From the looks of the R.E.M. setlist site remtimeline.com, early R.E.M. gigs were glorified bar-band hootenannies. Before they’d written Murmur and Chronic Town, the band cut their teeth seemingly as philistines, playing sloppy, drunken versions of everything from Creedence to the Sex Pistols. Time passed, R.E.M. became more sophisticated, and their covers eventually became more immaculately chosen, providing an essential glimpse into the band’s background. The recent iTunes clearinghouse release of some 143 B-sides showcases many, but not all, of the band’s best covers. They can also be found on rare fan club-only 45s, 1988’s Dead Letter Office compilation, and occasionally one-off performances that would never be properly recorded, not even at a bootleg level.

What the following lists provides is — to quote Peter Buck and the title of the band’s final singles compilation — evidence that R.E.M. were “part lies, part heart, part truth, and part garbage.” They forged their own sound over the course of 15 LPs, but it was in the ghetto of B-sides territory where some of their most spirited and revelatory performances were laid to tape. These songs were sloppy, often drunken, occasionally flippant — but you could never accuse them of lacking in passion, and many of these covers turned the band’s massive audience onto the music of R.E.M.’s heroes. They did it the right way, shining light on what was unacceptable or unheard by the mainstream, just as they did when they handpicked their opening acts throughout the years.

R.E.M. were certainly a polarizing band, as most greats are, but one would be hard pressed to accuse them of not caring. These 10 tracks illuminate 10 great bands that profoundly shaped R.E.M. These aren’t 10 great performances per se, but they run the stylistic gamut with a spirited élan. There were another 10-20 numbers that easily could’ve been included, as R.E.M.’s covers discography is a deep well of downright great songs by brilliant artists. But these 10 are deeply reverent love letters to R.E.M.’s spiritual antecedents, the DNA strands that cohere to form the double helix of one of the finest bands of our generation.

10. “Crazy” (Originally by Pylon, from Dead Letter Office, 1987)

Peter Buck laments on the Dead Letter Office liner notes how bummed he was upon hearing Pylon’s “Crazy,” which was released the same day as R.E.M.’s “Radio Free Europe,” because he felt Pylon’s new song was far superior to R.E.M.’s. Whether it’s superior is debatable, but it does deserve a place in the pantheon of great proto-indie singles. While Pylon’s version is herky-jerky, like a volcano bubbling beneath the surface, all hiccupy vocals courtesy of Vanessa Briscoe Hay that occasionally tease at full-on eruption, R.E.M.’s version strives for equanimity, and would’ve fit well on Murmur, with its enigmatic, inscrutable Stipe-consonant dropping and Buck’s murky, serpentine guitar line. Many of the bands R.E.M. covered went on to achieve something approximating mass popularity, thanks at least in some part to the exposure granted to them by their song being covered by such a popular band. Pylon unfortunately never attained mass success, despite supporting R.E.M. both early in the band’s career and much later on the massive Green amphitheater tour. Perhaps Pylon never had the constitution or populist sound for mass success, but they left an indelible mark on those that they did manage to reach, including their great friends and lifelong fans in R.E.M.

9. “Jesus Christ” (Originally by Big Star, from a fan club-only single, 2002)

Another Mike Mills-sung number, “Jesus Christ” — like so many of these tracks — paid tribute to one of R.E.M.’s biggest influences: Memphis, TN’s finest, Big Star. In Mills’ hands, the song feels like a sad lament, far removed from the caustic bile at the heart of Alex Chilton’s version. Yet the melodic instincts so endemic to Big Star are here in spades, offering a glistening take on one of Third’s finest moments. The track would later assume a more celebratory, ebullient tone when Mills sung it following Chilton’s sudden death in 2010, as a series of concerts took place honoring both Chilton and Third’s long-overdue recognition as a colossal artistic achievement. It celebrated Chilton and his band’s brilliance, and thankfully never went into over-the-top deification mode, which would’ve had the notoriously cantankerous Chilton, who showed downright contempt at times for his legacy, rolling in his grave.

8. “Ghost Rider” (Originally by Suicide, from the UK single “Orange Crush,” 1988)

R.E.M.’s cover of this Suicide classic jettisons the cloistered dread of the original in favor of an almost playful, bubblegum take on the song. The line “America is killing its youth” is delivered with a detached anomie by Stipe, but other than that, he seemingly vamps it up, as the protagonist is “sneaking around in a blue jumpsuit/ baby, baby he’s screaming the truth,” having a gas with the absurd notion of a frontman’s power to raise his arm up to elicit screams from tens of thousands, transforming this number into something decidedly different but no less interesting than the brilliant original. And like so many of these songs, it exposed legions to a woefully underappreciated act, in this case the quintessential purveyors of New York No-Wave noise, Martin Rev and Alan Vega.

7. “Strange” (Originally by Wire, from Document, 1987)

More joyous than Wire’s near dirge-like exercise in minimalism, “Strange” fits the mood of Document exceedingly well, leading nicely into the frenzied jubilation of the classic single “It’s The End Of The World As We Know It (And I Feel Fine).” When Stipe playfully inserts himself into the narrative, proclaiming, “Michael’s nervous and the lights are bright” (in lieu of Joey), R.E.M. makes this distinctive song their own. It took balls the size of church bells to veer off course from the original so radically, and they certainly took their fair share of criticism for such a brazen move. But much like R.E.M.’s Velvet Underground covers, their take on “Strange” likely exposed tens of thousands to the genius of Wire. And hell, Wire’s Colin Newman was no doubt pleased with the generous royalty checks his band received thanks to Document’s multi-platinum sales, likely outstripping the amount Wire made from the sales of their own LPs, hopefully assuaging any feelings of artistic betrayal.

6. “Dark Globe” (Originally by Syd Barrett, from the US single “Everybody Hurts,” 1993)

As Stipe croons wistfully over a resplendent acoustic guitar figure, “Won’t you miss me?” he, — like the song’s author, erstwhile Pink Floyd founder Syd Barrett — imbues a mundane line with a devastating pathos. R.E.M. played the tune extensively on the Green world tour, with Stipe often singing it a cappella, including a Roger Waters-attended gig in London, at which point Stipe apparently, in contrast to the other guys in the band who were stoked to meet the Pink Floyd legend, gave him something of a cold shoulder. Waters reportedly wasn’t sure what to make of the cover — whether it was a dig at him, an unorthodox tribute, or perhaps just a coincidence. No matter, the cover choice was always a gorgeous one, and saliently illustrated the fractious headspace Stipe often found himself in throughout the protracted grind of the Green tour.

5. “Indian Summer” (Originally by Beat Happening, from the single “Hollow Man,” 2008)

This dirge-like take on the quintessential Beat Happening number swaps the gossamer shambles of the original for a foreboding “All Tomorrow’s Parties” vibe. Stipe, at the tail end of his R.E.M. career, sounds more curmudgeonly than ever, lacking the sprightliness that highlighted so many of R.E.M.’s finest numbers, lending the track’s wistful naiveté a sepulchral sense of dread and in a sense emulating the monochromatic vocal style Beat Happening frontman Calvin Johnson achieved on the original. Of the late-career R.E.M. covers, this is easily the most successful, simply because it sounds so effortless. Bill Rieflin’s tom-tom rhythms fit the number like a glove, and Buck’s slow-motion jangle, building tastefully to a restrained crescendo of sustained feedback over a roller-rink-ish keyboard line, evokes the maddening desperation endemic to a relationship far past its expiration date. As Stipe intones, “We will never change/ no matter what they say,” the writing is neon bright on the chalkboard. Things had changed, and there was no looking back.

4. “Wall Of Death” (Originally by Richard Thompson, from the US single “E-Bow The Letter,” 1996)

This song equates the disintegration of a relationship with a carnival ride, one in which centripetal force keeps the rider from falling to harm. The sanguine instrumentation and sad-eyed slide guitar of R.E.M.’s fairly straightforward take is all jangle and stomp, leaving us with an ultimately R.E.M.-ish tune that’s lyrically akin to the likes of “Country Feedback” and “Fretless” in its dignified resignation. As the protagonist claims he’ll “take [his] chances on the Wall Of Death,” while passing on the other, more comforting carnival rides, it’s a veiled acknowledgement that the relationship is being sustained by smoke and mirrors, but he’ll give it hell anyway with the Sword Of Damocles dangling over his neck.

3. “First We Take Manhattan” (Originally by Leonard Cohen, from the UK single “Drive,” 1992)

Michael Stipe has long acknowledged the lyrical debt he owes to Leonard Cohen, particularly in post-Document R.E.M. Heck, “World Leader Pretend” was the first song the band ever printed lyrics to, and one can draw a straight line from that track’s megalomaniacal ethos to the deluded myopia of “First We Take Manhattan,” a conflation of fantasy-world domination with a harrowing interpersonal struggle. Cohen’s version of this track has a skittish, vaguely funk feel, while R.E.M.’s track is pure anthemic power-chord sizzle. But the message within each remains the same: The internal struggle of simply making it through the day is the only thing ultimately worth fighting for, and until it’s conquered, all the grandiose visions in the world are worth fuck-all.

2. “Superman” (Originally by the Clique, from Lifes Rich Pageant, 1986)

As an unlikely hit in the R.E.M. canon, a cover of a Clique B-side from Peter Buck’s absurdly arcane collection, “Superman” finds the band at their most playful, belying the studiously high political gravitas of much of the rest of the album. It was dismissed upon release in its Rolling Stone review by Anthony DeCurtis, who claimed that it and “Underneath The Bunker” prevented Pageant from being one of the “seminal American rock albums of the ’80s.” It’s a fair assessment, but the track acted as something of a bridge from the R.E.M. whose earliest sets were composed of a litany of sloppy covers, both obscure and populist, to the world-beating superstars who would stride confidently upon the release of 1987’s Document. It makes sense that Mike Mills would sing lead vocals, as the song doesn’t stray far thematically from the Mills-penned “(Don’t Go Back To) Rockville,” essentially a fervent plea to win back a woman. But while “Rockville” found Mills meekly “drinking himself to sleep” over a girlfriend who’s left him and skipped town, “Superman” finds him confidently keening, “If you go a million miles away I’ll track you down girl,” as Peter Buck’s Rickenbacker chimes like a clarion bell throughout the chorus, echoing Mills’ exaggerated, cartoonish confidence.

1. “Pale Blue Eyes” (Originally by the Velvet Underground, from Dead Letter Office, 1987)

Never mind that old saw about how everyone who heard the Velvet Underground back in the day started a band — in 2014 it feels as though everyone who’s heard the Velvet Underground has covered the Velvet Underground at some point. R.E.M.’s 1987 odds-and-sods collection Dead Letter Office featured three Velvets covers. Dead Letter Office singlehandedly exposed tens of thousands of listeners to the Velvet Underground for the first time. R.E.M. had such an affinity for “Pale Blue Eyes” that it often served as the opener during 1984’s Little America tour. The studio version didn’t stray far from the Velvets’ original, its plangent take on perfidy not far removed from the grief engendered on R.E.M. originals such as “So. Central Rain (I’m Sorry)” and “7 Chinese Brothers,” which dealt in a similar vernacular of love, regret, and betrayal. Michael Stipe once said in an interview a decade or so ago that Lou Reed still thanked him to that day for covering his songs over the years. The pleasure was all ours, Lou.