Mark Kozelek was born and raised in Massillon, Ohio. Upon relocating to Atlanta, Georgia, he met drummer Anthony Koutsos. After the pair moved once again to San Francisco, California, Red House Painters formed with the addition of guitarist Gorden Mack and bassist Jerry Vessel. From 1992 to 1996, Red House Painters released a series of acclaimed albums, showcasing Kozelek's intense, highly autobiographical songs.

However, Kozelek's parting with the record label 4AD, followed by a major label merger which left Red House Painters' 1998 album Old Ramon on the shelf, proved highly frustrating and stifled the group's momentum. Kozelek opted to occupy himself with a series of idiosyncratic solo recordings as Red House Painters dissolved.

In 2000, he released the solo EP Rock 'n' Roll Singer, which featured three original songs as well as covers of John Denver's "Around and Around" and three AC/DC songs ("Rock 'n' Roll Singer," "Bad Boy Boogie," and "You Ain't Got a Hold on Me"). Two of the AC/DC cover versions on Rock 'n' Roll Singer were radical re-arrangements of the originals which removed the Bon Scott-penned lyrics from their hard rock context and placed them into gentle, acoustic "folk ballad" settings.