Kowalczyk, Taylor, Dahlheimer and Gracey first played together at a middle-school talent show in York, Pennsylvania. They remained together throughout high school, playing new wave covers under band names such as First Aid, Club Fungus, Paisley Blues, Action Front, and Body Odor Boys. Eventually they settled on the name Public Affection and recorded a self-released cassette of original songs, The Death of a Dictionary, in 1989. In 1990 they released an EP of demos produced by Jay Healy, titled Divided Mind, Divided Planet, via their Black Coffee mailing list. The band played regular concerts at CBGB in New York City, which helped earn them a contract with Radioactive Records in 1991.
Under the new name Live, the band entered the studio with producer Jerry Harrison (of Talking Heads) and recorded the EP Four Songs. The single "Operation Spirit (The Tyranny of Tradition)" went to number nine on the Modern Rock chart and was followed by their debut album, 1991's Mental Jewelry, which Harrison again produced. Pat Dalheimer explained that, given the band's inexperience, Jerry Harrison's input was very valuable, he said, "So important to have somebody else in the room to help us, especially with arrangements. I mean, we were still learning how to write songs...Jerry just seemed to know everything...this guy's a wizard!" Some of the album's lyrics, written by Kowalczyk, were inspired by Indian philosopher and writer Jiddu Krishnamurti.