Together with Russell Simmons he founded the Def Jam record label while attending New York University in 1984. Their early records included LL Cool J's "I Need a Beat". They went on to release Public Enemy's records, starting with their Rubin-produced debut Yo! Bum Rush the Show (1987). Rubin also produced records by the Beastie Boys and Run DMC, both of them characterised by fusing rap with heavy rock. His work on Slayer's Hell Awaits and Reign in Blood saw him in a purer rock context.
In 1988, Simmons and Rubin went their separate ways, Simmons remaining in New York at the helm of Def Jam, and Rubin leaving to Los Angeles to found the Def American label. There he signed a number of heavy rock acts, including Slayer, Danzig, Masters of Reality, The Cult and Wolfsbane, as well as the indie rockers Jesus and Mary Chain and controversial stand up comedian Andrew Dice Clay, and he produced the Red Hot Chilli Peppers' break-through album Blood Sugar Sex Magik. He retained a close association with rap also, signing the Geto Boys and continuing to work with Public Enemy, LL Cool J and Run DMC among others.
In 1993, Rubin held a symbolic funeral for the word "Def" and Def American became American Recordings. The first major project on the renamed label was Johnny Cash's American Recordings (1994), a record including several covers by artists more closely associated with Rubin than Cash (such as Danzig) and which did much to revive Cash's career following a fallow period. The formula was repeated for Cash's Unchained, also produced by Rubin, and Rubin also made a number of records with other older artists, including Tom Petty (Wildflowers) and Donovan (Sutras).