He was a working class Jewish boy who grew up in Brooklyn, New York and received a scholarship to Harvard Law School. He graduated and practiced law in a small firm which folded, then moved on to the firm of Rosenman, Colin, Kaye, Petschek and Freund, which had CBS Records as a client. He was then hired by the legal department of CBS subsidiary Columbia Records.
He discovered a passion for music which led him up the ranks of Columbia and CBS. In 1967, he became president of CBS Records; at almost the same time, he became a convert to the newest generation of folk rock and rock and roll. That year he attended the Monterey Pop Festival and was inspired by what he saw as the future of music. He immediately signed Janis Joplin and Big Brother & the Holding Company, and went on to sign Electric Flag, Santana, Bruce Springsteen, Chicago, Billy Joel, Blood, Sweat and Tears, and Pink Floyd. The company, which had previously avoided rock music, doubled its market share in three years.
Amid government investigations of financial irregularities in the record industry, Davis was fired from CBS (nervous about losing its licenses) amid charges that he had used company money for personal expenses -- allegations he still denies.
In 1974, he founded Arista Records (named after his secondary school honor society). This label has one of the most diverse lineups in the record industry. It has been home to Barry Manilow, Whitney Houston, The Grateful Dead, Sarah McLachlan, Annie Lennox, Kenny G, Puffy Combs, Aretha Franklin, Toni Braxton and Patti Smith.
On January 28, 1997 Davis received a star on the Hollywood Walk of Fame.
In 2000, Davis ran into disagreements with new Arista owners BMG (a member of the Bertelsmann group) over the way he ran the label. Reports point to his age (late sixties at the time), lack of a succession plan, and free-spending habits as points of contention. As part of the deal to oust him, Davis was allowed to take some of his most successful artists (Luther Vandross and Alicia Keys, for example) as he founded yet another independent record label, J Records. Bertelsmann even invested $150 million in the company.
Three years later, at the beginning of 2003, BMG wooed Davis back into the fold, naming him head of RCA Records.