Rota was born into a musical family in Milan. He studied at the conservatory there under Ildebrando Pizzetti. Later, the conductor Arturo Toscanini encouraged him to go to Philadelphia, Pennsylvania to study. This he did, where he worked under Fritz Reiner, amongst others. He later returned to Milan, where he wrote a thesis on the renaissance composer Gioseffo Zarlino.
Rota wrote his first film score in 1944 for Zazą, a film directed by Renato Castellani. He later met the director Federico Fellini while the latter was working on his first film, Lo Sceicco Bianco. The two collaborated on many occasions, with Rota's score for 8 1/2 often cited as one of the main factors which makes the film hang together.
The scores for which Rota is probably best known among the general public are Franco Zeffirelli's Romeo and Juliet and Francis Ford Coppola's The Godfather and The Godfather, Part II.
Although best known for his work in films, Rota also composed ten operas, five ballets and many other instrumental works.