Ferrari was born in Paris and studied the piano under Alfred Cortot, musical analysis under Olivier Messiaen and composition under Arthur Honegger. His first works were freely atonal.
In 1954, Ferrari went to the United States to meet Edgard Varese, whose Déserts he had heard on the radio, and had impressed him. This seems to have had a great effect on him, with the tape part in Déserts serving as inspiration for Ferrari to use magnetic tape in his own music.
In 1958 he co-founded the Groupe des Recherche Musicales with Pierre Schaeffer and François-Bernard Mâche. He has taught in institutions around the world, and has worked for film, theatre and radio.
By the early 1960, Ferrari had begun work on his Hétérozygote, a piece for magnetic tape which uses ambient environmental sounds to suggest a dramatic narrative. The use of ambient recordings was to become a distinctive part of Ferrari's musical language.
Ferrari's Presque rien No. 1 "Le Lever du jour au bord de la mer" (1970) is regarded as a classic of its kind. In it, Ferrari takes a day-long recording of environmental sounds at a Yugoslavian beach and, through editing, makes a piece that lasts just twenty-one minutes. It has been seen as an affirmation of John Cage's idea that music is always going on all around us, and if only we were to stop to listen to it, we would realise this.
Ferrari has continued to write purely instrumental music as well as his tape pieces. He has also made a number of documentary films on contemporary composers in rehearsal, including Messiaen and Karlheinz Stockhausen.