Neumann was born in Prague and studied at the conservatoire there. He co-founded, and played the viola in, the Smetana Quartet, before holding conducting posts in Carlsbad and Brno. In 1956 he became conductor at the Komische Oper in Berlin, leaving in 1964 he left to become condcutor of the Leipzig Gewandhaus Orchestra. He stayed there until 1968 when he became principal conductor of the Czech Philharmonic Orchestra, a post he held until 1990.
Neumann was particularly noted as a champion of Czech music, and made the first studio recording of Leos Janacek's opera The Excursions of Mr. Broucek in 1962.