Born in Kirchheim-Bolanden, Pfalz, he finished his education in geophysics and hydrography in Munich in 1851. He then sailed as an ordinary sailor to Brazil and Australia.
In 1857 he established the Flagstaff Observatory for Geophysics, Magnetism and Nautical Science in Melbourne, Australia. He remained the director until 1864.
Later, he organized the "Gazelle Expedition." (1874-1876) and was director of the hydrographic organisation "Deutsche Seewarte" (1876-1903).
He chaired the International Polar Commission in 1879 together with Karl Weyprecht, founding the first International Polar Year 1882/83 and the Antarctic Year 1901.
In 1895, von Neumayer had established the German Commission for South Polar Exploration, which culminated in the first German Antarctic expedition in 1901, the so-called Gauss expedition.
Von Neumayer died in 1909 in Neustadt an der Weinstraße.