Marsh was born in Lockport, New York. He graduated at Yale College in 1860, and studied geology and mineralogy in the Sheffield scientific school, New Haven, and afterwards palaeontology and anatomy in Berlin, Heidelberg and Breslau. He returned to the United States in 1866 and was appointed professor of vertebrate paleontology at Yale University. He persuaded his uncle, George Peabody, to establish the Peabody Museum of Natural History at Yale.
In May 1871 Marsh found the first American pterodactyl fossils. He also discovered the remains of early horses. Marsh described the remains of Cretaceous toothed birds and flying reptiles, and Cretaceous and Jurassic dinosaurs, including Apatosaurus and Allosaurus.
Marsh is interred at the Grove Street Cemetery, New Haven, Connecticut.