Ormandy was born in Budapest and studied at the conservatoire there. In 1921 he moved to the United States of America, where he worked first as a violinist and later conductor of the Capitol Theater Orchestra in New York City, a group which accompanied silent movies. He was conductor of the Minneapolis Symphony Orchestra from 1931, but it was as conductor of the Philadelphia Orchestra that he became most famous. He was associate conductor there (alongside Leopold Stokowski) from 1936, and music director from 1938 until his retirement in 1980.
He shaped the Philadelphia Orchestra's famous lush, legato style, a style which attracted praise for its opulent sound while also being criticized for lack of any real individual touch. He was particularly noted for late Romantic and early 20th century works. He was a champion of Sergei Rachmaninov's music and gave the US premiere of several symphonies by Dmitri Shostakovich.
Ormandy died in Philadelphia, Pennsylvania.