Fiedler's father was an Austrian-born violinist who played in the Boston Symphony Orchestra, and his mother was a pianist and musician. He grew up in Boston until his father retired and returned to Austria, where he studied and worked until returning to Boston at the start of World War I. In 1915 joined the Boston Symphony Orchestra under Karl Muck as a violinist, but also worked as pianist, organist, and percussionist.
In 1924 he formed the Boston Sinfonietta, a chamber orchestra made up of Boston Symphony members, and started a series of free outdoor concerts. He was appointed the eighteenth conductor of the Boston Pops in 1930, a position he held for a half-century until his death.
Under Fiedler's direction, the Boston Pops has allegedly made more recordings than any other orchestra in the world, with total sales of albums, singles, tapes, and cassettes exceeding 50 million.
Fiedler was also associated with the San Francisco Pops Orchestra for 26 summers, and conducted many other orchestras throughout the world.
After his passing, the city of Boston, Massachusetts honored him with an abstract sculpture on the Esplenade, home of the free concert series that continues through the present day. The sculpture is an oversized bust of Fiedler.