Sargent was born in Ashford in Kent. He worked first as an organist before making his conducting debut at a Proms concert at the Queen's Hall in London in 1921 with his own piece, Impression on a Windy Day. Early in his career he worked at the D'Oyly Carte Opera Company and with Serge Diaghilev's Ballets Russes from 1927 to 1930. In 1928 he became conductor of the Royal Choral Society, a post he retained until his death. He was chief conductor of the Proms from 1948 to 1966, and of the BBC Symphony Orchestra from 1950 to 1957. He was knighted in 1947.
Sargent tackled a wide range of repertoire, but was particularly noted in choral music. He was a champion of contemporary British music, and conducted the premieres of William Walton's oratorio Belshazzar's Feast in 1931 and his opera Troilus and Cressida in 1954. He was also noted as a populariser of classical music, conducting many concerts for children.
He was sometimes nicknamed "Flash Harry" because of his impeccable appearance.