Tovey began to study the piano and compose at an early age. He eventually studied music under Hubert Parry.
Tovey became a close friend of Joseph Joachim, and played piano with the Joachim Quartet in a 1905 performance of Johannes Brahms' Piano Quintet. He gained some moderate fame as a composer, having his works performed in Berlin and Vienna as well as London. He performed his own piano concerto under the conductorship of Henry Wood in 1903 and under Hans Richter in 1906.
In 1914 he began to teach music at Edinburgh University and there he founded the Reid Orchestra. For their concerts he wrote a series of programme notes which were eventually developed into the books his is now best known for, the Essays in Musical Analysis.
Tovey began to compose and perform less often later in life. In 1913 he composed a symphony, in 1935 he wrote a cello concerto for Pablo Casals and he also wrote an opera, The Bride of Dionysus.
Tovey made several editions of other composers' music and in 1931 produced a completion of Johann Sebastian Bach's Die Kunst der Fuge.