Joachim was born in Kittsee (Köpcsény), near Pressburg (Bratislava). In 1833 his family moved to Budapest, where he had violin lessons from the age of five. He went on to study in Vienna and Leipzig, where he was mentored by Felix Mendelssohn. It was with Mendelssohn that Joachim made his first visit to London at the age of thirteen. He was a great success there, and went on to visit the city many more times.
Following Mendelssohn's death, Joachim moved to Weimar, where he became concertmaster and met Franz Liszt and Richard Wagner. However, when he moved to Hanover in 1852, he dissociated himself from their musical ideals, and instead became friends with Robert Schumann, Clara Schumann and Johannes Brahms (who sometimes accompanied him at the piano in recitals). Brahms and Joachim jointly wrote a manifesto against the music of Liszt, Wagner, and their associates. Also in Hanover, Joachim married the singer Amalie Weiss.
In 1866, Joachim moved to Berlin, becoming founding director of the Royal Academy of Music there. He founded his own orchestra, and, in 1869, the Joachim String Quartet, which gained a repuatation as one of the finest quartets in the world.
In 1884, Joachim divorced his wife after he became convinced that she was having an affair with Brahms' publisher, Fritz Simrock. Brahms, certain that Joachim's suspicions were groundless, wrote a long letter in support of Amalie, later produced in court as evidence. This led to a break up of his and Joachim's friendship, not restored for some years.
Joachim remained in Berlin until his death in 1907.
Joachim performed a wide range of repertoire, ranging from the concertos of Johann Sebastian Bach, through the Beethoven Violin Concerto (a work which he did much to establish as part of the standrad repertoire) to works by his young contemporaries, many of whom he knew personally. He was the dedicatee of Antonin Dvorak's Violin Concerto, though never performed the work.
Joachim was particularly closely associated with Brahms, who consulted him on points of instrumental technique while writing his Violin Concerto, a work dedicated to Joachim. Joachim also premiered Brahms' Double Concerto (written as a peace offering after their falling out over the divorce) and conducted the English premiere of his Symphony No. 1.
His activities as a composer are less well known. He has a reputation as a competent though rather characterless composer, and none of his works are regularly performed today. Among his compositions are various works for the violin (including three concertos) and overtures to Shakespeare's Hamlet and Henry IV. He also wrote cadenzas for a number of other composers' concertos (including ones for the Beethoven and Brahms).