Brahms began working on the piece in 1884, just a year after completing his Symphony No. 3, and completed it in 1885. Like most symphonies, it is in four movements:
The last movement is notable as a rare example of a symphonic passacaglia. For the repeating theme, Brahms' adapted the passacaglia theme in the closing movement of Johann Sebastian Bach's cantata, Nach dir, Herr, verlanget mich.
The work was given its premiere in Meiningen on October 25, 1885 with Brahms himself conducting the performance. It was well received and has remained popular since. The piece had earlier been given to a small private audience in a version for two pianos (one of them played by Brahms). Brahms' friend and biographer Max Kalbeck, reported that the critic Eduard Hanslick, on hearing the first movement in this performance, exclaimed "For this whole movement I had the feeling that I was being given a beating by two incredibly intelligent people." Hanslick later spoke more approvingly of it, however.