The orchestra was founded as the City of Birmingham Orchestra in 1920, with Edward Elgar conducting its first concert. Adrian Boult was its chief conductor from 1924 to 1930. It became a full time organisation in 1944, changing its name to the City of Birmingham Symphony Orchestra in 1948. The composer Andrzej Panufnik was chief conductor from 1957 to 1960.
The orchestra made recordings and gave regular concerts, but did not gain any international renown until Simon Rattle became its chief conductor in 1980. Under him, the orchestra became one of the leading ensembles in Britain, and gained a name for its interpretations of late romantic and 20th century works. During this period, the orchestra moved from Birmingham Town Hall to a new "home venue", Symphony Hall, inside Birmingham's renowned International Convention Centre. In 1990, the post of Radcliffe Composer in Association was created, with Mark-Anthony Turnage filling the role. In 1995 Judith Weir became Fairbairn Composer in Association.
The Finn Sakari Oramo took over from Simon Rattle in 1998, upon the latter securing a post at the Berlin Philharmonic Orchestra.
The CBSO records mainly for EMI records.