It is thought the mosque was first built by Azoud ad Dowleh in 977 CE but after being destroyed by fire it was rebuilt by Seljuk Malik Shah in 1086 CE. Since that time parts of the mosque have been rebuilt numerous times.
Since the U.S war on Iraq there have been a number of attacks at the mosque.
On April 10, 2003, the Shiite leader Sayed Abdul Majid al-Khoei (son of Grand Ayatollah Abu al Qasim al-Khoei) who had return from exile in Britain to encourage cooperation with the US occupation was shot at the mosque.
On August 29, 2003, a car bomb exploded outside the mosque just as the main Friday prayers were ending. More than 125 people were killed, including the influential cleric Ayatollah Sayed Mohammed Baqir al-Hakim, the Shiite leader of the Supreme Council for the Islamic Revolution in Iraq (SCIRI).