The Grey Mountains were the last remnecents of the wall of the Ered Engrin or Iron Mountains, which once stretched all over the north of Middle-earth, but were broken at the end of the First Age.
North of the Grey Mountains lay Forodwaith, or the Northern Waste. This land was known as Dor Daidelos during the First Age, but most of it was destroyed in the breaking of Arda.
In the maps of the Second and Third Age it looked like the Ered Mithrin were but a northern arm of the Hithaeglir, but in truth this mountain range was far older, stemming from the creation of Arda, whereas the Misty Mountains had not been raised until after the Years of the Lamps.
Where the Ered Mithrin met at their western end with the Hithaeglir lay Mount Gundabad, ancient Dwarven holy site and Angmar's old capital. The stretch of mountains west of the Hithaeglir which still formed one range with the Grey Mountains was known as the Mountains of Angmar, another remnant of the Ered Engrin.
The eastern end of the Ered Mithrin was split in two branches, and in between lay the Withered Heath, where dragons still bred. After that was a long gap, until the Iron Hills continued the old line of the Iron Mountains again. Erebor, the Lonely Mountain]], was not part of this range.
From East to West the mountains stretched some 350 Miles, and the sources of the Great River Anduin, the river Greylin, and the Forest River of Mirkwood arose in this range.
Of old the Grey Mountains had been mined by Dwarves, but by the Third Age all Dwarven strongholds had been abandoned or raided by dragons. Its sole purpose now seemd to be to divide Forodwaith from Wilderland.