It was originally a timber fortification upon an artificial hill and upgraded later by Earl Ralph in 1347 by building a massive stone keep.
In 1444, Humphrey Stafford was created Duke of Buckinghamshire and the castle reached its heyday.
Early on during the Civil War it was defended by Lady Isabel but was eventually abandoned and demolished.
Rebuilt in the Gothic Revival Style in 1813, the castle fell into ruin through this century but is still standing, open to visitors and visible from the M6 motorway.