Brumstead is recorded in the Domesday Book. Its name is believed to mean "place of broom". The manorial hall and church survive.
The village has also been known for hundreds of years as Brunstead, as old maps demonstrate. Brunstead is also the name by which the village is known locally, and the name on the village sign.