The village name is Anglo Saxon in origin, and means 'Hogg's farm' ('Hogg' being someone's name). In the Domesday Book of 1086 it was recorded as Hochestone.
The village was once granted (in 1314) a Royal charter to hold a market once a week, though this has long since been discontinued.
The parish church is dedicated to Holy Cross. Anciently there was a fair in the village every year on the exaltation of the Holy Cross (commonly called Holyrood Day).