The village name is Anglo Saxon in origin, and means 'Hædde's cliff' referring to the position of the village on a cliff overlooking the Thames.
The manor was anciently in the Hedsor family, who took their name from the village. It was they that owned it at the time of the Domesday Book in 1086. The modern manor however (built in 1778) stands on a hill in the village, commanding an excellent view of the Thames and of Berkshire beyond.
The parish church is dedicated to St Giles.