Latimer was originally joined with the adjacent village of Chenies. Both were anciently called Isenhampstead, at a time when there was a royal palace in the vicinity. However in the reign of King Edward III of England the lands were split between to manorial barons: Thomas Cheyne in the village that later became called 'Chenies', and William Latimer in this village. Latimer came into possession of the manor in 1326.
At the time of the English Civil War Latimer belonged to the Earl of Devonshire. When Charles I was captured by the Parliamentarian forces he was brought to Latimer on his way to London.
Today Latimer is also the name of a railway station on the Chiltern Line between Amersham and London Marylebone Station. See Chalfont and Latimer tube station.