The name is derived from Sir John Cavendish, the ancestor of the Dukes of Devonshire who was involved in the Peasants' Revolt. In 1381 his son had put to death Wat Tyler, the peasants' leader. As a result, John Cavendish tried to flee from the pursueing peasants, and he hung on to the handle of the door of St Mary's Church, Cavendish, to plead sanctuary. His plea was in vain, and he was taken to Bury St Edmunds and beheaded by a lynch mob headed by Jack Straw. He is buried in Bury St Edmunds.
St. Mary's Church had a bequest from Sir John, and its chancel was restored.
Leonard Cheshire and his wife Sue Ryder are buried in Cavendish. There is also the Sue Ryder Foundation Museum with many war memorabilia.