The abbey was originally founded by King Athelstan of England, and there are two medieval paintings of the king and his mother in the chancel. Another of the church's benefactors was John Tregonwell, whose family came into the possession of the buildings following the Dissolution. Tregonwell fell from the roof of the church in a childhood accident, but his life was saved when his wide pantaloons filled with air and broke his fall. In thanks, he bequeathed his library to the church.
The abbey church is built in a mixture of Ham stone, Chilmark stone and flint. Its style is mostly Decorated Gothic dating from the mid 14th century. The Earl and Countess of Dorchester were also generous to the church, and their joint tomb, designed by Robert Adam, is to be found in the north transept. Perhaps the most striking feature of the church's interior, however, is its south window, designed as a Tree of Jesse by August Pugin.