It lies in the Waveney valley, about 7 km west of Beccles.
Bungay Castle was owned by the Normans, but was later rebuilt by Roger Bigod and his family, who also owned Framlingham Castle. Bungay's village sign shows the castle.
The town was almost destroyed by a great fire in 1688.
The Butter Cross was constructed in 1689. It was the place where farmers displayed their butter, eggs and other farm produce for sale. Until 1810, there was also a Corn Cross, but it was taken down and replaced by a pump.
What was once the 12th century church of the Benedictine Priory (founded by Gundreda, wife of Roger Bigod), is now the parish church of St. Mary. A wooden panel behind the altar was presented to the church by the novelist Sir H. Rider Haggard who was born nearby in Bradenham near Kessingland and lived in Ditchingham.
Nearby, in the village of Earsham, is the Otter Trust where otters are raised in captivity and then released into the wild. As a result, the otter population in the Suffolk Broads has increased.