The term comes from the fact that such structures have often been dubbed "[name of architect or builder]'s Folly", in the sense of foolishness or madness.
Follies are usually found in parks or large grounds of houses; they may sometimes have been deliberately built to look partially in ruins.
The Parc de la Villette in Paris has a number of modern follies by different architects.
Examples include: