Fountain was born in New Orleans on July 3, 1930, and started playing clarinet heavily influenced by Irving Fazola. Early on he played with the bands of Monk Hazel and Al Hirt. With his long time friend trumpeter George Girard Fountain founded The Basin Street Six in 1950. After this band broke up 4 years later Fountain was hired to join the Lawrence Welk band, and became well known for the many solos he took on Welk's national television show. Fountain returned to New Orleans, played with The Dukes of Dixieland, then began leading bands under his own name, owning his own club in the French Quarter in the 1960s and 1970s, then getting a residency in the New Orleans Hilton Hotel where he plays when not on tour.
Fountain's clarinet work is noted for his sweet fluid tone. He has recorded over 100 LPs and CDs under his own name, some in the Dixieland style, many others with only peripheral relevence to any type jazz.
Fountain was a founder and is the most prominent member of The Half Fast Marching Club, one of the best known marching Krewes that parades in New Orleans on Mardi Gras Day.