Hovis was born in Wapato, Washington and moved to Houston, Texas as a small child. As a youth, he was a singer, appearing on Arthur Godfrey's Talent Scouts. Hovis attended the University of Houston, During the mid 1950s, Hovis sang in nightclubs as a singer. He attempted songwriting and signed with Capitol Records and released one album.
He also began appearing in local theater productions. After some success, he moved to New York City in 1959 and appeared in Broadway revue shows which highlighted his singing and comedy talents.
Hovis moved to California in 1963 where he performed stand-up comedy and tried to break into television. In 1964, he was discovered by Andy Griffith's manager and was hired to appear on the show Gomer Pyle, U.S.M.C where he played "Pvt. Larry Gotschalk".
In 1965, Hovis received a promotion of sorts, when he was cast as "Sgt. Andrew Carter" in the television show "Hogan's Heroes". While he was a regular on that series, he wrote the 1966 film spy-spoof "Out of Sight". He also appeared and wrote for Rowan and Martin's Laugh-In.
After Hogan's Heroes was cancelled in 1971, Hovis appeared in several TV shows. He also produced and appeared in the mid-70s game show "Liar's Club."
Beginning in the 1990s, Hovis taught drama at Texas State University in San Marcos, Texas.
Larry Hovis died in Austin, Texas of esophageal cancer.