He was born Joseph Goreed Williams in Georgia and moved to Chicago as a child. He rose to national prominence as the singer for Count Basie's Orchestra in 1954. He remained with Basie for several years. "Everyday I have the Blues" is his biggest hit.
Bill Cosby cast Williams as his father "Grandpa Al" in a recurring role on the 1980s sit-com The Cosby Show.
Williams enjoyed a successful career right until his death. He died at age 80, in Las Vegas, Nevada.