Born in San Francisco, California, he graduated from high school in Los Angeles and spent three years in the Army Corps of Engineers, where he took up boxing. After leaving the army, he enrolled in Los Angeles City College and the Los Angeles Academy of Dramatic Art. He had a small part in When Worlds Collide, then took small roles in such films as The All American, Brigadoon, Ten North Frederick and The Sound and the Fury. His first starring role was in Murder, Inc in 1960.
In 1961, Whitman was nominated for an Academy Award for Best Actor for his role as a child molester in The Mark. He has since appeared in starring and supporting roles in many genres of film.