He was born in New York City, and spent most of his early career doing stand up comedy, becoming known primarily as an "insult comedian". It is said that this characteristic developed when he replied to the audience's heckles by insulting the heckler back. The audience liked these insults more than his prepared material, so he developed that part of his act. There is a story that while working in a Los Angeles nightclub early in his career he spotted Frank Sinatra, and called out "Make yourself at home, Frank: hit somebody!"
He had been a drama student, and played occasional bit-parts on television alongside his work in nightclubs. In 1958, he made his film debut in Run Silent Run Deep, and he went on to appear in a number of others.
As his career progressed, he made more and more appearances on television talk shows, making his first appearance on Johnny Carson's Tonight Show in 1965. From the 1970s, he made several appearances on Dean Martin's show.
Recently, he has made a return to film, appearing in Martin Scorsese's film, Casino, and voicing Mr. Potato Head in Toy Story and Toy Story 2.