Berg could speak several languages and was sometimes called "the brainiest guy in baseball" by admiring newspaper sports writers, who featured him in their columns far more than was called for by his sports prowess. His reputation was fueled when he did very well as a guest on a radio quiz show called Information, Please in 1938. Berg answered questions about the derivation of words and names in Greek and Latin, historical events in Europe and the Far East, and current international conferences.
Berg was born in New York City to a Jewish family. He was never religiously observant, although this background did at times contribute to his sense of being an outsider in mid-20th century America.
Berg graduated Princeton University in 1923, after majoring in Modern Languages and playing brilliantly on their baseball team.
He played his first professional baseball game on June 27, 1923 as shortstop for the Brooklyn Robins. He spent only one season with Brooklyn. From 1923 to 1926 he didn't play professional baseball. It is suspected that during this time he attended Columbia Law School, but it is also possible that he went to school during the off-season. He was admitted to the New York State bar in 1928, but apparently never practiced as a lawyer
In 1926, he played various infield positions for the Chicago White Sox but by 1928 had become a catcher. He spent the rest of his career behind the plate, mostly as a backup, first with Chicago and later with the Washington Senators (or Nationals), Cleveland Indians and finally the Boston Red Sox, where he ended his playing career in 1939.
Berg batted and threw right-handed. He was 6' 1" tall and weighed 185 lbs. He was a lifetime .243 hitter. His best season was probably 1929, when he appeared in a career-high 107 games and batted .287.
In 1934, Berg toured Japan with an American all-star baseball team. While he was there, he took photographs of several areas which may have been used to help determine bombing targets during World War II.
During World War II he worked for America's first spy service, the Office of Strategic Services, most notably on a mission to Switzerland where he attempted to discover how much progress Germany was making towards building an atomic bomb.
After the war, the OSS made way for the Central Intelligence Agency, which developed a more professional, bureaucracy-based approach to espionage that didn't fit the prickly, enigmatic Berg. As a result, Berg only worked one other time as a spy, and without success. Many friends and acquaintences, however, continued to believe he was a spy due to his secretive nature.
As depicted in the 1995 book The Catcher was a Spy by Nicholas Dawidoff, Berg spent the last 25 years of his life with no real job, living off friends and relatives who put up with him because of his great charm.
He died in Belleville, New Jersey from injuries sustained in a fall at home. He was 70.