Born in Ireland, the son of notable Rabbi Yitzhak HaLevi Herzog, he immigrated to Palestine in 1935, and served in the Jewish paramilitary group Haganah during the Arab revolt of 1936-38.
He went on to earn a degree in law, but joined the British army during World War II, operating primarily in Germany. By the end of the war, he had become head of intelligence in northern Germany for the British, and participated in the liberation of several concentration camps.
Immediately following the war, he returned to Palestine to participate in the agitation for the formation of a Jewish state. Following the 1947 UN Partition Plan creating the state of Israel, he fought in the 1948 Arab-Israeli War, serving as an officer in the battles for Latrun.
His intelligence experience during World War II was seen as a valuable asset, and he subsequently became head of the IDF Military Intelligence Branch, a position in which he served from 1948 to 1950 and again from 1959 to 1962. In the intervening years, he had served as Israeli defense attache at the Israeli Embassy in the United States (from 1950 to 1954). By the time he retired from the IDF in 1962, he held the rank of major-general.
After his retirement, he entered private law practice, but entered public life again as the 1967 Six-Day War broke out, this time as a military commentator on news radio. Following the capture of the West Bank in that war, he did enter official public life again, becoming the first military governor of the newly-acquired territory.
In 1975 he was appointed Israel's Ambassador to the United Nations, in which capacity he served until 1978. In 1981 he entered politics for the first time, winning election to the Knesset as a member of the Labour Party.
In 1983 he was elected as the sixth President of Israel by the Knesset; he served two terms, retiring from political life in 1993.
He has also authored several books on the historical events in which he was involved, including War of Atonement: The Inside Story of the Yom Kippur War (1975), Who Stands Accused? : Israel Answers Its Critics (1978), The Arab-Israeli Wars: War and Peace in the Middle East (1982), and Heroes of Israel: Profiles of Jewish Courage (1989). His other notable writings include Battles of the Bible (1978), co-authored with military historian Mordecai Gichon, and his memoirs, Living History: A Memoir (1996).