Conrad was born in Bismarck, North Dakota, where he lived for much of his early life. During his childhood he and his family also lived in Tripoli, Libya. After graduating college, he became an assistant to the North Dakota tax commissioner, who later became his colleague in the Senate, Byron Dorgan. In 1980, Conrad succeeded Dorgan as tax commissioner. He was tax commissioner from 1980 until 1986, when he ran for senate.
In the 1986 election, Conrad defeated the Republican incumbent, Mark Andrews. In 1992, he did not run for reelection. However, he got an unusual opportunity to retain his senate position when the other North Dakota senator, the long-serving Democrat Quenton Burdick died in September 1992. Burdick's widdow was appointed to that seat temporarily, but a special election was needed to fill the rest of the term. Conrad got the democratic nomination and won the election, which was in November 1992. In a politically and constitutionally interesting move, Conrad had to resign from the senate in order to immediately return to it and start serving the term he had just won, to avoid holding both of North Dakota's senate seats at the same time. He was elected to another full term in 1994 and reelected in 2000. He was chairman of the Senate Budget Committee from 2001 until 2003, and still serves on that committee as the ranking Democrat.