Colfax was born in New York City. In 1836 he moved with his parents to New Carlisle, Indiana. He was appointed deputy auditor of Joseph County, Indiana in 1841. He became a legislative correspondent for the Indiana State Journal. He purchased an interest in the South Bend Free Press and changed its name in 1845 to the St. Joseph Valley Register, the Whig organ of northern Indiana.
Colfax was a member of the state constitutional convention in 1850 and an unsuccessful Whig candidate for election to the Thirty-second Congress. He was elected as a Republican to the Thirty-fourth and to the six succeeding Congresses (March 4, 1855-March 3, 1869). He was not a candidate for renomination in 1868, having become the Republican nominee for Vice President. He was Speaker of the House of Representatives, and elected Vice President of the United States on the Republican ticket headed by Gen. Ulysses S. Grant in 1868, inaugurated March 4, 1869, serving until March 3, 1873. Colfax was an unsuccessful candidate for renomination in 1872, owing to charges of corruption in connection with the Credit Mobilier of America scandal.
He was a lecturer after leaving public office, and died in Mankato, Blue Earth County, Minnesota. He was intered in the City Cemetery, South Bend, Indiana.