He was born in New Haven, Connecticut and became a lawyer. He was a member of the Connecticut state senate 4th District, 1837-38 and a member of Connecticut state house of representatives, 1840. He was a key figure in the Amistad case. He became Governor of Connecticut, 1844-46 and was United States Senator from Connecticut, 1847-51. He died in New Haven, Connecticut, and was interred at Grove Street Cemetery.
He was grandson of Roger Sherman, son of Simeon Baldwin, and father of Simeon Eben Baldwin.