A native of Nashua, New Hampshire, Gregg graduated from Yale University and Harvard Law School in 1942, after which he returned to Nashua and started a law practice. A Republican, he served as an alderman in the city for two years and was elected mayor in 1950, a term cut short because of military duty.
Gregg was also a local businessman involved with the family mill-working business. He was instrumental in setting up the Nashua Foundation, which helped the city recover from the loss of textile mills in the 1950s, by recruiting new industry, including defense electronics firms and, later, Digital Equipment Corp.
In later years, Gregg was best known for his defense of New Hampshire's first-in-the-nation presidential primary, as well as his contention that the Republican Party started in this state.
Gregg was known for a sense of humor, reflected in a small hardback book he published, titled All I learned about politics, by Hugh Gregg. All of its pages are blank.