His early government jobs included that of surveyor and King's Advocate for New England. He was apponted the royal Governor in 1741. Following his failure in a military expedition against Fort Niagara, he was recalled to England in June of 1756. He was later exonerated, and served as Governor of the Bahamas from 1761-1769.
He retired to live with his daughter and her husband (Eliakin Hutchinson) at the Roxbury house. He died there on March 24, 1771.
The Shirley House
He built a family home in Roxbury between 1744 and 1750. It still stands, at 33 Beverly Street, has been largely restored and is open to the public.