He was younger son of the 11th Earl of Pembroke. Educated at Harrow and Oriel College, Oxford, he made a reputation at the Oxford Union as a speaker, and entered the House of Commons as Conservative member for a division of Wiltshire in 1832. Under Peel he held minor offices, and in 1845 was included in the cabinet as Secretary at War, and again held this office in 1852 - 1855, being responsible for the War Office during the Crimean War, and again in 1859.
It was Sidney Herbert who sent Florence Nightingale out to the Crimea, and he led the movement for War Office reform after the war, the hard work entailed causing his breakdown in health, so that in July 1861, having been created a baron, he had to resign office, and died on the 2nd of August 1861. His statue was placed in front of the War Office in Pall Mall. He was succeeded in the title by his eldest son, who later became the 13th Earl of Pembroke, and the barony is now merged in that earldom; his second son became 14th earl. Another son, the Hon. Michael Herbert (1857-1904), was British Ambassador at Washington in succession to Lord Pauncefote.