Stump was born in Phoenix, Arizona and was a U.S. Navy World War II Combat Veteran, where he served on the USS Tulagi from 1943-1946. He graduated from Tolleson High School in 1947, and Arizona State University in 1951.
He served four terms in the Arizona House of Representatives from 1959-1967, and five terms in the Arizona State Senate, from 1967 to 1976. He served as President of the Arizona State Senate from 1975 to 1976.
He was first elected to the 95th Congress on November 3, 1976 representing the third district as a Republican. He was re-elected to the 96th through the 107th Congresses where he was a member of the House Armed Services Committee.
He decided in 2002 not to run for re-election due to declining health and died June 20, 2003.
There was some controversy over the fact that he seldom travelled back to Arizona from Washington D.C. and his residence was in Phoenix, which is outside the district he represented.