After serving army duty during World War II, Dillard returned to college and resumed athletics, to which he had been inspired by Jesse Owens, who was also from Cleveland. He particularly excelled in hurdling, and was probably the best hurdler in the world shortly after the war. However, at the trials for the 1948 Summer Olympics, Dillard failed to qualify for the 110 m hurdles event, although he qualified as third (and last) for the 100 m, not his specialty.
At the Games, Dillard reached the final, which seemed to end in a dead heat between Dillard and another American, Barney Ewell. The finish photo showed Dillard had won, equalling the World Record as well. As a member of the 4 x 100 m relay team, he won another gold medal at the London Games.
Four years later, still a strong hurdler, Dillard did qualify for the 110 m hurdles event, and won the event in Helsinki. Another 4 x 100 m relay victory yielded Dillard's fourth Olympic title.
Harrison Dillard attempted to qualify for a third Olympics in 1956, but failed.