He served in the Estonian army until Soviet troops occupied the country in 1940. Rebane joined the German army when it entered Estonia the following year. He was serving in the Estonian Legion of the German army: under the Waffen-SS, he fought against the Soviets. Rebane became one of Estonia's most prominent soldiers. During the war he was awarded with Knight's Cross and promoted to SS-Standartenführer (colonel). Since 1947 Rebane lived in Great Britain, in 1961 he moved to Germany and stayed there until his death in 1976. Rebane's ashes were returned to Estonia and reburied with honors in 1999.