Small cannon and large-calibre rifles were used against the early WWI tanks being introduced by the BEF, many of which proved to be almost useless. By the end of the war a number of light guns, typically 37mm (2-pounder in British measure) were being deployed on short carriages that proved to be considerably better. In addition most forces deployed large high-velocity rifles, typically of 50-cal (12.5mm) calibre, with enough power to puncture the thin armor of the tanks of the era.
At the start of World War II many of these weapons were still being used operationally, along with a newer generation of light guns that little changed from their WWI counterparts. In combat both proved entirely useless against the larger and better armored tanks they faced. All combatants quickly introduced newer and more powerful guns, and the anti-tank rifle had largely disappeared by 1942. As the guns grew in size they dropped in mobility, leading to the development of the anti-tank vehicle, typically a tank chassis with a much larger gun mounted on top. However the infantry were left with little support in most cases.
Fortunately for the infantry, anti-tank weapons based on HEAT warheads using a Monroe effect shaped charge were developed. These self-forging explosive munitions could penetrate even more armor than some of the largest anti-tank guns. Typically delivered by a small rocket fired at short range, HEAT formed the basis of the British PIAT, US Bazooka and German Panzerfaust, Panzerschreck and the recoilless rifle. In many cases these weapons were so effective that the concepts of Blitzkrieg could often be stopped cold by properly equipped troops.
In the post-WW II era HEAT became almost universal for a while, and rockets were supplanted by the wire guided missile as a delivery system. The British developed the HESH, or high explosive squash head, as an anti-concrete device for attacking fortifications, and found it surprisingly effective against tanks in the 1950s and 1960s. However, increases in depth of armor and improvements in armor technology meant that hand-held rockets were no longer large enough to deliver enough power by the 1970s, and the introduction of Chobham armour by the UK and reactive armor by the USSR, forced the HEAT rounds to be so large that they are no longer truly man-portable.
Today the anti-tank role is filled with a variety of weapons, from portable "top attack" missiles, to larger HEAT based missiles for use from jeeps and helicopters, a variety of high velocity autocannon, and ever-larger heavy tank guns.