Forced to leave Russia and with other radical political figures living in exile, Martov joined the Russian Social Democratic Labour Party (RSDLP). At the Second Congress of the RSDLP in London in 1903, there was a dispute between Martov and Vladimir Lenin. Lenin published his ideas for moving the party forward in his pamphlet What is to be Done?, arguing for a small party of professional revolutionaries with a large fringe of non-party sympathizers and supporters. Martov disagreed believing it was better to have a large party of activists.
Martov based his ideas on the socialist parties that existed in other European countries such as the British Labour Party. At the end of the debate Martov won the vote 28-23 . Vladimir Lenin was unwilling to accept the result and formed a faction known as the Bolsheviks. Those who remained loyal to Martov, such as George Plekhanov, Fedor Dan and Irakli Tsereteli, became known as Mensheviks. At this stage Leon Trotsky who would become one of the Bolshevik leaders in the Russian Revolution in 1917 supported Martov's approach.
After the reforms brought about by the 1905 Revolution, Martov argued that it was the role of revolutionaries to provide a militant opposition to the new bourgeois government. He advocated the joining a network of organisations such as trade unions, cooperatives, village councils and soviets to harass the bourgeois government until the economic and social conditions made it possible for a socialist revolution to take place.
After the February Revolution in 1917 Martov returned to Russia but was too late to stop some Mensheviks joining the Provisional Government. He strongly criticized those Mensheviks such as Irakli Tsereteli and Fedor Dan who now part of Russia's government, supported the war effort. However at a conference held on June 18, 1917, he failed to gain the support of the delegates for a policy of immediate peace negotiations with the Central Powers.
When the Bolsheviks came to power in October, 1917 Martov became politically marginalised, best exempified by the then Bolshevik, Trotsky's comment to him as he left a meeting of the council of Soviets in disgust at the way in which the Bolsheviks had seized political power, "go to where you belong, the dustbin of history". For a while Martov led the small Menshevik opposition group in the Constituent Assembly until the Bolsheviks abolished it. The Mensheviks were banned along with other political parties (except for the Communists) by the Soviet government.
Martov supported the Red Army against the White Army during the Russian Civil War, however, he continued to denounce the persecution of liberal newspapers, the nobility, the Cadets and the Socialist-Revolutionaries.
In 1920 Martov was forced into exile and he died in Schomberg, Germany in that year.