Colonies have long provided a convenient destination for surplus populations: landless younger sons (who go "out into the wide world to seek their fortunes"), "ne'er-do-wells" (idle, irresponsible persons) and remittance men (people who received a scheduled remittance or allowance from their families, often made in order to keep them away from their original country because they seemed embarrassingly eccentric and could cause problems for their family). But for states with vast empires, colonies also provide a cheap way of getting rid of convicted prisoners and exiling political dissidents: sending them sufficiently far away to discourage escape (or even return after sentence-expiry), and to places otherwise inhospitable where their (unpaid) labour can redound to the metropole's advantage before later bulk "free" immigration becomes viable. In this way governments may achieve fantasies of penology like "locking them up and throwing away the key".
Thus Great Britain once recycled its petty criminals by shipping them to North America as indentured labour. When that avenue closed in the 1780s after the American Revolution, Britain switched the destination to what later became parts of Australia: Norfolk Island, Van Diemen's Land and New South Wales. Awkward advocates of Irish Home Rule or of Trade Unionism (the Tolpuddle Martyrs) received sentences of transportation and joined the thieves and embezzlers and prostitutes in remote Australian colonies.
France elected to send serious criminals to uncomfortable tropical penal colonies: Devil's Island in French Guiana received forgers and other criminals; for a time New Caledonia received dissidents like the Communards.
Both Imperial Russia and the Soviet Union used Siberia as a dumping-ground penal colony for both criminals and dissidents. Though geographically contiguous with heartland Russia, Siberia provided both remoteness and an unwelcomingly harsh climate. Alongside the popular image of Siberian saltmines, the Gulag and its predecessors developed forestry and construction enterprises.
The idea of remote and inhospitable prison planets has also understandably appealed to science fiction writers. Some of the most famous examples include Salusa Secundus in Frank Herbert's Dune, and the penal colony in Alien 3, though many more exist.
The Penal Colony is a short story by Franz Kafka.