Atkinson was born in 1831 in the United Kingdom town of Broxton. He was educated in Britain, but chose to settle in New Zealand when he was twenty-two years old. He was accompanied by his brother Arthur. On arriving in New Zealand, the two bought farmland in Taranaki. According to his correspondence, he was highly satisfied with his decision to move to New Zealand, seeing it as an opportunity to prosper.
Atkinson was first involved in politics when he became a member of the provincial council. Of particular interest to him was policy regarding Maori-owned land, which he wished to see taken over by the British settlers. Continued Maori ownership, he believed, prevented economic development for the colony. Atkinson considered the Maori to be "savages", and believed that war was a reasonable option for ensuring Maori cooperation with British land acquisition.
When fighting broke out in Taranaki between Maori and the settlers, Atkinson helped to organize a number of volunteer units to fight the Maori. He himself fought in a number of battles. The importance of Atkinson's contribution is debated, but his endeavours earned him respect from like-minded politicians.
In 1861, Atkinson was elected to parliament, unopposed in his electorate. In 1864, he was made Defence Minister in the government of Frederick Weld. He was highly active in this portfolio. In 1866, however, he retired due to the death of his wife Amelia (whom he had married in 1856). The following year, he married his cousin Annie. He briefly returned to parliament from 1867 to 1869, but afterwards concentrated on maintaining his farm.
In 1972, Atkinson returned to politics in order to defeat a candidate who was allied with William Fox (a prominent supporter of Maori land rights). Atkinson declared that he would "not see a Foxite get in", and narrowly defeated the candidate. Once in parliament, Atikinson soon became involved in economic matters, opposing the policies of Julius Vogel (who also happened to be a supporter of Maori land rights). Vogel, who supported extensive borrowing to finance public works, was attacked by Atkinson as reckless. Vogel's response was that Atkinson was overly cautious, and would delay economic progress.
Atkinson and Vogel both agreed, however, that borrowing by provincial government (as opposed to the central government) was indeed out of control. The two also believed that provincial politicians were petty and self-interested, and that more cooperation was needed between provinces and the state. It was this shared view of provincial government than enabled Vogel and Atkinson to cooperate, although they never resolved their differences on borrowing by the central government or on dealings with the Maori. Atkinson eventually became part of Vogel's cabinet, but not with portfolios related to negotiations with the Maori or to finance. He did continue to express his opinions on these matters, but found it increasingly harder to convince people of his views.
In 1876, Vogel retired, and Atkinson managed to secure the Premiership. One of his first acts was to abolish the provinces. He also took over direct responsibility for financial policy, and implemented a less aggressive strategy for borrowing. He attempted to reform the system by which money was handled, placing all responsibility for borrowing with the government while increasing control of spending at a district or municipal level. However, growing economic problems caused his plan to encounter difficulties. As the economy declined, Atkinson became more and more unpopular.
Atkinson lost power in 1877, only slightly over a year after he gained it. He entered opposition, continuing to promote his ideas of financial caution. He also proposed a number of other measures, including national insurance. In 1883, he managed to make a comeback, gaining the Premiership for eleven months before losing it to Robert Stout. The two then engaged in a protracted struggle for the leadership. A strong counter-offensive by Atkinson enabled him to unseat Stout again after only twelve days. Stout, however, was not so easily defeated, and took the Premiership again after seven days. This time, Stout held his position for three years, defeating Atkinson's attempts to oust him. Eventually, however, Atkinson persevered, and gained the post in 1887.
In 1891, Atkinson was finally removed from the office of Premier by John Ballance of the newly created Liberal Party, the country's first organized political block. The Liberals, who represented the ideas of William Fox, Julius Vogel, and many other of Atkinson's opponents, were to hold power for twenty-one years after Atkinson's defeat, but Atkinson was not to see this - he died in his parliamentary office a year after the Liberal Party's victory.