Jim Mitchell began his political involvement at the age of 11 when he supported Sean MacBride, leader of the radical republican Clann na Poblachta in the 1957 general election. He joined Fine Gael in 1967, becoming that party's candidate in a by-election in 1970. He sought a party nomination to run in the 1973 Irish general election. However he agreed not to contest the seat to allow Declan Costello, a senior figure in his party and son of former Taoiseach John A. Costello, to be elected. Costello went on to serve as Attorney-General in the 1973-1977 National Coalition of Fine Gael and Labour.
Mitchell was elected to Dublin Corporation in 1974. In 1976, aged 29, he became the youngest ever Lord Mayor of Dublin. In 1976 Mitchell won a seat in Dáil Éireann in a by-election in the constituency of Dublin South West, moving to the new constituency of Dublin Ballyfermot in the 1977 general election. With the party's loss of power in 1977, the new leader, Garret FitzGerald appointed Mitchell to the Party's Front Bench as spokesman on Labour. In 1981, following a dramatic increase in party support, Garret FitzGerald on his appointment as Taoiseach caused some surprise by excluding some of the older conservative ex-ministers from his cabinet. Instead young liberals like Mitchell were appointed, with Mitchell receiving the high profile post of Minister for Justice, taking responsibility for policing, criminal and civil law reform, penal justice, etc. The Fine Gael-Labour government collapsed in January 1982, but regained power in December of that year. Mitchell again was included in a FitzGerald cabinet, as Minister for Transport.
Mitchell, who was seen as to the left of Fine Gael was however out of favour with John Bruton when he became Fine Gael leader in 1990. When Bruton formed the Rainbow Coalition in December 1994 Mitchell was not appointed to any cabinet post.
Mitchell contested and won Dáil elections in 1976, 1977, 1981, (February and November) 1982, 1987, 1989, 1992, 1997. He ran for his party as its candidate to become a member of the European Parliament in the 1994 and 1998 elections. He also was Director of Elections for Austin Currie, the Fine Gael candidate, in the Irish presidential election, 1990.
In 2001, Bruton was deposed as Fine Gael leader, and replaced by Michael Noonan. Mitchell served as his deputy from 2001 to 2002. Mitchell also chaired the key Oireachtas Public Accounts Committee. The Committee's work under Mitchell's chairmanship was widely praised for its inquiry into allegations of corruption and widescale tax evasion in the banking sector.
Though regarded in politics as one of Fine Gael's "survivors", who held onto his seat amid major boundary changes, constituency changes and by attracting working class votes in a party whose appeal was primarily middle class, Mitchell lost his seat in the June 2002 general election. That election witnessed a large scale collapse in the Fine Gael vote, with the party dropping from 54 to 31 seats in Dáil Éireann. MItchell had earlier had a liver transplant in an attempt to beat a rare form of cancer which had cost the lives of a number of his siblings. Though the operation was successful, the cancer returned. Though he appeared to be making a recovery, Jim Mitchell ultimately died of the disease in December 2002.
His former constituency colleague and rival, Taoiseach Bertie Ahern, described Jim Mitchell as having made an "outstanding contribution to Irish politics."