Born in Barnsley, McCarthy made his league debut for Barnsley F.C in 1977. A strong central defender, after 272 appearances for Barnsley he went to Manchester City (1983-87), Glasgow Celtic (1987-1989), Olympique Lyonnais (1989-90) and Millwall. His father being Irish, he was eligible for selection for the Republic of Ireland's national team; he made his international debut for them in 1984. He won 57 caps up to June 1992 and was the well-respected captain for his side, "Captain Fantastic".
He joined Millwall in March 1990 and became player-manager in 1991, succeeding Bruce Rioch. After relative success at Millwall on February 5, 1996 McCarthy was appointed successor to Jack Charlton as Ireland manager. Millwall went on to be relegated that season under Jimmy Nicholl.
Despite failure to qualify for the World Cup of 1998 or the European Championship of 2000 McCarthy held his job. Ireland qualified for the 2002 World Cup in Japan, but their tournament was overshadowed by a very public and bitter spat between McCarthy and the team's star player Roy Keane, who returned home without kicking a ball. The Irish media sided with Keane, and McCarthy was heavily criticised for his handling of the player; this, in spite of a relatively successful campaign, reaching the Second Round to be eliminated by Spain in a penalty shootout. The media's vilification became increasingly intense and personal after a poor beginning to their qualifying campaign for the European Championship of 2004; eventually, on November 5, 2002, McCarthy resigned from the post.
While his record as national manager was uneven - of 68 games his team won 29, drew 19, and lost 20 - he managed to keep Ireland in the top 20 international teams in the world, no mean achievement for a small country with few top quality players. However, it seems likely that his reputation as Ireland manager will always be overshadowed by the Keane affair.
On March 12, 2003 he was appointed manager of struggling Sunderland as an immediate replacement for Howard Wilkinson, who was sacked after six successive Premiership defeats left the club facing near-certain relegation. McCarthy's hiring did not stop Sunderland's slide, and the Black Cats were relegated at the end of the season. However, he largely escaped blame for the relegation, and was retained as manager.