Signed out of high school by the New York Mets on June 12, 1964, McGraw became a part of their bullpen a year later. The Mets tried him as a starting pitcher, but he only managed a 2-12 record in 16 starts over two years. After spending all of 1968 in the minor leagues, he became a full-time reliever in 1969. Relying on a good screwball, he racked up twelve savess for the Miracle Mets as they went on to win the World Series, but he did not himself pitch in the Fall Classic.
He became one of the more successful closers in baseball during the early 1970s, placing second in the National League in saves in 1972 and 1973. During the 1973 season, he coined a popular rallying cry for the Mets, "Ya Gotta Believe!" That year, the Mets won the National League East with only 82 wins, but managed to make the World Series, losing to the Oakland Athletics in seven games.
On December 3, 1974, McGraw was traded to the Philadelphia Phillies in a six-player deal. At the time of the trade, McGraw was the all-time Mets leader in saves, games pitched, and games finished.
As a Phillie, he continued his role as a reliable relief pitcher. In 1980, he finished fifth in the NL Cy Young Award voting, compiling 20 saves and a 1.46 ERA and helping the Phillies win the NL East. In the playoffs, he appeared in all five games of the National League Championship Series, saving two of them. His finest efforts came in the World Series, striking out ten batters in 7 2/3 innings. He saved the final game by striking out Willie Wilson, clinching the Phillies' first World Series championship.
He spent the next four seasons as a set-up man rather than a closer, and retired after the 1984 season. He compiled 180 saves in his career, tied for eighth-best in Major League history at the time.
McGraw played in two All-Star Games.
On March 12, 2003, McGraw was working as a spring training instructor for the Phillies when he was hospitalized with a brain tumor. Surgery done to remove it revealed that he had cancer. Given three weeks to live by doctors, he managed to survive nine months. During this time, he was part of the closing ceremonies of Veterans Stadium, where he recreated the final out of the Phillies' World Series triumph.
At the time of his death, McGraw was ranked:
McGraw was a popular and colorful player who uttered a number of memorable quotes: