He debuted as a professional in 1958 with a first round knockout of (boxer) George Hamilton, in New York. 12 wins in a row followed, 10 of them by knockout (including wins vs. contenders Ike Jenkins and Al Andrews, after which he was able to make his San Juan debut against Benny Paret, a world welterweight and Middleweight champion whose death after a fight would later on become one of the turning points in the history of boxing. Torres and Paret fought to a 10 round draw, and in 1960, Torres went back to campaigning in New York, where he scored three wins that year, all by decision, including two over Randy Sandy.
In 1961, Torres made his hometown debut with a four round knockout win in a rematch with Hamilton at Ponce. He made six more fights that year, winning all of them by knockout.
1962 came by and Torres kept his knockout streak alive with three more knockout wins, but in 1963, he suffered his first loss, being stopped in five by Argentina's Florentino Fernandez, the only boxer ever to beat Torres by a knockout as a professional. After that setback, Torres went back to training and had one more fight that year, and that time around, he was able to beat another top conten der in Don Fullmer, Gene Fullmer's brother, with a ten round decision win in New Jersey.
In 1964, Torres beat a group of name boxers, including Jose Gonzalez, Walker Simmons (twice), Frankie Olivera, Gomeo Brennan and former world Middleweight champion Carl Olson (Bobo), taken out in one round. After this, Torres was ranked number 1 among Light Heavyweight challengers, and his title shot would arrive soon.
In 1965, it finally did. Met at Madison Square Garden with fellow International Boxing Hall Of Fame member and then world Light Heavyweight champion Willie Pastrano, Torres became the third Puerto Rican world boxing champion in history and first Latin American to win the Light Heavyweight title, knockoing Pastrano out in round nine. He made a non title bout versus Tom McNeeley (father of former Mike Tyson rival Peter McNeeley) in San Juan, winning a ten round decision.
In 1966, he successfully defended his crown three times, with 15 round decisions over Wayne Thornton and Eddie Cotton and a two round knockout of Chic Calderwood. In his next defense, however, he would lose it to another Hall Of Fame member, Nigeria's Dick Tiger, by a decision in 15 rounds.
In 1967, he and Tiger had a rematch, and Torres lost a 15 round decision again. Many fans thought he should have won it that time, and as a consequence, a large scale riot followed the fight, with many New York City policemen called to the Garden arena to try to calm down the fans.
After his second defeat to Tiger, Torres only fought twice more, retiring after 1969.
In his years after retiring from boxing,he became a representant of the Puerto Rican community in New York, meeting political leaders, giving lectures and becoming the New York State Athletic Commission's Commissioner from 1984 to 1988. In 1986, he was chosen to sing the United States National Anthem before the world Lightweight championship bout between Jimmy Paul and Irleis Perez in Atlantic City, New Jersey, and in 1987, he released a book about Tyson. In 1990 he became President of the WBO, and he performed as President there until 1995.
He is also a member of the International Boxing Hall Of Fame.
He is the author of Sting Like a Bee (ISBN 0071395881), a biography of Muhammad Ali.