The split was co-ordinated by the American section, the Socialist Workers Party, and included the British section under Gerry Healy and the French Parti Communiste Internationaliste (PCI). Small groups in various other countries, notably Nahuel Moreno's group in Argentina, also joined.
The grouping's founding statement was the Open Letter written by James P. Cannon, outlining the disputes they had with the International Secretariat of the Fourth International.
Despite some initial strong rhetoric aimed at the ISFI, the American SWP were in fact eager for a reconciliation, and so discouraged the committee from officially declaring itself the Fourth International. Instead, they claimed to be dedicated to the founding principles of the International. The PCI and the British group, soon to rename itself the Socialist Labour League (SLL), had little interest in reconciliation, and instead stepped up their attacks, especially on Pablo.
The major groups in the ICFI actually spent little time working together. The first congress did not take place until 1958, and the SWP officially only acted as observers at the event being prevented from affiliating by law.
In the ISFI, Pablo had lost prestige, and as both the SWP and the ISFI hailed the Cuban Revolution as unconsciously furthering Trotskyism, they grew together. In 1963, the SWP left the ICFI to work with the ISFI, forming the United Secretariat of the Fourth International.
The ICFI continued, now essentially led by the SLL and a second congress was held in Leeds in 1966. Most of the remaining Americans were expelled in 1966 to form the International Spartacist Tendency. The French section, now renamed the Organisation Communiste Internationaliste (OCI), fell out with the SLL over Healy's support of revolutionary nationalism in various Arabic countries. The OCI encouraged the Bolivian Partido Obrero Revolucionario (POR) to join the ICFI, in order to support their positions, but by 1971 were forced to leave the group.
The OCI and POR split from the ICFI in 1971 to form an even smaller international, the Organising Committee for the Reconstruction of the Fourth International. The SLL, soon to rename itself once more as the Workers Revolutionary Party, maintained the ICFI name for itself and a few tiny affiliated sections. After the disintegration of the WRP in the mid-1980s, the remaining ICFI split into several groups, most claiming to be the official one. Perhaps only one of these survived to mark the ICFI's 50th anniversary, that of the various tiny Socialist Equality Party groups.