It is the heart of the Nintendo64 next to the MIPS 4300i . It was developed by SGI specially for Nintendo, and little is known about it, and the way it works.
The RCP features a vector processor that can perform over 500,000,000 arithmetic operations per second, making it a very powerfull chip compared even to Intel's processors at the time (in raw processing power).
It handles the audio and video of the Nintendo64 and can perform very well in operations such as: real-time edge anti-aliasing, high frame-rate on complex images, automatic load-management, advanced texture mapping, real-time depth buffering, etc.
In general, what made the Nintendo64 more advanced than other video game consoles of its time, was mainly the RCP.