He was the eldest son of Richard Woodville, 1st Earl Rivers and Jacquetta of Luxembourg. Like his father, he was originally a Lancastrian, fighting on that side at the Battle of Towton, but later became a Yorkist.
Rivers became very influential at the royal court after his sister Elizabeth married Edward IV. He joined the king in his temporary exile in 1470, and returned with him the next year, where he was wounded at the Battle of Barnet.
In 1473 the Edward IV appointed Rivers Governor of the Prince of Wales' household. When the king died in 1483, he accompanied the Prince, now King Edward V, on the way back to London. However, they were waylaid by the Duke of Gloucester, who imprisoned the Earl and then had him beheaded as part of his path toward becoming king (as Richard III).
Rivers was evidently quite learned. One of the first, if not the first, book printed in England was his translation from French of the Sayings of the Philosophers, printed by William Caxton in 1476. Lambeth Palace Library has an illustration showing Rivers presenting a copy of this book to Edward IV.
{| border="2" align="center"
|-
|width="30%" align="center"|Preceded by:
Richard Woodville
|width="40%" align="center"|Earl Rivers
|width="30%" align="center"|Followed by:
Richard Woodville
|}