Pryce was born on in Cardiff. He was in the underground band Anhrefn with Gruff Rhys and Dafydd Ieuan before Super Furry Animals was formed.