Born in London, the son of a clergyman, Greene became a choirboy at St Paul's Cathedral under Robert King. He studied the organ under Richard Brind, and after Brind died, Greene became organist at St Paul's.
With the death of William Croft in 1727, Greene became organist at the Chapel Royal, and in 1730 he became professor of music at Cambridge University. In 1735 he was appointed Master of the King's Musick. At his death, Greene was working on the compilation Cathedral Music, which he left to his student William Boyce, also his successor as Master of the King's Musick, to complete. Many items from that collection are still used in Anglican services today.
Greene wrote a good deal of vocal music, both sacred and secular, including the oratorios Jephtha and The Force of Truth, and settings of sonnets from Edmund Spenser's Amoretti.