Stevens was born in Brentford, the son of a tap dancer. He used to listen to jazz as a child, but was initially more interested in drawing and painting (mediums through which he expressed himself throughout his life). He studied at the Ealing College of Art and then started work in a design studio. He left at 19, however, to join the Royal Air Force. He studied the drums at the Royal Airforce School of Music in Uxbridge, and while there met Trevor Watts and Paul Rutherford, two musicians who became close collaborators.
In the mid-1960s, Stevens began to play in London jazz groups alongisde musicians like Tubby Hayes and Ronnie Scott, and in 1965 he fronted a septet. Influenced by the free jazz he was hearing coming out of the United States by players like Ornette Coleman and Albert Ayler, his style began to move away from fairly traditional be-bop to something more experimental.
In 1966, SME was formed with Watts and Rutherford and the group moved into the Little Theatre Club in the centre of London to develop their new music. In 1967 their first album, Challenge, was released. Stevens then became interested in the music of Anton Webern, and the SME began to play generally very quiet music. Stevens also became interested in non-Western musics.
The SME went on to make a large number of records with an ever changing line-up and an ever changing number of members, but Stevens was always there, at the centre of the group's activity. He also played in a number of other groups, drumming in Watts' group Amalgam and later forming bands like Freebop and Fast Colour, for example, but the SME remained at the centre of his activities.
In the latter part of 1967, Evan Parker joined the SME and worked closely with Stevens in the group, eventually becoming one of the longest standing members. He later summed up Stevens' approach to improvising in two basic maxims: if you can't hear another musician, then you're too loud; and there is no point in group improvisation if what you are playing doesn't relate to what other members of the group are playing.
Stevens also devised a number of basic starting points for improvisation. These were not "compositions" as such, but rather a means of getting improvisational activity started, which could then go off in any direction. One of these was the so-called "Click Piece" which essentially asked for each player to repeatedly play a note as short as possible.
Stevens played alongside a large number of prominent free improvisors in the SME, including Derek Bailey, Peter Kowald and Julie Tippetts, but from the 1970s, the make-up of the SME began to settle down to a regular group of Stevens, Nigel Coombes playing violin, and Roger Smith playing guitar.
From 1983, Stevens was involved with Community Music, an organisation through which he took his form of music making to youth clubs, mental health institutions and other unusual places. Notes taken during these sessions were later turned into a book for the Open University called Search and Reflect (1985).
The SME continued to play, the last time being in 1992 with a group including John Butcher. Stevens died in 1994.