He rose to prominence as a member of Duke Ellington's orchestra, with which he performed from 1929 to 1940. He also recorded his own sessions during this time, both freelance and with other Ellington sidemen. In 1940 he joined Benny Goodman's orchestra, then in 1941 formed his own orchestra, in which over the years he employed Charlie Parker, Eddie "Lockjaw" Davis, Bud Powell, Eddie Vinson, and other important young players.
He began to play more rhythm and blues in the late 1940s. In the 1950s he toured with small groups and fell into obscurity. In 1962 he rejoined Ellington and stayed with the orchestra till 1974, after Ellington's death.
He was renowned for his use of the plunger mute, and is reputed to have inspired Wynton Marsalis's use of it.