His first major literary sensation was The Autobiography of an Ex-Colored Man (1912), in which he describes his success in overcoming the color barrier in America at the turn of the century. Written while he was still serving in the diplomatic corps, he resigned shortly after, realizing that he had slim chances for further promotion.
It was while serving as executive secretary of the NAACP that he released God's Trombones, the work he is best remembered for today. For several years previously, he had collected and published anthologies of African American poetry and folklore, when he determined that "A good deal has been written on the folk creations of the American Negro; his music, sacred and secular; his plantation tales, and his dances; but that there are folk sermons, as well, is a fact that has passed unnoticed." Rather than collect the sermons, he transformed them into verse to capture the nuances of the "rhythmic intoning."
Other works by Johnson include an opera, Tolosa, and the lyrics for "Lift Every Voice and Sing", to which his brother wrote the music. The song later became known as the "Negro National Anthem." He also wrote Fifty Years and Other Poems (1917), The Book of American Negro Poetry (1922), The Book of American Negro Spirituals (1925), and The Second Book of Negro Spirituals (1926), the novel Black Manhattan (1930), another autobiography Along This Way (1933), and Negro Americans, What Now? (1934), a book calling for civil rights for African Americans.
Johnson was one of the first African-American professors at New York University. He died while on vacation in Wiscasset, Maine in 1938, when the car he was driving was hit by a train.