In 1945, Brown ran away from her home in Portsmouth along with a trumpeter, Jimmy Brown, whom she soon married. She then spent a month with Lucky Millinder's orchestra, but was fired after she brought drinks to the band for free. Blanch Calloway, Cab Calloway's sister, arranged a gig for Brown at a nightclub called Crystal Caverns and soon became her manager. Willis Conover, a local DJ, caught her act and recommended her to Atlantic Records bosses, Ahmet Ertegun and Herb Abramson. Brown was unable to audition as planned though, because of a serious car accident that resulted in a nine-month hospital visit.
At her first audition though, in 1949, she sang "So Long", which ended up becoming a hit. This was followed by "Teardrops in my Eyes" (1950), "I'll Wait for You" (1951), "I Know" (1951), "5-10-15 Hours" (1953), "(Mama) He Treats Your Daughter Mean" (1953), "Oh What a Dream" (1954), "Mambo Baby" (1954) and "Don't Deceive Me" (1960). She also became known as Miss Rhythm during this period.
During the 1960s, Brown faded from public view and only returned to music in the 1970s, followed by a series of comedic acting gigs, including a role in sitcom "Hello, Larry" and the John Waters film "Hairspray", as well as earning a Tony Award for her Broadway performance of "Black and Blue".