The library was initially located in a former schoolhouse located on Mason Street, and was opened to the public on March 20, 1854. Later that year, the construction of a new building was authorized by the Library Commission, to be located at the corner of Boylston Street and Dartmouth Street on Copley Square. Charles Follen McKim, of the architectural firm McKim, Mead, and White, was engaged to design the new building, which opened in 1895. The new building included a children's room, the first in the nation.
A modernist addition designed by Philip Johnson was completed in 1972; the Johnson Building now houses the circulating collection, while the research library is located in the McKim Building.
The main branch is located on Boylston Street in Boston. It the home of several large collections, including first edition folios by William Shakespeare, and the personal library of John Adams.