The term also describes the organization by that name which is the focus of the book. The organization's existence had been revealed in Foundation (and nothing more), searched for in Foundation and Empire, and it makes brief apperances in this novel. It would not be described in detail until Foundation's Edge.
Warning: Wikipedia contains spoilers.
Part I: Search By the Mule is about the Mule's search for the elusive Second Foundation, with the intent of destroying it. In the end, Preem Palver, the first Speaker of the Second Foundation, telepathically modifies the Mule to make him not care about finding the Second Foundation.
Part II: Search By the Foundation takes place a few decades after the first part, some years after the Mule's death (natural causes). The members of the (first) Foundation are now fully aware that the Second foundation is out there (they had known of its existence all along, but it had never sunk in until they were able to turn the Mule back)
After inventing a device that jams telepathic abilities while simultaneously causing telepaths great pain, the Foundation finds and locates telepaths on Terminus, `at the other end of the galaxy' (from the first foundation, also at Terminus). Thus, they declare the Second Foundation destroyed and are content to forget the matter. This would be revisited in Foundation's Edge.
The Novel
It is written in two distinct parts. The Organization
The Second Foundation was founded by Hari Seldon as budding colony of mentalics, people with telepathic abilites. Located at Star's End (The Great Library of Trantor), they are governed by a council of the strongest telepaths, called the Speakers. The leader of the group is the first speaker.