According to what is at present the most common typology among sociologists, religious groups are classified as ecclesia, denominationss, cults or sects. Note that sociologists give these words precise definitions which are different from how they are commonly used. Note especially that the words 'cult' and 'sect' as used by sociologists are free from prejudice, even though the popular use of these words is often highly pejorative.
The below text deals only with one theory; work of other authors needs to be included.
According to the theory of R. Stark & W. S. Bainbridge, religions are systems of "compensators". Compensators are a body of language and practices that compensate for some physical lack or frustrated goal. They can be divided into specific compensators (compensators for the failure to achieve specific goals), and general compensators (compensators for failure to achieve any goal).
It has been observed that social or political movements that fail to achieve their goals will often transform into religions. As it becomes clear that the goals of the movement will not be achieved by natural means (at least within their lifetimes), members of the movement will look to the supernatural to achieve what cannot be achieved naturally. The new religious beliefs are compensators for the failure to achieve the original goals. Examples of this include the counterculture movement in America: the early counterculture movement was intent on changing society and removing its injustice and boredom; but as members of the movement proved unable to achieve these goals they turned to Eastern and new religions as compensators.
The most general sociological theory of the formation of religions so far is contained in R. Stark & W. S. Bainbridge's book "Theory of Religion". This theory is outlined roughly below:
Most religions start out their lives as cults or sects, i.e. groups in high tension with the surrounding society. Over time, they tend to either die out, or become more established, mainstream and in less tension with society. Cults are new groups with a new novel theology, while sects are attempts to return mainstream religions to (what the sect views as) their original purity. Mainstream established groups are called denominations. The comments below about cult formation apply equally well to sect formation.
There are four models of cult formation: the Psychopathological Model, the Entrepreneurial Model, the Social Model and the Normal Revelations model.
According to the Psychopathological Model, religions are founded during a period of severe stress in the life of the founder. The founder suffers from psychological problems, which they resolve through the founding of the religion. (The development of the religion is for them a form of self-therapy, or self-medication.)
According to the Entrepreneurial Model, founders of religions act like entrepreneurs, developing new products (religions) to sell to consumers (to convert people to). According to this model, most founders of new religions already have experience in several religious groups before they begin their own. They take ideas from the pre-existing religions, and try to improve on them to make them more popular.
The Social Model emphasises not the founder of the religion, but rather the early religious group. According to this model, religions are founded by means of social implosions. Members of the religious group spend less and less time with people outside the group, and more and more time with each other within it. The level of affection and emotional bonding between members of a group increases, and their emotional bonds to members outside the group diminsh. According to the social model, when a social implosion occurs, the group will naturally develop a new theology and rituals to accompany it.
The Normal Revelations model was added to the theory by Stark in a latter work. According to the Normal Revelations model, religions are founded when the founder interprets ordinary natural phenomena as supernatural; for instance, ascribing his or her own creativity in inventing the religion to that of the deity.
Some religions are better described by one model than another, though all apply to differing degrees to all religions.
Once a cult or sect has been founded, the next problem for the founder is to convert new members to it. Prime candidates for religious conversion are those with an openness to religion, but who do not belong or fit well in any existing religious group. Those with no religion or no interest in religion are difficult to convert, especially since the cult and sect beliefs are so extreme by the standards of the surrounding society. But those already happy members of a religious group are difficult to convert as well, since they have strong social links to their pre-existing religion and are unlikely to want to sever them in order to join a new one. The best candidates for religious conversion are those who are members of or have been associated with religious groups (thereby showing an interest or openness to religion), yet exist on the fringe of these groups, without strong social ties to prevent them from joining a new group.
Potential converts vary in their level of social connection. New religions best spread through pre-existing friendship networks. Converts who are marginal with few friends are easy to convert, but having few friends to convert they cannot add much to the further growth of the organization. Converts with a large social network are harder to convert, since they tend to have more invested in mainstream society; but once converted they yield many new followers through their friendship network.
Cults initially can have quite high growth rates; but as the social networks that initially feed them are exhausted, their growth rate falls quickly. On the other hand, the rate of growth is exponential (ignoring the limited supply of potential converts): the more converts you have, the more missionaries you can have out looking for new converts. But nonetheless it can take a very long time for religions to grow to a large size by natural growth. This often leads to cult leaders giving up after several decades, and withdrawing the cult from the world.
It is difficult for cults and sects to maintain their initial enthusiasm for more than about a generation. As children are born into the cult or sect, members begin to demand a more stable life. When this happens, cults tend to lose or de-emphasise many of their more radical beliefs, and become more open to the surrounding society; they then become denominations.
The goal or dream of most founders of religions is to convert their entire society; but of the myriad religions founded throughout history, few have been very successful. Most of the world's religious market share is taken up by a few religions (Christianity, Islam, Hinduism, Buddhism). It is very difficult for a religion to grow to this size, and depends more on the luck than the skill of the religion's founder. Most of these religions (especially Christianity) established themselves by penetrating the social network of the society's elite. Once this network was penetrated, the religion quickly controlled the elite of the society. The religion of the common people took much longer to change (sometimes centuries), but once the elite had changed their religion the change of the whole society was inevitable.Typology of Religious Groups
The sociological view of religion
Sociological theories of the formation of religion