If G is finite, this is equivalent to requiring that the order of G (the number of its elements) itself be a power of p. Quite a lot is known about the structure of finite p-groups. One of the first standard results using the class equation is that the center of a finite p-group cannot be the trivial subgroup. More generally, every finite p-group is both nilpotent and solvable.
p-groups of the same order are not necessarily isomorphic; for example, the cyclic group C4 and the Klein group V4 are both 2-groups of order 4, but they are not isomorphic. Nor need a p-group be abelian; the dihedral group D8 is a non-abelian 2-group.
In an asymptotic sense, almost all finite groups are p-groups. In fact, almost all finite groups are 2-groups. The sense taken is that if you fix a number n and choose uniformly randomly from a list of all the isomorphism classes of groups of order at most n, then the probability that you pick a 2-group tends to 1 as n tends to infinity. For instance, if n = 2000, then the probability of picking a 2-group of order 1024 is greater than 99%.
Every non-trivial finite group contains a subgroup which is a p-group. The details are described the Sylow theorems.
For an infinite example, let G be the set of rational numbers of the form \m/pn where m and n are natural numbers and m < pn. This set becomes a group if we perform addition modulo 1. G is an infinite abelian p-group, and any group isomorphic to G is called a p∞-group. Groups of this type are important in the classification of infinite abelian groups.
The p∞-group can alternatively be described as the multiplicative subgroup of C - {0} consisting of all pn-th roots of unity, or as the direct limit of the groups Z / pnZ with respect to the homomorphisms Z / pnZ → Z / pn+1Z which are induced by multiplication with p.