It should be noted that the impact to the target can be no greater than the impact of the recoil, due to the law of conservation of momentum. However, the smaller size of the bullet, compared to the gun-and-shooter system, allows significantly higher energy to be imparted to the bullet than to the shooter, giving guns their lethal effect. Consider a system where the shooter-and-gun have mass M and the bullet has mass m, and the two systems move away from one another with new velocities V and v.
Then, by the law of conservation of momentum, MV = mv, and thus V = mv/M and the kinetic energies imparted to the two systems are respectively 1/2 MV2 and 1/2 mv2. The energy imparted to the shooter can then be written as
When the bullet strikes, its high velocity and small area means that it will exert large forces on any object it hits. This usually results in it penetrating any soft object, such as flesh. The energy is then dissipated in the wound tract formed by the passage of the bullet. See terminal ballistics for a fuller discussion of these effects.
The reversal of this mathematics is the reason why bulletproof vests work: the vest's material, usually Kevlar, works by presenting a series of material layers which catch the bullet and spread its momentum, hopefully bringing the round to a stop before it can penetrate into the body.
People who wear this kind of vest also have to remember that while it can prevent a bullet from penetrating, they can still be affected by the kinetic energy of the bullet which can produce serious internal injuries.
Hollywood depictions of firearm victims being thrown through plate-glass windows are inaccurate, as were this to be the case, the shooter would also be thrown backwards with equal force. Gunshot victims frequently do collapse when shot; however this is usually due to the effect of the energy of the bullet on their body systems, not the momentum of the bullet pushing them over.
(Note that the above does not apply if the victim is hit by heavy weapons fire such as aircraft cannon, where the momentum effects can be enormous; this is why these weapons need to be mounted on a weapons platform.)
See ballistics for a more technical discussion.