The concept of center of gravity was first introduced by the ancient Greek mathematician, physicist, and engineer Archimedes of Syracuse. Archimedes showed that the torque exerted on a lever by weights resting at various points along the lever is the same as what it would be if all of the weights were moved to a single point -- their center of gravity. In work on floating bodies he demonstrated that the position at object floats is the one that makes its center of gravity as low as possible. He developed mathematical techniques for finding the centers of gravity of objects of uniform density of various well-defined shapes, in particular a triangle, a hemisphere, and a frustum of a circular paraboloid.
Aeronautical significance
The CoG is an important point on an aircraft, as it defines the amount of mass forward or behind the center of gravity that needs to be moved in order to pitch the plane up or down without applying any external forces.
In conventional designs the CoG is often located very near the line 1/3rd back from the front of the wing. That is the line where most wings generate their lift, known as the center of pressure (CoP), so by balancing the plane at that point, the lift and weight balance out with no net torque. The CoG is sometimes moved slightly to the rear of this line in order to provide the plane with a natural "nose up" tendency when lift increases (like when applying more power).
If the balance of the plane is moved too far from the CoG, the control surfaces may have trouble controlling the plane. The actual force generated by the surfaces is typically quite small (a few pounds) but due to their location at the end of the tail (typically) they generate considerable torque to pitch the plane. If the CoG starts to move away from the CoP there will be an increasing amount of constant torque they have to counteract, and if it moves too far, it may be more than the controls can counter.
For this reason great pains are taken to keep the CoG from moving very much. Anything that could change the balance of the plane, like cargo, passengers or fuel, are placed as close as possible to the CoG. This is why small planes, which with their short arms for the control surfaces, only a few feet long, have very small cabins located right under or above the wing. Movement of even a few inches at full loads can cause serious control problems at low speeds when there isn't a lot of wind on the tail surfaces.