When light from a star or another astronomical object enters the Earth's atmosphere, the different temperature layers and different wind speeds distort and move the image in various ways (see astronomical seeing for a proper discussion). The net result is that an 8 meter or 10 meter telescope (like the VLT or Keck), while theoretically capable of milli-arcsecond resolution, is limited to what the atmosphere permits, which can easily be a factor of 50 or 100 worse.
An adaptive optics system tries to correct these distortions, using a quality-of-image detector, a deformable mirror, and a computer that receives input from the detector and calculates the optimal deformation of the mirror.
This is possible only because the theoretical, perfect image is known in advance: in the case of a point-like star, the image is a circle with certain characteristics dictated by the telescope's aperture.
The necessity of a reference star means that an adaptive optics system cannot work everywhere on the sky, but only where stars of sufficient luminosity (for current systems, about magnitude 11-12) can be found very near to the object of the observation. (An alternative is the use of a laser beam as a fake guide star).
The only other workable approach is to get rid of the atmosphere entirely, using a space-based telescope. This is what NASA's Hubble space telescope does.