The sky appears to drift overhead from east to west, completing a full circuit around the sky in 24 (sideral) hours. This phenomenon is due to the spinning of the Earth on its axis. The Earth's spin axis intersects the celestial sphere at two points. These points are the celestial poles. As the Earth spins, they remain fixed in the sky, and all other points seem to rotate around them. The celestial poles are also the poles of the celestial equatorial coordinate system, meaning they have declinations of +90 degrees and -90 degrees (for the north and south celestial poles, respectively).
The north celestial pole currently has nearly the same coordinates as the bright star Polaris (which is Latin for "Pole Star"). This makes Polaris useful for navigation: not only is it always above the North point of the horizon, but its altitude angle is always (nearly) equal to the observer's geographic latitude (however, Polaris can only be seen from locations in the Northern hemisphere).
The fact that Polaris is near the pole is purely a coincidence. In fact, because of precession of the equinoxes, Polaris is only near the pole for a small fraction of the time.