Typical compilers examine the source code for a program and produce a machine-readable executable program. This style of compiler is almost universal due to its performance, but is limited in that the executable can only be run on the single platform it was compiled for. Although in theory a different compiler on another platform could be used on the same source code, in practice this tends to be very difficult to achieve due to differences in compilers. Thus the vast majority of programs are limited to a single system.
Another solution is the interpreter, which examines the source code on whatever platform it finds itself on. As long as an interpreter exists for a platform, the program should be able to run on it. On the downside, interpreters tend to be quite slow, and are not widely used as a result.
There is an intermediate ground, known as p-code or bytecode, where the source code is compiled "half way" and then interpreted from there. Such systems have been in use for many years, but generally the performance has not been enough to make it a serious competitor. In more recent years the performance of computers has become so great that such approaches now seem practical.
The Java platform was the first modern platform to re-approach the bytecode solution. Speed was poor however, so newer runtimes were introduced that compiled the bytecode as it was run. This technique became known as JIT.
The basic idea behind a JIT is that the bytecode is smaller and easier to compile than the original source code, so that the time spent compiling it the "rest of the way" on any particular platform is quite small. In theory this means that the JIT system has the best of both worlds, it can run just as fast as other compiled code, yet the compiler for any particular platform is easy to write because the main bytecode compiler has already done most of the work.
Sun's HotSpot compiler was the first JIT to appear, around 1997. Since then almost all Java systems have introduced a JIT, and the technique has since become popular for other uses as well. Microsoft selected a JIT-based system for their .NET environment, and the Crusoe processor uses similar techniques in hardware.