From LQWiki

A Just-in-time, or JIT, compiler may be used to speed up execution of interpreted languages.

There are differing approaches, but generally a native, machine code, version of the interpreted code is maintained for at least the most used function calls.

In theory a JIT can improve the speed of a language like Java to compete with precompiled C++. Because (re)compilation is performed at runtime it can be targeted very specifically to the platform it is running under. In practice this has not yet been attained because the conversion to native code is required to be very fast, and therefore cannot find as many optimisations as normal compilation.

Sun's HotSpot is an example of the current state of the art in JIT compilers. Without JIT a typical Java program takes around five times longer to perform the same task. If a task is especially CPU-based the difference can be an order of magnitude.