Although liquid hydrogen is clearly the best fuel in terms of efficiency of the engine, it also requires huge structures to hold it due to its low density. These structures can weigh a lot, offsetting the light weight of the fuel itself to some degree, and also result in higher drag while in the atmosphere. While kerosene weighs considerably more, its higher density results in smaller structures, which implies less losses to atmospheric drag. In addition kerosene based engines generally provide higher thrust, which is important for takeoff. So in general terms there is a "sweet spot" in altitude where one type of fuel becomes more practical than the other.
Traditional rocket designs can easily use this sweet spot to their advantage via staging. For instance the Saturn Vs used a lower stage powered by RP-1 (kerosene) and upper stages powered by LH2. Some of the early Space Shuttle design efforts used similar designs, with one stage using kerosene into the upper atmosphere, where a LH2 powered upper stage would light and go on from there. The existing Shuttle design is somewhat similar, although it uses solid rockets for its lower stages.
However the eventual fallout of the Shuttle suggested that rockets should use a single stage to greatly simplify operations. Almost all of the cost of operating the Shuttle is operational, the payroll for the army of workers needed to re-build the Shuttle from its component parts after it has landed. A rocket using a single stage, the so-called SSTO, avoids this expensive step, and thereby lowers costs. In this case the staging solution is not available, by definition, so it becomes harder to use both fuels.
Designs could simply carry two sets of engines, but this would mean the spacecraft would be carrying one or the other set "turned off" for most of the flight. With light enough engines this might be reasonable, but the SSTO design has razor-thin margins for extra weight.
And thus the tripropellant engine. The engine is basically two engines in one, with a common engine core with the engine bell, combustion chamber and oxidizer pump, but two fuel pumps and feed lines. The engine is somewhat heavier and more complex than a single-fuel engine, but the complexity is generally a little less than 50% more than a single engine – or more importantly, 50% less than two engines would be. Of course there are numerous practical reasons why things are harder.
At liftoff the engine typically burns both fuels, gradually changing the mixture over altitude in order to keep the exhaust plume "tuned" (a strategy similar in concept to the plug nozzle but using a normal bell), eventually switching entirely to LH2 once the kerosene is burned off. At that point the engine is largely a straight LH2/LOX engine, with an extra fuel pump hanging onto it.
The concept was first explored in the US by Robert Salkeld, who published the first study on the concept in Mixed-Mode Propulsion for the Space Shuttle, Astronautics & Aeronautics August 1971. He studied a number of designs using such engines, both ground based and a number that were air-launched from large jet aircraft. He concluded that tripropellant engine would produce gains of over 100% in payload fraction, reductions of over 65% in propellant volume and >20% in dry weight. A second design series studied the replacement of the Shuttles SRBs with tripropellant based boosters, in which case the engine almost halved the overall weight of the designs. His last full study was on the Orbital Rocket Airplane which used both tripropellant and (in some versions) a plug nozzle, resulting in a spaceship only slightly larger than an Lockheed SR-71, able to operate from traditional runways.
Although invented in the US, the only tripropellant engines built were in Russia. Kosberg and Glushko developed a number of experimental engines in the early 1990s for a SSTO spaceplane called MAKS, but both the engines and MAKS were later cancelled due to a lack of money. Glushko's RD-701 was built and test fired however, and although there were some problems Energomash feels that the problems are entirely solvable and that the design does represent one way to reduce launch costs by about 10 times.