Figuratively, it was typically used when presenting a third choice or fact after two others, usually after the second had been presented with the phrase "on the other hand...". Thus, a discussion of choices could follow the typical English pattern of saying "We could do this, but on the other hand we could do that," followed by what would in conventional English be a non sequitur, "but on the gripping hand, there's another alternative". In keeping with the idea that this represents the strongest arm, the last fact is often one that is most unpleasant or difficult to deal with, in some way.
Some of the more devoted fans of Niven and Pournelle will sometimes slip into this metaphor in regular speech, often resulting in some confusion from the listener.