The cadenza was originally a vocal flourish improvised by a performer to elaborate a cadence in an aria. It was later used in instrumental music, and soon became a standard part of the concerto. Originally, it was improvised in this context as well, but during the 19th century, composers began to write cadenzas out in full. Third parties also wrote cadenzas for works in which it was intended by the composer to be improvised, so the soloist could have a well formed solo that they could practice in advance. Some of these have become so widely played as to virtually be a part of the original piece, as is the case with Joseph Joachim's cadenza for Johannes Brahms' Violin Concerto.
Nowadays, very few performers improvise their cadenzas, and very few composers have written concertos within the last hundred years that include the possibility of an improvised cadenza.
Cadenzas are also found in instrumental solos with piano or other accompaniment, where they are placed near the beginning or near the end or sometimes in both places. (e.g. "The Maid of the Mist," cornet solo by Herbert L. Clarke.)