Adûnaic derived from the related Bëorian and Hadorian tongues of the Atani, and during the Second Age a Bëorian accent still survived in Númenor. Adûnaic seems not to have been influenced by the language of the Second House of Men, the Haladin tongue, at all: when the Númenóreans returned to Middle-earth in the Second Age, they did not recognise the peoples of Enedwaith and Minhiriath as their distant kin, because these spoke Haladin languages.
The exact origin of the Bëorian/Hadorian tongue is not clear, but certain is that there are both Elvish and Dwarvish (Khuzdul) influences, suggesting the Atanatari (Fathers of Men) had contacts with both peoples before arriving in Beleriand.