Despite the name, the people are not from the Netherlands, but rather are from various parts of southwest Germany, Alsace and Switzerland. Pennsylvania Germans spoke a language known as Pennsylvania German, and some continue to speak it to this day. The word "Dutch" here is left over from an archaic sense of the English word, which once referred to Germany as well as to the Netherlands. This archaism may have survived for various reasons; for example, the Pennsylvania German word for "German" is "Deitsch", which sounds similar to the English "Dutch".
The German word for "German" (itself) is "deutsch" (it sounds similar to "dutch", too).
See also: Lunenburg, Nova Scotia.