For a time in the early 1800s it was the seat of government of the colony of New South Wales (which at that time encompassed all of the Australian mainland). Afterwards the seat of government was moved back to Sydney, but the Governor's former residence, Old Government House, remains as a historic site.
The name Parramatta is derived from a local Aboriginal expression meaning "place of many eels". It is located at a portion of the nearby Parramatta river which was indeed a gathering place for native species of eel.