The original St. Joseph's Church was built on the site of the current basilica in 1803, and was the first non-mission parish built in California. The original adobe structure was damaged by earthquakes in 1818 and 1822, and a new adobe church was built from 1835 to 1846. The second church was severly damaged by an earthquake in 1868, and work on the third church began in 1869. The third church was destroyed by fire in 1875, and a temporary fourth church was built a few blocks away while construction of the fifth and current church was being constructed. The fifth church was dedicated in 1877 while construction continued. The current porticio was completed in 1884, and the large dome was finished in 1885.
In 1981, a major renovation project was begun at the church, which was to become the cathedral for the new Roman Catholic Bishop of San Jose. In 1985, the church was elevated to a cathedral, pending completion of the restoration in 1990. The cathedral was made a minor basilica by Pope John Paul II in 1997.
The Cathedral Basilica of St. Joseph is listed as a National and California Historical Landmark.