In fields that have no non-academic counterpart, academic elitism is common because it actually is the case that there aren't very many people with anything worthwhile to say on the topic, other than the academics. The problem starts when scholarship blurs with the application of scholarship, such as in business or mathematics, or with technology in the case of physics and engineering. In such cases academic elitism arises when those in pursuit of scholarly knowledge deride the pursuit of application.
Academic elitism is closely related to anti-intellectualism, the belief that "ivory tower" academics are too far removed from reality to have anything possibly useful to say about practical matters.