A key argument is that mathematics and physics as perceived by humans have grown together, may simply reflect human cognitive bias, and that the rigorous application of empirical methods or mathematical practice in either field is insufficient to disprove credible alternate approaches.
Hilary Putnam argued convincingly in 1975 that real mathematics had accepted informal proofs and proof by authority, and made and corrected errors all through its history, and that Euclid's system of proving theorems about geometry was peculiar to the classical Greeks and did not evolve in other mathematical cultures in China, India, and Arabia. This and other evidence led many mathematicians to reject the label of Platonists, along with Plato's ontology - which, along with the methods and epistemology of Aristotle, had served as a foundation ontology for the Western world since its beginnings. A truly international culture of mathematics would, Putnam and others argued, necessarily be at least 'quasi'-empirical (embracing 'the scientific method' for consensus if not experiment).
Eugene Wigner had noted in 1960 that this culture need not be restricted to mathematics, physics, or even humans. He stated further that "The miracle of the appropriateness of the language of mathematics for the formulation of the laws of physics is a wonderful gift which we neither understand nor deserve. We should be grateful for it and hope that it will remain valid in future research and that it will extend, for better or for worse, to our pleasure, even though perhaps also to our bafflement, to wide branches of learning."
This simultaneous "hope" and "bafflement" regarding the undeserved "gift" was a frank admission that quasi-empiricism could apply to physics as well, and that other branches of learning need not necessarily be so compatible with mathematics as understood in the context of physics or hard sciences.
See also: philosophy of mathematics, foundations of mathematics
Sources