The bottom line of the controversy is our model of the Universe: it is generally thought that the universe is undergoing expansion, and that redshifts observed in extra-galactic objects are a confirmation of this effect. Quasars, in particular, are high-redshift objects thought to be very bright nuclei of very distant galaxies. The relationship between distance and redshift (interpreted as velocity) is known as Hubble's law, and is one of the fundamentals of modern cosmology.
Arp's argument is to show many quasars with high redshift but, it seems, linked in some ways to close-by objects like nearby galaxies, thus invalidating Hubble's law. He then proceeds to show that even some galaxies show strange redshifts, and that redshifts themselves could be quantized - something that current theories cannot explain at macroscopic scales.
Most astronomers reject Arp's interpretation of the data. Anomalous observations could be explained by perspective effects. Some of Arp's calculations seem to be simply bad mathematics. Arp asserts that many questions he posed to the scientific establishment are still unanswered and that, even worse, his requests for more observation time have been systematically rejected.