What are the main arguments against Hoag's object in particular and Hoag-like objects in general to be explained as Einstein rings formed by a farther edge-on galaxy?
It doesn't look like an Einstein ring. An Einstein ring shows highly distorted shapes: stretched and multiple images of the distant galaxy, and a massive foreground object (with different redshift) which would have to be a galaxy itself as nothing else is big enough.
Hoag's object shows none of these distortions, and the central bulge is at exactly the same redshift (so the same distance by Hubble's law). The areas of star formation in the ring look rather like those in more normal galaxies. It is just an unusual shape.