Related to the question "Are any Pluto-sized objects remaining to be discovered in the Kuiper Belt?" and the fact that most of the Kuiper Belt objects have very elliptical orbits, the question begs, what observational techniques are used to confirm that the object being viewed is an orbiting Kuiper Belt Object, as opposed to the transit of a rogue or 'orphan' planet?
I'm not speaking from an informed position here, but two things come to mind.
First is that a rogue planet is likely to be traveling very fast, relative to our solar system. An object from within our solar system (a Kuiper/Oort object) is going to have an orbital velocity. Something that isn't a part of the system at all could be traveling much, much faster.
Secondly, if a rogue planet somehow slowly drifted into our system at almost no speed, and then got pulled into a sort of orbital speed by the sun, the direction of its orbit might give it away. For example, it's orbital inclination could be way off--even perpendicular to the normal orbital plane. It also might be orbiting in the wrong direction--that is--clockwise, instead of ante-clockwise.
That being said, there's no reason that some disturbed Kuiper belt object might not have an eccentric orbital inclination or even an anti-clockwise orbit. But the larger the object, the less likely this is, because it would need a larger and larger disturbance to throw it so out of whack.
No special techniques are required that I am aware of, but several observations must be taken to make a reasonable estimate of the orbit. Once a sufficient number of observations are made so that the orbital elements can be determined, then anything that has an elliptical orbit (eccentricity >0 but >1) can be assumed to be orbiting as part of the solar system and thus not a transiting rogue object.
An object with an eccentricity to its orbit greater than 1 is following a hyperbolic escape trajectory, and, unless substantially perturbed by the gravity of another planet, is on its way out of the solar system. I am not aware of any such objects that are thought to have originated from outside the system, but comets have been known to have been flung from the system along hyperbolic paths from the influence of Jupiter.