Is it possible that this planet changed other planet orbits (from which we assume that it exists) but then was thrown out of the Solar System in past when Sun was close to some other star? And it doesn't exist in our system anymore and we can't find it because of it.
By definition Planet 9 is in the Solar System. Planet 9 is a hypothetical body, and we suspect that it might exist because of the motion of other bodies in the Solar System.
If a planet used to be in the Solar System but then got ejected, then that planet will no longer interact with the other objects in the Solar System. So it won't explain what we see happening in the Solar System.
It is not impossible that other planets existed in the Solar System and got ejected, but those hypothetical planets are different from the hypothetical object that is generally called "Planet 9". If it's not gravitationally bound, then it's not planet 9.