As Venus rotates in clockwise direction, while other planets rotate in anti clockwise direction:
Is it possible that Venus was once a rogue planet?
Astronomy Stack Exchange is a question and answer site for astronomers and astrophysicists. It only takes a minute to sign up.Sign up to join this community
This is considered very unlikely. It is actually very difficult for things to be captured into orbits. They pick up speed as they fall in towards the larger object, and that's automatically enough speed to carry them back out. Capture either requires dust and gas to slow the body down, or very lucky gravitational interaction with a third object.
This has happened with moons, but the solar system is much more crowded than the galaxy. For something the size of Venus to pass through the solar system at all would be exceedingly rare.
Finally, it is easy to explain the retrograde rotation by other means. This is where the other question that James K mentioned in the comments comes in.