How do we know that Mercury, Earth, Mars, Jupiter, Saturn, and Neptune rotate counterclockwise and Venus, Uranus, and Pluto rotate clockwise? How do scientists determine the direction of rotation of planets?
I'll focus first on the question of the title: "how do we find / measure rotation?"
The easiest method is the same as everyone else: look and see. Take images some time apart and you will see in which direction distinctive surface features moved around the rotation axis. The easiest might be Jupiter with its great red spot, but every planet has surface (but also for atmospheric features like storms) which move.
For planets with an atmosphere with little features you can employ spectroscopy and measure the velocity via Doppler shift. Easiest for these measurements is to pick a strong emission or absorption line for one of the main elements in that planet and look at the spectrum spatially resolved. You then will find that it is slightly blue-shifted on one side and slightly red-shifted on the other side. There is a nice description how to achieve that even on your own with slightly advanced amateur equipment.
Now, once we saw or measured the rotation, we know how the 3D-orientation of the rotational axis, we can look at what is prograde and counterclockwise: We consider prograde rotation when the rotational axis of the planet roughly (maximum +-90°) align with the rotational axis of its orbital plane. You can take your right hand: your fingers point in direction of rotation, then your thumb points in the direction of the rotational axis. In our Solar system that's towards the North and only deviates for Venus (177°) and somewhat for Uranus which kinda rolls on its orbit with an obliquity of 98°.
This question about the direction of rotation is a bit ambiguous, as "direction" can be interpreted as retrograde/prograde -or- as the direction of the north pole axis.
This answer is about the direction of a planet's axis of rotation. When it comes to the direction of the rotation, the method that most people use is described in the following paper:
Essentially, the idea is to calculate de right ascension and declination values of the north pole of each planet using formulas described in the document. Each formula is planet specific, due to its unique interactions with other bodies in the solar system. The values of the right ascension and declination are expressed using the planet's ecliptic equator (the intersection of the sphere and the International Celestial Reference Frame (ICRF), which is fixed to the stars and centered on the solar system barycenter).
Once, the north pole is determined, the equatorial plane can be deducted (the plane orthogonal to the vector defined between the north pole and the planet center and containing the planet center) as well as the line of nodes that define the intersections of the ICRF plane and the planet's equatorial plane.
The last step is to properly orient the north pole to a given direction. This is achieved by calculating the angle W that specifies the position of the line of nodes. Again, the formula to calculate the value for W is planet specific. Using the value of W, the north Pole axis can be rotated around the ICRF z-axis (0,0,1), to position the line of nodes properly.
In each formula, the right ascension, the declination, and the position of the prime meridian are calculated using:
- T = interval in Julian centuries (of 36525 days) from the standard epoch J2000
- d = interval in days from the standard epoch J2000
- The standard epoch is JD 2451545.0, i.e. 2000 January 1 12 hours TDB