There are such evidence in the case of Mars. Observations from Mars Global Surveyor show evidences of crustal magnetization. In particular, this magnetization has extensive, east-west trending linear features in Terra Cimmeria and Terra Sirenum. These are probably reminiscent of magnetic features associated with a reversing dipole.
There are no such evidence in the case of Venus: Venus has no intrinsic magnetic field today (contrary to the common belief, its inability to maintain a dynamo activity is either due to its chemical and physical conditions unable to form a solid core, either du to an early complete core solidification; it is not due to its slow rotation), and could not have maintained a remanent crustal magnetization, because its crust would have been to hot during its dynamo activity (above the Curie point).
Sources: