How do we know that other planetary systems aren't composed of antimatter? If they never touch matter, then they don't annihilate. All their observable physics (spectra) are the same as matter, right?
There are two major points about this, I think.
First, and ultimately most importantly, is that if we have matter here, and they have antimatter over there, then somewhere in between we must have a region that transitions from matter to antimatter. Even if this region is located in the intergalactic medium, where densities are typically very low, collisions between matter and antimatter would happen often enough that there would be a tell-tale radiation signature from this region. To date, no confirmed detections of such a region have been made.
Now for the second point, it is not actually true that matter and antimatter have the same observable physics, though the observable differences are hard to detect even within our own laboratories.
Electromagnetism (classical or quantum), gravity, and the strong force all obey C-symmetry, and so you would not see any observable differences between matter and antimatter with these forces.
However, C-symmetry is not preserved in the weak interaction/weak force. It is also known that parity symmetry can be violated in weak interactions. Parity symmetry, loosely, means reflecting all particles in a mirror, and violation of it means that something has "handedness": you can tell your left hand from a right hand, and the mirror makes your left look like a right. Remarkably, there is reason to believe that the weak force also violates the combined CP symmetry.
So the weak force introduces this sort of handedness into particle physics. It does so with the neutrinos and anti-neutrinos in particular.
Now this handedness of neutrinos is something that could, in principle, let us detect that a mass is composed of antimatter. Beta decay is a result of the weak force and emits neutrinos or anti-neutrinos, depending on whether the original atom is matter or anti-matter. So theoretically we might tell a planet is antimatter if it has some radioactive material undergoing beta decays.
Except that will actually be pretty impossible. The scale of even an entire Jupiter's worth of radioactive mass would just be too small a source of neutrino emissions. Current neutrino detectors confirm on the order of a handful of neutrinos that probably originated from outside of our solar system every decade. And those aren't really looking for neutrinos from any particular area; they'll look for any neutrino coming in from anywhere in the universe (well, where you are on the earth matters a bit).
So our current technology basically isn't good enough to detect these weak sector asymmetries from sources outside of our solar system.
Otherwise, assuming some improvements, a lot of time, and/or one heck of a lot of luck, perhaps our best shot at confirming a large source of antimatter comes from supernovae. Supernovae are very, very bright (they will outshine the entire galaxy they occur in), and therefore very noticeable. They also emit tremendous numbers of neutrinos very quickly. So if we chance to notice a supernova, and if we luck into our neutrino detectors having enough neutrino detections at the right time to correlate to that supernova, then if somehow that was an anti-matter star we might be able to tell due to detecting the difference in the neutrinos.
But that's really unlikely. We're substantially more likely to detect the energy signature of annihilation events at the matter/anti-matter interface, as mentioned in the first point.
See also this Q&A, which proposes using polarized light as a means of detection.
You would/should be able to detect the annihilation of electron-positron pairs at the boundary between normal-matter and anti-matter space. This has a distinctive energy and so could be identified unambiguously. For an example of the detection of electron-positron annihilation see this NASA news item.
Assuming the parent star of such a system emits a solar wind similar to our sun's, then those antimatter particles would collide at that system's heliopause with regular-matter cosmic ray particles. I would think such a matter-antimatter reaction would be visible across much of the galaxy.