There are three main formation scenarios for planetary moons.
The giant impact hypothesis: The satellite forms as a consecuence of an impact between the planet an a large planetesimal. The Moon is an example, and one of the arguments is that the chemical composition of the Moon matches that of Earth with a significant accuracy suggesting that it is a in part our planet and in part the original impactor (Theia). We also know that the Moon has been getting farther away from Earth because we have evidence that it gained orbital potential energy by absorbing it from the rotational energy of Earth. We know this because days were not 24 hours long a few million years ago and we can keep track of those changes in the rotational period of Earth using rings in fossillized corals (those have rigns like the tree rings but ones that generate on a daily basis). We then can see that the moon was extremely close to the Earth some billion years ago (we have more evidence of this from the fact that tides were huge in those times and led geological evidence of daily floods across the recently formed planet). If you keep going back in time you see that the Moon was basically emerging from Earth. There are many more evidences of this scenario for our moon.
The accretion scenario: The satellite coalesced from a disc of material around the new-born planet (just as the planet accreted from the protoplanetary disc), the so called circumplanetary disc. As an example we have the four galilean moons around Jupiter (Io, Europa, Ganymede and Callisto). Since the disk was relatively flat the moons formed in the same orbital plane, also they move in the same direction as the planet rotates (which makes sense as both are generated from the same material spining with a certain angular momentum). This is the most frequent scenario for large moons. Our moon couldn't form like this because the expected size of the circumplanetary disc was in no way as massive as our moon is today (Earth is a tiny planet and it has a huge moon in relative terms).
Capture scenario: The satellite formed elsewhere in the Solar System as an independent minor body. With time some dynamical interaction might have led the object close to a planet and both got gravitationally binded. An example of this is Triton, Neptune's largest moon. The retrograde orbit is inexplicable in terms of the accretion scenario and the energy needed for a gian-impact scenario to work on Neptune is too large. Triton was captured (we think it formed as another planetesimal in the Kuiper Belt as it shares many chemical features of Pluto and other objects of the region). There are not so many moons on Neptune probably because they dissapeared (crashing into the planet or been ejected) just as Triton arrived to the system and dynamically destabilized their orbits. Another clear example are the tiny irregular satellites of Jupiter. This scenario is very difficult to immagine for Earth since capturing a massive moon like ours and making the orbit circular would have been a feat in terms of how precisely tunned would the orbit insertion parameters had to be. The giant-impact scenario leads to the current situation in simulations for a larger range of impact parameters, thus is much more probable in statistical terms.
There are some less frequent and some speculative scenarios also:
Eyecta fragments from other moons: Some satellites might have their origin on other satellites. A large impact might eject material into orbit. An example would be Hippocamp (a Neptunian moon) which is now regarded to be a fragment stripped out from Proteus (a larger moon).
Lagrangian/Trojan moons: This is similar to the circumplanetary disc scenario but here the accretion in the disc of the planet is further stimulated in certain regions because of a moon that formed a bit earlier. An orbiting body can generate five equilibrium points (Lagrange points) by sculpting the gravitational landscape. Two of those equilibrium points (L4 and L5) are stable equilibrium points; thus they are like gravitational traps where matter can accumulate until a new moon is formed. As a potential example we have Telesto and Calypso in the Saturnian system. They both lie on the L4 and L5 Lagrange points of Tethys (a much larger moon with great gravitational influence). They might have formed as regular objects and then got trapped on the equilibrium points or they might have actually formed there as matter coalesced on those gravitational traps.
Sprayed into existence by cryovolcanism from another moon: It sounds ridiculous because it is an hypothetical scenario, I just think might happen somewhere in the universe. Take a look to Enceladus (a large active moon of Saturn). Enceladus has water plumes and jets that shoot material from its interior into space through craks in the icy crust (because tidal stresses heat the interior like a pressure cooker and the pressure gets liberated that way). The entire E-ring on Saturn was created by orbiting ice and dust grains sprayed by Enceladus. We know that the E-ring has a mass of nearly $12\cdot10^8\; kg$ and we know that it is so far from Saturn as to make possible the coalescence of this matter (tidal forces wouldn't disrupt it: see Roche limit). Thus it is possible that the material from the ring could make a moon with the density of Aegaeon (another saturnian moon) and with a diameter of just $162\; m$ (a third of the size of Aegaeon). Enceladus prohibits this because of the gravitational influence it excerts on the E-ring, it also prohibits it to grow (the ring would be more massive if the material was not continuosly reabsorbed by Enceladus). But if Enceladus migrated to another orbit in relatively short timescales then at least I think is possible for this to happen. The new moon greated from the material of the ring could then end up coliding with Enceladus since both would probably still interact chaotically. The material once sprayed out of its interior would had returned to home.
Centrifugal breakup scenario: This is also hypothetical but we think this happens a lot on asteroids. Lightweight moons could be ruble piles just as many comets and asteroids are. Losely bounded material with little cohesion. If the moon starts to spin faster and faster (because of some mechanism like the Yarkovsky-O'Keefe–Radzievskii–Paddack effect), it could eventually break into two pieces due to the extreme centrifugal forces (described as seen from the co-rotating reference frame of the original moon). It is believed that asteroids like $1999 \;KW_4$ splitted in two because of this effect. I don't see any reason why a moon couldn't do the same, forming a new independent moon in the process.
Moon made of chunks of other disrupted moons: As crazy as it seems, it is one of the formation hypothesis regarding the formation of Miranda (one of Uranus satellites). Miranda's surface is so complex and varied that some speculate it might have formed as several pieces that orbited Uranus came toghether gently. These pieces might have been chunks from other moons or might have been chunks from an earlier iteration of Miranda itself, fragmented after a disruptive event. The geology on each chunk would have evolved independently until they re-assembled together. But this is also quite speculative.