Essentially the amount of radiation reflected per spectrum band (UV, X, visible etc.) depends on atmospheric and object surface chemical composition. In the case of the Moon you have no atmosphere, so this is already a great difference with respect to Earth. But since there's also a different surface composition between Earth and Moon you can't say a priori: "if the Earth reflects x amount of UV radiation from the Sun because of the atmosphere, then the Moon reflects all of it cause atmosphere is absent". Of course it's just an example.
To make it simpler, you can think about all the objects you see around you in everyday life: their colours depends also on their chemical composition. If you put an atmosphere around a desk for example, the colour you see might change, and if you change the atmosphere chemical compositionAnswer edit for rephrasing: no, that colour will change again. If you remove the atmosphere around the desk and you change the composition of the desk you will see again another colourdifference can't exactly be visible light. ThisThe albedo is becauseestimated from the amount of radiation reflected per band changeslight, so the final result is a different colour. The albedo is referred to allwhich includes the whole spectrum of radiation coming from these bands.