The accepted answer is correct as far as it goes but to add a bit of why for the claim:
jmh> "Glass can also be polished to a high degree of accuracy without having defects"
Comes down to glass being amorphous instead of polycrystalline as metal is. When a material is polycrystalline its atoms have preferred (stronger) connections with some of its neighbors and weaker connections with others. It is these chains of preferred connections (conga lines) that metallurgists tweak to give the material trade off in properties of tensile strength, ductility, hardness. Amorphous material on the other hand has no preference and it is every atom for itself (mosh pit).
This means that when polishing/figuring, as atoms are removed from from a polycrystalline material they leave at an uneven rate resulting in pits where there were stronger connections and lumps where there were weaker atomic connections.
Whereas single atoms can be removed or jostled into lower energy configurations without prejudice in amorphous material resulting in a (nigh) atomically smooth surface.
At this point it is good to remember that the glass is not typically the mirror as glass no matter how smooth is not highly reflective, even worse it is transparent to the visible light we may be interested in. Instead the mirror is just the few atoms of thickness of highly reflective metal that
coats the glass.
If we could convince a single atom thick sheet of metal atoms to adopt the same shape without the glass it would be a great day for astronomy.