I would like to find a table where I can, say, look for K5 dwarfs and read off that stars of that type have on average an absolute magnitude of X.XX in the infrared L band and Y.YY in the G band. I've looked everywhere and I can't find a compilation of this kind of information. Does any such thing exist?
There are a couple of standard papers containing the table you want.
Kenyon & Hartmann (1995) http://adsabs.harvard.edu/abs/1995ApJS..101..117K
Table A5 contains many colours for stars as a function of spectral type. You need to combine this with something that gives absolute V magnitude along the main sequence, like that of Schmidt-Kaler (1982).
An alternative, that has colours for both main sequence and pre main sequence stars (though the absolute magnitude of a PMS star is age-dependent) is found in Pecaut & Mamajek (2013). http://arxiv.org/abs/1307.2657
There are a lot of stars  and they are organised into catalogs by various organisations, well known catalogs include the UNSO catalog (produced by the united states Naval office) and the Guide Star Catalog (used by Hubble to accurately posistion itself)
I think the best for you is the Hipparcos catalog of stars, as it can be browsed quite easily by spectral type.