This question has two parts:
A very useful diagram which shows surface temperatures, and also gives you the temperature of any star you can observe is the Herzsprung-Russell Diagram, this one from le.ac.uk.
As you can see, the yellow of our own sun places it in the 4.5 kKelvin to 6 kKelvin, as noted in the question. This temperature is down towards the lower end of average. The main sequence, where most stars are, tops out about 20 kKelvin, and there are some up towards the 40 kKelvin region - they aren't shown here as they are much rarer.
White dwarves are a little hotter than our sun - between 6 kKelvin and 10 kKelvin.
Neutron stars are way off the main sequence - young ones can be over 1 MKelvin!
Internally, the core temperatures are dependent on the mass of the star. In our sun, energy is delivered via the proton-proton chain mechanism, which occurs up to about 20 MKelvins, whereas more massive stars can use the Carbon-Nitrogen-Oxygen cycle - which happens from about 15 MKelvins upwards.
The differences are mainly down to convection and radiation differences - this extract from Wikipedia's Main Sequence page describes this in some detail:
Because there is a temperature difference between the core and the surface, or photosphere, energy is transported outward via radiation and convection. A radiation
zone, where energy is transported by radiation, is stable against
convection and there is very little mixing of the plasma. By contrast,
in a convection zone the energy is transported by bulk movement of
plasma, with hotter material rising and cooler material descending.
Convection is a more efficient mode for carrying energy than
radiation, but it will only occur under conditions that create a steep
temperature gradient. In massive stars (above 10 solar
masses) the rate of energy generation by the CNO cycle is very
sensitive to temperature, so the fusion is highly concentrated at the
core. Consequently, there is a high temperature gradient in the core
region, which results in a convection zone for more efficient energy
transport. This mixing of material around the core removes the
helium ash from the hydrogen-burning region, allowing more of the
hydrogen in the star to be consumed during the main-sequence lifetime.
The outer regions of a massive star transport energy by radiation,
with little or no convection. Intermediate mass stars such as
Sirius may transport energy primarily by radiation, with a small core
convection region. Medium-sized, low mass stars like the Sun have
a core region that is stable against convection, with a convection
zone near the surface that mixes the outer layers. This results in a
steady buildup of a helium-rich core, surrounded by a hydrogen-rich
outer region. By contrast, cool, very low-mass stars (below 0.4 solar
masses) are convective throughout. Thus the helium produced at the
core is distributed across the star, producing a relatively uniform
atmosphere and a proportionately longer main sequence lifespan.