Astronomy Stack Exchange is a question and answer site for astronomers and astrophysicists. It only takes a minute to sign up.
Sign up to join this community
From my understanding, the luminosity of a main sequence star should increase because, in its core, hydrogen is being burnt into helium which means that the number density of hydrogen decreases and the temperature increases and from the Stefan–Boltzmann law, the temperature is proportional to the luminosity so the luminosity of a star should increase while it's on the main sequence phase of evolution.
Please correct me if my reasoning is incorrect.
The luminosity of a star remains approximately constant during its time on the main sequence. There is a slight increase over time, which is, as you guessed, related to the changing composition of the star due to fusion.
The so-called Vogt-Russell theorem tells us that a star's mass and composition are the only two factors that affect its structure and, by extension, its luminosity.$^{\dagger}$ Most main sequence stars have negligible mass-loss rates (at least, compared to evolved stars like AGB stars), and so therefore compositional changes should be the only thing dictating luminosity evolution.
It turns out that the luminosity is related to the mean molecular mass $\mu$ by (approximately) $$L\propto\mu^4$$ which is not insignificant. Fusion turns hydrogen to helium in the star's core, and hence there should be an increase in $\mu$ and thus in luminosity (roughly a factor of two). The change is small compared to the orders of magnitude by which two main sequence stars may differ from one another, but it isn't zero.
As an aside, there is indeed a change in the central temperature (though I would note that your use of the Stefan-Boltzmann law requires a change in the surface temperature) to accommodate the increase in luminosity, but the energy generation rate is very sensitive to changes in temperature, particularly for stars using the CNO cycle.
$^{\dagger}$ This is, strictly speaking, not true. Rotation, as Rob Jeffries mentioned, is also important for certain stars, and it can have a role on the evolution of a star's luminosity.
