If we graph the variability of a small bright star which is lined up with the asteroid belt, is it less constant than stars that are far away from the asteroid belt? To what degree does variability change relative to distance away from the most dense areas of the solar system?
No, stars that are seen through the asteroid belt are not any more variable than stars in any other part of the sky.
All stars vary in brightness on measurable timescales, although some have been measured to have such small variations that they are recorded as "standard" stars and for most purposes can be considered to have a constant brightness and thus be used as reference for other brightness measurements.
What you are referring to, as noted, are occultations. There are many hundreds of thousands of known asteroids in our Solar System, and billions of stars in our galaxy, and so there could be hundreds of predicted occultation events every day, and there are many people interested in occultation events. For one thing, we can calculate the distance and speed of an asteroid quite accurately but we can't directly photograph it to determine its size. But when an asteroid passes in front of a star, we can measure the duration of the occultation and set precise limits on its size.
But for all the asteroids in the Main Belt, the space between them is huge and for any one star the chances of an occultation are quite (vanishingly) small, so the stars seen through the asteroid belt are not any more variable than anywhere else in the sky.