Proxima Centauri is the closest star to the earth other than the sun yet it isn't visible to the naked eye, but stars that are further away are easily found in the night sky. Why is that?
John's answer is correct. For a few more details:
Stars brightness works out to roughly the 4th power of their relative mass. This falls off a bit for very large stars, but for smaller stars it's in the ballpark. (Red-giant phases not included.)
Proxima Centauri is about 12% the mass of our sun, and about 14% the sun's diameter. At 12% its mass, its brightness, using the 4th power rule, is about 0.02% or 1/5,000th as bright. That's total light. Because it's smaller and colder, much of that light is in the infra-red range that we can't see. In terms of visible light or visual magnitude, it's about 1/20,000th as bright as our sun. That means, for it to be as bright as our sun from Earth in visible light, it would need to be about 1 million KM away. If it was as far away as Pluto, some 40 AU on average, it wouldn't look very different than Venus at Venus' peak brightness, with a distinct red color.
Larger stars can have a brightness over 100,000 times brighter than our Sun. So, because the mass of the star affects the brightness to about the 4th power, what it really comes down to is size matters.
Distance matters too, but only to the square of the distance, so mass of the star matters more. Here's an article with a chart of the visually brightest stars in the sky. Negative numbers and smaller numbers correspond to greater brightness.
It might go without saying, but virtually none of the stars that can be seen with the naked eye are red dwarf stars even though they're the most common type of star. They're too dim for us to see at multiple light-years distance.
Stars vary a lot in both size and brightness. The nearby stars you mentioned are less bright than many stars that are much further away. We see the bright stars but not the less bright ones. Its like comparing the effectiveness of the old 2-battery flashlight to the 5-battery one. Both are flashlights but one is much brighter than the other.