What barriers - of technology, physics and possibly economy (things that would be possible technologically but are simply too expensive) sets the upper bound on quality of telescopes for observation of the sky in visible spectrum - observing surface of distant objects at maximum resolution/detail?
Visual resolution of a telescope is directly proportional to the aperture of the telescope. The focal length, and hence the magnification that can be achieved, is then just following on the visual resolution.
The telescopes today are usually so well build that they are diffraction limited, which means optical resolution due to diffraction is the limiting factor. If you want to have "higher magnification" in a telescope, you always want to have a larger aperture. The longer focal length may help, but is not quite necessary.
And, as Jeremy said, the limiting resource in this is money. There are some engineering problems with building extremely large telescopes, but most of these can be solved, given enough money, time and resources.
Comparing telescopes that observe the visible spectrum to the radio spectrum, radio astronomers have been able to create telescopes with apertures of the order of kms, thanks to aperture synthesis. This is extremely hard in optical telescopes and the only telescope (afaik) that does so is the Large Binocular Telescope. The reason this is possible in radio astronomy is because we can measure the phase of the incoming wave using radio telescopes where as information about the phase is not captured by optical telescopes. Maybe in the future, technology will help us make optical detectors that can measure the phase of the wave.
coming to the size of the aperture itself, larger and larger sizes doesn't help as long as we don't account for the atmospheric seeing. the reason stars twinkle is because of atmospheric seeing. effects seeing can be negated using adaptive and active optics and advancement of these technologies will help astronomy move ahead.
coming to the actual detectors, the intrinsic noise from radio detectors (eg. bolometers) is much smaller than that in optical detectors (eg. CCDs). so again, maybe in the future, we have better detectors with extremely low noise.
(sorry coulnd't add more links. needed more rep :D)
To answer the question reworded as: "What limits the quality of telescopes currently?" The answer is mostly: money.
It used to be: atmosphere. But with advances in adaptive optics, ground-based telescopes are achieving what used to require a space telescope. Plus, we have the technology for space telescopes if we want, like the pending JWST.
So pretty much, it boils down to funding. Who is going to spend the money on the expensive technology to see better and further, when science is being squeezed for funding from all quarters, and astronomers can't put forward a business case for a return on investment like a biotech company can for developing a grass that will result in cows burping less methane.