A singularity in physics is basically a big signpost saying "the theory you are using has probably ceased to be a good approximation of reality by the time you get here". An example is the infinite amount of short-wavelength radiation produced by a blackbody under the Rayleigh-Jeans law. This is obviously an incorrect description of reality and the issue ended up being resolved by the development of quantum theory: taking into account the corrections gives Planck's law, and the infinity goes away.
Inside a black hole, general relativity does predict singularities, and it seems to be fairly common to talk of them as though they are physical objects. However we should be cautious: even though general relativity is our best description of gravity so far, we already know that it does not describe the actual universe. The issue here is that general relativity is incompatible with the Standard Model of quantum field theory, which is our best theory of how physics works excluding gravity.
By the time you get to the region near where general relativity predicts a singularity, you're working with a very small region with an intense gravitational field. Quantum corrections are almost certainly going to be relevant here, and unfortunately we do not yet have a self-consistent theory of "quantum gravity" to describe what should happen in such a region and so far experiments (such as the ones being performed at the Large Hadron Collider) haven't given much of a clue as to how to proceed with developing one.
In a hypothetical universe where general relativity is correct then there would be a singularity (which is fortunately hidden from the rest of the universe by the event horizon), but we do not live in such a universe.