Per the Big Bang Theory, the universe was formed from a dense singular point which existed 13 billion years ago and has been expanding ever since. This suggests that the point should have existed somewhere in space implying an outer universe. If we call this point the "Universe", then what do we call the space that held that point? Also how did this single point come into existence in the first place?
You are making the mistake of thinking that there was a single spot in space that everything expanded from. Rather, the expansion was not of things in space, it was space that expanded. It was an expansion OF space, not IN space. There wasn't "space" around the singularity.
This has not been answered yet for sure I guess. For example string theory (M-Theory?) suggests that our Universe lies on a higher dimensional "membrane" that "colided" with another one and thus created the big bang. This implies stuff like extra dimensions, strings, multiverses and other yet not proven theories.
Even why the big bang took place and how it could be that the matter in this universe is so unevenly spread out from a point that once consisted of perfectly even distributed energy is not clarified. "Who messed with it?" )
Big Question for sure! And you see how many quotation marks I use just to be careful with the wording. Maybe someone here can give a more scientific answer with some maths :) I can't.