So far our estimates of the size of the Universe is from what it is expected to be (i.e. calculations) rather than what we see. But there are several problems.
Age of the Universe
We are pretty sure of the age of the Universe, 13.8B years old, and the time when the first light was emitted. This gives us a relatively good idea of the size of the Universe since we consider that the furthest edge of our Universe is the light that it has emitted so far which has gone outward and never hit anything.
Assuming a perfect sphere, a.k.a. a perfect Euclidean Universe (which it isn't,) then you would think the Universe is a sphere with a radius of about 13.8 billion light years and so its size would be:
$$size_{universe} = {\frac {4 \pi \times r^3} {3}}$$
With r = 13.8 billion light years.
Hubble Discovery: Expansion of the Universe
Only we know that our Universe is in expansion and there is nothing that says it's going to stop expanding. This affects the size of the Universe. Unfortunately, when we look around the cosmos, we can't see the expansion happening. We inferred it because of what we call the redshift. Most of the far away galaxies are moving away from us, the further they are, the longer the wave length of their light.
The speed at which the expansion happens is not yet set in stone either. Therefore, we have many arguing about the current size of the Universe.
Is there an Horizon?
One problem we currently have is the weight of the observable Universe. We calculated a weight which is much higher than what we see... so we added Black Matter. Matter which we can't see because it doesn't emit any kind of radiation and it's not a small weight: 85% of the calculated weight is missing!
Another theory, in that regard, is the possibility that the Expansion of the Universe has already moved most of the would be visible Universe over the Horizon.
When you go to the Ocean and look at the water, at some point you see the edge of the Earth. This is called the Horizon. Whatever is behind that line, you can't see it unless you go toward the horizon (when we observe the Universe, though, we stay in our solar system...) At such a size, the Universe is expanding faster than the speed of light. So light over the Horizon will never reach us because it can't go faster than the Expansion.
One more theory which is going to be hard to prove...
Limited Size?
There is a funny word circulating these days: Circumnavigation. This is like Pacman going out on the left side of the screen to re-appear on the right side. There are theories, which so far have been disproved, that the Universe would be finite and that things going in one direction reappear in on the other side. (If I'm correct, this is directly linked to the string theory.)
I guess some people think that this would have been great since that would mean our Universe has a specific size and things will always be around. They just float and go around, come back where they were... in an infinite loop.
So far, we have not been able to see light of galaxies coming from the opposite direction. This is also quite contradictory with the possibility of a Horizon.
I also personally think this is contradictory with the Expansion theory. If the Universe had a limited size, how could it also expand?
So what's the size?
We actually calculate a size from what we can observe using a radius of 46.5 billion light years. This gives us an estimation of
$$3.57×10^{80} m^3$$
This size takes our best estimate of the redshift in account.
For additional details about how the size is calculated, I would suggest reading the Size of the Observable Universe section on Wikipedia.