There are more stars and galaxies but not necessarily more information.
You might choose to measure the information content in terms of photons received, in which case, for a given star/galaxy, this decreases as the inverse square of the distance, so the two effects cancel out.
Astronomy is a constant struggle with the tension between these two competing dependencies. As we look further, not only do we see more of particular types of object, but we also get to see examples of rare objects - i.e. the tails of the distributions. On the other hand as we expand our horizons, so our grasp of what is going on becomes increasingly blurred in terms of spatial, spectral and temporal resolution. It is for that reason that bigger and bigger telescopes are built!
The issue you raise concerning redshift is interesting. The co-moving volume versus look back time in any direction will not be a simple function (i.e. doesn't just go as the square of the lookback time) and depends critically on the adopted cosmological parameters.
There is of course a limit in any case. Once we get back to the microwave background at $z=1100$, then further probing backwards with electromagnetic radiation is stymied by the optically thick nature of the universe at early times.