The further away we see, the older states of the world we observe. And the volume of the part of space which is observable, increases by the square of the distance to it. So there should basically be a million times more phenomena (data) observable about what happened a billion years ago, than about what happened a million years ago. Given good enough observatories.

This is obviously counter intuitive, and sorting it out for so called redshift effects doesn't help in that respect. Do/can we really (potentially) know more about things/events the older they are? Is there some epistemological problem involved in that?

There is of course a limit in any case. Once we get back to the microwave background at $z=1100$, then further probing backwards with electromagnetic radiation is stymied by the optically thick nature of the universe at early times.