Regarding matter that has the highest redshifts, do we see such matter in every general direction we look (relative to Earth: up down left right front back)?
It's not possible to see photons from "beyond" the cosmic microwave background (CMB), because (assuming our cosmology is generally correct) the CMB is the result of recombination of electrons with protons in the early universe. Photons from before that era were absorbed in the plasma.
Essentially then, the earliest matter we can see (in photonic terms) is this glowing hot hydrogen, but massively red-shifted so it looks like hydrogen at 3K.
We see the CMB in every direction.