If the first stars in the universe were formed 400 million years after the big bang, and the universe is expanding, should that stars at that time not been in our cosmic backyard? And should their light not have reached earth much sooner?
Yes, and indeed it did. For some stars.
Some first stars were close to us, some were far away. The light from the ones that were very far away has yet to reach us, while the light from the ones that were close to us, reached us in the past; if we look today at their location, we no longer see first stars, but instead see evolved galaxies.
On the other hand, the light from the ones that happened to lie at the particular distance of roughly 2.7 billion lightyears (Gly) when the Universe was 400 Myr old reaches us today. An example of this is the galaxy GN-z11.
"2.7 Gly?", you say, "Shouldn't that light have reached us after 2.7 Gyr, then?"
No, because the Universe expands, it takes longer. In fact at that time, the part of the Universe that was at the distance of GN-z11 expanded at four times the speed of light. That means that even though light traveled in our direction (locally at the speed of light), the distance from us$^\dagger$ to the photons increased. Eventually, however, as the photons reached regions that expanded less rapidly, they "overtook" the expansion of the Universe, and slowly started decreasing their distance to us.
$^\dagger$The hypothetical "us"; our Sun wasn't even born at that time, and probably the Milky Way was just a small density fluctuation in the background density.
The very first stars were probably, massive, ultraluminous, and very short-lived. So if they were formed 400 million years after the big bang, then they would have ceased to exist only a few million years later.
Given that, if we wish to observe the first stars, then we have to look at the light from galaxies that has been travelling for 13.3 billion years to get to us. Hence we must look at very, very distant galaxies.
If we consider very old, but second generation stars (known as "Population II stars) - those born from material that incorporates the detritus of the first stars, then we can see those closer to home. That is because some of those stars were long lived and survive to the present day in our own and other nearby galaxies.