Firstly, Big Bang didn't happen at a point in space, away from which we are traveling. Big Bang was the creation of space. This space has been expanding ever since, so that the distances between everything increases, but Big Bang happened right where you are, where the Andromeda galaxy is, where GN-z11 is, and so on.
The Universe has evolved ever since. If we want to know how galaxies look 13.8 billion years (Gyr) after Big Bang, we can just look around in our neighborhood. If we look too far away, we don't see 13.8 Gyr old galaxies, because the light has taken some time to reach us. Thus, if we want to see galaxies that are 12.8 Gyr old, we simply look 1 billion lightyears away; if we want to se 10 Gyr old galaxies, we look (roughly) 3.8 billion lightyears away, and so on.$^\dagger$ In this way, we look back in time.
If we look away far enough, we would in principle be able to look 13.8 Gyr back in time. However, we face a problem in that until the Universe was 380,000 years old, it was opaque to light (why this is so is another story). It wasn't opaque to gravitational waves, however. And since lots of GWs are thought to have originated during the epoch called inflation, which were responsible for the expansion of the Universe, and which took place a fraction of a second after the creation of space, we say that GWs offer the possibility of looking all the way back to Big Bang.
$^\dagger$This is somewhat imprecise, since galaxies weren't created instantly after Big Bang, and since all galaxies weren't created at the same time. But for the sake of the arument, let's pretend they were.