I understand that as the universe continues to expand, eventually our supply of hydrogen will be exhausted and then it's "lights out." Is that really going to happen or is there something "eternal" about the universe? It matters because if the universe is eternal, and since we are a part of the universe, we too are eternal as well in some sense.
The metaphysical aspects of your question can't be answered. Our bodies aren't eternal, and astronomy can't answer questions about the soal.
Observations of the universe suggest that it will continue expanding.
As you note, eventually all the stars will run out of hydrogen, collapse to white dwarfs, which then slowly cool. What happens next is rather speculative. At this time there will be nowhere that is warm in the universe. Everywhere will cool to very close to absolute zero. Complex chemistry will no longer be possible, and there will be no life in the universe.
Some stars will fall into black holes. Others will be ejected from the galaxy. If protons decay, then very very very slowly, the matter will break down, leaving only black holes.
If black holes emit Hawking radiation, then they too will break down.
In this model, the spacetime of the universe is eternal, but nothing within it survives.
Wikipedia has a list of some of the events of the far future
The universe is eternal...in a sense. The universe is rapidly expanding, and with that, the amount of space in between everything. The point when astrophysicists consider the universe to have "ended" is when all stars have died and there components are too far apart to create new ones. When the planets of solar systems are too far from their stars to see their light. When our universe is ruled by black holes who swallow up what is left of our once beautiful universe. When (theoretically) the redshift effect would occur on such a vast scale that we would no longer be able to see the light from the stars in the visible universe( by that point we will all have been dead for billions of years). To the point where the universe can no longer harbor life. The universe will still exist in a technical sense, but it will be a cold and unrecognizable universe from the one we call home today.