I'm thinking, in particular, about general relativity. When we speak, for example, of neutrino decoupling, what do we mean when we say this happened in the first second after the Big Bang? Do we mean one second from the point of view of an imaginary observer subject to the same relativistic effects on time? How much can we even say about the affect of relativity in this context?
Do we mean one second from the point of view of an imaginary observer subject to the same relativistic effects on time?
That's basically correct. Typically, cosmologists use comoving coordinates:
Although general relativity allows one to formulate the laws of physics using arbitrary coordinates, some coordinate choices are more natural or easier to work with. Comoving coordinates are an example of such a natural coordinate choice. They assign constant spatial coordinate values to observers who perceive the universe as isotropic. Such observers are called "comoving" observers because they move along with the Hubble flow.
A comoving observer is the only observer that will perceive the universe, including the cosmic microwave background radiation, to be isotropic. Non-comoving observers will see regions of the sky systematically blue-shifted or red-shifted. Thus isotropy, particularly isotropy of the cosmic microwave background radiation, defines a special local frame of reference called the comoving frame.
The comoving time coordinate is the elapsed time since the Big Bang according to a clock of a comoving observer and is a measure of cosmological time.
So we use the frame in which the cosmic microwave background and cosmic neutrino background are isotropic. In practice, we use the isotropy of the CMB because the cosmic neutrino background is so redshifted that the neutrinos have very low energy, so they're impossible to detect with current technology.