The short answer is, "gravity". The time of "co-moving observers" (observers moving with the expansion of the universe") is expanding, but our "proper time" is not, because we're within a local group of galaxies that are bound together gravitationally. The principles involved are detailed in the Wikipedia article "Co-moving and proper distances", and its links.
However, in any reading about General Relativity--the currently-accepted theory of gravity-- it may be helpful to understand that "observers" are really any sort of hypothesized material or energetic object, however small: I believe they were supposed to be hypothetical but possible beings, when use of that terminology began--back around the time when GR's predecessor, Special Relativity, was written--but are now involved in so many inaccessible and inhospitable situations that "they" are not necessarily considered to be sentient, or even alive. It may also be useful to understand that, although gravity is realistically considered to be the curvature of spacetime, "time" itself is best viewed as comparable to the three well-known spatial dimensions, or at least as something comparably definite, like "duration": The temporal aspect of spacetime actually includes most of its curvature, as described in a website about General Relativity by the University of Pittsburgh's Dr. Norton.
Regarding your secondary question, the delay in the arrival of particles (photons) from distant stars is actually affected by several factors, generally based on inflationary cosmology, that are in a well-known diagram shown in the first Wikipedia item I suggested, but visible in larger versions with explanatory text: Searches by the names of its originators, "Lineweaver and Davis", will reach references on various levels of sophistication.
