This might be a dumb question.
This is an excellent question!
When we watch a solar eclipse, or the occultation of a star (or planet) by the Moon, and we want accurate timing information (or we want to record it) we must know our 3D position (lat, lon, elev) in order to either get an accurate time prediction or use measured timing to say something about the relative positions of objects in 3D space.
That's because of exactly what you point out; the two coordinate systems have different centers, and also because one (the celestial sphere) is at infinity.
Why do people use this diagram to derive transformation equations?
Because for say 99% of the time the things we look at are far enough away (e.g. light years) that the ~6370 (give or take) kilometers between the two centers make no difference at the level that we can measure.
When it does matter, astronomers do in fact go to more rigorous calculations in 3D cartesian space rather than exclusively in spherical coordinates, and besides the distance to the object even take into account things like the speed of light and astronomical aberration.
Shouldn't the equatorial system and the horizontal system have different centers?
the answer is a resounding Yes!