Does a gas star have a clear boundary between its core and shell? If it does, then what is keeping a helium core distinct from its hydrogen shell, for a main-sequence star that transforming to a giant star?
This depends on the initial mass of the star, but generally the gradient of the core is not sufficiently strong on the main sequence to justify a well defined core-envelope boundary, which is why if a star on the main sequence experiences Roche-lobe overflow the entire star is essentially destroyed, e.g., in a binary system.
The core of a main sequence star develops due to the nuclear fusion of hydrogen into helium. The main sequence ends when the core is exhausted of hydrogen, and then low mass stars enter the giant branch phase and high mass stars enter the Hurtzsprung-gap: in both cases the star begins shell burning of hydrogen around the core. Then, once the temperature in the core is sufficiently high, core helium burning begins which brings the star to the next stage of development - the supergiant phase.
The star chokes like an engine does when it switches from one form of nuclear fuel to another, resulting in contractions of the core and expansions of the envelope, resulting in the star's surface radius to generally increase as it ages.
If it does, then what is keeping a helium core distinct from its hydrogen shell, for a main-sequence star that transforming to a giant star?
So, once a star is on the supergiant phase, its core is sufficiently well defined to support itself under the envelope by the luminosty pressure from the core and the shells surrounding the core, and the core supports its own self gravity by the degenerate matter that develops composed mostly of oxygen and carbon (i.e., the electron degeneracy pressure).
There is also rotational support and its effects, which is an active area of research. The core-envelope boundary is (literally) a turbulent region where chemical mixing occurs which drives convective currents through the envelope and which can result in angular momentum transfer between the core and envelope resulting in spin coupling between core and envelope. This spin coupling can have implications for the late evolution of the star. I asked a question about this that's remained unanswered.
Enhanced rotational mixing of the chemical elements inside a main sequence star can lead to chemically homogeneous evolution, in which there is no longer a core-envelope boundary since the helium has been mixed throughout the star.