Are the stars we can see with the naked eye in the night sky only from our Milky Way galaxy or can we see stars from Andromeda? I am aware we can see other celestial objects like nebulas and the Andromeda Galaxy itself.
The resolved stars (those that can be seen as individuals) are all part of the Milky Way Galaxy (unless there are any interlopers that have been captured!).
The distances to the next nearest galaxies of any size are more than 100,000 light years. Andromeda is 2 million light years away. Unless one goes supernova, there basically aren't any types of star that are intrinsically bright enough to be seen with naked eye at these distances. Most of the stars we do see are at distances of one AU (the Sun) to a few thousand light years, with a median of a few hundred light years (See Aren't there more naked-eye-visible stars in the Milky Way plane? for more detail).