Light rays travel along things called geodesics in spacetime (specifically they travel along a certain class of geodesics called 'null geodesics').
If spacetime is flat, which will be the case in the absence of gravity, then those geodesics are just straight lines as we normally think of them, and so yes, light rays travel along straight lines in that case.
But there is gravity and spacetime is therefore not flat. This means there there is really no definition of 'straight line' any more: geodesics are the best thing we have (that's why they're called geodesics rather than straight lines).
But in very many cases the deviation caused by spacetime not being flat is pretty small: for instance it's possible to see the difference in apparent position of stars if the light from them passes close to the Sun, but it's a small deviation.
In some cases – if the light passes close to very massive objects, the deviation can be very large (and in fact if the object is massive enough, it can be as large as you like).
The general effect of light travelling along geodesics which are not well-approximated by straight lines is called 'gravitational lensing' and it's now a well-confirmed thing which is used in practice to detect things like exoplanets (amazingly, in this case it's the lensing of the light from stars by the planet which is observed, which is just astonishing.)
So a brief answer is yes, almost always, but occasionally not at all!