I have recently noticed an orange "star" (or what I presume to be a star), rise above the hill line behind my home. It rises later than every other star and rises within a few hours while the other stars are stationary! It moves off to the right when half way in the night sky this happens every night when clear for the past few months. I also believe I'm looking south. It's really bugging me. Could anyone identify this object from my description?
The object is (most likely) Mars! It's been rising in the south as of late. When it will rise of course depends on your location...for me it rises pretty close to sunset. Planets appear in an essentially constant position relative to the background stars over the course of the night, and as with the stars, the movement you see is due to the rotation of the Earth. Over the course of weeks, however, you can see the planets change position relative to the stars due to their motion in their orbit around the Sun. We don't see this wandering of position with stars - at least, not in an amount or on a timescale we can appreciate - because they're so far away.
Keep an eye out for other bright objects in the sky that appear to move faster and (unless conditions are poor) don't really twinkle, especially near sunset before the stars properly come out - Mars probably isn't the only planet you can see! Jupiter, Saturn and Venus are often visible as well.
Check out this site if you want to see what planets are visible from your location and when.
ETA: Corrected explanation of planetary motion vs. motion of stars based on James' comment.