Celestial Navigation - LOP uncertainty

Question

In doing a sight reduction of a celestial body at a given time the goal is to define a line on a chart that includes the location you think you are in. Given the angle of a sighting, the UTC time/date and the body, a great circle with a determined radius is specified.

That isn’t very helpful in immediate navigation so there is a process of finding a line on a local chart. It uses an assumed position and calculates what the angle of the sighting would be if you were standing at the AP. From the difference between that and the observed angle and the direction the body was from the AP, a nearby linear segment of the great circle is determined.

I’m at latitude 33 degrees 45‘ W and initially used 33 degrees 0’ W in the AP and calculated and drew a line of position (LOP). Realizing that 34 degrees 0’ was the better round number latitude to use, I did the calculation over again, expecting little difference in the resulting LOP. The AP is supposed to be arbitrary but the two LOPs are not very close.

They are parallel and about 11 nautical miles apart. I have no problem understanding being that far off in an actual sighting but this (almost) just math.

Do I misunderstand some approximation in the process that would naturally lead to this? Otherwise I’m making some mistake.

I used a sight reduction app, not tables in a book.

After posting:

I calculated the computed angle from the actual GPS fix I was standing at and my measurement was 20’ off. Using what should have been the sighting, I recalculated for the two AP locations and now the LOPs are only 3 nm apart. I believe that sighting was the only factor that wasn’t just math and correcting it improves things but I still have a discrepancy that might be due to an appropriation in the process that I do not appreciate or to my further error.

Answer 1

Unless you're planning to finish the sight reduction using Hydrographic Office multi-volume tables, there's no need to round off to the nearest whole degree, but doing so is good enough, and might reduce errors in data entry.

Start by solving the spherical triangle with vertices at a pole, your assumed (or best-guess) position, and the geographical position (GP) of the celestial body at the time of the sight. The azimuth of the celestial body calculated at this step will usually be more accurate than an observed azimuth. Then your LoP will be a line perpendicular to the line from you to the celestial body's GP, (a line whose azimuth you just found) at a distance from that GP equal to 90 degrees minus the celestial body's observed (sextant) altitude.

Using what you've calculated can be continued in a few different ways. If you were using a maneuvering board, you would have the assumed/estimated position at the center and draw the azimuth line through it. From the solution to the triangle that gave you azimuth you could also have found the distance from the assumed position to the celestial body's GP. The difference between that distance and the observed distance tells you where the LoP intercepts the azimuth line.

You should usually have an error of less than an arc minute in the location of LoPs using slightly different assumed positions. In the general case the lines won't be parallel. They'll be slightly off because of the different starting points.