What are the Rules of Significant Figures, Precision, and Uncertainty, Strictly Speaking? [closed]

Question

In the physical sciences (which are physics, chemistry, astronomy, materials science, etc.), we learned that the uncertainty is +/- the smallest unit (which is 1) of the last significant figure if the uncertainty is not given in a recording of data. So, if we have a digital measuring device that measures to the nearest millimeter, has a manufacturer's stated uncertainty of +/- 1 mm, and gives a reading of 914 mm, then it will obviously be recorded as just "914 mm".

However, does the true value actually lie somewhere between exactly 913 mm and exactly 915 mm, or may it stray outside even those numbers if higher precision is used? For example, if go down to the micrometer, is the uncertainty actually +/- 999 μm or +/- 1,499 μm according to the rules of significant figures? If we measure the same sample using a micrometer, is the reading guaranteed to be somewhere between 913,001 microns and 914,999 microns, or is it instead only guaranteed to be somewhere between 912,501 microns and 915,499 microns, respectively?

Answer 1

...is the reading guaranteed to be..."

Nothing is guaranteed.

I think that if there is nothing further said and the reference is to an already known spectral line, then we should assume there's only one line near 914 nm and it's between 913.5 and 914.5, and they're not talking about a line at 914.9 nm.

However if they are reporting and observation as 914 nm, then if nothing further is said about the error and there are no error bars or +/- stated, then we really can't infer much about the uncertainty at all!

Had they said 914+/-1 nm we'd assume that the uncertainty was gaussian with a standard deviation of 1 nm, so 2/3 of the time it might be more than 1 nm off from the "correct" value.

I've never seen an error that had hard limits. So to

...does the true value actually lie somewhere between exactly 913 mm and exactly 915 mm, or may it stray outside even those numbers...

The answer is generally pretty much "yes, it certainly may stray outside those numbers!"

The problem here is that an instrument has both a a precision and an accuracy and a measurement using that instrument has a statistical uncertainty and in reality we must understand all three of those concepts and how they interact.

It's really quite a challenge in some cases, and so we must always remain vigilant.

Just for a random example, see the abstract of Measurement of the Branching Ratio for the Beta Decay of ¹⁴O where they break out the errors due to statistical uncertainty and uncertainty due to systematic errors separately!

We present a new measurement of the branching ratio for the decay of ¹⁴O to the ground state of ¹⁴N. The experimental result, λ0/λ_total = (4.934±0.040 (stat.) ±0.061 (syst.))×10−3, is significantly smaller than previous determinations of this quantity. The new measurement allows an improved determination of the partial halflife for the superallowed 0+ → 0+ Fermi decay to the ¹⁴N first excited state, which impacts the determination of the V_ud element of the CKM matrix. With the new measurement in place, the corrected ¹⁴O Ft value is in good agreement with the average Ft for other superallowed 0+ → 0+ Fermi decays.

Answer 2

A supplement to the (good) answer by uhoh:

Quoted limits of accuracy of astronomical measures have sometimes been called 'formal' limits by their authors. This can be a recognition that even after best efforts to estimate the accuracy of a measure, the error-estimate may be way off. Sometimes a measure turns out more accurate than its estimated accuracy, sometimes less.

An example: the mean tidal or non-gravitational decelerative component in the moon's motion in longitude has been of interest in a number of branches of astronomy for a long time. It has not been the easiest parameter to measure. Nearly 50 years ago, after a previous history of somewhat scattered estimates by others, Morrison & Ward (1975) obtained a value of (-)26"/cy/cy, to which they attached an error-estimate of +/-2"/cy/cy.

Since then, numerous hopefully-improved estimates have been published, many of them the products of lunar laser ranging, a tool not available when Morrison and Ward did most of their work. Nearly all have been accompanied by much smaller error-estimates, but many of the deceleration estimates themselves have been considerably less than 26".

By contrast, the most recent estimate of this parameter that I can see, from Williams and Boggs (2016), returns almost to (-)26": their value is -25".97 +/-0".05 /cy/cy.

So the 1975 value now looks much better than its nominal error-limits suggested, and some of the intervening values now look worse than their authors hoped.

Was the goodness of the 1975 value just a fluke?

It might be very difficult now to establish or exclude that possibility, but there are some signs that certain authors have tended regularly to produce better measurements than their nominal error-estimates might suggest, while others have tended to produce worse.

The history of estimation of many astronomical quantities yields several examples of similar kind: The mean motion of the moon's apogee was better estimated by J Horrocks (1638-40) than by several eminent successors until about 1753 (date of Tobias Mayer's (good) result); the moon/earth mass ratio was better estimated by J D'Alembert (1749) than by other eminent successors for over half a century after; and similar effects can be seen in histories of estimates of the solar parallax from the 1760s, and of the difference of meridians Greenwich-to-Paris from the late 17th century.

These are some illustrations of how, indeed, nothing is guaranteed.