Reference library17 entries

Category: Precision

Core reference for precision, rounding, and significant figures, including the definitive sig fig calculator.

Oct 2026

When to Use Order of Magnitude in Uncertainty Budgets: A GUM Perspective

Order-of-magnitude estimates are useful for screening and Type B bounding in GUM uncertainty budgets, but they are not a substitute for evaluated standard uncertainties…

Read explanation

Oct 2026

GUM Guide

GUM Uncertainty Rounding: When to Use Round Half to Even for Measurement Results

GUM result expression uses ISO 80000-1 rounding: round the expanded uncertainty to one or two significant digits, match the estimate to the same decimal…

Read explanation

Oct 2026

Absolute vs Relative Error

How to Calculate Absolute and Relative Error in Excel: Rules, Formulas, and Precision Standards

Absolute error is the magnitude of the difference between a measured and reference value; relative error divides that difference by the reference value. This…

Read explanation

Oct 2026

Measurement Uncertainty

How to Handle Auto-Ranging Digital Meters: Why the Last Digit Changes and How to Record It

Auto-ranging digital meters change resolution and display counts as they select ranges. Record the displayed value with the meter's current resolution, do not add…

Read explanation

Oct 2026

Measurement Uncertainty

Reading a Vernier Caliper: Significant Figures and Uncertainty

A comprehensive guide to interpreting vernier caliper readings with strict adherence to significant figure rules and ISO/GUM uncertainty standards.

Read explanation

Oct 2026

Measurement Uncertainty

Why Do Some Textbooks Use the Full Scale Division for Uncertainty Instead of Half? A Comparison of Conventions

Textbooks disagree on whether reading uncertainty is half or one full scale division because they assume different interpolation models, conservatism levels, and digital count…

Read explanation

Oct 2026

Accuracy vs Precision

Trueness vs Accuracy: The VIM Definition and Why It Matters in Calibration

Understand the critical distinction between trueness and accuracy according to the VIM, and how these concepts govern precision measurement and rounding standards.

Read explanation

Aug 2026

Error Propagation

Error Propagation: How Uncertainty Moves Through a Calculation

Learn the rules for propagating uncertainty through addition, subtraction, multiplication, division, and powers, with worked examples, common pitfalls, and standards citations from GUM, ISO,…

Read explanation

Aug 2026

Accuracy vs Precision

Accuracy, Precision, and Uncertainty: What Each One Actually Measures

Understand the distinct meanings of accuracy, precision, and uncertainty in measurement, with standards-based definitions, worked examples, and common pitfalls.

Read explanation

Aug 2026

Measurement Uncertainty

How to Read Uncertainty From a Ruler, Burette, or Digital Meter

Learn the correct way to interpret measurement uncertainty from analog and digital instruments, including significant figures, rounding, and standard conventions.

Read explanation