Reference library22 entries

Category: Precision

Core reference for precision, rounding, and significant figures, including the definitive sig fig calculator.

Oct 2026

Accuracy vs Precision: Definitions, Differences, and Why Both Matter

Accuracy refers to how close a measurement is to the true value, while precision refers to the consistency of repeated measurements. Understanding the distinction…

Read explanation

Oct 2026

Accuracy vs Precision

Systematic vs Random Error: How to Identify and Fix Each

A comprehensive guide to distinguishing between systematic and random errors, including identification techniques and mitigation strategies based on ISO and NIST standards.

Read explanation

Oct 2026

Standard Deviation Reporting

Why Does My Teacher Round the Mean Differently Than the Standard Deviation?

The mean and standard deviation are rounded differently because they answer different questions: the mean follows decimal-place precision of the data, while the SD…

Read explanation

Oct 2026

GUM Guide

Why Leading Zeros Are Not Significant in the GUM: A Metrology Perspective

An authoritative explanation of why leading zeros serve only as placeholders and do not contribute to the precision of a measurement according to GUM…

Read explanation

Oct 2026

Measurement Uncertainty

Type A vs Type B Uncertainty: What’s the Difference and When to Use Each

Type A uncertainty is evaluated by statistical analysis of repeated observations; Type B uncertainty is evaluated by any other documented method, such as calibration…

Read explanation

Oct 2026

Measurement Uncertainty

When to Use Order of Magnitude in Uncertainty Budgets: A GUM Perspective

Order-of-magnitude estimates are useful for screening and Type B bounding in GUM uncertainty budgets, but they are not a substitute for evaluated standard uncertainties…

Read explanation

Oct 2026

GUM Guide

GUM Uncertainty Rounding: When to Use Round Half to Even for Measurement Results

GUM result expression uses ISO 80000-1 rounding: round the expanded uncertainty to one or two significant digits, match the estimate to the same decimal…

Read explanation

Oct 2026

Absolute vs Relative Error

How to Calculate Absolute and Relative Error in Excel: Rules, Formulas, and Precision Standards

Absolute error is the magnitude of the difference between a measured and reference value; relative error divides that difference by the reference value. This…

Read explanation

Oct 2026

Measurement Uncertainty

How to Handle Auto-Ranging Digital Meters: Why the Last Digit Changes and How to Record It

Auto-ranging digital meters change resolution and display counts as they select ranges. Record the displayed value with the meter's current resolution, do not add…

Read explanation

Oct 2026

Measurement Uncertainty

Reading a Vernier Caliper: Significant Figures and Uncertainty

A comprehensive guide to interpreting vernier caliper readings with strict adherence to significant figure rules and ISO/GUM uncertainty standards.

Read explanation