Reference library14 entries

Category: Precision

Core reference for precision, rounding, and significant figures, including the definitive sig fig calculator.

Oct 2026

How to Handle Auto-Ranging Digital Meters: Why the Last Digit Changes and How to Record It

Auto-ranging digital meters change resolution and display counts as they select ranges. Record the displayed value with the meter's current resolution, do not add…

Read explanation

Oct 2026

Measurement Uncertainty

Reading a Vernier Caliper: Significant Figures and Uncertainty

A comprehensive guide to interpreting vernier caliper readings with strict adherence to significant figure rules and ISO/GUM uncertainty standards.

Read explanation

Oct 2026

Measurement Uncertainty

Why Do Some Textbooks Use the Full Scale Division for Uncertainty Instead of Half? A Comparison of Conventions

Textbooks disagree on whether reading uncertainty is half or one full scale division because they assume different interpolation models, conservatism levels, and digital count…

Read explanation

Oct 2026

Accuracy vs Precision

Trueness vs Accuracy: The VIM Definition and Why It Matters in Calibration

Understand the critical distinction between trueness and accuracy according to the VIM, and how these concepts govern precision measurement and rounding standards.

Read explanation

Aug 2026

Error Propagation

Error Propagation: How Uncertainty Moves Through a Calculation

Learn the rules for propagating uncertainty through addition, subtraction, multiplication, division, and powers, with worked examples, common pitfalls, and standards citations from GUM, ISO,…

Read explanation

Aug 2026

Accuracy vs Precision

Accuracy, Precision, and Uncertainty: What Each One Actually Measures

Understand the distinct meanings of accuracy, precision, and uncertainty in measurement, with standards-based definitions, worked examples, and common pitfalls.

Read explanation

Aug 2026

Measurement Uncertainty

How to Read Uncertainty From a Ruler, Burette, or Digital Meter

Learn the correct way to interpret measurement uncertainty from analog and digital instruments, including significant figures, rounding, and standard conventions.

Read explanation

Aug 2026

Absolute vs Relative Error

Absolute Error vs Relative Error vs Percent Error: A Comprehensive Guide

Understand the differences between absolute, relative, and percent error, including formulas, when to use each, standards, and common pitfalls. A definitive reference for measurement…

Read explanation

Aug 2026

Percent Error

How to Calculate Percent Error (and Report It Correctly)

Learn the correct formula for percent error, how to handle significant figures and rounding, and avoid common pitfalls when reporting measurement accuracy.

Read explanation

Jul 2026

Standard Deviation Reporting

Reporting Mean and Standard Deviation With Correct Precision

When reporting experimental results, the mean and standard deviation (SD) are the most common summary statistics. However, the precision with which these values are…

Read explanation