Floating Point and Rounding Error
Computer arithmetic is approximate.
Key idea
Computer arithmetic is approximate.
In depth
Computers store numbers with finite binary precision, so most decimals are approximated: 0.1 + 0.2 ≠ 0.3 exactly. Errors propagate, and subtracting nearby numbers causes catastrophic cancellation. Machine epsilon is about 2.2 × 10⁻¹⁶ for doubles.
Worked example
(1 + 10⁻¹⁶) − 1 = 0 in double precision.
Real-world uses
- Understanding why spreadsheets and calculators sometimes show tiny errors such as 0.1 + 0.2
Common mistake
Testing floating-point numbers for exact equality.
How to learn this
- Read the key idea and say it in your own words.
- Work through the worked example, then try a similar problem.
- Use the interactive lesson’s visual and mini-quiz to check the pattern.
Frequently asked questions
What is Floating Point and Rounding Error?
Computer arithmetic is approximate. It is a topic at the expert level in Differential Equations, Abstract Algebra & Applied Math.
Where is Floating Point and Rounding Error used in real life?
Real-world examples: Understanding why spreadsheets and calculators sometimes show tiny errors such as 0.1 + 0.2.
What should I learn before Floating Point and Rounding Error?
Make sure you are comfortable with Numerical Differentiation, the previous topic in Differential Equations, Abstract Algebra & Applied Math.
What is a common mistake with Floating Point and Rounding Error?
Testing floating-point numbers for exact equality.
How can I practise Floating Point and Rounding Error?
Open the interactive lesson, play with the visual, then answer the three-question mini-quiz. Getting two right marks the topic complete.
What comes after Floating Point and Rounding Error?
Move on to Numerical Linear Algebra next.
Related expert topics in Differential Equations, Abstract Algebra & Applied Math
- Exact Equations – M dx + N dy = 0 with Mᵧ = Nₓ.
- Bernoulli Equations – Substitute v = y¹⁻ⁿ to linearise.
- Runge–Kutta Methods – Higher-accuracy numerical solvers.
- Second-Order Homogeneous Equations – Use the characteristic equation.
- Undetermined Coefficients – Guess the particular solution form.
- Variation of Parameters – General method for particular solutions.