Measuring Error

Numerical Methods

David Mayerich

Scalable Tissue Imaging and Modeling (STIM) Laboratory

Department of Electrical and Computer Engineering

Cullen College of Engineering

University of Houston

David Mayerich

STIM Laboratory, University of Houston

Error Behavior

David Mayerich

STIM Laboratory, University of Houston

  • Define error behavior using Big-O notation and error terms:

  • Bound differences between errors as parameters change:

x_1 = 0.50
\epsilon_1 \leq C\left|0.5\right|^3
\epsilon_2 \leq C\left|0.75 \right|^4
n_1 = 3
x_2 = 0.75
n_2 = 4
\epsilon_1 \leq 0.125 C
\epsilon_2 \leq 0.316 C
\frac{\epsilon_1}{\epsilon_2} \approx 0.396

  • We can calculate how error bounds change without precisely knowing \(\epsilon_1\) or \(\epsilon_2\)

  • More precise values for \(\epsilon\) require approximating \(C\) in some way:

    • theoretically using features of the function

    • practically by comparing the result to some ground truth

e^x = 1 + x + \frac{x^2}{2} + \frac{x^3}{6} + O\left(x^4\right)

where

O\left(x^n\right) \leq C\left| x^n \right|

Approximating Error Analytically

  • Consider the Lagrange remainder in a finite Maclaurin series expansion:

David Mayerich

STIM Laboratory, University of Houston

\frac{f^{(k+1)}(z)}{(k+1)!}x^{k+1} = O\left( x^{k+1} \right)
f(x) = \sum_{n=0}^{k} \frac{f^{(n)}(0)}{n!}(x)^n + \frac{f^{(k+1)}(z)}{(k+1)!}x^{k+1}

  • What is \(\epsilon\) for \(e^x\) using a 5th-order series?

\frac{f^{(k+1)}(z)}{(k+1)!}x^{k+1} = C\left| x \right|^{k+1}
C = \frac{f^{(k+1)}(z)}{(k+1)!}

where \(z \in [0, x]\)

e^x = 1 + x + \frac{x^2}{2} + \frac{x^3}{6} + \frac{x^4}{24} + \frac{x^5}{120} + \epsilon
  1. If \(\frac{d}{dx} e^x = e^x\) then \(f^{(d+1)}(z) = e^z\)
  2. \(e^x\) is monotonically increasing, so the largest value for \(z \in [0, x]\) is \(z=x\)
  3. The numerator is at most \(e^x\)
  4. The error in the worst case is \(\epsilon = \frac{e^x}{6!}x^6\)
y = exp(1.0);
\epsilon \leq \frac{2.718\cdots}{720} 0.5^6\leq 0.0038\cdots
y = exp(0.5);
\epsilon \leq \frac{1.65}{720}0.5^6 \leq 0.0000358

Approximating Error Practically

  • Absolute error is the magnitude of the difference between true and calculated values:

David Mayerich

STIM Laboratory, University of Houston

\epsilon_a = \left| \epsilon \right| = \left| t(x) - c(x) \right|

  • How do we get the true value \(t(x)\)?

  • Once again, consider \(e^x\) as a 5th-order Maclaurin series:

e^x = 1 + x + \frac{x^2}{2} + \frac{x^3}{6} + \frac{x^4}{24} + \frac{x^5}{120} + \epsilon
float exp5(x){

	float ex = 1;					// initialize with e^0
    float xp = 1;					// initialize with x^0
    float fac = 1;					// initialize 0!
	for(int i = 1; i <= 5; i++){
    	xp *= x;
        fac *= i;
    	ex += xp / fac;
    }
    return ex;
}

exp5(1.0f) = 2.716667

NASA has \(e^x\) out to 2M digits:
\(e^x \approx 2.718281828459045235\cdots\)

\epsilon_a = |\epsilon| = 0.00161516179\cdots
\epsilon \leq 0.00378\cdots

analytical result

Understanding Error

  • An error is a deviation of a result from reality

    • a systematic error is introduced by bias - this can often be correlated

    • a random error is a product of uncertainty and impossible to correct

    • random errors can often be controlled or bounded

  • Absolute vs. relative error:

David Mayerich

STIM Laboratory, University of Houston

\epsilon_a = \left| \epsilon \right| = \left| t(x) - c(x) \right|
\epsilon_r = \left| \frac{\epsilon_a}{t(x)} \right| = \left| \frac{t(x) - c(x)}{t(x)} \right|
t(x) = 1.5
c(x) = 1.0
t(x) = 100.5
c(x) = 100.0
\epsilon_a = 0.5
\epsilon_r \approx 0.3 \ (30\%)
\epsilon_a = 0.5
\epsilon_r \approx 0.005 \ (0.5\%)
t(x) = 0.03
c(x) = 0.53
\epsilon_a = 0.5
\epsilon_r \approx 16.7 \ (1670\%)

  • Consider the following true and calculated values:

Relative Error

David Mayerich

STIM Laboratory, University of Houston

  • Scientific and engineering applications are less sensitive to small errors in large values

    • \(\pm 1\) ton is a lot for a truck but not for a cargo ship

  • Relative error can compare values at widely varying scales

  • Relative error is often expressed as a percentage error: \(\epsilon_r \times 100\)

  • Most computing systems are designed to minimize relative error
     

  • Undefined when the \(t(x) = 0\), which can be resolved in a few ways:

regularization

\epsilon_r = \frac{\epsilon_a}{\left| t(x) \right| + \sigma}

specify a value

\epsilon_r = \begin{cases} \frac{\epsilon_a}{\left| t(x) \right|} & \text{if}\ \ t(x)\neq 0\\ 1 & \text{otherwise} \end{cases}

Discussion: Interval Measurements

David Mayerich

STIM Laboratory, University of Houston

  • Relative error requires a ratio scale with a natural zero

  • Ratios have to make sense

    • \(2x\): there is twice some quantity

    • \(\frac{1}{3}x\): there is one third of something

  • The zero indicates that there is zero of something

Celsius

\(t(x)=10.0^\circ \text{C}\)

\(c(x)=11.0^\circ\text{C}\)

\(\epsilon_a = 1^\circ\text{F}\)

\(\epsilon_r = 0.1 \ (10\%)\)

Kelvin

\(t(x)=283.0 \text{K}\)

\(c(x)=284.0\text{K}\)

\(\epsilon_a=1\text{K}\)

\(\epsilon_r=0.0035 \ (0.35\%)\)

Fahrenheit

\(t(x)=50.0^\circ \text{F}\)

\(c(x)=51.8^\circ\text{F}\)

\(\epsilon_a = 1.8^\circ\text{F}\)

\(\epsilon_r = 0.036 \ (3.6\%)\)