Numerical differentiation

From Wikipedia, the free encyclopedia

Numerical differentiation is a technique of numerical analysis to produce an estimate of the derivative of a mathematical function or function subroutine using values from the function and perhaps other knowledge about the function.

A simple two-point estimation is to compute the slope of a nearby secant line through the points (x,f(x)) and (x+h,f(x+h)). Choosing a small number h, h represents a small change in x, and it can be either positive or negative. The slope of this line is

${f(x+h)-f(x)\over h}.$

This expression is Newton's difference quotient.

The slope of this secant line differs from the slope of the tangent line by an amount that is approximately proportional to h. As h approaches zero, the slope of the secant line approaches the slope of the tangent line. Therefore, the true derivative of f at x is the limit of the value of the difference quotient as the secant lines get closer and closer to being a tangent line:

$f'(x)=\lim_{h\to 0}{f(x+h)-f(x)\over h}.$

Since immediately substituting 0 for h results in division by zero, calculating the derivative directly can be unintuitive.

A simple three-point estimation is to compute the slope of a nearby secant line through the points (x-h,f(x-h)) and (x+h,f(x+h)). The slope of this line is

${f(x+h)-f(x-h)\over 2h}.$

More generally, three-point estimation uses the secant line through the points $(x - h 1, f (x - h 1))$ and $(x + h 2, f (x + h 2))$ . The slope of this line is

${f(x+h_2)-f(x-h_1)\over {h_1+h_2}}.$

The slope of these secant lines differ from the slope of the tangent line by an amount that is approximately proportional to $h 2$ so that three-point estimation is a more accurate approximation to the tangent line than two-point estimation when h is small.

1 Practical considerations
2 Higher order methods
3 References
4 See also
5 External links

[edit] Practical considerations

Example showing the difficulty of choosing

h

due to both rounding error and formula error

An important consideration in practice when the function is approximated using floating point arithmetic is how small a value of h to choose. If chosen too small, the subtraction will yield a large rounding error. If too large, the calculation of the slope of the secant line will be more accurate, but the estimate of the slope of the tangent by using the secant could be worse.

As discussed in Chapter 5.7 of Numerical Recipes in C (http://www.nrbook.com/a/bookcpdf/c5-7.pdf), a suitable choice for $h$ is $s q r t (e p s) * x$ where the machine epsilon eps is typically of the order 2.2e-16. Another important consideration is to make sure that h and x+h are representable in floating point precision so that the difference between x+h and x is exactly h. This can be accomplished by placing their values into and out of memory as follows: $h = s q r t (e p s) * x$ , $t e m p = x + h$ and $h = t e m p - x$ . It may be necessary to declare $t e m p$ as a volatile variable to ensure the steps are not undone by compiler optimization.