The Remainder Term in Taylor Series

Taylor Series

The philosophy of Taylor series is to approximate a function with polynomials that use only the information of a single point in the domain of the function. Consider the function $f: \mathbb{R} \rightarrow \mathbb{R}$ and a point $a \in \mathbb{R}$ . Further, we would like to use $(x-a)^0$ , $(x- a)^1$ , $(x- a)^2$ , ..., $(x- a)^n$ as the basic building blocks of our approximation. The polynomial constructed is of the form:

p_n(x) = c_0 + c_1 (x - a) + c_2(x - a)^2 + ... + c_n (x - a)^n

Now, assume that the $n$ -th derivative of $f$ exits. To approximate $f$ well, $p_n$ should satisfy

p_n(a) = f(a)

so that $c_0 = f(a)$ .

Further, $f'(a)$ tell how the function $f(x)$ changes when $x$ moves from $a$ by a little bit. We require $p_n(x)$ to exhibit the same behavior:

p'_n(a) = f'(a)

so that $c_1 = f'(a)$ .

To fit $f(x)$ even better, we force the second order derivative of $p_n(x)$ at $x$ comply with $f''(a)$ , hence

p_n''(a) = f''(a)

and $c_2 = \frac{1}{2} f''(a)$ .

Following the above procedure,we get

c_i = \frac{1}{i!}f^{(i) } (a)

and

p_n(x) = \sum_{k = 0}^n \frac{ f^{( k ) }(a) }{ k! } (x - a)^k

where $f^{(0)} \doteq f$ , $0! \doteq 1$ . We have just show that the construct polynomial satisfies

Theorem. $p^{ (k) }_n (a) = f^{ (k) } (a)$ , for $k \in [n]$ .

Remainder

One natural to ask is that, how accurate is the $p_n(x)$ . In particular, given $x \neq a$ , we want to measure the error

r_n(x) = f(x) - p_n(x)

quantitively.

Note that if the function $f$ is itself a polynomial up to order $n$ , then the approximation is accurate that is $r_n(x) \equiv 0$ for $x \in \mathbb{R}$ . In this case, we recover $f$ globally using only local information at $a$ .

In general, if $f^{ (n + 1)} (x)$ exists and is continuous, then the error is characterized as

Theorem

r_n(x) = \frac{1}{n!} \int_a^x f^{ (n + 1) } (t) (x - t)^{n } dt = -\int_a^x f^{ (n + 1) } (t) d \left( \frac{(x - t)^{n + 1} }{(n + 1)!} \right)

Proof.

By fundamental theorem of calculus, we have

f(x) = f(a) + \int_{a}^x f'(t) dt

On the other hand

\frac{\partial [f'(t) (t - x)] }{\partial t} = f''(t) (t - x) + f'(t)

Therefore,

f'(t) (t - x) \mid_{t = a}^x = f'(a) (x -a) = \int_{a}^x f''(t) (t - x) dt + \int_{a}^x f'(t) dt

and

f(x) = f(a) + f'(a) (x - a) + \int_{a}^x f''(t) (x - t) dt

Similarly,

\begin{aligned} \int_{a}^x f''(t) (x - t) dt &= - \int_{a}^x f''(t) d \frac{ (x - t)^2}{2!} \\ &= f''(a) \frac{ (x - a)^2}{2!} + \int_{a}^x \frac{ (x - t)^2}{2!} f^{(3) } (t) dt \end{aligned}

As just shown, the proof can be finished by induction. $\square$

Weighted Mean Value Theorem. Given a continuous functions $f \ge 0$ and an Riemann integral function $g \ge 0$ defined on an closed interval $I$ , then

\int_I f(t) g(t) dt = f(c) \int_I g(t) dt

for some $c \in I$ .

Proof. As $f$ is continuous and $I$ is closed, $\exists m, M \in \mathbb{R}$ , s.t.,

m \le f(t) \le M, \qquad \forall t \in I

Therefore,

mg(t) \le f(t) g(t) \le M g(t), \qquad \forall t \in I

Define $S = \int_I g(t) dt$ . By monotonicity of Riemann integration, we have

m S \le \int_I f(t) g(t) dt \le M S

Note that $S f(t)$ is a continuous function in $I$ . Hence, by mean value theorem, $\exists c \in I$ , such that

S f(c) = \int_I f(t) g(t) dt

$\square$

Corollary 1. By weighted mean value theorem, we have

\begin{aligned} r_n(x) &= \frac{1}{n!} \int_a^x f^{ (n + 1) } (t) (x - t)^{n } dt \\ &= f^{ (n + 1) } (c) \int_a^x \frac{1}{n!} (x - t)^{n } dt \\ &= \frac{f^{ (n + 1) } (c ) }{(n + 1) ! } (x - a)^{n + 1 } \end{aligned}

for some $c \in [a, x]$ .

Corollary 2. Let $f:\rightarrow \mathbb{R}^n \rightarrow \mathbb{R}$ be a function with continuous second partial derivatives, then given $a \in \mathbb{R}^n$ , it holds that

f(x) = f(a) + [\nabla f(c)]^T (x - a)

for some $c \in a + t(x - a)$ , where $t \in [0, 1]$ .

Proof: Define $g(t) = f(a + t (x- a)) : [0,1] \rightarrow \mathbb{R}$ . Then

$g(0) = f(a)$ .
$g(1) = f(x)$ .
$g'(t) = [\nabla f(a + t (x- a) ) ]^T (x - a)$
By mean value theorem, $\exists t \in [0, 1]$ , s.t.,

\frac{g(1) - g(0)}{1} = f(x) - f(a) = g'(t) = [\nabla f(a + t (x- a) ) ]^T (x - a)

$\square$

Corollary 3. Let $f$ , $g$ and $c$ as defined before, then

f(x) = f(a) + [\nabla f(a)]^T (x - a) + \frac{1}{2} (x - a)^T [\nabla^2 f(c)] (x - a)

Proof:

g''(t) = (x - a)^T [\nabla^2 f(a + t (x- a) ) ] (x - a)

Then applying Corollary 1 gives

g(1) = g(0) + g'(0)(1 - 0) + \frac{1}{2} g''(t) (1 - 0)^2

for some $t \in [0, 1]$ .

$\square$