Characteristic Function

Definition

Let $(\Omega, \mathcal{F}, \mathbb{P})$ be a probability space, and $X$ be a random variable on the space. Further, denote $\mu$ the distribution of $X$, s.t.,

where $\mathcal{B}$ is the collection of Borel set on $\mathbb{R}$.

Definition. The characteristic function r.w.t. $\mu$ is given by

$$\varphi(t) = \int_\mathbb{R} e^{itx} \, \mu (dx) .$$

Theorem.

$$\varphi(t) = \mathbb{E} \left[ e^{itX} \right].$$

Proof. Recall that, by definition of expectation, we have

where in Equation $(a)$, we have applied the fact that for each function $f$ that is measurable $\mathcal{B} / \mathcal{B}$,

$\square$

Independence

Theorem. Let $X$ and $Y$ be independent random variables. Then

$$\varphi(X + Y) = \varphi(X) \varphi(Y).$$

Proof. Since $X$ and $Y$ are independent, the following pairs of random variables are independent

1. $\sin \big( t X(\omega) \big)$, $\sin \big( t Y(\omega) \big)$;
2. $\cos \big( t X(\omega) \big)$, $\cos \big( t Y(\omega) \big)$;
3. $\sin \big( t X(\omega) \big)$, $\cos \big( t Y(\omega) \big)$;
4. $\cos \big( t X(\omega) \big)$, $\sin \big( t Y(\omega) \big)$.

$\square$

Uniqueness

Theorem. Let $\mu$ and $\nu$ be probability measures on $\mathbb{R}$. If $\mu$ and $\nu$ has the same characteristic functions, then

$$\begin{aligned} &\mu(A) = \nu(A), &\forall A \in \mathcal{B}. \end{aligned}$$

The proof follows from the following lemma:

Lemma. Let $a, b \in \mathbb{R}$, s.t. $a < b$. If $\mu(\{a\}) = \mu(\{b\}) = 0$, then

$$\mu \Big( (a, b] \Big) = \lim_{T \rightarrow \infty} \frac{1}{2 \pi} \int_{-T}^T \frac{e^{-ita} - e^{-itb}}{it} \varphi(t) \, d t.$$

Proof of the Theorem. For each $n \in \mathbb{N}_+$, define $E_n \doteq \{ x \in \mathbb{R} : \mu( \{ x \} ) \ge 1 / n \}$. It is easy to see that $|E_n| \le n$. It follows that the $E \doteq \cup_{n \in \mathbb{N}_+} E_n$ is countable.

Consider the set $S = \{ (a, b] : a < b, \mu(\{a\}) = \mu(\{b\}) = 0 \}$. Obviously this is a $\pi$ system. Further, it generates $\mathcal{B}$. To see this, for each point $c \in E$, we can find

1. a sequence $a_1, a_2, \ldots$ from $\bar{E}$ that increases to $c$,
2. a sequence $b_1, b_2, \ldots$ from $\bar{E}$ that decreases to $c$.

Then $(a_i, b_i] \in S$, and $\cap_{i \ge 1} (a_i, b_i] = [c, c]$.

It follows that every interval $(a, b] \subseteq \mathbb{R}$ can be generated by intervals in $S$.

According to the lemma, $\mu$ and $\nu$ agrees on a $\pi$ system that generates $\mathcal{B}$. By $\pi$-$\lambda$ theorem, they agree on each set in $\mathcal{B}$.

$\square$

Proof of the Lemma. We are going to prove that, for each $a < b$,

Define

where

• Equation $(a)$ holds: since $e^{-i x} = 1 + \int_0^x e^{-is} \, d s$,

  $$\begin{aligned} \left| \frac{e^{-it(a - x)} - e^{-it(b - x)}}{it} \right| = \left| \int_{(b - x)}^{(a - x)} e^{-is} \, d s \right| \le |b - a| \end{aligned}$$

  is therefore bounded. Hence, by Fubini's theorem, we can exchange the order of the double integral over finite product measure.

• Equation $(b)$ holds: since $\cos \big( (x - a) t \big) / (it)$ and $\cos \big( (x - b) t \big) / (it)$ are odd functions over $[-T, T]$

• Equation $(c)$ holds: since $\sin \big( (x - a) t \big) / (it)$ and $\sin \big( (x - b) t \big) / (it)$ are even functions over $[-T, T]$

Next, define $S(T) \doteq \int_0^T \tfrac{\sin x}{x} \, dx$. Introducing the function

we can write

Since ${T \rightarrow \infty}$, $S(T) \rightarrow \pi / 2$, we have $R(\theta, T) \rightarrow \mathsf{sgn}(\theta) \cdot \pi / 2$. Hence,

Finally, $|R(\theta, T)| = \Big| S \big( |\theta| T \big) \Big| \le \sup_t S(t) < \infty$, by bounded convergence theorem,

which concludes the proof.

Remark. Here, let $f_T(x) \doteq R(x - a, T) - R(x - b, T)$, and $f(x) \doteq \lim_{T \rightarrow \infty} f_T(x)$. Since $f_T(x)$ is bounded, assume that $|f_T(x)| \le B, \forall x, T$.

Let $\epsilon > 0$. For each $T \in \mathbb{R}^+$, define $E_T \doteq \{ x \in \mathbb{R} : | f_T(x) - f(x) | \le \epsilon \}$. Then

Finally, for $x \notin (a - \epsilon, a + \epsilon) \cup (b - \epsilon, b + \epsilon)$, we have $|x - a| T, |x - b| T \ge \epsilon T$. Hence,

which approaches $0$ as $T \rightarrow \infty$. So

$\square$

Reference

[1] P. Billingsley, Probability and measure Anniversary Edition, 3rd ed. New York: Wiley, 1995.

[2] R. Durrett, “Probability : theory and examples,” p. 432.