Multivariate Fubini's Theorem

Let $k \in \mathbb{N}^+$, $k \ge 2$, and $(\Omega_{i}, \mathcal{F}_i, \mu_i), i \in [k]$ be $\sigma$-finite measure spaces.

Product Measure Space

Consider the product space $\Omega = \times_{i \in [k]} \Omega_{i}$.

Definition. A measurable rectangle is a product $\times_{i \in [k]} A_i$ for which $A_i \in \mathcal{F}_i, i \in [k]$.

Denote the collection of measurable rectangles as

Let $\times_{i \in [k]} \mathcal{F}_i \doteq \sigma(\mathcal{S})$ be the $\sigma$-field generated by $\mathcal{S}$. Following similar vein, for each $I \subseteq [k]$, we can define $\times_{i \in I} \mathcal{F}_i$ to be the $\sigma$-field generated by measurable rectangles $\{ \times_{i \in I} A_i, A_i \in \mathcal{F}_i, \forall i \in I \}$.

We would like to construct a measure $\mu$ on $\sigma(\mathcal{S})$, thus

1. For each $\times_{i \in [k]} A_i \in \mathcal{S}$, it's natural to define $\mu (\times_{i \in [k]} A_i) = \prod_{i \in [k]} \mu_i(A_i)$.

2. We prove that $\mathcal{S}$ is a semialgebra.

3. We show that $\mu$ is a measure on $\mathcal{S}$.

4. Finally, it follows by the Caratheodory’s Extension Theorem, $\mu$ has a unique extension on $\sigma(\mathcal{S})$.

Lemma 1. $\mathcal{S}$ is a semialgebra.

Proof.

• $\mathcal{S}$ is close under intersection: if $\times_{i \in [k]} A_i, \times_{i \in [k]} B_i \in \mathcal{S}$, then$$(\times_{i \in [k]} A_i) \cap (\times_{i \in [k]} B_i) = \times_{i \in [k]} (A_i \cap B_i).$$
• The complement of element in $\mathcal{S}$ is a finite disjoint union of elements in $\mathcal{S}$: if $\times_{i \in [k]} A_i \in \mathcal{S}$, since$$\Omega = \times_{i \in [k]} \big( A_i +\bar{A_i} \big),$$it holds that$$\overline{ \times_{i \in [k]} A_i } = \sum \big( \times_{i \in [k]} B_i \big),$$where $B_i = A_i$ or $B_i = \bar{A_i}$, and $\times_{i \in [k]} B_i \neq \sum_{i \in [k]} A_i$.

$\square$

Theorem. There is a unique extension $\mu$, that is a measure on $\sigma( \mathcal{S} )$.

Proof. We prove the theorem by induction. For each $\ell \in [k]$, define

and

Our goal is to show that $\mu_{1:\ell}$ is a $\sigma$-finite measure on $\mathcal{S}_{1:\ell}$. Note that $\mathcal{S}_{1:\ell}$ is a semialgebra and $\mu$ is a $\sigma_{1:\ell}$-finite measure. Applying Caratheodory’s Extension Theorem, we see that there is a unique extension $\mu_{1:\ell}$ of $\mu_{1:\ell}$, that is a measure on $\sigma( \mathcal{S}_{1:\ell} )$.

The claim clearly holds for $\ell = 1$.

Next, we prove that if the claim holds for $\ell < k$, then it holds for $\ell + 1$. The $\sigma$-finiteness follows directly from the finiteness of $\mu_1, \ldots, \mu_{\ell + 1}$. It suffices to prove $\sigma$-additivity of $\mu_{1 : \ell + 1}$.

Suppose $\times_{i \in [\ell + 1]} A_i = +_{m \ge 1} (\times_{i \in [\ell + 1]} B_{m, i})$ is a countable disjoint union of elements in $\mathcal{S}_{1: \ell + 1}$.

Consider an $x_{1 : \ell} \in \times_{i \in [\ell]} A_i$. For each $x_{\ell + 1} \in A_{\ell + 1}$, we have $(x_1, \ldots, x_{\ell + 1}) \in \times_{i \in [\ell + 1]} A_i$. As the sets $\begin{Bmatrix} \times_{i \in [\ell + 1]} B_{m, i} \end{Bmatrix}$ consists of a partition of $\times_{i \in [\ell + 1]} A_i$, it follows that $(x_1, \ldots, x_{\ell + 1})$ belongs to exactly one $\times_{i \in [\ell + 1]} B_{m, i}$. Denote

Then $A_{\ell + 1} = +_{m \in I(x)} B_{m, \ell + 1}$.

As $\mu_{\ell + 1}$ is a measure on $(\Omega_{\ell + 1}, \mathcal{F}_{\ell + 1})$, it is countably additive:

By monotone convergence theorem,

which is

$\square$

An Interesting Property

Define the operation ordered product $\vec{\times}$ on two measurable set $A \in \left( \times_{i \in I} \mathcal{F}_i \right)$ and $B \in \left( \times_{i \in \overline{I}} \mathcal{F}_i \right)$ as

and let $\left( \times_{i \in I} \mathcal{F}_i \right) \vec{\times} \left( \times_{i \in \overline{I}} \mathcal{F}_i \right)$ to be the $\sigma$-field generated by measurable rectangles

Theorem. Let $I \subseteq [k]$, and consider

$$\left( \times_{i \in I} \mathcal{F}_i \right) \vec{\times} \left( \times_{i \in \overline{I}} \mathcal{F}_i \right) = \times_{i \in [k]} \mathcal{F}_i$$

Proof. Since $\mathcal{S} \subseteq \mathcal{S}_{I, \overline{I}}$, clearly it holds that

On the other hand, for each $A \vec{\times} B \in \mathcal{S}_{I, \overline{I}}$, we claim that $A \vec{\times} B \in \times_{i \in [k]} \mathcal{F}_i$. Hence $\times_{i \in [k]} \mathcal{F}_i$ is a $\sigma$-field containing $\mathcal{S}_{I, \overline{I}}$. Via the definition of $\left( \times_{i \in I} \mathcal{F}_i \right) \vec{\times} \left( \times_{i \in \overline{I}} \mathcal{F}_i \right)$, we conclude that

To prove the claim, we will show $A \vec{\times} \left( \times_{i \in \overline{I} } \Omega_{i} \right) \in \times_{i \in [k]} \mathcal{F}_i$ and $\left( \times_{i \in I } \Omega_{i} \right) \vec{\times} B \in \times_{i \in [k]} \mathcal{F}_i$. By symmetry, we need only to prove the former.

Consider

Then $\mathcal{S}_I \in \mathcal{L}_{\times_{i \in I} \Omega_{i}}$. Further,

1. $\emptyset \in \mathcal{L}_{\times_{i \in I} \Omega_{i}}$;

2. If $A_I \in \mathcal{L}_{\times_{i \in I} \Omega_{i}}$, then $\bar A_I \in \mathcal{L}_{\times_{i \in I} \Omega_{i}}$, as $\bar A_I \vec{\times} \left( \times_{i \in \overline{I} } \Omega_{i} \right) \in \times_{i \in [k]} \mathcal{F}_i$

3. If $A_I^{(n)} \in \mathcal{L}_{\times_{i \in I} \Omega_{i}}$ for $n \in \mathbb{N}^+$, then $\cup_{n \ge 1} A_I^{(n)} \in \mathcal{L}_{\times_{i \in I} \Omega_{i}}$, as

   $$\left( \cup_{n \ge 1} A_I^{(n)} \right) \vec{\times} \left( \times_{i \in \overline{I} } \Omega_{i} \right) = \cup_{n \ge 1} \left( A_I^{(n)} \vec{\times} \left( \times_{i \in \overline{I} } \Omega_{i} \right) \right) \in \times_{i \in [k]} \mathcal{F}_i$$

It follows that $\mathcal{L}_{\times_{i \in I} \Omega_{i}}$ is a $\sigma$-field, and therefore $\mathcal{L}_{\times_{i \in I} \Omega_{i}} \supseteq \times_{i \in I} \mathcal{F}_i$. It concludes that $A \vec{\times} \left( \times_{i \in \overline{I} } \Omega_{i} \right) \in \times_{i \in [k]} \mathcal{F}_i$.

$\square$

Fubini's Theorem

Integration is a generalization of discrete sum. Given a measure space $(\Omega, \sigma( \mathcal{S} ), \bar \mu)$, and a measure function $f$, the integral of $f$ exists, if

1. $f \ge 0$;
2. Or $\int |f| d\, \bar \mu < \infty$.

In case that $(\Omega, \sigma( \mathcal{S} ), \bar \mu)$ is a produce measure space, the Fubini's theorem states that if the integral exists, it can be computed by iterated integrals.

Theorem (Fubini's Theorem). If $f \ge 0$ or $\int |f| d\, \bar \mu < \infty$, then there exists $A_i \in \mathcal{F}_i, i \in [k]$, s.t., $\mu_i (A_i) = 0$, and for each permutation $\pi$ of $[k]$,

$$\int_{\Omega } f(x_1, \ldots, x_k) d\, \bar \mu = \int_{\Omega_{\pi(1)} - A_{\pi(1)}} \ldots \int_{\Omega_{\pi(k - 1)} - A_{\pi(k - 1)}} \int_{\Omega_{\pi(k)}} f(x_1, \ldots, x_k) d\, \mu_{\pi(k)} \, \ldots d\, \mu_{\pi(1)}.$$

Remark. For the case concerning $f \ge 0$, we have $A_i = \emptyset, i \in [k]$, and the Theorem is called Tonelli’s theorem.

Proof of Fubini's Theorem

By symmetry, it suffices to prove that

Sketch of the Proof

Before proving that the iterated integral equals the double integral, we need to make sure that the first iterated integral is well-defined. Specifically,

• When $x_{1 : k - 1}$ is fixed, the function $f_{x_{1:k - 1}} : \Omega_{k} \rightarrow \mathbb{R}$ defined by $f_{x_{1:k - 1}}(x_k) \doteq f(x_1, \ldots, x_k)$ is $\mathcal{F}_k$ measurable, and the integral $\int_{\Omega_{k}} f(x_1, \ldots, x_k) \, d\, \mu_k$ exists.

• For each $\ell \in [k - 1]$, there is $\mathcal{F}_\ell$ measurable function $g_\ell : \Omega_{\ell} \rightarrow \mathbb{R}$ such that

We will first establish the theorem for indicator functions $\mathbb{I}_E$ for sets $E \in \sigma( \mathcal{S} )$, which serves as a basic building blocks for extending the theorem for more general functions.

Lemma. Given $E \in \sigma( \mathcal{S} )$, let $\mathbb{I}_E$ be the indicator for $(x_1, \ldots, x_k) \in E$. Then

1. For each $x_k \in \Omega_{k}$, the function $f_{x_{1:k - 1}}(x_k) \doteq \mathbb{I}_E (x_1, \ldots, x_k)$ is $\mathcal{F}_k$ measurable.

2. The function $g_\ell (x_\ell) \doteq \int_{\Omega_{\ell + 1}} \ldots \int_{\Omega_{k - 1}} \int_{\Omega_{k}} \mathbb{I}_E (x_1, \ldots, x_k) \, d\, \mu_k \, \ldots \, d\, \mu_{\ell + 1}$ is $\mathcal{F}_{\ell}$ measurable.

3. $$\int_{\Omega } \mathbb{I}_E (x_1, \ldots, x_k) \, d\, \bar \mu = \int_{\Omega_{1}} \ldots \int_{\Omega_{k - 1}} \int_{\Omega_{k}} \mathbb{I}_E (x_1, \ldots, x_k) \, d\, \mu_{k} \, \ldots d\, \mu_{1}.$$

The proof of the lemma is relegated to the end of the article.

Proof of the Theorem

For now, we apply it to prove Theorem for the following three increasingly more general cases.

Case 1. $f$ is a simple function on $(\Omega, \sigma(\mathcal{S}))$.

Case 2. $f \ge 0$ and is $\sigma(\mathcal{S})$ measurable.

Case 3. $f$ is measurable and integrable.

Remark: this follows exactly the same order with which we define Lebesgue integrals.

Case 1. Suppose that $f(x_1, \ldots, x_k) = \sum_{i \in [n]} c_i \cdot \mathbb{I}_{E_i} (x_1, \ldots, x_k)$ is a simple function on $(\Omega, \sigma(\mathcal{S}))$.

1. For fixed $x_{1 : k - 1} \in \Omega_{1 : k - 1}$, according the lemma, the functions $f_{x_{1 : k - 1}}^{ (i) } (x_k) \doteq \mathbb{I}_{E_i} (x_1, \ldots, x_k), i \in [n]$ are $\mathcal{F}_k$ measurable. It follows that function $f_{x_{1 : k - 1}} (x_k) = \sum_{i \in [n]} c_i \cdot f_x^{ (i) } (x_k)$ is $\mathcal{F}_k$ measurable.

2. For each $\ell \in [k - 1]$, since $g_\ell^{(i)} (x_\ell) \doteq \int_{\Omega_{\ell + 1}} \ldots \int_{\Omega_{k}} \mathbb{I}_{E_i} (x_1, \ldots, x_k) d\, \mu_k \ldots d\, \mu_\ell$ is $\mathcal{F}_\ell$ measurable, so is the function

3. Since $\int_{\Omega } \mathbb{I}_{E_i} (x_1, \ldots, x_k) d\, \bar \mu = \int_{\Omega_{1}} \ldots \int_{\Omega_{k - 1}} \int_{\Omega_{k}} \mathbb{I}_{E_i} (x_1, \ldots, x_k) d\, \mu_{k} \, \ldots d\, \mu_{1}$,

Case 2. Suppose that $f \ge 0$ and is $\sigma(\mathcal{S})$ measurable. Then there exists simple, nonnegative functions $f^{ (n) } \uparrow f$.

1. For fixed $x_k \in \Omega_K$. The functions $f_{x_{1:k - 1}}^{ (n) } (x_k) \doteq f^{ (n) } (x_1, \ldots, x_k) \uparrow f_{x_{1:k - 1}}(x_k) \doteq f(x_1, \ldots, x_k)$. Since they are $\mathcal{F}_k$ measurable, so is $f_{x_{1:k - 1}} (x_k)$.

2. For each $\ell \in [k - 1]$, let $g_\ell^{(n)} (x_\ell) \doteq \int_{\Omega_{\ell + 1}} \ldots \int_{\Omega_{k}} f^{ (n) } (x_1, \ldots, x_k) d\, \mu_k \ldots d\, \mu_\ell$. We prove by induction that $0 \le g_{\ell + 1}^{(n)} (x_{\ell + 1}) \uparrow g_{\ell + 1} (x_{\ell + 1})$. The claim is true when $\ell + 1 = k$. Next, if this is true for $\ell + 1$, we show that it is true for $\ell$. By monotone convergence theorem:

   $$\begin{aligned} \lim_n g_\ell^{(i)} (x_\ell) &= \lim_n \int_{\Omega_{\ell + 1}} g_{\ell + 1}^{(n)} (x_{\ell + 1}) d\, \mu_\ell \\ &= \int_{\Omega_{\ell + 1}} g_{\ell + 1} (x_{\ell + 1}) d\, \mu_\ell = g_\ell (x_\ell). \end{aligned}$$

   Hence $g_\ell^{(n)} (x_\ell) \uparrow g_\ell (x_\ell)$. Since they are $\mathcal{F}_\ell$ measurable, so is $g_\ell (x_\ell)$.

3. Since $g_1^{(n)} (x_1) \uparrow g_1 (x_1) \,$, $\int_{\Omega_1} g_1^{(n)} (x_1) d\, \mu_1 = \int_{\Omega } f^{ (n) } (x_1, \ldots, x_k) d\, \bar \mu \,$, and $f^{(n)} (x_1, \ldots, x_k) \uparrow f(x_1, \ldots, x_k)$, by monotone convergence theorem,

Case 3. Suppose that $f(x_1, \ldots, x_k)$ is a measurable and integrable function on $\Omega$, i.e., $\int_{\Omega } |f(x_1, \ldots, x_k)| d\, \bar \mu < \infty$. Write $f(x_1, \ldots, x_k) = f^+(x_1, \ldots, x_k) - f^-(x_1, \ldots, x_k)$, such that $f^+(x_1, \ldots, x_k), f^-(x_1, \ldots, x_k) \ge 0$ are measurable and integrable.

1. For a fixed $x \in \Omega_{k}$, since both $f^+_{x_{1:k - 1}}(x_k) \doteq f^+(x_1, \ldots, x_k),\, f^-_{x_{1:k - 1}}(x_k) \doteq f^-(x_1, \ldots, x_k)$ are $\mathcal{F}_k$ measurable, so is the function $f_{x_{1:k - 1}} (x_k) \doteq f(x_1, \ldots, x_k) = f^+(x_1, \ldots, x_k) - f^-(x_1, \ldots, x_k)$ is $\mathcal{F}_k$.

2. Since $f^+(x_1, \ldots, x_k), f^-(x_1, \ldots, x_k) \ge 0$ are measurable and integrable, we have

   $$\begin{aligned} \int_{\Omega_{1}} \ldots \int_{\Omega_{k - 1}} \int_{\Omega_{k}} f^+(x_1, \ldots, x_k) d\, \mu_{k} \, \ldots d\, \mu_{1} = \int_{\Omega } f^+(x_1, \ldots, x_k) d\, \bar \mu < \infty, \\ \int_{\Omega_{1}} \ldots \int_{\Omega_{k - 1}} \int_{\Omega_{k}} f^-(x_1, \ldots, x_k) d\, \mu_{k} \, \ldots d\, \mu_{1} = \int_{\Omega } f^-(x_1, \ldots, x_k) d\, \bar \mu < \infty. \end{aligned}$$

   It follows that $\exists A_\ell \subseteq \mathcal{F}_\ell$, s.t., $\mu_\ell (A_\ell) = 0$ for each $\ell \in [k - 1]$, and $g_\ell^{+} (x_\ell) \doteq \int_{\Omega_{\ell + 1} - A_\ell} \ldots \int_{\Omega_{k - 1} - A_{k - 1}} \int_{\Omega_{k}} f^{+} (x_1, \ldots, x_k) d\, \mu_k \ldots d\, \mu_\ell$ and $g_\ell^{-} (x_\ell) \doteq \int_{\Omega_{\ell + 1} - A_\ell} \ldots \int_{\Omega_{k - 1} - A_{k - 1}} \int_{\Omega_{k}} f^{-} (x_1, \ldots, x_k) d\, \mu_k \ldots d\, \mu_\ell$ are finite on $\Omega_{\ell} \setminus A_\ell$. Define

   $$g_\ell (x_\ell) = \begin{cases} g_\ell^{+} (x_\ell) - g_\ell^{-} (x_\ell), & \text{if } x_\ell \in \Omega_{\ell} \setminus A_\ell \\ 0, & \text{if } x_\ell \in A_\ell \end{cases}.$$

   Then $g_\ell (x_\ell)$ is $\mathcal{F}_\ell$ measurable.

3. We have

$\square$

Proof of the Lemma

Notations

For each $x = (x_1, \ldots, x_k) \in \Omega$ and $\ell \in [0 \,. \,. k]$, define

In general, we refer $x_{1:0}$ to $\varnothing$, or $x_{1:\ell}$ ($\ell \in [k]$) to a sequence in $\times_{i = 1}^\ell \Omega_i$.

Given $E \in \sigma(\mathcal{S})$, for each $x_{1 : \ell}$ ($\ell \in [k - 1]$), the cross-section of $E$ at $x$ is given by

Further, let

and $\mu_{\ell : k}$ be the unique measure defined on $( \times_{i = \ell}^k \Omega_i, \times_{i = \ell}^k \mathcal{F}_i)$, s.t.,

We restate the lemma, with slightly different notations.

Lemma. Given $E \in \sigma( \mathcal{S} )$, let $\mathbb{I}_E$ be the indicator for $(x_1, \ldots, x_k) \in E$. Then

1. For each $x_{1:\ell - 1}$, $\ell \in [k]$, the function

   $$g_{E, x_{1:\ell - 1}} (y_\ell) \doteq \begin{cases} \mathbb{I}_E (x_1, \ldots, x_{k - 1}, y_k), &\text{ if } \ell = k - 1, \\ \int_{\Omega_{\ell + 1}} \ldots \int_{\Omega_{k - 1}} \int_{\Omega_{k}} \mathbb{I}_E (x_1, \ldots, x_{\ell - 1}, y_\ell, z_{\ell + 1}, \ldots, z_k ) \, d\, \mu_k \, \ldots \, d\, \mu_{\ell + 1}, &\text{ otherwise }, \end{cases}$$

   is $\mathcal{F}_{\ell}$ measurable.

2. For each $x_{1:\ell - 1}$, $\ell \in [k]$, it holds that

   $$\begin{align} \mu_{\ell : k} (E_{x_{1:\ell - 1}}) = \int_{\Omega_{\ell}} \ldots \int_{\Omega_{k}} \mathbb{I}_E (x_1, \ldots, x_{\ell - 1}, y_\ell, \ldots, y_k) \, d\, \mu_{k} \, \ldots d\, \mu_{\ell} \end{align}$$

Proving that

• $g_{E, x_{1:k - 1}}$ is $\mathcal{F}_k$ measurable and

• $\mu_{k : k} (E_{x_{1:k - 1}}) = \int_{\Omega_{k}} \mathbb{I}_E (x_1, \ldots, x_{k - 1}, y_k) \, d\, \mu_{k}.$

It suffices to show that

Claim. For each $x_{1: k - 1} \in \Omega_{1 : k - 1}$ and each $E \in \sigma(\mathcal{S})$, $E_{x_{1 : k - 1}} \in \mathcal{F}_k$.

Proof of the Claim. Fix an $x_{1:k - 1} \in \Omega_{1 : k - 1}$. Define the function $T_{x_{1:k - 1}} : \Omega_{k} \rightarrow \times_{i \in [k]} \Omega_{i}$ by $T_{x_{1:k - 1}} (x_k) \doteq (x_1, \ldots, x_k)$. Then for each $E = \times_{i \in [k]} A_i \in \mathcal{S}$,

Therefore $T_{x_{1:k - 1}}^{-1} (E) = E_{x_{1:k - 1}} \in \mathcal{F}_k$. Since $\mathcal{S}$ generates $\sigma(\mathcal{S})$, we conclude that $T_{x_{1 : k - 1}}$ is measurable $\mathcal{F}_k / \sigma(\mathcal{S})$.