Measure Space

Let Ω\Omega be a set and 2Ω2^\Omega be the power set of Ω\Omega.

σ\sigma-Algebra

Definition. A collection of subsets F⊆2Ω\mathcal{F} \subseteq 2^\Omega is a σ\sigma-algebra (or σ\sigma-field) if

  • It is closed under countable union, intersection and complement.

Measure

Definition. Given a collection of subsets F⊆2Ω\mathcal{F} \subseteq 2^\Omega, a function μ:F→R\mu : \mathcal{F} \rightarrow \mathbb{R} is a measure if it satisfies

  1. It is non-negative: ∀A∈F,μ(A)≥0\forall A \in \mathcal{F}, \mu(A) \ge 0.

  2. The empty set is mapped to 0: μ(∅)=0\mu(\emptyset) = 0.

  3. σ\sigma-additivity: if Ai∈FA_i \in \mathcal{F} is a countable sequence of disjoint sets, s.t., ∪i≥1Ai∈F\cup_{i \ge 1} A_i \in \mathcal{F}, then

μ(∪i≥1Ai)=∑i≥1μ(Ai). \mu(\cup_{i \ge 1} A_i) = \sum_{i \ge 1} \mu(A_i ).

Measure Space

Definition. A measure space (Ω,F,μ)(\Omega, \mathcal{F}, \mu) consists of three part:

  1. A set Ω\Omega, on which it is defined;

  2. A σ\sigma-field F\mathcal{F} that contains all measurable sets;

  3. A measure μ\mu defined on F\mathcal{F}.

In general, it is not easy to construct a measure space on Ω\Omega directly. Our indirect construction consists of three steps:

  1. Consider a simple set class SS and a measure μ\mu defined on SS.
  2. We extend the domain of μ\mu from SS to a larger one Ω=2Ω\Omega=2^\Omega.
  3. We choose a subset from 2Ω2^\Omega, such that it is a σ\sigma-field and μ∗\mu^* is a measure defined on it.

In what follows, we illustrate the idea via constructing measures on Rd\mathbb{R}^d.

Semialgebra

Definition. A class of subsets S⊆2ΩS \subseteq 2^\Omega is a semialgebra if

  1. It is closed under intersection: ∀A,B∈S\forall A, B \in S, A∩B∈SA \cap B \in S.

  2. If A∈SA \in S, then Aˉ\bar A is a finite disjoint union of sets in SS.

Examples. For d∈Z+d \in \mathbb{Z}^+, the collection of half-open intervals in Rd\mathbb{R}^d plus the empty set

Sd≐{×i∈[d](ai,bi]:−∞≤ai<bi≤∞,∀i∈[d]}∪{∅} S_d \doteq \{ \times_{i \in [d] } (a_i, b_i] : -\infty \le a_i \lt b_i \le \infty, \forall i \in [d] \} \cup \{ \emptyset \}

Algebra

Definition. A class of subsets F⊆2Ω\mathcal{F} \subseteq 2^\Omega is an algebra (or field) if

  • It is closed under union, intersection and complement.

Remark. By De Morgan's laws, A∩B‾=Aˉ∪Bˉ\overline{A \cap B } = \bar A \cup \bar B and A∪B‾=Aˉ∩Bˉ\overline{A \cup B} = \bar A \cap \bar B. If we know a collection is closed under intersection (union) and complement, so is union (intersection).

Theorem If S⊆ΩS \subseteq \Omega is a semialgebra, then

S~≐{finite disjoint union of sets in S} \widetilde{S} \doteq \{ \text{finite disjoint union of sets in } S \}

is an algebra.

Proof.

  1. S~\widetilde{S} is closed under intersection: Let A,B∈S~A, B \in \widetilde{S} with A=+iAiA = +_i A_i and B=+jBjB = +_j B_j, where ++ denotes disjoint union and Ai,Bj∈SA_i, B_j \in S. Then A∩B=+i,jAi∩BjA \cap B = +_{i, j} A_i \cap B_j. As SS is a semialgebra, it follows that Ai∩Bj∈SA_i \cap B_j \in S and A∩B∈S~A \cap B \in \widetilde{S}.

  2. S~\widetilde{S} is closed under complement. Let A∈S~A \in \widetilde{S} with A=+iAiA = +_i A_i, where Ai∈SA_i \in S. Then Aˉ=∩iAˉi\bar A = \cap_i \bar A_i. As SS is a semialgebra, Aˉi\bar A_i is a finite disjoint union of sets in SS, which implies that Aˉi∈S~\bar A_i \in \widetilde{S}. Therefore Aˉ\bar A is a finite intersection of sets in S~\widetilde{S}. Since S~\widetilde{S} is closed under intersection, Aˉ∈S~\bar A \in \widetilde{S}.

□\square

Measure Extension: Semialgebra →\rightarrow Algebra

Theorem. Let SS be a semialgebra and μ\mu be a measure on SS. Then μ\mu has a unique extension μˉ\bar \mu that is a measure on S~\widetilde S, the algebra generated by SS.

Remark. The σ\sigma-additivity condition of μ\mu being a measure can be replaced by

  1. Finite additivity. If A,Ai∈SA, A_i \in S with A=+iAiA = +_i A_i, then μ(A)=∑iμ(Ai)\mu(A) = \sum_i \mu (A_i).
  2. Subadditivity. If A,Ai∈SA, A_i \in S with A=+i≥1AiA = +_{i \ge 1} A_i, then μ(A)≤∑i≥1μ(Ai)\mu (A) \le \sum_{i \ge 1} \mu (A_i).

The subscript i≥1i \ge 1 in the conditions indicates a countable union, while a plain subscript ii or jj indicates a finite union.

The relation between the condition stated in the Theorem and the relaxed ones is established as follows.

Exercise. A finitely additive nonnegative set function on an algebra S~\widetilde{S} is monotone and the following conditions are equivalent

  1. σ\sigma-additivity.
  2. Subadditivity.

The proof for the Theorem consists of two steps.

▶\blacktriangleright Defining μˉ\bar \mu

If μ\mu is finitely additive in SS, then it has a finitely additive extension μˉ\bar \mu in S~\widetilde{S}.

  1. Extension. If A=+iAi∈S~A = +_i A_i \in \widetilde{S} with Ai∈SA_i \in S, define

    μˉ(A)=∑iμ(Ai). \bar \mu(A) = \sum_i \mu (A_i).

    We need to verify the value of μˉ(A)\bar \mu(A) is unique. Suppose AA has another decomposition: A=+jBjA = +_j B_j with Bj∈SB_j \in S. Since SS is a semialgebra, Ai∩Bj∈SA_i \cap B_j \in S. Using that μ\mu is a measure

    ∑iμ(Ai)=∑i,jμ(Ai∩Bj)=∑jμ(Bj). \sum_i \mu (A_i) = \sum_{i, j} \mu(A_i \cap B_j) = \sum_j \mu(B_j).

    It follows that μˉ(A)\bar \mu(A) does not change with the decomposition of AA and hence is well-defined.

  2. Finite additivity. Let A=+iAi∈S~A = +_i A_i \in \widetilde{S} with Ai∈S~A_i \in \widetilde{S} and suppose that Ai=+jBi,jA_i = +_j B_{i,j} with Bj∈SB_j \in S. Then A=+i,jBi,jA = +_{i,j} B_{i,j}. By definition of μˉ(A)\bar \mu (A) and μˉ(Ai)\bar \mu (A_i),

μˉ(A)=∑i,jμ(Bi,j)=∑iμˉ(Ai). \bar \mu (A) = \sum_{i,j} \mu (B_{i,j}) = \sum_{i} \bar \mu ( A_i ).

We list two more properties about μˉ\bar \mu induced by finite additivity.

  • Monotonicity. If A,B∈S~A, B \in \widetilde{S} with A⊆BA \subseteq B, then μˉ(A)≤μˉ(B)\bar \mu (A) \le \bar \mu (B). As S~\widetilde{S} is an algebra, B−A∈S~B - A \in \widetilde{S} and by finite additivity of μˉ\bar \mu,

    μˉ(A)≤μˉ(A)+μˉ(B−A)=μˉ(B). \bar \mu (A) \le \bar \mu (A) + \bar \mu (B - A) = \bar \mu (B).
  • Finite subadditivity. If A⊆∪i∈[n]AiA \subseteq \cup_{i \in [n] } A_i with A∈S~A \in \widetilde{S}, Ai∈S~,∀i∈[n]A_i \in \widetilde{S}, \forall i \in [n], then μˉ(A)≤∑iμˉ(Ai)\bar \mu(A) \le \sum_i \bar \mu (A_i). Let B1=A1B_1 = A_1 and Bi=Ai−Ai−1−…−A1B_i = A_i - A_{i - 1} - \ldots - A_1 for i≥1i \ge 1. As S~\widetilde{S} is an algebra, Bi∈S~B_i \in \widetilde{S}. By monotonicity and finite additivity of μˉ\bar \mu,

    μˉ(A)≤μˉ(+iBi)=∑iμˉ(Bi)≤∑iμˉ(Ai). \bar \mu (A) \le \bar \mu( +_i B_i ) = \sum_i \bar \mu (B_i) \le \sum_i \bar \mu (A_i).

□\square

▶\blacktriangleright σ\sigma-Additivity of μˉ\bar \mu

If A,A1,…∈S~A, A_1, \ldots \in \widetilde S with A=+i≥1AiA = +_{i \ge 1} A_i, then

μˉ(A)=∑i≥1μˉ(Ai).\small \bar \mu (A) = \sum_{i \ge 1} \bar \mu (A_i).

Proof. Let Ai=+jBi,jA_i = +_j B_{i,j} with Bi,j∈SB_{i,j} \in S. Then A=+i,jBi,jA = +_{i,j} B_{i,j} and ∑i≥1μˉ(Ai)=∑i≥1∑jμ(Bi,j)\sum_{i \ge 1} \bar \mu(A_i) = \sum_{i \ge 1} \sum_j \mu ( B_{i,j} ). WLOG, we assume that AiA_i are countable union of disjoint sets in SS.

Next, assume that A=+kCkA = +_k C_k with Ck∈SC_k \in S. Using that SS is closed under intersection, and that μ\mu is a measure on SS (therefore σ\sigma-additivity holds),

μ(Ck)=∑i≥1μ(Ck∩Ai). \mu(C_k) = \sum_{i \ge 1} \mu (C_k \cap A_i).

Observing that nonnegative numbers can be summed in any order,

μˉ(A)=∑kμ(Ck)=∑k∑i≥1μ(Ck∩Ai)=∑i≥1∑kμ(Ck∩Ai)=∑i≥1μ(Ai). \small \begin{aligned} \bar \mu(A) = \sum_k \mu (C_k) = \sum_k \sum_{i \ge 1} \mu (C_k \cap A_i) = \sum_{i \ge 1} \sum_k \mu (C_k \cap A_i) = \sum_{i \ge 1} \mu (A_i). \end{aligned}

□\square

Measure Extension: Algebra →\rightarrow σ\sigma-Algebra

Definition. μˉ\bar \mu is said to be σ\sigma-finite if there is a sequence of sets Ai∈S~A_i \in \widetilde{S} such that μ(Ai)<∞\mu(A_i) < \infty and ∪i≥1Ai=Ω\cup_{i \ge 1} A_i = \Omega.

Caratheodory’s Extension Theorem. Let μˉ\bar \mu be a σ\sigma-finite measure on an algebra S~\widetilde S. Then μˉ\bar \mu has a unique extension μ∗\mu^* (that is a measure) to σ(S~)\sigma(\widetilde S), the smallest σ\sigma-field containing S~\widetilde S.

Remark: Observe that σ(S~)=σ(S)\sigma(\widetilde S) = \sigma(S).

The proof is included at the end of the article.

Lebesgue-Stieltjes Measure

In this section, we construct concrete measures on Rd\mathbb{R}^d.

Definition. A function F:R→RF: \mathbb{R} \rightarrow \mathbb{R} is a Stieltjes measure function if

  1. FF is non-decreasing.

  2. FF is right continuous, i.e., lim⁡y↓xF(y)=F(x)\lim_{y \downarrow x} F(y) = F(x).

Definition (Borel Sets). Let Rd=σ(Sd)\mathcal{R}^d = \sigma (S_d). It is the smallest σ\sigma-field containing all open sets in Rd\mathbb{R}^d.

Theorem 3. Associated with each Stieltjes measure function FF there is a unique measure μ\mu on (R,R1)(\mathbb{R},\mathcal{R}^1) with

μ((a,b])=F(b)−F(a). \mu \big( (a, b] \big) = F(b) - F(a).

When F(x)=xF(x) = x, the measure μ\mu is called Lebesgue measure.

Proof. We just to verity that μ\mu is a measure defined on the following semi-algebra:

S1≐{(a,b]:−∞≤a<b≤∞}∪{∅}. S_1 \doteq \{ (a, b] : -\infty \le a \lt b \le \infty \} \cup \{ \emptyset \}.
  1. μ((a,b])≥0\mu((a, b]) \ge 0, since FF is non-decreasing.

  2. Finite additivity. If (a,b]=+i(ai,bi](a, b] = +_i (a_i, b_i], μ((a,b])=∑iμ((ai,bi])\mu \big( (a, b] \big) = \sum_i \mu \big( ( a_i, b_i ] \big). We can rearrange the {(ai,bi]}\{ (a_i, b_i] \}, such that a=a1a = a_1, bi=ai+1b_i = a_{i + 1} for all 1≤i<n1 \le i < n, and bn=bb_n = b. Hence,

    μ((a,b])=F(b)−F(a)=∑i(F(bi)−F(ai))=∑iμ((ai,bi]). \mu \big( (a, b] \big) = F(b) - F(a) = \sum_i ( F(b_i) - F(a_i) ) = \sum_i \mu \big( ( a_i, b_i ] \big).
  3. Finite subadditivity. If (a,b]⊆∪i∈[n](ai,bi](a, b] \subseteq \cup_{i \in [n] } (a_i, b_i], μ((a,b])≤∑i∈[n]μ((ai,bi])\mu \big( (a, b] \big) \le \sum_{i \in [ n ] } \mu \big( ( a_i, b_i ] \big). Recall that the definition of μˉ\bar \mu only requires finite additivity of μ\mu. Working with S1~\widetilde{S_1} and μˉ\bar \mu significantly simplifies the proof. As for all A∈S1A \in S_1, it holds that μ(A)=μˉ(A)\mu(A) = \bar \mu(A). It suffices to prove that

    μˉ((a,b])≤∑i∈[n]μˉ((ai,bi]). \bar \mu \big( (a, b] \big) \le \sum_{i \in [ n ] } \bar \mu \big( (a_i, b_i] \big).

    But this follows exactly from finite subadditivity of μˉ\bar \mu.

  4. σ\sigma-additivity. If (a,b]=+i≥1(ai,bi](a, b] = +_{i \ge 1} (a_i, b_i], then μ((a,b])=∑i≥1μ((ai,bi])\mu \big( (a, b] \big) = \sum_{i \ge 1} \mu \big( ( a_i, b_i ] \big). Using finite subadditivity of μ\mu, for all n∈Z+n \in \mathbb{Z}^+,

    μ((a,b])≥∑i∈[n]μ((ai,bi]). \mu \big( (a, b] \big) \ge \sum_{i \in [n] } \mu \big( ( a_i, b_i ] \big).

    Letting n→∞n \rightarrow \infty, we get μ((a,b])≥∑i≥1μ((ai,bi])\mu \big( (a, b] \big) \ge \sum_{i \ge 1} \mu \big( ( a_i, b_i ] \big).

    It is left to prove the reverse direction of the inequality (σ\sigma-subadditivity). If μ((ai,bi])=∞\mu \big( ( a_i, b_i ] \big) = \infty for some ii, then the inequality is trivial. WLOG, assume that −∞<ai<bi<∞-\infty < a_i < b_i < \infty for all ii.

    First, suppose that both aa and bb are finite. For each ϵ>0\epsilon > 0, as FF is right continuous, there exists some δ>0\delta > 0, such that

    F(a+δ)≤F(a)+ϵ , and μ((a,b])≤μ((a+δ,b])+ϵ. \begin{aligned} F(a + \delta) &\le F(a) + \epsilon\,,\qquad \text{ and } \\ \mu \Big( (a, b] \Big) &\le \mu \Big( (a + \delta, b] \Big) + \epsilon. \end{aligned}

    For each ii, there exists some δi>0\delta_i > 0, such that

    F(bi+δi)≤F(bi)+ϵ/2i , and μ((ai,bi+δi])≤μ((ai,bi])+ϵ/2i. \begin{aligned} F(b_i + \delta_i) &\le F(b_i) + \epsilon / 2^i\,,\qquad \text{ and } \\ \mu \Big( (a_i, b_i + \delta_i] \Big) &\le \mu \Big( (a_i, b_i] \Big) + \epsilon / 2^i. \end{aligned}

    Since {(ai,bi+δi)}\{ (a_i, b_i + \delta_i) \} is an open cover of the closed interval [a+δ,b][a + \delta, b], there is a finite sub-cover {(si,ti)}\{ (s_i, t_i) \}. It follows that {(si,ti]}\{ (s_i, t_i] \} covers (a+δ,b](a + \delta, b]. Therefore,

    μ((a+δ,b])≤∑iμ((si,ti])≤∑i≥1μ((ai,bi+δi]) \mu \Big( (a + \delta, b] \Big) \le \sum_i \mu \Big( (s_i, t_i] \Big) \le \sum_{i \ge 1} \mu \Big( (a_i, b_i + \delta_i] \Big)

    It follows that

    μ((a,b])≤∑i≥1μ((ai,bi])+2ϵ. \mu \Big( (a, b] \Big) \le \sum_{i \ge 1} \mu \Big( (a_i, b_i] \Big) + 2 \epsilon.

    Letting ϵ→0\epsilon \rightarrow 0 finishes the proof.

    If a=−∞a = -\infty or b=∞b = \infty, the inequality μ((a′,b′])≤∑i≥1μ((ai,bi])\small \mu \Big( (a', b'] \Big) \le \sum_{i \ge 1} \mu \Big( (a_i, b_i] \Big) holds for all finite subset (a′,b′]⊆(a,b](a', b'] \subseteq (a, b]. Hence it holds for (a,b](a, b].

□\square

Caratheodory’s Extension Theorem

Caratheodory’s Extension Theorem. Let μˉ\bar \mu be a σ\sigma-finite measure on an algebra S~\widetilde S. Then μˉ\bar \mu has a unique extension μ∗\mu^* (that is a measure) to σ(S~)\sigma(\widetilde S), the smallest σ\sigma-field containing S~\widetilde S.

♠\spadesuit Outer Measure

Define the function μ∗:2Ω→R≥0\mu^*: 2^\Omega \rightarrow \mathbb{R}_{\ge 0} as ∀A⊆Ω\forall A \subseteq \Omega,

μ∗(A)=inf⁡{∑i≥1μˉ(Ai):A⊆∪i≥1Ai and Ai∈S~ ,∀i} \mu^*(A) = \inf \left\{ \sum_{i \ge 1} \bar{ \mu }(A_i) : A \subseteq \cup_{i \ge 1} A_i \text{ and } A_i \in \widetilde{S}\,,\forall i \right\}

Intuitively, μ∗\mu^* is an upper bound on the measure of AA. The following properties follows immediately form the definition.

  1. μ∗(∅)≤∑i=1∞μˉ(∅)=0\mu^*(\emptyset) \le \sum_{i = 1}^\infty \bar{ \mu }(\emptyset) = 0.

  2. μ∗\mu^* is monotone: A⊆BA \subseteq B implies that μ∗(A)≤μ∗(B)\mu^*(A) \le \mu^*(B).

  3. Subadditivity. If A⊆∪i≥1AiA \subseteq \cup_{i \ge 1} A_i, then μ∗(A)≤∑i≥1μ∗(Ai)\mu^*(A) \le \sum_{i \ge 1} \mu^*(A_i). Given ϵ>0\epsilon > 0, for each AiA_i, there exists a countable sequence {Bi,j:j∈Z+}\{ B_{i, j} : j \in \mathbb{Z}^+ \} in S~\widetilde{S} such that it covers AiA_i and

    ∑j≥1μˉ(Bi,j)≤μ∗(Ai)+ϵ/2i. \sum_{j \ge 1} \bar{ \mu }(B_{i, j}) \le \mu^*(A_i) + \epsilon / 2^i.

    The sequence {Bi,j:i,j∈Z+}\{ B_{i, j} : i, j \in \mathbb{Z}^+ \} is countable and covers AA. Further

    μ∗(A)≤∑i≥1∑j≥1μˉ(Bi,j)≤∑i≥1μ∗(Ai)+ϵ. \mu^*( A ) \le \sum_{i \ge 1} \sum_{j \ge 1} \bar{ \mu }(B_{i, j}) \le \sum_{i \ge 1} \mu^* (A_i) + \epsilon.

    The inequality holds for all ϵ>0\epsilon > 0 hence it holds that

    μ∗(A)≤∑i≥1μ∗(Ai). \mu^*( A ) \le \sum_{i \ge 1} \mu^* (A_i).

Now, μ∗\mu^* is almost a measure: it is an nonnegative set function and satisfies subadditivity. By previous analysis, we know that

Given a nonnegative, finitely additive set function defined on an algebra, the following conditions are equivalent

  • σ\sigma-additivity.
  • σ\sigma-subadditivity.

If we can prove that μ∗\mu^* is finitely additive, then it is a measure on the σ\sigma-algebra, 2Ω2^\Omega (which is also an algebra). However, in general, finite additivity does not hold on 2Ω2^\Omega. Therefore, we restrict our discussion on a subset of 2Ω2^\Omega.

Measurable Set

Intuitively, the measurable sets are the ones on which finite additivity of μ∗\mu^* holds. Formally, we say that a set A⊆ΩA \subseteq \Omega is called measurable if

μ∗(B)=μ∗(B∩A)+μ∗(B∩Aˉ),∀B⊆Ω. \mu^* (B) = \mu^* (B \cap A) + \mu^* (B \cap \bar A),\qquad \forall B \subseteq \Omega.

Using subadditivity, to prove AA is measurable, it is enough to show that

μ∗(B)≥μ∗(B∩A)+μ∗(B∩Aˉ),∀B⊆Ω. \mu^* (B) \ge \mu^* (B \cap A) + \mu^* (B \cap \bar A),\qquad \forall B \subseteq \Omega.

Definition. S∗=S^* = { the collection of all measurable sets }.

Theorem. S∗S^* is a σ\sigma-field that contains S~\widetilde{S} and the restriction of μ∗\mu^* to S∗S^* is a measure.

▶S∗\blacktriangleright \quad S^* is a σ\sigma-field
  1. If A∈S∗A \in S^*, then so is Aˉ\bar A.

    Proof. By definition.

  2. If A1,A2∈S∗A_1, A_2 \in S^*, then A1∪A2∈S∗A_1 \cup A_2 \in S^* and A1∩A2∈S∗A_1 \cap A_2 \in S^*.

    Proof. Observe that

    A1∪A2⊆(A1∩Aˉ2)∪(A1∩A2)∪(Aˉ1∩A2). A_1 \cup A_2 \subseteq (A_1 \cap \bar A_2) \cup (A_1 \cap A_2) \cup (\bar A_1 \cap A_2).

    For all B⊆ΩB \subseteq \Omega, it holds that

    μ∗(B)=μ∗(B∩A1)+μ∗(B∩Aˉ1)=μ∗(B∩A1∩A2)+μ∗(B∩A1∩Aˉ2)+μ∗(B∩Aˉ1∩A2)+μ∗(B∩Aˉ1∩Aˉ2)≥μ∗(B∩(A1∪A2))++μ∗(B∩Aˉ1∩Aˉ2) \begin{aligned} \mu^*(B) &= \mu^*(B \cap A_1) + \mu^* (B \cap \bar A_1) \\ &= \mu^*(B \cap A_1 \cap A_2 ) + \mu^*(B \cap A_1 \cap \bar A_2 ) \\ &\quad + \mu^* (B \cap \bar A_1 \cap A_2 ) + \mu^* (B \cap \bar A_1 \cap \bar A_2 ) \\ &\ge \mu^*(B \cap (A_1 \cup A_2) ) + + \mu^* (B \cap \bar A_1 \cap \bar A_2 ) \end{aligned}

    The last inequality follows form subadditivity of μ∗\mu^*. Hence A1∪A2∈S∗A_1 \cup A_2 \in S^*. Finally, A1∩A2=Aˉ1∪Aˉ2‾∈S∗A_1 \cap A_2 = \overline{ \bar A_1 \cup \bar A_2} \in S^*.

    □\square

  3. If Ai∈S∗A_i \in S^*, then A=∪i≥1Ai∈S∗A = \cup_{i \ge 1} A_i \in S^*.

    Proof. We have proven that S∗S^* is an algebra. Let B1=A1B_1 = A_1 and Bi=Ai−Ai−1−…−A1B_i = A_i - A_{i - 1} - \ldots - A_1 for i≥2i \ge 2. It holds that Bi∈S∗B_i \in S^* and ∪iAi=∪Bi\cup_i A_i = \cup B_i. Replacing AiA_i with BiB_i we can assume without lose of generality that the AiA_i are disjoint. For all n∈Z+n \in \mathbb{Z}^+, let Cn=+i∈[n]AiC_n = +_{i \in [n] } A_i. For all B⊆ΩB \subseteq \Omega and n>1n > 1,

    μ∗(B)=μ∗(B∩Cn)+μ∗(B∩Cˉn)=μ∗(B∩Cn∩An)+μ∗(B∩Cn∩Aˉn)+μ∗(B∩Cˉn)=μ∗(B∩An)+μ∗(B∩Cn−1)+μ∗(B∩Cˉn) \begin{array}{ll} \mu^*(B) &= \mu^* (B \cap C_n ) + \mu^* (B \cap \bar C_n ) \\ &= \mu^* (B \cap C_n \cap A_n ) + \mu^* (B \cap C_n \cap \bar A_n ) + \mu^* (B \cap \bar C_n ) \\ &= \mu^* (B \cap A_n ) + \mu^* (B \cap C_{n - 1} ) + \mu^* (B \cap \bar C_n ) \\ \end{array}

    The last equality follows from that Cn∩An=AnC_n \cap A_n = A_n and Cn∩Aˉn=Cn−1C_n \cap \bar A_n = C_{n - 1}. By induction, and monotonicity of μ∗\mu^*, we get

    μ∗(B)=∑i∈[n]μ∗(B∩Ai)+μ∗(B∩Cˉn)≥∑i∈[n]μ∗(B∩Ai)+μ∗(B∩Aˉ) \begin{array}{ll} \mu^*(B) &= \sum_{i \in [n] } \mu^* (B \cap A_i ) + \mu^* (B \cap \bar C_n ) \\ &\ge \sum_{i \in [n] } \mu^* (B \cap A_i ) + \mu^* (B \cap \bar A ) \end{array}

    Letting n→∞n \rightarrow \infty finishes the proof.

    □\square

▶S∗⊇S~\blacktriangleright \quad S^* \supseteq \widetilde{S}

Proof. For all B⊆ΩB \subseteq \Omega, and all ϵ>0\epsilon > 0, there exists a sequence {Bi}\{ B_i \} in S~\widetilde{S}, such that B⊆∪iBiB \subseteq \cup_i B_i and

μ∗(B)+ϵ≥∑i≥1μˉ(Bi). \begin{array}{ll} \mu^* (B) + \epsilon \ge \sum_{i \ge 1} \bar \mu (B_i). \end{array}

Let A∈S~A \in \widetilde{S}. Observe that Bi∩A,Bi∩Aˉ∈S~B_i \cap A, B_i \cap \bar A \in \widetilde{S}, and that

B∩A⊆∪i(Bi∩A) ,B∩Aˉ⊆∪i(Bi∩Aˉ). \begin{array}{ll} B \cap A \subseteq \cup_i (B_i \cap A)\,, \qquad B \cap \bar A \subseteq \cup_i (B_i \cap \bar A). \end{array}

By the definition of μ∗\mu^* and finite additivity of μˉ\bar \mu, it holds that

μ∗(B∩A)+μ∗(B∩Aˉ)≤∑i≥1μˉ(Bi∩A)+∑i≥1μˉ(Bi∩Aˉ)=∑i≥1μˉ(Bi). \small \begin{aligned} \mu^* (B \cap A) + \mu^* (B \cap \bar A) \le \sum_{i \ge 1} \bar \mu (B_i \cap A) + \sum_{i \ge 1} \bar \mu (B_i \cap \bar A) = \sum_{i \ge 1 } \bar \mu (B_i). \end{aligned}

Letting ϵ→0\epsilon \rightarrow 0, it concludes that

μ∗(B)≥μ∗(B∩A)+μ∗(B∩Aˉ). \mu^*(B) \ge \mu^* (B \cap A) + \mu^* (B \cap \bar A).

□\square

▶μ∗\blacktriangleright \quad \mu^* is a measure restricted to S∗S^*

Sigma additivity of μ∗\mu^*. Let A=+i≥1AiA = +_{i \ge 1} A_i, with Ai∈S∗A_i \in S^*, then

μ∗(A)=∑i≥1μ∗(Ai). \mu^*(A) = \sum_{i \ge 1 } \mu^*( A_i ).

Proof. As S∗S^* is an algebra and μ∗\mu^* is σ\sigma-subadditive, it suffices to prove that μ∗\mu^* is finitely additive. Let A=+i∈[n]AiA = +_{i \in [n] } A_i.

μ∗(A)=μ∗(A∩An)+μ∗(A∩Aˉn)=μ∗(An)+μ∗(+i∈[n−1]Ai)… =∑i∈[n]μ∗(Ai). \begin{aligned} \mu^*(A) &= \mu^*(A \cap A_n) + \mu^*(A \cap \bar A_n) \\ &= \mu^*(A_n) + \mu^*( +_{i \in [n - 1] } A_i ) \\ &\ldots\, \\ &= \sum_{ i \in [n] } \mu^* (A_i). \end{aligned}

□\square

♠\spadesuit Uniqueness

▶  π\blacktriangleright\,\,\pi-system

Definition. P\mathcal{P} is said to be a π\pi-system if it is closed under intersection.

▶  λ\blacktriangleright\,\,\lambda-system

Definition. L\mathcal{L} is said to be a λ\lambda-system if
(1). Ω∈L\Omega \in \mathcal{L}.
(2). If A∈LA \in \mathcal{L}, then Aˉ∈L\bar A \in \mathcal{L}.
(3). If A1,A2,…∈LA_1, A_2, \ldots \in \mathcal{L} with An∩Am=∅A_n \cap A_m = \emptyset for n≠mn \neq m, then ∪n≥1An∈L\cup_{n \ge 1} A_n \in \mathcal{L}.

Note that a λ\lambda-system is almost a σ\sigma-algebra, expect that it is only closed under disjoint union.

Lemma. Replacing (2) in the definition with the following one results in equivalent definition
(2'). If A,B∈LA, B \in \mathcal{L} with A⊆BA \subseteq B, then B−A∈LB - A \in \mathcal{L}.

Proof.

Assume that (1), (2), (3) hold. If A,B∈LA, B \in \mathcal{L} with A⊆BA \subseteq B, then

Bˉ∈L  ⟹  A∪Bˉ∈L  ⟹  A∪Bˉ‾=B−A∈L. \bar B \in \mathcal{L} \implies A \cup \bar B \in \mathcal{L} \implies \overline{A \cup \bar B} = B - A \in \mathcal{L}.

Hence (2') holds.

Assume that (1), (2'), (3) hold. Since Ω∈L\Omega \in \mathcal{L}, if A∈LA \in \mathcal{L}, then

Aˉ=Ω−A∈L. \bar A = \Omega - A \in \mathcal{L}.

Hence (2) holds.

□\square

Lemma. If a λ\lambda-system L\mathcal{L} is closed under intersection, then it is a σ\sigma-field.

Proof. Since it is closed under complement and intersection, it is an algebra. It is a σ\sigma-field as it satisfies monotonicity.

□\square

▶  \blacktriangleright\,\, π-λ\pi \text{-} \lambda Lemma

π-λ\pi \text{-} \lambda Lemma. Let P\mathcal{P} be a π\pi system and L\mathcal{L} be a λ\lambda-system. If P⊆L\mathcal{P} \subseteq \mathcal{L}, then σ(P)⊆L\sigma(\mathcal{P}) \subseteq \mathcal{L}.

Proof. Let λ(P)\lambda(\mathcal{P}) the smallest λ\lambda-system that contains P\mathcal{P}. We will show that λ(P)\lambda(\mathcal{P}) is a σ\sigma-field. Then

σ(P)⊆λ(P)⊆L. \sigma(\mathcal{P}) \subseteq \lambda(\mathcal{P}) \subseteq \mathcal{L}.

It suffices to prove that λ(P)\lambda(\mathcal{P}) is a π\pi-system.

​For each A∈λ(P)A \in \lambda (\mathcal{P}), define

ΠA≐{B∈λ(P):B∩A∈λ(P)}. \Pi_A \doteq \{ B \in \lambda (\mathcal{P}) : B \cap A \in \lambda (\mathcal{P}) \}.

We will prove that ΠA\Pi_A is a λ\lambda-system and ΠA⊇P\Pi_A \supseteq \mathcal{P}. It follows that ΠA⊇λ(P)\Pi_A \supseteq \lambda(\mathcal{P}) and λ(P)\lambda(\mathcal{P}) is closed under intersection, and therefore a π\pi-system.

ΠA\Pi_A is a λ\lambda-system:

  1. Ω∈ΠA\Omega \in \Pi_A.

  2. If B∈ΠAB \in \Pi_A, we need to prove that Bˉ∈ΠA\bar B \in \Pi_A. We need to prove that A∩Bˉ=A−(A∩B)∈λ(P)A \cap \bar B = A - \big( A \cap B \big) \in \lambda(\mathcal{P}). This follows from definition (2') of λ\lambda-system and that

    A,(A∩B)∈λ(P) ∧ A⊇(A∩B). A, \big( A \cap B \big) \in \lambda(\mathcal{P})\, \wedge\, A \supseteq \big( A \cap B \big).
  3. If B1,B2,…∈ΠAB_1, B_2, \ldots \in \Pi_A with Bn∩Bm=∅B_n \cap B_m = \emptyset, then ∪n≥1Bn∈ΠA\cup_{n \ge 1} B_n\in \Pi_A, since

(∪n≥1Bn)∩A=∪n≥1(Bn∩A)∈λ(P). \big( \cup_{n \ge 1} B_n \big) \cap A = \cup_{n \ge 1} (B_n \cap A) \in \lambda (\mathcal{P}).

ΠA\Pi_A contains P\mathcal{P}:

If B∈PB \in \mathcal{P}, denote ΠB={A⊆Ω:A∩B∈λ(P)}\Pi_B = \{ A \subseteq \Omega : A \cap B \in \lambda (\mathcal{P}) \}. Since P\mathcal{P} is closed under intersection, ΠB⊇P\Pi_B \supseteq \mathcal{P}. Since ΠB\Pi_B is a λ\lambda-system, ΠB⊇λ(P)\Pi_B \supseteq \lambda (\mathcal{P}).

Hence, for all A∈λ(P),B∈PA \in \lambda (\mathcal{P}), B \in \mathcal{P}, we have A∩B∈λ(P)A \cap B \in \lambda (\mathcal{P}). Therefore ΠA⊇P\Pi_A \supseteq \mathcal{P}.

□\square

Unique Extension Theorem

Theorem. Let P\mathcal{P} be a π\pi-system. If ν1\nu_1 and ν2\nu_2 are measures (on σ\sigma-fields F1\mathcal{F}_1 and F2\mathcal{F}_2) that agree on P\mathcal{P} and there is a sequence An∈PA_n \in \mathcal{P} with An↑ΩA_n \uparrow \Omega and νi(An)<∞\nu_i (A_n) < \infty for i=1,2i = 1, 2, then ν1\nu_1 and ν2\nu_2 agree on σ(P)\sigma( \mathcal{P} ).

Proof. Since νi\nu_i are measures on σ(P)\sigma(\mathcal{P} ), for all A∈σ(P)A \in \sigma(\mathcal{P}),

νi(A)=lim⁡n→∞νi(A∩An). \nu_i (A) = \lim_{n \rightarrow \infty} \nu_i (A \cap A_n).

It suffices to prove that ν1(A∩An)=ν2(A∩An)\nu_1 (A \cap A_n) = \nu_2 (A \cap A_n) holds for all n∈N+n \in \mathbb{N}^+.

For a fixed nn, now consider the set

L={B∈σ(P):ν1(B∩An)=ν2(B∩An)}⊆σ(P). \mathcal{L} = \{ B \in \sigma(\mathcal{P} ) : \nu_1 (B \cap A_n) = \nu_2 (B \cap A_n) \} \subseteq \sigma( \mathcal{P} ).

▹\triangleright P⊆L:\mathcal{P} \subseteq \mathcal{L}: via assumption that P\mathcal{P} is a π\pi-system and that ν1\nu_1 and ν2\nu_2 agrees on P\mathcal{P}.

▹\triangleright L\mathcal{L} is a λ\lambda system:

  • Ω∈L\Omega \in \mathcal{L}.

  • If B∈LB \in \mathcal{L}, then Bˉ∈L\bar B \in \mathcal{L} since

    ν1(Bˉ∩An)=ν1(An)−ν1(B∩An)=ν2(An)−ν2(B∩An)=ν2(Bˉ∩An), \begin{aligned} \nu_1 ( \bar B \cap A_n) &= \nu_1 ( A_n ) - \nu_1 (B \cap A_n) \\ &= \nu_2 ( A_n ) - \nu_2 (B \cap A_n) = \nu_2 ( \bar B \cap A_n ), \end{aligned}

    where the third equality follows from that ν1(An)=ν2(An)\nu_1 ( A_n ) = \nu_2 ( A_n ) and ν1(B∩An)=ν2(B∩An)\nu_1 (B \cap A_n) = \nu_2 (B \cap A_n).

  • If BjB_j are disjoint sets in L\mathcal{L}, then ∪j≥1Bj∈L\cup_{j \ge 1} B_j \in \mathcal{L}, since

    ν1((∪j≥1Bj)∩An)=∑jν1(Bj∩An)=∑jν2(Bj∩An)=ν2((∪j≥1Bj)∩An). \begin{aligned} \nu_1 \Big( \big( \cup_{j \ge 1} B_j \big) \cap A_n \Big) &= \sum_{j} \nu_1 (B_j \cap A_n) \\ &= \sum_{j} \nu_2 (B_j \cap A_n) = \nu_2 \Big( \big( \cup_{j \ge 1} B_j \big) \cap A_n \Big). \end{aligned}

Using the π\pi-λ\lambda lemma, we get L⊇σ(P)\mathcal{L} \supseteq \sigma( \mathcal{P} ), which finishes the proof.

□\square

Reference

[1]. R. Durrett, “Probability : theory and examples,” p. 432.
[2]. P. Billingsley, Probability and measure Anniversary Edition, 3rd ed. New York: Wiley, 1995.