Definition. A collection of subsets F⊆2Ω is a σ-algebra (or σ-field) if
It is closed under countable union, intersection and complement.
Measure
Definition. Given a collection of subsets F⊆2Ω, a function μ:F→R is a measure if it satisfies
It is non-negative: ∀A∈F,μ(A)≥0.
The empty set is mapped to 0: μ(∅)=0.
σ-additivity: if Ai∈F is a countable sequence of disjoint sets, s.t., ∪i≥1Ai∈F, then
μ(∪i≥1Ai)=i≥1∑μ(Ai).
Measure Space
Definition. A measure space (Ω,F,μ) consists of three part:
A set Ω, on which it is defined;
A σ-field F that contains all measurable sets;
A measure μ defined on F.
In general, it is not easy to construct a measure space on Ω directly. Our indirect construction consists of three steps:
Consider a simple set class S and a measure μ defined on S.
We extend the domain of μ from S to a larger one Ω=2Ω.
We choose a subset from 2Ω, such that it is a σ-field and μ∗ is a measure defined on it.
In what follows, we illustrate the idea via constructing measures on Rd.
Semialgebra
Definition. A class of subsets S⊆2Ω is a semialgebra if
It is closed under intersection: ∀A,B∈S, A∩B∈S.
If A∈S, then Aˉ is a finite disjoint union of sets in S.
Examples. For d∈Z+, the collection of half-open intervals in Rd plus the empty set
Sd≐{×i∈[d](ai,bi]:−∞≤ai<bi≤∞,∀i∈[d]}∪{∅}
Algebra
Definition. A class of subsets F⊆2Ω is an algebra (or field) if
It is closed under union, intersection and complement.
Remark.By De Morgan's laws, A∩B=Aˉ∪Bˉ and A∪B=Aˉ∩Bˉ.
If we know a collection is closed under intersection (union) and complement, so is union (intersection).
Theorem If S⊆Ω is a semialgebra, then
S≐{finite disjoint union of sets in S}
is an algebra.
Proof.
S is closed under intersection: Let A,B∈S with A=+iAi and B=+jBj, where + denotes disjoint union and Ai,Bj∈S.
Then A∩B=+i,jAi∩Bj.
As S is a semialgebra, it follows that Ai∩Bj∈S and A∩B∈S.
S is closed under complement. Let A∈S with A=+iAi, where Ai∈S.
Then Aˉ=∩iAˉi.
As S is a semialgebra, Aˉi is a finite disjoint union of sets in S, which implies that Aˉi∈S. Therefore Aˉ is a finite intersection of sets in S. Since S is closed under intersection, Aˉ∈S.
□
Measure Extension: Semialgebra → Algebra
Theorem. Let S be a semialgebra and μ be a measure on S. Then μ has a unique extension μˉ that is a measure on S, the algebra generated by S.
Remark.The σ-additivity condition of μ being a measure can be replaced by
Finite additivity. If A,Ai∈S with A=+iAi, then μ(A)=∑iμ(Ai).
Subadditivity. If A,Ai∈S with A=+i≥1Ai, then μ(A)≤∑i≥1μ(Ai).
The subscript i≥1 in the conditions indicates a countable union, while a plain subscript i or j indicates a finite union.
The relation between the condition stated in the Theorem and the relaxed ones is established as follows.
Exercise. A finitely additive nonnegative set function on an algebra S is monotone and the following conditions are equivalent
σ-additivity.
Subadditivity.
The proof for the Theorem consists of two steps.
▶ Defining μˉ
If μ is finitely additive in S, then it has a finitely additive extension μˉ in S.
Extension. If A=+iAi∈S with Ai∈S, define
μˉ(A)=i∑μ(Ai).
We need to verify the value of μˉ(A) is unique.
Suppose A has another decomposition: A=+jBj with Bj∈S.
Since S is a semialgebra, Ai∩Bj∈S.
Using that μ is a measure
i∑μ(Ai)=i,j∑μ(Ai∩Bj)=j∑μ(Bj).
It follows that μˉ(A) does not change with the decomposition of A and hence is well-defined.
Finite additivity. Let A=+iAi∈S with Ai∈S and suppose that Ai=+jBi,j with Bj∈S.
Then A=+i,jBi,j.
By definition of μˉ(A) and μˉ(Ai),
μˉ(A)=i,j∑μ(Bi,j)=i∑μˉ(Ai).
We list two more properties about μˉ induced by finite additivity.
Monotonicity. If A,B∈S with A⊆B, then μˉ(A)≤μˉ(B). As S is an algebra, B−A∈S and by finite additivity of μˉ,
μˉ(A)≤μˉ(A)+μˉ(B−A)=μˉ(B).
Finite subadditivity. If A⊆∪i∈[n]Ai with A∈S, Ai∈S,∀i∈[n], then μˉ(A)≤∑iμˉ(Ai). Let B1=A1 and Bi=Ai−Ai−1−…−A1 for i≥1. As S is an algebra, Bi∈S. By monotonicity and finite additivity of μˉ,
μˉ(A)≤μˉ(+iBi)=i∑μˉ(Bi)≤i∑μˉ(Ai).
□
▶σ-Additivity of μˉ
If A,A1,…∈S with A=+i≥1Ai, then
μˉ(A)=i≥1∑μˉ(Ai).
Proof. Let Ai=+jBi,j with Bi,j∈S.
Then A=+i,jBi,j and ∑i≥1μˉ(Ai)=∑i≥1∑jμ(Bi,j).
WLOG, we assume that Ai are countable union of disjoint sets in S.
Next, assume that A=+kCk with Ck∈S.
Using that S is closed under intersection, and that μ is a measure on S (therefore σ-additivity holds),
μ(Ck)=i≥1∑μ(Ck∩Ai).
Observing that nonnegative numbers can be summed in any order,
Definition.μˉ is said to be σ-finite if there is a sequence of sets Ai∈S such that μ(Ai)<∞ and ∪i≥1Ai=Ω.
Caratheodory’s Extension Theorem.
Let μˉ be a σ-finite measure on an algebra S.
Then μˉ has a unique extension μ∗ (that is a measure) to σ(S), the smallest σ-field containing S.
Remark:Observe that σ(S)=σ(S).
The proof is included at the end of the article.
Lebesgue-Stieltjes Measure
In this section, we construct concrete measures on Rd.
Definition. A function F:R→R is a Stieltjes measure function if
F is non-decreasing.
F is right continuous, i.e., limy↓xF(y)=F(x).
Definition (Borel Sets). Let Rd=σ(Sd). It is the smallest σ-field containing all open sets in Rd.
Theorem 3. Associated with each Stieltjes measure function F there is a unique measure μ on (R,R1) with
μ((a,b])=F(b)−F(a).
When F(x)=x, the measure μ is called Lebesgue measure.
Proof. We just to verity that μ is a measure defined on the following semi-algebra:
S1≐{(a,b]:−∞≤a<b≤∞}∪{∅}.
μ((a,b])≥0, since F is non-decreasing.
Finite additivity. If (a,b]=+i(ai,bi], μ((a,b])=∑iμ((ai,bi]). We can rearrange the {(ai,bi]}, such that a=a1, bi=ai+1 for all 1≤i<n, and bn=b. Hence,
Finite subadditivity. If (a,b]⊆∪i∈[n](ai,bi], μ((a,b])≤∑i∈[n]μ((ai,bi]).
Recall that the definition of μˉ only requires finite additivity of μ.
Working with S1 and μˉ significantly simplifies the proof.
As for all A∈S1, it holds that μ(A)=μˉ(A).
It suffices to prove that
μˉ((a,b])≤i∈[n]∑μˉ((ai,bi]).
But this follows exactly from finite subadditivity of μˉ.
σ-additivity. If (a,b]=+i≥1(ai,bi], then μ((a,b])=∑i≥1μ((ai,bi]).
Using finite subadditivity of μ, for all n∈Z+,
μ((a,b])≥i∈[n]∑μ((ai,bi]).
Letting n→∞, we get
μ((a,b])≥∑i≥1μ((ai,bi]).
It is left to prove the reverse direction of the inequality (σ-subadditivity).
If μ((ai,bi])=∞ for some i, then the inequality is trivial.
WLOG, assume that −∞<ai<bi<∞ for all i.
First, suppose that both a and b are finite.
For each ϵ>0, as F is right continuous, there exists some δ>0, such that
F(a+δ)μ((a,b])≤F(a)+ϵ, and ≤μ((a+δ,b])+ϵ.
For each i, there exists some δi>0, such that
F(bi+δi)μ((ai,bi+δi])≤F(bi)+ϵ/2i, and ≤μ((ai,bi])+ϵ/2i.
Since {(ai,bi+δi)} is an open cover of the closed interval [a+δ,b], there is a finite sub-cover {(si,ti)}.
It follows that {(si,ti]} covers (a+δ,b]. Therefore,
μ((a+δ,b])≤i∑μ((si,ti])≤i≥1∑μ((ai,bi+δi])
It follows that
μ((a,b])≤i≥1∑μ((ai,bi])+2ϵ.
Letting ϵ→0 finishes the proof.
If a=−∞ or b=∞, the inequality
μ((a′,b′])≤∑i≥1μ((ai,bi])
holds for all finite subset (a′,b′]⊆(a,b].
Hence it holds for (a,b].
□
Caratheodory’s Extension Theorem
Caratheodory’s Extension Theorem.
Let μˉ be a σ-finite measure on an algebra S.
Then μˉ has a unique extension μ∗ (that is a measure) to σ(S), the smallest σ-field containing S.
♠ Outer Measure
Define the function μ∗:2Ω→R≥0 as ∀A⊆Ω,
μ∗(A)=inf{i≥1∑μˉ(Ai):A⊆∪i≥1Ai and Ai∈S,∀i}
Intuitively, μ∗ is an upper bound on the measure of A.
The following properties follows immediately form the definition.
μ∗(∅)≤∑i=1∞μˉ(∅)=0.
μ∗ is monotone: A⊆B implies that μ∗(A)≤μ∗(B).
Subadditivity. If A⊆∪i≥1Ai, then μ∗(A)≤∑i≥1μ∗(Ai).
Given ϵ>0, for each Ai, there exists a countable sequence {Bi,j:j∈Z+} in S such that it covers Ai and
j≥1∑μˉ(Bi,j)≤μ∗(Ai)+ϵ/2i.
The sequence {Bi,j:i,j∈Z+} is countable and covers A.
Further
μ∗(A)≤i≥1∑j≥1∑μˉ(Bi,j)≤i≥1∑μ∗(Ai)+ϵ.
The inequality holds for all ϵ>0 hence it holds that
μ∗(A)≤i≥1∑μ∗(Ai).
Now, μ∗ is almost a measure: it is an nonnegative set function and satisfies subadditivity. By previous analysis, we know that
Given a nonnegative, finitely additive set function defined on an algebra, the following conditions are equivalent
σ-additivity.
σ-subadditivity.
If we can prove that μ∗ is finitely additive, then it is a measure on the σ-algebra, 2Ω (which is also an algebra). However, in general, finite additivity does not hold on 2Ω. Therefore, we restrict our discussion on a subset of 2Ω.
Measurable Set
Intuitively, the measurable sets are the ones on which finite additivity of μ∗ holds.
Formally, we say that a set A⊆Ω is called measurable if
μ∗(B)=μ∗(B∩A)+μ∗(B∩Aˉ),∀B⊆Ω.
Using subadditivity, to prove A is measurable, it is enough to show that
μ∗(B)≥μ∗(B∩A)+μ∗(B∩Aˉ),∀B⊆Ω.
Definition.S∗= { the collection of all measurable sets }.
Theorem.S∗ is a σ-field that contains S and the restriction of μ∗ to S∗ is a measure.
The last inequality follows form subadditivity of μ∗.
Hence A1∪A2∈S∗. Finally, A1∩A2=Aˉ1∪Aˉ2∈S∗.
□
If Ai∈S∗, then A=∪i≥1Ai∈S∗.
Proof. We have proven that S∗ is an algebra.
Let B1=A1 and Bi=Ai−Ai−1−…−A1 for i≥2.
It holds that Bi∈S∗ and ∪iAi=∪Bi. Replacing Ai with Bi we can assume without lose of generality that the Ai are disjoint.
For all n∈Z+, let Cn=+i∈[n]Ai.
For all B⊆Ω and n>1,
Definition.P is said to be a π-system if it is closed under intersection.
▶λ-system
Definition.L is said to be a λ-system if
(1). Ω∈L.
(2). If A∈L, then Aˉ∈L.
(3). If A1,A2,…∈L with An∩Am=∅ for n=m, then ∪n≥1An∈L.
Note that a λ-system is almost a σ-algebra, expect that it is only closed under disjoint union.
Lemma. Replacing (2) in the definition with the following one results in equivalent definition
(2'). If A,B∈L with A⊆B, then B−A∈L.
Proof.
Assume that (1), (2), (3) hold.
If A,B∈L with A⊆B, then
Bˉ∈L⟹A∪Bˉ∈L⟹A∪Bˉ=B−A∈L.
Hence (2') holds.
Assume that (1), (2'), (3) hold.
Since Ω∈L, if A∈L, then
Aˉ=Ω−A∈L.
Hence (2) holds.
□
Lemma. If a λ-system L is closed under intersection, then it is a σ-field.
Proof. Since it is closed under complement and intersection, it is an algebra. It is a σ-field as it satisfies monotonicity.
□
▶π-λ Lemma
π-λ Lemma. Let P be a π system and L be a λ-system.
If P⊆L, then σ(P)⊆L.
Proof. Let λ(P) the smallest λ-system that contains P.
We will show that λ(P) is a σ-field. Then
σ(P)⊆λ(P)⊆L.
It suffices to prove that λ(P) is a π-system.
For each A∈λ(P), define
ΠA≐{B∈λ(P):B∩A∈λ(P)}.
We will prove that ΠA is a λ-system and ΠA⊇P.
It follows that ΠA⊇λ(P) and λ(P) is closed under intersection, and therefore a π-system.
ΠA is a λ-system:
Ω∈ΠA.
If B∈ΠA, we need to prove that Bˉ∈ΠA. We need to prove that A∩Bˉ=A−(A∩B)∈λ(P).
This follows from definition (2') of λ-system and that
A,(A∩B)∈λ(P)∧A⊇(A∩B).
If B1,B2,…∈ΠA with Bn∩Bm=∅, then ∪n≥1Bn∈ΠA, since
(∪n≥1Bn)∩A=∪n≥1(Bn∩A)∈λ(P).
ΠA contains P:
If B∈P, denote ΠB={A⊆Ω:A∩B∈λ(P)}. Since P is closed under intersection, ΠB⊇P. Since ΠB is a λ-system, ΠB⊇λ(P).
Hence, for all A∈λ(P),B∈P, we have A∩B∈λ(P). Therefore ΠA⊇P.
□
Unique Extension Theorem
Theorem. Let P be a π-system. If ν1 and ν2 are measures (on σ-fields F1 and F2) that agree on P and there is a sequence An∈P with An↑Ω and νi(An)<∞ for i=1,2, then ν1 and ν2 agree on σ(P).
Proof. Since νi are measures on σ(P), for all A∈σ(P),
νi(A)=n→∞limνi(A∩An).
It suffices to prove that ν1(A∩An)=ν2(A∩An) holds for all n∈N+.
For a fixed n, now consider the set
L={B∈σ(P):ν1(B∩An)=ν2(B∩An)}⊆σ(P).
▹P⊆L: via assumption that P is a π-system and that ν1 and ν2 agrees on P.
Using the π-λ lemma, we get L⊇σ(P), which finishes the proof.
□
Reference
[1]. R. Durrett, “Probability : theory and examples,” p. 432.
[2]. P. Billingsley, Probability and measure Anniversary Edition, 3rd ed. New York: Wiley, 1995.