We consider a measure space (Ω,F,μ)(\Omega, \mathcal{F}, \mu), where Ω\Omega is a set of elements, F\mathcal{F} is a σ\sigma-field, and μ:F→R\mu : \mathcal{F} \rightarrow R a σ\sigma-finite measure defined on F\mathcal{F}.

Definition (Mutually Singular). Measures μ\mu and ν\nu on a measurable space (Ω,F)(\Omega, \mathcal{F}) are by definition mutually singular if they have disjoint supports, i.e., if

∃Sμ,Sν∈F,  s.t.  {Sμ∩Sν=∅, μ(Sˉμ)=ν(Sˉν)=0. \exists S_\mu, S_\nu \in \mathcal{F},\, \text{ s.t. }\, \begin{cases} S_\mu \cap S_\nu = \emptyset,\, \\ \mu(\bar S_\mu) = \nu(\bar S_\nu) = 0. \end{cases}

Definition (Absolute Continuous). Measure ν\nu is absolute continuous with μ\mu (or dominated by μ\mu), denoted as ν≪μ\nu \ll \mu, if

μ(E)=0  ⟹  ν(E)=0,∀E∈F. \mu(E) = 0 \implies \nu(E) = 0, \qquad \forall E \in \mathcal{F}.

Motivation. Let ff be a nonnegative and measurable function. Then the measure defined by

ν(E)=∫Ef dμ,∀E∈F, \nu(E) = \int_E f \, d \mu, \qquad \forall E \in \mathcal{F},

is absolutely continuous with respect to μ\mu. The The Radon-Nikodym Theorem establishes the opposite direction.

The Radon-Nikodym Theorem.
Let μ,ν\mu, \nu be σ\sigma-finite measures such that ν≪μ\nu \ll \mu. There exists a nonnegative measurable function ff, a density, such that

ν(E)=∫Ef dμ,∀E∈F. \nu(E) = \int_E f \, d \mu, \qquad \forall E \in \mathcal{F}.

For two such function ff and f′f', μ[f≠f′]=0\mu[ f \neq f' ] = 0. ff is called the Radon-Nikodym derivative of ν\nu with respect to μ\mu and is denoted dμ/dνd \mu / d \nu.

The proof of the Theorem relies on the following lemma, the proof of which is deferred to the end of the article.

Lemma. Let μ\mu and ν\nu be two finite measures that are not mutually singular. Then there exits A∈FA \in \mathcal{F} and ϵ>0\epsilon > 0, such that μ(A)>0\mu(A) > 0, and for each measurable set EE in AA, it holds that ϵ⋅μ(E)≤ν(E)\epsilon \cdot \mu(E) \le \nu(E).

Proof of the Theorem:

Since both μ\mu and ν\nu are σ\sigma-finite, there is a disjoint sequence En∈FE_n \in \mathcal{F} such that ∪n≥1En=Ω\cup_{n \ge 1} E_n = \Omega and that μ(En)<∞\mu(E_n) < \infty, ν(En)<∞\nu(E_n) < \infty for each n∈N+n \in \mathbb{N}^+.

Define finite measures μn\mu_n and νn\nu_n by

μn(E)=μ(E∩En),νn(E)=ν(E∩En),∀E∈F. \mu_n (E) = \mu(E \cap E_n),\quad \nu_n (E) = \nu(E \cap E_n),\quad \forall E \in \mathcal{F}.

Suppose that we can find densities fn,n∈N+f_n, n \in \mathbb{N}^+, such that νn(E)=∫Efn dμn\nu_n(E) = \int_E f_n \, d \mu_n. As μn\mu_n has density 1En\mathbf{1}_{E_n}, for each E∈FE \in \mathcal{F}, it holds that ∫Efn dμn=∫Efn⋅1En dμ\int_E f_n \, d \mu_n = \int_E f_n \cdot \mathbf{1}_{E_n} \, d \mu. Via monotone convergence theorem,

μ(E)=∑n≥1μn(E)=∑n≥1∫Efn dμn=∑n≥1∫Efn⋅1En dμ=∫E∑n≥1fn⋅1En dμ. \begin{aligned} \mu(E) &= \sum_{n \ge 1} \mu_n(E) = \sum_{n \ge 1} \int_E f_n \, d \mu_n \\ &= \sum_{n \ge 1} \int_E f_n \cdot \mathbf{1}_{E_n} \, d \mu = \int_E \sum_{n \ge 1} f_n \cdot \mathbf{1}_{E_n} \, d \mu. \end{aligned}

WLOG, we can assume that both μ\mu and ν\nu are finite.

Define the collection of functions

G={g is nonnegative, measurable and ∫Eg dμ≤ν(E),∀E∈F}. \mathcal{G} = \left\{ g \text{ is nonnegative, measurable and } \int_E g \, d \mu \le \nu(E), \quad \forall E \in \mathcal{F} \right\}.

Note that G\mathcal{G} is non-empty, since at least g≡0g \equiv 0 satisfies the conditions and belongs to G\mathcal{G}. Moreover,

  • G\mathcal{G} is closed under finite maxima: let g1,g2∈Gg_1, g_2 \in \mathcal{G}, then for each E∈FE \in \mathcal{F},

    ∫Emax⁡(g1,g2) dμ=∫E∩[g1≥g2]g1 dμ+∫E∩[g1<g2]g2 dμ≤ν(E∩[g1≥g2])+ν(E∩[g1<g2])=ν(E). \begin{aligned} \int_E \max (g_1, g_2) \, d \mu &= \int_{E \cap [g_1 \ge g_2]} g_1 \, d \mu + \int_{E \cap [g_1 < g_2]} g_2 \, d \mu \\ &\le \nu \big( E \cap [g_1 \ge g_2] \big) + \nu \big( E \cap [g_1 < g_2] \big) = \nu (E). \end{aligned}

  • G\mathcal{G} is closed under non-decreasing limit: let gn∈Gg_n \in \mathcal{G} such that gn↑gg_n \uparrow g, then by monotone convergence theorem it holds for each E∈FE \in \mathcal{F} that

    lim⁡n∫Egn dμ=∫Elim⁡ngn dμ≤ν(E). \lim_n \int_E g_n \, d \mu = \int_E \lim_n g_n \, d \mu \le \nu(E).

Next, consider the supremum of integrals of function in G\mathcal{G}:

α≐sup⁡g∈G∫g dμ≤ν(Ω)<∞. \alpha \doteq \sup_{g \in \mathcal{G} } \int g \ d \mu \le \nu(\Omega) < \infty.

Let gn∈G,n∈N+g_n \in \mathcal{G}, n \in \mathbb{N}^+ be a sequence of functions such that $ \lim_n \int g_n \ d \mu = \alpha. $

For each n∈N+n \in \mathbb{N}^+, define fn=max⁡i∈[n]gif_n = \max_{i \in [n]} g_i. Then the fnf_n is a non-decreasing sequence of functions in G\mathcal{G}. Denote f=lim⁡nfnf = \lim_n f_n. We have f∈Gf \in \mathcal{G}, and

∫f dμ=lim⁡n∫gn dμ=α. \int f \, d \mu = \lim_n \int g_n \ d \mu = \alpha.

It is left to show that λ(E)=∫Ef\lambda(E) = \int_E f for each measurable set EE. Define function λ:F→R\lambda : \mathcal{F} \rightarrow R as

λ(E)=ν(E)−∫Ef dμ \lambda(E) = \nu(E) - \int_E f \ d \mu

We complete the proof by showing that λ(E)≡0\lambda(E) \equiv 0.

It is easy to verify that λ\lambda is a measure (λ≥0\lambda \ge 0) as ν(E)≥∫Efdμ\nu(E) \ge \int_E f d \mu. Further, ∫Ef dμ≪μ\int_E f \ d \mu \ll \mu and ν≪μ\nu \ll \mu imply that λ≪μ\lambda \ll \mu.

Suppose λ≠0\lambda \ne 0. By assumption μ(Ω)<∞\mu(\Omega) < \infty, then there exists ϵ>0\epsilon > 0, such that λ(Ω)−ϵμ(Ω)>0\lambda(\Omega) - \epsilon \mu(\Omega) > 0. Let {A,B}\{ A, B\} be the Hahn decomposition of λ−ϵμ\lambda - \epsilon \mu such that AA is positive. Let g=f+ϵχAg = f + \epsilon \chi_{A}. It satisfies

  1. g∈Gg \in \mathcal{G}, since ∀E∈F\forall E \in \mathcal{F},

    λ(E)−∫Eg dμ=λ(E)−∫Ef dμ−∫EϵχA dμ≥0 \lambda(E) - \int_E g \ d \mu = \lambda(E) - \int_E f \ d \mu - \int_E \epsilon \chi_{A} \ d \mu \ge 0

  2. ∫g dμ≥α\int g \ d \mu \ge \alpha,

    ∫g dμ=∫f+ϵχA dμ=∫f dμ+∫ϵχA dμ=α+ϵμ(A)≥α \int g \ d \mu = \int f + \epsilon \chi_{A} \ d \mu = \int f \ d \mu + \int \epsilon \chi_A \ d \mu = \alpha + \epsilon \mu(A) \ge \alpha

If follows that μ(A)=0\mu(A) = 0. Now 0<(λ−ϵμ)(Ω)=(λ−ϵμ)(A)+(λ−ϵμ)(B)=0+(λ−ϵμ)(B)≤00 < (\lambda - \epsilon \mu)(\Omega) = (\lambda - \epsilon \mu)(A) + (\lambda - \epsilon \mu)(B) = 0 + (\lambda - \epsilon \mu)(B) \le 0. Contradiction. We thus have proved that λ=0\lambda = 0.

□\square

Proof of the Lemma.

For each n∈N+n \in \mathbb{N}^+, ν−(1/n)⋅μ\nu - (1 / n) \cdot \mu is a signed measure. Let Sn+,Sn−S_n^+, S_n^- be its Hahn decomposition. Define S+=∪n≥1Sn+S^+ = \cup_{n \ge 1} S_n^+, and S−=∩n≥1Sn−S^- = \cap_{n \ge 1} S_n^-. Since S−⊂Sn−S^- \subset S_n^-, we have

ν(S−)≤1n⋅μ(S−),∀n∈N+. \nu(S^-) \le \frac{1}{n} \cdot \mu(S^-), \qquad \forall n \in \mathbb{N}^+.

It follows that ν(S−)=0\nu(S^-) = 0. Since μ\mu and ν\nu are not mutually singular, S−S^- does not support μ\mu. Hence μ(S+)>0\mu(S^+) > 0. There exits some n∈N+n \in \mathbb{N}^+, such that μ(Sn+)>0\mu(S_n^+) > 0. But

ν(E)≥1n⋅μ(E),∀E⊂Sn+,E∈F. \nu(E) \ge \frac{1}{n} \cdot \mu(E), \qquad \forall E \subset S_n^+, E \in \mathcal{F}.

□\square

Theorem (Chain Rule). Let μ1,μ2,μ3\mu_1, \mu_2, \mu_3 be σ\sigma-finite measures on (Ω,F)(\Omega, \mathcal{F}), s.t., μ1≪μ2\mu_1 \ll \mu_2, μ2≪μ3\mu_2 \ll \mu_3. Prove that μ1≪μ3\mu_1 \ll \mu_3, and

dμ1dμ3(ω)=dμ1dμ2(ω)⋅dμ2dμ3(ω)a.e. with respect to  μ3. \frac{d \mu_1}{d \mu_3} (\omega) = \frac{d \mu_1}{d \mu_2} (\omega) \cdot \frac{d \mu_2}{d \mu_3} (\omega) \qquad \text{a.e. with respect to }\, \mu_3.

Proof. μ1≪μ3\mu_1 \ll \mu_3 is straightforward.

Next, to ease the notations, denote

f=dμ1dμ3,h=dμ1dμ2,δ=dμ2dμ3,g=h⋅δ. f = \frac{d \mu_1}{d \mu_3}, \quad h = \frac{d \mu_1}{d \mu_2}, \quad \delta = \frac{d \mu_2}{d \mu_3}, \quad g = h \cdot \delta.

Via Radon-Nikodym theorem, for each E∈FE \in \mathcal{F}, it holds that

μ1(E)=∫Ef dμ3,μ1(E)=∫Eh dμ2,μ2(E)=∫Eδ dμ3. \mu_1(E) = \int_E f \, d \mu_3, \quad \mu_1(E) = \int_E h \, d \mu_2, \quad \mu_2(E) = \int_E \delta \, d \mu_3.

The goal is to prove that

Lemma.

∫Ef dμ3=μ1(E)=∫Eh⋅δ dμ3,∀E∈F. \int_E f \, d \mu_3 = \mu_1(E) = \int_E h \cdot \delta \, d \mu_3, \quad \forall E \in \mathcal{F}.

Define En+≐{f−h⋅δ≥1/n},∀n∈N+E^+_n \doteq \{ f - h \cdot \delta \ge 1 / n \}, \forall n \in \mathbb{N}^+. Then

0=∫En+(f−h⋅δ) dμ3≥1n⋅μ3(En+)  ⟹  μ3(En+)=0. 0 = \int_{E^+_n} (f - h \cdot \delta) \, d \mu_3 \ge \frac{1}{n} \cdot \mu_3 (E^+_n) \implies \mu_3 (E^+_n) = 0.

Let E+≐{f>h⋅δ}E^+ \doteq \{ f > h \cdot \delta \}. Since En+↗EE_n^+ \nearrow E, we get μ3(E+)=lim⁡nμ3(En+)=0\mu_3 (E^+) = \lim_n \mu_3 (E^+_n) = 0.

In the same vein one can prove that μ3(f<h⋅δ)=0\mu_3 ( f < h \cdot \delta ) = 0. It concludes that f=h⋅δf = h \cdot \delta almost everywhere respect to μ3\mu_3.

Proof of the Lemma.

Consider a fixed E∈FE \in \mathcal{F}.

Case One. h=IS,S∈Fh = \mathbb{I}_S, S \in \mathcal{F} is an indicator random variable. Then

∫EIS dμ2=μ2(E∩S)=∫E∩Sδ dμ3=∫EIS⋅δ dμ3. \int_E \mathbb{I}_S \, d \mu_2 = \mu_2(E \cap S) = \int_{E \cap S} \delta \, d \mu_3 = \int_E \mathbb{I}_S \cdot \delta \, d \mu_3.

Case Two. hh is a simple function. The claim follows from the linearity of expectation.

Case Two. hh is a non-negative function. One can construct a sequence of simple functions which increases to hh. The claim follows from the monotone convergence theorem.

□\square

Reference

[1]. Billingsley. Patrick, Probability and measure