Regular conditional probability

In probability theory, regular conditional probability is a concept that formalizes the notion of conditioning on the outcome of a random variable. The resulting conditional probability distribution is a parametrized family of probability measures called a Markov kernel.

Motivation

Consider two random variables X and Y, where the represents the roll of a die. The conditional probability of Y being in a Borel set $A\subseteq \mathbb {R}$ is given by

$P(Y\in A|X=x)={\frac {P(Y\in A,X=x)}{P(X=x)}}.$

Conditional probability forms a two-variable function $\nu :\mathbb {R} \times {\mathcal {F}}\to \mathbb {R}$

$\nu (x,A)=P(A|X=x)$

Note that when x is not a possible outcome of X, the function is undefined: the roll of a die coming up 27 is a probability zero event. The function $\nu$ is defined almost everywhere in x.

Now consider two continuous random variables, X and Y, with density $f_{X,Y}(x,y)$. The conditional probability of Y being in A is given by

$P(Y\in A|X=x)={\frac {\int _{A}f_{X,Y}(x,y)\mathrm {d} y}{\int _{\mathbb {R} }f_{X,Y}(x,y)\mathrm {d} y}}.$

Conditional probability is a two variable function as before, undefined outside of the support of the distribution of X.

Relation to conditional expectation

In probability theory, the theory of conditional expectation is developed before that of regular conditional distributions.

For discrete and continuous random variables, the conditional expectation is given by

${\begin{aligned}\mathbb {E} [X|Y=y]&=\sum _{x}xP(X=x|Y=y)\\\mathbb {E} [X|Y=y]&=\int xf_{X|Y}(x,y)\mathrm {d} x\end{aligned}}$

where $f_{X|Y}(x,y)$ is the conditional density of X given Y.

It is natural to ask whether measure theoretical conditional expectation can also be expressed as

$\mathbb {E} [X|Y](\omega )=\int x\nu (\omega ,\mathrm {d} x)$

where $\nu :\Omega \times {\mathcal {B}}({\overline {\mathbb {R} }})\to [0,1]$ is a family of measures parametrized by outcome $\omega$.

Such a Markov kernel can be defined using conditional expectation:

$\nu (\omega ,A)=\mathbb {E} [1_{X\in A}|Y](\omega ).$

It can be shown that for almost all $\omega$, this is a probability measure if $X:\Omega \to \mathbb {R}$. There are, however, counterexamples when the random variable X takes values in a more general space E. A space E can be constructed where $\nu$ does not form a probability measure almost everywhere.

Definition

Let $(\Omega ,{\mathcal {F}},P)$ be a probability space, and let $T:\Omega \rightarrow E$ be a random variable, defined as a Borel-measurable function from $\Omega$ to its state space $(E,{\mathcal {E}})$. One should think of $T$ as a way to "disintegrate" the sample space $\Omega$ into $\{T^{-1}(x)\}_{x\in E}$. Using the disintegration theorem from the measure theory, it allows us to "disintegrate" the measure $P$ into a collection of measures, one for each $x\in E$. Formally, a regular conditional probability is defined as a function $\nu :E\times {\mathcal {F}}\rightarrow [0,1],$ called a "transition probability", where:

• For every $x\in E$, $\nu (x,\cdot )$ is a probability measure on ${\mathcal {F}}$. Thus we provide one measure for each $x\in E$.
• For all $A\in {\mathcal {F}}$, $\nu (\cdot ,A)$ (a mapping $E\mapsto [0,1]$) is ${\mathcal {E}}$-measurable, and
• For all $A\in {\mathcal {F}}$ and all $B\in {\mathcal {E}}$

$P{\big (}A\cap T^{-1}(B){\big )}=\int _{B}\nu (x,A)\,P{\big (}T^{-1}(dx){\big )}.$

where $P\circ T^{-1}$ is the pushforward measure $T_{*}P$ of the distribution of the random element $T$, $x\in \mathrm {supp} \,T,$ i.e. the topological support of the $T_{*}P$. Specifically, if we take $B=E$, then $A\cap T^{-1}(E)=A$, and so

$P(A)=\int _{E}\nu (x,A)\,P{\big (}T^{-1}(dx){\big )}$,

where $\nu (x,A)$ can be denoted, using more familiar terms $P(A\ |\ T=x)$ (this is "defined" to be conditional probability of $A$ given $x$, which can be undefined in elementary constructions of conditional probability). As can be seen from the integral above, the value of $\nu$ for points x outside the support of the random variable is meaningless; its significance as a conditional probability is strictly limited to the support of T.

The measurable space $(\Omega ,{\mathcal {F}})$ is said to have the regular conditional probability property if for all probability measures $P$ on $(\Omega ,{\mathcal {F}}),$ all random variables on $(\Omega ,{\mathcal {F}},P)$ admit a regular conditional probability. A Radon space, in particular, has this property.

See also conditional probability and conditional probability distribution.

Alternate definition

Consider a Radon space $\Omega$ (that is a probability measure defined on a Radon space endowed with the Borel sigma-algebra) and a real-valued random variable T. As discussed above, in this case there exists a regular conditional probability with respect to T. Moreover, we can alternatively define the regular conditional probability for an event A given a particular value t of the random variable T in the following manner:

$P(A|T=t)=\lim _{U\supset \{T=t\}}{\frac {P(A\cap U)}{P(U)}},$

where the limit is taken over the net of open neighborhoods U of t as they become smaller with respect to set inclusion. This limit is defined if and only if the probability space is Radon, and only in the support of T, as described in the article. This is the restriction of the transition probability to the support of T. To describe this limiting process rigorously:

For every $\epsilon >0,$ there exists an open neighborhood U of the event {T=t}, such that for every open V with $\{T=t\}\subset V\subset U,$

$\left|{\frac {P(A\cap V)}{P(V)}}-L\right|<\epsilon ,$

where $L=P(A|T=t)$ is the limit.

Example

To continue with our motivating example above, we consider a real-valued random variable X and write

$P(A|X=x_{0})=\nu (x_{0},A)=\lim _{\epsilon \rightarrow 0+}{\frac {P(A\cap \{x_{0}-\epsilon <X<x_{0}+\epsilon \})}{P(\{x_{0}-\epsilon <X<x_{0}+\epsilon \})}},$

(where $x_{0}=2/3$ for the example given.) This limit, if it exists, is a regular conditional probability for X, restricted to $\mathrm {supp} \,X.$

In any case, it is easy to see that this limit fails to exist for $x_{0}$ outside the support of X: since the support of a random variable is defined as the set of all points in its state space whose every neighborhood has positive probability, for every point $x_{0}$ outside the support of X (by definition) there will be an $\epsilon >0$ such that $P(\{x_{0}-\epsilon <X<x_{0}+\epsilon \})=0.$

Thus if X is distributed uniformly on $[0,1],$ it is truly meaningless to condition a probability on "$X=3/2$".

See also

• Conditioning (probability)
• Disintegration theorem
• Adherent point
• Limit point

External links

• Regular Conditional Probability on PlanetMath