Chain rule (probability)

The chain rule for two random events ${\displaystyle A}$ and ${\displaystyle B}$ says

$${\displaystyle P(A\cap B)=P(B\mid A)\cdot P(A).}$$

This rule is illustrated in the following example. Urn 1 has 1 black ball and 2 white balls and Urn 2 has 1 black ball and 3 white balls. Suppose we pick an urn at random and then select a ball from that urn. Let event ${\displaystyle A}$ be choosing the first urn: ${\displaystyle P(A)=P({\overline {A}})=1/2.}$ Let event ${\displaystyle B}$ be the chance we choose a white ball. The chance of choosing a white ball, given that we have chosen the first urn, is ${\displaystyle P(B|A)=2/3.}$ Event ${\displaystyle A\cap B}$ would be their intersection: choosing the first urn and a white ball from it. The probability can be found by the chain rule for probability:

$${\displaystyle \mathrm {P} (A\cap B)=\mathrm {P} (B\mid A)\mathrm {P} (A)=2/3\times 1/2=1/3.}$$

For more than two events ${\displaystyle A_{1},\ldots ,A_{n}}$ the chain rule extends to the formula

$${\displaystyle \mathrm {P} \left(A_{n}\cap \ldots \cap A_{1}\right)=\mathrm {P} \left(A_{n}|A_{n-1}\cap \ldots \cap A_{1}\right)\cdot \mathrm {P} \left(A_{n-1}\cap \ldots \cap A_{1}\right)}$$

which by induction may be turned into

$${\displaystyle \mathrm {P} \left(A_{n}\cap \ldots \cap A_{1}\right)=\prod _{k=1}^{n}\mathrm {P} \left(A_{k}\,{\Bigg |}\,\bigcap _{j=1}^{k-1}A_{j}\right).}$$

With four events (${\displaystyle n=4}$), the chain rule is

$${\displaystyle {\begin{aligned}\mathrm {P} (A_{1}\cap A_{2}\cap A_{3}\cap A_{4})&=\mathrm {P} (A_{4}\mid A_{3}\cap A_{2}\cap A_{1})\cdot \mathrm {P} (A_{3}\cap A_{2}\cap A_{1})\\&=\mathrm {P} (A_{4}\mid A_{3}\cap A_{2}\cap A_{1})\cdot \mathrm {P} (A_{3}\mid A_{2}\cap A_{1})\cdot \mathrm {P} (A_{2}\cap A_{1})\\&=\mathrm {P} (A_{4}\mid A_{3}\cap A_{2}\cap A_{1})\cdot \mathrm {P} (A_{3}\mid A_{2}\cap A_{1})\cdot \mathrm {P} (A_{2}\mid A_{1})\cdot \mathrm {P} (A_{1})\end{aligned}}}$$

For two random variables ${\displaystyle X,Y}$, to find the joint distribution, we can apply the definition of conditional probability to obtain:

$${\displaystyle \mathrm {P} (X=x,Y=y)=\mathrm {P} (X=x\mid Y=y)\cdot \mathrm {P} (Y=y)}$$

for any possible values ${\displaystyle x}$ of ${\displaystyle X}$ and ${\displaystyle y}$ of ${\displaystyle Y}$ in the discrete case or, in general,

$${\displaystyle \mathrm {P} (X\in A,Y\in B)=\mathrm {P} (X\in A\mid Y\in B)\cdot \mathrm {P} (Y\in B)}$$

for any possible measurable sets ${\displaystyle A}$ and ${\displaystyle B}$.

If one desires a notation for the probability distribution of ${\displaystyle X}$, one can use ${\displaystyle P_{X}}$, so that ${\displaystyle P_{X}(x):=P(X=x)}$ in the discrete case or, in general, ${\displaystyle P_{X}(A):=P(X\in A)}$ for a measurable set ${\displaystyle A}$.

Consider an indexed collection of random variables ${\displaystyle X_{1},\ldots ,X_{n}}$ taking possible values ${\displaystyle x_{1},\dots ,x_{n}}$ respectively. Then, to find the value of this member of the joint distribution, we can apply the definition of conditional probability to obtain:

$${\displaystyle \mathrm {P} \left(X_{n}=x_{n},\ldots ,X_{1}=x_{1}\right)=\mathrm {P} \left(X_{n}=x_{n}|X_{n-1}=x_{n-1},\ldots ,X_{1}=x_{1}\right)\cdot \mathrm {P} \left(X_{n-1}=x_{n-1},\ldots ,X_{1}=x_{1}\right)}$$

Repeating this process with each final term and letting ${\displaystyle A_{k}}$ denote the event ${\displaystyle X_{k}=x_{k}}$ creates the product:

$${\displaystyle \mathrm {P} \left(\bigcap _{k=1}^{n}A_{k}\right)=\prod _{k=1}^{n}\mathrm {P} \left(A_{k}\,{\Bigg |}\,\bigcap _{j=1}^{k-1}A_{j}\right)=\prod _{k=1}^{n}\mathrm {P} \left(X_{k}=x_{k}\,|\,X_{1}=x_{1},\dots X_{k-1}=x_{k-1}\right).}$$

With four variables (${\displaystyle n=4}$), denote ${\displaystyle P(x_{n}\,|\,x_{n-1}\dots ,x_{1}):=P(X_{n}=x_{n}\,|\,X_{n-1}=x_{n-1}\dots ,X_{1}=x_{1})}$ for brevity. Then, the chain rule produces this product of conditional probabilities:

$${\displaystyle {\begin{aligned}\mathrm {P} (x_{4},x_{3},x_{2},x_{1})&=\mathrm {P} (x_{4}\mid x_{3},x_{2},x_{1})\cdot \mathrm {P} (x_{3},x_{2},x_{1})\\&=\mathrm {P} (x_{4}\mid x_{3},x_{2},x_{1})\cdot \mathrm {P} (x_{3}\mid x_{2},x_{1})\cdot \mathrm {P} (x_{2},x_{1})\\&=\mathrm {P} (x_{4}\mid x_{3},x_{2},x_{1})\cdot \mathrm {P} (x_{3}\mid x_{2},x_{1})\cdot \mathrm {P} (x_{2}\mid x_{1})\cdot \mathrm {P} (x_{1})\end{aligned}}}$$