Week D-3. Independence of random variables

Definition

Independence

\( X_1, \cdots, X_n\) are indepdendent
iff for all \(x_1, \cdots, x_n \in R\)
$$ f_{X_1, \cdots, X_n}(x_1, \cdots, x_n)= \Pi_{i=1}^n f_{X_i}(x_i).$$
iff for all \(x_1, \cdots, x_n \in R\)
$$ F_{X_1, \cdots, X_n}(x_1, \cdots, x_n)= \Pi_{i=1}^n F_{X_i}(x_i).$$

iid (independently identically distributioned)

We say \( X_1, \cdots, X_n \) are iid if

1. \( X_1, \cdots, X_n\) are indepdendent
2. They have the same distribution, ie, \( F_{i}(x)=F(x), f_{X_i}(x)=f(x)\) for all i and where \( F_{i}(x), f_{X_i}(x) \) is the (marginal) cdf and pdf/pmf of \(X_i \) respectively.

Implications

If \( X_1, \cdots, X_n\) are indepdendent then

• Probability $$ P(X_1 \in A_1, \cdots, X_n \in A_n)= \Pi_{i=1}^n P(X_i \in A_i).$$

• For any functions \( g_i, \ i=1, \cdots, n\)
  $$E(\Pi_{i=1}^n g_i(X_i))= \Pi_{i=1}^n E( g_i(X_i)$$
  provided the expectations on the right-hand side exist.

• It is much easier to work out what \( Y=g(X_1, \cdots, X_n )\) is (ie, find out what its cdf or pdf/pmf or mgf is) or at least some snapshots such as expectation, variance or moments of \( Y\).

Illustrations

• Let \( X_1, \cdots, X_n \sim_{iid} Bernoulli(p), p \in (0,1)\) then
  $$ \sum_{i=1}^n X_i \sim Bin(n, p).$$
• Let \( X_1, \cdots, X_n \sim_{iid} N(\mu, \sigma^2))\) then
  $$ \bar{X}=\frac{1}{n}\sum_{i=1}^n X_i \sim N(\mu, \frac{\sigma^2}{n}).$$
• Even when the distribution is only known to expectation and variance, we can still get some “pictures” of it. In the sense,
  if \( X_1, \cdots, X_n \sim_{iid}\) with \( \mu=E(X_i), \sigma^2=Var(X_i)\) then
  $$ E(\bar{X})= \mu, Var(\bar{X})=\frac{\sigma^2}{n}.
  $$
  Note that this is consistent with our Bernoulli and normal cases.

Homework 7

Textbook: Sec. 8.6: 8.1, 8.5, 8.6, 8.9; Sec 9.7: 9.1, 9.3, 9.5, 9.10, 9.11, 9.12. Sec 10.5: 10.2, 10.6, 10.8, 10.16. To be discussed in class 12/27.