ELEC 321

# Continous Random Vectors

Updated 2017-12-04

## Continuous Random Vectors

• All the entries are continuous random vectors
• The joint behavior are determined by the continuous joint density function $f(\vec x)$
• $f(\vec x)$ is a function that maps $\mathbb R^m\rightarrow \mathbb R$ where $m$ is the number of items in the vector
• $f(\vec x)$ satisfies the following:
1. $f(x_1, x_2,\dotsc, x_m)\geq 0$ for all $\vec x=(x_1, x_2,\dotsc, x_m)\in \mathbb R^m$
2. $\int_{\infty}^\infty\cdots\int_{\infty}^\infty f(x_1,\dotsc,x_m)\mathrm dx_1\dotsc\mathrm dx_m=1$

Take $m=2$ for instance, the corresponding joint distribution function is a multivariable function given by $F(x_1, x_2)$.

\begin{align}F(x_1,x_2)&=\mathbb P(X_1\leq x_1, X_2\leq x_2)\\ &=\int_{-\infty}^{x_2} \int_{-\infty}^{x_1}f(t_1, t_2)\mathrm dt_1\mathrm dt_2 \end{align}

By the FTC, we obtain the joint density function:

$f(x_1,x_2)=\frac{\partial^2}{\partial x_2\partial x_2}F(x_1, x_2)$

### Bivariate Normal (More in module 6)

The main model is given by:

$f(x_1,x_2)=\frac{(1-\rho^2)^{-\frac12}}{2\pi\sigma_1\sigma_2}\exp\left(-\frac{\frac{(x_1-\mu_1)^2}{\sigma_1^2}+\frac{(x_2-\mu_2)^2}{\sigma_2^2}-\frac{2\rho(x_1-\mu_1)(x_2-\mu_2)}{\sigma_1\sigma_2}}{2(1-\rho^2)}\right)$

### Marginal Density Function

Suppose we have the joint density function, $f(x_1, x_2)$, for random variables $X_1, X_2$.

Similar to discrete random variables, the marginal density function of one random variable is obtained by integrating the other random variables out.

$f_1(x_1)=\int_{-\infty}^\infty f(x_1, x_2)\mathrm dx_2$

### Conditional Density Function

The conditional densities work identical to the discrete version:

$f(x_2\vert x_1)=\frac{f(x_1, x_2)}{f_1(x_1)}$

Where $f(x_1, x_2)$ is the joint density and $f_1(x_1)$ is the marginal density.

It follows that given conditional and joint densities, we can obtain the joint density:

$f(x_1, x_2)=f(x_2\vert x_1)f_1(x_1)=f(x_1\vert x_2)f_2(x_2)$

### Conditional Mean

Let $\mu_{y\vert x}$ denote the mean of $Y$ given some realization of $X$. It is defined as

$\mu_{y\vert x}=\mathbb E(Y\vert X=x)=\int_{-\infty}^\infty y\cdot f(y\vert x)\mathrm dy$

### Conditional Variance

The variance of $Y$ given some $X=x$ is defined as

$\sigma_{y\vert x}^2=\text{Var}(Y\vert X=x)=\int_{-\infty}^{\infty}(y-\mu_{y\vert x})^2f(y\vert x)\mathrm dy$

### Independence

If the continuous random variables in the random vector are independent, then the joint density is all the marginal densities multiplied together.

$f(x_1,x_2,\dots,x_m)=f_1(x_1)f_2(x_2)\cdots f_m(x_m)$

Thus, any conditional density is zero.

Furthermore, given its covariance matrix, all non-diagonal elements is 0. In particular, $\sigma_{ij}=0$ for $i\neq j$.

Example:

Suppose $X\sim \text{Unif}(0,10)$ and that $Y\vert X=x \sim\text{Exp}(\frac{1}{x})$. What is the mean and variance of $Y$? What fractional of the total variance is explained by $X$?

Recall exponential random variables: the expected value is $1/\lambda$ and variance is $1/\lambda^2$. Since $Y\vert X=x\sim\text{Exp} (\frac{1}{x})$, then the expected value of $Y\vert X$, an exponential random variable is

$\mathbb E(Y\vert X)=X$

Recall uniform random variables: the expected value is $\frac{(a+b)}{2}$ and the variance is $\frac{(b-a)^2}{12}$. Plugging $a=0, b=10$, we get $\mathbb E(X)=5$ and $\text{Var}(X)=\frac{100}{12}$.

Putting everything together, we get

$\mathbb E(Y)=\mathbb\{\mathbb E(Y\vert X\}=\mathbb E\{X\}=5\\$

We compute the total variance:

\begin{align} \text{Var}(Y)&=\mathbb E\{\text{Var}(Y\vert X)\}+\text{Var}\{\mathbb E(Y\vert X)\}\\ &=\mathbb E\{X^2\}+\text{Var}\{X\}\\ &=\text{Var}(X)+[\mathbb E(X)]^2+\text{Var}(X)\\ &=\frac{200}{12}+25\\ &=41.7 \end{align}

To compute the percentage of explained variance, we look at the explained variance, which is $\text{Var}(\mathbb E(Y\vert X))$, and divide it by the total variance, which gives

$\frac{\text{Var}\{\mathbb E(Y\vert X)\}}{\text{Var}(Y)}=\frac{100/12}{41.7}=0.20$

## Functions of Continuous Random Vectors

Suppose we have a function $\mathbf h$ that is bijective, then for some random vectors $\mathbf x$ and $\mathbf y$,

$\mathbf y=\mathbf h(\mathbf x)\\ \mathbf x=\mathbf h^{-1}(\mathbf y)$

If given the density $f_{\mathbf X}$, then the density for $\mathbf y$ is

$f_{\mathbf Y(\mathbf y)}=f_{\mathbf X}(\mathbf h^{-1}(\mathbf y))\cdot \underbrace{\left\vert \det\left(\frac{\partial x_i}{\partial y_j}\right)\right\vert }_{J(\mathbf y)}$

Where $J(\mathbf y)$ is the Jacobian which is used for transformations.

### Linear Transformations

Suppose we have random vector $\mathbf Y=\mathbf A\mathbf X + \mathbf b$, where $\mathbf Y$ and $\mathbf X$ is a $n$-dimensional random vector, $\mathbf A$ is an $n\times n$ matrix, and $\mathbf b$ is an $n$-dimensional vector.

Then the expected value and covariance is transformed as follows:

$\boldsymbol\mu_{\mathbf Y}=\mathbb E\{\mathbf Y\}=\boxed{\mathbf{A}\boldsymbol \mu_{\mathbf X}+\mathbf b}\\ \text{Cov}(\mathbf Y)=\boxed{\mathbf A \text{Cov}(\mathbf X) \mathbf A^T}$