ELEC 321

Multivariate Normal

Updated 2017-12-07

Standard Multivariate Normal

Recall that a standard normal random variable has a expected value / mean of 0 and a variance of 1. Then a random vector of standard normal is denoted as . Where is the random vector with a size of , is a vector of zeroes, and is a identity matrix.

Density

Suppose there are independent standard normal random variables then their joint density is the product:

Recall that

Then the joint density can be simplified to

Where is a vector that contains , and is the dot product of itself.

Mean

The expected value of the random vector composed of standard multivariate normal is a vector of zeroes:

Covariance

For random variables, the covariance matrix is an identity matrix.

General Multivariate Normal

Given that is the random vector of standard multivariate normal; any multivariate normal random vector can take the form:

Where is the random vector comprised of random variables with general normal distribution; vector has a length of . is an invertible matrix. And is a vector of constants.

Because is invertible, then

Mean

The mean of a generate multivariate normal is since the mean for the standard multivariate normal is :

Covariance

The covariance is where is the transpose of :

Jacobian

Notice that is in fact a transformation, therefore its corresponding Jacobian is simply

Density

Now that we know the Jacobian, it follows that the density is given by

Plugging in , and we obtain

Note that the covariance matrix , which also implies .

Also note that the determinant of the covariance matrix is , working it out we see that .

Plugging these equations in, the above density simplify down to

Properties of Multivariate Normal

1. Linear transformation of normal vectors results in normal vectors

Suppose we have a vector , and a matrix is full rank. then let . The mean of is ; the variance is .

2. Marginal distributions are normal

Suppose we have a random vector that has size 2:

Then we get

3. Conditional distributions are normal

Using the previous case, suppose we have as a realization for , then the conditional on is

Where and .