ELEC 321

Discrete Random Vectors

Updated 2017-11-30

We are interested in joint behavior of multiple random variables. An example of random vector would be rolling two dies and we are looking at the joint behavior of two random variables.

There are continuous and discrete random variables.

Discrete Random Vectors

Joint Probability Mass Function

The probability for a realization of all considered random variables.

Example: consider the random vector from above (rolling two dies)

Out of all possibilities, sum of points being 6 has 6 possible outcomes: (1, 5), (2, 4), (3, 3), (4, 2), (1, 6). Thus the probability of is . Then out of those possibilities, only (3, 3) has the absolute difference 0, so .

Therefore the joint probabilities is

Marginal Densities

PMF that only takes interest in a single random variable in a random variable.

For instance if the random vector has 2 elements, then is the discrete joint PMF function.

Then the marginal densities is the PMF with the other variables (not of interest) summed up.

Mean of Random Vector

The mean/expected value of a random vector just applies to each individual random variables.

Covariance

The variance of the random vector can be described as covariance, in a covariance matrix.

Where the random vector subtracting the mean of the random vector, and is the transpose. This results in a matrix multiplication that returns a matrix:

Where for any ,

And for any and ,

The covariance of the two individual random variable is

Computing Covariance

Correlation Coefficient

It describes how correlated two random variables are. It is defined as follows.

Where .

Independent Random Variables

The random variables are independent from each other if and only if

Consider random variables and , if they are independent then,

Covariance is 0 if they are independent.

Note that independence of and , but that Y$$ are independent.

Conditional PMF

Consider two random variables and . Then the conditional PMF is:

Thus, it’s clear to see that

Independence

If and are independent, then

It follows that

and similarly

Conditional Mean and Variance

Conditional Mean is the expected value of one random variable given the realization of another random variable.

Conditional Variance:

Independence

If events and are independent, then:

Conditional Mean As A Function

Generally the conditional mean is expressed as , which is a function of . Thus, we can express that as:

Since is only a realization of a random variable, we can consider as a random function as:

Two Step Average

We may find the conditional mean of a random variable in two steps.

Where and . It follows that

Total Variance

Consider the following plot of two random variables and . Where the red circles represent and the red whiskers represents the variance . Then the blue circle is the overall mean and the blue whisker is the overall variance .

In this case, the Total Variance is given by

In the example above, the unexplained variance is the inner variance (red whiskers). The explained variance is the variance due to the red dots increasing.

The Percentage of Explained Variance is given by:

This gives an idea of how good a prediction is. If the percentage of explained variance is closer to 1, then one could use it for a good prediction. On the other hand, a percentage of near 0 is useless.

For the best prediction, use .