Updated 2017-11-30
We are interested in joint behavior of multiple random variables. An example of random vector would be rolling two dies and we are looking at the joint behavior of two random variables.
There are continuous and discrete random variables.
The probability for a realization of all considered random variables.
Example: consider the random vector from above (rolling two dies)
Out of all possibilities, sum of points being 6 has 6 possible outcomes: (1, 5), (2, 4), (3, 3), (4, 2), (1, 6). Thus the probability of is . Then out of those possibilities, only (3, 3) has the absolute difference 0, so .
Therefore the joint probabilities is
PMF that only takes interest in a single random variable in a random variable.
For instance if the random vector has 2 elements, then is the discrete joint PMF function.
Then the marginal densities is the PMF with the other variables (not of interest) summed up.
The mean/expected value of a random vector just applies to each individual random variables.
The variance of the random vector can be described as covariance, in a covariance matrix.
Where the random vector subtracting the mean of the random vector, and is the transpose. This results in a matrix multiplication that returns a matrix:
Where for any ,
And for any and ,
The covariance of the two individual random variable is
It describes how correlated two random variables are. It is defined as follows.
Where .
The random variables are independent from each other if and only if
Consider random variables and , if they are independent then,
Covariance is 0 if they are independent.
Note that independence of and , but that Y$$ are independent.
Consider two random variables and . Then the conditional PMF is:
Thus, it’s clear to see that
If and are independent, then
It follows that
and similarly
Conditional Mean is the expected value of one random variable given the realization of another random variable.
Conditional Variance:
If events and are independent, then:
Generally the conditional mean is expressed as , which is a function of . Thus, we can express that as:
Since is only a realization of a random variable, we can consider as a random function as:
We may find the conditional mean of a random variable in two steps.
Where and . It follows that
Consider the following plot of two random variables and . Where the red circles represent and the red whiskers represents the variance . Then the blue circle is the overall mean and the blue whisker is the overall variance .
In this case, the Total Variance is given by
In the example above, the unexplained variance is the inner variance (red whiskers). The explained variance is the variance due to the red dots increasing.
The Percentage of Explained Variance is given by:
This gives an idea of how good a prediction is. If the percentage of explained variance is closer to 1, then one could use it for a good prediction. On the other hand, a percentage of near 0 is useless.
For the best prediction, use .