Updated 2017-11-05
Given an uniform random variable, how do we generate random numbers with a specific probability distribution?
To generate discrete random variables with distribution , for , we follow the following algorithm.
To generate continuous random variables with distribution , follow these steps:
“Pseudo” Proof: Suppose we have the uniform random variable and we use the inverse distribution function to obtain random . Note that the distribution function for is, by definition, .
Which is what we wanted.
Note: This method does not work directly on Gaussian random variables, use Polar Algorithm for Gaussian random variables instead.
As noted above, we can’t use inverse transformation method for Gaussian random variables. In order to simulate continuous random variable to have the distribution , we need to follow these steps:
To obtain a random variable with non-standard normal distribution , we use the linear properties of the Gaussian random variables.
Use this method if the distribution function is composed of a sum of other distribution functions :
Where are CDFs.
To get a random variable with distribution , follow these steps:
We can use this method if it’s hard to use previous methods to generate a random variable with distribution .
So we find a proposal distribution where it is easy to sample. Note that we must ensure the range of the density functions, and are the same.
Next, we need to know the upper bound , where .
With all that in mind, we may begin the algorithm:
Note: The probability of being accepted is , therefore we should always choose to be as small as possible given .
Example: half normal distribution
The half normal distribution function has the density . The inverse CDF is very difficult to find, so we use the A-R method.
The range of the density of this distribution spans , so it would be appropriate to choose the exponential random variable distribution as our proposal distribution. The density of the proposal distribution is .
Thus
Taking the derivative and the second derivative, we find the maximum is at . Thus, our upper bound is
Now we apply the A-R algorithm. First we generate an exponential random variable, .
Next, we generate .
Lastly, we set if
Suppose we have a vector consists of two random variables where .
If and are independent. Great! Then we can just generate each of them independently.
If not, then the algorithm is as follows:
Express the joint PDF of the two random variables as a conditional PDF
For do:
Example:
Suppose and , then the marginal PDF for is
Next, using the marginal PDF for , we can find the conditional PDF:
Following the rest of the algorithm, we sample and sample for all ’s.