ELEC 321

Random Variable Generation

Updated 2017-11-05

Given an uniform random variable, how do we generate random numbers with a specific probability distribution?

Inverse Transform Algorithm

Discrete Random Variables

To generate discrete random variables with distribution , for , we follow the following algorithm.

  1. Generate continuous uniform random variable
  2. Set based on where is on the distribution:

Continuous Random Variables

To generate continuous random variables with distribution , follow these steps:

  1. Generate continuous uniform random variable
  2. Find the inverse of the continuous distribution
  3. Set

“Pseudo” Proof: Suppose we have the uniform random variable and we use the inverse distribution function to obtain random . Note that the distribution function for is, by definition, .

Which is what we wanted.

Note: This method does not work directly on Gaussian random variables, use Polar Algorithm for Gaussian random variables instead.

Polar Algorithm

As noted above, we can’t use inverse transformation method for Gaussian random variables. In order to simulate continuous random variable to have the distribution , we need to follow these steps:

  1. Generate continuous uniform random variables and
  2. Set
  3. Set
  4. Set ; is now a random variable with distribution
  5. Set ; is now a random variable with distribution that is also independent from

To obtain a random variable with non-standard normal distribution , we use the linear properties of the Gaussian random variables.

  1. Generate using steps 1-4 above
  2. Set ; is now a random variable with distribution

Composition Method

Use this method if the distribution function is composed of a sum of other distribution functions :

Where are CDFs.

To get a random variable with distribution , follow these steps:

  1. Generate a random variable with discrete distribution for using the Inverse Transform method for discrete random variables
  2. Generate a random variable with distribution using the Inverse Transform method for continuous random variables
  3. Set =$Y_IX$ is now a random variable with distribution

Acceptance-Rejection Method

We can use this method if it’s hard to use previous methods to generate a random variable with distribution .

So we find a proposal distribution where it is easy to sample. Note that we must ensure the range of the density functions, and are the same.

Next, we need to know the upper bound , where .

With all that in mind, we may begin the algorithm:

  1. Generate a random variable from the distribution using methods previously described
  2. Generate a uniform random variable
  3. Check if
    • If true (accept), then set
    • Else (reject), go back to step 1 and try again

Note: The probability of being accepted is , therefore we should always choose to be as small as possible given .

Example: half normal distribution

The half normal distribution function has the density . The inverse CDF is very difficult to find, so we use the A-R method.

The range of the density of this distribution spans , so it would be appropriate to choose the exponential random variable distribution as our proposal distribution. The density of the proposal distribution is .

Thus

Taking the derivative and the second derivative, we find the maximum is at . Thus, our upper bound is

Now we apply the A-R algorithm. First we generate an exponential random variable, .

Next, we generate .

Lastly, we set if

Vector Random Variables Generation

Suppose we have a vector consists of two random variables where .

If and are independent. Great! Then we can just generate each of them independently.

If not, then the algorithm is as follows:

  1. Express the joint PDF of the two random variables as a conditional PDF

  2. For do:

    1. Generate
    2. Generate

Example:

Suppose and , then the marginal PDF for is

Next, using the marginal PDF for , we can find the conditional PDF:

Following the rest of the algorithm, we sample and sample for all ’s.