Economics 8300

Lecture Note 3: Random Variables (CB 1.4–1.6, 2.1)

Often it is convenient to work with numbers rather than events. To do this we use random vari-
ables:

Definition 1 A random variable is a function from the sample space to the real numbers. RV=random
variable
Example 1: Toss a coin three times. The sample space is

Ω = {H H H , H H T, H T H , H T T, T H H , T H T, T T H , T T T }.

We can define a random variable X by taking the number of heads in the three tosses. Thus the
random variable can take on values 0,1,2,3. So, for example, X ({H H T }) = 2. 

Definition 2 The cumulative distribution function (CDF) of a random variable X , denoted by

F X (x), is be careful about
F X (x) = P ({ω ∈ Ω : X (ω) ≤ x}). the distinction
between X and x

This is often written more informally as P (X ≤ x). sometimes we also

write P r for P
Example 1 continued: Here the CDF is


 0 x <0


 1/8


 0≤x <1
F X (x) = 4/8 1≤x <2





 7/8 2≤x <3

1 3 ≤ x.


Some useful properties of CDFs:

Proposition 1 The function F (x) is a CDF if and only if:

1. limx→−∞ F (x) = 0 and limx→∞ F (x) = 1.

2. F (x) is nondecreasing.

3. F (x) is right-continuous; that is, for every x 0 , limx↓x0 F (x) = F (x 0 ). x ↓ x 0 means x

approaching x 0
from above
Definition 3 A random variable X is discrete if F X (x) is a step function of x. A random variable X
is continuous if F X (x) is a continuous function of x.

1
Discrete random variables take on a countable (finite or infinite) number of values. Our example
above is clearly a discrete random variable. Continuous random variables take on values in some
continuous range.

Definition 4 The probability mass function (PMF) of a discrete random variable X is given by

f X (x) = P (X = x).

So, for any set A, the probability

P ({ω ∈ Ω : X (ω) ∈ A}) = P (X ∈ A) =

X
f X (x).
x∈A

P P
In particular, P (X ≤ b) = x≤b f X (x). Also, note that with A = (−∞, ∞), we have x∈A f X (x) =
P (ω ∈ Ω) = 1, so the probability function always adds up to unity.

Example 1 continued: The probability mass function here is



 1/8 x =0


 3/8


 x =1
f X (x) = 3/8 x =2





 1/8 x =3

0 otherwise.


For continuous random variables, which take on an uncountable number of values, there is an
analogous idea:

Definition 5 A function f X (x) is a probability density function (PDF) of a continuous random

variable if for all events A, the probability
Z
P (X ∈ A) = P ({ω ∈ Ω : X (ω) ∈ A}) = f X (x)d x.
A

Note that the probability density function is not unique. Because we only care about integrals
over the probability density function, we can change its value at a countable number of points
without changing any of the associated probabilities, and thus without changing the distribution
of the random variable.

Also note that the probability density function integrates to one, in contrast to the probability
function which sums to one.1
1 Since the PDF and PMF work similarly, many people just use “probability density function” to refer to both con-

2
By the definition of PDF, we have that
Z x
P (X ≤ x) = F X (x) = f X (t )d t .
−∞

So, if f is continuous, by the First

d Fundamental
F X (x) = f X (x).
dx Theorem of
Calculus
Example 2: Consider the experiment of picking a point randomly on the interval from zero to
two and defining the random variable X as the distance to zero. It may be reasonable to assign
the probability to the point picked being in any subinterval as being proportional to the length
of that interval. In that case the CDF is:

 0

 x <0
F X (x) = x/2 0≤x <2


 1 2 ≤ x.

and the PDF is: 

 0

 x <0
f X (x) = 1/2 0≤x <2


 0 2 ≤ x.

Not all random variables taking on an uncountable number of values are continuous. There
are mixed random variables, which are partly continuous and partly discrete. For example, a
variable such as hours worked per year, or expenditures on cars, might have some positive mass
at 0, and be continuously distributed for values greater than zero. This creates some conceptual
difficulties in defining what the probability (mass or density) function is, but one can always
work with the CDF.

Suppose we have a random variable X with distribution function F X (x) and PMF or PDF f X (x).
Sometimes we are interested in the distribution of a function of this random variable, say Y =
g (X ). The distribution of this new random variable is defined by the equation

P (Y ∈ A) = P (g (X ) ∈ A) = P ({ω ∈ Ω : g (X (ω)) ∈ A}).

We are interested in calculating the PDF and PMF/PDF for this transformation. The following
result gives the general case:

tinuous and discrete cases. In measure-theoretic probability, there is no need to make a distinction between the two,
so this saves a bit of excess terminology.

3
Result 1 Define the mapping g −1 (A) by:

g −1 (A) = {x : g (x) ∈ A}.

Then

F Y (y) = P ({ω ∈ Ω : Y (ω) ∈ (−∞, y]})

= P ({ω ∈ Ω : g (X (ω)) ∈ (−∞, y]})
= P ({ω ∈ Ω : X (ω) ∈ g −1 ((−∞, y])}).

This result is not always that easy to apply. More useful results are available when the transfor-
mation satisfies particular conditions. Suppose that X is a random variable with PDF or PMF
f X (x). Let
X = {x : f X (x) > 0}.

This is called (by CB) the support set or support of the distribution of X , and intuitively is the set
of values that the RV X can take on. (See below for some more discussion of supports.)

Result 2 Suppose g (x) is a strictly monotone function with inverse g −1 (·). Let Y = {y : g (x) =
y for some x ∈ X }. Then
(i) If X is a discrete random variable with PMF f X (x) on X , then Y is a discrete random variable
with PMF
f Y (y) = f X (g −1 (y)),

on Y ,
(ii) If X is a continuous random variable with PDF f X (x) on X , then Y is a continuous random
variable with PDF
¯ ∂g −1 ¯
f Y (y) = f X (g −1 (y)) · ¯ (y)¯,
¯ ¯
∂y
on Y .

The argument for the continuous case goes as follows, assuming g (·) is increasing,

F Y (y) = P (Y ≤ y) = P (g (X ) ≤ y)

= P (x ≤ g −1 (y)) = F X (g −1 (y)).

Now use the chain rule to take the derivative with respect to y to get the desired result.

Example 3: Suppose X has a Poisson distribution with parameter λ. That is, the PMF of X is This is an example
of a parametric
family of
distributions
4
 e −λ λx
(
x! x = 0, 1, 2, . . .
f X (x) =
0 otherwise,

for some positive λ. This is a distribution that is often used for counts of events, for example
counts of emissions of radioactive particles, counts of job offers, number of patents, etc. Differ-
ent choices for λ lead to different probability distributions, but they all have a similar form.

Suppose we are interested in the distribution of Y = g (X ) = 2 · X . The inverse of the transforma-

tion is X = g −1 (Y ) = Y /2. The PMF of Y is

e −λ λ(y/2)
(
(y/2)! y = 0, 2, 4, . . .
f X (x) =
0 otherwise.

 note how Y
reflects the
Example 4: Suppose X has an exponential distribution with PDF transformation

(
e −x x >0
f X (x) =
0 otherwise.

The CDF for this distribution is F X (x) = 1 − e −x . This distribution, or rather a generalization with
f X (x) = λ exp(−λx), for positive λ, is widely used for modelling durations such as survival times
after heart transplants, or durations of unemployment spells.

Suppose we are interested in the distribution of Y = g (X ) = 1−e −X . This is a monotone transfor-

mation with inverse X = g −1 (Y ) = − ln(1−Y ). The derivative of the inverse of the transformation
is
∂g −1 1
(y) = .
∂y 1− y
The range of Y is Y = (0, 1). The PDF of Y is

 f (g −1 (y)) · ¯¯ ∂g −1 (y)¯¯ = exp(ln(1 − y)) · 1 = 1
¯ ¯
X ∂y 1−y 0<y <1
f Y (y) =
 0 otherwise.

This is known as a uniform distribution (the PDF is constant over the unit interval). 

Example 5: Finally consider a non–monotone transformation. Let X be a random variable with

PDF (
1/2 −1 < x < 1
f X (x) =
0 otherwise,

a uniform distribution on the interval [−1, 1]. Suppose we are interested in the distribution of
Y = g (X ) = X 2 . Because of the non–monotonicity we have to do this on a more ad hoc basis. In
this example we work directly through the cumulative distribution functions. First we calculate

5
the CDF for X : 
 0

 x ≤ −1
F X (x) = (x + 1)/2 −1 < x ≤ 1


 1 1 < x.

Then the CDF for Y = X 2 is

p p p p
P (Y ≤ y) = P (X 2 < y) = P (− y ≤ X ≤ y) = F X ( y) − F X (− y).

Hence 
 0

 y ≤0
p p p
F Y (y) = ( y + 1)/2 − (− y + 1)/2 = y 0<y ≤1


 1 1 < y.

The PDF is then (

y −1/2 /2 0<y <1
f Y (y) =
0 otherwise.

Note on Support of a Distribution:

The definition of the support X of a random variable X given in CB (and above) is a bit loose,
because when X is a continuous RV, its PDF f X (x) is not unique. A more precise definition is
that X should be the set of all points x such that every open neighborhood of x has positive
probability. A consequence of this definition is that supports are closed sets. For example, the
uniform distribution in Example 4 has support [0, 1]. In practice the informal definition usually
suffices.