Random Variables

Statistics (ECON 511)

Josef Ruzicka

Department of Economics
School of Sciences and Humanities
Nazarbayev University

1 / 49
Overview
1 Random variables – basic notions

2 Jointly distributed random variables

3 Independent random variables

4 Conditional distributions

5 Expectation and other moments

6 Inequalities and weak law of large numbers

7 Problems

• Reading: Ross, Chapter 4

2 / 49
Random variables

• It is often useful to work with quantities determined by an experiment.

• For example, when measuring the lifetime of 100 lightbulbs, we can ask
how many keep working at least 200 days. We get this quantity once the
experiment is over.

Definition
A random variable is a quantity determined by the result of an
experiment.

Example
We throw three dice and denote X the sum of the numbers that show
up. X is a random variable. X can take values in {3, 4, . . . , 18}. For
instance, P(X = 4) = P(two dice give 1, one die gives 2) = 633 . In
principle, we could calculate P(X = k) for any k.

3 / 49
Random variables
Example
Let X be the number of visits a website of a company registers in a day.
X can take values 0, 1, 2, 3, . . .

Example
Let X be the time (in days) that a newly started device keeps working
before failure. Suppose it satisfies

P(device still working after x days) = e −0.2x , where x ≥ 0

So the probability that the device will work for at least two days is
e −0.2·2 ≈ 67%.
• When a random variable takes values in a countable set, we call it a
discrete random variable. (The first and second example).
• When a random variable takes values in a continuum of possible values,
we call it a continuous random variable. (The third example.)
4 / 49
Cumulative distribution function
Definition
For any random variable X , we define its cumulative distribution
function or just distribution function F as

F (x) = P(X ≤ x)

• When F is the distribution function of X , we write X ∼ F .

Example
A random variable X has the distribution function
(
0 if x ≤ 0
F (x) = −
√
x
1−e if x > 0

What is the probability that X > 2? √

P(X > 2) = 1 − P(X ≤ 2) = 1 − F (2) = e − 2 ≈ 24%

5 / 49
Probability mass function

Definition
For any discrete random variable X , we define its probability mass
function p as
(
P(X = a) when a is in the set of possible values of X
p(a) =
0 otherwise

• The probability mass function is often abbreviated pmf.

• The probability mass function is positive for at most countably many
different values, which we can denote x1 , x2 , x3 , . . . . We can write
p(xi ) = P(X = xi ), i = 1, 2, 3, . . .
X∞
• Also, it holds p(xi ) = 1
i=1

6 / 49
Probability mass function

Example
Consider a random variable X that can only take values 10, 20, 30, 40 and
p(10) = 81 , p(20) = 12 , p(30) = 81 , p(40) = 41 .

Example
Consider a random variable X that can only take values 0, 1, 2, 3, . . . and
1
p(k) = 2k+1 for k = 0, 1, 2, 3, . . .

7 / 49
Probability density function
Definition
For any continuous random variable X , we define its probability density
function f as
Z
P(X ∈ B) = f (x)dx where B is any set of real numbers
B

• The probability density function is often called density or pdf.

• We can calculate the probability that X will fall into the set B by
integrating f over B.
R R∞
• When we take B = R, then 1 = P(X ∈ R) = R f (x)dx = −∞ f (x)dx
• When we take B = [a, b], where a ≤ b, then
Rb
P(a ≤ X ≤ b) = a f (x)dx
• When we take B = [a, b], where
R aa = b, then
P(a ≤ X ≤ b) = P(X = a) = a f (x)dx = 0

8 / 49
Probability density function
• The density is closely related to the distribution function.
• If we take B = (−∞, a], then
Z a
F (a) = P(X ∈ (−∞, a]) = P(X ≤ a) = f (x)dx
−∞

• Differentiating the last equation w.r.t. a we get

Z a
′ ∂
F (a) = f (x)dx = f (a)
∂a −∞
• This means that the density is the derivative of the distribution
function.
• Take any a ∈ R and a small number ε > 0. Then
Z a+ ε Z a+ ε
 h ε ε i 2 2
P X ∈ a − ,a + = f (x)dx ≈ f (a)dx = εf (a)
2 2 a− ε a− ε
2 2

• So f (a) tells us approximately how likely it is that X will occur near a.

9 / 49
Probability density function
Example
Let X be a continuous random variable with density
(
C (x + (x − 1)2 ) when x ∈ [0, 2]
f (x) =
0 otherwise

(a) What is the value of C ?

We need to integrate f over [0, 2] and set it equal to 1. But first let’s find
Z 2 2
x2 (x − 1)3 22 (2 − 1)3
 2
(0 − 1)3

2 0 8
(x + (x − 1) )dx = + = + − + =
0 2 3 0 2 3 2 3 3

Thus, Z 2
8 3
C (x + (x − 1)2 )dx = C = 1, so C = .
0 3 8
(b) What is P(X > 1)?
Z 2
3 22 (2 − 1)3
 2
(1 − 1)3
 
3 1 11
(x + (x − 1)2 )dx = + − + =
1 8 8 2 3 2 3 16

10 / 49
Jointly distributed random variables
• Following an experiment, we often recover several quantities and are
interested in their relationship.
• For example, when measuring the lifetime of light bulbs, we can study
if and how it depends on temperature, voltage etc.
Definition
For random variables X and Y , we define the joint cumulative
distribution function F as F (x, y ) = P(X ≤ x, Y ≤ y )
• F is also called just joint distribution function.
• If we know the joint distribution function F of X and Y , we can find
the distribution function FX of X as follows:
FX (x) = P(X ≤ x) = P(X ≤ x, Y ≤ ∞) = F (x, ∞)
• Similarly, the distribution function FY of Y is
FY (y ) = P(Y ≤ y ) = P(X ≤ ∞, Y ≤ y ) = F (∞, y )
• We call FX the marginal distribution function of X (and FY the
marginal distribution function of Y ). 11 / 49
Jointly distributed random variables
• Although we can always recover marginal distribution functions from
joint distribution function, in general we can’t recover joint distribution
function from marginal distribution functions.
• F is more informative than FX and FY because F shows how X and Y
interact, while FX and FY don’t.
Definition
When X and Y are both discrete random variables with possible values
x1 , x2 , . . . and y1 , y2 , . . . , respectively, the joint probability mass
function p is p(xi , yj ) = P(X = xi , Y = yj )
• We can recover the probability mass function pX of X from the joint
probability mass function as follows:
X X
pX (xi ) = P(X = xi ) = P(X = xi , Y = yj ) = p(xi , yj )
j j

• Similarly, we get the probability mass function pY of Y :

X X
pY (yj ) = P(Y = yj ) = P(X = xi , Y = yj ) = p(xi , yj )
i i 12 / 49
Jointly distributed random variables
Example
Marketing department classifies customers into categories: man (represented by
X = 1), woman (represented by X = 2), teenager (represented by Y = 1), adult
(represented by Y = 2), and senior (represented by Y = 3). Their proportions:

Y=1 Y=2 Y=3

X=1 0.1 0.2 0.1
X=2 0.1 0.3 0.2

The table means the joint probability mass function p is p(1, 1) = 0.1,
p(1, 2) = 0.2, p(1, 3) = 0.1, p(2, 1) = 0.1, p(2, 2) = 0.3, p(2, 3) = 0.2. We get
(
0.1 + 0.2 + 0.1 = 0.4 when x = 1
pX (x) =
0.1 + 0.3 + 0.2 = 0.6 when x = 2

0.1 + 0.1 = 0.2
 when y = 1
pY (y ) = 0.2 + 0.3 = 0.5 when y = 2

0.1 + 0.2 = 0.3 when y = 3


13 / 49
Jointly distributed random variables

Definition
We say that random variables X and Y are jointly continuous if there
exists a function f (x, y ) : R2 7→ R such that for any set C ⊂ R2
ZZ
P((X , Y ) ∈ C ) = f (x, y )dxdy
(x,y )∈C

We call f the joint probability density function of X and Y .

• Joint probability density function is often called joint density.

• When C takes the form C = {(x, y ) : x ∈ A, y ∈ B} where A, B ⊂ R
Z Z
P((X , Y ) ∈ C ) = P(X ∈ A, Y ∈ B) = f (x, y )dxdy
B A

14 / 49
Jointly distributed random variables
• Also, when A = (−∞, a], B = (−∞, b],
F (a, b) = P(X ≤ a, Y ≤ b) = P(X ∈ (−∞, a], Y ∈ (−∞, b])
Z b Z a
∂2
= f (x, y )dxdy =⇒ f (a, b) = F (a, b)
−∞ −∞ ∂a∂b
• When X and Y are jointly continuous, they are individually continuous,
too.
• We can get the density fX of X by integrating out Y over R:
Z ∞Z
P(X ∈ A) = P(X ∈ A, Y ∈ R) = f (x, y )dxdy
−∞ A
Z Z ∞ Z
= f (x, y )dy dx = fX (x)dx
A −∞ A
| {z }
fX (x)

• Similarly, the density fY of Y is

Z ∞
fY (y ) = f (x, y )dx
−∞
15 / 49
Jointly distributed random variables

Example
Consider random variables X and Y with joint density
( −x
e
when 0 ≤ x < ∞, 1 ≤ y ≤ e 2
f (x, y ) = 2y
0 otherwise

a) Find P(X > 1, Y > 2).

e2 Z e2
∞
e −x e −x ∞
Z Z
P(X > 1, Y > 2) = dxdy = − dy
2 1 2y 2 2y 1
Z e2   −1  Z e2
e 1 1 1 2
= 0− − dy = dy = log(y )|e2 =
2 2y 2e 2 y 2e
1 2 − log 2
(log (e 2 ) − log (2)) = ≈ 0.24
2e 2e

16 / 49
Jointly distributed random variables

Example (Continued)
b) Find P(X < Y ).
e2 Z e2
e −x y
e −x y
Z Z
P(X < Y ) = dxdy = − dy
1 0 2y 1 2y 0
Z e 2  −y  0  Z e2 
1 − e −y

e e
= − − − dy = dy ≈ 0.89
1 2y 2y 1 2y

where the last integral doesn’t have a closed form solution, but can be
calculated by mathematical software.

17 / 49
Jointly distributed random variables

Example (Continued)
c) Find P(X < a) where a ∈ R.
We know that X cannot take negative values, so P(X < a) = 0 for any
a < 0. Now let’s consider a ≥ 0.
e2 e2
a
e −x e −x a
Z Z Z
P(X < a) = P(0 ≤ X < a) = dxdy = − dy
1 0 2y 1 2y 0
e2
 −a Z 2
e − e0 1 − e −a e 1 1 − e −a
Z 
2
= − dy = dy = log y |e1
1 2y 2 1 y 2
1−e −a
= (log e 2 − log 1) = 1 − e −a
2

18 / 49
Independent random variables

Definition
We say that random variables X and Y are independent if for any two
sets A, B ⊂ R

P(X ∈ A, Y ∈ B) = P(X ∈ A)P(Y ∈ B)

• So X and Y are independent if and only if the events {X ∈ A} and

{Y ∈ B} are independent.
• Also, X and Y are independent if and only if
P(X ≤ a, Y ≤ b) = P(X ≤ a)P(Y ≤ b) for any a, b ∈ R.
• Denote F the joint distribution function of X and Y . Denote FX the
marginal distribution function of X and FY the marginal distribution
function of Y . Then we have that X and Y are independent if and
only if F (a, b) = FX (a)FY (b) for any a, b ∈ R.

19 / 49
Independent random variables

• For X and Y discrete, denote p the joint pmf of X and Y , while px the
pmf of X and pY the pmf of Y . Then X and Y are independent if and
only if
p(x, y ) = pX (x)pY (y ) for all x, y
• For X and Y jointly continuous, denote f the joint density of X and
Y , while fx the density of X and fY the density of Y . Then X and Y
are independent if and only if

f (x, y ) = fX (x)fY (y ) for all x, y

20 / 49
Independent random variables
• The independence notion can be extended to more than two variables.

Definition
We say that random variables X1 , X2 , . . . , Xn are independent if for any sets
A1 , A2 , . . . , An ⊂ R

P(X1 ∈ A1 , X2 ∈ A2 , . . . , Xn ∈ An ) = P(X1 ∈ A1 )P(X2 ∈ A2 ) · · · P(Xn ∈ An )

• When X1 , X2 , . . . , Xn are discrete, denote p their joint pmf and pi the

pmf of Xi . They are independent if and only if

p(x1 , x2 , . . . , xn ) = p1 (x1 )p2 (x2 ) · · · pn (xn ) for all x1 , x2 , . . . , xn ∈ R

• When X1 , X2 , . . . , Xn are jointly continuous, denote f their joint density

and fi the density of Xi . They are independent if and only if

f (x1 , x2 , . . . , xn ) = f1 (x1 )f2 (x2 ) · · · fn (xn ) for all x1 , x2 , . . . , xn ∈ R

21 / 49
Conditional distributions

• Conditional distributions describe the distribution of one variable given

the value of other variable(s).
• For example, conditional distribution can help us understand how the
distribution of personal income varies with age, or how the distribution
of daily temperature varies with atmospheric pressure.

Definition
When X and Y are discrete random variables, the conditional
probability mass function of X given Y is

pX |Y (x|y ) = P(X = x|Y = y )

22 / 49
Conditional distributions
Example
Marketing department classifies customers into categories: man (represented by
X = 1), woman (represented by X = 2), teenager (represented by Y = 1), adult
(represented by Y = 2), and senior (represented by Y = 3). Their proportions:

Y=1 Y=2 Y=3

X=1 0.1 0.2 0.1
X=2 0.1 0.3 0.2

For example,
0.1
pX |Y (1|1) = P(X = 1|Y = 1) = = 0.5
0.1 + 0.1
0.1
pY |X (1|1) = P(Y = 1|X = 1) = = 0.25
0.1 + 0.2 + 0.1
0.2 2
pX |Y (2|3) = P(X = 2|Y = 3) = =
0.1 + 0.2 3

23 / 49
Conditional distributions
Definition
If X and Y have joint density f (x, y ) and the density of Y is fY (y ), then the
conditional density of X given Y is
f (x, y )
fX |Y (x|y ) =
fY (y )

Example
Consider again random variables X and Y with joint density
( −x
e
when 0 ≤ x < ∞, 1 ≤ y ≤ e 2
f (x, y ) = 2y
0 otherwise

What is the conditional density of X given Y ?

R ∞ −x −x
We first need to find fY (y ) = 0 e2y dx = − e2y |∞ 1
0 = 0 − (− 2y ) =
1
2y .
e −x
2y
fX |Y (x|y ) = 1 = e −x
2y
In this example, fX |Y (x|y ) doesn’t depend on y , so X and Y are independent.
24 / 49
Expectation
Definition
For a discrete random variable X with possible values x1 , x2 , . . . , its
expectation is X
E[X ] = xi P(X = xi )
i
For a continuous random variable X with density f , its expectation is
Z ∞
E[X ] = xf (x)dx
−∞

• Expectation of X is a weighted average of its possible values, with weights

proportional to how likely such values are.
• Expectation is also called expected value or mean.

Example
We roll a die. What is the expectation of the number that we get?
X 1 1 1 1 1 1 7
E[X ] = xi P(X = xi ) = 1 · +2· +3· +4· +5· +6· =
6 6 6 6 6 6 2
i
25 / 49
Expectation

Example
Consider a random variable with density
(
3
(x + (x − 1)2 ) when x ∈ [0, 2]
f (x) = 8
0 otherwise

What is its expectation?

Z 2 Z 2
3
E[X ] = xf (x)dx = x (x + (x − 1)2 )dx =
0 0 8
3 x4 x3 x2 2 5
Z 2  
3 3 2
= (x − x + x)dx = − + =
8 0 8 4 3 2 0 4

26 / 49
Properties of expectation
Theorem
If X is a discrete random variable with possible values x1 , x2 , . . . , then for
any function g : R 7→ R
X
E[g (X )] = g (xi )P(X = xi )
i

If X is a continuous random variable with density f , then for any

function g : R 7→ R
Z ∞
E[g (X )] = g (x)f (x)dx
−∞

For any a, b ∈ R
E[aX + b] = aE[X ] + b

• The last property is the linearity of expectation.

27 / 49
Moments

Definition
For a random variable X , its nth moment is E[X n ].

• Moments reveal lot of information about a random variable.

• They are related to dispersion and whether the variable is right-skewed
or left-skewed.
• If X is a discrete random variable
X
E[X n ] = xin P(X = xi )
i

• If X is a continuous random variable with density f

Z ∞
n
E[X ] = x n f (x)dx
−∞

28 / 49
Expectation of sums of random variables

Theorem
If X1 , X2 , . . . , Xn are random variables, then

E[X1 + X2 + · · · + Xn ] = E[X1 ] + E[X2 ] + · · · + E[Xn ]

Example
We roll 10 dice. What is the expectation of the sum of the numbers that
we get?
There are many possible outcomes: the sum could take any value in
{10, 11, . . . , 60}. Calculating all the probabilities would be tedious.
However, using the result above, it follows that the expected sum is just
10 times the expectation of a single die, which we saw is 72 . Thus, the
expected sum is 10 · 27 = 35.

29 / 49
Variance
Definition
The variance of a random variable X is Var[X ] = E[(X − E[X ])2 ]

• An alternative formula for variance can be obtained:

Var[X ] = E[(X − E[X ])2 ] = E[X 2 − 2X E[X ] + E[X ]2 ]

= E[X 2 ] − E[2X E[X ]] + E[E[X ]2 ] = E[X 2 ] − 2E[X ]E[X ] + E[X ]2
= E[X 2 ] − 2E[X ]2 + E[X ]2 = E[X 2 ] − E[X ]2

Example
Let X be the result of rolling a die. What is its variance?
1 1 1 1 1 1 91
E[X 2 ] = 12 + 22 + 32 + 42 + 52 + 62 =
6 6 6 6 6 6 6
7 2
Earlier we saw E[X ] = 72 , so Var [X ] = E[X 2 ] − E[X ]2 = 91 35


6 − 2 = 12
30 / 49
Variance
Theorem
For a random variable X and any constants a, b ∈ R

Var[aX + b] = a2 Var[X ]

• Variance of a constant is zero, which follows from setting a = 0.

• Adding or subtracting a constant from X doesn’t change its variance –
this follows from setting a = 1.
• However, changing the scale changes variance nonlinearly. For example,
if X is measured in cm, and we want to convert it to mm by setting
a = 10, b = 0, we get that the variance is going to increase 102 = 100
times.

Definition
p
The standard deviation of a random variable X is Var[X ].

31 / 49
Covariance and variance of sums of random variables
Definition
For random variables X and Y , their covariance is defined by

Cov[X , Y ] = E[(X − E[X ])(Y − E[Y ])]

• An alternative formula is Cov[X , Y ] = E[XY ] − E[X ]E[Y ].

• For random variables X1 , X2 , Y
Cov[X1 + X2 , Y ] = Cov[X1 , Y ] + Cov[X2 , Y ]
• We can generalize this result:

Theorem
For random variables X1 , X2 , . . . , Xn , Y1 , Y2 , . . . , Ym
 
Xn Xm X n X m
Cov  Xi , Yj =
 Cov[Xi , Yj ]
i=1 j=1 i=1 j=1

32 / 49
Covariance and variance of sums of random variables
Theorem
For random variables X1 , X2 , . . . , Xn
" n # n n X
n
X X X
Var Xi = Var[Xi ] + Cov[Xi , Xj ]
i=1 i=1 i=1 j=1
j̸=i

• When n = 2, we get Var[X + Y ] = Var[X ] + Var[Y ] + 2Cov[X , Y ]

Theorem
When random variables X and Y are independent, Cov[X , Y ] = 0.

Theorem
For random variables X1 , X2 , . . . , Xn that are independent
" n # n
X X
Var Xi = Var[Xi ]
i=1 i=1
33 / 49
Correlation

• Covariance depends on the scale of X and Y , which makes it hard to

interpret and compare across different variables.
• Another measure of association between X and Y is the correlation.

Definition
The correlation of random variables X and Y is
Cov[X , Y ]
Corr[X , Y ] = p p
Var[X ] Var[Y ]

• Correlation is scale-invariant and its values are between −1 and 1.

34 / 49
Moment generating function
• Calculating moments of random variables can be tedious.
• If we need to calculate various moments of a random variable, it can usually
be done faster with the help of the moment generating function.

Definition
The moment generating function ϕ of a random variable X is
X


 e txi P(X = xi ) when X is discrete
ϕ(t) = E[e tX ] = Z i ∞


 e tx f (x)dx when X is continuous with density f
−∞

• To calculate the nth moment of X , we take the nth derivative of ϕ with

respect to t and evaluate it at t = 0, that is E[X n ] = ϕ(n) (0).
• The moment generating function of a sum of independent random variables is
the product of their moment generating functions, that is
ϕX1 +X2 +···+Xn (t) = ϕX1 (t)ϕX2 (t) · · · ϕXn (t) when X1 , X1 , . . . , Xn are independent
35 / 49
Inequalities

Theorem (Markov’s inequality)

Let X be a random variable that only takes non-negative values. Then
for any a > 0
E[X ]
P(X ≥ a) ≤
a

• The Markov’s inequality gives us an upper bound on the probability

that X exceeds the value a.

Theorem (Chebyshev’s inequality)

Let X be a random variable with mean µ and variance σ 2 . Then for any
k >0
σ2
P(|X − µ| ≥ k) ≤ 2
k

36 / 49
Inequalities
Example
The number of calls that a call center receives per hour has mean 10.
a) What can we say about the probability that at least 40 calls are received in
the next hour?
From Markov’s inequality
10
P(X ≥ 40) ≤ = 0.25
40
Thus, the probability that at least 40 calls are received is at most 25%.
b) Suppose we also know that the variance of the calls per hour is 8. What can
we say about the probability that the number of calls next hour will differ from
the mean by at least 4?
From Chebyshev’s inequality
8
P(|X − 10| ≥ 4) ≤ = 0.5
42
We conclude: The probability that the number of calls will differ from the mean
by at least 4 is at most 50%.
37 / 49
The weak law of large numbers

Theorem
Let X1 , X2 , . . . be a sequence of independent and identically distributed
random variables with mean µ. Then for any ε > 0
n
!
1X
P Xi − µ > ε → 0 as n → ∞
n
i=1

• It tells us that the sample mean x̄ converges to the population mean µ.

38 / 49
Problem 1
Let a > 0, µ > 0, c > 0 be constants. Consider the following function
(
cx a for x ∈ (0, µ)
f (x) =
0 otherwise

For which values of a and µ is the function f a probability density function? What is the corresponding value of c?

• For f to be a density, it must be nonnegative and integrate to 1. We know

a > 0, µ > 0, c > 0, so f is nonnegative.
• Let’s integrate f Z ∞ Z µ cx a+1 µ cµa+1
f (x)dx = cx a dx = =
−∞ 0 a+1 0 a+1
• The integral must be 1, so
cµa+1 a+1
= 1 =⇒ c = a+1
a+1 µ
• For any µ > 0, a > 0, we can see that c > 0. So f is a density for any a > 0 and µ > 0 and
the value of c is given above.
• Discussion of the distribution given by f :
• If a were equal to 0, it would be uniform on [0, µ].
• Relative to the uniform distribution, the distribution given by f puts more probability mass to the right as f is
increasing in x.
• The greater is the value of a, the more probability mass shifts to the right.

39 / 49
Problem 2
A child got lost in a forest whose shape is an equilateral triangle. We assume the probability that the child is in any region of
the forest is proportional to the area of that region. Denote the forest sides a, b and c. What is the distribution of the
distance of the child from forest side a?

• Let D be the distance of the child from forest side a. D is a random variable.
• Let h be the height of the triangle from side a.
• Distance is nonnegative, so P(D < 0) = 0.
• The distance cannot be greater than h, so P(D ≤ h) = 1.
• Take any d ∈ [0, h]. Let’s find the probability that D ≤ d.

√
• The area of the whole equilateral triangle equals h2 / 3
√
• The area of the smaller equilateral triangle equals (h − d)2 / 3
√
(h − d)2 / 3 d 2
 
P(D ≤ d) = 1 − P(D > d) = 1 − √ =1− 1−
2
h / 3 h
• The distribution of the distance of the child from forest side a is given by

0 if d < 0 2


P(D ≤ d) = 1 − 1 − dh if d ∈ [0, h]


1 if d > h

40 / 49
Problem 3
A new lottery is being designed. The number of tickets to be issued is N. Each ticket may be associated either with one prize
or with no prize. There are various prizes, each of them possibly included multiple times. Specifically, there are mi prizes of
value qi , where i ∈ {1, 2, . . . , k}. Determine the price of the lottery ticket so that its expected value equals half of the
ticket price.

Pk
• What is the value of all the tickets in total? i=1 mi qi .
• By symmetry, the expected value of every ticket is the same.
• Thus, the expected value of each ticket is N1 ki=1 mi qi .
P

• The price of the lottery ticket must be N2 ki=1 mi qi .

P

• Additional question: If all the tickets are sold, what is going to be the profit from
the lottery?
k k k
2 X X X
N mi qi − mi qi = mi qi
N i=1 i=1 i=1

• The profit is equal to the value of all the tickets; the profit is 50%.

41 / 49
Problem 4
Show that if a random variable X is independent of itself, then X is constant with probability one. (X being independent of
itself means X is independent of X .)

• Recall the definition of independence of two random variables X and Y .

• We say that X is independent of Y if for all x, y ∈ R it holds
P(X ≤ x, Y ≤ y ) = P(X ≤ x)P(Y ≤ y ).
• Suppose X is independent of X . Then the definition implies that for all x, y ∈ R it
holds P(X ≤ x, X ≤ y ) = P(X ≤ x)P(X ≤ y ). Let’s take the special case of x = y .
Then we have P(X ≤ x, X ≤ x) = P(X ≤ x)P(X ≤ x), which must hold for all
x ∈ R. This implies

P(X ≤ x, X ≤ x) = P(X ≤ x)P(X ≤ x)

P(X ≤ x) = P(X ≤ x)P(X ≤ x)
P(X ≤ x)(1 − P(X ≤ x)) = 0

• The last equation means that either P(X ≤ x) = 0 or P(X ≤ x) = 1. Thus, the cdf
of X can take only two values: zero or one. This is the cdf of a random variable that
is constant with probability one.

42 / 49
Problem 5
Let a > 0, µ > 0, c > 0 be constants. Consider the following function
(
cx −a for x > µ
f (x) =
0 otherwise

For which values of a and µ is the function f a probability density function? What is the corresponding value of c?

• For f to be a density, it must be nonnegative and integrate to 1. We know

µ > 0, c > 0, so f is nonnegative.
• Let’s integrate f Z Z∞ ∞
f (x)dx = cx −a dx
−∞ µ
• Let’s consider 3 cases: a ∈ (0, 1), a = 1 and a > 1. If a ∈ (0, 1), then
Z ∞
cx −a+1 ∞ cµ−a+1
cx −a dx = =∞− =∞
µ −a + 1 µ −a + 1
If a = 1, then
Z ∞ ∞
cx −1 dx = c log(x) = ∞ − c log(µ) = ∞
µ µ

If a > 1, then
∞
cx −a+1 cµ−a+1 cµ1−a
Z ∞
cx −a dx = =0− =
µ −a + 1 µ −a + 1 a−1
43 / 49
• The integral must be 1, so the only possibility is a > 1.
• For a > 1, we must have
Z ∞
cµ1−a
cx −a dx = =1
µ a−1

which implies c = (a − 1)µa−1

• For any µ > 0, a > 1, we can see that c > 0. So f is a density for any
a > 1 and µ > 0 and the value of c is given above.

44 / 49
Problem 6
For a random variable with pdf f , the median is defined as the number m ∈ R which satisfies

Z m Z ∞
1
f (x)dx = f (x)dx =
−∞ m 2

Find the median of the distribution with pdf

(
3x 2 if x ∈ [0, 1]
f (x) =
0 otherwise

• We must solve the following equation for m

Z m Z m
1
f (x)dx = 3x 2 dx =
−∞ 0 2

• We have
Z m m 1
3x 2 dx = x 3 = m3 =
0 0 2
1
• As a result, m = √
3
2
45 / 49
Problem 7
A child got lost in a forest whose shape is an equilateral triangle. We assume the probability that the child is in any region of
the forest is proportional to the area of that region. What is the distribution of the distance of the child from the nearest
forest side?

• Let D be the distance of the child from the nearest forest side. D is a random variable.
• Distance is nonnegative, so P(D < 0) = 0.
• Let h be the height of the triangle.
• The maximal distance of the child from each side of the triangle occurs when the child is at the center. In this case the
distance from each side is h/3.
• The distance cannot be greater than h/3, so P(D ≤ h/3) = 1.
• Take any d ∈ [0, h/3]. Let’s find the probability that D ≤ d.

√
• The area of the whole equilateral triangle equals h2 / 3
√
• The area of the smaller equilateral triangle equals (h − 3d)2 / 3
√
(h − 3d)2 / 3 3d 2
 
P(D ≤ d) = 1 − P(D > d) = 1 − √ =1− 1−
h2 / 3 h
• The distribution of the distance of the child from forest side a is given by

0 if d < 0

 2
P(D ≤ d) = 1 − 1 − 3d h
if d ∈ [0, h/3]


1 if d > h/3
46 / 49
Problem 8
Let X and Y be discrete random variables with joint distribution given by the table below. (X can take values 1 or 2, Y can
take values 1, 2 or 3. The probabilities of the corresponding events are given inside the table.) For the two variables, find their
(i) marginal distributions,
(ii) expectations,
(iii) variances.

Y =1 Y =2 Y =3

X =1 0.1 0.2 0.3

X =2 0.2 0.1 0.1

• (i)

P(X = 1) = 0.1 + 0.2 + 0.3 = 0.6

P(X = 2) = 0.2 + 0.1 + 0.1 = 0.4
P(Y = 1) = 0.1 + 0.2 = 0.3
P(Y = 2) = 0.2 + 0.1 = 0.3
P(Y = 3) = 0.3 + 0.1 = 0.4

• (ii) E[X ] = 1 · 0.6 + 2 · 0.4 = 1.4, E[Y ] = 1 · 0.3 + 2 · 0.3 + 3 · 0.4 = 2.1
• (iii) Var[X ] = E[(X − E[X ])2 ] = (1 − 1.4)2 · 0.6 + (2 − 1.4)2 · 0.4 = 0.24,
Var[Y ] = E[(Y − E[Y ])2 ] = (1 − 2.1)2 · 0.3 + (2 − 2.1)2 · 0.3 + (3 − 2.1)2 · 0.4 = 0.69

47 / 49
Remark on previous home assignment
• Recall the Gini index G satisfies G = 1 − 2B,
1 s1 + s2 + · · · + sn−1
B= +
2n nsn

• Adding wealth A > 0 to each individual changes B to B̃,

1 (s1 + A) + (s2 + 2A) + · · · + (sn−1 + (n − 1)A)

B̃ = +
2n n(sn + nA)
n(n−1)A
1 s1 + s2 + · · · + sn−1 + 2
= +
2n n(sn + nA)
n(n−1) n(n−1)A
∂ B̃ 2 n(sn + nA) − n2 (s1 + s2 + · · · + sn−1 + 2 )
=
∂A n2 (sn + nA)2
n(n−1)
2 nsn − n2 (s1 + s2 + · · · + sn−1 )
=
n2 (sn + nA)2

48 / 49
 n(n−1)
∂ B̃ 2 nsn − n2 (s1 + s2 + · · · + sn−1 )
=
∂A n2 (sn + nA)2
(n−1) sn−1
2 − ( ssn1 + s2
sn + ··· + sn )
= (sn +nA)2
sn

sj
• Notice that sn ≤ nj , j ∈ {1, 2, . . . , n}. Thus,

(n−1) (n−1)
∂ B̃ 2 − ( n1 + 2
n + ··· + n−1
n ) 2 − ( n(n−1)
2n )
≥ (sn +nA)2
= (sn +nA)2
=0
∂A
sn sn

Since ∂∂A
B̃
≥ 0, it follows that B̃ is nondecreasing in A, so the Gini index
is nonincreasing in A. Hence, if the wealth of all individuals is raised by
A, the Gini index either stays the same or decreases.

49 / 49