University of Mumbai

INSTITUTE OF DISTANCE AND OPEN LEARNING

PSMT/PAMT 205

PROBABILITY THEORY
M.Sc. (MATHEMATICS)
SEMESTER - II
PAPER - V
 © UNIVERSITY OF MUMBAI

PROBABILITY THEORY
M.Sc. (Mathematics) SEMESTER - II
PAPER - V
Dr. Suhas Pednekar
Vice Chancellor
University of Mumbai, Mumbai

Dr. Prakash Mahanwar Dr. Sanjay Ratnaparakhi

Director Incharge Asst. Director & Incharge
Institute of Distance & Open Learning, Study Material Section,
University of Mumbai, Mumbai University of Mumbai, Mumbai

Programme Co-ordinator, I/c Faculty of Science & Technology

Asst. Prof. Mandar Bhanushe
Department of Mathematics, University of Mumbai, IDOL

Course Writers
Prof. Kalpana Phal
B. N. Bandodkar College, Thane.

Published by
Dr. Prakash Mahanwar, Director Incharge
on behalf of Institute of Distance & Open Learning,University of Mumbai, Mumbai
and Printed at
Name & Address.
DTP Composed by 7skills

ii
 CONTENTS
Unit No. Title Page No.

1. Basics of Probability .............................................................................................. 1

2. Fields and Sigma fields ........................................................................................ 12

3. Probability Measure and Lebesgue Measure...................................................... 20

4. Conditional Probability and Independence ....................................................... 32

5. Random variable and its distribution function ................................................. 43

6. Some Special Random variables and their distributions .................................. 53

7. Two Dimensional R.V.S. ...................................................................................... 62

8. Expectation, Variance and their Properties ........................................................ 72

9. Theorems on expectation and Conditional Expectation................................... 84

10. Law of Large Numbers and Central Limit Theorem ......................................... 91

iii
 PROBABILITY THEORY
M.SC. (MATHEMATICS) SEMESTER - II PAPER - V
SYLLABUS

PSMT205/PAMT205 PROBABILITY THEORY

Course Outcomes

1. Students will understand the concept of Modelling Random Experiments, Classical

probability spaces, -fields generated by a family of sets, -field of Borel sets, Limit
superior and limit inferior for a sequence of events.

2. Students will be able to know about probability measure, Continuity of probabili-

ties, First Borel-Cantelli lemma, Discussion of Lebesgue measure on -field of Borel
subsets of assuming its existence, Discussion of Lebesgue integral for non-negative
Borel functions assuming its construction.

3. Students will be able to earn knowledge of discrete and absolutely continuous prob-
ability measures, conditional probability, total probability formula, Bayes formula.

4. Students will learn distribution of a random variable, distribution function of a

random variable, Bernoulli, Binomial, Poisson and Normal distributions,

5. Students will be able to understand Chebyshev inequality, Weak law of large num-
bers, Convergence of random variables, Kolmogorov strong law of large numbers,
Central limit theorem and Application of Probability Theory.

Unit I. Probability basics (15 Lectures)

Modelling Random Experiments: Introduction to probability, probability space, events.
Classical probability spaces: uniform probability measure, fields, finite fields, finitely
additive probability, Inclusion-exclusion principle, σ-fields, σ-fields generated by a family
of sets, σ-field of Borel sets, Limit superior and limit inferior for a sequence of events.

Unit II. Probability measure (15 Lectures)

Probability measure, Continuity of probabilities, First Borel-Cantelli lemma, Discussion
of Lebesgue measure on σ-field of Borel subsets of assuming its existence, Discussion of
Lebesgue integral for non-negative Borel functions assuming its construction. Discrete
and absolutely continuous probability measures, conditional probability, total probability
formula, Bayes formula, Independent events.

Unit III. Random variables (15 Lectures)

Random variables, simple random variables, discrete and absolutely continuous random
variables, distribution of a random variable, distribution function of a random variable,
Bernoulli, Binomial, Poisson and Normal distributions, Independent random variables,
Expectation and variance of random variables both discrete and absolutely continuous.

iv
Unit IV. Limit Theorems (15 Lectures)
Conditional expectations and their properties, characteristic functions, examples, Higher
moments examples, Chebyshev inequality, Weak law of large numbers, Convergence of
random variables, Kolmogorov strong law of large numbers (statement only), Central
limit theorem (statement only).

Recommended Text Books

1. M. Capinski, Tomasz Zastawniak: Probability Through Problems.
2. J. F. Rosenthal: A First Look at Rigorous Probability Theory, World Scientic.
3. Kai Lai Chung, Farid AitSahlia: Elementary Probability Theory, Springer Verlag.
4. Ross, Sheldon M. A first course in probability(8th Ed), Pearson.

v
 1
BASICS OF PROBABILITY
Unit Structure

1.0 Objectives

1.1 Introduction

1.2 Some Terminologies and notations

1.3 Different Approaches of Probability

1.4 Chapter End Exercises

1.0 Objectives

After going through this chapter you will learn

x What is random experiment? How it forms the basis for the ͆probability͇
x Notion of sample space and its types
x Various types of events
x Operations of the events and the laws these operations obey.
x Mathematical and Statistical definition of probability and their limitations.

1.1 Introduction
In basic sciences we usually come across deterministic experiments whose results
are not uncertain. Theory of probability is based on Statistical or random
experiments. These experiments have peculiar features

Definition 1.1. Random Experiment: A non deterministic experiment is called as

a random experiment if
1. It is not known in advance, what will be the result of a performance of trial
of such experiment.
2. It is possible to list out all possible of this experiment outcomes prior to
conduct it.
3. Under identical conditions, it w possible to repeat such experiment as many
times as one wishes.

1
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Definition 1.2. Sample space: Collection of all possible outcomes of a random

experiment is known as sample space
Sample space is denoted by : . And an element of : by Z
1. Each Z represents a single outcome of the experiment.
2. Number of elements of : are called sample points, and total number of
sample points are denoted by #  : 

3. Number of elements of : may be finite, or it may have one one

correspondence with the  , or with .
4. Depending on its nature : is called as finite, countable or uncountable.

Example 1.1.

1. A roulette wheel with pointer fixed at the center is spinned. When it comes to
rest, the angle made by pointer with positive direction is noted. This
experiment is random. Since we do not know before spinning where the
pointer would rest. But it may make angle any where between (O, 3600) thus
here sample space :  0,360  It is subset of  . It w uncountable
0

2. A coin is tossed until it turns up head. Number of tosses before we get head
are noted. This is a random experiment. The corresponding sample space
:  0,1, 2,3} has one one correspondence with the  , so it is countable

3. A gambler enters a casino with initial capital ͆ C ͇ .If his policy is to

continuing to bet for a unit stake until, either his fortune reaches to ͆ C ͇
or his funds are exhausted . Gambler̓s fortune after any game is though
uncertain we can list it out. The sample space of this random experiment is
:  0,1, 2,3}, C  . Here sample space is finite.

1.2 Some Terminologies and notations

Event: Any subset of : is termed as an event. Thus corresponding to random

experiment a phenomenon may or may not be observed as a result of a random
experiment is called as an event.

Note: Event is made up of one or many outcomes Outcomes which entails

happening of the event is said to be favorable to the event. An event is generally

2
 Chapter 1: Basics of Probability

denoted by alphabets. Number of sample points in an event “ A ” is denoted by

#  A .

Algebra of events: Since events are sets algebraic operations on sets work for the
events.

x Union of two events: A and B are two events of : ,then their union is an
event representing occurrence of (at least one of them) A or B or both and
denoted by A ‰ B
Thus, A ‰ B { Z : Z  A or Z  B or  A and 13 both}

x Intersection of two events: A and B are two events of : ,then their

Intersection is an event representing simultaneous occurrence of A and B both
and denoted by A ˆ B
Thus, A ˆ B { Z : Z  A and Z  B }

x Complement of an event: Non occurrence of an event is its complementary

event. Complement of an event is denoted by ̺A. It contains Z that are not
in A. Thus , A { Z : Z does not belong to A }

x Relative complementarity: Out of the two events occurrence of exactly one

event is relative complement of the other. In particular if an event A occurs
but B does not, it is relative complement of B relative to A. It is denoted by
A  B or A ˆ B .This event contains all sample points of A that are not in B.
Similarly B  A or B ˆ A represents an event that contains all sample points
of 13 that are not in A. Thus , A  B { Z : Z  A and Z does not belong to
13 }

x Finite Union and Countable Union: A1 , A2 ,}, An be the events of the sample
n
space U i 1 Ai is called as finite union of the events.

f
If n o f we have U i 1Ai which is called as countable union of the events

x Finite intersection and Countable intersection: A1 , A2 ,}, An be the events of

n
the sample space  A is called as finite intersection of the events
i 1 i

f
If n o f we have  A which is called as countable intersection of the
i 1 i

events

3
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Laws of Operations: Union and intersection are the set operations, they obey
following laws.

x Commutative law i) A ‰ B B ‰ A and ii) A ˆ B Bˆ A

x Reflexive law i) A ‰ A A and ii) A ˆ A A

x Associative law i)  A ‰ B  ‰ C A ‰ B ‰ C
ii)  A ˆ B  ˆ C A ˆ B ˆ C

x Distributive law i) A ‰  B ˆ C   A ‰ B ˆ  A ‰ C
ii) A ˆ 13 ‰ C  A ˆ B ‰ A ˆ C

x 
De Morgan’s Law i) A ‰ B   A ˆ B . ii)  A ˆ B  A ‰ B
Impossible event: An event corresponding to an empty set.

Certain event: An event corresponding to :.

Mutually Exclusive Event: When occurrence of one event excludes the

occurrence of the other for all choices of it then the two events are called as
Mutually exclusive events. Alternately, when the two events do not occur
simultaneously then the two events are called as Mutually exclusive events. Here
A ˆ B M.

Exhaustive events: The two events are said to be Exhaustive events if they
together form the sample space Alternately when all sample points are included in
them they are called Exhaustive events. Here AUB :

Equally likely events: If we have no reason to expect any of the events in

preference to the others, we call the events as Equally likely events.

Indicator function: Indicator function of an event denoted by I A Z  and defined

as

­1 Z  A
I A Z  ® (1.1)
¯0 Z  A

Partition of sample space: A1 , A2 ,}, An be the events of the sample space such
that they are Mutually exclusive and Exhaustive then are said to form (finite)
partition of the sample space.

4
 Chapter 1: Basics of Probability

So, if A1 , A2 ,} An are forming partition of a sample space, for every i z j 1, 2,} n


; Ai ˆ Aj  M And ‰ A n
i 1 i
:

Note: Concepts of Mutually Exclusive Event and Exhaustive events and hence for
partition can be generalized for countable events A1 , A2 , }

Example 1.2. : ^e1 , e2 , e3 , e4 , e5 ` . If A ^e1 , e3 , e5 ` , and B ^e1 , e2 , e3 , e4 ` .

Answer the following (i) Are A, B mutually exclusive? (ii) Are A, B exhaustive9.
(iii) IfC ^e2 , e4 ` . find A ‰  B ˆ C  and A ˆ B  
Solution: (i) Since  A ˆ B  ^e1 , e3 ` which is non null, so A, B are not mutually
exclusive. (ii)  A ‰ B  : , so A, B are exhaustive.

(iii) A ‰  B ˆ C  : and  A ˆ B  C

1.3 Different Approaches of Probability

Definition 1.3. Classical or Mathematical definition (Leplace): If a random

experiment is conducted results into N mutually exclusive, exhaustive and equally
likely outcomes, M of which are favorable to the occurrence of the event A , then
M
probability of an event A is defined as the ratio , and denoted by P  A 
N

#  A M
P  A
#: N

This definition has limitations

1 It is not applicable when outcomes are not equally likely.

2 We may not always come across a random experiment that results into a finite
number of outcomes.
3 Even if outcomes are finite, can not be enumerated or the number favorable
to the event of interest may not be possible to count.

5
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Definition 1.4. Empirical or Statistical definition (Von Mises):If a random

experiment is conducted N times, out of which M times it results into outcomes
M
favorable to an event A, then the limiting value of the ratio is called probability
N
of A.
M
P  A lim
nof N

This definition also has limitations.

1 This definition gives a stabilized value of the relative frequency, and
overcomes to some extent the drawbacks of classical approach
2 This definition also has some limitations first is, it may not be possible to
repeat the experiment under identical conditions large number of times ,due
to budgeted time and cost.
3 In repeatation of the experiment large number of times conditions no more
remain identical.
4 Since it is based on concept of limit, drawbacks of limit are there with the
definition also. However it it works satisfactorily and is widely used

Example 1.3.

What is the probability that a positive integer selected at random from the set of
positive integers not exceeding 100 is divisible by (i) 5, (ii)5 or 3 (iii)5 and 3 ͂.?

Solution: : ^1, 2,},100` so, #  :  100

(i) Let A be an event that no. is divisible by 5, so A , ^5,10},100`

so, #  A  20
#  A 20
P  A 0.2
# : 100

(ii) Let B be an event that no. is divisible by 3, so B ^3,6,},99`

so, #  B  33
# B 33
P B 0.33
#: 100

6
 Chapter 1: Basics of Probability

(iii) Let C be an event that no. is divisible by 5 or 3, C A‰ B

so, #  A ‰ 13  47
#C  47
P C  0.47
#: 100

(iv) Let D be an event that no. is divisible by 5 and 3, D Aˆ B

so, #  A ˆ B  6
# D 6
P  D 0.06
# : 100

Example 1.4. What is the probability that in a random arrangement of alphabets

of word (“REGULATIONS͇

(i) All vowels are together. (ii)No two vowels are together ?

Solution: Since there are 11 letters in the word, they can be arranged in 11!
distinct ways so, #  :  11!

Let A be an event that the random arrangement has all vowels together.
Since the 5 vowels is one group to be kept together and remaining 6
consonants, which is random arrangement of 7 entities in all can be done in
7! ways. In the group of 5 vowels the random arrangement can be done 5!.
ways. so, #  A  7! X 5!

#  A 7!5!
P  A .01515
#: 11!

(ii) Let B be an event that the random arrangement no to vowels together. The
consonants can be arranged as *C *C *C *C *C *C * , where C stands for
consonants. 5 vowels can be arranged a t 7, * positions in 7 P5 ways and 6
consonants in 6! ways, all such random arrangements 7 P5 X 6! ways so,
7! X 6!
# B
2!
# B 7!6!
P B .04545
#: 11!21

Example 1.5. From a pack of well shuffled 52 cards four cards are selected
without replacing the selected card. Jack, queen, king or ace cards are treated as
honor card. a) What is the probability that are there are i) all honor cards ii) More

7
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

honor cards 9. b) What will be these probabilities if cards are drawn with
replacement?

Solution:

a) Since there are 52 cards in the pack of cards,4 can be selected without
52 52
replacement in C 4 distinct ways so, #  :  C4

i) Let A be an event that the random selection has 4 honor cards. Since there
16
are in all 4 X 4 honour cards, #  A  C4

#  A 16C4
P  A 0.0067
#: 52C4

ii) Let B be an event that the random selection has more , that is 4 or 3 honor
cards.
16
# B C4  36 C116C3 21980

# B 21980
P B 0.08119
#: 270725

b) 4 cards can be selected with replacement in 524 ways so, #  :  52 4

i) Let C be an event that the random selection has 4 honor cards. #  C  16 4

# C  164
P C  0.00896
# : 524

ii) Let D be an event that the random selection has more ) that is 4 or 3 honor
cards.
#  D  16 4  36 X 163 212992

# D 212992
P  D 0.02913
# : 7311616

Example 1.6. In a party of 22 people, find the probability that (i) All have different
birtday (ii) Two persons have sme bithday (iii) 11 persons have birtday in same
month.

Solution: We assume that none of them have birthday on 29th February.

8
 Chapter 1: Basics of Probability

(i) Since all 22 people can have any of 365 days as their bithday in 36522 ways.
Thus #  :  36522
365
A be the event that all have different birhday, #  A  P22

Hence P  A  0.5243

(ii) B be the event that two have same birthday and remaining 20 have different
birhday, Any 2 out of 22 can be chosen to have same birthday in 22C2 ways,
and remaining 21 different birthdays can be chosen from 365 days in 365P22
ways.
365
#B P21 u22 C 2

Hence P  B  0.352

(iii) C be the event that 11 have birhday in same month and remaining 11 in
different months.

Now #  :  12 22
12
And #  C  P11 u22 C11
Hence P  C  0.000011

The notion of probability is given modern approach which is based on measure

theory. For this it is necessary to introduce class of sets of :

In next chapter we will discuss various classes of sets.

1.4 Chapter End Exercises

1. Cards are to be prepared bearing a four digit number formed by choosing

digits among 1, 4, 5, 6 and 8. * ind the probability that a randomly chosen
cards among them bear (i) An even number (ii) A number divisible by 4 (iii)A
number has all four digits same.
2. A sample of 50 people surveyed for their blood group. If 22 people have ‘A’
blood group, 5 have ‘B’ blood group, 21 have ‘O’ blood group and 2 have
‘AB’ blood group. Find the probability that a randomly chosen person has
(i) Either ‘A’ or ‘13’ blood group (ii)Neither ‘A’ nor ‘B’ blood group.

3. A roulette wheel has 40 spaces numbered from 1 to 40. Find the probability
of getting (i) number greater than 25(ii) An odd number (iii) A prime number.

9
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

4. A, B, C forms a partition. If the event A is twice as likely as B ,and event C

is thrice as likely as A . Find their respective probabilities.

5. What is the probability that in a random arrangement of alphabets of word

“CHILDREN͇
(i) All vowels are together.
(ii) No two vowels are together?

6. A committee of 5 is to be formed from among a coordinator, chairperson, five

research guides and three research students. What is the probability that
committee (i) Do not have coordinator and chairperson. (ii) All research
guides (iii) None of the students

7. 9 people are randomly seated at a round table. What is the probability that a
particular couple sit next to each other?

8. In a box there are 10 bulbs out of which 4 are not working. An electrician
selects 3 bulbs from that box at random what is the probability that at least
one of the bulb is working?

9. : ^1, 2,},50` A denote number divisible by 5, B denotes number up to

30 ,C is number greater than 25 and D is number less than or equal to 4.
Answer the following
(i) Which events are exhaustive?
(ii) Which events are mutually exclusive?
(iii) Give a pair of events which is mutually exclusive but not exhaustive.
(iv) Give a pair of events which is not mutually exclusive but exhaustive.
(v) Give a pair of events which is neither mutually exclusive nor
exhaustive.

10. A pair of fair dice is thrown what is the probability that the sum of the
numbers on faces of the dice is (i) 6, 7 or 8. (ii) Divisible by 5.(iii)a prime
number?

11. What is the probability that in a group of 25 people (i) all have different
birtdays (ii)11 have birhday in different month and 14 in the same month?

12. Five letters are to be kept in five self addressed envelopes. What is the
probability that (i) All goes to correct envelope(ii)none of them goes to
correct envelope?

10
 Chapter 1: Basics of Probability

13. The coefficients a,b,c of the quadratic equation ax 2  bx  c 0 , are obtained

by throwing a die thrice. Find the probability that equation has real roots.

14. What is the probability that there are 53 Thursdays and 53 Fridays in a leap
year?

15. A sequence of 10 bits is randomly generated. What is the probability that

(i) atleast one of these bits is 0? (ii) a sequence has equal number of 0 and 1.

16. The odds against an event A are 3: 5, the odds in favor of an event 13 are 7:
5, What are the probabilities of the events?

17. In a group of 12 persons what is the probability that (i) each of them have
different birthday (ii) each of them have birthday in different calendar month?

1  3x 1  4x
18. A, B, C are mutually exclusive. P  A  , P  B and
2 3
1 x 1 1
P  C . (i)Show that the range for x is,  x  (ii) are they
6 4 3
exhaustive?

19. Express A   B ˆ C  as union of three events.

™™™

11
 2
FIELDS AND SIGMA FIELDS
Unit Structure

2.0 Objectives

2.1 Class of Sets

2.2 Field

2.3 V  field and Borel V  field

2.4 Limit of sequence of events

2.5 Chapter End Exercises

2.0 Objectives

After going through this chapter you will learn

x A class of sets and variouus closure properties that it may follow.

x Concept of field and its properties.

x Sigma field and its properties.

x Borel Sigma field, minimal Sigma field.

x Limit superior and limit inferior of sequence of events.

2.1 Class of Sets

Before introducing modern approach of probability, we need to define some terms

from measure theory. Subsequent sections are also explaining their role in
probability theory.

A collection of subsets of : is termed as Class of subsets of : .It plays an

important role in measure theory. They have some closure properties with respect
to different set operations.

A be the class of subsets of :.

12
 Chapter 2: Fields and Sigma Fields

Complement: A is said to be closed under the complement, if for any set A  A ,

A is  A

Union: A is said to be closed under the union if for any sets A, B  A, A ‰ B is

 A Intersection : A is said to be closed under the intersection if for any sets A ,
B  A , A ‰ B is  A

Finite Union and Countable Union : A is said to be closed under the finite union
if for any sets A1 , A2 , ..., An . A , * ni 1 Ai is  A .Further if n o f and if we have
Uif 1 Ai is  A, A is said to be closed under countable unions

Finite intersection and Countable intersection: A is said to be closed under the

finite intersection if for any sets A1 , A2 , ..., An .  A,

 ni 1 Ai is  A .Further if n o f and if we have fi 1 Ai is  A, A is said to be

closed under countable intersection. Note: Closure property for countable operation
implies closed for finite operation

2.2 Field

Definition 2.1. Field :

A class  of subsets of a non̺empty set : is called a field on : if
1. :   .
2. It is closed under complement.
3. It is closed under finite Union

Notationally
A class  of subsets of a non̺empty set : is called a field on : if
1. :  
2. for any set A   , A   .

3. for any sets A1 , A2 , ... , An .   , *ni 1 Ai  

Following points we should keep, in mind regarding field.

x Closure for complement and finite Union implies closure for
intersection.So,field is closed under finite intersections.
x ^M ,:` is a field.

13
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

x Power set P  :  ,which is set of all subsets of : is a field.

x For any A  :,^M , :, A, A` is smallest field containing A.

x For any sets A, B   A  B   ,hence A ' B   .

x 1 and 2 are two fields on : , then 1n2 is a field.

x Field is also called as an Algebra.

Example 2.1. : ^1, 2,3` , 1 ^M , :,^1`,^2,3`` and 2 ^M , :,^2`,^2,3`` are

two fields on : . Is union of these two fields is a field i )

Solution: 1 ‰ 2 {M , :,^1` , {2}, {2, 3} }

let A ^1, 2` ^1` * ^2`  1U2

? f1 * 2 is not a field.

Example 2.2.  { A  : such that A is finite} . Is  a field?

Solution: No, if : is infinite then, : does not belong to  and hence  cannot
be a field.

Example 2.3. Complete the following class to obtain a field. Given : >0,1@ and
­ ª 1 ·½
 ®M ,1, «0, ¸ ¾
¯ ¬ 2 ¹¿

­ ª1 · ª 1 · ª 1 º½
Solution: Add ®> 0,1@ , « ,1¸ , «0, ¸ U ^1` , > 0,1 , « ,1» ¾ in  to make it a field
¯ ¬2 ¹ ¬ 2 ¹ ¬ 2 ¼¿

2.3 D  field and Borel D  field

Definition 2.2. V  field: A class C of subsets of a non̺empty set : is called a

V ̺field on : if

1. :  C.

2. It is closed under complement.

3. It is closed under countable Unions.

14
 Chapter 2: Fields and Sigma Fields

Notationally

A class C of subsets of a non̺empty set : is called a V ̺ field on : if

1. :C

2. for any set A  C , A  C .

3. for any sets A1 , A2 , ..., .  C , then *fi 1 Ai  C

x Field which is closed under countable unions is a D ̺field.

x Like fields the intersection of arbitrary V ̺ fields is also V ̺ field

but their union is not a V ̺field

x Power set P  :  ,which is collection of all subsets of : is ao .̺field.

x Given a class of sets consisting all countable and complements of

countable sets is a V ̺ field

Example 2.4. A class C of subsets A of : such that either A or its complement

is finite. Is C is  I  a field d p  II  a V  field 9.

Solution:

(I) C { A Ž : | A is finite or ̺ is finite}

Note that  i  C is closed under complementation, since either of A or ̺ is

finite (ii) If A, B  C both finite then A * B is finite
If A is finite B is infinite A * B is infinite. But A * B A  B .Since ̺ is
finite ̺ is infinite

A * B finite, hence A U B. Similarly, we can check the case when both A

and B are infinite.

Thus, For any A, B  C , A * B is also  C . So C is a field

(II) But if Ai are finite *fi 1 Ai does not belong to C.

C is not closed for countable unions, hence cannot be 0˜ ̺field.

Definition 2.3. Minimal V  field : A class C of subsets of : is called a minimal

V ̺field on : , if it is the smallest 0  field containing C

15
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

x Minimal D ̺field can be generated by taking intersection of all the D ̺

fields containing C

x If A is family of subsets of : , and C A n ^C | A  C` ,which is intersection

of all 0˜ ̺fields containing A then C A is a minimal V ̺field.

x If A itself is a V ̺field, then C A A

Hence onwards we term the pair  :, C  as a sample space

In theory of probability : IR has specific features and sample space (IR, B )

plays vital role.

Definition 2.4. Borel V  field: Let C is the class of all open intervals  f, x 
where x   , a minimal V  field generated by C is called as Borel V ̺ field,
and denoted by B.

Borel V ̺ field has following features.

1. It is clear that [ x,f ) is complement of  f, x  but it does not belong to

C .Thus C is not closed under complement C is also not closed under
countable intersections, as

§ 1 ·
fn 1 ¨ x  , f ¸ > x, f  . But B is closed under complements as well as
© n ¹
countable unions or intersections.
Hence  contains all intervals of the type [ x,f )

§ 1·
2.  f, x@ fn 1 ¨ f, x  ¸ .So B contains all intervals of the type (f, x ]
© n¹

3.  x,f  is complement of (f, x ]. Thus B contains all intervals of the type

 x, f   a, b   f, b    a, f  , where a  b . So  contains all

. 4.
intervals of the type  a, b  . And contains even intervals of the type [ a , b ),
(a, b ] for all a, b  .

Note that sets of B are called as borel sets.

16
 Chapter 2: Fields and Sigma Fields

2.4 Limit of sequence of events

In this chapter the concept of limit of a sequence of events is introduced.

Definition 2.5. Limit Superior: ^ An ` be the sequence of events of space  :, C  .

Limit superior of An is an event which contains all points of : that belong to An
for infinitely many n and it is denoted by lim sup An or  limAn , termed as limit
superior of An
x Z  lim sup An iff for each n t 1 there exists an integer m t n such that
Z  Am for all m t n
x Thus lim sup n fn 1 *fm n Am

x It can be clearly seen that lim sup An  C

x lim An ^ Ani.0` , where i.o infinitely often.

Definition 2.6. Limit Inferior: ^ An ` be the sequence of events of space  :, C  .

Limit inferior of An is an event which contains all points of : that belongs to An
but for finite values of n . and it is denoted by lim inf An or limAn , termed as limit
inferior of An
x Z  lim inf An iff there exists some n t 1 such that Z  Am for all m t n

x Thus lim inf An *fn 1 fm n Am

x It can be clearly seen that limin  An  C

x lim inf An subseteq lim sup An

x If lim An exists, lim An lim inf An lim sup An .

Definition 2.7. ^ An ` be the sequence of events of space  :, C  such that

A1  A2 } , then ^ An ` is called as expanding or increasing sequence and
lim An *fn 1 An

Definition 2.2. ^ An ` be the sequence of events of space  :, C  such that

A1 Š A2 } , then ^ An ` is called as contracting or decreasing sequence and
lim An fn 1 An

17
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Remark 2.1. ^ An ` be the sequence of events of space  :, C  then  i  Cn *fm n Am

is decreasing sequence. Cn p C , where C fn 1 cn lim sup An (ii) Bn fm Am

is increasing sequence. Bn n B , where B *fn 1 Bn lim inf An

Remark 2.2. ^ An ` be the sequence of events of space  :, C  then *fm n Am is also

called as Supmtn Am , and fm n Am is also called as Inf mtn Am .

2.5 Chapter End Exercises

1.  is a field .If A, B   then show that A  B and A+ B are also events off

2. : ^1,2,3,4`.
Which of the following classes is a field on : ?

(i) 1 ^M ,^1, 4`,^2,3``

(ii) 2 ^M , :,^2`,^3`,^2,3`,^1, 4`,^1, 2, 4`,^1,3, 4``.
3. Complete the following class to obtain a field. Given :  0,1 and (i)
­ § 1 · § 1 ·½ § 1 · ª1 · § 2º § 2 ·
 ®M ,  0,1 , ¨ 0, ¸ , ¨ ,1¸ ¾ (ii)  {M ,  0,1 , ¨ 0, ¸ , « ¸1) ¨ 0, » , ¨ ,1¸}
¯ © 2 ¹ © 2 ¹¿ © 2 ¹ ¬2 ¹ © 3¼ © 3 ¹

4. A class C of subsets A of : such that either A or its complement is

countable. Is C is a V ̺field?

5. A, B, C forms partition of : , obtain a smallest field containing A,13, C

6. Show that following are the Borel sets.

(i) > a, b @  ii ^a` (iii) Any finite set (iv)Any countable set (v) A set of rational
numbers(v) A set of natural numbers

ª1 1 º
7. C is V  field on : >0,1@ such that « n, , n  1»  C for n = 1,2,....Show that
¬ ¼
1 1
following are the events of C. (i) ( ,1] ( ii ) (0, ]
n n

­ A n 1,3,5,
8. An ®
¯ B n 2, 4,6}
Find lim inf An , lim sup A7 l and show that lim An does not exists.

18
 Chapter 2: Fields and Sigma Fields

9. Prove that (i) lim  An * Bn  limAn * limBn , (ii) limAn lim An , (iii)

lim  An  Bn  

limAn  limBn

10. Are the above results true for lim inf?

11. If An o A then Ѹ n o

™™™

19
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

3
PROBABILITY MEASURE AND
LEBESGUE MEASURE
Unit Structure
3.0 Objectives
3.1 Probability Measure
3.2 Lebesgue Measure and integral
3.3 Discrete and absolutely continuous probability measures
3.4 Chapter End Exercises

3.0 Objectives
After going through this chapter you will learn

x A function defined on sample space called as probability measure.

x Types of probability measure ,discrete and continuous.

x Lebesgue measure and Lebesgue integral.

x Properties of probability function

x Probability of limit of sequence of events.

3.1 Probability Measure

The modern approach of probability is based on measure theory. Following

definition is due to Kolmogorov (1933)
Definition 3.1. Axiomatic definition of probability: C be the Cf field associated
with the sample space : . A function P . defined on C to > 0,1@ is called as
probability measure or simply probability if it satisfies following axioms.
1. P  A  t 0 for all A  C.
2. P : 1
3. A1 , A2 , ... is sequence of mutually exclusive events of C then


P * fi 1 Ai  6if 1 P  Ai  (3.1)

20
 Chapter 3: Probability Measure and Lebesgue Measure

x Axioms are respectively called as non̺negativity, normality and countable

additivity.

x In this definition probabilities have been already assigned to the events, by

some methods or by past information.

x The triplet  :, C , P  is called as a probability space

x Depending on : different types of probability space are decided.

x If : is finite or countable(at most countable)probability space is discrete.

x If : has one one correspondence with IR probability space is continuous.

Properties of the Probability function.

Complement
P  A  1  P  A
Proof:
: A* A

A and A are mutually exclusive events So

P : P  A  P A 
…by countable additivity axiom
L. H. S 1 ….. by normality axiom

R.H. S  
P  A  P A Hence

P  A  P  A 1

P  A 1  P  A

Monotone A and B are C such thatA A  B then P  A  d P  B 

Proof:
B 
A* B  A 
 
Since A and B  A are mutually exclusive, so by countable additivity axiom

P  B P  A  P   B  A 

So, P  A  d P  B  as P B  A t 0 
21
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Substantivity A and B are C such thatA  B . Then P  B  A  P  B   P  A

Proof: From the above proof
PB P  A  P  B  A 
Thus
P  B  A P  B  A P  B   F  A
Similarly we can say that
A and B are C such that B  A Then P  A  B  P  A  P  B 
Continuity lim An A ,then lim P  An  P  A
nof nof

Theorem 3.1.  i ^ Aj ` is expanding or increasing sequence of events of space

 :, C  then

lim P  An 
n of

P * fn 1 An  (3.2)

(ii) ^ A j ` is the contracting or decreasing sequence events of space  :, C  then

lim P  An 
n of

P  fn 1 An  (3.3)

Proof:  i ^ Aj ` be the sequence of increasing events, so A1  A2 } . Let

Bj A j n A j 1B j are mutually exclusive.
So An * nj 1 B j
*fj 1 Aj *fj 1 B j (3.4)
By  9.4 


P * fj 1 A j  
P * fj 1 B j 
f
¦ j 1
P  B j  By countable additivity
n
lim ¦ j 1P  B j  By definition of sum of series
nof

 
lim P * nj 1 B j By finite additivity
n of

lim P  An  By definition of An
nof

lim P  An 
n of

P * fn 1 An 
 ii ^ Aj ` be the sequence of decreasing events, so A1 Š A2 } . hence ^  j` be the
sequence of increasing events, so Ѹ 1  ̺ 2" ˜ Applying result in (i) to ^  j`

P * fj 1  j  lim P   n 
n of
(3.5)

22
 Chapter 3: Probability Measure and Lebesgue Measure

 
1  P fj 1 Aj L.H.S by De Morgan’s law

1  lim ª¬1  P  An  º¼ R.H.S By complementation

nof

1  lim P  An 
nof

lim P  An 
nof

P n fn 1 An 
Theorem 3.2. The continuity property of probability.
lim An
nof
A , then lim P  An 
nof

P lim An
nof
 P  A (3.6)

Proof: lim inf An * fm 1 n fm n Am * fn 1 Bn

were
Bn fm n Am
These Bns are increasing events n *fn 1 Bn B say
using (3.2)
lim P  Bn 
n of

P * fn 1 Bn  P B

lim sup fn 1 *fm n Am 

P fn 1 Cn 
where
Cn *fm n Am
These Cns are decreasing events p fn 1 cn Csay
using (3.3)
lim P  Cn 
n of

P  fn 1 Cn  P C 

Now consider,
fm n Am  An  U fm n Am (3.7)
Bn ‫ ؿ‬An ‫ ؿ‬Cn
By monotone property of P
P  Bn  d P  An  d P  Cn 
taking limits
lim P  Bn  d lim P  An  d lim P  Cn 
nof nof nof

So,
P  B  d lim P  ArL  d P  C 
nof

But, lim An A

A lim AJL lim inf An B lim sup An C

23
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

implies
P  B P  A P C 

lim P  An 
nof
P  A P lim An nof
 (3.8)

Example 3.1. Which of the following are Probability functions?

(i) : ^1, 2,3}` , , C is V  field on : . A function P

defined on space  :, C  as
1
P i 
2i
for i :
Solution: a)
f 1
P : ¦ 1
i 1
2i
b)

P  A  t 0 for all A  C

c ) Let us define mutually exclusive events, Ai i we can verify countable

additivity.

P * fi 1 Ai  6if 1 P  Ai  (3.9)

By a) ,b)and c) Pis Probability function.

 ii  :  0, f  , Borel V  field B defined on :. A function F

defined on space  :, B  as, for any I  

x
PI  ³e dx (3.10)
I
f
Solution: a) P  :  ³ e x d x 1
0

b) P  A  t 0 for all A  B
c) Ai  i, i  1 we can verify countable additivity

. P  * fi 1 Ai  ³ e x d x (3.11)
U if 1

Ai s are mutually exclusive. From the properties of integrals,

f f
¦ i 1
f i 1e  x d x ¦ i 1
P  Ai  (3.12)

By a) ,b), c) P is Probability function.

24
 Chapter 3: Probability Measure and Lebesgue Measure

(iii) :  f, f  , V  field C defined on : A function P defined

on space  :, C  as, for any I C

­0 I   f,1
°
PI  ®1
°2 I  >1f 
¯
Solution: a )
f 1
P : ³ z1
1 2
P is not a Probability function.
Theorem 3.3. Borel Catelli lemma

If ¦ P A   f
i then P lim An   P  Ani.0  0

Proof:


P lim An  
P  fn 1 * fm n Am 

d P * fm n Am 
f
d ¦ m n
P  Am  as events need not be mutually exclusive

If ¦ P A   f
i , then ¦ P  A  tends to zero as n o f Hence the proof.
m

Remark 3.1. Other half of the above result is stated as follows. But it needs
independent events.
For Ai are independent events of the sample space, If ¦ P A  i f then


P  lim An  P  Ani.0  1

3.2 Lebesgue Measure and integral

Definition 3.2. Lebesgue Measure A function P defined on space  , B  is called

as Lebesgue Measure if it satisfies following
1. P  a, b @ b  a
2. P M  0
3. Ei s are mutually exclusive intervals of B then,
P  * fi 1 E? ˜ X if 1P  Et 

25
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Example 3.2. Find Lebesgue Measure for following sets.

ª1 1 · ª1 1 · ª2 7º 1 1
(i)
«¬ 2 , 6 ¸¹ (ii) «¬ 3 , 2 ¸¹ (iii) «¬ 3 , 9 »¼ (iv) n ,: n 1, 2,.} (v) {x : x  n  2n forn  }

Solution:
§ 1 1º 1 1 1
(i) P ¨ , » 
© 9 6¼ 6 9 18
1 1 1 f
­1 ½
Similarly (ii) (iii) (iv) P{ : n 1, 2,...} ¦ P ®¯ n ¾¿ 0
9 9 n i 1

1 1 1
(v) x  n  n
implies n  n  x  n  n
2 2 2
1
x lies in mutually exclusive intervals of length
2n 1
­ 1½ f
§ 1 1 ·
P ® x : x  n  n ¾ for n  ` ¦ P ¨© n  2 n
,n  ¸ 2
¯ 2 ¿ i 0 2n ¹
Remark 3.2.
x If : >0,1@ then P P is Probability measure.

x P is actually an extended measure. It is V ̺finite measure,

1
x Since P  x  P (lim( x  , x ])
n
§ 1 º 1
P  x  lim P ¨ x  , x » lim
© n ¼ n
by continuity
Thus P  x  0

x From above P ( a, b ] P  a, b  b  a

x The sets whose P measure is zero is called as P ̺null set.

Definition 3.3. f . is a function defined on \ is a called as Borel function if

inverse image is a Borel set.

Definition 3.4. Lebesgue integral: Lebesgue integral is a mapping on non-negative

burel function f which satisfy following,

1 ³ fd P   0, f 

2 ³I A dP P  A  for any A  \

26
 Chapter 3: Probability Measure and Lebesgue Measure

3 ³  f  gdP ³ fd P  ³ gd P and ³ cfd P c ³ fd P . Where c t 0

4 lim ³ f n d P ³ fd P if lim f n  x  f  x  for any x  

x For any nonnegative piecewise continuous function f

b
J >a ,b@ fd P ³ D f  x  dx F b  F  a 
, (3.13)
Where P is antiderivative of f.

x P is nondecreasing function with F  f   F  f  is finite.

x If we divide F by F  f   F  f  we get probability measure.

x We will revisit this function in next chapters.

3.3 Discrete and absolutely continuous probability measures

Definition 3.5. Density function: A non-negative Borel function f :  o [0, f ) is
called as a density if

³ fd P 1 (3.14)

Theorem 3.4. If f is a density then P satisfying

P  A ³ fd P (3.15)
A

is a probability measure on Borel subsets A of \

proof: Since f is a density on \

P ³ fd P 1 (3.16)


Now consider A1 , A2 } be mutually exclusive Borel sets. Let Bn *ni 1 At and
B *fi 1 Ai Since fI Bn / fI B By monotone convergence of Lebesgue measure

P  Bn  ³ fd P ³ fI Bn d P / ³ fI B d P ³ fd P PB (3.17)
Bn B

Thus P is countably additive, hence it is Probability measure.

Example 3.3. Find the constant k if following are the density functions.

(i) f  x  kI> 2,3@  x 

27
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

(ii) f  x  ke2 x ; x ! 0

Solution:(i) The density is 0 outside [̺2, 3] and on [̺2, 3] it is 1

3
. ³ f  x  dx k ³ 1dx 5k 1 (3.18)
 2

1
k
5
(ii) The density is 0 outside  0,f  and on  0,f  it L sf  x  ke2 x
f k
. ³ f  x  dx k ³ e2 x dx 1 (3.19)
 0 2
k 2
Definition 3.6. Absolutely Continuous Probability Measure:
P is a probability measure on Borel subsets A of  is said to be Absolutely
Continuous probability Measure, if there exists a density f such that

P  A ³ fd P (3.20)
A

Definition 3.7. Dirac Measure:

Let : be at most countable arbitrary set,  be the family of subsets of : . A
measure G w on  defined as

­1 Z  A
GZ  A ® (3.21)
¯0 Z  A
is called as Dirac Measure concentrated at w
Definition 3.8. : be at most countable arbitrary set )  be the family of subsets
: A Dirac measure on  say GZ A probability measure P defined as
f
P  A ¦ k 1 k
D Gk (3.22)

such that D k t 0 and

f

¦D
k 1
k 1

is said to be discrete probability measure.

Remark 3.3. Dirac Measure is a probability measure.

x :  ^Z` is largest set with measure O, and its every subset has also measure
0 . Smallest set with measure 1 is w

28
 Chapter 3: Probability Measure and Lebesgue Measure

x For Z1 , Z2 : , P  A  DG Z1  A   1  D  G Z2  A  is a probability measure.

Where 0  D  1.
x Sometimes we come across measures which are neither discrete nor
absolutely continuous. Following theorem is for such mixed probability
measures.
Theorem 3.5. P1 , and P2 are the two probability measures,

P  A  D P1  A  1  D  P2  A is a probability measure. Where 0  D  1.

Proof:
1. P  :  D P1  :   1  D  P2  :  1 ,as both P1 and P2 are probability
measures.
2. P  A  D P1  A  1  D  P2  A t 0 as Pi  A  t 0, i 1,2.

3. An be the countable sequence of mutually exclusive events. I3y countable

additivity of P1 and P2 ,


P * fn 1 An  D P1  * fn 1 An   1  D  P2  * fn 1 An 
f f
D ¦ n 1P1  An   1  D  ¦ n 1P2  An 
f
¦ n 1
P  An 

This shows that P  A  is a probability measure.

Remark 3.4. Generalization of above result can be stated as : Pi , s are the

probability measures, P  A  ¦ D P A 
i i is a probability measure. Where

0  Di  1 and ¦D i 1

Example 3.4. Find the P  A  if following are the density functions and F is
absolutely Continuous probability measure w.r.t it and A (0, 2 ].
1
(i) f  x  I  x  . (ii) f  x  e x ) x ! 0
6 >3,3@
Solution:(i)
1 2 1
P   0,2@ 1dx
6 ³0 3
(ii)
2
P   0, 2@ ³e
x
dx 0.8647
0

29
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

1 3 1
Example 3.5. Define P by P  A  G1  A   G 2  A  P3  A ,
8 8 2
1
P3 has density f  x  I
6 >3,3@
 x  Compute > 2,3@ .
Solution: G1 > 2,3@ 1, G 2 > 2,3@ 1 , and
1 3 1 1 3 1 7
P3 > 2,3@ 1dx P > 2,3@ .
6 ³2
 
6 8 8 12 12

3.4 Chapter End Exercises

1. Find the constant k if following are the density functions.

(i) f  x  kI ª 5 , 5 º¼
 x
¬

(ii) f  x  kx3 ;0  x  1

2. Find the P  A  , if following are the density functions and P is absolutely

Continuous probability measure w.r. t it. A (1,0.5 ].

1
(i) f  x  I  x
4 > 1,4@

(ii) f  x  6 x 1  x  ;0  x  1

3. :  f, f V ̺ field C defined on : .A function P defined on space

 :, C  as , for any I  C

­1 Iisfinite
PI  ®
¯0 Iisinfinite
1 2
4. Find Lebesgue Measure for following sets    , > 0,1@ , (, , ].
8 5
5. Show that Dirac Measure is a probability measure.
1 1 1
6. Define Paneity  A  G1  A   G 2  A  P3  A
4 4 2

P3 has density f  x  2e2 x Compute P >1,3@ ).̓

30
 Chapter 3: Probability Measure and Lebesgue Measure

1 3 1
7. Define P by P  A  G1  A  P2  A   P3  A , P2 has density
8 8 2
1
f  x
2
I> 1,1@  x  and P3 has density f  x  x 2 ;0  x  1 .Compute P >0,1@ .
8. If P  An 'A  o 0 as n o f then show that P  An  o P  A .

9. Show that P  AnBnC  t 1  P     P     P    .

10. If A and B implies C then show that P    d P     P    .

™™™

31
 4
CONDITIONAL PROBABILITY AND
INDEPENDENCE
Unit Structure

4.0 Objectives

4.1 Conditional Probability and multiplication theorem

4.2 Independence of the events

4.3 Bayes’ Theorem

4.4 Chapter End Exercises

4.0 Objectives

After going through this chapter, you will learn

x Conditional probability and its role in finding probability of simultaneous

occurrence of events.

x Notion of independence of events and its consequences

x Total probability theorem.

x Bayes̓ theorem and its use to land posterior probabilities.

4.1 Conditional Probability and multiplication theorem

Let 13 be arbitrary set of : . Let A be the class of events of :.

AB ^B  A | A  A`
We can easily verify that AB is a V ̺field. And  B, AB  is a measurable space. P

measure on this space is not a probability as P  B  z 1,

let PB is defined as
P A  B
PB  A  (4.1)
P B

32
 Chapter 4: Conditional Probability and Independence

PB is called as conditional probability measure or simply conditional probability of

an event A
Theorem 4.1. B be arbitrary set of : A be the class of events of
:. AB ^B  A | A  A`
P A  B
PB  A  (4.2)
P B

PB is a probability measure on  B, AB 

proof:

1. PB  A  t 0 for all A  A

2. PB  B  1

3. A1 , A2 } be mutually exclusive sets of  AB

From above it is clear that PB is a probability measure on  B, AB 

Remark 4.1. Conditional probability is denoted by P  A / B  and called as

conditional probability of an event A given event B has occurred Thus it is
necessary to have P  B  ! 0

x P  A / A 1

x P  A / : 1

x P  A / B  z P  B / A

x From the definition of conditional probability, it follows that

P  A  B P  B P  A / B

which is also known as a Multiplication theorem on probability.

x For three events Multiplication theorem on probability is stated as

P  A1  A2  A3  P  A1 / A2  A3  P  A2 / A3  P  A3 

33
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

The conditional probability is not defined when probability of given event is zero.
The conditional probability leads to another concept related with events, known as
independence.
Example 4.1. Show that

P A* B / C P A / C  PB / C  P A  B / C

Solution:
P  A * B   C 
P A* B / C
P C 

.. By definition of conditional prob.

P  A  c  *  B  C 
P C 

.. By Distributive law
P  A  C   P  B  C   P  A  C  B 
P C 

.. By Addition theorem on probability.

P  A  C  P B  C  P  A  C  B
 
P C  P C  P C 

P A / C  PB / C  P A B / C

By definition of conditional prob.

Example 4.2. Probability that it rains today is 0.4) probability that it will rain
tomorrow is is 0.5, probability that it will rain tomorrow and rains today is 0.3.
Given that it has rained today, what is the probability that it will rain tomorrow.?
Solution: Let
P  A P (it rains today) 0.4
P B P ( it will rain tomorrow) 0.5
P  A  B P ( it will rain tomorrow and rains today) 0.3

Required probability is
P  A  B
P A / B 0.6
P B

34
 Chapter 4: Conditional Probability and Independence

Example 4.3
A box contains cards numbered 1 to 25. A card bearing even number was drawn,
but the number was not known. What / s the probability that it is card bearing
number divisible by 5c ?

Solution: : ^1,2,3}25` ,
12
P B P ( even no. card)
25
5
P  A P ( card with no. divisible by 5 )
25
2
P  A  B P ( An even no. divisible by 5 )
25
Required probability is

P A  B 2
P  A / B
P  B 12

4.2 Independence of the events

The occurrence and nonoccurrence of the event, when does not depend on
occurrence and nonoccurrence of the other event the two events are said to be
independent. Since occurrence and nonoccurrence of the event is measured in terms
of probability. Instead of only independence we say stochastic independence or
independence in probability sense. Let us first define independence of two events.
we will later call it as pair wise independence

Definition 4.1. Independence of the events: Let  :, A, P  be a probability space.

invents A and B of this space are said to be stochastically independent or
independent in probability sense if and only if P  A  B  P  A P  B 

x Above definition works for any pair of events even when either P  A  or
P  B  is equal to zero.

x Property of independence is reflexive.

x If A and B are independent then conditional probability and unconditional

probabilities are same. That means if A is independent of B P  A / B  P  A
and P  B / A  P B

35
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Theorem 4.2. If events A and B are independent so are(i) A and B  ii  B and A

(iii) A and B .
proof: (i) Consider
P Aˆ B P  A  P  A ˆ B 

Since A and B are independent

P  A ˆ B P  A P  B 

Consider
P Aˆ B P  A  P  A ˆ B 

P  A  P  A P  B  P  A P  B 

Thus
P Aˆ B P  A P  B 

So, A and B are independent. Similarly we can prove (ii)

(iii) Consider
P A ˆ B 
P A‰ B 
....By De Morgan’s law

P A‰ B  1 P  A ‰ B

1  ª¬ P  A   P  B   P  A ˆ B  º¼ 1  ª¬ P  A   P  B   P  A  P  B  º¼

.......since A and I3 are independent

Thus
P  A ˆ B  [1  ª¬ P  A  º¼ ª¬1  P  B  º¼

P  A P B  
So, A and B are independent.

Definition 4.2. At , i 1}n events are mutually or completely independent if and

only if for every sub collection
k
P  A1nA2 " nAk  P  A  i
i 1

for k 2}n

36
 Chapter 4: Conditional Probability and Independence

2 we say that events are pairwise

Remark 4.2. If the above condition holds for k
independent. There are such nC2 pairs, and those many conditions have to be
checked. And for n events, to be completely independent, there are 2n  n  1 .
conditions have to be checked.
Remark 4.3. If A, B, C are three events

x They are pairwise independent if

1. P  A ˆ B P  A P  B 

2. P AˆC P  A P  C 

3. PB ˆC P  B  P C 

x They are completely independent if

1. P  A ˆ B P  A P  B 

2. P AˆC P  A P  C 

3. PB ˆC P  B  P  C  And

4. P Aˆ B ˆC P  A P  B  P  C 

4.3 Bayes’ Theorem

It is possible to find probability of an event if conditional probabilities of such event

given various situations. The situations need to be exhaustive and non-overlapping

Example 4.4. An urn contains 5 white and 7 black balls.3 balls are drawn in
succession. What is the probability that all are white’? If ball are drawn (i) with
replacement (ii) Without replacement.

Solution: Let Ai be the event that i th drawn ball is white, i 1, 2,3 . (i) When balls
are drawn Without replacement, events are not independent. Using multiplication
theorem, Required prob. P  A1 ˆ A2 ˆ A3 

5 4 3 1
P  A1  P  A2 / A1  P  A3 / A1 ˆ A2  u u
12 11 10 22

(ii) When balls are drawn with replacement, events are independent
3
§ 5·
P  A1 ˆ A2 ˆ A3  P  A1  P  A2  P  A3  ¨ ¸
© 12 ¹

37
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Example 4.5. A problem is given to three students whose chances of solving the
problem are 0.2,0.3 and 0.5 respectively. If all of them solve the problem
independently, find the probability that (i) None of them solves it. (ii) the problem
is solved by exactly two students

(iii) the problem is solved.

Solution: Let Ai be the event that i th , i 1,2,3 . student solves the problem

i 1, 2,3. (i) P ( None of them solves it ) 

P A1 ˆ A2 ˆ A3 
Since they solve the problem independently, Ai are independent, so are Ai

     
P A1 P A2 P A3 0.8 u 0.7 u 0.5 0.28

(ii) P ( the problem is solved by exactly two students)

  
P A1 ˆ A2 ˆ A3  P A1 ˆ A2 ˆ A3  P A1 ˆ A2 ˆ A3   
0.2 u 0.7 u 0.5  0.8 u 0.3 u 0.5  0.8 u 0.7 u 0.5 0.47

(iii) P (the problem is solved) 1  P ( None of them solves it ) 0.72

Example 4.6. : ^^1,1,1`^1, 2,1`^1,1, 2`^2,1,1`` , A : first no. is 1, B: second no. is1
, C:third no. is 1, examine whether A, B, C are completely independent?.

Solution:
P  A P  B P C  0.5, P  A ˆ B  PB ˆ C P Aˆ C 0.25 P  A ˆ B 
0.25 P  A ˆ B  P  A P  B  , P  A ˆ C  P  A P  C  , P  B ˆ C  P  B  P C 
so A, B, C are pairwise independent. But P  A ˆ B  0.25 z P  A P  C  P  B  0.125 ,
hence they are not completely independent.

Theorem 4.3. Theorem of total probability: A1, A2 ,. An are forming partition of a

: , Let B be another event, B  : then we can find probability of B by following
relation

n
P B ¦ P B / A  P A 
i 1 i i (4.5)

38
 Chapter 4: Conditional Probability and Independence

proof: A1 , A2 , . An are forming partition of a sample space, for every i z j 1, 2 , ...

n ;  Ai ˆ A j  I And  * ni 1 Ai  :

B:

? P  B P  B ˆ :

P  B  
P B ˆ *ni 1 Ai 
Since Ais forms partition


P U in 1B ˆ  Ai  
...... By distributive law.

n
¦ P  B ˆ  A  .
i 1 i

By finite additivity of Probability function

n
¦ P  B / A  P  A 
i 1 i i

.. ... By multiplication theorem. Hence

n
P B ¦ P B / A  P A 
i 1 i i

x Though the theorem is proved for finite partition, it is also true for countable
partition.
x At least two Ais are should have nonzero probability.

x If B is an effect and Ais are different causes P  B  summarizes chance of the

effect due to all possible causes.

Example 4.7. Screws are manufactured by two machines A and B. Chances of

producing defective screws are by machine A and B are 4r0 and 1r0
respectively. For a large consignment A produced 70r0 and B produced 30r0
screws. What is the probability that a randomly selected screw from this
consignment is defective?

Solution: Let A be the event that screws are manufactured by two machines A and
B be the event that screws are manufactured by machines B. D be the event that

39
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

defective screws are manufactured. Given P  A 0.7 , P  B  0.3 ,

P  D / A  0.04 , P  D / B  0.01 . By Theorem of total probability
P D P  A  P  D / A  P  B  P  D / B  .031

Example 4.8. A ball is selected at random a box containing 3 white 7 black balls.
If a ball selected is white it is removed and then second ball is drawn. If the first
ball is black it is put back with 2 additional black balls and then second ball is
drawn. What is the probability that second drawn ball is white ̓?

Solution: Let AW be the event that ball drawn at the first draw is white AB be the
event that ball drawn at the first draw is black. D be the event that ball drawn at
2
the second draw is white Given P  AW  0.3, P  AB  0.7, P  D / AW  ,
9
3
P  D / AB  . By total probability theorem,
12

P D P  AW  P  D / AW   P  AB  P  D / AB  0.2417

Theorem 4.4. Bayes’ Theorem : A1, A2 ,. An are forming partition of a : , Let B

be another event, B  : then we can find probability of B by following relation

( P  B / Aj  P  Aj 
P  Aj / B  (4.6)
¦ P  B / Ai  P   Ai  

proof:

P  Aj ˆ B 
P  Aj / B 
P B

By multiplication theorem.

And using P  B  from total probability theorem the proof of theorem follows.

x This theorem is useful for posterior analysis of cause and effect.

x Given P  Ai  , which are prior probabilities of the i th cause.Where as

P  Ai / B  are posterior probability of the cause Ai given that I3 is effect
observed.

Example 4.9. Three people X , Y , Z have been nominated for the Manager ̓ s
post. The chances for getting elected for them are 0 . 4, 0.35 and 0.25 respectively.
If X will be selected the probability that he will introduce Bonus scheme is 0.6 the
respective chances in respect of Y and Z are 0.3 and 0.4 respectively. If it w

40
 Chapter 4: Conditional Probability and Independence

known that Bonus scheme has been introduced, what is the probability that X is
selected as a Manager?
Solution: Let B be the event that bonus scheme is introduced. X , Y , Z denotes
respectively that X ,Y , Z are elected. Thus given P  X  0.4 ,,
P Y  0.35, P  Z  0.25, P  B / X  0.6, P  B / Y  0.3, P  B / Z  0.4 By Bayes
Theorem,

PB / X  P X 
P X / B
P  B / X  P  X   P  B / Y  P Y   P  B / Z  P  Z 

0.4 u 0.6
0.5275
0.4 u 0.6  0.25 u 0.3  0.35 u 0.4
Example 4.10. 1% of the population suffer from a dreadful disease. A suspected
person undergoes a test. However the test making correct diagnosis 90% of times.
Find the probability that person who has really caught by that disease given that
the test resulted positive? Solution: Let A1 be the event that person was really
caught by that disease

A2 be the event that person was healthy

D be the event that person got the test positive

P  D / A1  0.9 , P  D / A2  0.1, A1 0.01, A2 0.99

P D P  A1  P  D / A1   P  A2  P  D / A2  0.108

Required Probability is

( P  A1  P  D / A1 
P  A1 / D  0.08333
( P  A1  P  D / A1   ( P  D / A2  P  A2 

4.4 Chapter End Exercises

1. What is the probability that(i)husband, wife and daughter have same birthday
(ii)two children have birthday in March?
2. 4 soldiers A,B,C and D fire at a target .Their chances of hitting the target are
0.4,0.3,0.75, and 0.6 respectively. They fire simultaneously. What is the
chance that(i) the target is not hit? (ii) the target is hit by exactly one of them.
3. If A, B, C are independent show that, (i) A, B ˆ C are independent(ii)A,
B ‰ C are independent(iii) A , B  C are independent

41
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

4. : ^1, 2,3,4` , A ^1,2` List all ]3 such that A, B are independent.

5. If P  A / B   P  A then P  A / B  ! P  A  , and vice versa.

6. Show that
P  A   P  An B 
P A / B
1 P B
, P  B  z 0 ,hence prove P  AnB  t P  A   P  B   1

7. Examine for pairwise and mutual independence of events K, R , and S

which are respectively getting of a king, red and spade card in a random draw
from a well shuffled pack of 52 cards.
8. Urn A contains numbers 1 to 10 and B contains numbers 6 to 15. An urn is
selected at random and from it a number is drawn at random. What is the
probability of urn l 3 was selected, if the number drawn is less than 7.
9. In a population of 55 % males and 45% females, 4% of the males and 1% of
females are colorblind. *ind the probability a randomly selected person is
colorblind person 7.
10. A man is equally likely to drive by one of the three routes A,B,and C from
his home to office.The chances of being late to the office are 0.2,
0.4,0.3 ,provided he has chosen the routes A, B,c respectively.If he was late
on riday what is the prob. that he has chosen route C ?

™™™

42
 5
RANDOM VARIABLE AND ITS
DISTRIBUTION FUNCTION
Unit Structure
5.0 Objectives
5.1 Random Variable
5.2 Distribution unction
5.3 Discrete random variable and its p.m.f
5.4 Continuous random variable and its p.d.f
5.5 Chapter End Exercises

5.0 Objectives
After going through this chapter you will learn
x A real valued function defined on sample space,known as random variable
x Discrete and continuous r.v
x Distribution function of a r.v and its association with probability measure.
x Properties of Distribution function
x Probability mass function of a r.v.
x Probability density function of a r.v.

5.1 Random Variable

Definition 5.1. Random Variable  :, C  be a measurable space. A real valued
function X defined on this space is called as a random variable if every inverse
image is a Borel set. That n s for all B  B we have

X 1  B  ^Z | X  w   B`  C
x Random variable is abbreviated by ‘ r.v ’

x X is a r.v iff for each x  ,^ X d x`  C.

x X : : o  .Further ^Z  : : X Z   B` is an event or it is C.

43
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

x In chapter 2 we have seen that all intervals semi̺ open,semi̺ closed,single

tones are  B .That is B may be  f, a  or [a, f ) etc.,so

^ x`,{a  X  b},{a  X d b},{a d X  b},^a d X d b` are all   hence are

events.
Example 5.1. Show that Indicator function is a r.v.
Solution: Indicator function is defined for a A Ž :

­1 Z  A
I A Z  ® (5.1)
¯0 Z  A

I A Z  is a r.v on  :, C  iff A  C.

Example 5.2. Consider  :, C  be a sample space, where : {HH , HT.TH, TT }

If X  w  Number of heads in Z , C is sigma field, Is X a r.v ̓?

Solution: If X Z  Number of heads in Z then X  HH  2, X ( fi[T )

1, X TH  1, X  I J T  0

­M x0
°TT 0 d x 1
°
X 1  f, x @ ® (5.2)
° HT , TH , HH 1 d x  2
°¯: xt2

All X 1 are events, so X is a r.v.

1
Example 5.3. If X is a r.v are following functions r.v ? (i) aX  b (ii)
X

Solution: If X is a r.v ^Z : X d x`  C

­ x b½
Case I : a ! 0, b   then ®Z : X d ¾C
¯ a ¿

{Z : aX  b d x}  C

­ x b½
Case II: a  0, b   then as a complement of ®Z : X d ¾C
¯ a ¿

­ xb ½
®Z : X ! , a  0¾  C
¯ a ¿
{Z : aX d x  b}  C

44
 Chapter 5: Random Variable and its Distribution Function

^Z : aX  b d x`  C
Case III: a 0

­: x  b t 0
^Z : aX  b d x` ^Z : X d x  b` ® (5.3)
¯M x  b  0

Thus aX  b is a r.v.

(ii) Let

ª1 º ­1 1 ½ ­1 ½
«¬ X d x »¼ ® d x, X ! 0}U{ d x, X  0 ¾ U ® d x, X
¯X X ¿ ¯X
0¾
¿

­{ X  0} x 0
° 1 when x positive
ª 1 º °°{ X d x, X  0}U{ X t , X ! 0}
«¬ w : X d x »¼ ® x,0
°­ 1 1
°® X t ¾½ U ®­ X d , X ! 0}¾½ when x is negative
°¯¯ x¿ ¯ x ¿

ª1 º 1
All events are  C So, « d x » is an event, hence is ar. v
¬X ¼ X

Example 5.4. : ^1, 2,3, 4`, C ^M , :,^1`^2,3, 4``

Is X Z  1  Z is a random variable with respect to  :, C  ?

Solution: inverse image of ^Z  : : X Z  3` ^2`  C

So X Z  1  Z is not a random variable.

5.2 Distribution Function

Define a probability measure PX by PX P  X  B  ,this is a mapping PX :

 o > 0,1@. is a sigma field of borel sets. We define a point function P associated
with probability space  :, C , P 

Definition 5.2. Distribution Function of a random variable: A mapping FX :

45
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

IR \ o > 0,1@ defined by FX P ª¬Z : X Z  d x º¼ is called as a distribution

function FX  x  of X.

Example 5.5. One of the numbers 2, ... 1, 2 is chosen at random by throwing a pair
of dice and adding the numbers shown on the two faces. You win $9 in case 2,3, 11
or 12 comes out or lose $ 10 if the outcome is 7, otherwise you do not lose or win
anything. Find P[ X ! 0] and P[ X  0]

Solution : : ^ a, b  : a, b 1, 2,3, 4,5,6` X : : o  define as

­9 if a  b 2,3,11,12
°
X ( v  X  a, b  ®10 if a  b 7 (5.4)
°0 if a  b 4,5,6,8,9,10
¯
1 1
P[ X ! 0] ^Z : X Z  9` P ¬ª1,1 , 1, 2  ,  2,1 ,  6,5  5,6  6,6  º¼
6
P[ X  0] ^Z : X Z  7` 6

Example 5.6. X Z  1 for (Z  A, X Z  2 for Z  B, X Z  2 otherwise.

1 1
B are dis / oint . Find d . f of X. P  A  , P  B
3 2
Solution: x is r.v , inverse image must be an event,
­0 x  2
°B  2 d x 1
°
X 1  f, x @ ®
°A 1d x  2
°¯: xt2
d . fFX  x  of X , then FX  x  PX (f, x ]
­0 x  2
°
°P  B  1 2 d x  1
° 2
FX  x  P  f, x @ ®
° P  A  P  B  1 1 5
 1d x  2
° 2 3 6
°P : 1 xt2
¯
x We can establish suitable correspondence between P and F as
FX PX (f, x ]

x P[a  X d b] FXJ  b   FX  a 

Distribution function (d.f) has following properties.

x FX  x  is non negative for all x  

46
 Chapter 5: Random Variable and its Distribution Function

proof: We can easily verify this as FX P > X d x @ ,and P is a probability

measure.

x FX  x  is Monotonically non decreasing.

proof: Let x1, x2   such that x1 d x2
 f, x1    f, x2 
Using monotone property of P, PX  f, x1  d PX  f, x2 
FX  x1  d FX  x2 

x FX  x  is right continuous.

proof: consider a sequence xn p x such that x1 ! x2 } ! xn , and events

Bn ( x, xn ] now as Bn are decreasing events as seen in chapter 1,
Bn p n n Bn M , and using continuity property of P

0 P M  
P lim Bn
n
 lim P  Bn  lim P  x, xn @ lim FX  xn   FX  x 
n n n

This implies right continuity,

lim FX  xn  FX  x  (5.5)
n

(i) lim FX  x  FX  f  0.

(ii) lim FX  f  1.

proof: (i)Let x1 ! x2 } ! xn , and events Bn (f, xn ] now as Bn are decreasing

events as seen in chapter 1, Bn p n n Bn M , and using continuity property of P

0 P M  P ( lim Bn ) lim P  Bn  lim P(f, xn ] (5.6)

n n n

0 lim FX  xrL   FXJ  f 

n

lim FX  xn  FX  f  0 (5.7)
nof

(ii)Let x1  x2 } xn , and events Bn (f, xn ] now as Bn are increasing events as

seen in chapter 1, Bn n U n Bn : , and using continuity property of P

47
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

1 P : P  U n Bn  P ( lim Bn ) lim P  Bn  (5.8)

n n

1 lim P  f, xn @ lim FX  xn   FX  f 

n n

But FX  f  0
lim FX  xn  1 (5.9)
nof

Theorem 5.1. Every d . f is a d . f of some r.v.

Remark 5.1. If X is a r.v on  :, CP  with FX P ª¬Z : X Z  d x º¼ is associated

d. f .

By above theorem for every r.v we associate a d.f.on some prob. space.Thus given
a r.v there exists a d.f and conversely.
Example 5.7. 1. Write d . f of the following r.v.s

(i) X Z  C , for all Z  : C is constant

(ii) X is no. heads in tossing two coins.

Solution :  i  FX  x  P > X d x @ PX  f, x @ 0 i f x  C

FX  x  P > X d x @ PX  f, x @ 1 i f x t C

1
(ii) X No . of heads, : ^HH , HT ,TH ,TT ` P> X 0@ ) P> X 1@
4
1 1
, P> X 2@
2 4
­0, x  0;
°1
° ) 0 d x  1)
°4
FX x  P > X d x @®
°3, 1 d x  2;
°4
°1 x t 2.
¯

5.3 Discrete random variable and its p.m.f

Definition 5.3. Discrete random variable: A random variable X is called as
discrete if there exist an at most countable set D such that P > X  D @ 1

x Set D contains countable points ^X xi ` .They have non negative

mass .They are called as jump points or the point of increase of d.f.As seen

48
 Chapter 5: Random Variable and its Distribution Function

before in chapter 1, ^ X xi `   . And ^Z : X Z  xi ` is an event.We can

assignPX ª¬Z : X Z  xi º¼ denoted by p  xi  such that (i) p  xi  t 0 and

¦ p x  i 1

x X is a discrete random variable if and only if PX is a discrete probability

measure.
x The distribution function of a discrete r.v is a step function.As
P > X x @ FX  x   FX  x  this jump at x.WhereFX  x  lim FX  x  h 

x Random variable has its characteristic probability law. *or discrete r.v it is
also called as probability ass function (p.m.f)
x For X discrete,  i  P(a  X d b) FX  b   FX  a 

 ii  P(a  X  b) FX  b   FX  a   P > X b@

 iii  P  a d X d b  FX  b   FX  a   P > X a@

 iv  P(a d X  b) FX  b   FX  a   P > X a@  P > X b@

Definition 5.4. Probability mass function: A collection p  xi  which is

representing P  X xi  satisfying (i) p  xi  t 0 and ¦ p x  i 1 is called as

probability mass function  p.m. f  of a discrete random variable X .

Example 5.8. Let X be no. of tosses of a coin up to and including the toss showing
head first time. (i) Write p.m. f of X hence find P [ X is even]. (ii) Also write
d . f of X .

Solution: (i) Let P̓ be the chance of showing head. 1  p q ) is chance of

showing tail.

P> X x @ P { x  1 tosses are tail, x th toss is head} pq x1 ; for x 1, 2}

P [ X is even] P ^w : x even` P  Ui even ^ x i` ¦i evenP ^ x i`

i
pq ª ¦ i  q  º»¼
f
P [ X is even] p ª¬ q1  q 3  q 5 }º¼ 2
«¬ 1

49
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Using infinite geometric series with common ratio q 2

pq q
P [ X is even]
1  q2 q 1

(ii) d . f of X is

n
FX  x  P > X d x@ ¦ x 1
p  x  1  q7l

­0 x 1
FX  x  P > X d x@ ® n
¯1  q n d x  n  1; n 1, 2,

5.4 Continuous random variable and its p.d.f

Definition 5.5. Continuous random variable : Reject random variable X is

defined on  :, CP  with d . f FX is said to be continuous if F is absolutely
continuous.

* is absolutely continuous if there exists a density f X : X o > 0,1@ defined as

b
P ª¬ X   a, b @º¼ PX  a, b @ ³ f  x  dx (5.10)
a X

For every a,b   This function f is called as probability density function

(p.d.f)of a continuous r.v X

Definition 5.6. Probability density function: If f is p.d . f of a continuous r.vX

with d . f F , it satisfies
1. f t 0
f
2. ³ f X
f  x  dx 1
b
3. P ª¬ X   a, b @º¼ ³ f  x  dx FX  b   FX  a 
a X

x For P absolutely continuous and f continuous for all x then

dFX
f  x (5.11)
dx

x For continuous r.v FX is continuous, right as well as left.

FX  x  FX  x  FX  x  Where FXJ  x  lim FX  x  h 

50
 Chapter 5: Random Variable and its Distribution Function

x From above it is clear that P > X x @ 0 for continuous r.v

x For X continuous, P(a  X  b) P(a  X d b) P  a d X d b  P(a d X  b)

FX  b   FX  a 

Example 5.9. Coin is tossed. If it shows head you pay Rs.2 . If it show tail you spin
a wheel which gives the amount to you, distributed with uniform prob. between
Rs.0 to 10 you gain or loss is a random variable. Find the distribution function
and use it to compute the probability that you will win at least 5.
1 1 x
Solution : P > X 2@ and for > 0,10@ , f  x  , so FX  x 
2 10 10

­0 x  2
°
° 1
FX  x  ® 0 d x  10  2 d x  0
° 0 2,1x
°¯1 x t 10.

1
P [ X is at least 5 ] 1  F  5 .
2
Example 5.10. A r. X has p.d . f

­k
° 2 x t 100
f  x ®x
°¯0 otherwise

1
Find  i  k  ii  P[50  X  200] (iii) M such that P[ X  M ]
2
f
Solution: (i) ³ f X
f  x  dx 1 So,

f k k
³ dx k[  x 1 |100
f
i00 x2 100
k
? 1 , gives k 100
100
(ii)
200 100 1
P[50  X  200] ³ dx
150 x2 3
1
 iii  M such that P[ X  M ] so
2

51
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

M 100 1
³ dx
100 x2 2
gives M 200

5.5 Chapter End Exercises

1. Find the smallest V ̺ field on : .Let X D f  c Š is a random variable

on :
^2, 1,0,1,2`
2. Two dice are rolled. Let r.v X be the larger of the two numbers shown.
Compute
PX > 2, 4@ .

3. : >0,1@ and C is a V ̺ field of borel sets in :

1
Is(i) X ( v  : ii  X  w w is a random variable on : w.r.t C
2

4. A r.X has p.d.f .*ind its d.f . Hence P[ X ! 0.5]

­1  x 1  x  0
°
f  x ®1  x 0 d x  1
°0 otherwise
¯

5. A r.X has d.f , find its p.d.f

­ 0 0 x
° x
° 0 d x 1
° 2
°° 1
FX  x  ® 1d x  2
° 2
°3  x
° 2 2d x3
°
°̄ 1 xt3

6. ff X is a r.v are following functions r.v?  i  X 2  ii  X  iii  X

™™™

52
 6
SOME SPECIAL RANDOM VARIABLES AND
THEIR DISTRIBUTIONS
Unit Structure
6.0 Objectives
6.1 Bernoulli and Binomial distribution
6.2 Poisson distribution
6.3 Normal Distribution
6.4 Chapter End Exercises

6.0 Objectives

After going through this chapter you will learn

x Bernoulli and Binomial distribution and their properties.
x Poisson distribution its relation with Binomial distribution
x Normal distribution and its applications.

6.1 Bernoulli and Binomial distribution

In this chapter we will come across some typical r.v s and their distributions. In
real life situation we come across many experiments which result into only two
mutually exclusive outcomes. Generally the outcome of interest is called as ͂
Success’ and other as ͂Failure̓.We assign a positive probability ͂p̓ to
success and 1 p  to failure.

Definition 6.1. Bernoulli r.v : Ar.vX assuming values 1 and 0 with probabilities
‘p̓ and 1 p  is called as Bernoulli r .v

x Thus Bernoulli r.v is same indicator function I A with ͂A̓ as success.

x The probability law of Bernoulli is also written as

1 x 
P> X x@ p x 1  p  x 0,1

x Hence onward we denote 1 p by ͂q̓.Note that p  q 1

53
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Example 6.1. 1. An indicator function I A is a Bernoulli r .v , if we assign

probability P  A  p and P  A  1  p

When the trial of a Bernoulli experiment is repeated independently finite number

of times say ͂n̓ times, it gives rise to Binomial situation. If we count total
number of successes in such n trials it is Binomial r.v .The probability law for
n
Binomial r.v has nth term of binomial expansion of  p  q 

Definition 6.2. Binomial distribution: A r.vX assuming values 0,1, 2}n is said
to follow Binomial distribution if its p.m. f is given by P[ X x1

­° nC x p x q  n x  x 0,1,} n;0  p  1, p  q 1
® (6.1)
°̄0otherwise

x The notation X o B  n, p  is used for showing X follows Binomial

distribution with parameters n and p.

x Bernoulli r.v is particular case of Binomial with n 1 . And Binomial arises

from sum of n independent Bernoulli r.v.s.

x The Binomial probabilities can be evaluated by the following recurrence

relation, starting from P > X 0@ q n And then using recursive formula,

nx§ p·
P> X x  1@ ¨ ¸ P> X x@ (6.2)
x 1 © q ¹

This is forward formula. We can also start from P > X n@ and use the equation as
a backward formula.

x A r.v counting number of successes in n bernoulli trials follows 13(n,p) and

counting number of failures n  X Y say, in n bernoulli trials follows
B(n,q).

x We can easily verify that P ª¬ X   0,1} n  º¼ 1 , hence Binomial is a discrete

r.v.
Example 6.2.

X o B  n, p  and if y n  x, Y o B  n, q  , then show that

n
PX > X r @ PY >Y n  r@ Cr p r q nr

54
 Chapter 6: Some Special Random variables and their Distributions

Solution:
n
PX > X r@ Cr p r q n  r

n
PY >Y n  r@ Cn  r q n  r p r

n
But nCnr Cr
so,
n
PX > X r @ PY >Y n  r@ Cr p r q nr

Head is thrice as likely as tail for a coin. It is flipped 4 times (i) Write p.m. f of X,
representing number of heads observed in this experiment.  ii  find probability of
getting 3 or 4 heads.
3
Solution : P  H  3P T  , so P  H  p ,tossing coin once is a Bernoulli trial.
4
X counting number of heads observed in tossing such bais coin 4 times.  i  X
§ 3·
follows B ¨ n 4, p ¸
© 4¹

P> X x@

­ 4 § 3 · x § 1 · n  x 
° Cx ¨ ¸ ¨ ¸ x 0,1, 2,3, 4
® ©4¹ ©4¹ (6.3)
°0 otherwise
¯

Required Prob P > X t 3@ 1  P > X d 2@

4 3 2 2
§1· §1· §3· §1· §3·
P > X d 2@ ¨ ¸  4 ¨ ¸ ¨ ¸  6 ¨ ¸ ¨ ¸ 0.9198
©4¹ ©4¹ ©4¹ ©4¹ ©4¹

So, P > X t 3@ 0.08016

Example 6.3. It is found that 60r0 of the health care victims are senior citizens . If
a random sample of 10 victims is taken, what is the probability of getting exactly 3
senior citizens in this sample ,
Solution: X is number of victims who are senior citizens in this sample. X
10
follows B  n 10, p 0.6  P > X 3@ C3 0.630.47 0.04247

Example 6.4. X follows B  n 6, p  such that P > X 2@ P > X 4@ Find p.

55
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Solution: P > X 2@ P> X 4@

6
C2 p 2 q  4 6
C4 p 4 q 2

1
p2 q 2 means p q
2

Example 6.5. X follows B  n, p  , Y follows B  m, p  , If X , Y are independent.

Find the probability distribution of X  Y . Solution: Consider P > X  Y k @ , for
k 0,1 , ... m  n

P> X  Y k@ P ª¬ U nx 0 X x, Y k  x º¼

n
¦ x 0
P> X x, Y k  x@ (6.4)

n
¦ x 0
P> X x @ P >Y k  x @ Since r.v.s are independent

n
¦ ªC x p x q  n  x  º ªC x p k  x q  m  k  x  º (6.5)
x 0 ¬ ¼¬ ¼

n
¦ ªC xmCk  x p k q  mnk  º (6.6)
x 0 ¬ ¼

n
¦ ªCkm p k q  mnk  º (6.7)
x 0 ¬ ¼

Thus X  Y follows B  n  m, p  .

6.2 Poisson distribution

Now let us introduce another commonly used discrete r.v . Many a times we come
across a r.v counting number of occurrences in a fixed duration of time. *or
example, Number of deaths due to Maleria per month in Mumbai, Number of
accidents per hour on a express highway. Thc number of defects in cloth per square
metor is similar occasion where Poisson distribution is appropriate

Definition 6.3. Poisson distribution: A discrete r.vX assuming values 0,1, 2}f is
said to follow Poisson distribution if its p.m. f is given by

­ eO O x
° ; x 0,1,}f; O ! 0
P> X x@ ® x! (6.8)
°0 otherwise
¯

56
 Chapter 6: Some Special Random variables and their Distributions

x The notation X o (  O  is used for showing X follows Poisson distribution

with parameter O

x The Poisson probabilities can be evaluated by the following recurrence

relation,starting from P > X 0@ e  O And then using recursive formula,
O
P> X x  1@ P> X x@ (6.9)
x 1
all probabilities can be evaluated. Tables are also available for various values of O

x We can easily verify that P ª¬ X   0,1}f  º¼ 1 , hence Poisson is a discrete

r.v.

x In Binomial situation if number of successes are very large and the chance of
success is very small, but average number ͂ np ̓ is fixed say O then,
Binomial probabilities tends to Poisson as n becomes very large

Theorem 6.1. Poisson as a limiting distribution of Binomial X o B  n, p  , then if

p is very small and n becomes very large, but ͂np̓ remains constant O then
Binomial probabilities tends to Poisson probabilities

eO O x
lim nC x p x q  n x  x 0,1 , ... f (6.10)
nof x!

Proof: X o B  n, p  , so P > X x@ n
C x p x q  n x 

O
By putting p
n
x n x
n! § O · § O · § O ·
¨ ¸ ¨1  ¸ ¨1  ¸
x !n  x ! © n ¹ © n ¹ © n ¹
x
n! n  n  1 n  2 } n  x  1  O  Ox
lim x
= 1
rL of x !n  x ! x! n x
n x
§ O· § O·
lim ¨ 1  ¸ ¨ 1  ¸ eO
nof
© n¹ © n¹
Hence

n x  n x eO O x
lim C x p q x 0,1 , ... f . (6.11)
n of x!

57
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Limiting distribution of Binomial is Poisson.

Example 6.6. A sales firm receives on an average three toll̺free calls per hour.
For any given hour find the probability that firm receives  i  At most three calls.
 ii  At least three calls
Solution: X No . of toll free calls a firm receives X o (  O 3
(i) required prob
3 eO O x
P > X d 3@ ¦ 0.6472
x 0
x!
(ii) required prob
f eO O x
P > X t 3@ ¦ 1  P > X d 2@ 1  0.5768 0.4232
x 3
x!
Example 6.7. A r.vX (  O  , such that P > X 4@ P > x 5@ . Find O Solution: As
P> X 4 @ P > x 5@ .
e O O 4 eO O 5
4! 5!
So, O 5
Example 6.8. A safety device in laboratory is set to activate an alarm, if it register
5 or more radio active particles within one second. If the back ground radiation is
such that, the no of particles reaching the device has the poison distribution with
parameter O 0.5 flow likely is it that alarm will be activated within a given one
second period?
Solution: Let X be number of particles reaching safety device within a one sec.
period . X o (  O 0.5  The alarm will be activate if X t 5
P > x t 5@
f e O O x
¦ 0.112
x 5
x!
Example 6.9. X 1 o (  O1  and an independent r.vX 2 o (  O2 
Show that X 1  X 2 o (  O1  O2  .
Solution: Consider P > X  Y k @ , for k 0,1 , ... f
f
P> X  Y k @ P ª¬ Ufx 0 X 1 x, X 2 k  x º¼ ¦ x 0
P > X1 x, X 2 k  x@
f
¦ P > X1
x 0
x@ P > X 2 k  x @ Since r.vs are independent

f ª e  O1  O1  x º ª e O2  O2  k  x  º f ª e O1 O2   O1  O2 k º
¦ x 0
«
«¬
»«
x ! »¼ «  k  x ! »
» ¦ x 0
«
«¬ k!
»
»¼
¬ ¼
Thus X 1  X 2 o (  O1  O2  .

58
 Chapter 6: Some Special Random variables and their Distributions

Example 6.10. 2% students are left̺handed. In a class of 200 students find the
probability that exactly 5 are left̺handed.

Solution: X is no. of left ̺ handed in 200, p .02 and n 200 thus X has
B(n 200, p . 02) . Using Poisson as a limiting distribution of Binomial,

X o ( O np 4

e 4 O 5
P > x t 5@ 0.1563
5!

6.3 Normal Distribution

Definition 6.4. Normal Distribution: A continuous r .v is said to follow Normal
distribution if its p.d . f is given by

­ 1 § x  P ·2
°e z¨ ¸
f  x
° © V ¹  f  xf; P   V ! 0 (6.12)
®
° 2SV
°̄

x The notation X o N  P ,0 2  is used to show that X follows normal

distribution with parameters P and V 2

x Normal distribution is applicable to wide range of situations in real life.

x P Mean of X and 02 Variance of X

x When P 0 and V 2 1 it is called as standard normal distribution. The

tables for P > X d x @ are available for this distribution.

x Since any X o N  P ,V 2  its linear combination Y aX  b also has

Normal distribution with parameters aP  b and a 2V 2

X P
x If X o N  P ,V 2  then Z has standard normal distribution.
D
x We denote by M  z  P > Z d z @ as d.f of N  0,1 .

x    
X 1 o N P1 , V 12 and X 2 o N P 2 , V 22 ,if X1, X 2 are independent r.vs,then


X 1  X 2 o N P1  P1 , V 22  V 12 

59
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Example 6.11. The score of the test are normally distributed with
N  P 100,V 15 . Find the probability that score is below 112.

Solution: Score is denoted by X , X o N  P 100,V 15 

112  100
P[ X  112 @ P> Z  ] M  0.8  0.7881
15
X P
, By normal tables. Since Z has standard normal distribution.
V

Example 6.12. X 1 o N  4,1.52  and X 2 o N  2, 2 2  , if X1, X 2 are independent

r.vs , then find P > X 1  X 2 t 1@

Solution. X 1 o N  P1 ,D12  and X 2 o N  P 2 ,022  , if X1, X 2 are independent

r.v.s., then

X 1 X 2 o N (2,  2.25  4 )asX 1 , X 2 areindependenat r.v.s

ª § 1  2 ·º
P > X 1   X 2 t 1@ P «Z t ¨ ¸ » 1  M (0.4) I  0.4  0.6554
¬ © 25 ¹ ¼

6.4 Chapter End Exercises

1. In order to qualify police academy, candidates must have score in top 10%in
the general ability test. If the test scores X o N  P 200,D 20  , find the
lowest possible score to qualify.

2. X 1 o N 10, 2.52  and X 2 o N 12, 22  ,if X1 , X 2 are independent

r.vs,then,find mean and variance of X1  X 2

3. On an average 0.2% of the screws are defective. *ind the probability that in
a random sample of such 200 screws we get exactly 3 defective screws.

P> X 4@ 3
4. X o (  O  . *indO if .
P> X 3@ 8

P> X 4@ 3
5. X follows B  n 5, p  . Find p if .
P> X 3@ 8

6. A video tape on an average one defect every 1000 feet.What is the probability
of atleast one defect in 3000 feet?

60
 Chapter 6: Some Special Random variables and their Distributions

7. 3% of all the cars fail emission inspection. *ind the probability that in a
sample of 90 cars three will fail.Use(i) Binomial distribution (ii)Poisson
approximation to Binomial.

8. If a student randomly guesses at five multiple choice questions, find the

probability of getting three or more correct answers .There are four possible
options for each question.

9. X o N  P 10, V 3 . Find the probability that

(i) X is less than 13 but X is greater than 7. ii  Y 2 X  3 ,then Y  26.

 iii  X 2100  iv  X ! 8.

§ i·
10. X i follows B ¨ n 5, ¸ . where i 1,2. i  Write p.m f of X1  X 2
© 3¹
(ii) Find P > X 1  X 2 d 3@

™™™

61
 7
TWO-DIMENSIONAL R.V.S
Unit Structure
7.0 Objectives
7.1 Probability Distributions of two-dimensional discrete r.v.s
7.2 Probability Distributions of two dimensional continuous r.v. s
7.3 Conditional Probability distributions
7.4 Independence of r.v.s
7.5 Chapter End Exercises

7.0 Objectives

After going through this chapter, you will learn

x Two dimensionaldiscrete r.v.s and its Joint Probability mass function.

x Two-dimensional continuous r.v.s and its joint Probability density function.

x From joint probability function of two-dimensional r.v.s finding marginal

Probability laws.

x The conditional distributions of the r.v.s.

x Notion of independence of r.v.s and its consequences.

7.1 Probability Distributions of two-dimensional discrete r.v.s

The notion of r.v can be extended to multivariate case .In particular if X and Y are
two r.v.s. defined on same probability spacc  :, C ,, P  then ^ X ,Y   B`  C , for
any borel set B in  2 .Note that this Borel set is a 0˜ field generated by rectangles
 a , b  X  c, d  The mapping  X , Y  :  :, C  o  2 is a two dimensional r.v.

Definition 7.1. Joint Probability Distribution of two dimensional r.v.s : The

probability measure PX ,Y defined on  2 is called as Joint Probability Distribution
of two dimensional r.v.s  X , Y  where

62
 Chapter 7: Two-Dimensional R.V.S

PX ,Y  B  P ª¬ X , Y   B º¼ for every borel set B   2 (7.1)

Definition 7.2. Two dimensional discrete r.v . The two-dimensional random
variable  X , Y  is called as discrete if there exist an at most countable set D such
that PX ,Y  D  1
Two r.v.s are jointly discrete if and only if they are discrete.
x The joint probability law PX ,Y Of two dimensional discrete r.v s satisfy
following
1. PX ,Y  x, y  t 0 for all x,y

2. ¦ ¦ P  x, y 
x y X ,Y
1

x The joint probability law PX ,Y Of two dimensional discrete r.v s is also

called as joint probability mass function of  X , Y 

x PX  x  ¦ P  x, y  is called as Marginal p.m.f of X.

y X ,Y

x PY  y  ¦ P  x, y  is called as Marginal p.m.f of Y.

x X ,y

x Marginal p.mfs are proper p.m.fs of one dimensional discrete r.v.s.

Example 7.1. Given following verify which are joint probability mass function of
joint p.m. f , if so, find the constant K .
(i) PX ,Y  x, y  K  x  y  for x 1, 2,3 y 1, 2.
(ii) PX ,Y  x, y  Kxy for x 1,0,1; y 1,0,1
Solution:  i  PX ,Y  x, y  t 0 for all x, y if K ! 0
And

¦ ¦ P  x, y 
x y X ,Y

P 1,1  P 1,2   P  2,1  P  2,2   P  3,1  P  3,2  21K 1

1
So, for PX,Y  x, y  to be proper joint p.m.f K ,
21
(ii) P 1, 1  0 , we can not have positive prob for remaining pairs, if K is
selected negative. Which means that for no K , PX ,y  x, y  ! 0 PX ,y  x, y  is not
proper joint p.m. f
Example 7.2. Two cards are drawn from a pack of cards. Let X denotes no. of
heart cards and Y no. of red cards. Find the joint p.m. f of r.v  X , Y  . Hence
P> X Y@

63
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Solution: x, y 0,1, 2 . following will be / ointp.m. f of  X , Y 

13 u 25 25
P 1, 2  P >1heart , 2red @
52C2 102
and so on.
YЎ XЍ 0 1 2 P[Y = y]
0 25 0 0 25
102 102
1 26 26 0 52
102 102 102
2 6 13 6 25
102 102 102 102
P[X = x] 57 39 6 1
102 102 102
And
25  26  6 19
P> X Y @ P  0,0   P 1,1  P  2,2 
102 34

Example 7.3. Using the above joint p.m. f of X , Y

Solution: PX  x  ¦ P  x, y 
X ,y find marginal p.m. f .s of r.v.sX , Y . is as
Marginal p.m. f of X .

and PY  y  ¦ P  x, y  is Marginal
X ,Y p.m. f of Y .
X 0 1 2
PX(x) 19 13 1
34 34 17

Y 0 1 2
Py(x) 25 52 25
102 102 102

7.2 Probability Distributions of two dimensional continuous r.v.s

Definition 7.3. Two dimensional continuous r.v . and their joint probability density
function: The two dimensional random variable  X , Y  is called as continuous if
there exists a function f X ,Y :  2 o > 0,1@

64
 Chapter 7: Two-Dimensional R.V.S

satisfying
1. f X ,Y  x, y  t 0 for all x, y
f f
2. ³ ³
f f X ,Y
f  x, y  dxdy 1
b d
3. ³³ f X ,Y dxdy P > a d X d b, c d Y d d @ where a, b, c , and dare  
a c

f
gX  x ³ f  x, y  dy is called as Marginal p.d.f of X.
f X ,y

f
hy  y  ³ f  x, y  dx is called as Marginal p.d.f of Y.
f X ,y

Marginal p.d.fs are proper p.m.fs of one dimensional continuous r.v.s

Example 7.4. Given following verify which are joint probability density function
of  X , Y ) if so, find the constant K .

(i) f X ,y  x, y  Ke  x y  for x t 0 y t 0.

(ii) f X ,y  x, y  Kxy for 0  x  y  1

Solution: (i) f X ,Y  x, y  t 0 for all x, y if K ! 0

And
f f
³ ³
f f X ,Y
f  x, y  dxdy
f f f
³³ Ke  x y  dxdy K .³ e  y (e  x |0f dy K
0 0 0

So , for f X ,Y  x, y  to be proper joint p.d . fK 1

(ii) f X ,Y  x, y  t 0 for all x, y if K ! 0

And
f f 1 y
³ ³ f f X ,Y
f  x, y  dxdy ³ 0 ³ 0 Kxydxdy
1 x2 y y4 1 K
K ³ y. |0 d y K |0
0 2 8 8
So, for f X ,Y  x, y  to be proper joint p.d . fK 8

Example 7.5. For the above two joint p.d . fs find  I  P[ X  0.5, y  0.5]
(II)Marginal p d . f .s of X and Y .

Solution:(i) f X ,y  x, y  e  x y  for x t 0 y t 0.

(I)
0.5 0.5  x  y
0.5 2
P[ X  0.5, Y  0.5] ³ ³
0 0
 
e dxdy 1  e 
65
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

f
 II  g X  x  ³ f f X ,Y  x, y  dy is Marginal p.d . f of X .
f  x y
 
Marginal p.d . f of X. g X  x  ³ e dy e  x for x t 0
0
f
hY  y  ³ f  x, y  dx is Marginal p.d . f of Y .
f X ,Y
f
Marginal p.d . f of Y. hY  y  ³ f X ,Y  x, y  dx e  y for y t 0
0

(ii) f X ,y  x, y  8 xy for 0  x  y  1
(I)
0.5 y
P[ X  0.5, y  0.5] ³ ³ 8 xydxdy
0 0
2 3
0.5 x y 0.5 y 4
8³ y |0 dy 8³ dy  0.5  0.0625
0 2 0 2

1
(II) Marginal p.d . f of X. g X  x  ³ 8 xydy
x
 
4 x 1  x 2 for 0  x  1
y
Marginal p.d . f of Y. hY  y  ³ 8 xydx 4 y 3 for 0  y  1
0

Example 7.6. The joint probability density function of  X , Y 

­1  xy
° x  1, y  1
f  x, y  ® 4
°¯0 Othrewise

Find P[ X 2  u,Y 2  v]
Solution:
P[ X 2  u , Y 2  v @ P>  u  X  u , v  Y  v ]
u v 1  xy
³ ³ dxdy uv
 u  v 4

7.3 Conditional Probability distributions

Definition 7.4. Conditional Probability mass function: Let the joint probability law
of a two dimensional discrete r.v  X , Y  be PX ,Y and the marginal p.m. f of X
and Y be PX  x  , PY  y  respectively, then the conditional p.m. f of X given
Y y is given by

PX ,y  x, y 
PX /y y  x for all x, providedPy  y  z 0 (7.2)
Py  y 

66
 Chapter 7: Two-Dimensional R.V.S

And the conditional p.m. f of Y given X x is given by

PX ,Y  x, y 
PY / X x  y for all y , provided PX  x  z 0 (7.3)
PX  x 

Note that conditional p.m.f s are proper p.m.f.

Definition 7.5. Conditional Probability density function : Let the joint probability
law of a two dimensional continuous r.v. X , Y  is f X ,Y  x, y  , and the marginal
p.d . f of X and Y be g X  x  , hY  y  respectively, then the conditional p.d . f of
X given Y y is given by
f X ,Y  x, y 
g X /y y  x for all x , provided hY  y  z 0 (7.4)
hY  y 

And the conditional p.d . f of Y given X x is given by

f X ,Y  x, y 
hY / X x  y for all y , provided g x  x  z 0 (7.5)
gX  x

Note that conditional p.d.f s are proper p.d.f.s

Example 7.7. There are 4 tickets in a bowl, two are numbered 1 and other
numbered numbers 2. Two tickets are chosen at random from the bowl. X denotes
the smaller of the numbers on the tickets drawn, and Y denotes the smaller of the
numbers on the tickets drawn.
(i)Find the joint p.m. f . of r.v. X , Y 

(ii)Find the conditional p.m. f of Y given X 2 (iii) Find the conditional p.m. f
of X given Y 2.

Solution: : ^1,1 , 1, 2  ,  2,1 ,  2, 2 `

joint p . m.f. of r.v. X , y 
YЎ XЍ 1 2 PY(y)
1 1 0 1
4 4
2 1 1 3
2 4 4
PX(x) 3 1 1
4 4
The conditional p.m. f of Y given X 2 is given by
PX ,Y  x, y 
PY / X 2  y 1
for y 1, 2 . (7.6)

67
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

y 1 2
PY/X=2(y) 0 1

The conditional p.m. f of X given Y 2 is given by

PX ,Y  x, y 
PX /Y 2  x 3
forx 1, 2 . (7.7)
4
x 1 2
PX/Y=2(x) 2 1
3 3

Example 7.8. The joint p.d . f of f X ,Y  x, y  8 xy for 0  x  y  1

r.v. X , Y 

(i)Find the conditional p.d . f of Y given X x (ii) Find the conditional p.d . f of
X given Y y.

Solution: f X ,y  x, y  8 xy for 0  x  y  1
1
Marginal p.d . f of X. g X  x  ³ 8 xydy 4 x 1  x 2  for 0  x  1
x
y
Marginal p.d . f of y.hY  y  ³ 8 xydx 4 y 3 for 0  y  1
0

(i) The conditional p.d . f of X given Y y is given by

f X ,y  x, y  8 xy 2x
gX /y y  x for 0  x  y (7.8)
hY  y  4 y3 y2

And  ii  the conditional p.d . f of Y given X x is given by

f X ,Y  x, y  8 xy 2y
hY / X x  y for x  y  1 (7.9)
gX  x 
4 x 1  x2  1  x2

7.4 Independence of r.v.s

Definition 7.6. Independence of r.v.s :  X , Y  be two dimensional r.vs they are

said to be independent if and only if, the events X  A and Y  B are independent
for any Borel sets A, B  

x For the two dimensional discrete r.v s  X , Y  are independent if and only if
the joint probability mass function is equal to the product of marginal p.m .fs
that is
PX ,Y  x, y  PX  x  Py  y  for all x,y (7.10)

68
 Chapter 7: Two-Dimensional R.V.S

x For the two dimensional continuous r.v s  X , Y  are independent if and only
if the joint probability density is equal to the product of marginal p.d .fs that
is

f X ,Y  x, y  g X  x  hY  y  for all x,y (7.11)

x When the r.v.s are independent their conditional p.m.f. s / p.d.fs are same as
marginal p.m.f. s / p.d.fs

Example 7.9. 1. Verify whether (X, Y) are independent r ˜ .v.s.

1 1 5 2
(i) The joint p.m. f . of  X , Y  is P  0,0  P 1,1 , P  0,1 , P 1,0 
9 9 9 9
(ii) The joint p.d . f of f X ,Y  x, y  8 xy for 0  x  y  1

Solution:(i)The joint p.m. f . of  X , Y  is

YЎ XЍ O 1 PY  y 
0 1 2 1
9 9 3
1 5 1 2
9 9 3
PX  x  2 1 1
3 3
2 1
PX  0  PY  0  u
3 3
1
PX ,Y  0,0 
9
PX  0  PY  0  z PX ,Y  0,0 
? X , Y are not independent.
(ii) The joint density is f X ,Y  x, y  8 xy for 0  x  y  1 Marginal p.d . f of
1
X .g X  x  ³ 8 xydy 4 x 1  x 2  for 0  x  1
x
y
Marginal p.d . f of Y. hY  y  ³ 8 xydx 4 y 3 for 0  y  1
0

f X ,Y  x, y  z g X  x  u hY  y 
X , Y are not independent.

69
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

7.5 Chapter End Exercises

1. Given following verify which are joint probability mass function of  X , Y  ,

if so, find the constant K.

(i) PX ,Y  x, y  Kx for x 1, 2,3; y 0,1, 2.

x
(ii) PX ,Y  x, y  K for x 1, 2; y 1, 4
y

2. Given following verify which are joint probability density function of

 X ,Y  . If so,

(i) find the constant K.

(ii) find marginal p.d.f of X,Y,

(iii) Verify whether they are independent.

(a) f X ,y  x, y  K for x, y  [0, 1],

(b) f X ,Y  x, y  OP .e  O x P y  for x t 0 y t 0;.O , P ! 0

3. Given following joint probability mass function of (X, y)

(i) Find the conditional p.m.f of Y given X 2

(ii) Find the conditional p.m.f of X given Y 2.

(iii) Also verify their independence.

(I)
YЎ X̺Ѝ 1 2 3 P[Y = y]
0 1 2 3 1
18 18 18 3
1 1 2 3 1
18 18 18 3
2 1 2 3 1
18 18 18 3
P[X = x] 1 1 1 1
6 3 2
(II) Two fair dice are tossed . X is maximum of the number on two
faces and Y is sum of the number on them.

70
 Chapter 7: Two-Dimensional R.V.S

4. The joint p.d.f of f X ,Y  x, y  2 for 0  x  y  1

r.v. X , Y 
(ii)Find the conditional p.d.f of Y given X x (iii) Find the conditional p..f
of X given Y y

5. Find the constant K , if the joint p.m.f of  X , Y  is given as

1
P  x, y  
K 3x14 y  for x, y 1, 2 , ....Also verify whether X, Y are
independent.

™™™

71
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

8
EXPECTATION, VARIANCE AND
THEIR PROPERTIES
Unit Structure
8.0 Objectives
8.1 Expectation of a r.v.
8.2 Variance of a r.v.
8.3 Characteristic function of a r.v.
8.4 Chapter End Exercises

8.0 Objectives

After going through this chapter you will learn

x The expected value of functions of r.v.s.
x Properties of expectation.
x Variance and its role in studying r.v
x Characteristic function and its properties.

8.1 Expectation of a r. v.

Definition 8.1. Expectation:

Case (I)Expected value of a discrete r .vX assuming values x1, x2 } xrL and with
p.m. f P > X xt @ is defined as
n
B X  ¦ x p  xi 
i 1 i
(8.1)

Provided the sum is convergent.

Case (II) Expected value of a continuous r .vX with p.d . f . f  x  is defined as

f
EX  ³ xf  x  dx (8.2)
f

Provided the integral is convergent.

Expected value of a r.v is its average, simply called as a mean of r.v

72
 Chapter 8: Expectation, Variance and their Properties

Example 8.1. Find expectation of X if

(i) A r.v X assuming values 0,1, 2 , ... n with probability proportional to n C x
(ii) X o B  n, p 
(iii) X o ( O 
(iv) X be no. of tosses of a coin up to and including the first toss showing heads.

Solution:(i)A r .vX assuming values 0,1, 2 , ... n with probability proportional to

n
Cx
P> X x @ K  Cx 
n
K¦x 0
Cx  K 2n 1

so, K 2 n
By definition of expectation,
n
EX  ¦ x 0
xp  x 
n
2  n ¦ x 0x  n
Cx 
As

 n
Cx  nx  n 1
Cx 1 
n n
2 n n¦ x 1
 n 1
Cx 1  2 n n2n1
2
(ii) X o (  O 
f f e O O x
EX  ¦ xp  x  ¦ x
x 0 x 0
x!
e  O O x1
f
O¦x 1
 x  1!
Using
ª O O2 º O
«1  11  21 }» e
¬ ¼

E  X  O e  O eO O.

(iii) Let X be no. of tosses of a coin up to and including the first toss showing
heads. Let ͂p̓ be the chance of showing head. 1 p q , is chance of showing
tail.
P> X x@ pq x1

73
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

for x 1, 2}
f f
EX  ¦ x 1
xp  x  ¦ x 1
xpq x 1
p 1
p ª¬1  2q  3q 2 }º¼ 2
1  q  p
Example 8.2. Find expectation of r.v X if
(i) A r .vX assuming values  0,f  with p.d . ff  x  O e O x . [This is Exponential
distribution with parameter O ]
(ii) A r .vX assuming values  a, b  , a  b real numbers, with p.d . ff  x 
constant. [This is Rectagular or Uniform distribution ]
(iii) 
X o N P ,V 2 
Solution:(i) Since X is absolutely continuous r.v with density f  x 
f
EX  ³ xf  x  dx (8.3)
f
f
EX  ³ O xe  O x dx (8.4)
0

e  O x f e  O x 1
xO |0f O ³ dx
O 0 O O
(ii) Since density is constant over  a, b  ,
b
³ Kdx 1
0

1
gives f  x  K
ba
x b x2 b2  a 2 ba
B X  ³ D b  a dx |ba (8.5)
2 b  a  2 b  a  2
(iii)
2
­  1 §¨ x  P ·¸
° e 2© V ¹
f  x ®  f d x d f; P  ,V ! 0 (8.6)
° 2SD
¯
2
1 § xP ·
 ¨ ¸
2© V ¹
f e
EX  ³ x dx (8.7)
f 2SV
xP
Put z then x V z  P and dx V dz
D
1
 z2
f e 2

³ V z  P 
f
2S
dz

74
 Chapter 8: Expectation, Variance and their Properties

1 1
 z2  z2
f e 2 f e 2
D³ z dz  P ³ dz (8.8)
f 2S f 2S

V u0 P P. (8.9)

Since the first integral is an even function, and

1
 z2
f e 2

³ dz 1
f 2S

Property 8.1. Properties of Expectation

x Expectation of a function of a r.v : If g  X  is a monotonic function of a r.v
then expected value of g  X  deoted by E  g  X   is defined as Case (I)
Discrete r.v
n
E ª¬ g  X  º¼ ¦ g x  px  i i (8.10)
i 1

Provided the sum is convergent.

Case (II) Continuous r.v
f
E ª¬ g  X  º¼ ³ g  x  f  x  dx (8.11)
f

Provided the integral is convergent.

x Expectation of a constant : E >C @ C Where C is constant.
x Effect of change of origin : E > X  A@ E  X   A Where A is constant.
x Effect of change of Scale : Pi > AX @ APi  X  Where A is constant.
x Linearity: Combining above two we may write E > AX  B @ AE  X   B
Where A, B are constants
x Monotone If X t Y , E > X @ t E >Y @

Example 8.3. A r .vX has mean  1 , find mean of following r.v.s , (i) X , (ii) 2X
X 3 2 X
(iii) , (iv)
2 2

Solution: (i) E   X   E  X  1 (ii) E  2 X  2 E  X  2

(iii)
§ X 3· EX 3
E¨ ¸ 1
© 2 ¹ 2
(iv)
§ 2 X · 2  B X 
E¨ ¸ 1.5
© 2 ¹ 2

75
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

8.2 Variance of a r.v.

The function of r.v g  X  X r , has special role in the study of a r.v

Definition 8.2. r th raw moment of a r.v : r th raw moment of a r.v X is defined
 
asE X r and is denoted by P r'
x We can check that for ' r 1 we get first raw moment and it is equal to mean
E  X  P1'
x r th raw moment of a r.v X is also called as moment about zero
r
x Moments can also be defined around arbitrary origin, that is E > X  A@
x In particular if arbitrary origin is mean the moments are called as central
W
moments So E ª¬ X  P1' º¼ is called as r th central moment. Second central
moment is important tool in the study of r.v . and it gives the idea about
spread or scatterness of the values of the variable.
Example 8.4. Show that first central moment E  X  P1'  0
Solution:

E X  P1'  E  X   P1' 0
, since E  X  P1'
2
Example 8.5. Show that E  X  a  is minimum when a E  X  , hence variance
is least mean square
Solution: Consider
2
dfij  X  a  § d 2·
E ¨  X  a  ¸ 2 E  X  a  0
da © da ¹
2
d 2E  X  a 2
when E  X  a , provided 2
! 0 . Thus E  X  E  X   V  X  mean
da
square deviation about mean is minimum
Definition 8.3. Variance of a r.v . : Variance of a r.v is its second central moment

x [Variance of a r.v ] If X is a r.v then variance of X denoted by V(X) is defined

as: Case (I) Discrete r.v :
n 2
V X  ¦ i
ª x  P1' º¼ p  xi 
1¬ i
(8.12)

Provided the sum is convergent.

Case (II) Continuous r.v . :

' 2
f
V X  ³  X  P  f  x  dx
f 1 (8.13)

76
 Chapter 8: Expectation, Variance and their Properties

Provided thc integral is convergent.

2
x V X   
E X 2  ª¬ E  X  º¼ , for computational purpose we use this formula.

x Variance of a constant : V >C @ 0 Where C is constant.

x Effect of change of origin : V > X  A@ V  X  Where A is constant.

x Effect of change of Scalc : V > AX @ A2V  X  Where A is constant.

x Combining above two we may write V > AX  B @ A2V  X  Where A, B are

constants.

x Positive square root of variance is called as standard deviation (s.d)of the r.v.

Example 8.6. A r .vX has variance 4, find variance and s.d of following r.v.s (i)
X 3 2 X
X , (ii) 2X , (iii) , (iv)
2 2
Solution:
(i) V   X  V  X  4, s.d 2.
(ii) V  2x  22 V  X  16,s.d 4
§ X3· V(X)
(iii) V ¨ ¸ 1
© 2 ¹ 22
§2X· V(X)
(iv) V ¨ ¸ 1,s.d 1
© 2 ¹ 22

Example 8.7. Find variance of following r.v.s.

(i) X o (  O 
(ii) X has Exponential distribution with parameter O
(iii) X has Uniform  a, b 

Solution:(i) X o (  O  so , as shown in above exercise E  X  O

Now consider,
E X2   E  X  X  1   E  X 

f
f
e O O x
E  X  X  1  ¦  x  x  1  p  x  ¦ x  x  1
x 0
x 1 x!
f e  O O x2
O2¦x 2 1
 x  2

77
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Using
ª O O2 º O
«1  11  21 }» e
¬ ¼
E  X  X  1  O 2e  O eO O2
So,
 
E X2 E  X  X  1   E  X  O 2  O.
2
V X   
E X 2  ª¬ E  X  º¼ O2  O  O2 O
Thus X o (  O  , then V  X  O
(ii) X has Exponential distribution with parameter O
f
E X2  ³ 0
O x 2e  O x dx (8.14)

e  O x f e  O x 2
x 2O |f0 2O ³ x dx
O 0 O O2
Since
f 1
EX  ³ O xe  O x dx
0 O
2
2 2 §1· 1
V X  Pi  X 2
  ª¬ E  X º¼ 2
¨ ¸
O2 O ©O¹
ba
(iii) Since X has Uniform over  a, b  , as seen above, E  X 
2
b x2 x3 b3  a 3 b 2  ab  a 2
EX 2 ³ a b  a dx |ba (8.15)
3 b  a  2 b  a  3
2 2
2 b 2  ab  a 2 § b  a · b  a 
V X E X   ª¬ H  X º¼
2

3
¨
© 2 ¹
¸
12

8.3 Characteristic function of a r.v.

A complex valued function of a r.v that is useful to study various properties of a r.v
is known as characteristic function(ch.f).

Definition 8.4. Characteristic function: X be a r.v a complex valued function

denoted by ) X  t  is defined as ) X  t   
E e ztX where t   and i 1

I) For discrete r.v : A discrete r.v X having p.m.f PX , then its ch.f is given by
f
) X t  ¦ P
x 0 X
 x  eitx (8.16)

t   and i 1

78
 Chapter 8: Expectation, Variance and their Properties

II) For continuous r.v A continuous r.v X having p.d.f f X  x  , then its ch. f is
given by
f
) X t  ³ f X
f  x  eitx dx (8.17)

t   and i 1

f
x We can also write ) X  t  ³ eitx dFX  x  which includes all r.v.
f

x ) X t   
E eitX E (cos  tX   iE  sin  tX  

x Re  ) X  t   E  cos  tX   , which is real part of ) X t  .And

I m) X  t  E  sin  tX   is the imaginary part of ) X  t 

Example 8.8. Find ch. f of following r.v.s

(i) X o B  n, p 

(ii) X has p.m. fp  x  pq x x 0,1,2} (Geometric distribution with parameter p)

(iii) X o N  P , V 2 

Solution:(i)

n
¦ C  p q x n  x  itx
) X t  x 0 x e (8.18)

t   and i 1

n x n
¦  C  q   pe   q  pe 
n x it it
) X t  x 0 x (8.19)

n
Using Binomial expansion. Ch. f of X o B  n, p  is ) X  t   q  pe it
(ii)

f
x itx p
) X t  ¦ pq e (8.20)
x 0 1  qeit

Using geometric series with common ratio qeit  1

p
So, Ch. f of Geometric distribution with parameter p is ) X  t  (iii)
1  qet

79
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

2
1 § xP ·
 ¨ ¸
2© V ¹
f e
) X t  ³ eitx dx (8.21)
f 2SV
xP
Put z then x V z  P and dx V dz
V
1
 z2
f it V z  P  e 2

³ e dz
f
2S
1
1   z itD 2
it P  D 2t 2 f e 2
e 2
³ dz (8.22)
f 2S
1
it P  V 2t 2
e 2
(8.23)
Since
1
  z itV 2
f e 2

³ dz 1
f 2S
1
it P  V 2t 2
X o N P ,V 2
 then its ch. f ) X t  e 2

Property 8.2. Properties of ch. f

x ) X  0  1.

x ) X  t  d 1 for all t  .

x ) X  t  ) X t  ) X  t  which is complex conjugate of ) X  t  .

x ) X  t  is uniformly continuous function of t.
x ) aX b  t  eibl ) X (at).
r
x ) X  t  generates moments of r.v X. The coefficient of  it  in the expansion
of ) X is r th moment of X. We can also get it from r th derivative of ) X .
1 dr
Pr' ) X  0 (8.24)
i r dt r
x 2 0 2
p(x) ̺4l ̺2l ̺4l

x If X and Y are independent r.vsch. f of X  Y is equal to product of their

ch. fs

x Product of any two ch. fs is also a ch, f . Thus any power of ) X  t  is also
a ch, f .

80
 Chapter 8: Expectation, Variance and their Properties

Example 8.9. Find ch. f of X which is Uniform r.v over  0,1 . Hence that of
̺X. Solution : f  x  1 for 0  x  1

1 eit  1
) X t  ³ eitx dx
0 it
eit  1
) X t 
it
Example 8.10. Using above result find ch. f of X̺ Y if X , Y are i.i.d Uniform
 0,1 .
eit  1
Solution: ) X  t  )Y  t 
it
eit  1
) Y  t 
it
X and Y are independent, X ,̺Y are also independent, by property of ch. f

eit  1 eit  1 eit  eit  2

) X Y  t  ) X  t  ) Y  t 
it it t2
Example 8.11. Is cos 2  t  a ch. f ?

Solution:
eil  eil e2il  e2il  2
cost
2
So, cos 2  t 
4
 
E eitX

Where X has p.m. f .

X -2 0 2
p(x) 1 1 1
4 2 4

cos 2  t  is a ch. f .

Theorem 8.1. X1 , X 2 } X n be the r.vs with d . fs FX1 , FX 2 } FX n respectively and

ch. fs ) X1 , ) X 9 }) X n respectively, then for the constants a1 , a2 }an such that

ai t 0 , and ¦a i 1, ¦ ) X x ai is a ch. f of ¦ aF i Xx

Following theorem characterizes the ch. f and its density or d.f. Uniquely.

81
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Theorem 8.2. Inversion theorem: X is absolutely continuous r.v such that

f
³ | ) X  t  | dt  f
f

then its p.d . f is given by

1 f
fX  x ³ e itx) X  t  dt (8.25)
2S f

Example 8.12. Show that e) t 1w a ch. f . if )  t  isch. f of some d . fF  x 

Solution:
2 3
) t  ) t 
1  ) t    
e
)  t  1 2! 3!
e
f
¦ a ) j t 
j 1 j

1 f
where a a j with ¦a 1 1 and ) j  t  ) j  t  , ch. f . of j̺fold convolution
j !e j 1

) t  1
of F ’ x  . hence e is a ch.f

e x
Example 8.13. f  x  for f  x  f then find its ch. f .
2
Solution:

x
f
i tx e
) X t  ³ e dx
f 2

0 ex f e x 1§ 1 1 · 1
³ eitx dx  ³ eitx dx ¨  ¸
f 2 0 2 2 © 1  it 1  it ¹ 1  t 2

t
) X t  e find its p.d . f
1
Solution: Since from the above example for f  x  e x ch f is , we write
1 t2
x
1 f
itx e
³ e dx
1 t2 f 2

replace t by ̺y in above equation gives

1 f e x iyx

1 y2 ³ fe 2 dx

82
 Chapter 8: Expectation, Variance and their Properties

1
replace x by ̺t in above equation and multiply by gives
S
1 1 1 f t
³ eiyt e dt
S 1  y2 2S f

By inversion theorem, e  t is a ch. f of r.v whose density is

1 1
f  y for f  x  f This is Cauchy density.
S 1  y2

8.4 Chapter End Exercises

1. Find expectation and variance of X if

(i) X o B  n, p 
(ii) X has Uniform over  1,1

(iii) p.d.f of X is

­1  x for  1  x d 0
°
f  x ®1  x for0  x d 1 (8.26)
°0 Otherwise
¯

(iv) X has p.m.f p  x  pq x x 0,1,2}

2. Find ch. f of  i  X has Poisson with parameter O . (ii) X1  X 2 , where

X 1 o N  P1 ,V 12  and X 2 o N  P 2 ,V 22  are independent r.v.s.

k
3. Are following ch. f ? (i) cost (ii) Re) X  t  (iii) P  ) X  t   ¦ ) t  X pk .
1
(iv)
1 t

™™™

83
 9
THEOREMS ON EXPECTATION AND
CONDITIONAL EXPECTATION
Unit Structure
9.0 Objectives
9.1 Expectation of a function of two dimensional r.v.s
9.2 Conditional Expectation
9.3 Chapter End Exercises

9.0 Objectives

After going through this chapter you will learn

x Expectation of a function of two dimensional r.v

x Theorems on expectation.

x Some inequalities based on expectations.

x Conditional expectation and its relation with simple expectation.

9.1 Expectation of a function of two dimensional r.v.s

Definition 9.1. Expectation of g  X , Y  : Let the function of two dimensional

r.vs  X , Y  be g  X , Y  its expected value denoted by E ª¬ g  X , Y  º¼ is defined as

Case (I)Discrete r v . with joint p.m. fPX ,y  xi , y j 

m n
Pi ª¬ g  X , Y  º¼ ¦ ¦ g x , y  p x , y 
j 1 i 1 i j i j (9.1)

Provided the sum is convergent. Case (II)Continuous r v . with joint

p.d . f . f X ,y  x, y 
f f
Pi ª¬ g  X , Y  º¼ ³ ³ g  x, y  f  x, y  dxdy (9.2)
f f

Provided the integral is convergent.

84
 Chapter 9: Theorems on Expectation and Conditional Expectation

Theorem 9.1. Addition Theorem on expectation:  X , Y  be two dimensional r.vs

then

E X Y  E  X   E Y  (9.3)

Proof: We assume that the r.v.s (X, Y) are continuous with joint p.d.f. f X ,Y  x, y 
and marginal p.d.f .sg X  x  , hY  y  .

f f
E  X  y ³ ³  x  y  f  x, y  dxdy X ,Y
f f

f f f f
³ ³ xf X ,y  x, y  dxdy  ³ ³ yf X ,Y  x, y  dxdy
f f f f

f
x ª f f  x, y  dy º dx  f y ª f f  x, y  dx º dy
³ f «¬ ³ f X ,Y »¼ ³ f «¬ ³ f X ,Y »¼
f f
³ xg X  x  dx  ³ yhY  y  dy*rom the definition g X  x  , hY  y 
f f

Pi  X   Pi Y  From the definition of E > X @ and E > Y @

Hence the proof.

Theorem 9.2. Multiplication Theorem on expectation:  X ,Y  be two

dimensional independent r.vs then

E  XY  E  X  E Y  (9.4)

Proof: We assume that the r.v.s  X , Y  are continuous with joint p.d.f. f X ,Y  x, y 
and marginal p.d.f .sg X  x  , hY  y  .

f f
E  XY  ³ ³ xyf X ,y  x, y  dxdy
f f

f f
³ ³ xyf X ,Y  x, y  dxdy
f f

f f
³ ³ xyg X  x  hY  y  dxdy By independence of X and Y
f f

f f
³ xg X  x  ³ yhy  y  dxdy
f f

E  X  Pi Y  *rom the definition of E > X @ and E > Y @

Hence the proof.

Above theorems can be generalized for n variables X1, X 2 , ... X n

85
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

E  X 1  X 2 } X n  E  X 1   E  X 2  } E  X n  (9.5)

*orn independent random variables X1, X 2 , ... X n

E  X1 X 2 } X n  E  X 1  E  X 2 } E  X n  (9.6)

Example 9.1. Show that E  aX  bY  c  aE  X   bE Y   c

Solution: Consider continuous r.v. X , Y  with joint p.d . f . f X ,Y  x, y  , and

marginal p.d . f .s

g X  x  , hY  y 
f f
Pi > aX  bY  c ] ³ ³  ax  by  c  f  x, y  dxdy (9.7)
f f

f f f f f f
³ ³ axf X ,Y  x, y  dxdy  ³ ³ by f X ,y  x, y  dxdy  c ³ ³ f  x, y  dxdy
f f f f f f X ,Y

f f f f
a ³ x ª ³ f X ,y  x, y  dy º dx  b ³ y ª ³ f X ,Y  x, y  dx º dy  cBy property of joint
« f
f ¬ ¼» « f
f ¬ ¼»
f f
p.d . f a ³ xg X  x  dx  b ³ yhY  y  dy  c By the definition 9 x  x  , hY  y 
f f

aE  X   bE Y   c By the definition of l X 1 and E[Y ]

If X t Y , then prove that E > X @ t E >Y @

Solution: If X t y , then > X  Y @ t 0 , and hence E > X  Y @ t 0

From the property of expectation and addition theorem,

E > X @  E >Y @ t 0 or E > X @ t E >Y @.

Example 9.2. Prove for any two r.v.sX , Y ,

2
E > Xy @ d E ª¬ X 2 º¼ E ª¬Y 2 º¼

[This is Cauchy Schwarz̓s inequality.1

2
Solution: consider a function h  a  E > X  aY @

ha E ª¬ x 2 º¼  a 2 E ª¬Y 2 º¼  2aE > XY @

dh  a 
2aE ª¬Y 2 º¼  2 E > XY @ 0
da
gives
pj > XY @
a
E ª¬Y 2 º¼

86
 Chapter 9: Theorems on Expectation and Conditional Expectation

And
d 2h  a 
2 E ª¬Y 2 º¼ t 0
da 2
E > XY @
Thus h  a  is minimum vjhen a
E ª¬Y 2 º¼
2
ª E > XY @ º E > XY @ E > XY @
ha t E « X  Y» fij ª¬ X 2 º¼  E ª¬Y 2 º¼ 2  2 fii > XY @
«¬ E ª¬Y 2 º¼ »¼ 2
E ª¬Y º¼ E ª¬Y 2 º¼
2
2 E > XY @
E ¬ª X ¼º  t0
pj ª¬Y 2 º¼
gives
2
E > XY @ d E ª¬ X 2 º¼ E ª¬Y 2 º¼
Example 9.3. Show that
§ 1 · 1
E¨ 2 ¸ t 2
© X ¹ E ª¬ X º¼
1
Solution: Using Cauchy Schwarz̓s inequality for y
X
ª 1 º
1 d E ª¬ X 2 º¼ E « 2 »
¬X ¼
divide inequality by E ª¬ X 2 º¼ ! 0 to get,

§ 1 · 1
B¨ 2 ¸ t 2
© X ¹ E ª¬ X º¼
Example 9.4. Show that for any two r.v.sX , Y ,
2
E > X  Y @ d E ª¬ X 2 º¼  E ª¬Y 2 º¼
Solution: Consider,
2
E>X Y @ E ª¬ X 2  2 Xy  Y 2 º¼

E ª¬ X 2 º¼  2 E > XY @  E ª¬Y 2 º¼

d E ª¬ X 2 º¼  2 E ª¬ X 2 º¼ E ª¬Y 2 º¼  E ª¬Y 2 º¼ By Cauchy Schwarz̓s inequality

2
ª E ª X 2 º  E ªY 2 º º
¬« ¬ ¼ ¬ ¼ ¼»
Taking square root,
2
pj > X  Y @ d B ª¬ X 2 º¼  pj ª¬Y 2 º¼

87
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

9.2 Conditional Expectation

Definition 9.2. Conditional Expectation of X :

Case (I) Let  X , Y  be the two dimensional discrete r.vwith / oint probability
mass function PX ,Y  xi , y j  i 1 ton, j 1tom the conditional expected value of X
given Y y j denoted by E X / y  y j  and defined as

n
EX /Y yj  ¦ x P X
i 1 i
xi / Y yj  (9.8)

Case (II) Let  X , Y  be the two dimensional continuous r .v with joint probability
density function f X ,Y  x, y  x, y   2 the conditional expected value of X given
Y y denoted by E  X / y y  and defined as

f
EX /Y y ³ xg  X / Y y  dx (9.9)
f

Similarly using conditional p.m.f s or p.d.f s of Y given X we can define

conditional expected value of Y.

Definition 9.3. Conditional Expectation of Y : Case (I) Let  X , Y  be the two

dimensional discrete r .v with joint probability mass function PX ,Y  xi , y j  i 1ton,
j 1tom the conditional expected value of Y given X xi denoted by
Pi Y / X xi  and defined as

m
E Y / X xi  ¦ j 1 j
y P Y yj / X xi  (9.10)

Case (II) Let  X , Y  be the two dimensional continuous r .v with joint probability
density function f X ,Y  x, y  x, y   2 the conditional expected value of X given
Y y denoted by E Y / X x  and defined as

f
E Y / X x ³ yh Y / X x  dy (9.11)
f

Theorem 9.3. For any two r.v.sX and Y

Ey E  X / Y y E X 
and Ex E Y / X x E Y 

88
 Chapter 9: Theorems on Expectation and Conditional Expectation

Proof: Consider
f
EX / y y ³ xg  X / y  dx
f

Multiply both sides of above equation by h  y  and itegrate with respect to y,to get
f f f
³ h  y  Pi  X / y  dy ³ ³
f f f
xh  y  g  X / y  dxdy

L.H.S becomes E y E  X / y  E  X  and since h  y  g  X / y  f X ,y  x, y  R.H.S

becomes
f
x ª f f  x, y  dy º dx
³ f «¬ ³ f X ,Y »¼
f
³ xg X  x  dx By definition of g X  x  (9.12)
f

Pi  X  (9.13)
Hence the proof. We can similarly prove Ex E Y / X x E Y 
Example 9.5. Find conditional means of X and Y of the following r.v. The joint
p.d . f of f X ,Y  x, y  8 xy for 0  x  y  1
Solution: From 13.8  the conditional p, d . f of X givenY yis
2x
g X /Y y  x for 0  x  y
y2
y 2x 2
E  X / y ³ xy 2
dx y
0 3
From (13.9) the conditional p.d . f of Ygiven X xis
2y
hY / X x  y for x  y  1
1  x2

E Y / x 
1
y
2y
dy

2 1  x  x2 
³ x 1  x2 3 1  x 
Example 9.6. Find E  X / Y 2  , and E Y / X 2  if the joint p.m. f . of
r.v. X , Y  is as given below.
YЎ XЍ 1 2 PY(y)
1 1 0 1
4 4
2 1 1 3
2 4 4
3 1 PX(x) 1
4 4
Solution: The conditional p.m. f of Y given X 2 is given by

89
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Y 1 2
PX/Y=2(y) 0 1̺3l

E Y / X 2  2 : yP Y y/ X 2 2
The conditional p.m. f of X given Y 2 is given by
x 1 2
PX/Y=2(x) 2 1
3 3

4
EX /Y 2 ¦ xP  X x /Y 2
3

9.3 Chapter End Exercises

1. If X and Y are independent then show that conditional means are same same
as simple means.

2
2. Show that E X  P d E  X  P 

3. Find conditional means of X and Y of the following r.v. The joint p.d.f of
f X ,Y  x, y  2 for 0  x  y  1

4. Given the joint p.m.f of X,Y as

x  3y
P> X x, Y y@ for x, y 1, 2 . Find conditional mean of Y givenX
24
1 and conditional mean of X given Y 1

5. For the above problem 3 and 4,find Ei  XY  ,E  X  Y  .

6. Two balls are drawn from an urn containing one yel of blue balls
drawn.low,two red and three blue balls. Let X be the number of red balls, and
Y be the number of blue balls drawn. *ind the joint p.m.f of X, Y ,hence
E X /Y 2  and E Y / X 2 .

™™™

90
 10
LAW OF LARGE NUMBERS AND
CENTRAL LIMIT THEOREM
Unit Structure
10.0 Objectives
10.1 Chebyshev’s inequality
10.2 Modes of Convergence
10.3 Laws of large numbers
10.4 Central Limit Theorem
10.5 Chapter End Exercises

10.0 Objectives

After going through this chapter you will learn

x Chebyshev̓s inequality and its applications

x Various modes of convergence and their interrelations.

x Weak law of large numbersand necessary condition for asequence to obey

this law.

x Strong law of large numbersand necessary condition for asequence to obey

this law.

x CLT : An important theorem for finding probabilities using Normal

approximation.

10.1 Chebyshev’s inequality

In this chapter we will study asymptotic behavior of the sequence of r.v.s.

Theorem 10.1. X is a non̺negative r.v with finite mean then for any C ! 0

EX 
P> X t C@ d (10.1)
C

91
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

proof :
f
E> X @ ³ xf X  x  dx (10.2)
f

Consider : ^ X t C` U{ X  C} So,
E> X @ ³ xf X  x  dx  ³ xf X  x  dx (10.3)
X tC X C

t³ xf X  x  dx
X tC

t³ Cf X  x 
X tC

CP > X t C @

Hence we get
B X 
P> X t C@ d (10.4)
C
Following inequality is directly followed from above theorem
Theorem 10.2. Chebyshev’s inequality X is a r.v with mean P and variance
V 2 then for any C ! 0
V2
P ª¬ X  P t C º¼ d 2
C
2
proof: X  P t C implies  X  P  t C 2

So
2
2 E ( X  P 
E ª¬ X  P t C º¼ d P ª X  P  t C 2 º d
¬ ¼ C2
2
Using above theorem for  X  P 

V X 
C2
Hence
V2
P ª¬ X  P t C º¼ d 2 (10.5)
C
x Inequality can also be written as
V2
P[ X  P  C ] t 1  (10.6)
C2
we get an lower bound on the probability that r.v deviates from its mean by C

92
 Chapter 10: Law of Large Numbers and Central Limit Theorem

x If C is replaced by k(J , where k ! 0 then inequality reduces to give an upper

bound
1
P ª¬ X  P t kCf º¼ d (10.7)
k2

x By complementation can also write a lower bound.

1
P[ X  P  kV ] t 1  (10.8)
k2
3
x If k 2 , lower bound is , which means that 75% of the times r.v assumes
4
values in  P  2D , P  2D 

x The bounds given by Chebyshev’s inequality are theoretical and not practical,
in the sense that the bounds are rarely attained by the r.v.

x Inequality is useful when the information regarding probability distribution

of r.v is not available but the mean and variance is known.
Example 10.1. A r.vX has mean 40 and variance 12. Find the bound for the
probability P > X d 32@  P > X t 48@.

Solution:
P > X d 32@  P > X t 48@ P ª¬ X  40 t 8 º¼

By Chebyshev’s inequality,
variance
P ª¬ X  40 t 8º¼ d 0.1875
82
Example 10.2. A unbiased coin is tossed 400 times,find the probability that number
of heads lie between (160, 240).

§ 1·
Solution: X has B ¨ 400, ¸ . X has Mean 200 and Variance 100.
© 2¹

So, P >160 d X d 240@ P ª¬ X  200 d 40 º¼ ,By Chebyshev ̓ s inequality, with

k 4 and V 10
1
P ª¬ X  200 d 40 º¼ t 1  . This gives the lower bound 0.9375
16

93
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

10.2 Modes of Convergence

Th modes of convergence are itroduced so as to define further laws of large

numbers.

Definition 10.1. Convergence in Probability:  :,A, P  be a probability space

^ X n` is a sequence of a r.v.s X n is said to converge in probability to a r.vX ,
from the same space, if for any H ! 0
lim P (| X n  X | H ) 0 (10.9)
n of

P
We say that X n  oX

Definition 10.2. Almost Sure Convergence  :,A, P  be a probability space^ X n `

is a sequence of a r.v.s X n is said to converge almost surely to a r.vX , from the
same space, if for any H ! 0
P (lim | X n  X |! H ) 0 (10.10)
nof

a.s
We say that X n  oX

Definition 10.3. Convergence in Distribution:  :,A, P  be a probability space

^ X n` is a sequence of a r.v.s with d . f .s ^ Fn `. X n is said to converge in
distribution to a r.vX , from the same space, if there exists a d . fF of X such that
Fn converges to F at all continuity points of F .

x Almost sure convergence implies convergence in probability.

x Convergence in probability implies convergence in distribution.

10.3 Laws of large numbers

Theorem 10.3. Weak law of Large Numbers(WLLN): X1 , X 2 } X n be the

independent r.vs with means P1, P2 }Pn respectively and finite variances

V 12 ,V 22 }V n2 respectively and Sn ¦X i for any H ! 0 , if

n
S ¦ Pi
lim P( n  i 1 ! H ) 0 (10.11)
nof n n

We say the Weak Law of Large Numbers (WLLN) holds for the sequence of
r.v.s ^ X i `

94
 Chapter 10: Law of Large Numbers and Central Limit Theorem

proof: Consider
ܵ௡ σ௡௜ ൌ ͳ ߤ௜ ܵ௡ ܵ௡
ܲሺቤ െ ቤ ൐ ߝሻ ൌ ܲሺฬ െ ‫ ܧ‬൬ ൰ฬ ൐ ߝሻ
݊ ݊ ݊ ݊
§S ·
Var ¨ n ¸
d © n ¹
H2
Because of independence of r.vs and by Chebyshev̓s Inequality .Further
2
¦V i
(10.12)
n 2H 2

Taking limit as n tends to f of both sides of inequality we get R.H. S limit zero
since variances are finite, finally
S n ¦ Pi
lim P (  !H) 0 (10.13)
nof n n

Theorem 10.4. Khintchine̓s Weak law of Large Numbers(WLLN) X1 , X 2 } X n

be the i.i.dr.vs with common mean P , then Sn ¦X i for any H ! 0 , if

Sn
lim P(  P !H) 0 (10.14)
nof n
We say the Weak Law of Large Numbers (WLLN) holds for the sequence of r.v .
s^ X i `
The law can be equivalently stated using complementation as
S
lim P( n  P  H ) 1 (10.15)
nof n

x In short when WLLN holds for thc sequence if Sn o P ¦ Pi .

x The limiting value of chance that average values of the r.v.s becomes close to
the mean is nearly 1,as n approaches to f
x Assumption of finite variance is required for non identically distributed
r.v.s.The condition for WLLN to hold for such sequence is that

V  SrL 
n2
tends to zero as n approaches to infinity. *or i.i.d.r.vs only existence of
finite mean is required.

x Above law is a weak law in the sense that there is another law which implies
this law

95
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Example 10.3. Examine whether WLLN holds for the following sequence of the
independent r.v.s.
1.
­ 1
°° 2k  1 withprob 2
Xk ®
° 2k  1 1
withprob
°̄ 2

Solution: E > X k @ 0 and V > X k @ Zk  1,V  Sn  n 2

V  Sn 
does not tend to zero as n approaches to f . WLLN does not hold for the
n2
sequence
2.
­ k 1
°°r2 withprob
2 2 k 1
Xk ®
°0 1
withprob 1 
°̄ 22 k

Solution: E > X k @ 0 and V > X k @ 1,V  S n  n

V  Sn 
tends to zero as n approaches to f , therefore WLLN holds for the
n2
sequence
Theorem 10.5. Strong Law of Large Numbers(SLLN): X1 , X 2 } X n be the
independent r.vs with means P1, P2 }P7l respectively and finite variances

V 12 ,V 22 }V n2 respectively and Sn ¦X i for any H ! 0 , if

n
S ¦ Pi
P[lim | n  i 1 |! H ] 0 (10.16)
n of n n
We say that Strong Law of Large Numbers (SLLN) holds for the sequence of
r.v.s ^ X i `

x In short when SLLN holds for the sequence if Sn o as ¦ Pi .

x The average values of the r.v.s becomes close to the mean as n approaches
to f with very high probability.That is almost surely.
x Assumption of finite variance is required for non identically distributed r.v.s.
The condition for SIದLN to hold for such sequence is that

96
 Chapter 10: Law of Large Numbers and Central Limit Theorem

f V  Xi 
¦ i 1
i2
f

as n approaches to infinity.This condition is known as Kolmogorov’s

Condition. For i.i.d.r.v.s only existence of finite mean is required.
x Above law is a Strong law in the sense that which implies Weak law
Example 10.4. Examine whether SLLN holds for the following sequence of the
independent r.v.s.
1.
­ 1
°°r k withprob
2 k
Xk ®
°0 1
withprob 1 
°̄ k
3
V  Xk 
Solution: E > X k @ 0 and V > X k @ k , ¦ 2
4f SLLN does not hold for the
k2
sequence
2.
­ k 1
°°2 withprob
2
Xk ®
° 2k 1
withprob
°̄ 2
V  Xk 
Solution: E > X k @ 0 and V > X k @ 22 k , ¦  f SLLN holds for the
k2
sequence

Weak law of large numbers gives an idea about whether the difference between
average value of r.v.s and their mean becomes small But following theorem gives
the limiting probability of this becomes less than small number H

10.4 Central Limit Theorem

Central Limit Theorem is basically used to find the approximate probabilities, using
Normal distribution.The theorem was initially proved for bernoulli r.v.s It has been
proved by many mathematicians and statisticians, by imposing different conditions.
Theorem 10.6. Central Limit Theorem by Lindberg̺Léévy(CLT):
X1, X 2 } X n be the i.i.dr.vs with common mean P and common variance V 2 , let

Sn ¦X i for any a   ! 0 ,

97
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

Sn
P 1
n 1 a  x2
lim P (  a) ³ e 2
dx (10.17)
nof D 271 a

We say that CLT holds for the sequence of r.v . s ^ X i `

x This theorem is useful to find the probabilities using normal approximation.

Normal distribution tables are available for d.f of N  0,1 for all a   ! 0

1
1 a  x2
M a ³ e 2
dx (10.18)
271 f

Sn
P
Sn V
x P( n  a) P( P  a) (10.19)
V n n
n

V n
Now if the choice of H is arbitrary take H a that is a H , as n
n V
approaches to f the above probability becomes,

Sn
lim P(  P  H ) M  f  1 (10.20)
nof n

n
x Sn o P ¦ i 1Pi

Sn
.Thus WLLN holds CLT gives probability bound for  P ,where as
n
WLLN gives only the limiting value.

x If X1 , X 2 } X n be the i.i. d bernoulli r.v s CLT becomes

Sn
p 1
n 1 o  x2
lim P (  a) ³ e 2
dx (10.21)
nof pq 2S f

Example 10.5. A fair coin is tossed 10, 000 independently. Find the probability
that number of heads (i)differs by less than 1% from 5000  ii  is greater than 5100

98
 Chapter 10: Law of Large Numbers and Central Limit Theorem

Solution: Sn be the number of heads in 10, 000 independent tosses of a fair coin.
Pi  Sn  5000 and V  Sn  2500
By CLT  i 
Sn  5000
lim P( Sn  5000  50) lim P(  1) (10.22)
nof nof 2500
1
1 1  x2
³ e 2
dx 2M 1  1 0.6826 (10.23)
2S 1

(ii)
Sn  5000 5100  5000
lim P( Sn ! 5100) lim P( ! ) (10.24)
nof nof 2500 2500
1
1 f  x2
³ e 2
dx 1  M  2  0.0228 (10.25)
27 r 2

Example 10.6. How many independent tosses of a fair die are required for the
1
probability that average number of sixes differ from by less than 6% to be at
6
least 0.95?

Sn 5th Sn
Solution. bee number average of sixes in n independent tosses of a
n 36n )n
§S ·
fair die. E ¨ n ¸ By CLT  i 
© n ¹

Sn 1 5
lim P(   .01 ) t 0.95 (10.26)
nof n 6 36n

5 1
1 .01  x2 § 5 ·
³ 36 n
e
2
dx 2M ¨¨ .01 ¸  1 t 0.95 (10.27)
2S .01
5
36 n © 36n ¸¹

2M 1.96   1 M 1.96   M  1.96  0.95 (10.28)

5 1962 x5
So,.01 t 1.96 or it gives n ! 5336 Toss the die at least 5336 times
36n 36 x.012
to get the result.

99
SET THEORY AND LOGIC & ELEMENTRY PROBABILITY THEORY

10.5 Chapter End Exercises

1. X is a r.v assuming values Ѹ l, 0,1 with probabilities 0.125, 0.75, 0.125

respectively.Find the bounds on P[ X ! 1]

2. Find K such that probability of getting hcad between 450toK is 0.9, in 1000
tosscs of a fair coin.

3. 3. f  x  e xx t 0*ind the bound on the probability P[ X  1 ! 2] , and

compare it with actual probability

4. Examine whether SLIದN holds for the following sequence of the

independent r.v.s.

­ k 1
°°r2 withprob
2 2 k 1
Xk ®
°0 1
withprob 1 
°̄ 2k

5. Examine whether WIದLN holds for the following sequence of the

independent r.v.s.

­ 1
°° k withprob
2
Xk ®
° k 1
withprob
°̄ 2

6. Suppose a large lot contains l% defectives. By using CLT, find approximate

probability of getting at least 20 defectives in a random sample of size 1000
units.

7. ^Xi` is sequence of independent r.vs such that E  X i  0 and

1 100
V  Xi  S100 ¦ X i approximately the P[ Sn ! 0.2]
3 i 1

™™™

100