Axioms of Probability

C.M. Liu Feb. 25, 2007 www.csie.nctu.edu.tw/~cmliu/probability

Contents
Introduction Sample Space and Events Axioms of Probability Basic Theorems. Continuity of Probability Function Probabilities 0 and 1 Random Selection of Points from Intervals

Introduction
Ancient Egypians, 3500 B.C.
Use a four-sided die-shaped bone. Hounds and Jackals.

Studies of chances of events in 15th century

Luca Paccioli (1445 1514) Niccolo Tartaglia (1499 1557) Girolamo Cardano (1501 1576) Galileo Galilei (1564 1642)

Introduction
Real Progress from 1654
Blaise Pascal (1623 1662). Pierre de Fermat (1601 1665). Christian Huygens (1629 1695). On Calculations in Games of
Chance

Fermat

Pascal

Major Breakthrough
James Bernoulli (1654 1705). Abraham de Moivre (1667 1754).

Moivre

Bernoulli

Christiaan Huygens

Laplace

Introduction
18th century
Laplace, Poisson, Gauss expanded the growth of probability and its application.

19th Century
Advanced the works to put it on firm mathematical grounds. Pafnuty Chebyshev Andrei Markov. Aleksandre Lyapunov.

Kolmogorov

20th Century
David Hilbert (1862 1943) 23 problems whose solutions were crucial to the advancement of mathematics. Andrei Kolmogorov (1903-1987)
Combined the notion of sample space, introduced by Richard von
Mises, and measure theory and presented his axiom system for probability theory in 1933. 5

2. Sample Space and Events

Basic definitions:
Sample Space, S: The set of all possible
outcomes.

Sample Points: The outcomes. Event: A subset of the sample space.

2. Sample Space and Events

Experiment (eg. Tossing a die) Outcome(sample point) Sample space={all outcomes} Event: subset of sample space Ex1.1 tossing a coin once sample space S = {H, T} Ex1.2 flipping a coin and tossing a die if T or flipping a coin again if H S={T1,T2,T3,T4,T5,T6,HT,HH

2. Sample Space and Events

Ex1.3 measuring the lifetime of a light bulb S={x: x E={x: x 0} 100} is the event that the light bulb lasts at least 100 hours

Ex1.4 all families with 1, 2, or 3 children (genders specified) S={b,g,bg,gb,bb,gg,bbb,bgb,bbg,bgg, ggg,gbg,ggb,gbb}

2. Sample Space and Events-Subset Relationship

Subset
EFxExF

Equal
E=FEF and FE

Intersection
The intersection of E and F, written E F, is the set of elements that belong to both E and F. EF=EF={x: xE and xF}

Union
The union of E and F, written E F, is the set of elements that belong to either E or F. EF={x: xE or xF}.

2. Sample Space and Events-Subset Relationship

Complement
The complement of E, written Ec, is the set of all elements that are not in E. Ec={x: xE}. Difference of Two Events.
The set of elements belong to E but not in F. E-F={x: xE and xF}.

Mutually Exclusive. The joint occurrence of any two event is impossible. EF= 10

2. Sample Space and Events-Subset Relationship

Venn Diagrams
EF E F E Intuitive justification, create counterexamples, and shows invalidity. EF F

Ec E

(Ec G)F E G
11

2. Sample Space and Events-Subset Relationship

For any three events A, B, and C defined on a sample space S,
Commutativity: EF=FE, EF=FE. Associativity: E(FG)=(EF)G, E(FG)=(EF)G Distributative Laws: (EF)H =(EH)(FH) (EF)H=(EH)(FH), DeMorgans Laws: c n c n c c=EcFc, U Ei = I Ei (EF) U Ei = I Eic i =0 i =0 i =1 i =1 (EF)c=EcFc c c n n c I Ei = U Ei I Ei = U Eic
i =0
i =0

i =1

i =1

12

Proof of DeMorgans Laws

( E F )c E c F c Let x ( E F )
Then
c

( E F )c E c F c Let x E c F c
Then x E c and x F c So, x E and x F

x (E F )

So, x E and x F

Hence, x ( E c F c )

Therefore, x E F Thus, x ( E F ) c

13

3. Axioms of Probability
Axioms of Probability:
Let S be the sample space of a random phenomenon. Suppose that to each event A of S. a number denoted by P(A), is associated with A. If P satisfies the following axioms, then it is called a probability and the number P(A) is said to be the probability of A. P(A) 0 for any event A. P(S) = 1 where S is the sample space. If {Ai}, i=1,2,, is a sequence of mutually exclusive
events (that is, AiAj= for all ij), then

P ( U Ai ) = P ( Ai )
i =1 i =1
14

3. Axioms of Probability
Theorem 1.1 The probability of the empty set P()=0.

15

3. Axioms of Probability
Theorem 1.2 If {Ai}, i=1,2,n, are mutually exclusive (that n n is, AiAj= for all ij), then P(U A ) = P( A )
i =1 i

i =1

16

Examples
Flipping a fair or unbiased Coin
Events Sample space, S Probability on sample space and events. Probability on unbiased coin. Probability on biased coin.

17

3. Axioms of Probability
Theorem 1.3
Let S be the sample space of an experiment. If S has N points that are all equally likely occur, then for any event A of S, N ( A) P ( A) = N where N(A) is the number of points of A.

Example
Flipping a fair coin three times and A be the event at least two heads.
18

Proof on Theorem 1.3

19

Examples

20

Solutions

21

4. Basic Theorems
Theorem 1.4

For any event A, P ( Ac ) = 1 P ( A).

Theorem 1.5

If A B , then P ( B A) = P ( BAc ) = P ( B ) P ( A)
Corollary

If A B, then P( A) P( B)

22

Proof

23

4. Basic Theorems
Theorem 1.6 P ( A B ) = P ( A ) + P ( B ) P ( AB ). Inclusion-Exclusion Principle
n

P (U Ai ) =
i =1 n2 i =1 n 1 n

i =1

P ( Ai )

i =1 j = i +1

n 1

P ( Ai A j )

P ( A i A j A k ) ... + ( 1 ) n 1 P ( A1 A 2 A n ).

j = i + 1k = j + 1

Theorem 1.7
P ( A ) = P ( AB ) + P ( AB
24
c

4. Basic Theorems

25

4. Basic Theorems
Ex 1.15 In a community of 400 adults, 300 bike or swim or do both, 160 swim, and 120 swim and bike. What is the probability that an adult, selected at random from this community, bike? Sol: A: event that the person swims B: event that the person bikes P(AUB)=300/400, P(A)=160/400, P(AB)=120/400 P(B)=P(AUB)+P(AB)-P(A) = 300/400+120/400-160/400=260/400= 0.65
26

4. Basic Theorems
Ex 1.16 A number is chosen at random from the set of numbers {1, 2, 3, , 1000}. What is the probability that it is divisible by 3 or 5(I.e. either 3 or 5 or both)? Sol: A: event that the outcome is divisible by 3 B: event that the outcome is divisible by 5 P(AUB)=P(A)+P(B)-P(AB) =333/1000+200/1000-66/1000 =467/1000

27

4. Basic Theorems Inclusion-Exclusion Principle

28

4. Basic Theorems
Inclusion-Exclusion Principle

P ( A1 U A2 U ... U An ) = +

P( Ai ) P( Ai A j )

P ( Ai A j Ak ) ... + ( 1)n 1 P ( A1 A2 ... An )

29

4. Basic Theorems-- Examples

30

4. Basic Theorems-- Examples (c.1)

31

4. Basic Theorems
Theorem 1.7 P(A) = P(AB) + P(ABc)
Proof:

A = AS = A( B B c ) = AB AB c Since AB and AB c are mutually exclusive P( A) = P( AB AB c ) = P( AB ) + P( AB c )

32

Examples (c.2)

33

5. Continuity of Probability Functions

Continuous Functions
Let R denote the set of all real numbers. f : R R. is called continuous at a point c R if

lim f ( x ) = f (c )
xc

It is called continuous on R if it is continuous at all points.

cR Sequential Criterion
f(x) is continuous on R if and only if, for every convergent sequence { xn }=1 in R. n

lim f ( xn ) = f ( lim xn )
n n

34

5. Continuity of Probability Functions

Increasing Sequence of Events of Sample Space

E1 E 2 E n E n +1
Decreasing Sequence of Events of Sample Space

E1 E 2 E n E n +1
For increasing events

Applicable to the probability density function ?

means the event that

For increasing sequence of events, lim E n n at least one Ei occurs

lim E n = E n
n n =1

For decreasing sequence of events, lim E n n every Ei occurs

means the event that

lim E n = E n
n n =1

35

5. Continuity of Probability Functions

Theorem 1.8 Continuity of Probability Function
For any increasing or decreasing sequence of events,

lim P ( E n ) = P lim E n .
n n

36

Example

37

6. Probabilities 0 and 1
Not correct speculation
If E and F are events with probabilities 1 and 0, respectively, it is not correct to say that E is the sample space and F is the empty space. Ex. P(1/3) in (0, 1).

38

7. Random Selection of Points from Intervals

Randomly selected from an Interval
A point is said to be randomly selected from an interval (a, b) if any two subintervals of (a, b) that have the same length are equally likely to include the point. The probability associated with the event that the subinterval (, ) contains the point is defined to be (-)/(b-a).

39

dealine
Section 1.1-1.2 (page 10): 11, 12, 14, 16, 19. Section 1.4 (page 23): 14, 22, 28, 31 Section 1.7 (page 34): 3, 10. Review (page 36): 10, 12, 14.

40