PROBABILITY & STATISTICS

Sample Spaces
Chapter 2: Probability
Probability
LEARNING OBJECTIVES
Addition Rules

Conditional Pro.
1. Sample Spaces and Events
2. Interpretations of Probability
Multi. Rules
3. Addition Rules
Total Pro. Rule
4. Conditional Probability
Independence
5. Multiplication and Total Probability Rules
Bayes’Theorem
6. Independence
Summary
7. Bayes’ Theorem
19/05/2021 Department of Mathematics 1/38
 Sample spaces and events
Sample Spaces
Sample Spaces

Probability
Definition Random experiment
Addition Rules
• An experiment that can result in different outcomes,
Conditional Pro. even though it is repeated in the same manner every
time, is called a random experiment.
Multi. Rules
• The set of all possible outcomes of a random
Total Pro. Rule experiment is called the sample space of the
experiment. The sample space is denoted as S.
Independence
• An event is a subset of the sample space of a random
Bayes’Theorem
experiment.
Summary

19/05/2021 Department of Mathematics 2/38

 Sample spaces and events
Sample spaces
Sample Spaces

Probability

Addition Rules
Example
Conditional Pro.
Random experiment: Roll a die
Multi. Rules
Sample space: S ={1, 2, 3, 4, 5, 6}
Total Pro. Rule Event: E1 = {Die is even}={2, 4, 6}
Independence E2 = {Die is odd}={1, 3, 5}

Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 3/38

 Sample spaces and events
Sample spaces
Sample Spaces
Tree Diagrams
Probability
Sample spaces can also be described graphically with
Addition Rules tree diagrams.
Conditional Pro.
– When a sample space can be constructed in
several steps or stages, we can represent each of
Multi. Rules the n1 ways of completing the first step as a
branch of a tree.
Total Pro. Rule
– Each of the ways of completing the second step
Independence can be represented as n2 branches starting from
the ends of the original branches, and so forth.
Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 4/38

 Sample spaces and events
Sample spaces
Sample Spaces

Example
Probability
A probability experiment consists of tossing a
Addition Rules
coin and then rolling a six-sided die. Describe the
Conditional Pro. sample space.
Multi. Rules Tree diagram:
Total Pro. Rule

Independence

Bayes’Theorem H1 H2 H3 H4 H5 H6 T1 T2 T3 T4 T5 T6

Summary The sample space has 12 outcomes:

S = {H1, H2, H3, H4, H5, H6, T1, T2, T3, T4, T5, T6}
19/05/2021 Department of Mathematics 5/38
 Sample spaces and events
Sample spaces
Sample Spaces
Example
Probability Each message in a digital communication system is classified
as to whether it is received within the time specified by the
Addition Rules
system design. If three messages are classified, use a tree
Conditional Pro.
diagram to represent the sample space of possible outcomes.

Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 6/38

 Sample spaces and events
Sample Spaces
Sample Spaces
Basic Set Operations
Probability

The union of two events is the event that consists of all

Addition Rules
outcomes that are contained in either of the two events.
Conditional Pro. We denote the union as E1∪E2.
Multi. Rules The intersection of two events is the event that consists
of all outcomes that are contained in both of the two
Total Pro. Rule
events. We denote the intersection as E1∩E2.
Independence The complement of an event in a sample space is the set
Bayes’Theorem
of outcomes in the sample space that are not in the event.
We denote the component of the event E as E’.
Summary

19/05/2021 Department of Mathematics 7/38

 Sample spaces and events
Sample Spaces
Sample Spaces
Venn Diagrams
Probability mutually exclusive

Addition Rules

Conditional Pro.

Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 8/38

 Sample spaces and events
Sample Spaces
Sample Spaces
Important properties:
Probability

Addition Rules A ∪ (B ∪ C) = (A ∪ B) ∪ C

Conditional Pro. A ∩ (B ∩ C) = (A ∩ B) ∩ C

Multi. Rules
A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)
(A ∪ B)’ = A’ ∩ B’
Total Pro. Rule
(A ∩ B)’ = A’ ∪ B’
Independence
A = (A ∩ B) ∪ (A ∩ B’)
Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 9/38

 Interpretations of Probability
Sample Spaces
Introduction
Probability
Probability
There are three approaches to assessing the probability of an
Addition Rules uncertain event:
Conditional Pro. 1. a priori classical probability: the probability of an event
is based on prior knowledge of the process involved.
Multi. Rules

Total Pro. Rule

2. empirical classical probability: the probability of an event
is based on observed data.
Independence

3. subjective probability: the probability of an event is

Bayes’Theorem
determined by an individual, based on that person’s past
Summary experience, personal opinion, and/or analysis of a
particular situation.
19/05/2021 Department of Mathematics 10/38
 Interpretations of Probability
Sample Spaces

Probability
Equally Likely Outcomes
Probability
Whenever a sample space consists of N possible outcomes that are
Addition Rules equally likely, the probability of each outcome is 1/N.

Conditional Pro.
1. a priori classical probability
Multi. Rules
X number of ways the event can occur
Total Pro. Rule Probability of Occurrence = =
T total number of possible outcomes

Independence 2. empirical classical probability

Bayes’Theorem
number of favorable outcomes observed
Probability of Occurrence =
total number of outcomes observed
Summary

19/05/2021 Department of Mathematics 11/38

 Interpretations of Probability
Sample Spaces

Example priori classical probability

Probability
Probability
Find the probability of selecting a face card (Jack, Queen,
Addition Rules or King) from a standard deck of 52 cards.
Conditional Pro.

Multi. Rules X number of face cards

Probability of Face Card = =
Total Pro. Rule T total number of cards

Independence X 12 face cards 3

= =
T 52 total cards 13
Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 12/38

 Interpretations of Probability
Sample Spaces

Example empirical classical probability

Probability
Probability
Find the probability of selecting a male taking statistics
Addition Rules from the population described in the following table:
Conditional Pro.
Taking Stats Not Taking Stats Total
Multi. Rules Male 84 145 229
Female 76 134 210
Total Pro. Rule
Total 160 279 439
Independence

Bayes’Theorem number of males taking stats

Probability of Male Taking Stats =
total number of people
Summary
84
= = 0.191
439
19/05/2021 Department of Mathematics 13/38
 Interpretations of Probability
Sample Spaces
Axioms of Probability
Probability
Probability
Probability is a number that is assigned to each member of a
Addition Rules collection of events from a random experiment that satisfies the
following properties:
Conditional Pro.
If S is the sample space and E is any event in a random
Multi. Rules experiment,

Total Pro. Rule

(1) P(S) = 1
(2) 0 ≤ P(E) ≤ 1
Independence
(3) For two events E1 and E2 with E1∩E2 = Ø
Bayes’Theorem
P(E1 ∪ E2) = P(E1) + P(E2)
Summary

19/05/2021 Department of Mathematics 14/38

 Addition Rules
Sample Spaces
The special addition rule
Probability

Addition Rules
Rules
1. If A and B are mutually exclusive events,
Addition
P(A ∪ B) = P(A) + P(B)
Conditional Pro.
2. A collection of events, E1, E2, …, Ek is said to be
Multi. Rules
mutually exclusive if for all pairs,
Total Pro. Rule Ei ∩ Ej = Ø
Independence For a collection of mutually exclusive events,
Bayes’Theorem
P(E1 ∪ E2 ∪… ∪ Ek) = P(E1) + P(E2) +…+ P(Ek)
Summary

19/05/2021 Department of Mathematics 15/38

 Addition Rules
Sample Spaces
The general addition rule
Probability

Addition Rules
Rules
1. Two events: A and B are any events,
Addition
P(A ∪ B) = P(A) + P(B) – P(A ∩ B)
Conditional Pro.

Multi. Rules
2. Three events:
Total Pro. Rule

Independence P(A ∪ B ∪ C) = P(A) + P(B) + P(C) - P(A ∩ B)

Bayes’Theorem - P(B ∩ C) - P(C ∩ A) + P(A ∩ B ∩ C)

Summary

19/05/2021 Department of Mathematics 16/38

 Addition Rules
Sample Spaces
Example
Probability
Find the probability of selecting a male or a statistics
Addition Rules
Addition Rules
student from the population described in the following
table:
Conditional Pro.
Taking Not Taking Total
Multi. Rules Stats Stats
Male 84 145 229
Total Pro. Rule
Female 76 134 210
Independence Total 160 279 439

Bayes’Theorem
P(Male or Stat) = P(M) + P(S) – P(M or S)
Summary
= 229/439 + 160/439 – 84/439 = 305/439

19/05/2021 Department of Mathematics 17/38

 Conditional Probability
Sample Spaces
Conditional Probability
Probability
To introduce conditional probability, consider an
Addition Rules
example involving manufactured parts.
Conditional Pro.
Conditional Pro.
Let D denote the event that a part is defective and let F
Multi. Rules
denote the event that a part has a surface flaw.
Total Pro. Rule

Then, we denote the probability of D given, or

Independence
assuming, that a part has a surface flaw as P(D|F). This
Bayes’Theorem notation is read as the conditional probability of D
Summary
given F, and it is interpreted as the probability that a
part is defective, given that the part has a surface flaw.
19/05/2021 Department of Mathematics 18/38
 Conditional Probability
Sample Spaces
Example 2-22(page 42)
Probability

Addition Rules

Conditional Pro.
Conditional Pro.

Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 19/38

 Conditional Probability
Sample Spaces

Probability

Addition Rules

Conditional Pro.
Conditional Pro.

Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 20/38

 Conditional Probability
Sample Spaces
Definition Conditional Probability
Probability
The conditional probability of an event B given an event
Addition Rules A, denoted as P(B|A), is
Conditional Pro.
Conditional Pro. P(B|A) = P(B ∩ A)/P(A)
Multi. Rules for P(A) > 0.

Total Pro. Rule Special case: all outcomes are equally likely
Independence

Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 21/38

 Conditional Probability
Sample Spaces
Example 2-23 (page 43)
Probability

Addition Rules

Conditional Pro.
Conditional Pro. Note that in this example all four of the following
probabilities are different:
Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem
Here, P(D) and P(D|F) are probabilities of the same
event, but they are computed under two different states
Summary of knowledge. Similarly, P(F) and P(F|D) are computed
under two different states of knowledge.
19/05/2021 Department of Mathematics 22/38
 Conditional Probability
Sample Spaces
Example
Probability
Of the cars on a used car lot, 70% have air conditioning
Addition Rules
(AC) and 40% have a CD player (CD). 20% of the cars
have both.
Conditional Pro.
Conditional Pro.
What is the probability that a car has a CD player, given
Multi. Rules that it has AC ?
Total Pro. Rule
CD No CD Total
P(CD and AC)
Independence P(CD | AC) =
AC 0.2 0.5 0.7 P(AC)
Bayes’Theorem
No AC 0.2 0.1 0.3 0.2
= = .2857
0.7
Summary Total 0.4 0.6 1.0

19/05/2021 Department of Mathematics 23/38

 Multiplication rules
Sample Spaces
Multiplication Rule
Probability
P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A)
Addition Rules

Conditional Pro.
Exam (NCOVID19). Only 1 in 100 adults is
afflicted with Ncovid19 for which a diagnostic test
Multi. Rules
Rules
Multi. has been developed. The test is such that when
Total Pro. Rule an individual actually has the disease, a
positive result will occur 99% of the time.
Independence
If a randomly selected individual is tested. Find the
Bayes’Theorem
probability of that person has disease and positive test.
Summary
A: has disease; B: positive test. Compute P(A ∩ B)
19/05/2021 Department of Mathematics 24/38
 Total Probability Rule
Sample Spaces
NCOVID 19
Probability

Addition Rules Only 1 in 100 adults is afflicted with Ncovid19

for which a diagnostic test has been
Conditional Pro.
developed. The test is such that when an
Multi. Rules individual actually has the disease, a
Total Pro.
Total Pro. Rule
Rule
positive result will occur 99% of the time,
whereas an individual without the disease
Independence
will show a positive test result only 2% of
Bayes’Theorem the time.
Summary If a randomly selected individual is tested. Find
the probability of that person has positive test.
19/05/2021 Department of Mathematics 25/38
 Multiplication rules
Sample Spaces
Multiplication Rule
Probability
P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A)
Addition Rules

Conditional Pro.
Example
The probability that an automobile battery subject to high
Multi. Rules
Multi. Rules engine compartment temperature suffers low charging current
is 0.7. The probability that a battery is subject to high engine
Total Pro. Rule
compartment temperature is 0.05.
Independence The probability that a battery is subject to low charging current
and high engine compartment temperature is
Bayes’Theorem

Summary
C={a battery suffers T ={a battery is subject to high
low charging current} engine compartment temperature}
19/05/2021 Department of Mathematics 26/38
 Total Probability Rule
Sample Spaces
Partition of an event
Probability

Addition Rules Figure 2-15

Partitioning an event
Conditional Pro. into two mutually
exclusive subsets.
Multi. Rules

Total Pro.
Total Pro. Rule
Rule

Independence Figure 2-16

Partitioning an
Bayes’Theorem event into several
mutually
Summary exclusive subsets.

19/05/2021 Department of Mathematics 27/38

 Total Probability Rule
Sample Spaces
Total Probability Rule: two events
Probability

Addition Rules P(B) = P(B ∩ A) + P(B ∩ A’) = P(B|A)P(A) + P(B|A’)P(A’)

Conditional Pro.
Total Probability Rule: multiple events
Multi. Rules

Total Pro.
Pro. Rule
Rule
E1 ∪ E2 ∪ … Ek =S
Total

Independence

Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 28/38

 Total Probability Rule
Sample Spaces
NCOVID 19
Probability

Addition Rules Only 1 in 100 adults is afflicted with Ncovid19

for which a diagnostic test has been
Conditional Pro.
developed. The test is such that when an
Multi. Rules individual actually has the disease, a
Total Pro.
Total Pro. Rule
Rule
positive result will occur 99% of the time,
whereas an individual without the disease
Independence
will show a positive test result only 2% of
Bayes’Theorem the time.
Summary If a randomly selected individual is tested. Find
the probability of that person has positive test.
19/05/2021 Department of Mathematics 29/38
 Bayes’ Theorem
Sample Spaces
Bayes’ Theorem
Probability

Addition Rules If E1, E2, …, Ek are k mutually exclusive and exhaustive

events and B is any event,
Conditional Pro.
P(B | E1 )P( E1 )
P( E1 | B) =
Multi. Rules P(B | E1 )P( E1 ) + P(B | E2 )P(E2 ) + ...+ P(B | Ek )P(Ek )
Total Pro. Rule for P(B) > 0

Independence
In special case:
Bayes’Theorem
Bayes’Theorem P ( B | A) P ( A)
P( A | B) = for P ( B ) > 0
Summary P( B)

19/05/2021 Department of Mathematics 30/38

 Bayes’ Theorem
Sample Spaces
NCOVID 19 again
Probability
Only 1 in 100 adults is afflicted with Ncovid19 for
Addition Rules
which a diagnostic test has been developed. The test
Conditional Pro. is such that when an individual actually has the
disease, a positive result will occur 99% of the
Multi. Rules time, whereas an individual without the disease
Total Pro.
Pro. Rule
Rule
will show a positive test result only 2% of the
Total
time.
Independence
If a randomly selected individual is tested and has positive
Bayes’Theorem test. What is the probability that the individual has the
disease?
Summary
A: has disease; B: positive test. Compute P(A|B)
19/05/2021 Department of Mathematics 31/38
 Bayes’ Theorem
Sample Spaces
Example
Probability
In a state where cars have to be tested for the emission of
pollutants, 25% of all cars emit excessive amount of pollutants.
Addition Rules
When tested, 99% of all cars that emit excessive amount of
Conditional Pro. pollutants will fail, but 17% of all cars that do not emit
excessive amount of pollutants will also fail. What is the
Multi. Rules probability that a car that fails the test actually emits excessive
amounts of pollutants?
Total Pro. Rule

Independence Let A denote the event that a car fails the test and B the event
that it emits excessive amounts of pollutants:
Bayes’Theorem
Bayes’Theorem
P(B) = 0.25, P(A|B) = 0.99 and P(A|B’) = 0.17.
Summary
We have to find P(B|A)?

19/05/2021 Department of Mathematics 32/38

 Bayes’ Theorem
Sample Spaces

P( A | B) P( B)
Probability P( B | A) =
P( A)
Addition Rules
where
Conditional Pro.
P(A) = P(AB) + P(AB’) = P(A|B)P(B) + P(A|B’)P(B’)
Multi. Rules
= 0.375
Total Pro. Rule
Substitution into the formula for P(B|A) yields
Independence
P(B|A) = 0.66
Bayes’Theorem
Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 33/38

 Bayes’ Theorem
Sample Spaces
Example
Probability
Two events A and B are such that P[A∩B] = 0.15, P[A∪B]
Addition Rules = 0.65, and P[A|B] = 0.5. Find P[B|A].

Conditional Pro.

Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem
Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 34/38

 Independence
Sample Spaces

Definition
Probability
Two events is called independent if any one of the
Addition Rules following equivalent statements is true:
Conditional Pro. (1) P(A|B) = P(A)
Multi. Rules (2) P(A ∩ B) = P(A)P(B)
Total Pro. Rule (3) P(B|A) = P(B)

Independence
Independence
Proposition: If A and B are independent events, then
Bayes’Theorem so are events A and B’, events A’ and B, and events A’
Summary
and B’.

19/05/2021 Department of Mathematics 35/38

 Independence
Sample Spaces
Example 2-31 (page 52)
Probability
A day’s production of 850 manufactured parts contains 50
Addition Rules
parts that do not meet customer requirements. Two parts are
selected at random, without replacement, from the batch. Let
Conditional Pro. A ={the first part is defective}, and let B ={the second part is
defective}.
Multi. Rules
We suspect that these two events are not independent because
Total Pro. Rule knowledge that the first part is defective suggests that it is
less likely that the second part selected is defective.
Independence
Independence
P(B|A) = 49/849 the two events
Bayes’Theorem are not
independent, as
Summary
we suspected.

19/05/2021 Department of Mathematics 36/38

 Independence
Sample Spaces

Definition
Probability
The events E1, E2, …, En are independent if and only if
Addition Rules for any subset of these events Ei1 , Ei2 , ... , Eik
Conditional Pro.
P(Ei1 ∩ Ei2 ∩...∩ Eik ) = P(Ei1 )P(Ei2 ) ... P(Eik )
Multi. Rules

Total Pro. Rule

Exercise
Independence
Independence
Two coins are tossed. Let A denote the event “at most
Bayes’Theorem one head on the two tosses,” and let B denote the event
“one head and one tail in both tosses.” Are A and B
Summary
independent events?

19/05/2021 Department of Mathematics 37/38

 Independence
Sample Spaces
Example 2-34 (page 53)
Probability
The following circuit operates only if there is a path of
Addition Rules functional devices from left to right. The probability that each
device functions is shown on the graph. Assume that devices
Conditional Pro. fail independently. What is the probability that the circuit
operates?
P(T ∪B) = 1- P[(T ∪B)’]
Multi. Rules

Total Pro. Rule

= 1-P(T’ ∩B’)
Independence
P(T’ ∩B’) = P(T’)P(B’)
Independence

Bayes’Theorem
= (1-0.95)2 =0.052
Summary
P(T ∪B) = 1 - 0.052 =0.9975
19/05/2021 Department of Mathematics 38/38
 Random Variables
Sample Spaces

Probability
Definition
Addition Rules A random variable is a function that assigns a real
number to each outcome in the sample space of a
Conditional Pro.
random experiment.
Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem
Bayes’Theorem A random variable is denoted by an uppercase letter such
as X. After an experiment is conducted, the measured
Summary
value of the random variable is denoted by a lowercase
letter such as x= 70 milliamperes.
19/05/2021 Department of Mathematics 39/38
 Random Variables
Sample Spaces

Probability

Addition Rules
Definition
A discrete random variable is a random variable with a
Conditional Pro. finite (or countably infinite) range.
Multi. Rules A continuous random variable is a random variable with
an interval (either finite or infinite) of real numbers for
Total Pro. Rule
its range.
Independence

Bayes’Theorem
Bayes’Theorem

Summary

19/05/2021 Department of Mathematics 40/38

 Summary
Sample Spaces
We have studied:
Probability
1. Sample Spaces and Events
Addition Rules
2. Interpretations of Probability
Conditional Pro. 3. Addition Rules

Multi. Rules 4. Conditional Probability

5. Multiplication and Total Probability Rules
Total Pro. Rule
6. Bayes’ Theorem
Independence
7. Independence
Bayes’Theorem
Homework: Read slides of the next lecture.
Summary
Summary

19/05/2021 Department of Mathematics 41/38