Probability Bernoulli Random Variable

Probability Model 25. Notation: X ∼ Ber(p)

1. Ingredients of a probability model: 26. P(X = 1) = p and P(X = 0) = 1 − p
(a) Sample space Ω: set of all possible outcomes. 27. EX = p
(b) Events: subsets A ⊂ Ω to which a probability can be assigned. 28. Var(X) = p(1 − p)
(c) Probability: function P that assigns a number to each possible event Binomial Random Variable
A, satisfying 0 ≤ P(A) ≤ 1, P(Ω) = 1, and the Sum Rule: if 29. Notation: X ∼ Bin(n, p)
A1 , A2 , . . . cannot occur at the same time, then
30. P(X = x) = n

x
! x
p (1 − p)n−x , x = 0, 1, . . . , n.
31. EX = np
[ X
P Ai = P (Ai ) .
i i 32. Var(X) = np(1 − p).
2. Ac : A does not occur (complement of A). Uniform Random Variable
3. A ∩ B: Both A and B occur (also written AB). 33. Notation: X ∼ U[a, b]
4. A ∪ B: Either A or B (or both) occur. 34. Probability density function:
5. If A ∩ B = ∅. A and B are disjoint. 1
f (x) = , a≤x≤b (and f (x) = 0 otherwise).
6. If A ⊂ B then P(A) ≤ P(B). b−a
7. P(Ac ) = 1 − P(A). 35. EX = (a + b)/2
8. P(A ∪ B) = P(A) + P(B) − P(A ∩ B). 36. Var(X) = (b − a)2 /12
Probability by Counting Normal Random Variable
9. When all (finitely many) outcomes in Ω are equally likely, the 37. Notation: X ∼ N(µ, σ 2 )
probability of an event A ⊂ Ω is
38. Probability density function:
|A| Number of elements in A 
x−µ
2
P(A) = = . 1 −1
|Ω| Number of elements in Ω f (x) = √ e 2 σ , x∈R
σ 2π
Replacement 39. EX = µ
Order Yes No 40. Var(X) = σ 2
10. 41. Standardization: X−µ
∼ N(0, 1).
Yes nk n(n − 1) · · · (n − k + 1) σ

No — n

=
n(n−1)···(n−k+1) Multiple Random Variables
k k(k−1)···1
42. Joint cdf of X1 , . . . , Xn :
Conditional Probability F (x1 , . . . , xn ) = P(X1 ≤ x1 , . . . , Xn ≤ xn ).
11. Definition:
P(A ∩ B) 43. Joint pmf of discrete X1 , . . . , Xn :
P(A | B) = .
P(B) f (x1 , . . . , xn ) = P(X1 = x1 , . . . , Xn = xn ).
12. Events A and B are said to be independent if
44. Joint pdf of continuous X1 , . . . , Xn (summarised as X): positive
P(A ∩ B) = P(A) P(B). function f with total integral 1 such that
Z
13. Product Rule (notation AB = A ∩ B): P(X ∈ B) = f (x) dx for all sets B.
P(A1 A2 . . . An ) = P(A1 ) P(A2 | A1 ) · · · P(An | A1 . . . An−1 ). x∈B

14. Product Rule for independent events: 45. X1 , . . . , Xn with joint pmf or pdf f are independent if

P(A1 A2 . . . An ) = P(A1 ) P(A2 ) · · · P(An | A1 ). f (x1 , . . . , xn ) = fX1 (x1 ) · · · fXn (xn ) for all x1 , . . . , xn .
46. Covariance: Cov(X, Y ) = E[(X − µX )(Y − µY )], where µX = EX and
Random Variables µY = EY .
15. Cumulative distribution function (cdf) of a random variable X: Cov(X,Y )
47. Correlation coefficient: ρ(X, Y ) = σ σ 2 = Var(X) and
, where σX
F (x) = P(X ≤ x), x ∈ R. X Y
2 = Var(Y ).
σY
16. Probability mass function (pmf) of a discrete random variable:
48. Properties of Var and Cov:
f (x) = P(X = x) for all x. Var(X) = EX 2 − (EX)2 .
17. Probability density function (pdf) of a continous random variable X: Var(aX + b) = a2 Var(X).
positive function f with integral 1 such that: Cov(X, Y ) = EXY − EXEY .
Z b Cov(X, Y ) = Cov(Y, X).
P(a < X ≤ b) = F (b) − F (a) = f (u) du. Cov(aX + bY, Z) = a Cov(X, Z) + b Cov(Y, Z).
a Cov(X, X) = Var(X).
18. Relations between the cdf and pdf of a continuous random variable: Var(X + Y ) = Var(X) + Var(Y ) + 2 Cov(X, Y ).
Z x
d X and Y independent =⇒ Cov(X, Y ) = 0.
F (x) = f (t) dt, f (x) = F (x).
−∞ dx 49. For all random variables X1 , . . . , Xn and constants a, b1 , . . . , bn :
Expectation E(a + b1 X1 + · · · + bn Xn ) = a + b1 EX1 + · · · + bn EXn .
19. Expectation of a discrete random variable with pmf f : 50. For independent random variables,
X
EX = x f (x). Var(a + b1 X1 + · · · + bn Xn ) = b21 Var(X1 ) + · · · + b2n Var(Xn ).
x
51. For independent random variables,
20. Expectation of a continuous random variable with pdf f :
Z ∞ E[X1 X2 · · · Xn ] = EX1 EX2 · · · EXn .
EX = x f (x) dx. 52. For independent normal random variables X1 , X2 , . . . , Xn :
−∞
n n
!
21. Expectation of a function of a discrete random variable X with pmf f : X X
X Y = a + b1 X1 + · · · + bn Xn ∼ N a+ bi EXi , b2i Var(Xi ) .
Eg(X) = g(x) f (x) . i=1 i=1
x
53. Law of large numbers: If X1 , . . . , Xn are iid with expectation µ < ∞,
22. Properties of expectation: then for large n,
X1 + · · · + Xn
(a) E(a X + b) = a E(X) + b, X̄ = ≈ µ.
n
(b) E(g(X) + h(X)) = E(g(X)) + E(h(X)).
54. Central limit theorem (CLT): If X1 , . . . , Xn are iid with expectation
23. Variance: Var(X) = E(X − µ)2 , where µ = EX. µ < ∞ and variance σ 2 < ∞, then for large n:
24. Properties of variance: approx.
X1 + · · · + Xn ∼ N(nµ, nσ 2 ).
(a) Var(X) = EX 2 − (EX)2 . 55. By the CLT, X ∼ Bin(n, p) has approximately a N(np, np(1 − p))
(b) Var(a + bX) = b2 Var(X). distribution.
Statistics Pn
72. 2-sample F -test. Test statistic:
56. Sample mean: X̄ = i=1 Xi /n. 2
SX
57. Sample variance: S 2 =
Pn
− X̄)2 /(n − 1). F = .
i=1 (Xi SY2
Confidence Intervals 2 = σ 2 , we have F ∼ F(m − 1, n − 1).
Under H0 : σX Y
58. 1-sample iid data. Approximate confidence interval for µ: 73. 2-sample binomial test. Test statistic:
S
X̄ ± q √ , X/m − Y /n
n Z= q
,
1 1
P̂ (1 − P̂ ) m + n
where q is the 1 − α/2 quantile of N(0, 1). For normal data replace N(0, 1)
with tt−1 to get an exact CI. X+Y approx.
where P̂ := m+n
. Under H0 : pX = pY , we have Z ∼ N(0, 1).
59. 1-sample normal data. Exact confidence interval for σ 2 :

(n − 1)S 2 (n − 1)S 2
 Linear Models
, ,
q2 q1 74. 1-factor ANOVA model. Let Yik be the response for the k-th
replication at level i. Then,
where q1 and q2 are the α/2 and 1 − α/2 quantiles of the χ2n−1 distribution.
Yik = µi + εik , k = 1, . . . , ni , i = 1, . . . , d,
60. 2-sample normal data. Exact confidence interval for σ 2 /σY
2 :
iid
where {εik } ∼ N(0, σ 2 ).
!
2 2
1 SX 1 SX
, , MSF
q2 SY2 q1 SY2 75. 1-factor ANOVA. Test statistic F = MSE
. Under H0 : all {µi } are
equal, we have F ∼ F(d − 1, n − d).
where q1 and q2 are the α/2 and 1 − α/2 quantiles of F(m − 1, n − 1).
76. 2-factor ANOVA model (factor effects formulation). Let Yijk be the
61. Pooled sample variance: response for the k-th replication at level (i, j). Then,
2 + (n − 1)S 2
(m − 1)SX
Sp2 = Y
. Yijk = µ + αi + βj + γij + εijk ,
m+n−2
k = 1, . . . , nij , i = 1, . . . , d1 , j = 1, . . . , d2 ,
62. 2-sample normal data (assuming equal variances). Exact confidence
interval for µX − µY : where {εijk } ∼iid N(0, σ 2 ).
r 77. ANOVA table:
1 1
X̄ − Ȳ ± q Sp + , Source DF SS MS F P(F > f )
m n Factor 1
where q is the 1 − α/2 quantile of tm+n−2 . Factor 2
63. 2-sample normal data (unequal variances). Approximate confidence Residuals
interval for µX − µY : 78. Simple linear regression:
s
2
SX S2 Yi = β 0 + β 1 x i + ε i , i = 1, . . . , n,
X̄ − Ȳ ± q + Y ,
m n iid
where {εi } ∼ N(0, σ 2 ).
where q is the 1 − α/2 quantile of N(0, 1).
79. Multiple linear regression:
64. 1-sample binomial data. Approximate confidence interval for p:
s Yi = β0 + β1 xi1 + · · · + βd xid + εi , i = 1, . . . , n,
P̂ (1 − P̂ ) iid
P̂ ± q , where {εi } ∼ N(0, σ 2 ).
n
80. Normal linear model:
where P̂ = X/n and q is the 1 − α/2 quantile of N(0, 1).
Y = X β + ε,
65. 2-sample binomial data. Approximate confidence interval for pX − pY : where β is a vector of parameters and ε a vector of independent error
terms, each N(0, σ 2 ) distributed.
s
P̂X (1 − P̂X ) P̂Y (1 − P̂Y )
P̂X − P̂Y ± q + ,
m n Other Mathematical Formulas
n n
81. Binomial theorem: (a + b)n = ak bn−k .
P

where P̂X = X/m, PˆY = Y /n and q is the 1 − α/2 quantile of N(0, 1). k=0 k
1−an+1
Hypothesis Testing 82. Geometric sum: 1 + a + a2 + · · · + an = 1−a
(a 6= 1).
1
66. The 7 steps for a statistical test: model, hypotheses, test statistic, If |a| < 1 then 1 + a + a2 + ··· = 1−a
.
distribution of the test statistic under H0 , outcome of the test statistic, 83. Logarithms:
p-value, accept or reject H0 .
67. p-value: the probability that under H0 the (random) test statistic (a) log(x y) = log x + log y.
takes a value as extreme as or more extreme than the one observed. (b) elog x = x.
68. 1-sample t-test. Test statistic: 84. Exponential:
X̄ − µ0 x2 x3
T = √ . (a) ex = 1 + x + 2!
+ 3!
+ ···.
S/ n
x n
(b) ex = limn→∞ 1 +


Under H0 : µ = µ0 , we have T ∼ tn−1 . n
.

69. 1-sample binomial test. Test statistic X. Under H0 : p = p0 , we have (c) ex+y = ex ey .
X ∼ Bin(n, p0 ).
85. Differentiation:
70. 2-sample t-test (assuming equal variances). Test statistic:
(a) (f + g)0 = f 0 + g 0 ,
X̄ − Ȳ
T = q , (b) (f g)0 = f 0 g + f g 0 ,
1 1
Sp m +  0 0
n g0
(c) fg = f g−f g2
.
Under H0 : µX = µY , we have T ∼ tm+n−2 .
d n
71. 2-sample t-test (unequal variances). Test statistic: (d) dx
x = n xn−1 .
d x
X̄ − Ȳ (e) dx
e = ex .
T = q 2 2
.
SX SY d 1
+ (f) dx
log(x) = x
.
m n

Under H0 : µX = µY , we have T
approx.
∼ tdf distribution with 86. Chain rule: (f (g(x)))0 = f 0 (g(x)) g 0 (x).
87. Integration: ab f (x) dx = [F (x)]ba = F (b) − F (a), where F 0 = f .
 2 R
s2
X s2
Y
m
+ n 88. Integration by parts:
df = 2 2 . Rb b
Rb 0
a f (x) G(x) dx = [F (x) G(x)]a − a F (x) g(x) dx. (Here F = f and
 
1 s1
X 1 s1
Y
m−1 m
+ m−1 n G0 = f .)
 Standard Normal Distribution

This table gives the cummulative distribution function (cdf) Φ of a N(0, 1)-distributed
random variable Z: Z z
1 2
Φ(z) = P(Z ≤ z) = √ e−x /2 dx.
2π −∞
Example: Φ(1.65) = P(Z ≤ 1.65) = 0.9505.

z .00 .01 .02 .03 .04 .05 .06 .07 .08 .09
0.0 5000 5040 5080 5120 5160 5199 5239 5279 5319 5359
0.1 5398 5438 5478 5517 5557 5596 5636 5675 5714 5753
0.2 5793 5832 5871 5910 5948 5987 6026 6064 6103 6141
0.3 6179 6217 6255 6293 6331 6368 6406 6443 6480 6517
0.4 6554 6591 6628 6664 6700 6736 6772 6808 6844 6879
0.5 6915 6950 6985 7019 7054 7088 7123 7157 7190 7224
0.6 7257 7291 7324 7357 7389 7422 7454 7486 7517 7549
0.7 7580 7611 7642 7673 7704 7734 7764 7794 7823 7852
0.8 7881 7910 7939 7967 7995 8023 8051 8078 8106 8133
0.9 8159 8186 8212 8238 8264 8289 8315 8340 8365 8389
1.0 8413 8438 8461 8485 8508 8531 8554 8577 8599 8621
1.1 8643 8665 8686 8708 8729 8749 8770 8790 8810 8830
1.2 8849 8869 8888 8907 8925 8944 8962 8980 8997 9015
1.3 9032 9049 9066 9082 9099 9115 9131 9147 9162 9177
1.4 9192 9207 9222 9236 9251 9265 9279 9292 9306 9319
1.5 9332 9345 9357 9370 9382 9394 9406 9418 9429 9441
1.6 9452 9463 9474 9484 9495 9505 9515 9525 9535 9545
1.7 9554 9564 9573 9582 9591 9599 9608 9616 9625 9633
1.8 9641 9649 9656 9664 9671 9678 9686 9693 9699 9706
1.9 9713 9719 9726 9732 9738 9744 9750 9756 9761 9767
2.0 9772 9778 9783 9788 9793 9798 9803 9808 9812 9817
2.1 9821 9826 9830 9834 9838 9842 9846 9850 9854 9857
2.2 9861 9864 9868 9871 9875 9878 9881 9884 9887 9890
2.3 9893 9896 9898 9901 9904 9906 9909 9911 9913 9916
2.4 9918 9920 9922 9925 9927 9929 9931 9932 9934 9936
2.5 9938 9940 9941 9943 9945 9946 9948 9949 9951 9952
2.6 9953 9955 9956 9957 9959 9960 9961 9962 9963 9964
2.7 9965 9966 9967 9968 9969 9970 9971 9972 9973 9974
2.8 9974 9975 9976 9977 9977 9978 9979 9979 9980 9981
2.9 9981 9982 9982 9983 9984 9984 9985 9985 9986 9986
3.0 9987 9987 9987 9988 9988 9989 9989 9989 9990 9990
3.1 9990 9991 9991 9991 9992 9992 9992 9992 9993 9993
3.2 9993 9993 9994 9994 9994 9994 9994 9995 9995 9995
3.3 9995 9995 9995 9996 9996 9996 9996 9996 9996 9997
3.4 9997 9997 9997 9997 9997 9997 9997 9997 9997 9998
3.5 9998 9998 9998 9998 9998 9998 9998 9998 9998 9998
3.6 9998 9998 9999 9999 9999 9999 9999 9999 9999 9999