Download Probability Distributions and Functions: A Comprehensive Guide for Students and more Summaries Statistics in PDF only on Docsity!
Probability Distributions and
Functions
Probability Concepts and Distributions
1.2 Set Notation
Sample Space (S) : The set of all possible sample points (i.e., the collection of all the sample events). Complement (A^c) : The set of all elements not in the event A; P(A^c) = 1 - P(A). Union (A ∪ B) : The set of all elements either in set A, B, or both. Intersection (A ∩ B) : The set of all elements occurring in both sets A and B. Empty Set (∅) : The set where neither sets A nor B have elements in common. If A ∩ B = ∅, then sets A and B are mutually exclusive or disjoint.
1.3 Additive Law
The probability that at least one of the two events A and B occurs is: P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
If events A and B are mutually exclusive, then: P(A ∪ B) = P(A) + P(B)
1.4 Conditional Probability
The probability that both events A and B occur is: P(A ∩ B) = P(A)P(B|A) = P(B)P(A|B)
Two events are said to be independent if any one of the following holds: - P(A ∩ B) = P(A)P(B) - P(A|B) = P(A) - P(B|A) = P(B)
1.6 Law of Total Probability and Bayes' Rule
Let B1, ..., Bk be a partition of the sample space S, where every Bi has P(Bi)
- Then, for any event A: P(A) = Σ P(A|Bi)P(Bi)
Bayes' Rule (Conditioning on a Partition): P(Bj|A) = (P(A|Bj)P(Bj)) / Σ P(A| Bi)P(Bi)
1.7 Random Sampling
Random Variable : A numeric variable whose value is determined by the outcome of a chance experiment. Its domain is the sample space. Statistics : The use of samples to make statements about populations (i.e., inference).
Simple Random Sample (SRS) : Selection of n individuals from a population of size N so that every set of n individuals has the same probability of selection. The number of possible SRS are: N!/(n!(N-n)!).
2 Discrete Distributions
2.1 Bernoulli(p)
Counts successes of a single trial with Boolean responses. Special case of the binomial distribution when n = 1. P(X = x|p) = p^x(1 - p)^(1-x); x = 0, 1; 0 ≤ p ≤ 1 E[X] = p, Var[X] = p(1 - p) Moment Generating Function (MGF): MX(t) = (1 - p) + pe^t
2.2 Binomial(n, p)
Counts successes of n independent trials with Boolean responses (samples with replacement). Related to the multinomial distribution, a multivariate version of the binomial distribution. Can be approximated by the Poisson as n approaches ∞. P(X = x|n, p) = (n choose x) p^x(1 - p)^(n-x); x = 0, 1, 2, ...; 0 ≤ p ≤ 1 E[X] = np, Var[X] = np(1 - p) MGF: MX(t) = [pe^t + (1 - p)]^n
2.3 Geometric(p)
Determines the probability until the first success. Y = X - 1 is negative binomial (1, p). The distribution is memoryless: P(X > s|X > t) = P(X > s - t). P(X = x|p) = p(1 - p)^(x-1); x = 1, 2, ...; 0 ≤ p ≤ 1 E[X] = 1/p, Var[X] = (1 - p)/p^ MGF: MX(t) = p/(1 - (1 - p)e^t), t < -log(1 - p)
2.4 Hypergeometric
Counts successes of n trials with Boolean responses (samples without replacement). If K << M and N, the range of x = 0, 1, 2, ..., K will be appropriate. P(X = x|N, M, K) = (M choose x)(N-M choose K-x)/(N choose K); x = 0, 1, 2, ..., K; M-(N-K) ≤ x ≤ M; N, M, K ≥ 0 E[X] = KM/N, Var[X] = KM(N-M)(N-K)/N^2(N-1) Moment Generating Function does not exist.
2.5 Negative Binomial(r, p)
Counts fails of n trials with Boolean responses. Can be approximated by a geometric with n = 1. P(X = x|r, p) = (r+x-1 choose x) p^r(1 - p)^x; x = 0, 1, ...; 0 ≤ p ≤ 1
3.4 Double Exponential(μ, σ)
The difference between two independent identically distributed exponential random variables. Also known as the Laplace distribution. f(x|μ, σ) = (1/(2σ))e^(-(|x-μ|/σ)), -∞ < x < ∞, -∞ < μ < ∞, σ > 0 E[X] = μ, Var[X] = 2σ^ MGF: MX(t) = 1/(1 - (σt)^2), |t| < 1/σ
3.5 Exponential(β)
Measures the lifetime of items. Special case of the gamma distribution. Has the memoryless property. f(x|β) = (1/β)e^(-x/β), 0 ≤ x < ∞, β > 0 E[X] = β, Var[X] = β^ Alternative Parameterization: f(x|λ) = λe^(-λx), 0 ≤ x < ∞, λ > 0; E[X] = 1/λ, Var[X] = 1/λ^ MGF: MX(t) = λ/(λ-t), t < λ
3.6 F
Related to chi squared (Fν1,ν2 = (χ^2_ν1/ν1)/(χ^2_ν2/ν2)) and t (F1,ν = t^2). f(x|ν1, ν2) = (Γ((ν1+ν2)/2)/(Γ(ν1/2)Γ(ν2/2))) (ν1/ν2)^(ν1/2) x^((ν1-2)/ 2)/(1 + (ν1/ν2)x)^((ν1+ν2)/2); 0 ≤ x < ∞; ν1, ν2 = 1, ... E[X] = ν2/(ν2-2), ν2 > 2; Var[X] = 2ν2^2(ν1+ν2-2)/(ν1(ν2-2)^2(ν2-4)), ν2 > 4 Moments: E[X^n] = Γ((ν1+2n)/2)(ν2-2n)/(Γ(ν1/2)Γ(ν2/2))(ν1/ν2)^n, n < ν2/
3.7 Gamma(α, β)
Used for non-negative and skewed right data. Some special cases are exponential (α = 1) and chi squared (α = p/2, β = 2). f(x|α, β) = (1/(Γ(α)β^α)) x^(α-1)e^(-x/β), 0 ≤ x < ∞, α, β > 0 E[X] = αβ, Var[X] = αβ^ Alternative Parameterization: f(x|α, β) = (β^α/Γ(α)) x^(α-1)e^(-βx), E[X] = (α-1)/β, Var[X] = (α-1)/β^ MGF: MX(t) = (1/(1-βt))^α, t < 1/β
3.8 Lognormal(μ, σ^2)
Convenient for measurements in exact and engineering sciences, medicine, and economics. f(x|μ, σ^2) = (1/(x√(2πσ^2))) e^(-(log x-μ)^2/(2σ^2)), 0 ≤ x < ∞, -∞ < μ < ∞, σ > 0 E[X] = e^(μ+σ^2/2), Var[X] = e^(2μ+σ^2)(e^(σ^2)-1) Moments: E[X^n] = e^(nμ+n^2σ^2/2)
3.9 Normal(μ, σ^2)
f(x|μ, σ^2) = (1/(√(2πσ^2))) e^(-(x-μ)^2/(2σ^2)), -∞ < x < ∞, -∞ < μ < ∞, σ > 0 E[X] = μ, Var[X] = σ^ MGF: MX(t) = e^(μt+σ^2t^2/2)
3.10 t
Related to F (F1,ν = t^2). f(x|ν) = Γ((ν+1)/2)/(√(νπ)Γ(ν/2)) (1 + x^2/ν)^(-(ν+1)/2), -∞ < x < ∞, ν = 1, ... E[X] = 0, Var[X] = ν/(ν-2), ν > 2 Moments: E[X^n] = 0 if n is odd, and √(2)Γ((n+1)/2)Γ((ν-n)/2)/ (√(πν)Γ(ν/2)) if n is even and n < ν
3.11 Uniform(a, b)
If a = 0 and b = 1, this is a special case of the beta (α = β = 1). f(x|a, b) = 1/(b-a), a ≤ x ≤ b E[X] = (a+b)/2, Var[X] = (b-a)^2/ MGF: MX(t) = (e^(bt) - e^(at))/(t(b-a))
3.12 Weibull (γ, β)
The MGF exists only for γ ≥ 1. Its form is not very useful. f(x|γ, β) = (γ/β)(x/β)^(γ-1)e^(-(x/β)^γ), 0 ≤ x < ∞, γ > 0, β > 0 E[X] = βΓ(1 + 1/γ), Var[X] = β^2(Γ(1 + 2/γ) - Γ(1 + 1/γ)^2) Moments: E[X
Functions of Random Variables
6.1 Method of Distribution Functions
To find the density for a function U of random variables Y1, ..., Yn with joint density function f(y1, ..., yn):
Determine the region where U takes the value in the y1, ..., yn space. Determine the region in the y1, ..., yn space where U ≤ u. Find the distribution function FU(u) = P(U ≤ u) for U by integrating f(y1, ..., yn) over the region where U ≤ u. Calculate the density for U as the derivative of its distribution: fu(u) = δFU(u)/δu.
Example: Find the density for U = 4Y + 2.
Given: f(x, y) = ( 3y^2, 0 ≤ y ≤ 1 0, otherwise )
U ≤ u: 4Y + 2 ≤ U, so Y ≤ (U - 2)/4.
