工程概率统计 学习笔记

Chapter 1. Basis

Basic Concepts

Random Experiment (随机试验)

Sample Point $\omega$ (样本点): Each possible fundamental outcome.

Sample Space $\Omega$​ (样本空间): A set that includes all sample points.

Random Event (随机事件): A subset of $\Omega$.

Inclusion: $A \subset B$.

Sum / Union: $A \cup B$, $A + B$.

Product / Intersection: $A \cap B$, $AB$.

Difference: $A - B$, $A \setminus B$.

Mutually Exclusive / Disjoint (互斥): $A \cap B = \varnothing$.

Complement (对立, 互补): $A \cap B = \varnothing \land A \cup B = \Omega$. Denoted as $B = A^c = \overline{A}$.

De Morgan’s Laws: $\overline{\bigcup_{i = 1}^{\infty} A_i} = \bigcap_{i = 1}^{\infty} \overline{A_i}$, $\overline{\bigcap_{i = 1}^{\infty} A_i} = \bigcup_{i = 1}^{\infty} \overline{A_i}$.

Probability (Measure) (概率(测度)): a real-valued function defined on subsets of $\Omega$ that satisfies:

• Non-negativity: $A \subseteq \Omega$, $P(A) \ge 0$​.
• Normalization: $P(\Omega) = 1$.
• Additivity: $P\left(\bigcup_{i = 1}^{\infty} A_i\right) = \sum_{i = 1}^{\infty} P\left(A_i\right)$.

Properties of Probability:

• $P(\varnothing) = 0$​.
• Finite Additivity: $P\left(\bigcup_{i = 1}^n A_i\right) = \sum_{i = 1}^n P\left(A_i\right)$​.
• The Complement Rule: $P(\overline{A}) = 1 - P(A)$.
• The numeric bound: $0 \le P(A) \le 1$.
• Monotonicity: if $A \subseteq B$, then $P(A) \le P(B)$ and $P(B - A) = P(B) - P(A)$.
• The Addition Law: $P(A \cup B) = P(A) + P(B) - P(AB)$.
• The Inclusion-exclusion Principle (容斥原理)

Computing Probabilities

Classical Model of Probability (古典概型): each sample point is equally likely to occur. $P(A) = \frac{k}{n}$.

Geometric Model of Probability (几何概型)

Conditional Probability & Independence

Conditional Probability: $P(A | B) = \frac{P(AB)}{P(B)}$.

Multiplication Law: $P(AB) = P(A | B) P(B)$.

Law of Total Probability: For a partition (分划) $B_1, B_2, \cdots, B_n$, $P(A) = \sum P(A | B_i) P(B_i)$.

Bayes’ Theorem / Bayes’ Rule: $P(B_i | A) = \frac{P(B_i) P(A | B_i)}{P(A)} = \frac{P(B_i) P(A | B_i)}{\sum P(B_j) P(A | B_j)}$​.

• Posterior Probaility (后验概率): $P(B_i | A)$ (of $B_i$).
• Prior Probability / Marginal Probability (边缘概率): $P(B_i)$ (of $B_i$).

Independence: If $P(AB) = P(A) P(B)$, then $A$ and $B$​ are independent.

Mutually Exclusive: If $P(AB) = 0$, then $A$ and $B$ are mutually exclusive.

---

Chapter 2. Random Variables and Distributions

Random Variables

Definition:

• is a real-valued function defined on the sample space, typically denoted by capital letters like $X, Y, Z$.

Discrete Random Variable (离散型随机变量):

• 取有限或可数的值。使用 Probability Mass Function (PMF, 概率质量函数) 描述：

  $$p(a_i) = p_i = P(X = a_i)$$

  满足 $p(a_i) \ge 0$（Non-negativity），$\sum p(a_i) = 1$​（Normalization）。

  

Continuous Random Variable (连续型随机变量):

• 取连续、不可数的值。使用 Probability Density Function (PDF, 概率密度函数) 描述。

  $$P(a \le X \le b) = \int_{x = a}^b f(x) \mathrm{d}x$$

  满足 $f(x) \ge 0$（Non-negativity），$\int_{-\infty}^{\infty} f(x) \mathrm{d}x = 1$​（Normalization）。

  

二者都可以用 Cumulative Distribution Function (CDF, 累积分布函数) 描述：

• $$F(x) = P(X \le x), \quad \forall x \in \R$$

  对于离散型随机变量，CDF 是阶梯函数；对于连续型随机变量，CDF 是连续函数。

  

Expectation (期望):

• 对于离散型随机变量：（前提：$\sum_{k = 1}^{\infty}x_k p_k < \infty$）

  $$E(X) \triangleq \sum_{x = 1}^{\infty} x_k p_k$$

  对于连续型随机变量：（前提：$\int_{-\infty}^{\infty}|x|f(x)\mathrm{d}x < \infty$）

  $$E(X) \triangleq \int_{-\infty}^{\infty} x f(x) \mathrm{d}x$$

Variance (方差):

• Describes the stability (稳定性) / variability (波动性) of a random variable around its expectation.

• If $E\left(X^2\right) < \infty$, then

  $$\text{Var}(X) \triangleq E\left(X - E(X)\right)^2$$

  and Standard Deviation (SD, 标准差) is defined as

  $$\text{SD}(X) \triangleq \sqrt{\text{Var}(X)}$$

• Properties:

  • $E(aX + b) = aE(X) + b$.
  • $\text{Var}(aX + b) = a^2 \text{Var}(x)$, $\text{SD}(aX + b) = a \text{SD}(X)$.
  • $\text{Var}(X) = E\left(X^2\right) - [E(X)]^2$​.
  • $E[g_1(X) \pm g_2(X)] = E[g_1(X)] \pm E[g_2(X)]$.

Discrete Distributionsss

Bernoulli Distribution (伯努利分布): (抽一次奖)

$$p(x) = p^x (1 - p)^{1 - x}, \quad x = 0, 1$$

$$E(X) = p, \quad \text{Var}(X) = p(1 - p)$$

Binomial Distribution (二项分布): (抽 $n$ 次奖)

$$X \sim \text{Binomial}(n, p) / \text{Bin}(n, p) / B(n, p)$$

$$p(x) = \binom{n}{x} p^x (1 - p)^{n - x}, \quad x = 0, 1, 2, \cdots, n$$

$$E(X) = np, \quad \text{Var}(X) = np(1 - p)$$

Geometric Distribution (几何分布): (抽奖直到第 $x$ 次中)

$$X \sim \text{Geometric}(p)$$

$$p(x) = p(1 - p)^{x - 1}, \quad x = 1, 2, \cdots$$

$$E(X) = \frac{1}{p}, \quad \text{Var}(X) = \frac{1 - p}{p^2}$$

• 独有性质：Memoryless Property (无记忆性): $P(X > m + n \mid X > m) = P(X > n)$​.

Poisson Distribution (泊松分布): (单位时间发生率为 $\lambda$，发生 $x$ 次的概率)

$$X \sim \text{Poisson}(\lambda)$$

$$p(x) = \frac{\lambda^x}{x!} e^{-\lambda}, \quad x = 0, 1, 2, \cdots$$

$$E(X) = \lambda, \quad \text{Var}(X) = \lambda$$

• where the parameter $\lambda$​ is called the intensity (泊松强度).
• The Poisson distribution can be derived as a limiting case of the binomial distribution.

Continuous Distributions

Uniform Distribution (均匀分布):

$$X \sim \text{Uniform}(a, b) / X\sim U(a, b)$$

$$f(x) = \begin{cases} \frac{1}{b - a}, & a < x < b \\ 0, & \text{otherwise} \end{cases}$$

$$P(X \le x) = \begin{cases} 0, & x \le a \\ \frac{x - a}{b - a}, & a < x < b \\ 1, & x \ge b \end{cases}$$

Exponential Distribution (指数分布):

$$X \sim \text{Exp}(\lambda)$$

$$f(x) = \begin{cases} \lambda e^{-\lambda x}, & x \ge 0 \\ 0, & \text{otherwise} \end{cases}$$

$$P(X \le x) = \begin{cases} 1 - e^{-\lambda x}, & x \ge 0 \\ 0, & \text{otherwise} \end{cases}$$

$$E(X) = \frac{1}{\lambda}, \quad \text{Var}(X) = \frac{1}{\lambda^2}$$

• Memoryless Property (无记忆性): $P(X > m + n \mid X > m) = P(X > n)$​​.
• 泊松过程（Poisson Process）可以简单理解为随机事件沿时间轴独立且速率恒定发生的过程。指数分布可以用来描述泊松过程中事件之间时间间隔的分布。

Normal Distribution (正态分布) / Gaussian Distribution (高斯分布):

$$X \sim N\left(\mu, \sigma^2\right)$$

$$f(x) = \frac{1}{\sqrt{2 \pi} \sigma} e^{-\frac{(x - \mu)^2}{2 \sigma^2}}, \quad x \in \R$$

$$E(X) = \mu, \quad \text{Var}(X) = \sigma^2$$

• $N(0, 1)$ is called Standard Normal Distribution (标准正态分布), whose CDF is:

  $$\Phi(x) = P(X \le x) = \int_{-\infty}^{x} \frac{1}{\sqrt{2 \pi}} e^{-\frac{t^2}{2 \sigma^2}} \mathrm{d}t, \quad x \in \R$$

  If $X \sim N(\mu, \sigma^2)$ and $Z = (X - \mu) / \sigma$, then $Z \sim N(0, 1)$. So:

  $$P(X \le x) = P\left(\frac{X - \mu}{\sigma} \le \frac{x - \mu}{\sigma}\right) = P\left(Z \le \frac{x - \mu}{\sigma}\right) = \Phi\left(\frac{x - \mu}{\sigma}\right)$$

  By symmetry of PDF:

  $$\Phi(x) + \Phi(-x) = 1, \quad x \in \R$$

• $\mu$ 变大，PDF 向右移。$\sigma$ 变大，PDF 变平坦。

Transformation

For a discrete r.v. $X$ with PMF $p_X(x)$:

$$p_Y(y) = P(Y = y) = \sum_{x: g(x) = y} p_X(x)$$

$$E(Y) = E(g(X)) = \sum_{x} g(x) p_X(x)$$

For a continuous r.v. $X$ with PDF $f_X(x)$:

$$p_Y(y) = P(Y = y) = \int_{x: g(x) = y} f_X(x) \mathrm{d}x$$

$$f_Y(y) = \begin{cases} |h'(y)| \cdot f_X(h(y)), & \text{where }y\text{ is defined} \\ 0, & \text{otherwise} \end{cases}$$

$$E(Y) = E(g(X)) = \int_{-\infty}^{\infty} g(x) f(x) \mathrm{d}x$$

• where $h(y) = g^{-1}(y)$​ and it’s continuously-differentiable.

Jensen’s Inequality (琴生不等式):

• For any convex function (凸函数):

  $$E(g(X)) \ge g(E(X))$$

  For any concave fuction (凹函数):

  $$E(g(X)) \le g(E(X))$$

• Inferences:

  • $E(|X|) \ge |E(X)|$.
  • $E\left(X^2\right) \ge (E(X))^2$.
  • $E\left({|X|}^p\right) \ge {|E(X)|}^p$.
  • $E\left(e^{cX}\right) \ge e^{cE(X)}$.

---

Chapter 3. Joint Distributions

Random Vector

A random vector $(X, Y)$​ is also called a two-dimensional random variable.

For discrete random vectors: (Non-negativity, Normalization)

• Joint CDF (联合累积分布函数): $F(x, y) = P(X \le x, Y \le y)$, $\forall x, y \in \R$.

  Marginal CDF (边缘累积分布函数): $F_X(x) = F(x, \infty)$, $F_Y(y) = F(\infty, y)$.

• Joint PMF (联合概率质量函数): $p(x_i, y_i) = P(X = x_i, Y = y_i) = p_{ij}$, $i, j = 1, 2, \cdots$.

  Marginal PMF (边缘概率质量函数): $p_X(x_i) = P(X = x_i) = \sum_{j = 1}^{\infty} p_{ij} = p_i$, $i = 1, 2, \cdots$.

For continous random vectors: (Non-negativity, Normalization)

• Joint CDF (联合累积分布函数): $F(x, y) = \int_{-\infty}^{x}\int_{-\infty}^{y} f(u, v) \mathrm{d}u \mathrm{d}v$, so that $f(x, y) = \frac{\partial^2}{\partial x \partial y} F(x, y)$.

• Joint PDF (联合概率密度函数): $P\big((X, Y) \in D\big) = \iint_{(x, y) \in D} f(x, y) \mathrm{d}x\mathrm{d}y$.

  Marginal PDF (边缘概率密度函数): $f_X(x) = \int_{-\infty}^{\infty} f(x, y) \mathrm{d}y$.

Conditinally: (Non-negativity, Normalization)

• Conditional PMF (条件概率质量函数): $p_{X \mid Y}(x \mid y) = P(X = x \mid Y = y) = \frac{p(x, y)}{p_Y(y)}$​. ($p_Y(y) > 0$)

  Conditional PDF (条件概率密度函数): $f_{X \mid Y}(x \mid y) = \frac{f(x, y)}{f_Y(y)}$​. ($f_Y(y) > 0$)

Relationship

Independence of Random Variables:

• Let $F(x_1, x_2, \dots, x_n)$ be the joint CDF of $(X_1, X_2, \dots, X_n)$, $F_{X_i}(x_i)$ be the marginal CDF of $X_i$, then if for $\forall x_1, x_2, \dots, x_n \in \mathbb{R}$ we have

  $$F(x_1, x_2, \dots, x_n) = F_{X_1}(x_1) F_{X_2}(x_2) \dots F_{X_n}(x_n)$$

  then they are (mutally) independent.

  For discrete random variables $X_1, X_2, \dots, X_n$, if they are independent, then the PMF satisfies

  $$p(x_1, x_2, \dots, x_n) = P(X_1 = x_1, \dots, X_n = x_n) = p_{X_1}(x_1) p_{X_2}(x_2) \dots p_{X_n}(x_n)$$

  For continuous random variables $X_1, X_2, \dots, X_n$, if they are independent, then the PDF satisfies

  $$f(x_1, x_2, \dots, x_n) = f_{X_1}(x_1) f_{X_2}(x_2) \dots f_{X_n}(x_n)$$

• If $X$ and $Y$ are independent random variables, then for any functions $g$ and $h$, we have

  $$E[g(X)h(Y)] = E[g(X)]E[h(Y)],$$

  $$\mathrm{Var}[g(X) \pm h(Y)] = \mathrm{Var}[g(X)] + \mathrm{Var}[h(Y)]. \quad (\text{注意符号})$$

Covariance (协方差):

• The covariance between $X$ and $Y$, denoted by $\mathrm{Cov}(X, Y)$, is defined by

  $$\mathrm{Cov}(X, Y) = E[(X - E(X))(Y - E(Y))] = E(XY) - E(X)E(Y).$$

  If $X$ and $Y$ are independent, then $\mathrm{Cov}(X, Y) = 0$.

  However, if $\mathrm{Cov}(X, Y) = 0$, $X$ and $Y$ may not be independent; we can only say that $X$ and $Y$ are uncorrelated (不相关的).

• Note that $\mathrm{Cov}(X, Y) > 0$ when $X$ and $Y$ tend to vary in the same direction and $< 0$ when they tend to vary in the opposite direction.

• Properties ($a, b, c$ are constants):

  • Covariance-variance relationship:

    $$\mathrm{Cov}(X, X) = \mathrm{Var}(X);$$

    $$\mathrm{Var}(X \pm Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) \pm 2\mathrm{Cov}(X, Y).$$

  • Symmetry:

    $$\mathrm{Cov}(X, Y) = \mathrm{Cov}(Y, X).$$

  • Constants cannot covary:

    $$\mathrm{Cov}(X, c) = 0.$$

  • Pulling out constants:

    $$\mathrm{Cov}(aX, bY) = ab\,\mathrm{Cov}(X, Y).$$

  • Distributive property:

    $$\mathrm{Cov}(X_1 + X_2, Y) = \mathrm{Cov}(X_1, Y) + \mathrm{Cov}(X_2, Y).$$

  • Bilinear property:

    $$\mathrm{Cov}\!\left(\sum_{i=1}^{n} a_i X_i,\; \sum_{j=1}^{m} b_j Y_j\right) = \sum_{i=1}^{n}\sum_{j=1}^{m} a_i b_j\,\mathrm{Cov}(X_i, Y_j).$$

Correlation Coefficient (相关系数):

• The correlation coefficient (相关系数) between $X$ and $Y$, denoted by $\mathrm{Cor}(X, Y)$ or $\rho_{XY}$, is defined by

  $$\rho_{XY} = \mathrm{Cor}(X, Y) = \frac{\mathrm{Cov}(X, Y)}{\mathrm{SD}(X)\,\mathrm{SD}(Y)} = \frac{\mathrm{Cov}(X, Y)}{\sqrt{\mathrm{Var}(X)\mathrm{Var}(Y)}}.$$

  $\rho_{XY}$​ is a normalized version of the covariance, which is a dimensionless quantity (无量纲数值).

• $\rho_{XY} \in [-1, 1]$, measure the direction and strength of the linear relationship between $X$ and $Y$.

Conditional Expectation (条件期望):

• For a discrete random vector $(X, Y)$, the conditional expectation of $X$ given $Y = y$ is

  $$E(X \mid Y = y) = \sum_k x_k \cdot P(X = x_k \mid Y = y).$$

  For a continuous random vector $(X, Y)$, the conditional expectation of $X$ given $Y = y$ is

  $$E(X \mid Y = y) = \int_{-\infty}^{\infty} x \cdot f_{X \mid Y}(x \mid y)\,dx,$$

  where $f_{X \mid Y}(x \mid y)$ is the conditional PDF of $X$ given $Y = y$.

Law of Total Expectation (全期望公式):

• For a random vector $(X, Y)$, if $E(X)$ exists, then

  $$E(X) = E\!\bigl(E(X \mid Y)\bigr),$$

  called double expectation.

• Specifically,

  • For a discrete random vector $(X, Y)$,

    $$E(X) = \sum_j E(X \mid Y = y_j) \cdot P(Y = y_j).$$

  • For a continuous random vector $(X, Y)$,

    $$E(X) = \int_{-\infty}^{\infty} E(X \mid Y = y)\,f_Y(y)\,dy.$$

Function of Multiple Random Variables

PDF of $Z = X + Y$ - Continous Case:

• Let $f(x, y)$ be the PDF of random vector $(X, Y)$, then the PDF of $Z = X + Y$ is

  $$f_Z(z) = \int_{-\infty}^{\infty} f(z - y, y) \mathrm{d}y = \int_{-\infty}^{\infty} f(x, z - x) \mathrm{d}x$$

• Specifically, if $X$ and $Y$ are independent, then

  $$f_Z(z) = f_X * f_Y \triangleq \int_{-\infty}^{\infty} f_X(z - y) f_Y(y) \mathrm{d}y = \int_{-\infty}^{\infty} f_X(x) f_Y(z - x) \mathrm{d}x$$

  where the two integrals are called the Convolution (卷积) of $f_X$ and $f_Y$, denoted as $f_X * f_Y$.

• Sum of Independent Normal r.v.: If $X_1, X_2, \dots, X_n$ are independent and $X_i \sim N(\mu_i, \sigma_i^2)$, then for constants $a_1, a_2, \dots, a_n$,

  $$\sum_{i = 1}^n a_i X_i \sim N\left(\sum_{i = 1}^n a_i \mu_i, \sum_{i = 1}^n a_i \sigma_i^2\right)$$

PDF of $Z = X + Y$ - Discrete Case:

• Let $X$ and $Y$ be two discrete r.v. $\in \{0, 1, 2, \dots\}$, then the PMF of $Z = X + Y$ is

  $$P(Z = k) = \sum_{i = 0}^k P(X = i, Y = k - i) = \sum_{j = 0}^k P(X = k - j, Y = j)$$

• Spcifically, if $X$ and $Y$ are independent, then

  $$P(Z = k) = \sum_{i = 0}^k P(X = i) \cdot P(Y = k - i) = \sum_{j = 0}^k P(X = k - j) \cdot P(Y = j)$$

  which is the Discrete Convolution Formula (离散卷积公式).

Central Limit Theorem (CLT, 中心极限定理):

• For a sequence of i.i.d. (independent and identically distributed, 独立同分布) r.v. $X_1, X_2, \dots, X_n$ with $\mu \triangleq E(X_i)$ and $\sigma^2 \triangleq \mathrm{Var}(X_i)$, let $S_n = \sum X_i$ and $\bar X_n = S_n / n$, then

  $$Z_n = \frac{S_n - E(S_n)}{\sqrt{\mathrm{Var}(S_n)}} = \frac{S_n - n\mu}{\sqrt{n\sigma^2}} = \frac{\sqrt{n}(\bar X_n - \mu)}{\sigma}$$

• As $n \to \infty$, $Z_n$ converges in distribution (依分布收敛) to a standatd normal r.v., that is $\forall z$,

  $$F_{Z_n}(z) = P(Z_n \le z) \xrightarrow{n \to \infty} \Phi(z)$$

  If $X_1, X_2, \dotsm X_n$ are normal r.v.s, $F_{Z_n}(z) = P(Z_n \le z) = \Phi(z)$ even for small $n$.

• The rule of thumb (经验法则): $n \ge 30$ should be to apply the CLT.

Approximation of $\text{Binomial}(n, p)$:

• Poisson Approximation $\text{Poisson}(np)$: Works well when $n$ is large and $p$ is small, e.g., $n > 100$ and $p < 0.05$.
• Normal Approximation $N\big(np, np(1 - p)\big)$: Works well when $np \ge 5$ and $n(1 - p) \ge 5$.
• Continuity Correction (连续性修正): Expand the interval by $0.5$ in each direction.

CDF of $\max(X, Y)$ & $\min(X, Y)$ - Continuous Case:

• Let $f(x, y)$ be the PDF of random vector $(X, Y)$, then the CDFs of $\max(X, Y)$ and $\min(X, Y)$ are

  $$F_{\max}(z) = \int_{-\infty}^{z} \int_{-\infty}^{z} f(x, y) \mathrm{d}x \mathrm{d}y$$

  $$F_{\min}(z) = 1 - \int_{z}^{\infty} \int_{z}^{\infty} f(x, y) \mathrm{d}x \mathrm{d}y$$

• Specifically, if $X$ and $Y$ are independent, then

  $$F_{\max}(z) = F_X(z)F_Y(z)$$

  $$F_{\min}(z) = 1 - [1 - F_X(z)][1 - F_Y(z)]$$

• Generally, if $X_1, X_2, ..., X_n \overset{\text{i.i.d.}}{\sim} F(x)$ with PDF $f(x)$, then the CDFs and PDFs of $\max(X_1, X_2, ..., X_n)$ and $\min(X_1, X_2, ..., X_n)$ are

  $$F_{\max}(z) = [F(z)]^n, \quad f_{\max}(z) = n f(z)[F(z)]^{n-1}$$

  $$F_{\min}(z) = 1 - [1 - F(z)]^n, \quad f_{\min}(z) = n f(z)[1 - F(z)]^{n-1}$$

Multivariate Normal Distribution

Bivariate Normal Distribution (二元正态分布):

• Random vector (𝑋, 𝑌) is said to be bivariate normally distributed with means $\mu_X, \mu_Y$ and variances $\sigma_X^2, \sigma_Y^2$, and with correlation coefficient $\rho$, then it can be expressed as

  $$(X, Y) \sim N\left(\begin{pmatrix}u_X \\ \mu_Y\end{pmatrix} \cdot \begin{pmatrix} \sigma_X^2 & \rho \sigma_X \sigma_Y \\ \rho \sigma_X \sigma_Y & \sigma_Y^2 \end{pmatrix}\right) = N(\boldsymbol\mu, \boldsymbol\Sigma)$$

  where $\boldsymbol\mu$ is the Mean Vector (均值向量) and $\boldsymbol\Sigma$ is the Convariance Matrix (协方差矩阵).

• Generally, for an $n$-dim random vector $\boldsymbol X = (X_1, X_2, \dots, X_n)^\top$:

  • The mean vector is $E(\boldsymbol X) \triangleq \big(E(X_1), E(X_2), \dots, E(X_n)\big)^\top$.
  • The covariance matrix $\mathrm{Cov}(\boldsymbol X) \triangleq E\Big[\big(\boldsymbol X - E(\boldsymbol X)\big) \big(\boldsymbol X - E(\boldsymbol X)\big)^\top\Big]$.

• Properties:

  • Marginals: If $(X, Y)$ is jointly normal, then the marginal distributions of $X$ and $Y$ are univariate normal. 二元正态 -> 分别正态

  • Independence & Non-Correlation: For bivariate normal distribution, “Independent” is equal to “Uncorrelated ($\rho = 0$)”. 服从二元正态分布，则独立和不相关等价

  • Conditional Distributions: $Y \mid X = x$ and $X \mid Y = y$​ is also normal.

    • Conditional Mean: $E(Y \mid X = x) = \mu_Y + \rho \frac{\sigma_Y}{\sigma_X}(x - \mu_X)$.
    • Conditional Variance: $\text{Var}(Y \mid X = x) = \left(1 - \rho^2\right)\sigma_Y^2$​.

  • Linear Combinations: For any real matrix $\boldsymbol A = \begin{pmatrix}a_{11} & a_{12} \\ a_{21} & a_{22}\end{pmatrix}$, then

    $$\boldsymbol{AX} = \begin{pmatrix} a_{11}X + a_{12}Y \\ a_{21}X + a_{22}Y \end{pmatrix} \sim N(\boldsymbol{A\mu}, \boldsymbol{A\Sigma A^T})$$

• Application: Gaussian Mixture Model (GMM, ⾼斯混合模型).

---

Chapter 4. Monte Carlo Methods

Law of Large Numbers (LLN)

For a sequence of i.i.d. r.v.s $X_1, X_2, \dots, X_n$ with $\mu \triangleq E(X_i) < \infty$, $\bar X_n = (X_1 + X_2 + \dots + X_n) / n$ converges in probability (依概率收敛) to $\mu$ as $n \to \infty$:

$$\bar X_n \xrightarrow{P} \mu, \text{ as } n \to \infty$$

Simulation of Random Variables

Pseudo-Random Number Generator (PRNG):

• Computers generate pseudo-random numbers, not truly random.
• They are determined by an initial value called the seed.
• Pros: Fast and Reproducible (same seed = same sequence).
• The Building Block ($U[0,1]$):
  • PRNGs typically provide a sequence of independent uniformly distributed numbers on $[0, 1]$.
  • Goal: Transform $U \sim U[0,1]$ into other distributions (Bernoulli, Poisson, Normal, etc.).

Discrete Distributions:

• Bernoulli ($p$): Generate $u$. If $u < p$, output 1; else output 0.
• Poisson ($\lambda$): Divide interval $(0,1)$ into sub-intervals based on the PMF (probability mass function). Check which interval $u$ falls into.

Inverse Transformation Sampling (逆变换采样):

• Theory: If $F(x)$ is the CDF, let $U \sim U[0,1]$, then $X = F^{-1}(U)$ follows the distribution $F$.
• e.g. Exponential ($\lambda$):
  • $F(x) = 1 - e^{-\lambda x} \rightarrow F^{-1}(u) = -\frac{\ln(1-u)}{\lambda}$.
  • Since $1-u$ is also uniform, we can use $x = -\frac{\ln(u)}{\lambda}$.

Rejection Sampling (拒绝采样):

• Concept:
  • Target Distribution: $f(x)$ (The difficult one we want).
  • Proposal Distribution: $g(x)$ (An easy one we can sample from, e.g., Uniform or Normal).
  • Condition: We need a constant $c > 1$ such that $f(x) \le c \cdot g(x)$ for all $x$. (The curve $cg(x)$ must cover $f(x)$ completely).
• Algorithm:
  • Step 1: Generate a candidate $x$ from $g(x)$.
  • Step 2: Generate a random $u$ from $U[0,1]$.
  • Step 3: Accept $x$ if $u \le \frac{f(x)}{c \cdot g(x)}$. Otherwise, Reject and repeat.
• Rationale (Dartboard Analogy):
  • Throw darts uniformly at the area under $c \cdot g(x)$.
  • Keep only the darts that fall under the curve $f(x)$.
• Efficiency:
  • Acceptance probability = $1/c$.
  • Ideally, $c$ should be as small as possible (close to 1) to reduce waste (rejected samples).

---

Chapter 5. Basic Statistics

Basic Concepts

Population (总体): The entire set of individuals, objects, events, or measurements that share a common characteristic and are of interest in a particular study.

Parameter (参数): Any numerical characteristic of a population.

Sample (样本): A subset of the population, which is used to make inferences about the population.

• $n$ individuals are selected from the population, their quantitative index are recorded as $X_1, X_2, \dots, X_n$, which is called a sample of size $n$, $n$ is the sample size (样本容量).
• The process of selecting individuals from the population is called sampling (抽样).
• The observed values $x_1, x_2, \dots, x_n$ of $X_1, X_2, \cdots, X_n$ we obtained after sampling are called sample observed values (样本观测值).

Simple Random Sampling (简单随机抽样): To select samples in a completely random manner, ensuring that every individual in the population has an equal probability of being chosen.

Statisic (统计量):

• If $X_1, ..., X_n$ is a sample from population $X \sim f(x; \boldsymbol{\theta})$ and $g(x_1, ..., x_n)$ is an $n$-variate function ($n$ 元函数). then $T = g(X_1, ..., X_n)$ is called a statistic if $g()$ does not involve any unknown parameter.

• e.g. that commonly used:

  • Sample Mean (样本均值):

    $$\bar{X} = \frac{1}{n} \sum_{i=1}^{n} X_i$$

  • Sample Variance (样本方差) & Sample Standard Deviation (样本标准差):

    $$S^2 = \frac{1}{n-1} \sum_{i=1}^{n} (X_i - \bar{X})^2, \quad S = \sqrt{S^2}$$

  • Order Statistics (顺序统计量):

    $$(X_{(1)}, X_{(2)}, \dots, X_{(n)})$$

    It implys that $X_{(1)} = \min\{X_1, \dots, X_n\}$, $X_{(n)} = \max\{X_1, \dots, X_n\}$.

  • Sample $p$-Quantile (样本 $p$ 分位数): number that exceeds at most $100p\%$ of the sample and is exceeded by at most $100(1-p)\%$ of the sample.

    $$Q_p = \begin{cases} X_{(\lfloor np+1 \rfloor)}, & \text{if } np \text{ is not an integer} \\ \frac{(X_{(np)} + X_{(np+1)})}{2}, & \text{if } np \text{ is an integer} \end{cases}$$

    Specifically,

    • $Q_{0.5}$ is the Sample Median (样本中位数).
    • $Q_{0.25}$ ($Q_{0.75}$) is the Sample Lower (Upper) Quartile (样本下(上)四分位数) or the Sample First (Third) Quartile (样本第一(三)四分位数) , also written as $Q_1$ ($Q_3$).
    • $Q_3 - Q_1$ is the Sample Interquartile Range (Sample IQR, 样本四分位距).

Sampling Distribution: The distribution of a statistic.

Point Estimation

Point Estimation: Estimating an unknown parameter with a statistic.

• Estimator (估计量): A statistic $\hat\theta_l(X_1, X_2, \dots, X_n)$ (an $n$-variate function), used to estimate $\theta_l$.
• Estimate / Estimated Value (估计值): $\hat\theta_l(x_1, x_2, \dots, x_n)$​​.
• Parameter Space (参数空间) $\boldsymbol \Theta$: The set of values that $\boldsymbol\theta = (\theta_1, ..., \theta_k)$ can take.

Unbiasedness (无偏性):

• Unbiased Estimator (无偏估计量): If for $\forall \boldsymbol\theta \in \boldsymbol\Theta$, we have

  $$E_{\boldsymbol{\theta}}(\hat{\theta}_l) = \theta_l$$

  where $E_{\boldsymbol{\theta}}(\hat{\theta}_l)$ is the expected value of the estimator $\theta_l$, assuming the true population parameter is $\boldsymbol\theta$.

  Then $\hat\theta_l$ is called an unbiased estimator of $\theta_l$. Otherwise, it is called a biased estimator of $\theta_l$.

• Bias: $E_{\boldsymbol{\theta}}(\hat{\theta}_l) - \theta_l$.

• Asymptotic Unbiased Estimator (渐进无偏估计量): The bias is not $0$ but converge to $0$ as $n \to \infty$.

• $\hat{\mu} = \bar{X}$ and $\hat{\sigma}^2 = S^2$ are unbiased estimators of $\mu$ and $\sigma^2$, if the population mean $\mu = E(X)$ and population variance $\sigma^2 = \mathrm{Var}(X)$​ exist.

要求给出的估计量 $\hat\theta$ 的期望，利用期望的性质将其转化为 $E(X)$ 等可以用 $\theta$ 表示的式子并化简。

Relative Efficiency (相对有效性):

• For two unbiased estimators $\hat\theta_{l1}$ and $\hat\theta_{l2}$ of $\theta_l$, if for $\forall \boldsymbol{\theta} \in \mathbf{\Theta}$, we have

  $$\mathrm{Var}_{\boldsymbol{\theta}}(\hat{\theta}_{l1}) \leq \mathrm{Var}_{\boldsymbol{\theta}}(\hat{\theta}_{l2})$$

  and $\mathrm{Var}_{\boldsymbol{\theta}}(\hat{\theta}_{l1}) < \mathrm{Var}_{\boldsymbol{\theta}}(\hat{\theta}_{l2})$ for at least one $\boldsymbol{\theta}$, then $\hat{\theta}_{l1}$ is said to be more efficient (更有效) than $\hat{\theta}_{l2}$​.

求方差，利用方差的性质不断代入化简：

$$\mathrm{Var}(g(X) \pm h(Y)) = \mathrm{Var}(g(X)) + \mathrm{Var}(h(Y))$$

$$\mathrm{Var}(aX + b) = a^2 \mathrm{Var}(X)$$

Consistency (相合性):

• If for $\forall \boldsymbol{\theta} \in \mathbf{\Theta}$ and $\forall \epsilon > 0$, we have

  $$\lim_{n \to \infty} P_{\boldsymbol{\theta}}(|\hat{\theta}_l - \theta_l| > \epsilon) = 0$$

• If $\hat\theta$ is an asymptotic unbiased estimator of $\theta$ and $\mathrm{Var}(\hat\theta) \to 0$ as $n \to \infty$, then $\hat\theta$ is a consistent estimator of $\theta$​. 渐进无偏 + 方差收敛到 0 -> 相合

Chebyshev’s Inequuality (切比雪夫不等式):

• Let $X$ be a r.v. with mean $\mu = E(X)$ and variance $\sigma^2 = \mathrm{Var}(X)$ both exists, then

  $$P(|X - \mu| \ge k\sigma) \le \frac{1}{k^2}$$

Method of Moments

The $k$-th Population Moment (总体 $k$ 阶矩): $\mu_k = E\left(X^k\right)$​​.

The $k$-th Population Central Moment (总体 $k$ 阶中心矩): $\tilde{\mu}_k = E\left[(X - \mu_1)^k\right]$. ($k \ge 2$)

• $\mu_1 = E(X)$, $\tilde\mu_2 = \mathrm{Var}(X)$.

The $k$-th Sample Moment (样本 $k$ 阶矩): $M_k = \frac{1}{n} \sum_{i=1}^{n} X_i^k$.

The $k$-th Sample Central Moment (样本 $k$ 阶中心矩): $\tilde{M}_k = \frac{1}{n} \sum_{i=1}^{n} (X_i - \bar{X})^k$. ($k \ge 2$)

• $M_1 = \bar X$, $\tilde M_2 = \tilde S^2 = (n - 1) S^2 / n$.
• Actually, $\tilde{M}_2 = M_2 - M_1^2$. (just like $\text{Var}(X) = E\left(X^2\right) - [E(X)]^2$)

Method of Moments (矩估计法):

If we have $k$ unknown parameters $(\theta_1, \dots, \theta_k)$:

1. Calculate Theoretical Moments: Express $\mu_1, \mu_2, ...$ as functions of $(\theta_1, \dots, \theta_k)$.

   $$\mu_1(\theta_1, \dots, \theta_k), \mu_2(\theta_1, \dots, \theta_k), \dots$$

2. Set up Equations: Equate population moments to sample moments.

   $$\begin{cases} \mu_1(\theta_1, \dots, \theta_k) = M_1 \\ \mu_2(\theta_1, \dots, \theta_k) = M_2 \\ \dots \end{cases}$$

3. Solve for $\boldsymbol\theta$: The solutions $\hat{\theta}_1, \dots, \hat{\theta}_k$​ are called the Moment Estimators (矩估计量).

有多少参数求多少阶矩。以两个参数为例，求出 $\mu_1 = E(X)$、$\mu_2 = E\left(X^2\right)$ 用参数 $\theta_1, \theta_2$ 表示的结果，形成方程组：

$$\begin{cases} E(X) = M_1 \\ E\left(X^2\right) = M_2 \end{cases}$$

把 $M_1, M_2$ 当作已知数解出 $\hat\theta_1, \hat\theta_2$，再代入（或者说明）：

$$M_1 = \frac{1}{n} \sum_{i = 1}^n X_i, \quad M_2 = \frac{1}{n} \sum_{i = 1}^n X_i^2$$

即可。

• For Mean and Variance (in Any Distribution):

  • Estimator for Population Mean $\mu$:

    $$\hat{\mu} = M_1 = \bar{X}$$

  • Estimator for Population Variance $\sigma^2$:

    $$\hat{\sigma}^2 = \tilde M_2 = \tilde S^2 = \frac{n - 1}{n} S^2 = \frac{1}{n}\sum_{i=1}^n (X_i - \bar{X})^2$$

    Note: This is the biased variance (dividing by $n$, but it’s consistent), not the unbiased sample variance $S^2$.

• The moment estimator of a parameter may not be unique.

Method of Maximum Likelihood

Likelihood Function (似然函数) $\mathcal{L}$:

• Since the sample variables $X_1, ..., X_n$ are i.i.d., the joint probability is the product of individual densities/probabilities:

  $$\mathcal{L}(\boldsymbol\theta; x_1, ..., x_n) = \prod_{i=1}^{n} f(x_i; \boldsymbol\theta)$$

• If there exists $\hat{\boldsymbol{\theta}} = \hat{\boldsymbol{\theta}}(x_1, \dots, x_n)$ such that

  $$\mathcal{L}(\hat{\boldsymbol{\theta}}) = \max_{\boldsymbol{\theta} \in \boldsymbol{\Theta}} \mathcal{L}(\boldsymbol{\theta}; x_1, \dots, x_n),$$

  then $\hat{\boldsymbol{\theta}}(x_1, \dots, x_n)$ is the Maximum Likelihood Estimate (极大似然估计值) of $\boldsymbol{\theta}$, the corresponding estimator $\hat{\boldsymbol{\theta}}(X_1, \dots, X_n)$ is the Maximum Likelihood Estimator (MLE, 极大似然估计量) of $\boldsymbol{\theta}$.

Log-Likelihood Function (对数似然函数) $\ell$:

• Maximizing $\mathcal{L}(\boldsymbol\theta; \boldsymbol{x})$ is equivalent to maximizing the log-likelihood function:

  $$\ell(\boldsymbol\theta; \boldsymbol{x}) = \log \mathcal{L}(\boldsymbol\theta; \boldsymbol{x}) = \sum_{i=1}^{n} \log f(x_i; \boldsymbol\theta)$$

Method of Maximum Likelihood:

1. Write the Likelihood Function:

   $$\mathcal{L}(\theta) = \prod_{i=1}^n f(x_i; \theta)$$

2. Take the Logarithm:

   $$\ell(\theta) = \ln \mathcal{L}(\theta) = \sum_{i=1}^n \ln f(x_i; \theta)$$

3. Differentiate: Take the derivative with respect to $\theta$ and set it to $0$ to find the critical point.

   $$\frac{\mathrm{d}}{\mathrm{d}\theta} \ell(\theta) = 0$$

4. Solve for $\theta$: The solution is the MLE, denoted as $\hat{\theta}$.

   (Check second derivative $< 0$​​ to ensure it is a maximum, though usually assumed in exams.)

对于连续 r.v.，求出 PDF 后，把 $\sum \ln X_i$ 当作已知数。对 $\ell(\theta)$ 求导 $= 0$ 解出 $\hat\theta$ 后，可以验证一下二阶导正负。

Summary:

• Under mild regularity conditions, the moment estimators and maximum likelihood estimators are consistent and asymptotic unbiased estimators.
• Asymptotic Normality: Under mild regularity conditions, for large samples, a maximum likelihood estimator has an approximately normal distribution.

Interval Estimation

Confidence Interval (置信区间):

• For $\forall \alpha \in (0, 1)$, if for $\forall \boldsymbol{\theta} \in \boldsymbol{\Theta}$, there exists two statistics $\hat{\theta}_{l1}(X_1, ..., X_n)$ and $\hat{\theta}_{l2}(X_1, ..., X_n)$ such that

  $$P_{\boldsymbol{\theta}}\left(\hat{\theta}_{l1} < \theta_l < \hat{\theta}_{l2}\right) \ge 1 - \alpha$$

  then $(\hat{\theta}_{l1}, \hat{\theta}_{l2})$ is called a Confidence Interval of $\theta_l$ with Confidence Level (置信水平) $1 - \alpha$, or simply a $100(1 - \alpha)\%$ confidence interval. $\hat{\theta}_{l1}$ and $\hat{\theta}_{l2}$ are called the Confidence Lower Limit (置信下限) and Confidence Upper Limit (置信上限), respectively.

General Method: $\text{置信区间} = \text{点估计} \pm (\text{系数} \times \text{标准误差})$.

• For a consistent estimator $\hat\theta_l(X_1, X_2, \dots, X_n)$ of $\theta_l$ with the Standard Error (标准误) $\sigma(\hat\theta_l) = \mathrm{SD}(\hat\theta_l)$:

  • If $\hat{\theta}_l$ exactly follows a normal distribution $N\big(\theta_l, \sigma^2(\hat\theta_l)\big)$, and $\sigma^2(\hat\theta_l)$ is known, then an exact $100(1 - \alpha)\%$ confidence interval of $\theta_l$ is

    $$\hat{\theta}_l \pm z_{\alpha/2}\sigma(\hat{\theta}_l) = \left(\hat{\theta}_l - z_{\alpha/2}\sigma(\hat{\theta}_l), \hat{\theta}_l + z_{\alpha/2}\sigma(\hat{\theta}_l)\right)$$

  • If $\hat{\theta}_l$ approximately follows a normal distribution $N\big(\theta_l, \sigma^2(\hat\theta_l)\big)$ (due to CLT as $n \to \infty$), or the true variance is unknown and replaced by a consistent estimator $\hat{\sigma}^2(\hat{\theta}_l)$, then a large sample $100(1 - \alpha)\%$ confidence interval of $\theta_l$ is

    $$\hat{\theta}_l \pm z_{\alpha/2}\hat{\sigma}(\hat{\theta}_l) = \left(\hat{\theta}_l - z_{\alpha/2}\hat{\sigma}(\hat{\theta}_l), \hat{\theta}_l + z_{\alpha/2}\hat{\sigma}(\hat{\theta}_l)\right)$$

• Commonly used values: $z_{0.10} = 1.282$, $z_{0.05} = 1.645$, $z_{0.025} = 1.960$, $z_{0.01} = 2.326$, $z_{0.005} = 2.576$.

---

Chapter 6. Hypothesis Testing

Basic Ideas

Hypothesis:

• Null Hypothesis (零假设/原假设) $H_0$: The default assumption, status quo, no effect, or “innocent”. We assume it is true until evidence proves otherwise.
• Alternative Hypothesis (备择假设) $H_1$: The claim the researcher wants to prove.
  • $H_0$ and $H_1$ are simply two mutually exclusive statements.
• Proof by Contradiction: We try to find sufficient evidence to reject $H_0$​.

Test Statistic:

• A value calculated and standardized from sample data, usually obtained by a suitable transformation of its estimator $\hat\theta$.

Rejection Region (RR, 拒绝域):

• The set of samples $\boldsymbol X = (X_1, X_2, \dots, X_n)$ such that $\bar X - 8 > c$, denoted by $\{\boldsymbol X: \bar X - 8 > c\}$.

2 Types of Errors:

• Type I Error: Reject $H_0$ when it is actually true. 冤枉好人
  • Type I Error Rate: $P(X \in \mathrm{RR} \mid H_0 \text{ is true})$.
• Type II Error: Fail to reject $H_0$ when it is not true. 放过坏人
  • Type II Error Rate: $\beta = P(X \not\in \mathrm{RR} \mid H_1 \text{ is true})$.
• For a given sample size and form of the RR, we cannot reduce the two error rates at the same time.

Significance Test (显著性检验):

• Let $\boldsymbol X = (X_1, X_2, \dots, X_n)$ be a simple random sample from the population $X \sim f(x; \theta_1, \theta_2, \dots, \theta_k)$. For testing problem $H_0: \theta_l \in \Theta_{l0} \leftrightarrow H_1: \theta_1 \in \Theta_{l1}$ and $\forall \alpha \in (0, 1)$​, if a test has rejection region RR such that

  $$P_{\boldsymbol\theta}(\boldsymbol X \in \mathrm{RR}) \le \alpha, \quad \forall \boldsymbol\theta \in \boldsymbol\Theta \land \theta_l \in \Theta_{l0}$$

  then the test is called a Significance Test of $\theta_l$ with Significance Level (显著性水平) $\alpha$. $\alpha$ is the upper bound of Type I Error Rate.

P-value:

• The probability of obtaining results at least as extreme as the observed data, assuming $H_0$ is true.
• Given a significance level $\alpha$, we would reject $H_0$ if $p\text{-value} < \alpha$; otherwise, we fail to reject $H_0$.

Statistical Power (统计功效):

• The probability of correctly detecting an effect when it actually exists.

  $$\text{Power} = 1 - \beta = P(\text{Reject } H_0 \mid H_1 \text{ is true})$$

• Factors affecting Power:

  • Significance Level $\alpha$: Larger $\alpha$ -> Smaller $\beta$ -> Higher Power.
  • Effect Size: Larger difference between the true parameter value and the values in $H_0$ -> Higher Power.
  • Sample Size $n$: Larger $n$ -> Smaller sampling error rate -> Higher Power.

Testing Process

4 Steps:

1. Formulate Hypotheses: Set up $H_0$ and $H_1$ (One-sided or Two-sided).
2. Test Statistic: Choose formula (usually Z-test for large samples).
3. Decision Rule: Calculate P-value or check if Statistic falls in Rejection Region based on $\alpha$.
4. Conclusion: Reject or Fail to reject.

3 Forms of Parametric Hypothesis Testing:

• $H_0: \theta = \theta_0 \leftrightarrow H_1: \theta < \theta_0$, a one-sided, left-tail alternative hypothesis.
• $H_0: \theta = \theta_0 \leftrightarrow H_1: \theta > \theta_0$, a one-sided, right-tail alternative hypothesis.
• $H_0: \theta = \theta_0 \leftrightarrow H_1: \theta \neq \theta_0$, a two-sided alternative hypothesis.

Common Test Types:

• Mean Test (Z-test): Testing if population mean $\mu$ is greater/less/equal to a value.

  $$Z = \frac{\bar{X} - \mu_0}{\sigma / \sqrt{n}}$$

• Two-Sample Mean Difference: Testing if $\mu_1 = \mu_2$.

  $$Z = \frac{(\bar{X} - \bar{Y}) - 0}{\sqrt{\frac{\sigma_X^2}{n} + \frac{\sigma_Y^2}{m}}}$$

• Proportion Test (Bernoulli): Testing defect rates or percentages ($p$).

  $$Z = \frac{\hat{p} - p_0}{\sqrt{p_0(1-p_0)/n}}$$

Summary

• Confidence intervals and significance tests are similar in that they both rely on the distribution of an estimator of the parameter of interest.
• Limitations:
  • Statistical Significance: The effect is not due to chance (P-value is small).
  • Practical Significance: The effect is large enough to matter in the real world.
  • With a huge sample size, even a tiny, meaningless difference can be “statistically significant.”

---

Practice Questions

Problem 1

English:
Customers arrive at a coffee shop according to a Poisson process with a rate of $\lambda = 10$ customers per hour.
(1) What is the probability that the time between two consecutive customer arrivals is more than 15 minutes?
(2) What is the probability that exactly 12 customers arrive in a specific 1-hour interval?
(3) The shop defines an hour as a “Peak Hour” if more than 12 customers arrive during that hour. Suppose the shop is open for 8 hours a day. Let $X$ be the number of “Peak Hours” in a day. Assuming customer arrivals in different hours are independent, find the probability that there are exactly 3 “Peak Hours” in a day. (You may express the probability $p$ of a single hour being a Peak Hour in terms of a summation).

中文：
顾客按泊松过程到达一家咖啡店，到达速率为 $\lambda = 10$ 人/小时。
(1) 两个连续顾客到达的时间间隔超过 15 分钟的概率是多少？
(2) 在特定的 1 小时内，恰好有 12 位顾客到达的概率是多少？
(3) 如果某小时内到达的顾客超过 12 人，该店将这小时定义为“高峰小时”。假设该店每天营业 8 小时。令 $X$ 为一天中“高峰小时”的数量。假设不同小时的顾客到达情况是独立的，求一天中恰好有 3 个“高峰小时”的概率。（你可以用求和公式表示单个小时为高峰小时的概率 $p$）。

(1) 考察指数分布 (Exponential Distribution)

设 $T$ 为两个连续顾客到达的时间间隔（以小时为单位）。
根据泊松过程的性质，$T$ 服从参数为 $\lambda = 10$ 的指数分布，即 $T \sim \text{Exp}(10)$。
其概率密度函数（PDF）为 $f(t) = 10e^{-10t}, t \ge 0$。累计分布函数（CDF）为 $F(t) = 1 - e^{-10t}$。
15 分钟等于 $0.25$ 小时。我们要求的是 $P(T > 0.25)$：

$$P(T > 0.25) = e^{-\lambda t} = e^{-10 \times 0.25} = e^{-2.5} \approx 0.082$$

(2) 考察泊松分布 (Poisson Distribution)

设 $N$ 为 1 小时内到达的顾客数量。
$N$ 服从参数为 $\lambda = 10$ 的泊松分布，即 $N \sim \text{Poisson}(10)$。
泊松分布的概率质量函数为 $P(N=k) = \frac{e^{-\lambda}\lambda^k}{k!}$。
我们要求的是 $P(N=12)$：

$$P(N=12) = \frac{e^{-10} \cdot 10^{12}}{12!} \approx 0.0948$$

(3) 考察二项分布 (Binomial Distribution)

首先，我们需要计算任意一个小时是“高峰小时”的概率，记为 $p$。
根据题目，“高峰小时”意味着 $N > 12$。利用第 (2) 问中的泊松分布模型：

$$p = P(N > 12) = 1 - P(N \le 12) = 1 - \sum_{k=0}^{12} \frac{e^{-10} \cdot 10^k}{k!}$$

接着，考虑一天营业的 8 个小时。这可以看作是 $n=8$ 次独立的伯努利试验，每次试验成功的概率为 $p$。
设 $X$ 为一天中“高峰小时”的数量，则 $X$ 服从二项分布 $X \sim B(8, p)$。
我们需要求恰好有 3 个高峰小时的概率 $P(X=3)$：

$$P(X=3) = \binom{8}{3} p^3 (1-p)^{8-3} = \frac{8!}{3!5!} p^3 (1-p)^5 = 56 p^3 (1-p)^5$$

其中 $p$ 的值如上式定义。

---

Problem 2

English:
A stick of unit length $[0, 1]$ is broken at a random point $X$, which is uniformly distributed on $[0, 1]$. Then, a second point $Y$ is chosen uniformly from the remaining segment $[X, 1]$.
(1) Write down the conditional probability density function (PDF) $f_{Y|X}(y|x)$ and compute the conditional expectation $E(Y|X=x)$.
(2) Using the Law of Total Expectation, compute the expected value of $Y$, i.e., $E(Y)$.
(3) Compute the expected value of the product $XY$, i.e., $E(XY)$.

中文：
一根长度为 1 的木棍 $[0, 1]$ 在随机点 $X$ 处折断，$X$ 服从 $[0, 1]$ 上的均匀分布。然后，在剩余的线段 $[X, 1]$ 上均匀选取第二个点 $Y$。
(1) 写出条件概率密度函数 (PDF) $f_{Y|X}(y|x)$ 并计算条件期望 $E(Y|X=x)$。
(2) 利用全期望公式，计算 $Y$ 的期望值 $E(Y)$。
(3) 计算乘积 $XY$ 的期望值 $E(XY)$。

(1) 条件 PDF 与条件期望

根据题意，给定 $X=x$，随机变量 $Y$ 服从区间 $[x, 1]$ 上的均匀分布。
因此，条件 PDF 为：

$$f_{Y|X}(y|x) = \begin{cases} \frac{1}{1-x}, & x < y < 1 \\ 0, & \text{otherwise} \end{cases}$$

均匀分布 $U[a, b]$ 的期望是区间的中点 $\frac{a+b}{2}$。对于 $U[x, 1]$：

$$E(Y|X=x) = \frac{x + 1}{2}$$

(2) 利用全期望公式求 $E(Y)$

已知 $X \sim U[0, 1]$，其 PDF 为 $f_X(x) = 1, 0 \le x \le 1$。
根据全期望公式（Law of Total Expectation），$E(Y) = E[E(Y|X)]$。
我们将 $X$ 视为随机变量代入 (1) 中的结果：

$$E(Y) = E\left( \frac{X + 1}{2} \right) = \frac{1}{2}E(X) + \frac{1}{2}$$

由于 $X \sim U[0, 1]$，可知 $E(X) = \frac{1}{2}$。因此：

$$E(Y) = \frac{1}{2} \cdot \frac{1}{2} + \frac{1}{2} = \frac{1}{4} + \frac{2}{4} = \frac{3}{4}$$

(3) 求 $E(XY)$

我们需要计算 $E(XY)$。利用全期望公式 conditioning on $X$：

$$E(XY) = E[E(XY|X)]$$

在条件期望内部，给定 $X$ 时，$X$ 可以看作常数提取出来：

$$E(XY|X) = X \cdot E(Y|X) = X \cdot \frac{X + 1}{2} = \frac{X^2 + X}{2}$$

现在对外层求期望：

$$E(XY) = E\left( \frac{X^2 + X}{2} \right) = \frac{1}{2} \left[ E(X^2) + E(X) \right]$$

我们需要计算 $E(X^2)$。对于 $U[0, 1]$ 分布：

$$E(X^2) = \int_{0}^{1} x^2 \cdot 1 \, dx = \left[ \frac{x^3}{3} \right]_0^1 = \frac{1}{3}$$

（或者利用方差公式：$E(X^2) = \text{Var}(X) + [E(X)]^2 = \frac{1}{12} + (\frac{1}{2})^2 = \frac{1}{3}$）

代入数值：

$$E(XY) = \frac{1}{2} \left( \frac{1}{3} + \frac{1}{2} \right) = \frac{1}{2} \left( \frac{2}{6} + \frac{3}{6} \right) = \frac{1}{2} \cdot \frac{5}{6} = \frac{5}{12}$$

---

Problem 3

English:
Let the random vector $(X, Y)$ follow a Bivariate Normal distribution with parameters: mean $\mu_X = 10, \mu_Y = 5$, standard deviation $\sigma_X = 3, \sigma_Y = 4$, and the correlation coefficient between $X$ and $Y$ is $\rho = \frac{1}{2}$.
Define two new random variables: $U = X - Y$ and $V = X + cY$, where $c$ is a real constant.
(1) Calculate the covariance $\text{Cov}(X, Y)$ and the variance $\text{Var}(U)$.
(2) Find the value of the constant $c$ such that $U$ and $V$ are independent. Explain the theoretical basis for your calculation.
(3) Using the value of $c$ found in (2), calculate the correlation coefficient between $X$ and $V$, i.e., $\rho_{XV}$.

中文：
设随机向量 $(X, Y)$ 服从二元正态分布，参数如下：均值 $\mu_X = 10, \mu_Y = 5$，标准差 $\sigma_X = 3, \sigma_Y = 4$，且 $X$ 与 $Y$ 之间的相关系数为 $\rho = \frac{1}{2}$。
定义两个新的随机变量：$U = X - Y$ 和 $V = X + cY$，其中 $c$ 为实常数。
(1) 计算协方差 $\text{Cov}(X, Y)$ 以及方差 $\text{Var}(U)$。
(2) 求常数 $c$ 的值，使得 $U$ 和 $V$ 相互独立。请说明你计算的理论依据。
(3) 利用 (2) 中求出的 $c$ 值，计算 $X$ 与 $V$ 之间的相关系数 $\rho_{XV}$。

(1) 计算协方差与方差

首先利用相关系数定义求协方差：

$$\text{Cov}(X, Y) = \rho \cdot \sigma_X \cdot \sigma_Y = \frac{1}{2} \cdot 3 \cdot 4 = 6$$

已知 $\text{Var}(X) = 3^2 = 9$，$\text{Var}(Y) = 4^2 = 16$。
根据方差的运算性质：

$$\begin{aligned} \text{Var}(U) &= \text{Var}(X - Y) \\ &= \text{Var}(X) + \text{Var}(Y) - 2\text{Cov}(X, Y) \\ &= 9 + 16 - 2(6) \\ &= 13 \end{aligned}$$

(2) 利用正态分布性质确定常数 $c$

理论依据： 由于 $(X, Y)$ 服从二元正态分布，$X$ 和 $Y$ 的线性组合 $U$ 和 $V$ 也会服从联合正态分布。在正态分布中，两个随机变量不相关（协方差为 0）等价于相互独立。因此，要使 $U$ 和 $V$ 独立，只需令 $\text{Cov}(U, V) = 0$。

计算 $\text{Cov}(U, V)$ 并令其为 0：

$$\begin{aligned} \text{Cov}(U, V) &= \text{Cov}(X - Y, X + cY) \\ &= \text{Cov}(X, X) + c\text{Cov}(X, Y) - \text{Cov}(Y, X) - c\text{Cov}(Y, Y) \\ &= \text{Var}(X) + (c-1)\text{Cov}(X, Y) - c\text{Var}(Y) \\ &= 9 + (c-1)(6) - c(16) \\ &= 9 + 6c - 6 - 16c \\ &= 3 - 10c \end{aligned}$$

令 $3 - 10c = 0$，解得：

$$c = \frac{3}{10} = 0.3$$

(3) 计算相关系数 $\rho_{XV}$

首先需要计算 $\text{Var}(V)$ 和 $\text{Cov}(X, V)$。
已知 $c = 0.3$：

$$\begin{aligned} \text{Var}(V) &= \text{Var}(X + 0.3Y) \\ &= \text{Var}(X) + 0.3^2\text{Var}(Y) + 2(0.3)\text{Cov}(X, Y) \\ &= 9 + 0.09(16) + 0.6(6) \\ &= 9 + 1.44 + 3.6 \\ &= 14.04 \end{aligned}$$

计算 $X$ 与 $V$ 的协方差：

$$\begin{aligned} \text{Cov}(X, V) &= \text{Cov}(X, X + 0.3Y) \\ &= \text{Var}(X) + 0.3\text{Cov}(X, Y) \\ &= 9 + 0.3(6) \\ &= 9 + 1.8 \\ &= 10.8 \end{aligned}$$

计算相关系数：

$$\rho_{XV} = \frac{\text{Cov}(X, V)}{\sqrt{\text{Var}(X)}\sqrt{\text{Var}(V)}} = \frac{10.8}{\sqrt{9} \cdot \sqrt{14.04}} = \frac{10.8}{3 \cdot \sqrt{14.04}} = \frac{3.6}{\sqrt{14.04}} \approx 0.96$$

---

Problem 4

English:
Suppose continuous random variables $X$ and $Y$ have the joint probability density function (PDF):

$$f(x, y) = \begin{cases} x + y, & 0 \le x \le 1, \ 0 \le y \le 1 \\ 0, & \text{otherwise} \end{cases}$$

(1) Let $Z = X - Y$. Derive the probability density function (PDF) of $Z$.
(2) Let $W = \min(X, Y)$. Derive the probability density function (PDF) of $W$.

中文：
设连续随机变量 $X$ 和 $Y$ 具有如下联合概率密度函数 (PDF)：

$$f(x, y) = \begin{cases} x + y, & 0 \le x \le 1, \ 0 \le y \le 1 \\ 0, & \text{其他} \end{cases}$$

(1) 令 $Z = X - Y$，求 $Z$ 的概率密度函数 (PDF)。
(2) 令 $W = \min(X, Y)$，求 $W$ 的概率密度函数 (PDF)。

(1) 求 $Z = X - Y$ 的 PDF

$Z$ 的取值范围是 $[-1, 1]$。我们需要计算分布函数 $F_Z(z) = P(X - Y \le z)$，然后求导。
积分区域为单位正方形 $[0,1] \times [0,1]$ 被直线 $y = x - z$ 切割的部分。

• 情形 1： $-1 \le z < 0$
  不等式 $X - Y \le z \Rightarrow Y \ge X - z$ (即 $Y \ge X + |z|$)。
  积分区域为正方形左上角的三角形，顶点为 $(0, -z), (0, 1), (1+z, 1)$。

  $$\begin{aligned} F_Z(z) &= \int_0^{1+z} \int_{x-z}^1 (x + y) \, dy \, dx \\ &= \frac{(1+z)^2 (z+2)}{2} = \frac{z^3 + 4z^2 + 5z + 2}{2} \end{aligned}$$

  求导得：$f_Z(z) = \frac{3z^2 + 8z + 5}{2} = \frac{(z+1)(3z+5)}{2}$。

• 情形 2： $0 \le z \le 1$
  利用补集更简单：$P(X - Y \le z) = 1 - P(X - Y > z) = 1 - P(Y < X - z)$。
  $Y < X - z$ 对应正方形右下角的三角形区域，顶点为 $(z, 0), (1, 0), (1, 1-z)$。

  $$\begin{aligned} P(Y < X - z) &= \int_z^1 \int_0^{x-z} (x + y) \, dy \, dx \\ &= \frac{(1-z)^2 (z+2)}{2} \end{aligned}$$

  所以 $F_Z(z) = 1 - \frac{(1-z)^2 (z+2)}{2}$。
  求导得：$f_Z(z) = \frac{3z^2 - 8z + 5}{2} = \frac{(1-z)(5-3z)}{2}$。

• 综上所述：

  $$f_Z(z) = \begin{cases} \frac{3z^2 + 8z + 5}{2}, & -1 \le z < 0 \\ \frac{3z^2 - 8z + 5}{2}, & 0 \le z \le 1 \\ 0, & \text{otherwise} \end{cases}$$

  (注：也可以合并写为 $f_Z(z) = \frac{3z^2 - 8|z| + 5}{2}$，因为 $x+y$ 关于 $y=x$ 对称，分布也是对称的)

(2) 求 $W = \min(X, Y)$ 的 PDF

$W$ 的取值范围是 $[0, 1]$。
利用生存函数（Survival Function）计算 $P(W > w)$ 会更简便，因为 $\min(X, Y) > w$ 等价于 $X > w$ 且 $Y > w$。

$$\begin{aligned} P(W > w) &= P(X > w, Y > w) \\ &= \int_w^1 \int_w^1 (x + y) \, dx \, dy \end{aligned}$$

计算该积分：

$$\begin{aligned} \int_w^1 \int_w^1 (x + y) \, dx \, dy &= \int_w^1 \left[ \frac{1}{2}x^2 + yx \right]_w^1 \, dy \\ &= \int_w^1 \left( (\frac{1}{2} + y) - (\frac{w^2}{2} + wy) \right) \, dy \\ &= \left[ \frac{1}{2}y + \frac{1}{2}y^2 - \frac{w^2}{2}y - \frac{w}{2}y^2 \right]_w^1 \\ &= 1 - w - w^2 + w^3 \end{aligned}$$

因此，CDF 为：

$$F_W(w) = 1 - P(W > w) = 1 - (1 - w - w^2 + w^3) = w + w^2 - w^3, \quad 0 \le w \le 1$$

求导得到 PDF：

$$f_W(w) = 1 + 2w - 3w^2, \quad 0 \le w \le 1$$

---

Problem 5

English:
Let $X$ and $Y$ be the midterm and final exam scores of a student, respectively. Assume that the joint distribution of scores follows a bivariate normal distribution with the following parameters:
Mean of midterm: $\mu_X = 70$, Standard deviation of midterm: $\sigma_X = 10$.
Mean of final: $\mu_Y = 75$, Standard deviation of final: $\sigma_Y = 12$.
The correlation coefficient between the two scores is $\rho = 0.8$.
(1) Suppose a student scores 80 on the midterm (i.e., $X = 80$). Determine the conditional probability density function (PDF) of his final score $Y$. Specifically, calculate the conditional mean $E(Y|X=80)$ and conditional variance $\text{Var}(Y|X=80)$.
(2) Given that the student scored 80 on the midterm, what is the probability that his final score is greater than 85?

中文：
设 $X$ 和 $Y$ 分别为一个学生的期中考试成绩和期末考试成绩。假设成绩的联合分布服从二元正态分布，参数如下：
期中均值：$\mu_X = 70$，期中标准差：$\sigma_X = 10$。
期末均值：$\mu_Y = 75$，期末标准差：$\sigma_Y = 12$。
两次成绩之间的相关系数为 $\rho = 0.8$。
(1) 假设某学生期中考了 80 分（即 $X = 80$）。确定他期末成绩 $Y$ 的条件概率密度函数 (PDF)。具体地，计算条件均值 $E(Y|X=80)$ 和条件方差 $\text{Var}(Y|X=80)$。
(2) 在该生期中考了 80 分的条件下，他期末成绩大于 85 分的概率是多少？

(1) 确定条件分布的参数

对于二元正态分布，给定 $X=x$ 时，$Y$ 的条件分布仍然是正态分布 $N(\mu_{Y|x}, \sigma_{Y|x}^2)$。
公式如下：

• 条件均值： $\mu_{Y|x} = \mu_Y + \rho \frac{\sigma_Y}{\sigma_X} (x - \mu_X)$
• 条件方差： $\sigma_{Y|x}^2 = \sigma_Y^2 (1 - \rho^2)$

代入题目给定的数值：$x = 80, \mu_X = 70, \sigma_X = 10, \mu_Y = 75, \sigma_Y = 12, \rho = 0.8$。

$$\begin{aligned} E(Y|X=80) &= 75 + 0.8 \times \frac{12}{10} \times (80 - 70) \\ &= 75 + 0.8 \times 1.2 \times 10 \\ &= 75 + 9.6 \\ &= 84.6 \end{aligned}$$

$$\begin{aligned} \text{Var}(Y|X=80) &= 12^2 \times (1 - 0.8^2) \\ &= 144 \times (1 - 0.64) \\ &= 144 \times 0.36 \\ &= 51.84 \end{aligned}$$

因此，条件 PDF $f_{Y|X}(y|80)$ 是均值为 $84.6$，方差为 $51.84$ 的正态分布密度函数：

$$Y|X=80 \sim N(84.6, 51.84)$$

(2) 计算条件概率

我们需要计算 $P(Y > 85 | X = 80)$。
首先对 $Y$ 进行标准化。利用上面求得的条件标准差 $\sigma_{Y|x} = \sqrt{51.84} = 7.2$。

$$\begin{aligned} P(Y > 85 | X = 80) &= P\left( \frac{Y - \mu_{Y|x}}{\sigma_{Y|x}} > \frac{85 - 84.6}{7.2} \right) \\ &= P\left( Z > \frac{0.4}{7.2} \right) \\ &= P\left( Z > \frac{1}{18} \right) \\ &\approx P(Z > 0.056) \end{aligned}$$

利用标准正态分布的对称性：

$$P(Z > 0.056) = 1 - \Phi(0.056)$$

---

Problem 6

English:
Let $X_1, X_2, \dots, X_n$ be a simple random sample from a population with mean $E(X) = \mu$ and finite variance $Var(X) = \sigma^2$. Consider the following estimator for $\mu$:

$$\hat{\mu}_n = \frac{2}{n^2} \sum_{k=1}^n k X_k$$

(1) Is $\hat{\mu}_n$ an unbiased estimator of $\mu$? If not, calculate the bias $B(\hat{\mu}_n) = E(\hat{\mu}_n) - \mu$.
(2) Prove that $\hat{\mu}_n$ is a consistent estimator of $\mu$.
(Note that $\sum_{k=1}^n k = \frac{n(n+1)}{2}$ and $\sum_{k=1}^n k^2 = \frac{n(n+1)(2n+1)}{6}$.)

中文：
设 $X_1, X_2, \dots, X_n$ 是来自总体均值为 $E(X) = \mu$、方差为 $Var(X) = \sigma^2$（有限）的简单随机样本。考虑 $\mu$ 的如下估计量：

$$\hat{\mu}_n = \frac{2}{n^2} \sum_{k=1}^n k X_k$$

(1) $\hat{\mu}_n$ 是 $\mu$ 的无偏估计量吗？如果不是，请计算偏差 $B(\hat{\mu}_n) = E(\hat{\mu}_n) - \mu$。
(2) 证明 $\hat{\mu}_n$ 是 $\mu$ 的相合估计量（一致估计量）。
（注：$\sum_{k=1}^n k = \frac{n(n+1)}{2}$ 以及 $\sum_{k=1}^n k^2 = \frac{n(n+1)(2n+1)}{6}$。）

(1) 检查无偏性 (Unbiasedness)

计算估计量的期望：

$$\begin{aligned} E(\hat{\mu}_n) &= E\left( \frac{2}{n^2} \sum_{k=1}^n k X_k \right) \\ &= \frac{2}{n^2} \sum_{k=1}^n k E(X_k) \quad (\text{期望的线性性质}) \\ &= \frac{2}{n^2} \cdot \mu \sum_{k=1}^n k \end{aligned}$$

利用求和公式 $\sum_{k=1}^n k = \frac{n(n+1)}{2}$：

$$E(\hat{\mu}_n) = \frac{2\mu}{n^2} \cdot \frac{n(n+1)}{2} = \frac{n(n+1)}{n^2} \mu = \frac{n+1}{n} \mu = \left(1 + \frac{1}{n}\right)\mu$$

显然 $E(\hat{\mu}_n) \neq \mu$（除非 $\mu=0$），所以 $\hat{\mu}_n$ 不是无偏估计量。
偏差为：

$$B(\hat{\mu}_n) = E(\hat{\mu}_n) - \mu = \left(1 + \frac{1}{n}\right)\mu - \mu = \frac{\mu}{n}$$

(2) 证明相合性 (Consistency)

根据均方误差（MSE）准则，若满足以下两个条件，则估计量是相合的：

1. 渐近无偏性：$\lim_{n \to \infty} E(\hat{\mu}_n) = \mu$
2. 方差趋于零：$\lim_{n \to \infty} \text{Var}(\hat{\mu}_n) = 0$

第一步：验证渐近无偏性
由 (1) 可知：

$$\lim_{n \to \infty} E(\hat{\mu}_n) = \lim_{n \to \infty} \left(1 + \frac{1}{n}\right)\mu = 1 \cdot \mu = \mu$$

条件 1 满足。

第二步：验证方差趋于零
由于 $X_1, \dots, X_n$ 是简单随机样本，相互独立，故：

$$\begin{aligned} \text{Var}(\hat{\mu}_n) &= \text{Var}\left( \frac{2}{n^2} \sum_{k=1}^n k X_k \right) \\ &= \left( \frac{2}{n^2} \right)^2 \sum_{k=1}^n k^2 \text{Var}(X_k) \quad (\text{独立的性质}) \\ &= \frac{4}{n^4} \cdot \sigma^2 \sum_{k=1}^n k^2 \end{aligned}$$

利用求和公式 $\sum_{k=1}^n k^2 = \frac{n(n+1)(2n+1)}{6}$：

$$\begin{aligned} \text{Var}(\hat{\mu}_n) &= \frac{4\sigma^2}{n^4} \cdot \frac{n(n+1)(2n+1)}{6} \\ &= \frac{2\sigma^2}{3} \cdot \frac{n(2n^2 + 3n + 1)}{n^4} \\ &= \frac{2\sigma^2}{3} \cdot \frac{2n^3 + 3n^2 + n}{n^4} \end{aligned}$$

当 $n \to \infty$ 时，分母阶数（$n^4$）高于分子阶数（$n^3$），因此：

$$\lim_{n \to \infty} \text{Var}(\hat{\mu}_n) = 0$$

条件 2 满足。

结论：
因为 $\hat{\mu}_n$ 满足渐近无偏且方差收敛于 0，所以 $\hat{\mu}_n$ 是 $\mu$ 的相合估计量。

---

Problem 7

English:
Consider a random variable $X$ that follows an Exponential distribution with an unknown scale parameter $\theta$ ($\theta > 0$). The probability density function (PDF) is given by:

$$f(x; \theta) = \begin{cases} \frac{1}{\theta} e^{-x/\theta}, & x > 0 \\ 0, & \text{otherwise} \end{cases}$$

(Note: In this parameterization, $E(X) = \theta$ and $\text{Var}(X) = \theta^2$).

A random sample of size $n=8$ is drawn from this population:

$$2, \ 4, \ 6, \ 8, \ 10, \ 12, \ 16, \ 22$$

(1) Compute the sample mean $\bar{x}$, sample variance $S^2$, sample lower quartile $Q_1$, sample upper quartile $Q_3$, and the sample interquartile range (IQR).
(2) Derive the Method of Moments Estimator (MME) $\hat{\theta}_{MME}$ for the parameter $\theta$. Calculate its value based on the sample data.
(3) Derive the Maximum Likelihood Estimator (MLE) $\hat{\theta}_{MLE}$ for the parameter $\theta$ using calculus (derivation). Calculate its value based on the sample data.
(4) Calculate the estimated Standard Error (SE) of the estimator $\hat{\theta}$ obtained in (3).

中文：
考虑一个服从指数分布的随机变量 $X$，其具有未知的尺度参数 $\theta$ ($\theta > 0$)。概率密度函数 (PDF) 如下：

$$f(x; \theta) = \begin{cases} \frac{1}{\theta} e^{-x/\theta}, & x > 0 \\ 0, & \text{其他} \end{cases}$$

（注：在该参数化下，总体均值 $E(X) = \theta$，总体方差 $\text{Var}(X) = \theta^2$）。

现从该总体中抽取了一个容量为 $n=8$ 的随机样本：

$$2, \ 4, \ 6, \ 8, \ 10, \ 12, \ 16, \ 22$$

(1) 计算样本均值 $\bar{x}$、样本方差 $S^2$、样本下四分位数 $Q_1$、样本上四分位数 $Q_3$ 以及样本四分位距 (IQR)。
(2) 推导参数 $\theta$ 的矩估计量 (MME) $\hat{\theta}_{MME}$，并根据样本数据计算其值。
(3) 利用微积分（求导）推导参数 $\theta$ 的极大似然估计量 (MLE) $\hat{\theta}_{MLE}$，并根据样本数据计算其值。
(4) 计算 (3) 中得到的估计量 $\hat{\theta}$ 的估计标准误 (Standard Error)。

(1) 描述性统计

首先整理数据（已排序）：$2, 4, 6, 8, 10, 12, 16, 22$。样本量 $n=8$。

• 样本均值：

  $$\bar{x} = \frac{2+4+6+8+10+12+16+22}{8} = \frac{80}{8} = 10$$

• 样本方差：
  计算 $\sum x_i^2 = 4 + 16 + 36 + 64 + 100 + 144 + 256 + 484 = 1104$。

  $$S^2 = \frac{1}{n-1} \left( \sum x_i^2 - n\bar{x}^2 \right) = \frac{1}{7} (1104 - 8 \times 100) = \frac{304}{7} \approx 43.43$$

• 四分位数与 IQR：
  数据分为下半区 $\{2, 4, 6, 8\}$ 和上半区 $\{10, 12, 16, 22\}$。
  $Q_1$ 为下半区中位数：$\frac{4+6}{2} = 5$。
  $Q_3$ 为上半区中位数：$\frac{12+16}{2} = 14$。

  $$IQR = Q_3 - Q_1 = 14 - 5 = 9$$

(2) 矩估计 (MME)

总体一阶矩（均值）为：

$$E(X) = \int_0^\infty x \cdot \frac{1}{\theta} e^{-x/\theta} dx = \theta$$

令总体矩等于样本矩：$E(X) = \bar{X}$。
即 $\theta = \bar{X}$。
所以矩估计量为：

$$\hat{\theta}_{MME} = \bar{X}$$

根据样本计算：

$$\hat{\theta}_{MME} = 10$$

(3) 极大似然估计 (MLE)

似然函数 $L(\theta)$ 为样本联合概率密度：

$$L(\theta) = \prod_{i=1}^n f(x_i; \theta) = \prod_{i=1}^n \frac{1}{\theta} e^{-x_i/\theta} = \frac{1}{\theta^n} e^{-\frac{1}{\theta}\sum_{i=1}^n x_i}$$

取对数似然函数 $l(\theta) = \ln L(\theta)$：

$$l(\theta) = -n \ln \theta - \frac{1}{\theta} \sum_{i=1}^n x_i$$

对 $\theta$ 求导并令其为 0：

$$\frac{d l(\theta)}{d \theta} = -\frac{n}{\theta} + \frac{1}{\theta^2} \sum_{i=1}^n x_i = 0$$

$$\frac{n}{\theta} = \frac{\sum x_i}{\theta^2} \implies n\theta = \sum x_i$$

解得极大似然估计量：

$$\hat{\theta}_{MLE} = \frac{1}{n} \sum_{i=1}^n X_i = \bar{X}$$

根据样本计算：

$$\hat{\theta}_{MLE} = 10$$

(4) 估计标准误 (Standard Error)

我们在 (3) 中得到的估计量是样本均值，即 $\hat{\theta} = \bar{X}$。
因此，该参数估计量的标准误即为样本均值的标准误。
公式为 $SE(\hat{\theta}) = SE(\bar{X}) = \frac{S}{\sqrt{n}}$。

利用 (1) 中计算出的样本方差 $S^2 = \frac{304}{7} \approx 43.43$。
样本标准差 $S = \sqrt{43.43} \approx 6.59$。

$$SE(\hat{\theta}) = \frac{6.59}{\sqrt{8}} = \frac{6.59}{2.828} \approx 2.33$$

---

Problem 8

English:
A manufacturer claims that the mean lifetime of a specific type of battery is at least 1000 hours. A quality control engineer suspects that the mean lifetime is actually less than 1000 hours. To verify this, a simple random sample of $n = 36$ batteries is selected from the production line. Assume that the battery lifetimes follow a normal distribution with a known standard deviation $\sigma = 60$ hours. The significance level is set at $\alpha = 0.05$.

The testing problem is: $H_0: \mu = 1000$ vs. $H_1: \mu < 1000$.

(1) Construct the rejection region for the sample mean $\bar{X}$ at significance level $\alpha = 0.05$. What is the probability of Type I error for this test?
(2) Suppose the observed sample mean is $\bar{x} = 982$ hours. Calculate the $p$-value of this observed statistic. Based on the $p$-value, should we reject the null hypothesis?
(3) Calculate the probability of making a Type II error ($\beta$) if the true population mean is actually $\mu_1 = 975$ hours.
(4) Construct a 95% two-sided confidence interval for the population mean $\mu$ based on the observed sample mean $\bar{x} = 982$. Does this interval contain the claimed value 1000? Briefly discuss whether the conclusion derived from the 95% two-sided confidence interval is consistent with the conclusion from the one-sided hypothesis test in (2) at $\alpha = 0.05$.
(5) If the engineer wants to achieve a statistical power of at least 0.90 when the true mean is $\mu_1 = 975$, what is the minimum sample size $n$ required? (Assume $\alpha$ remains 0.05).

中文：
某制造商声称某种电池的平均寿命至少为 1000 小时。质量控制工程师怀疑平均寿命实际上少于 1000 小时。为了验证这一点，从生产线上随机抽取了 $n = 36$ 个电池作为简单随机样本。假设电池寿命服从正态分布，且标准差已知为 $\sigma = 60$ 小时。显著性水平设定为 $\alpha = 0.05$。

检验问题为：$H_0: \mu = 1000$ vs. $H_1: \mu < 1000$。

(1) 在显著性水平 $\alpha = 0.05$ 下，构建基于样本均值 $\bar{X}$ 的拒绝域。该检验犯第一类错误的概率是多少？
(2) 假设观测到的样本均值为 $\bar{x} = 982$ 小时。计算该观测统计量的 $p$ 值 ($p$-value)。根据 $p$ 值，我们是否应该拒绝原假设？
(3) 如果真实的总体均值实际上是 $\mu_1 = 975$ 小时，计算犯第二类错误的概率 ($\beta$)。
(4) 基于观测均值 $\bar{x} = 982$，构建总体均值 $\mu$ 的 95% 双侧置信区间。该区间是否包含声称值 1000？请简要讨论基于 95% 双侧置信区间的结论与 (2) 中 $\alpha = 0.05$ 的单边假设检验结论是否一致。
(5) 如果工程师希望在真实均值为 $\mu_1 = 975$ 时，检验的统计功效 (Power) 至少达到 0.90，则需要的最小样本量 $n$ 是多少？（假设 $\alpha$ 保持 0.05）。

(1) 拒绝域与第一类错误

这是左单边检验。
标准误 $SE = \frac{\sigma}{\sqrt{n}} = \frac{60}{\sqrt{36}} = 10$。
在 $\alpha = 0.05$ 下，标准正态分布的左侧临界值 $z_{0.05} = 1.645$（即 $P(Z < -1.645) = 0.05$）。
拒绝域的形式为：

$$\frac{\bar{X} - \mu_0}{\sigma/\sqrt{n}} < -1.645 \implies \bar{X} < \mu_0 - 1.645 \left( \frac{\sigma}{\sqrt{n}} \right)$$

代入数值：

$$\bar{X} < 1000 - 1.645 \times 10 = 983.55$$

所以，拒绝域为 $\{ \bar{X} : \bar{X} < 983.55 \}$。
犯第一类错误的概率即为显著性水平 0.05。

(2) P-value 计算与决策

观测值 $\bar{x} = 982$。
计算检验统计量 $Z_{obs}$：

$$Z_{obs} = \frac{982 - 1000}{10} = -1.8$$

对于左侧检验，$p$-value 为：

$$p = P(Z < -1.8) = \Phi(-1.8) = 1 - \Phi(1.8) \approx 1 - 0.9641 = 0.0359$$

决策：
因为 $p\text{-value} = 0.0359 < 0.05$（或者因为 $\bar{x} = 982$ 落入了拒绝域 $\bar{X} < 983.55$），我们拒绝原假设 $H_0$。有显著证据表明电池平均寿命少于 1000 小时。

(3) 第二类错误率 $\beta$ 计算

当真实均值 $\mu_1 = 975$ 时，犯第二类错误意味着我们未能拒绝 $H_0$。
未能拒绝的区域是 $\bar{X} \ge 983.55$。
标准化：

$$Z = \frac{\bar{X} - \mu_1}{\sigma/\sqrt{n}} = \frac{983.55 - 975}{10} = \frac{8.55}{10} = 0.855$$

$$\beta = P(Z \ge 0.855) = 1 - \Phi(0.855)$$

查表（取 $\Phi(0.86) \approx 0.8051$）：

$$\beta \approx 1 - 0.8051 = 0.1949$$

(注：若使用计算器精确计算，$\beta \approx 0.1963$)

(4) 95% 双侧置信区间与讨论

95% 双侧置信区间公式：$\bar{x} \pm 1.96 \frac{\sigma}{\sqrt{n}}$。

$$CI = [982 - 1.96 \times 10, \ 982 + 1.96 \times 10] = [962.4, \ 1001.6]$$

该区间包含 1000。
讨论：
基于 95% 双侧 CI，由于 1000 在区间内，结论是“不拒绝 $H_0$”。
但在 (2) 中，单边检验（$\alpha=0.05$）结论是“拒绝 $H_0$”。
原因：单边检验将 5% 的错误率全部放在左侧，对左侧偏移更敏感；而 95% 双侧 CI 对应的是 $\alpha=0.05$ 的双侧检验，左侧只有 2.5% 的拒绝空间。$\bar{x}=982$ 位于分布的左侧 3.6% 处（$p=0.0359$），这小于 5% 但大于 2.5%。因此单边检验拒绝，双侧检验（CI）不拒绝。若只关心“是否小于”，应采信单边检验结果。

(5) 基于 Power 的样本量计算

给定条件：

1. Type I error $\alpha = 0.05 \implies z_\alpha = 1.645$（单侧）
2. Power $\ge 0.90 \implies \beta \le 0.10 \implies z_\beta = z_{0.10} = 1.282$（或者查表取 1.28）

单边检验样本量公式：

$$n = \frac{(z_\alpha + z_\beta)^2 \sigma^2}{(\mu_0 - \mu_1)^2}$$

代入数值：

$$n = \frac{(1.645 + 1.282)^2 \times 60^2}{(1000 - 975)^2} = \frac{(2.927)^2 \times 3600}{625}$$

$$n = \frac{8.567 \times 3600}{625} \approx 49.34$$

样本量必须为整数，故向上取整：

$$n = 50$$

所需的最小样本量为 50。

---

Problem 9

English:
An electronic component manufacturer has two production lines, Line A and Line B, producing the same type of chip. The quality control department wants to determine if there is a significant difference in the defective rates between the two lines.
They collected independent random samples from both lines:

• From Line A: $n_A = 200$ chips were tested, and $24$ were found to be defective.
• From Line B: $n_B = 300$ chips were tested, and $54$ were found to be defective.

Let $p_A$ and $p_B$ denote the true defective rates of Line A and Line B, respectively. Use a significance level of $\alpha = 0.05$.

(1) Construct the testing problem $H_0: p_A = p_B$ vs. $H_1: p_A \neq p_B$. Construct a rejection region based on the test statistic at $\alpha = 0.05$. Then, calculate the observed value of the statistic and decide whether to reject the null hypothesis based on whether the value falls into the rejection region.
(2) Alternatively, compute the $p$-value of the observed test statistic. Comparing the $p$-value with the significance level $\alpha$, is the conclusion consistent with (1)?
(3) Construct a 95% confidence interval for the difference in defective rates $p_B - p_A$. Based on whether the interval contains 0, can we conclude that the defective rates of the two lines are significantly different? Briefly discuss the consistency of this result with the hypothesis test in (1).
(4) Suppose the manager plans a new study to confirm the difference. If the true defective rates are indeed $p_A = 0.12$ and $p_B = 0.18$ (as observed in the samples above), what is the minimum sample size $n$ required for each line (assuming equal sample sizes $n_A = n_B = n$) to ensure that the Type II error rate ($\beta$) does not exceed 0.10 at the $\alpha = 0.05$ significance level?

中文：
一家电子元件制造商有两条生产线（生产线 A 和生产线 B）生产同类型的芯片。质量控制部门希望确定这两条生产线的次品率是否存在显著差异。
他们分别从两条生产线收集了独立的随机样本：

• 生产线 A：测试了 $n_A = 200$ 个芯片，发现 $24$ 个次品。
• 生产线 B：测试了 $n_B = 300$ 个芯片，发现 $54$ 个次品。

令 $p_A$ 和 $p_B$ 分别表示生产线 A 和生产线 B 的真实次品率。显著性水平设定为 $\alpha = 0.05$。

(1) 构建检验问题 $H_0: p_A = p_B$ vs. $H_1: p_A \neq p_B$。在 $\alpha = 0.05$ 水平下构建基于检验统计量的拒绝域。然后，计算统计量的观测值，并根据其是否落入拒绝域来决定是否拒绝原假设。
(2) 或者，计算观测检验统计量的 $p$ 值 ($p$-value)。将 $p$ 值与显著性水平 $\alpha$ 进行比较，判断结论是否与 (1) 一致？
(3) 构建次品率之差 $p_B - p_A$ 的 95% 置信区间。基于该区间是否包含 0，我们能否得出两条生产线次品率存在显著差异的结论？简要讨论该结果与 (1) 中假设检验的一致性。
(4) 假设经理计划进行一项新的研究以确认差异。如果真实的次品率确为 $p_A = 0.12$ 和 $p_B = 0.18$（即上述样本观测值），为了确保在 $\alpha = 0.05$ 的显著性水平下，第二类错误率 ($\beta$) 不超过 0.10，每条生产线所需的最小样本量 $n$ 是多少？（假设两组样本量相等 $n_A = n_B = n$）。

(1) 拒绝域法 (Rejection Region Method)

• 假设： $H_0: p_A = p_B$ vs $H_1: p_A \neq p_B$。
• 拒绝域构造：
  双侧检验，$\alpha = 0.05$，临界值为 $z_{0.025} = 1.96$。
  拒绝域 $R = \{ Z : |Z| > 1.96 \}$。
• 计算统计量：
  样本比例：$\hat{p}_A = 0.12, \hat{p}_B = 0.18$。
  在 $H_0$ 下需使用混合样本比例：

  $$\hat{p} = \frac{24 + 54}{200 + 300} = 0.156$$

  检验统计量 $Z_{\text{obs}}$：

  $$Z_{\text{obs}} = \frac{0.12 - 0.18}{\sqrt{0.156(1-0.156)(\frac{1}{200} + \frac{1}{300})}} \approx \frac{-0.06}{0.0331} \approx -1.81$$

• 决策：
  $|Z_{\text{obs}}| = 1.81 < 1.96$，未落入拒绝域。
  结论： 不能拒绝 $H_0$，无显著证据表明次品率不同。

(2) P值法 (P-value Method)

• 计算 P 值：

  $$p\text{-value} = 2 \times P(Z < -1.81) = 2 \times (1 - \Phi(1.81))$$

  查表 $\Phi(1.81) \approx 0.9649$。

  $$p\text{-value} \approx 2 \times 0.0351 = 0.0702$$

• 决策：
  $p\text{-value} = 0.0702 > 0.05$，不能拒绝 $H_0$。结论与 (1) 一致。

(3) 置信区间法 (Confidence Interval Method)

• 计算 CI：
  使用各自的样本方差计算标准误（不使用混合比例）：

  $$SE = \sqrt{\frac{0.12(0.88)}{200} + \frac{0.18(0.82)}{300}} \approx 0.0319$$

  95% CI：

  $$(0.18 - 0.12) \pm 1.96(0.0319) = 0.06 \pm 0.0625 = [-0.0025, \ 0.1225]$$

• 结论：
  区间包含 0，说明差异可能为 0，不能拒绝 $H_0$。与假设检验结论一致。

(4) 基于 Power 的样本量计算

• 参数设定：
  $\alpha = 0.05 \Rightarrow z_{\alpha/2} = 1.96$。
  $\beta = 0.10 \Rightarrow z_{\beta} = 1.28$。
  真实比例：$p_A = 0.12, p_B = 0.18$。
  差异 $\Delta = |p_A - p_B| = 0.06$。
  因为假设 $n_A = n_B = n$，在 $H_0$ 下的平均比例 $\bar{p} = \frac{p_A + p_B}{2} = \frac{0.12+0.18}{2} = 0.15$。
• 公式：
  双样本比例检验的样本量公式（每组）：

  $$n = \frac{\left( z_{\alpha/2} \sqrt{2\bar{p}(1-\bar{p})} + z_{\beta} \sqrt{p_A(1-p_A) + p_B(1-p_B)} \right)^2}{(p_A - p_B)^2}$$

• 计算分子各项：
  1. $H_0$ 部分 (使用 $\bar{p}=0.15$)：
     $\sqrt{2 \times 0.15 \times 0.85} = \sqrt{0.255} \approx 0.5050$
     $Term_1 = 1.96 \times 0.5050 \approx 0.9898$
  2. $H_1$ 部分 (使用 $p_A, p_B$)：
     $\sqrt{0.12(0.88) + 0.18(0.82)} = \sqrt{0.1056 + 0.1476} = \sqrt{0.2532} \approx 0.5032$
     $Term_2 = 1.28 \times 0.5032 \approx 0.6441$
• 计算 $n$：

  $$n = \frac{(0.9898 + 0.6441)^2}{(0.06)^2} = \frac{(1.6339)^2}{0.0036} = \frac{2.6696}{0.0036} \approx 741.56$$

• 结果：
  向上取整，每条生产线需要至少 742 个样本。