Point Estimation: Unbiased Estimators and Estimation Methods | 点估计：无偏估计量与估计方法

📚 Point Estimation: Unbiased Estimators and Estimation Methods | 点估计：无偏估计量与估计方法

Point estimation is a cornerstone of inferential statistics. When we have a random sample from a population, we often want to estimate an unknown parameter such as the population mean μ or variance σ². A point estimator is a rule or formula that uses sample data to produce a single number as our best guess for the parameter. In this article, we will explore what makes an estimator “good”, with special attention to unbiasedness, and we will review the most common estimation methods: the method of moments and maximum likelihood estimation.

点估计是推断统计的基石。当我们从总体中获取一个随机样本时，通常需要估计未知参数，例如总体均值 μ 或方差 σ²。点估计量是利用样本数据产生一个单一数值来作为参数最佳猜测的规则或公式。在本文中，我们将探讨什么使估计量“良好”，特别关注无偏性，并回顾最常用的估计方法：矩估计法和极大似然估计法。

1. What Is a Point Estimator? | 什么是点估计量？

A point estimator is a function of the sample data, usually denoted with a “hat” over the parameter, e.g., μ̂ for μ or σ̂² for σ². Because the estimator depends on the random sample, it is itself a random variable. The value it takes for a particular sample is called the point estimate. For example, the sample mean X̄ = (X₁ + X₂ + … + Xₙ)/n is an estimator for the population mean μ. If we observe a sample with mean 7.2, then 7.2 is the point estimate.

点估计量是样本数据的函数，通常用参数上方加“帽”表示，例如 μ̂ 表示 μ，σ̂² 表示 σ²。由于估计量依赖于随机样本，它本身也是一个随机变量。对于特定样本所取的值称为点估计值。例如，样本均值 X̄ = (X₁ + X₂ + … + Xₙ)/n 是总体均值 μ 的估计量。如果我们观测到一个样本均值为 7.2，那么 7.2 就是点估计值。

Not every estimator is equally good. We need criteria to compare different estimators. The most important properties are unbiasedness, efficiency, and consistency. This article focuses on unbiasedness, while also introducing how to derive estimators.

并非每个估计量都同样好。我们需要标准来比较不同的估计量。最重要的性质包括无偏性、有效性和相合性。本文重点讨论无偏性，同时介绍如何推导估计量。

2. Unbiased Estimators: Definition | 无偏估计量：定义

An estimator θ̂ is said to be an unbiased estimator of a parameter θ if its expected value equals the true parameter value for all possible values of θ:

如果估计量 θ̂ 的期望值等于真实参数值 θ，则称 θ̂ 是参数 θ 的无偏估计量，即对 θ 的所有可能取值都有：

E(θ̂) = θ for all θ.

If E(θ̂) ≠ θ, the estimator is biased, and the bias is defined as Bias(θ̂) = E(θ̂) − θ. A positive bias means the estimator tends to overestimate the parameter; a negative bias means it tends to underestimate.

如果 E(θ̂) ≠ θ，则称该估计量是有偏的，偏差定义为 Bias(θ̂) = E(θ̂) − θ。正偏差意味着估计量倾向于高估参数；负偏差意味着倾向于低估参数。

Unbiasedness is a desirable property because, on average, the estimator hits the true parameter. However, unbiasedness alone is not enough — a good estimator also should have low variance, which we will discuss later.

无偏性是一个理想的性质，因为平均而言，估计量能够击中真实参数。然而，仅有无偏性还不够——一个好的估计量还应当具有较小的方差，这一点我们稍后讨论。

3. The Sample Mean Is Unbiased for μ | 样本均值是 μ 的无偏估计量

Let X₁, X₂, …, Xₙ be a random sample from a distribution with mean μ and variance σ². The sample mean is defined as X̄ = (1/n) Σ Xᵢ. Using linearity of expectation:

设 X₁, X₂, …, Xₙ 是来自均值为 μ、方差为 σ² 的分布的随机样本。样本均值定义为 X̄ = (1/n) Σ Xᵢ。利用期望的线性性质：

E(X̄) = (1/n) Σ E(Xᵢ) = (1/n) × nμ = μ.

Therefore, X̄ is an unbiased estimator of μ. This holds regardless of the underlying distribution, as long as the mean exists. For example, whether the population is normal, Poisson, or exponential, the sample mean always unbiasedly estimates the population mean.

因此，X̄ 是 μ 的无偏估计量。只要均值存在，无论总体分布如何，该结论都成立。例如，无论总体是正态分布、泊松分布还是指数分布，样本均值总是总体均值的无偏估计量。

4. Sample Variance: The Classic Bias Trap | 样本方差：经典的偏差陷阱

When estimating the population variance σ², a natural idea is to use the average squared deviation from the sample mean: (1/n) Σ (Xᵢ − X̄)². However, this estimator is biased. Because X̄ is itself estimated from the data, the deviations Xᵢ − X̄ tend to be slightly smaller than the deviations from the true mean μ. Mathematically,

在估计总体方差 σ² 时，一个自然的想法是使用离样本均值的平均平方偏差：(1/n) Σ (Xᵢ − X̄)²。然而，这个估计量是有偏的。因为 X̄ 本身是从数据中估计的，偏差 Xᵢ − X̄ 往往比离真实均值 μ 的偏差略小。数学上，

E[ (1/n) Σ (Xᵢ − X̄)² ] = ((n−1)/n) σ².

Thus the naive estimator underestimates the variance by a factor of (n−1)/n. To correct this bias, we divide by n − 1 instead of n:

因此，朴素估计量会低估方差，低估因子为 (n−1)/n。为修正该偏差，我们用 n − 1 而不是 n 作除数：

S² = (1/(n−1)) Σ (Xᵢ − X̄)².

This S² is the usual sample variance and it is unbiased: E(S²) = σ². The divisor n − 1 is known as the degrees of freedom, because only n − 1 of the deviations are independent given that their sum is zero.

这个 S² 是常用的样本方差，且是无偏的：E(S²) = σ²。除数 n − 1 称为自由度，因为给定偏差之和为零后，n − 1 个偏差是独立的。

5. Unbiasedness of Sample Proportion | 样本比例的无偏性

For a Bernoulli population with parameter p (the probability of success), the sample proportion p̂ = X̄ = (number of successes)/n is a natural estimator for p. Since each Xᵢ has expectation p, we have E(p̂) = p. Thus the sample proportion is an unbiased estimator of the population proportion. This is fundamental in opinion polls and quality control, where we estimate the probability of an event from the observed frequency.

对于参数为 p（成功概率）的伯努利总体，样本比例 p̂ = X̄ =（成功次数）/n 是 p 的自然估计量。由于每个 Xᵢ 的期望为 p，我们有 E(p̂) = p。因此，样本比例是总体比例的无偏估计量。这在民意调查和质量控制中至关重要，因为我们需要从观测频率来估计事件发生的概率。

6. Method of Moments | 矩估计法

The method of moments, introduced by Karl Pearson, is one of the oldest and simplest methods for finding point estimators. The idea is to equate the sample moments (such as the sample mean and sample second moment) to the theoretical moments of the distribution, and then solve for the parameters.

矩估计法由卡尔·皮尔逊提出，是寻找点估计量最古老、最简单的方法之一。其思想是将样本矩（如样本均值、样本二阶矩）与分布的理论矩相等，然后解出参数。

For a single parameter θ, we set the first sample moment equal to the first population moment:

对于单参数 θ，我们令一阶样本矩等于一阶总体矩：

m₁ = (1/n) Σ Xᵢ = E(X) = μ(θ).

Solve this equation for θ to obtain the method-of-moments estimator. If there are two parameters, we also equate the second moments: m₂ = (1/n) Σ Xᵢ² = E(X²). For example, for a normal distribution N(μ, σ²), the method of moments gives μ̂ = X̄ and σ̂² = (1/n) Σ (Xᵢ − X̄)². Note that the latter is biased, so the method of moments does not always produce unbiased estimators.

解此方程得到 θ 的矩估计量。如果有两个参数，我们还需要令二阶矩相等：m₂ = (1/n) Σ Xᵢ² = E(X²)。例如，对于正态分布 N(μ, σ²)，矩估计法给出 μ̂ = X̄，σ̂² = (1/n) Σ (Xᵢ − X̄)²。注意后者是有偏的，因此矩估计法并不总能产生无偏估计量。

7. Maximum Likelihood Estimation | 极大似然估计法

Maximum likelihood estimation (MLE) is the most widely used estimation method due to its desirable properties. The likelihood function L(θ) is the joint probability (or density) of the observed sample, viewed as a function of the parameter θ. For independent observations, L(θ) = ∏ f(xᵢ; θ). The MLE is the value θ̂ that maximizes L(θ), i.e., the parameter value under which the observed data are most probable.

极大似然估计（MLE）因其优良性质而成为最广泛使用的估计方法。似然函数 L(θ) 是观测样本的联合概率（或密度），看作参数 θ 的函数。对于独立观测，L(θ) = ∏ f(xᵢ; θ)。MLE 是使 L(θ) 最大的 θ̂，即在该参数取值下，观测数据出现的概率最大。

In practice, we maximize the log-likelihood ℓ(θ) = ln L(θ), because the logarithm is a monotonic function and turns products into sums. The derivative is set to zero:

实际操作中，我们最大化对数似然 ℓ(θ) = ln L(θ)，因为对数函数是单调的，并且将乘积转化为求和。令导数为零：

d/dθ ln L(θ) = 0.

For example, for a normal sample, maximizing ℓ(μ, σ²) yields μ̂ = X̄ and σ̂² = (1/n) Σ (Xᵢ − X̄)². Here the MLE of σ² is biased, though the bias tends to zero as n grows.

例如，对于正态样本，最大化 ℓ(μ, σ²) 得到 μ̂ = X̄ 和 σ̂² = (1/n) Σ (Xᵢ − X̄)²。这里 σ² 的 MLE 是有偏的，但偏差随 n 增大而趋于零。

8. Unbiasedness vs. Efficiency | 无偏性与有效性

Among all unbiased estimators, we prefer those with smaller variance, because they are more likely to produce estimates close to the true parameter. If an unbiased estimator has the minimum variance among all unbiased estimators, it is called the uniformly minimum variance unbiased estimator (UMVUE). The Cramér–Rao lower bound provides a lower bound for the variance of any unbiased estimator under certain regularity conditions. If an estimator achieves this bound, it is efficient.

在所有无偏估计量中，我们更偏好方差较小的，因为它们更可能产生接近真实参数的估计值。如果某个无偏估计量在所有无偏估计量中具有最小方差，则称其为一致最小方差无偏估计量（UMVUE）。克拉美–罗下界在正则条件下给出了任何无偏估计量方差的 lower 界。如果某个估计量达到该界，则称其为有效的。

For example, for a normal population, the sample mean X̄ is the UMVUE of μ because it achieves the Cramér–Rao bound. However, an unbiased estimator can sometimes have very large variance, making it impractical. Thus unbiasedness should be balanced with variance considerations.

例如，对于正态总体，样本均值 X̄ 是 μ 的 UMVUE，因为它达到了克拉美–罗下界。然而，一个无偏估计量有时可能方差很大，使其不实用。因此，无偏性应与方差考虑相权衡。

9. Consistency: A Large-Sample Property | 相合性：大样本性质

An estimator θ̂ₙ (depending on sample size n) is consistent if it converges in probability to the true parameter θ as n → ∞. Formally, for any ε > 0,

如果估计量 θ̂ₙ（依赖于样本容量 n）在 n → ∞ 时依概率收敛于真实参数 θ，则称其为相合的。形式化地，对任意 ε > 0，

limₙ→∞ P(|θ̂ₙ − θ| < ε) = 1.

Both unbiasedness and consistency are desirable, but they are different: an estimator can be unbiased but not consistent (e.g., always using only the first observation to estimate μ), and it can be consistent but biased for finite samples (e.g., the MLE of variance). Typically, MLEs are consistent and asymptotically unbiased.

无偏性和相合性都是理想的，但两者不同：估计量可以无偏但不相合（例如，总是只用第一个观测值来估计 μ），也可以有限样本下有偏但相合（例如，方差的 MLE）。通常，MLE 具有相合性和渐近无偏性。

10. Worked Example: Unbiased Estimation of Mean and Variance | 例题：均值与方差的无偏估计

Consider a random sample of size n = 10 from a normal population. The observed data are: 4.5, 5.1, 6.2, 5.8, 4.9, 6.5, 5.3, 6.0, 5.6, 5.2. Compute the unbiased estimates of μ and σ².

设 n = 10 的随机样本来自正态总体。观测数据为：4.5, 5.1, 6.2, 5.8, 4.9, 6.5, 5.3, 6.0, 5.6, 5.2。计算 μ 和 σ² 的无偏估计。

First, compute the sample mean:

首先计算样本均值：

X̄ = (4.5 + 5.1 + 6.2 + 5.8 + 4.9 + 6.5 + 5.3 + 6.0 + 5.6 + 5.2) / 10 = 55.1 / 10 = 5.51.

Next, compute the sum of squared deviations from the mean. The deviations and their squares are listed in the table below:

接下来计算离均值的平方和。偏差及其平方如下表所示：

xᵢ	xᵢ − X̄	(xᵢ − X̄)²
4.5	−1.01	1.0201
5.1	−0.41	0.1681
6.2	0.69	0.4761
5.8	0.29	0.0841
4.9	−0.61	0.3721
6.5	0.99	0.9801
5.3	−0.21	0.0441
6.0	0.49	0.2401
5.6	0.09	0.0081
5.2	−0.31	0.0961

The sum of squared deviations is 1.0201 + 0.1681 + 0.4761 + 0.0841 + 0.3721 + 0.9801 + 0.0441 + 0.2401 + 0.0081 + 0.0961 = 3.489. Then the unbiased sample variance is:

平方和为 1.0201 + 0.1681 + 0.4761 + 0.0841 + 0.3721 + 0.9801 + 0.0441 + 0.2401 + 0.0081 + 0.0961 = 3.489。因此无偏样本方差为：

S² = 3.489 / (10 − 1) = 3.489 / 9 ≈ 0.3877.

Thus the unbiased point estimates are μ̂ = 5.51 and σ̂² ≈ 0.3877.

因此，无偏点估计为 μ̂ = 5.51，σ̂² ≈ 0.3877。

11. Common Mistakes and Exam Tips | 常见错误与考试提示

Forgetting to use n − 1 in the sample variance. Always check the divisor: S² uses n − 1 to be unbiased.
Confusing an estimator (a random variable) with an estimate (a realized value). The estimator has a distribution; the estimate is a fixed number from one sample.
Assuming that the maximum likelihood estimator is always unbiased. In many cases (e.g., σ² for normal data), the MLE is biased, though asymptotically unbiased.
When using the method of moments, be careful to equate the correct moments: first moment for location, second central moment (or raw moment) for scale parameters.
Remember that unbiasedness does not imply consistency, and consistency does not imply unbiasedness in finite samples.

常见错误包括：忘记在样本方差中使用 n − 1；混淆估计量（随机变量）与估计值（一个样本的实现值）；误认为极大似然估计总是无偏的；使用矩估计时未正确匹配矩；以及混淆无偏性与相合性的概念。

In IB exams, you may be asked to verify unbiasedness by computing E(θ̂), to derive estimators using the method of moments or MLE, or to compare two estimators. Always write down the definitions and show expected value calculations clearly.

在 IB 考试中，你可能会被要求通过计算 E(θ̂) 来验证无偏性，使用矩估计法或极大似然估计法推导估计量，或者比较两个估计量。务必写出定义并清晰展示期望值的计算过程。

12. Summary | 总结

Point estimation aims to find good estimators for unknown parameters. Unbiasedness is a key criterion: an estimator should have an expected value equal to the parameter it estimates. The sample mean is unbiased for μ; the sample variance with n − 1 divisor is unbiased for σ². The method of moments and maximum likelihood are two systematic ways to construct estimators, with MLE enjoying many asymptotic optimality properties. A complete understanding of unbiasedness, along with its relationship to bias, variance, and consistency, is essential for mastering statistical inference.

点估计的目标是为未知参数寻找良好的估计量。无偏性是一个关键标准：估计量的期望值应等于其所估计的参数。样本均值是 μ 的无偏估计量；以 n − 1 为除数的样本方差是 σ² 的无偏估计量。矩估计法和极大似然估计法是构造估计量的两种系统方法，其中极大似然估计具有许多渐近最优性。深刻理解无偏性及其与偏差、方差和相合性的关系，是掌握统计推断的基础。

Published by TutorHao | Mathematics Revision Series | aleveler.com

更多咨询请联系16621398022（同微信）

Point Estimation: Unbiased Estimators and Estimation Methods | 点估计：无偏估计量与估计方法