📚 Resampling Methods and Their Applications in Statistics | 重采样方法及其在统计中的应用
Resampling methods are a class of statistical techniques that repeatedly draw new samples from an original dataset in order to estimate uncertainty, validate models, or perform hypothesis tests. They have become increasingly important because modern computation allows us to simulate the sampling distribution of a statistic without relying on restrictive assumptions such as normality.
重采样方法是一类统计技术，它通过反复从原始数据集中抽取新样本，来估计不确定性、验证模型或进行假设检验。由于现代计算能力的提升，我们可以在不依赖正态性等严格假设的情况下，模拟一个统计量的抽样分布，因此重采样方法变得越来越重要。
1. Why Do We Need Resampling Methods? | 为什么需要重采样方法？
Traditional statistical inference often assumes that the data follow a known parametric distribution, such as the normal distribution. Under these assumptions, formulas for standard errors and confidence intervals can be derived analytically. However, if the data are skewed, heavy-tailed, or the estimator is complicated, these formulas may be unreliable.
传统的统计推断通常假设数据服从已知的参数分布，例如正态分布。在这些假设下，我们可以通过解析方法推导出标准误和置信区间的公式。然而，当数据偏斜、具有厚尾，或者估计量很复杂时，这些公式可能并不可靠。
Resampling methods offer a flexible alternative. They use the empirical distribution of the observed data to approximate the true population distribution, and then simulate the variability of an estimator by drawing many resamples. This allows us to obtain valid inference under much weaker assumptions.
重采样方法提供了一种灵活的替代方案。它利用观测数据的经验分布来近似总体分布，然后通过抽取多个重样本来模拟估计量的变异性。这使我们在更弱的假设下也能获得有效的推断。
2. The Bootstrap Principle | Bootstrap 原理
The bootstrap was introduced by Bradley Efron in 1979. Its key idea is to treat the observed sample as if it were the population. We draw a large number of resamples of the same size n from the original data, with replacement. Each resample is called a bootstrap sample.
Bootstrap 方法由 Bradley Efron 于 1979 年提出。其核心思想是将观测样本当作总体来看待。我们从原始数据中有放回地抽取大量与原样本容量相同的重样本，即 n 个观测。每次抽取的重样本称为一个 bootstrap 样本。
For a statistic θ̂, we compute its value on each bootstrap sample. If we draw B bootstrap samples, we obtain B bootstrap estimates θ̂₁, θ̂₂, …, θ̂_B. The spread of these B values gives us an estimate of the standard error of θ̂.
对于一个统计量 θ̂，我们在每个 bootstrap 样本上计算其取值。如果抽取 B 个 bootstrap 样本，我们就会得到 B 个 bootstrap 估计值 θ̂₁, θ̂₂, …, θ̂_B。这 B 个值的离散程度就给出了 θ̂ 标准误的估计。
Mathematically, the bootstrap estimate of standard error is the sample standard deviation of the bootstrap estimates:
数学上，bootstrap 标准误的估计就是这些 bootstrap 估计值的样本标准差：
SE_B = √[ (1/(B−1)) × Σ (θ̂_b − θ̂_mean)² ]
where θ̂_mean is the average of the B bootstrap estimates, and the sum runs from b = 1 to B.
其中 θ̂_mean 是 B 个 bootstrap 估计值的平均值，求和从 b = 1 到 B。
3. Bootstrap for Bias Correction | Bootstrap 用于偏差校正
The bootstrap can also be used to estimate the bias of an estimator. Bias is the difference between the expected value of the estimator and the true parameter value. In many cases, the bootstrap provides a simple way to correct this bias.
Bootstrap 还可以用于估计估计量的偏差。偏差是估计量的期望值与真实参数值之间的差异。在许多情况下，bootstrap 提供了一种简单的偏差校正方法。
Let θ̂ be the estimate from the original sample, and let θ̂_mean be the average of the bootstrap estimates. The bootstrap estimate of bias is θ̂_mean − θ̂. A bias-corrected estimate is then θ̂ − (θ̂_mean − θ̂) = 2θ̂ − θ̂_mean.
设 θ̂ 是原始样本上的估计值，θ̂_mean 是 bootstrap 估计值的平均值。则偏差的 bootstrap 估计为 θ̂_mean − θ̂。偏差校正后的估计为 θ̂ − (θ̂_mean − θ̂) = 2θ̂ − θ̂_mean。
For example, if a sample variance calculated with denominator n is known to be biased downward, the bootstrap can empirically estimate that bias and adjust the estimate upward.
例如，如果使用分母为 n 计算的样本方差已知会偏低，则 bootstrap 可以经验地估计出该偏差，并将估计值向上调整。
4. Bootstrap Confidence Intervals | Bootstrap 置信区间
One of the most valuable applications of the bootstrap is constructing confidence intervals without normal assumptions. The simplest method is the percentile bootstrap, which uses the empirical percentiles of the bootstrap distribution.
Bootstrap 最有价值的应用之一是在没有正态假设的情况下构造置信区间。最简单的方法是百分位 bootstrap，它使用 bootstrap 分布的经验百分位数。
If we have B bootstrap estimates θ̂₁, …, θ̂_B, we sort them and take the values at the 2.5% and 97.5% percentiles as the endpoints of a 95% confidence interval. This interval directly reflects the shape and skewness of the sampling distribution.
如果我们有 B 个 bootstrap 估计值 θ̂₁, …, θ̂_B，将它们排序后，取第 2.5% 和第 97.5% 百分位处的值作为 95% 置信区间的端点。该区间直接反映了抽样分布的形状和偏斜性。
There are also more advanced bootstrap intervals, such as the bias-corrected and accelerated (BCa) intervals, which adjust for both bias and skewness. These intervals have better coverage properties in many situations.
还有一些更高级的 bootstrap 区间，例如偏差校正且加速的（BCa）区间，它们同时校正了偏差和偏斜。在许多情况下，这些区间具有更好的覆盖率性质。
5. Permutation Tests: The Basics | 置换检验：基本原理
Permutation tests, also called randomization tests, are resampling methods used for hypothesis testing. The idea is to break any association between the response variable and the explanatory variable by randomly reassigning the labels of the data. Under the null hypothesis, the labels are exchangeable.
置换检验，也称为随机化检验，是一种用于假设检验的重采样方法。其思想是通过随机重新分配数据的标签来打破响应变量与解释变量之间的任何关联。在原假设下，标签是可以交换的。
For example, suppose we want to compare the means of two groups, A and B. If there is no real difference between the groups, then the labels “A” and “B” are arbitrary. We can randomly permute the labels, recalculate the difference in means, and repeat this many times.
例如，假设我们想比较两组 A 和 B 的均值。如果两组之间没有真正的差异，那么标签“A”和“B”就是任意的。我们可以随机置换标签，重新计算均值差，并重复多次。
The resulting distribution of the test statistic under random permutations is called the permutation distribution. The p-value is the proportion of permutation statistics that are as extreme as or more extreme than the observed statistic.
随机置换下的检验统计量分布称为置换分布。p 值是置换统计量中与观测统计量一样极端或更极端的比例。
6. A Two-Sample Permutation Test Example | 两样本置换检验示例
Let’s illustrate the permutation test with a small example. Suppose Group A has sample mean x̄_A and Group B has sample mean x̄_B. The observed test statistic is D_obs = x̄_A − x̄_B.
让我们通过一个小例子来说明置换检验。假设 A 组的样本均值为 x̄_A，B 组的样本均值为 x̄_B。观测检验统计量为 D_obs = x̄_A − x̄_B。
We combine all observations into one pool. Then we repeatedly divide the pooled data into two groups of sizes n_A and n_B, randomly. For each random division, we compute D = x̄_A* − x̄_B*, where x̄_A* and x̄_B* are the means of the random groups.
我们将所有观测混合在一起。然后反复将混合数据随机分成大小分别为 n_A 和 n_B 的两组。对于每次随机划分，我们计算 D = x̄_A* − x̄_B*，其中 x̄_A* 和 x̄_B* 是随机组的均值。
If we do this 10,000 times, we approximate the null distribution of D. If D_obs is in the far right tail, say the 95th percentile, then the one-sided p-value is about 0.05. This is a nonparametric alternative to the two-sample t-test.
如果我们这样做 10,000 次，就可以近似 D 的零分布。如果 D_obs 位于分布的极右尾，例如第 95 百分位数处，那么单侧 p 值约为 0.05。这是两样本 t 检验的一种非参数替代方法。
7. The Jackknife Method | Jackknife 方法
The jackknife is an earlier resampling technique that works by systematically leaving out one observation at a time. For a dataset of size n, we create n “leave-one-out” samples, each of size n−1.
Jackknife 是一种更早的重采样技术，它通过每次系统性地删除一个观测值来工作。对于容量为 n 的数据集，我们创建 n 个“留一”样本，每个样本容量为 n−1。
If the estimator is θ̂ computed from the full data, and θ̂_(i) is the estimate obtained from the data with the i-th observation removed, then the jackknife estimate of bias is:
如果 θ̂ 是基于完整数据计算的估计量，θ̂_(i) 是删除第 i 个观测后得到的估计值，那么 jackknife 偏差估计为：
Bias_jack = (n−1) × (θ̂_mean_jack − θ̂)
where θ̂_mean_jack is the average of all θ̂_(i) values. The jackknife can also estimate the variance of θ̂ using the spread of the leave-one-out estimates.
其中 θ̂_mean_jack 是所有 θ̂_(i) 值的平均值。Jackknife 还可以利用留一估计值的离散程度来估计 θ̂ 的方差。
The jackknife is computationally cheaper than the bootstrap since it only requires n recomputations. However, it is less effective for highly nonlinear statistics such as medians.
与 bootstrap 相比，jackknife 的计算成本更低，因为它只需要 n 次重新计算。然而，对于中位数等高度非线性的统计量，它的效果较差。
8. Cross-Validation for Model Evaluation | 交叉验证用于模型评估
Cross-validation is a resampling method widely used in model selection and predictive accuracy assessment. In k-fold cross-validation, the dataset is randomly divided into k equal-sized parts. The model is trained on k−1 parts and validated on the remaining part.
交叉验证是一种在模型选择和预测准确性评估中广泛使用的重采样方法。在 k 折交叉验证中，数据集被随机分成 k 个大小相等的部分。模型在 k−1 个部分上训练，并在剩余的 1 个部分上验证。
This process is repeated k times, with each part used exactly once as the validation set. The validation errors are averaged to obtain a more stable estimate of the model’s prediction error.
这个过程重复 k 次，每个部分恰好被用作一次验证集。所有验证集误差的平均值可作为模型预测误差更稳定的估计。
The leave-one-out cross-validation is a special case where k = n, i.e., each training set is the full data minus one observation. This has high variance and is computationally expensive, while 5-fold or 10-fold CV often produce good results with lower cost.
留一交叉验证是 k = n 的特殊情况，即每个训练集都是完整数据减去一个观测。这具有较高的方差且计算成本高，而 5 折或 10 折交叉验证通常能以较低成本产生良好结果。
9. Practical Considerations and Limitations | 实际注意事项与局限性
Resampling methods are powerful but not assumption-free. The bootstrap approximates the population distribution by the empirical distribution; if the original sample is not representative, the resampled results will also be biased.
重采样方法虽然强大，但并非没有假设。Bootstrap 用经验分布来近似总体分布；如果原始样本不具有代表性，重采样结果也会有偏差。
For small sample sizes, bootstrap confidence intervals may be too narrow or too wide. The percentile method can have poor coverage when the sampling distribution is skewed. In such cases, BCa intervals or studentized bootstrap intervals are preferred.
对于小样本量，bootstrap 置信区间可能过窄或过宽。当抽样分布偏斜时，百分位法的覆盖率可能较差。在这种情况下，应优先使用 BCa 区间或学生化 bootstrap 区间。
Permutation tests require exchangeability under the null hypothesis. They work well for comparing groups when the observations are independent, but they may not be valid for complex dependence structures such as time series data.
置换检验要求原假设下的可交换性。在比较组间差异时，若观测独立则效果良好，但对于时间序列数据等复杂依赖结构，置换检验可能无效。
Finally, resampling methods can be computationally intensive. However, with modern computers and efficient algorithms, even B = 10,000 resamples is usually fast for moderate dataset sizes.
最后，重采样方法可能计算量较大。然而，借助现代计算机和高效算法，即使 B = 10,000 次重采样，对于中等规模的数据集通常也很快。
10. Summary and Applications | 总结与应用
Resampling methods provide a unified framework for estimating precision, correcting bias, performing hypothesis tests, and evaluating predictive models. They are essential tools in modern statistics and data science.
重采样方法为估计精度、校正偏差、进行假设检验和评估预测模型提供了统一的框架。它们是现代统计学和数据科学中的重要工具。
Bootstrap methods are widely applied in medical research, economics, and genetics to obtain robust confidence intervals. Permutation tests are common in clinical trials and ecology when standard parametric assumptions fail. Cross-validation is the standard in machine learning for model selection and hyperparameter tuning.
Bootstrap 方法广泛应用于医学研究、经济学和遗传学中，以获得稳健的置信区间。当标准参数假设不成立时，置换检验常用于临床试验和生态学。交叉验证则是机器学习中模型选择和超参数调优的标准方法。
Understanding the strengths and limitations of resampling methods allows analysts to use them appropriately and avoid common pitfalls. As computational power grows, these methods will only become more central to statistical practice.
了解重采样方法的优缺点，有助于分析者正确使用它们并避免常见陷阱。随着计算能力的增长，这些方法只会更加成为统计实践的核心。
Published by TutorHao | Mathematics Revision Series | aleveler.com
更多咨询请联系16621398022（同微信）
屏轩国际教育cambridge primary/secondary checkpoint, cat4, ukiset,ukcat,igcse,alevel,PAT,STEP,MAT, ibdp,ap,ssat,sat,sat2课程辅导，国外大学本科硕士研究生博士课程论文辅导