Advanced Mathematics: Residual Analysis and Regression Diagnostics | 进阶数学：残差分析与回归诊断

📚 Advanced Mathematics: Residual Analysis and Regression Diagnostics | 进阶数学：残差分析与回归诊断

In A-Level Further Mathematics, regression is not merely about fitting a line or curve; it is about testing whether that fit is valid. Residual analysis and regression diagnostics give you the tools to check assumptions, detect outliers, and decide if your model truly explains the data.

在 A-Level 进阶数学中，回归不仅仅是拟合一条直线或曲线，更关键的是检验这个拟合是否有效。残差分析与回归诊断为检查假设、识别异常值以及判断模型是否真正解释数据提供了有力工具。

1. What Is a Residual? | 什么是残差？

A residual is the vertical difference between an observed value \(y_i\) and the value predicted by the regression model, \(\hat{y}_i\). It is written as \(e_i = y_i – \hat{y}_i\).

残差是观测值 \(y_i\) 与回归模型预测值 \(\hat{y}_i\) 之间的垂直差，记为 \(e_i = y_i – \hat{y}_i\)。

eᵢ = yᵢ − ŷᵢ

Each data point has its own residual. A small residual indicates the model closely predicts that observation; a large residual suggests the model struggles at that point.

每个数据点都有其残差。残差小说明模型对该观测值的预测很准确；残差大则说明模型在该点拟合欠佳。

2. Residuals and Fitted Values | 残差与拟合值

Residuals can be positive or negative. A positive residual means the actual value lies above the fitted line; a negative residual means it lies below.

残差可以为正或为负。正残差表示实际值位于拟合线上方；负残差表示实际值位于拟合线下方。

The fitted value is denoted \(\hat{y}_i = a + bx_i\) for simple linear regression. The residual for the \(i\)-th observation is therefore:

拟合值记为 \(\hat{y}_i = a + bx_i\)（一元线性回归中）。因此第 \(i\) 个观测点的残差为：

eᵢ = yᵢ − (a + bxᵢ)

In vector terms, the vector of residuals \(\mathbf{e} = \mathbf{y} – \mathbf{\hat{y}}\) is orthogonal to the column space of the design matrix in ordinary least squares.

从向量角度看，残差向量 \(\mathbf{e} = \mathbf{y} – \mathbf{\hat{y}}\) 与普通最小二乘中设计矩阵的列空间正交。

3. Key Properties of Residuals | 残差的重要性质

Under the classical linear regression assumptions, residuals have several important properties:

在线性回归经典假设下，残差具有以下几个重要性质：

The sum of residuals is exactly zero when the model includes an intercept term: \(\sum e_i = 0\).
当模型包含截距项时，残差之和恰好为零：\(\sum e_i = 0\)。
The mean of residuals is zero.
残差的均值为零。
Residuals are uncorrelated with the fitted values \(\hat{y}_i\).
残差与拟合值 \(\hat{y}_i\) 不相关。
Residuals are uncorrelated with the explanatory variable \(x_i\).
残差与解释变量 \(x_i\) 不相关。

These properties hold exactly for ordinary least squares with an intercept. They form the basis for many diagnostic checks.

这些性质在含截距的普通最小二乘中严格成立，是许多诊断检验的基础。

4. Sum of Squared Residuals (SSE) | 残差平方和（SSE）

The sum of squared residuals, often written as SSE or RSS, measures the total unexplained variation in the model:

残差平方和，通常记为 SSE 或 RSS，衡量模型中无法解释的总变异：

SSE = Σ eᵢ² = Σ (yᵢ − ŷᵢ)²

A smaller SSE indicates a better fit, but SSE alone is not comparable across datasets because it depends on the scale of the response variable.

SSE 越小说明拟合越好，但 SSE 本身不能跨数据集比较，因为它受响应变量尺度的影响。

For example, if you double all \(y\)-values, SSE becomes four times larger even though the model structure is unchanged.

例如，如果将所有 \(y\) 值变为原来的两倍，SSE 将变为原来的四倍，即使模型结构并未改变。

5. Residual Plots | 残差图

A residual plot is a scatter plot of residuals \(e_i\) against fitted values \(\hat{y}_i\), or against another explanatory variable.

残差图是以残差 \(e_i\) 对拟合值 \(\hat{y}_i\)（或对另一个解释变量）绘制的散点图。

Ideally, the residual plot should show a random cloud of points around zero with no clear pattern.

理想情况下，残差图应显示点围绕零线随机分布，无明显的规律性图案。

If a curve or curvature appears, the model may be misspecified (e.g., a linear model for nonlinear data).
如果出现曲线或弯曲形状，说明模型可能设定错误（例如用线性模型拟合非线性数据）。
If the spread of residuals increases or decreases systematically, heteroscedasticity is present.
如果残差的离散程度系统性增大或减小，则存在异方差性。
If residuals form a funnel shape, the constant variance assumption is violated.
如果残差呈漏斗状，则方差齐性假设被违反。

6. Standardized Residuals | 标准化残差

Standardized residuals divide each residual by an estimate of its standard deviation, making them comparable across observations:

标准化残差将每个残差除以其标准差的估计值，使不同观测点的残差可以相互比较：

rᵢ = eᵢ / (s √(1 − hᵢᵢ))

where \(h_{ii}\) is the leverage of observation \(i\), and \(s\) is the estimated standard deviation of the residuals.

其中 \(h_{ii}\) 是第 \(i\) 个观测点的杠杆值，\(s\) 是残差标准差的估计值。

Most standardized residuals should lie between −2 and +2. Values outside −3 and +3 are potential outliers.

大多数标准化残差应在 −2 到 +2 之间。超出 −3 到 +3 的范围通常视为潜在异常值。

7. Studentized Residuals | 学生化残差

A studentized residual is a special type of standardized residual where the standard deviation is estimated after deleting the \(i\)-th observation from the dataset.

学生化残差是一种特殊的标准化残差，其标准差是在删去数据集中第 \(i\) 个观测点后重新估计得到的。

If the \(i\)-th observation has an unusually large influence, the residual variance estimate with that point included may be inflated, masking the outlier.

如果第 \(i\) 个观测点具有异常大的影响，包含该点估计出的残差方差可能被夸大，从而掩盖异常值的存在。

Studentized residuals are therefore more sensitive for detecting outliers. A common rule is to flag observations with \(|t_i| > 2\) when the sample size is large, and \(|t_i| > 3\) in smaller samples.

因此学生化残差对检测异常值更为敏感。常用规则是：样本量较大时，\(|t_i| > 2\) 即标记，样本量较小时取 \(|t_i| > 3\)。

8. Leverage and Cook’s Distance | 杠杆值与库克距离

Leverage \(h_{ii}\) measures how far an observation’s explanatory variable values are from the mean of the explanatory variables. High-leverage points have the potential to strongly affect the fitted regression line.

杠杆值 \(h_{ii}\) 衡量一个观测点的解释变量取值偏离解释变量均值的程度。高杠杆点可能对拟合回归线产生强烈影响。

Cook’s Distance: Dᵢ = (eᵢ² / (p s²)) × (hᵢᵢ / (1 − hᵢᵢ)²)

where \(p\) is the number of parameters in the model. Cook’s distance combines residuals and leverage to identify influential points.

其中 \(p\) 是模型中参数的个数。库克距离将残差与杠杆值结合，用于识别有影响力的观测点。

Dᵢ > 4/n 或 Dᵢ > 1 通常视为有影响力的点

As a practical guide, if \(D_i > 1\) the point is almost certainly influential; some texts use the threshold \(D_i > 4/n\).

实用指南：若 \(D_i > 1\)，该点几乎肯定具有影响力；有些教材使用阈值 \(D_i > 4/n\)。

9. Detecting Autocorrelation: Durbin–Watson Test | 自相关检测：Durbin–Watson 检验

Autocorrelation occurs when residuals are correlated with themselves at adjacent time points. This is critical in time-series regression.

自相关是指残差在相邻时间点上彼此相关。这在时间序列回归中尤为关键。

d = Σ (eᵢ − eᵢ₋₁)² / Σ eᵢ²

The Durbin–Watson statistic \(d\) ranges from 0 to 4.

Durbin–Watson 统计量 \(d\) 的取值范围为 0 到 4。

\(d \approx 2\): no autocorrelation.
\(d \approx 2\)：无自相关。
\(d < 2\): positive autocorrelation, with \(d\) close to 0 indicating strong positive autocorrelation.
\(d < 2\)：存在正自相关，\(d\) 接近 0 时表示很强的正自相关。
\(d > 2\): negative autocorrelation, with \(d\) close to 4 indicating strong negative autocorrelation.
\(d > 2\)：存在负自相关，\(d\) 接近 4 时表示很强的负自相关。

10. Testing Normality of Residuals | 残差正态性检验

Many inference procedures in regression assume residuals are normally distributed. This is especially important for constructing valid confidence intervals and \(t\)-tests.

回归中的许多推断过程假设残差服从正态分布。这对于构造有效的置信区间和 \(t\) 检验尤为重要。

A normal quantile–quantile (Q–Q) plot compares the ordered residuals with the expected quantiles of a standard normal distribution. If the points lie approximately on a straight line, normality is supported.
正态 Q–Q 图将有序残差与标准正态分布的理论分位数进行比较。如果点大致落在一条直线上，则支持正态性假设。
The Shapiro–Wilk test and the Kolmogorov–Smirnov test provide formal hypothesis tests for normality.
Shapiro–Wilk 检验和 Kolmogorov–Smirnov 检验为正态性提供了正式的假设检验。

In the Edexcel Further Mathematics syllabus, you are expected to be able to interpret these diagnostics rather than derive their distributions from first principles.

在 Edexcel 进阶数学考纲中，要求能够解读这些诊断方法，而不是从第一性原理推导它们的分布。

11. Homoscedasticity: Constant Variance | 同方差性：方差齐性

Homoscedasticity means the variance of the residuals is constant across all fitted values. When this assumption fails, the standard errors of the coefficients become unreliable.

同方差性是指残差的方差在所有拟合值上保持恒定。当这一假设不成立时，系数的标准误将变得不可靠。

Heteroscedasticity is often visible in residual plots as a fan or cone pattern: residuals spread wider as fitted values increase.

异方差性通常可以从残差图中看出，表现为扇形或锥形图案：残差随拟合值增大而愈发分散。

如果残差图呈漏斗状，则 Var(eᵢ) = σ² 的假设被违反

Formal tests include the Breusch–Pagan test and the White test. In the A-Level context, visual inspection of residual plots is usually sufficient.

正式检验包括 Breusch–Pagan 检验和 White 检验。在 A-Level 阶段，通常通过目视检查残差图即可判断。

12. Putting It All Together: A Diagnostic Checklist | 综合应用：诊断检查清单

Before trusting any regression output, apply this diagnostic checklist:

在相信任何回归输出之前，请依次完成以下诊断清单：

Plot residuals against fitted values: check for randomness and constant spread.
绘制残差对拟合值的散点图：检查随机性与等方差性。
Plot residuals against each explanatory variable: look for curvature or trends.
绘制每个解释变量的残差图：寻找曲线或趋势。
Calculate standardized or studentized residuals: identify outliers.
计算标准化或学生化残差：识别异常值。
Check leverage values and Cook’s distance: assess influential points.
检查杠杆值和库克距离：评估有影响力点。
Use a Q–Q plot or Shapiro–Wilk test: verify residual normality.
使用 Q–Q 图或 Shapiro–Wilk 检验：验证残差正态性。
For time-series data, compute the Durbin–Watson statistic.
对于时间序列数据，计算 Durbin–Watson 统计量。

If any assumption is violated, you may need to transform variables, add nonlinear terms, or use robust estimation methods.

如果任何假设被违反，你可能需要变换变量、添加非线性项，或使用稳健估计方法。

Published by TutorHao | Further Mathematics Revision Series | aleveler.com

更多咨询请联系16621398022（同微信）

Advanced Mathematics: Residual Analysis and Regression Diagnostics | 进阶数学：残差分析与回归诊断