Least Squares Linear Regression Model Explained | 最小二乘线性回归模型解析

📚 Least Squares Linear Regression Model Explained | 最小二乘线性回归模型解析

Linear regression is one of the core topics in Edexcel A-Level Further Mathematics. It models the relationship between an explanatory variable and a response variable using a straight line, and the method of least squares gives the best-fitting line in a precise mathematical sense.

线性回归是 Edexcel 进阶数学的核心内容之一。它用一条直线刻画解释变量与响应变量之间的关系，而最小二乘法从精确的数学角度给出“最佳拟合直线”。

1. Model Definition and Assumptions | 模型定义与假设

In a simple linear regression model, we suppose that for n paired observations (xᵢ, yᵢ), the true relationship is yᵢ = α + βxᵢ + εᵢ. Here α is the intercept, β is the slope, and εᵢ is a random error term. For the standard theory to apply, the errors are assumed to be independent and normally distributed with mean 0 and constant variance σ².

在简单线性回归模型中，对 n 组配对观测 (xᵢ, yᵢ)，我们假设真实关系为 yᵢ = α + βxᵢ + εᵢ。其中 α 是截距，β 是斜率，εᵢ 是随机误差项。为保证标准理论成立，误差被假定为独立同分布，服从均值为 0、方差恒为 σ² 的正态分布。

The parameters α and β are unknown. In Edexcel Further Statistics, we use sample data to obtain estimates a and b. The fitted regression line is then written as ŷ = a + bx, where ŷ is the predicted value of y for a given x.

参数 α 与 β 未知。在 Edexcel 进阶统计中，我们用样本数据得到估计量 a 和 b，拟合回归线记为 ŷ = a + bx，其中 ŷ 是给定 x 时 y 的预测值。

2. The Principle of Least Squares | 最小二乘原理

The method of least squares chooses a and b to make the sum of squared vertical distances between the observed yᵢ and the fitted line as small as possible. For a candidate line y = a + bx, the vertical discrepancy is yᵢ − (a + bxᵢ). The total loss is therefore:

最小二乘法选择 a 和 b，使所有观测点 yᵢ 到拟合直线的竖直距离平方和尽可能小。对于候选直线 y = a + bx，竖直偏差为 yᵢ − (a + bxᵢ)，因此总损失为：

S(a, b) = Σ(yᵢ − a − bxᵢ)²

Squaring has two important effects: it removes the problem of positive and negative errors cancelling out, and it gives greater weight to large deviations. This makes the estimates sensitive to outliers, but it also makes the optimisation problem solvable by calculus.

平方有两个重要作用：既避免了正负误差相互抵消，又给予较大偏差更大的权重。这使估计对异常值较为敏感，但也使优化问题可以通过微积分求解。

3. Deriving the Normal Equations | 推导正规方程

We minimise S(a, b) by taking partial derivatives with respect to a and b and setting them equal to zero. The derivative conditions are ∂S/∂a = 0 and ∂S/∂b = 0, which yield the following normal equations:

为最小化 S(a, b)，我们分别对 a 和 b 求偏导并令其为零。导数条件为 ∂S/∂a = 0 和 ∂S/∂b = 0，由此得到正规方程：

Σyᵢ = n a + b Σxᵢ
Σxᵢyᵢ = a Σxᵢ + b Σxᵢ²

The first equation shows that the fitted line must balance the data around the mean. The second equation incorporates the covariance-like information between x and y. Solving these two linear equations simultaneously gives the least squares estimates.

第一个方程表明拟合直线必须围绕均值平衡数据；第二个方程则引入了 x 与 y 之间类似协方差的信息。联立求解这两个线性方程，即可得到最小二乘估计。

4. Slope and Intercept Formulas | 斜率与截距公式

The solution of the normal equations is usually written in terms of the corrected sums of squares. Define Sxx = Σxᵢ² − (Σxᵢ)²/n and Sxy = Σxᵢyᵢ − (Σxᵢ)(Σyᵢ)/n. Then the slope estimate is:

正规方程组的解通常用修正平方和表示。定义 Sxx = Σxᵢ² − (Σxᵢ)²/n，Sxy = Σxᵢyᵢ − (Σxᵢ)(Σyᵢ)/n。则斜率估计为：

b = Sxy / Sxx

Once b is known, the intercept is derived from the first normal equation: a = ȳ − b x̄. This shows that the fitted line always passes through the centroid (x̄, ȳ).

求得 b 后，由第一个正规方程可导出截距：a = ȳ − b x̄。这说明拟合直线必过质心点 (x̄, ȳ)。

5. Properties of the Regression Line | 回归线的性质

The least squares line has several useful algebraic properties. First, it always passes through the mean point (x̄, ȳ). Second, the sum of the residuals is exactly zero. Third, the sum of the fitted values equals the sum of the observed values.

最小二乘直线具有若干重要的代数性质。第一，它必过均值点 (x̄, ȳ)。第二，残差之和恰好为零。第三，拟合值之和等于观测值之和。

The residuals are uncorrelated with the explanatory variable: Σxᵢeᵢ = 0.

残差与解释变量不相关：Σxᵢeᵢ = 0。
The fitted values are uncorrelated with the residuals: Σŷᵢeᵢ = 0.

拟合值与残差不相关：Σŷᵢeᵢ = 0。

These properties are consequences of the normal equations and are often used in proof-style exam questions.

这些性质是正规方程的直接推论，也常出现在证明类的考试题目中。

6. Residuals and Residual Sum of Squares | 残差与残差平方和

The residual for the i-th observation is defined as eᵢ = yᵢ − ŷᵢ. The residual sum of squares, RSS = Σeᵢ², measures the unexplained variation after fitting the line. Smaller RSS means the fitted line is closer to the data.

第 i 个观测的残差定义为 eᵢ = yᵢ − ŷᵢ。残差平方和 RSS = Σeᵢ² 度量拟合后未被解释的变异。RSS 越小，表示拟合直线越贴近数据。

The error variance σ² is unknown, but it can be estimated unbiasedly by s² = RSS/(n − 2). The denominator n − 2 reflects the fact that two parameters a and b have been estimated from the data. This quantity s² appears in all standard error formulas.

误差方差 σ² 未知，但可用 s² = RSS/(n − 2) 进行无偏估计。分母 n − 2 反映我们从数据中估计了两个参数 a 和 b。这个 s² 会出现在所有标准误公式中。

7. Coefficient of Determination R² | 决定系数 R²

The total variation in y is Syy = Σ(yᵢ − ȳ)². The coefficient of determination is defined as the proportion of this variation explained by the regression:

y 的总变异为 Syy = Σ(yᵢ − ȳ)²。决定系数定义为回归所解释变异占总变异的比例：

R² = 1 − RSS / Syy

For a simple linear regression with one explanatory variable, this is exactly the square of the Pearson correlation coefficient r. It can also be computed as R² = (Sxy)² / (Sxx Syy). The value of R² always lies between 0 and 1.

对仅含一个解释变量的简单线性回归，R² 恰好等于 Pearson 相关系数 r 的平方。它也可以计算为 R² = (Sxy)² / (Sxx Syy)。R² 的值总在 0 与 1 之间。

An R² close to 1 means most of the variation in y is captured by the linear relationship with x, while a value close to 0 suggests little linear association.

R² 接近 1 表示 y 的变异大部分能被 x 的线性关系解释；R² 接近 0 则表明线性关联很弱。

8. Hypothesis Test for the Slope | 斜率假设检验

A common question in Edexcel Further Mathematics is whether the slope β is significantly different from zero. The null hypothesis is H₀: β = 0, and the alternative can be H₁: β ≠ 0 for a two-tailed test. Under H₀, the test statistic follows a t-distribution with n − 2 degrees of freedom:

在 Edexcel 进阶数学中，常见问题是判断斜率 β 是否显著不为零。原假设为 H₀: β = 0，双侧检验的备择假设为 H₁: β ≠ 0。在原假设下，检验统计量服从自由度为 n − 2 的 t 分布：

t = (b − β₀) / SE(b) ~ t(n − 2)

If β₀ = 0, this test checks whether there is evidence of a linear relationship between x and y. A large absolute t-value compared with the critical value leads to rejection of H₀. The standard error SE(b) is obtained from the formula in the next section.

当 β₀ = 0 时，该检验用于判断是否存在证据表明 x 与 y 之间具有线性关系。若 |t| 大于临界值，则拒绝 H₀。标准误 SE(b) 由下一节公式给出。

9. Standard Errors and Confidence Intervals | 标准误与置信区间

The standard error of the slope is SE(b) = √(s² / Sxx), where s² = RSS/(n − 2). The standard error of the intercept is more involved:

斜率的标准误为 SE(b) = √(s² / Sxx)，其中 s² = RSS/(n − 2)。截距的标准误则更复杂：

SE(a) = √(s² (1/n + x̄² / Sxx))

From these standard errors, a (1 − α) confidence interval for β can be constructed as b ± t(n − 2, α/2) × SE(b). Similarly, a confidence interval for α is a ± t(n − 2, α/2) × SE(a).

基于这些标准误，β 的 (1 − α) 置信区间可构造为 b ± t(n − 2, α/2) × SE(b)。类似地，α 的置信区间为 a ± t(n − 2, α/2) × SE(a)。

The interval narrows as the sample size increases or when the x-values are more spread out, because Sxx becomes larger.

当样本量增大或 x 取值更分散时，Sxx 变大，置信区间会变窄。

10. Prediction and Its Error | 预测与误差

Once the regression line is fitted, it can be used to predict the response at a new value x₀. The point prediction is simply ŷ₀ = a + b x₀. However, the uncertainty of this prediction depends on whether we are predicting the mean response or a single new observation.

拟合回归线后，可用于预测自变量取新值 x₀ 时的响应。点预测为 ŷ₀ = a + b x₀。不过，预测的不确定性取决于我们是在预测均值响应，还是预测单个新观测值。

For the mean response at x₀, the standard error is:

对 x₀ 处的均值响应，标准误为：

SE(mean) = s √(1/n + (x₀ − x̄)² / Sxx)

For a single new observation, one extra source of variation must be included:

对单个新观测值，还需额外加入一项变异来源：

SE(individual) = s √(1 + 1/n + (x₀ − x̄)² / Sxx)

Thus prediction intervals for individual observations are wider than confidence intervals for the mean. Both become wider as x₀ moves away from x̄, reflecting the greater uncertainty in extrapolation.

因此单个观测的预测区间宽于均值的置信区间。当 x₀ 离 x̄ 越远时，两者都会变宽，反映了外推时更大的不确定性。

11. Common Pitfalls and Practical Considerations | 常见误区与实践注意

Least squares regression is powerful, but it can be misapplied. The following points are especially important in exams and in real data analysis.

最小二乘回归非常强大，但也容易被误用。以下几点在考试和实际数据分析中尤为重要。

A significant regression does not imply causation; the relationship may be due to confounding variables.

显著的回归关系并不代表因果关系；这种关系可能由混杂变量导致。
Extrapolation beyond the observed x-range is risky because the linear relationship may not continue.

超出观测 x 范围的外推有风险，因为线性关系未必延续。
Outliers with extreme x-values, called high-leverage points, can pull the regression line strongly.

具有极端 x 值的异常值称为高杠杆点，会强烈拉动回归线。
Always inspect residual plots to check the assumptions of constant variance and linearity.

应始终检查残差图，以验证方差恒定与线性假设是否成立。

If the residual plot shows a curved pattern, a linear model is inappropriate even if R² is high.

若残差图呈现弯曲模式，则即使 R² 很高，线性模型也不合适。

12. Worked Example | 例题

Consider four paired observations: (1, 2), (2, 4), (3, 5), (4, 7). We can compute the least squares line and test whether the slope is significant.

设有四组配对观测：(1, 2)、(2, 4)、(3, 5)、(4, 7)。我们来计算最小二乘直线并检验斜率是否显著。

First compute the summary statistics: n = 4, Σx = 10, Σy = 18, Σxy = 53, Σx² = 30. Then Sxx = 30 − 10²/4 = 5 and Sxy = 53 − (10 × 18)/4 = 8. Hence b = 8/5 = 1.6, and a = 4.5 − 1.6 × 2.5 = 0.5. The fitted line is:

先计算汇总统计量：n = 4，Σx = 10，Σy = 18，Σxy = 53，Σx² = 30。于是 Sxx = 30 − 10²/4 = 5，Sxy = 53 − (10 × 18)/4 = 8。因此 b = 8/5 = 1.6，a = 4.5 − 1.6 × 2.5 = 0.5。拟合直线为：

ŷ = 0.5 + 1.6x

x	y	ŷ	Residual eᵢ
1	2	2.1	−0.1
2	4	3.7	0.3
3	5	5.3	−0.3
4	7	6.9	0.1

RSS = (−0.1)² + 0.3² + (−0.3)² + 0.1² = 0.20, so s² = 0.20/2 = 0.10. Then SE(b) = √(0.10/5) ≈ 0.1414, and the test statistic for H₀: β = 0 is t = 1.6/0.1414 ≈ 11.31. With 2 degrees of freedom, the 5% two-tailed critical value is 4.303, so we strongly reject H₀ and conclude that the slope is significantly different from zero.

RSS = (−0.1)² + 0.3² + (−0.3)² + 0.1² = 0.20，故 s² = 0.20/2 = 0.10。于是 SE(b) = √(0.10/5) ≈ 0.1414，检验 H₀: β = 0 的统计量为 t = 1.6/0.1414 ≈ 11.31。自由度为 2 时，5% 双侧检验临界值为 4.303，因此我们有充分理由拒绝 H₀，认为斜率显著不为零。

Published by TutorHao | Further Mathematics Revision Series | aleveler.com

更多咨询请联系16621398022（同微信）

Least Squares Linear Regression Model Explained | 最小二乘线性回归模型解析