📚 Outlier Detection and Treatment Strategies in Statistics | 统计中的异常值识别与处理策略
In statistics, an outlier is a data point that differs significantly from other observations. It may arise from measurement error, recording mistakes, or genuine rare events. Understanding how to detect and handle outliers is essential for accurate data analysis, especially in exam questions where a single extreme value can distort the mean, standard deviation, and regression results.
在统计学中，异常值是指与其他观测值差异显著的数据点。它可能来自测量误差、记录错误或真实的稀有事件。理解如何识别和处理异常值对于准确的数据分析至关重要，尤其是在考试题目中，一个极端值就可能扭曲均值、标准差和回归结果。
1. What Is an Outlier? | 什么是异常值？
An outlier is an observation that lies an abnormal distance from other values in a random sample from a population. In a simple dataset, if most values cluster around 10 and one value is 100, that 100 is a candidate outlier.
异常值是指在与总体随机样本中其他数值距离异常远的观测值。在一个简单数据集中，如果大多数值集中在10附近，而某个值为100，那么这个100就是候选异常值。
Outliers can be classified into three types: (1) genuine outliers caused by natural variability, such as an exceptionally tall person in a height survey; (2) measurement errors, such as a thermometer reading recorded incorrectly; and (3) data entry errors, such as typing 2.5 as 25. Each type requires different treatment.
异常值可分为三类：（1）由自然变异性引起的真实异常值，例如身高调查中特别高的人；（2）测量误差，例如温度计读数记录错误；（3）数据录入错误，例如将2.5误输为25。每种类型需要不同的处理方式。
2. Why Outliers Matter | 异常值为何重要
Outliers matter because they can heavily influence statistical measures. The mean is particularly sensitive to outliers: adding one extreme value can shift the mean substantially, while the median and mode are more resistant. The standard deviation and variance also inflate in the presence of outliers, leading to overestimated variability.
异常值之所以重要，是因为它们会严重影响统计指标。均值对异常值尤其敏感：加入一个极端值会显著改变均值，而中位数和众数则更具抗性。标准差和方差也会因异常值的存在而膨胀，导致对变异性的高估。
For example, consider the dataset {2, 3, 4, 5, 6, 7, 8}. The mean is 5. If we replace 8 with 80, the new mean becomes approximately 15.3, while the median remains 5. This demonstrates why we need formal rules for outlier detection rather than relying on visual inspection alone.
例如，考虑数据集{2, 3, 4, 5, 6, 7, 8}，均值为5。如果将8替换为80，新的均值约为15.3，而中位数仍然是5。这说明了为什么我们需要正式的异常值检测规则，而不是仅靠目测。
3. The 1.5 × IQR Rule | 1.5倍四分位距法则
The most common method for outlier detection in univariate data is the 1.5 × IQR rule. The interquartile range (IQR) is the difference between the third quartile (Q₃) and the first quartile (Q₁): IQR = Q₃ − Q₁. Any data point below Q₁ − 1.5 × IQR or above Q₃ + 1.5 × IQR is considered an outlier.
单变量数据中最常用的异常值检测方法是1.5倍四分位距法则。四分位距（IQR）是第三四分位数（Q₃）与第一四分位数（Q₁）之差：IQR = Q₃ − Q₁。任何低于 Q₁ − 1.5 × IQR 或高于 Q₃ + 1.5 × IQR 的数据点都被视为异常值。
Step-by-step procedure: (1) Sort the data in ascending order. (2) Find Q₁, the median of the lower half. (3) Find Q₃, the median of the upper half. (4) Calculate IQR. (5) Compute the lower fence and upper fence. (6) Identify points outside these fences as outliers.
具体步骤如下：（1）将数据按升序排列。（2）找到 Q₁，即下半部分的中位数。（3）找到 Q₃，即上半部分的中位数。（4）计算IQR。（5）计算下边界和上边界。（6）将位于边界之外的点识别为异常值。
Example: For the sorted data {1, 2, 3, 5, 7, 9, 10, 12, 15, 60}, Q₁ = 3, Q₃ = 12, IQR = 9. The lower fence is 3 − 13.5 = −10.5, and the upper fence is 12 + 13.5 = 25.5. The value 60 exceeds 25.5, so it is an outlier.
示例：对于有序数据{1, 2, 3, 5, 7, 9, 10, 12, 15, 60}，Q₁ = 3，Q₃ = 12，IQR = 9。下边界为 3 − 13.5 = −10.5，上边界为 12 + 13.5 = 25.5。数值60超过25.5，因此它是异常值。
4. Z-Score Method | Z分数法
The Z-score method assumes the data is approximately normally distributed. For each observation x, the Z-score is calculated as:
Z分数法假设数据近似服从正态分布。对于每个观测值x，Z分数的计算公式为：
Z = (x − μ) / σ
where μ is the population mean and σ is the population standard deviation. In practice, when μ and σ are unknown, we use the sample mean x̄ and sample standard deviation s. A common threshold is |Z| > 3, meaning the value is more than 3 standard deviations away from the mean.
其中μ是总体均值，σ是总体标准差。在实际中，当μ和σ未知时，我们使用样本均值x̄和样本标准差s。常见的阈值为|Z| > 3，即该值距离均值超过3个标准差。
However, the Z-score method has a limitation: both the mean and standard deviation themselves are sensitive to outliers. This is called the masking effect, where a cluster of outliers can pull the mean and inflate the standard deviation, making the outliers appear less extreme. In such cases, robust statistics like the median and IQR are preferable.
然而，Z分数法有一个局限性：均值和标准差本身对异常值敏感。这称为掩蔽效应，即一组异常值会拉动均值并增大标准差，使得异常值看起来不那么极端。在这种情况下，使用中位数和IQR等稳健统计量更为合适。
5. Box Plots and Visual Detection | 箱线图与可视化检测
A box plot (box-and-whisker plot) is an excellent visual tool for detecting outliers. The box represents the IQR, with the median marked inside. Whiskers extend to the most extreme data points within 1.5 × IQR of the quartiles. Any point beyond the whiskers is plotted individually as an outlier.
箱线图（盒须图）是检测异常值的极佳可视化工具。箱体代表IQR，中位数标记在内部。须线延伸到距离四分位数1.5倍IQR范围内的最极端数据点。任何超出须线的点都单独绘制为异常值。
In an exam, you may be asked to draw a box plot or to interpret one. Remember that the whiskers should not extend to the outlier itself; instead, the whisker ends at the last non-outlier value, and outliers are shown as dots or asterisks.
在考试中，你可能会被要求绘制箱线图或解读箱线图。记住，须线不应延伸到异常值本身；相反，须线止于最后一个非异常值，异常值以点或星号表示。
Other visual methods include scatter plots for bivariate data, where points that deviate from the overall pattern may indicate outliers. Histograms can also reveal gaps or isolated bars that suggest potential outliers.
其他可视化方法包括用于双变量数据的散点图，偏离整体模式的点可能表明异常值。直方图也能揭示表明潜在异常值的间隙或孤立条柱。
6. Modified Z-Score and Median Absolute Deviation | 修正Z分数与中位数绝对偏差
The modified Z-score uses the median and the median absolute deviation (MAD) instead of the mean and standard deviation, making it more robust. MAD is defined as:
修正Z分数使用中位数和中位数绝对偏差（MAD）代替均值和标准差，因此更加稳健。MAD的定义为：
MAD = median(|xᵢ − median(x)|)
The modified Z-score is then computed as:
修正Z分数的计算公式为：
Mᵢ = (0.6745 × (xᵢ − median(x))) / MAD
The constant 0.6745 makes the modified Z-score comparable to the standard Z-score under normality. A common cutoff is |Mᵢ| > 3.5. This method is preferred when the dataset contains multiple outliers, as the median and MAD are not pulled by extreme values.
常数0.6745使修正Z分数在正态性下与标准Z分数具有可比性。常用的截断值为|Mᵢ| > 3.5。当数据集中包含多个异常值时，此方法更受青睐，因为中位数和MAD不会被极端值拉动。
7. Handling Outliers: Option 1 — Keep Them | 处理异常值：方案一——保留
Not all outliers should be removed. If an outlier is a genuine observation that represents real variability in the population, it should be kept. For example, in a study of earthquake magnitudes, a very large earthquake is not an error; it is an essential part of the data.
并非所有异常值都应被移除。如果异常值是代表总体真实变异性的合法观测，则应保留。例如，在地震震级研究中，一次非常大的地震不是错误，而是数据的重要组成部分。
When keeping outliers, consider using robust statistical measures. Report the median instead of the mean, and the IQR instead of the standard deviation. These measures are less affected by extreme values and provide a more accurate summary of the central tendency and spread.
保留异常值时，考虑使用稳健统计量。用中位数代替均值，用IQR代替标准差。这些指标受极端值影响较小，能更准确地概括集中趋势和离散程度。
In some analyses, outliers may contain valuable information about unusual events, system failures, or novel discoveries. Removing them without justification can lead to loss of critical insight.
在某些分析中，异常值可能包含关于异常事件、系统故障或新发现的宝贵信息。未经合理理由就将其移除，可能导致关键洞察的丢失。
8. Handling Outliers: Option 2 — Remove Them | 处理异常值：方案二——移除
If an outlier is identified as a measurement error or data entry error, it is often appropriate to remove it from the dataset. For example, if a person’s height is recorded as 8.5 meters, this is clearly a typographical error and should be corrected or deleted.
如果异常值被确认为测量误差或数据录入错误，则通常应将其从数据集中移除。例如，如果某人的身高记录为8.5米，这显然是录入错误，应予以更正或删除。
Before removing an outlier, always investigate its source. Ask: Was the measurement instrument calibrated? Was there a transcription mistake? Is the value physically possible? Only after confirming an error should you remove the point, and you must document this decision.
在移除异常值之前，务必调查其来源。问一问：测量仪器是否经过校准？是否存在转录错误？该数值在物理上是否可能？只有在确认错误之后，你才能移除该点，并且必须记录这一决定。
In exam contexts, if a question states that a value is “an outlier due to recording error,” you may remove it and recalculate the mean and standard deviation. You should clearly show both the original and revised statistics.
在考试情境中，如果题目说明某个值”因记录错误而成为异常值”，你可以将其移除并重新计算均值和标准差。你应该清楚展示原始统计量和修正后的统计量。
9. Handling Outliers: Option 3 — Transform or Winsorize | 处理异常值：方案三——变换或缩尾处理
Transformation is a mathematical approach to reduce the impact of outliers. Common transformations include the logarithmic transformation y = log(x), square root transformation y = √x, and reciprocal transformation y = 1/x. These compress large values and spread out small values, making skewed distributions more symmetric.
变换是减少异常值影响的数学方法。常见变换包括对数变换 y = log(x)、平方根变换 y = √x 和倒数变换 y = 1/x。这些变换压缩大值并展开小值，使偏斜分布更加对称。
Winsorizing is another technique where extreme values are replaced with the nearest non-outlier values. For example, if the 5th percentile is 10 and the 95th percentile is 90, then all values below 10 are set to 10, and all values above 90 are set to 90. This retains the number of observations while reducing the influence of extremes.
缩尾处理是另一种技术，将极端值替换为最近的非异常值。例如，如果第5百分位数为10，第95百分位数为90，那么所有低于10的值都设为10，所有高于90的值都设为90。这保留了观测数量，同时减少了极端值的影响。
Transformations are particularly useful in regression analysis, where outliers can distort the fitted line. Applying a log transformation can often stabilize variance and make the relationship more linear.
变换在回归分析中特别有用，因为异常值会扭曲拟合线。应用对数变换通常能稳定方差并使关系更加线性。
10. Outliers in Regression | 回归中的异常值
In bivariate or multivariate data, outliers can be categorized into two types: outliers in the x-direction (leverage points) and outliers in the y-direction (regression outliers). A leverage point has an extreme predictor value, while a regression outlier has an unusual response value given its predictor.
在双变量或多变量数据中，异常值可分为两类：x方向异常值（杠杆点）和y方向异常值（回归异常值）。杠杆点具有极端的预测变量值，而回归异常值在给定预测变量时具有不寻常的响应值。
A point with high leverage can dramatically change the slope of the regression line. The residual (the difference between observed and predicted y) helps identify regression outliers. Points with large residuals are potential outliers that may indicate model failure or special circumstances.
高杠杆点可能显著改变回归线的斜率。残差（观测y与预测y之差）有助于识别回归异常值。具有大残差的点是潜在的异常值，可能表明模型失败或存在特殊情形。
In exam questions, you may be asked to calculate the residual for a given point or to comment on the effect of a specific point on the regression equation. Always check whether removing the point substantially changes the slope and intercept.
在考试题目中，你可能会被要求计算给定点的残差，或评论特定点对回归方程的影响。务必检查移除该点是否会显著改变斜率和截距。
11. Practical Worked Example | 实际计算示例
Consider the dataset representing the test scores of 10 students: {45, 48, 50, 52, 55, 57, 60, 62, 65, 98}. Using the 1.5 × IQR rule, first sort the data (already sorted). Q₁ is the median of the lower half {45, 48, 50, 52, 55}, which is 50. Q₃ is the median of the upper half {57, 60, 62, 65, 98}, which is 62. Thus IQR = 62 − 50 = 12.
考虑代表10名学生考试成绩的数据集：{45, 48, 50, 52, 55, 57, 60, 62, 65, 98}。使用1.5倍IQR法则，首先对数据排序（已排序）。Q₁是下半部分{45, 48, 50, 52, 55}的中位数，即50。Q₃是上半部分{57, 60, 62, 65, 98}的中位数，即62。因此IQR = 62 − 50 = 12。
Lower fence = 50 − 1.5 × 12 = 50 − 18 = 32. Upper fence = 62 + 1.5 × 12 = 62 + 18 = 80. The value 98 is greater than 80, so it is flagged as an outlier.
下边界 = 50 − 1.5 × 12 = 50 − 18 = 32。上边界 = 62 + 1.5 × 12 = 62 + 18 = 80。数值98大于80，因此被标记为异常值。
Original mean = (45+48+50+52+55+57+60+62+65+98)/10 = 592/10 = 59.2. After removing 98, the trimmed mean = (592 − 98)/9 = 494/9 ≈ 54.9. The median without the outlier: with 9 values, the median is the 5th value, which is 55. The outlier increased the mean by over 4 points, showing its strong influence.
原始均值 = (45+48+50+52+55+57+60+62+65+98)/10 = 592/10 = 59.2。移除98后，截尾均值 = (592 − 98)/9 = 494/9 ≈ 54.9。无异常值时的中位数：9个值的中位数是第5个值，即55。异常值使均值增加了超过4分，显示出其强大影响。
12. Summary and Exam Tips | 总结与考试要点
Outlier detection relies on rules such as the 1.5 × IQR method, Z-scores, box plots, and residual analysis. Each method has assumptions and limitations. Treatment strategies include keeping, removing, transforming, or winsorizing outliers, depending on the context and the cause of the outlier.
异常值检测依赖于1.5倍IQR法、Z分数、箱线图和残差分析等规则。每种方法都有假设和局限性。处理策略包括保留、移除、变换或缩尾处理，具体取决于背景和异常值的成因。
For exams, remember these key points: (1) Always show your working for quartiles and fences. (2) State clearly whether you are using a sample or population formula for Z-scores. (3) When removing an outlier, recalculate all statistics and compare. (4) Justify your decision using the context of the question.
对于考试，记住以下要点：（1）始终展示四分位数和边界的计算过程。（2）明确说明你在计算Z分数时使用的是样本还是总体公式。（3）移除异常值时，重新计算所有统计量并进行比较。（4）结合题目背景为你的决定提供依据。
Also note that the mode is unaffected by outliers, and the median is more robust than the mean. When a dataset is skewed, the median is generally preferred as the measure of central tendency. Understanding these concepts will help you choose the right strategy in any statistical problem.
还要注意，众数不受异常值影响，中位数比均值更稳健。当数据集偏斜时，通常首选众数和中位数作为集中趋势的度量。理解这些概念将帮助你在任何统计问题中选择正确的策略。
Published by TutorHao | Mathematics Revision Series | aleveler.com
更多咨询请联系16621398022（同微信）
屏轩国际教育cambridge primary/secondary checkpoint, cat4, ukiset,ukcat,igcse,alevel,PAT,STEP,MAT, ibdp,ap,ssat,sat,sat2课程辅导，国外大学本科硕士研究生博士课程论文辅导Cancel reply