Computer Vision Experiment: Image Processing and Feature Extraction | 计算机视觉实验：图像处理与特征提取

📚 Computer Vision Experiment: Image Processing and Feature Extraction | 计算机视觉实验：图像处理与特征提取

Computer vision enables machines to interpret visual information from the world. In this experimental guide, we explore fundamental image processing techniques and feature extraction methods, providing a practical roadmap for building a computer vision pipeline.

计算机视觉使机器能够解读来自世界的视觉信息。在本实验指南中，我们探索基础的图像处理技术与特征提取方法，为构建计算机视觉流水线提供一条实用路线。

---

1. Image Acquisition and Digital Representation | 图像采集与数字表示

Every computer vision experiment begins with acquiring a digital image. A camera sensor converts light into electrical signals, then an analog-to-digital converter samples the signal to produce a discrete pixel grid.

每个计算机视觉实验都从采集数字图像开始。相机传感器将光转换为电信号，随后模数转换器对信号采样，生成离散的像素网格。

A grayscale image is stored as a 2D matrix whose values range from 0 (black) to 255 (white). A colour image typically uses three channels: red, green and blue, each with the same 0–255 range.

灰度图像以二维矩阵存储，其取值范围为 0（黑色）到 255（白色）。彩色图像通常使用三个通道：红、绿、蓝，每个通道同样具有 0–255 的取值范围。

• Resolution: Number of pixels in width × height, e.g. 640 × 480.
• Resolution | 分辨率：宽 × 高的像素数量，例如 640 × 480。
• Colour depth: Bits per pixel; 8-bit grayscale yields 256 intensity levels.
• 颜色深度：每像素位数；8 位灰度可产生 256 个亮度级别。
• Coordinate system: Origin at top-left corner, x-axis rightwards, y-axis downwards.
• 坐标系：原点位于左上角，x 轴向右，y 轴向下。

Understanding pixel representation is essential because every subsequent operation – filtering, edge detection and feature extraction – operates directly on this numeric grid.

理解像素表示至关重要，因为后续所有操作——滤波、边缘检测和特征提取——都直接作用于这个数值网格上。

---

2. Image Pre-processing: Denoising and Normalisation | 图像预处理：去噪与归一化

Raw images often contain noise from sensor electronics or ambient lighting. Pre-processing aims to improve image quality while preserving useful structural information.

原始图像往往包含来自传感器电子器件或环境光照的噪声。预处理旨在提高图像质量，同时保留有用的结构信息。

Gaussian blur is a common denoising method. It applies a convolution kernel with weights sampled from a Gaussian function; the standard deviation σ controls blur strength.

高斯模糊是常用的去噪方法。它应用一个权重来自高斯函数的卷积核；标准差 σ 控制模糊强度。

G(x, y) = (1 / (2πσ²)) × e^(−(x² + y²) / (2σ²))

Median filtering replaces each pixel with the median value in its neighbourhood. It is particularly effective against salt-and-pepper noise, while preserving sharp edges better than Gaussian blur.

中值滤波将每个像素替换为其邻域内的中值。它对椒盐噪声尤为有效，并且比高斯模糊更好地保留尖锐边缘。

Normalisation scales intensity values to a fixed range, such as 0–1 or 0–255, which helps subsequent gradient-based methods perform consistently under varying illumination.

归一化将强度值缩放到固定范围，例如 0–1 或 0–255，这有助于后续基于梯度的方法在不同光照下表现一致。

---

3. Edge Detection with Sobel and Canny Operators | 基于 Sobel 与 Canny 算子的边缘检测

Edges represent significant changes in intensity and are key structural features. The Sobel operator computes approximate gradients using two 3×3 kernels.

边缘代表强度的显著变化，是关键的结构特征。Sobel 算子使用两个 3×3 卷积核计算近似梯度。

Gₓ = [[−1, 0, 1], [−2, 0, 2], [−1, 0, 1]] * I

Gᵧ = [[−1, −2, −1], [0, 0, 0], [1, 2, 1]] * I

The gradient magnitude G and direction θ are calculated as:

梯度幅值 G 与方向 θ 计算如下：

G = √(Gₓ² + Gᵧ²), θ = arctan(Gᵧ / Gₓ)

The Canny operator is a multi-stage algorithm: it first applies Gaussian smoothing, then computes gradients, suppresses non-maximum pixels, and finally applies hysteresis thresholding with two thresholds. Canny produces thin, well-connected edges and is widely used in experiments.

Canny 算子是一种多阶段算法：它首先进行高斯平滑，然后计算梯度，抑制非极大值像素，最后使用双阈值进行滞后阈值处理。Canny 生成细且连接良好的边缘，在实验中被广泛使用。

Operator | 算子	Advantages | 优点	Disadvantages | 缺点
Sobel	Simple, fast, noise-resistant | 简单、快速、抗噪	Thick edges, sensitive to threshold choice | 边缘较粗，对阈值选择敏感
Canny	Thin edges, good localisation | 边缘细，定位准确	Computationally heavier, two thresholds to tune | 计算量较大，需调节双阈值

---

4. Global Thresholding and Otsu’s Method | 全局阈值化与 Otsu 方法

Thresholding segments an image into foreground and background by comparing each pixel with a threshold T. A simple binary rule is:

阈值化通过将每个像素与阈值 T 比较，将图像分割为前景和背景。一个简单的二值规则是：

output(x, y) = 255 if I(x, y) > T, else 0

Selecting T manually is often unreliable. Otsu’s method automatically finds the threshold that minimises the weighted within-class variance of the two pixel groups.

手动选择 T 往往不可靠。Otsu 方法自动寻找使两组像素的加权类内方差最小的阈值。

Otsu’s algorithm computes the inter-class variance σ²_B for every possible T and chooses T* that maximises it. This works well for bimodal histograms, where background and object form two distinct peaks.

Otsu 算法对所有可能的 T 计算类间方差 σ²_B，并选择使该值最大的 T*。这对于双峰直方图效果良好，因为背景与物体构成两个不同的峰值。

In practice, Otsu thresholding is often applied after grayscale conversion and Gaussian blur. It provides an unsupervised, robust baseline for image segmentation.

在实践中，Otsu 阈值化常在灰度转换和高斯模糊后应用。它为图像分割提供了一个无监督、稳健的基线方法。

---

5. Corner Detection: Harris Operator | 角点检测：Harris 算子

Corners are points where intensity changes significantly in two orthogonal directions. They are stable across image transformations, making them valuable primitive features.

角点是强度在两个正交方向上显著变化的点。它们在图像变换下具有稳定性，因此是宝贵的原始特征。

The Harris detector computes the structure tensor M from image gradients Iₓ and Iᵧ within a local window.

Harris 检测器从局部窗口内的图像梯度 Iₓ 和 Iᵧ 计算结构张量 M。

M = Σ [Iₓ² , IₓIᵧ ; IₓIᵧ , Iᵧ²]

A corner response R is then computed as R = det(M) − k × trace(M)², where det(M) = λ₁λ₂ and trace(M) = λ₁ + λ₂. Pixels with large positive R are selected as corners; k is an empirical constant, typically 0.04–0.06.

随后计算角点响应 R = det(M) − k × trace(M)²，其中 det(M) = λ₁λ₂，trace(M) = λ₁ + λ₂。R 为较大正值的像素被选为角点；k 是经验常数，通常取 0.04–0.06。

A critical property of Harris corners is rotation invariance: when the image rotates, the response R at the same physical corner remains approximately unchanged, though it is not scale invariant – a corner may disappear when the image is magnified.

Harris 角点的一个重要性质是旋转不变性：当图像旋转时，同一物理角点处的响应 R 近似不变，但它不具备尺度不变性——当图像放大时，角点可能消失。

---

6. Local Feature Descriptors: SIFT and ORB | 局部特征描述子：SIFT 与 ORB

Beyond single-point detectors, modern feature extraction uses keypoints plus descriptors – compact vectors summarising the local appearance around each keypoint.

除单点检测器外，现代特征提取使用“关键点 + 描述子”——即围绕每个关键点局部外观的紧凑向量。

SIFT (Scale-Invariant Feature Transform) identifies keypoints across multiple scales using a Difference-of-Gaussian pyramid. It then assigns an orientation based on gradient histograms and constructs a 128-dimensional descriptor from 4×4 sub-regions.

SIFT（尺度不变特征变换）使用高斯差分金字塔在多个尺度上识别关键点，然后根据梯度直方图分配方向，并从 4×4 子区域构建 128 维描述子。

ORB (Oriented FAST and Rotated BRIEF) is a faster alternative. It uses FAST keypoint detection, computes a dominant orientation, and generates a binary descriptor using BRIEF, enabling very fast matching using Hamming distance.

ORB（定向 FAST 与旋转 BRIEF）是更快速的替代方案。它使用 FAST 关键点检测，计算主导方向，并使用 BRIEF 生成二进制描述子，从而通过汉明距离实现极快匹配。

Method | 方法	Scale robust | 尺度鲁棒	Rotation robust | 旋转鲁棒	Descriptor size | 描述子大小
SIFT	Yes | 是	Yes | 是	128 floats
ORB	No | 否	Yes | 是	256 bits

In experiments, SIFT is preferred when scale changes are expected, while ORB suits real-time applications such as robotics and mobile augmented reality.

在实验中，当预期存在尺度变化时优先选择 SIFT；而 ORB 适用于机器人、移动增强现实等实时应用。

---

7. Experimental Workflow: A Step-by-Step Pipeline | 实验流程：逐步流水线

A typical computer vision experiment follows a structured pipeline. Below is a recommended sequence for a feature-based image matching experiment.

一个典型的计算机视觉实验遵循结构化流水线。下面是为基于特征的图像匹配实验推荐的顺序。

1. Acquire images: Capture or load a dataset containing the target objects under varied lighting and viewing angles.
2. 采集图像：拍摄或加载包含目标物体且光照、视角多变的数据集。
3. Pre-process: Convert to grayscale, reduce noise with Gaussian blur, and normalise intensity.
4. 预处理：转换为灰度，使用高斯模糊降噪，并归一化强度。
5. Detect features: Apply Harris, SIFT or ORB to obtain keypoint locations and orientations.
6. 检测特征：应用 Harris、SIFT 或 ORB 获取关键点位置和方向。
7. Extract descriptors: Compute a descriptor vector for every keypoint.
8. 提取描述子：为每个关键点计算描述子向量。
9. Match: Compare descriptors between two images using Euclidean or Hamming distance; apply Lowe’s ratio test to reject ambiguous matches.
10. 匹配：使用欧氏距离或汉明距离比较两幅图像的描述子；应用 Lowe 比率检验去除模糊匹配。
11. Evaluate: Calculate matching accuracy, visualise keypoints and quantify repeatability.
12. 评估：计算匹配准确率，可视化关键点并量化可重复性。

---

8. Evaluation Metrics for Feature Extraction | 特征提取的评估指标

Quantitative evaluation is crucial to compare methods. Common metrics include precision, recall, and the F1 score applied to feature matches.

定量评估对比较方法至关重要。常用指标包括准确率、召回率以及应用于特征匹配的 F1 分数。

For corner detection, repeatability measures the ratio of keypoints detected in an image that are also detected in a transformed version of the same scene. Higher repeatability implies better robustness.

对于角点检测，可重复性衡量在一幅图像中检测到的关键点在同一场景的变换版本中仍被检测到的比例。可重复性越高说明鲁棒性越好。

For matching tasks, define:

对于匹配任务，定义如下：

Precision = TP / (TP + FP), Recall = TP / (TP + FN)

Here TP counts true matches, FP counts false matches, and FN counts missed true matches. The F1 score is the harmonic mean: F1 = 2 × Precision × Recall / (Precision + Recall).

其中 TP 为正确匹配数，FP 为错误匹配数，FN 为遗漏的正确匹配数。F1 分数是调和平均值：F1 = 2 × Precision × Recall / (Precision + Recall)。

---

9. Common Pitfalls and Practical Optimisations | 常见陷阱与实践优化

Students often encounter several recurring issues. Recognising them early saves significant debugging time.

学生经常遇到几个反复出现的问题。尽早识别它们可以节省大量调试时间。

• Over-blurring: A large Gaussian kernel destroys fine edges and keypoints. Choose σ relative to object size.
• 过度模糊：过大的高斯核会破坏精细边缘和关键点。应根据物体尺寸选择 σ。
• Poor threshold: A fixed threshold fails under varying illumination; use Otsu or adaptive thresholding.
• 阈值不佳：固定阈值在光照变化时会失效；应使用 Otsu 或自适应阈值化。
• Scale mismatch: Harris corners are not scale-invariant; if object size varies, use SIFT or resize images consistently.
• 尺度不匹配：Harris 角点不具备尺度不变性；若物体大小变化，应使用 SIFT 或统一调整图像尺寸。
• Edge responses: Harris may falsely label strong edges as corners; refine by checking eigenvalue ratio. A pixel is a corner only when both λ₁ and λ₂ are large.
• 边缘响应：Harris 可能将强边缘误判为角点；应通过检查特征值比率来修正。仅当 λ₁ 和 λ₂ 都较大时，像素才是角点。

---

10. Applications and Extensions | 应用与拓展

Image processing and feature extraction underpin many real-world systems. Optical character recognition (OCR) segments characters and extracts shape features; facial recognition relies on keypoint alignment; autonomous vehicles use edge and corner features for lane and obstacle detection.

图像处理与特征提取支撑许多真实世界系统。光学字符识别（OCR）分割字符并提取形状特征；人脸识别依赖关键点对齐；自动驾驶使用边缘与角点特征进行车道和障碍物检测。

Medical imaging applies these tools to identify tumours or measure anatomical structures. Remote sensing uses feature extraction to align satellite images over time for environmental monitoring.

医学成像应用这些工具来识别肿瘤或测量解剖结构。遥感使用特征提取来对齐不同时间的卫星图像，以进行环境监测。

Future extensions include deep learning-based descriptors, such as SuperPoint, which learn features directly from data. However, classical methods remain essential for understanding fundamental principles and for environments with limited computational resources.

未来扩展包括基于深度学习的描述子（如 SuperPoint），它们直接从数据中学习特征。然而，经典方法始终是理解基本原理以及算力受限环境中的必要基础。

---

Through this experiment, you have built a complete computer vision pipeline: image acquisition, denoising, edge detection, thresholding, corner detection, descriptor extraction and quantitative evaluation. Mastering these techniques provides a solid foundation for advanced vision research and practical systems.

通过本实验，你已构建了一条完整的计算机视觉流水线：图像采集、去噪、边缘检测、阈值化、角点检测、描述子提取以及定量评估。掌握这些技术，将为高级视觉研究和实际系统奠定坚实基础。

Published by TutorHao | Computer Science Revision Series | aleveler.com

更多咨询请联系16621398022（同微信）