Implicit Differentiation | 隐函数微分

📚 Implicit Differentiation | 隐函数微分

In computer science, we frequently work with geometric objects, physical simulations, and optimisation landscapes that are defined by equations rather than explicit functions. Understanding how to differentiate such implicitly defined relationships is essential for rendering, physics engines, and machine learning pipelines. This article explores implicit differentiation from a computational perspective, bridging mathematical foundations with practical algorithmic implementations.

在计算机科学中，我们经常处理由方程而非显式函数定义的几何对象、物理仿真和优化场景。理解如何对此类隐式定义的关系进行微分，对于渲染、物理引擎和机器学习流程至关重要。本文从计算的视角探索隐函数微分，在数学基础与实际算法实现之间架起桥梁。

1. From Explicit to Implicit: A Computational View | 从显式到隐式：计算视角

Traditional calculus presents functions in the explicit form y = f(x), where the output is computed directly from the input. In computational geometry, however, curves and surfaces are often stored as levelsets: the set of points satisfying F(x, y) = 0 or F(x, y, z) = 0.

传统微积分以显式形式 y = f(x) 呈现函数，输出由输入直接计算得到。然而在计算几何中，曲线和曲面通常以水平集的形式存储：即满足 F(x, y) = 0 或 F(x, y, z) = 0 的点集。

For example, the unit circle x² + y² − 1 = 0 represents an implicit relationship. A computer cannot simply plug in an x-value and obtain a unique y without additional logic, yet we still need to compute tangent lines or normals for rendering.

例如，单位圆 x² + y² − 1 = 0 表示一种隐式关系。计算机无法在没有额外逻辑的情况下直接代入 x 值获得唯一的 y，但我们仍然需要计算切线或法线来进行渲染。

This shift from explicit to implicit representations demands a different differentiation technique. Instead of applying differentiation rules directly to y = f(x), we differentiate both sides of the equation while treating y as a function of x, using the chain rule throughout.

从显式表达到隐式表达的这一转变，需要一种不同的微分技术。我们不再直接将微分规则应用于 y = f(x)，而是对方程两边同时求导，同时借助链式法则将 y 视为 x 的函数进行处理。

---

2. The Chain Rule and Implicit Differentiation | 链式法则与隐函数微分

The core of implicit differentiation is the chain rule. Given an equation F(x, y) = 0, we assume y can be expressed locally as a function of x, denoted y(x). Differentiating both sides with respect to x yields:

隐函数微分的核心是链式法则。给定方程 F(x, y) = 0，我们假设 y 可以局部表示为 x 的函数，记为 y(x)。对等式两边关于 x 求导得到：

∂F/∂x + ∂F/∂y · dy/dx = 0

Consequently, the derivative of y with respect to x can be isolated as:

因此，y 对 x 的导数可以解出为：

dy/dx = − (∂F/∂x) / (∂F/∂y)

This formula provides a direct computational recipe: evaluate the partial derivatives of F with respect to x and y, take their ratio with a negative sign, and you obtain the local slope. It works wherever ∂F/∂y ≠ 0.

这个公式提供了一个直接的计算配方：计算 F 对 x 和 y 的偏导数，取比值的相反数，即得到局部斜率。只要 ∂F/∂y ≠ 0，该公式便成立。

In a computer program, these partial derivatives can be computed either symbolically, numerically, or via automatic differentiation. Each approach has distinct trade-offs in speed, accuracy, and code complexity.

在计算机程序中，这些偏导数可以通过符号计算、数值计算或自动微分来获得。每种方法在速度、准确性和代码复杂度方面各有千秋。

---

3. Symbolic Computation of Implicit Derivatives | 隐函数导数的符号计算

Symbolic differentiation manipulates mathematical expressions as abstract syntax trees. Libraries like SymPy (Python) or Mathematica can apply differentiation rules analytically, yielding exact expressions for derivatives.

符号微分将数学表达式作为抽象语法树进行操作。像 SymPy（Python）或 Mathematica 这样的库可以解析地应用微分规则，产生导数的精确表达式。

Given an implicit equation, a symbolic engine computes ∂F/∂x and ∂F/∂y using standard derivative tables and the chain rule. Then it applies the implicit differentiation formula to produce a closed-form expression for dy/dx.

给定一个隐式方程，符号引擎使用标准导数表和链式法则计算 ∂F/∂x 和 ∂F/∂y。然后它应用隐函数微分公式，生成 dy/dx 的闭式表达式。

For instance, for F(x, y) = x³ + y³ − 6xy = 0 (the folium of Descartes), the symbolic derivative dy/dx = (2y − x²) / (y² − 2x). This exact formula can then be evaluated at any point on the curve.

例如，对于 F(x, y) = x³ + y³ − 6xy = 0（笛卡尔叶形线），符号导数 dy/dx = (2y − x²) / (y² − 2x)。这个精确公式可以在曲线上任意点处求值。

Symbolic methods excel in precision but suffer from expression swell for complex functions, where intermediate formulas grow exponentially. In computational geometry, symbolic differentiation is often avoided in performance-critical inner loops.

符号方法在精度方面表现出色，但对复杂函数会出现表达式膨胀问题，中间公式可能呈指数级增长。在计算几何中，符号微分在性能关键的内部循环中常被避免使用。

---

4. Automatic Differentiation and the Implicit Function Theorem | 自动微分与隐函数定理

Automatic differentiation (AD) is a technique that computes exact derivatives by repeatedly applying the chain rule to elementary operations. Unlike symbolic differentiation, AD does not produce large expressions—it evaluates derivatives alongside the function values in a forward or backward pass.

自动微分（AD）是一种通过对基本运算重复应用链式法则来计算精确导数的技术。与符号微分不同，AD 不产生庞大的表达式——它在前向或反向传递中与函数值同步计算导数。

For an implicitly defined relationship, we can leverage the implicit function theorem in forward-mode AD. By treating the equation F(x, y) = 0 as a constraint, the tangent linear system can be solved numerically without forming full symbolic derivatives.

对于隐式定义的关系，我们可以借助前向模式 AD 中的隐函数定理。通过将方程 F(x, y) = 0 视为约束条件，可以在不建立完整符号导数的情况下数值求解切线性系统。

In practice, we can track dual numbers for both x and y, enforce the equation F = 0, and solve a small linear system to obtain dy/dx. Reverse-mode AD is especially powerful when many inputs influence one output, as in neural network training.

实际操作中，我们可以同时追踪 x 和 y 的对偶数，强制方程 F = 0，并求解一个小型线性系统以获得 dy/dx。当多个输入影响一个输出时，反向模式 AD 尤为强大，例如神经网络训练中。

This approach is fundamental to differentiable rendering and physics-informed neural networks, where implicit constraints define the system’s state.

这种方法对于可微分渲染和物理信息神经网络至关重要，其中隐式约束定义了系统的状态。

---

5. Numerical Approaches: Finite Differences for Implicit Curves | 数值方法：隐式曲线的有限差分

When symbolic or AD infrastructure is unavailable, a simple numerical alternative is the finite difference method. We can approximate the derivative dy/dx along an implicit curve by taking small steps in the parameter and observing changes in the coordinates.

当缺乏符号或 AD 基础设施时，一种简单的数值替代方案是有限差分法。我们可以通过在参数中取小步长并观察坐标的变化，来近似隐式曲线上的导数 dy/dx。

For a point (x₀, y₀) satisfying F(x₀, y₀) = 0, we can find another nearby point (x₀ + h, y₁) on the curve using root-finding, then estimate dy/dx ≈ (y₁ − y₀) / h. Repeating this yields a numerical slope.

对于满足 F(x₀, y₀) = 0 的点，我们可以使用求根方法找到曲线上附近的点 (x₀ + h, y₁)，然后用 dy/dx ≈ (y₁ − y₀) / h 进行估计。重复此过程即可得到数值斜率。

More elegantly, we can apply the finite difference directly to the partial derivatives: ∂F/∂x ≈ [F(x+h, y) − F(x−h, y)]/(2h) and similarly for ∂F/∂y. Substituting into the implicit formula gives an approximation of dy/dx that converges as h → 0.

更优雅的方法是直接将有限差分应用到偏导数上：∂F/∂x ≈ [F(x+h, y) − F(x−h, y)]/(2h)，对 ∂F/∂y 同理。代入隐函数公式便可得到 dy/dx 的近似值，当 h → 0 时收敛。

Numerical methods are easy to implement but suffer from truncation and round-off errors. The choice of step size h is a delicate balance between accuracy and stability.

数值方法易于实现，但存在截断误差和舍入误差。步长 h 的选择需要在准确性和稳定性之间仔细权衡。

---

6. Applications in Computer Graphics: Surface Normals | 计算机图形学应用：曲面法线

One of the most immediate applications of implicit differentiation is computing surface normals for rendering. Given an implicit surface F(x, y, z) = 0, the gradient ∇F = (∂F/∂x, ∂F/∂y, ∂F/∂z) is perpendicular to the surface at every point.

隐函数微分最直接的应用之一，是在渲染中计算曲面法线。给定隐式曲面 F(x, y, z) = 0，梯度 ∇F = (∂F/∂x, ∂F/∂y, ∂F/∂z) 在处处均垂直于曲面。

This gradient can be derived implicitly without solving for z = f(x, y). By differentiating F(x, y, z) = 0 with respect to x and y separately, we demonstrate that the normal vector is indeed the gradient of F.

该梯度可以通过隐函数微分导出，无需解出 z = f(x, y)。通过分别对 x 和 y 求导 F(x, y, z) = 0，我们可以证明法向量确实是 F 的梯度。

Ray marching algorithms that render metaballs, blobby objects, or fractals rely on evaluating the implicit function and its gradient at sampled points. The gradient, computed via implicit differentiation, enables efficient sphere tracing and lighting calculations.

渲染元球、软体对象或分形的光线行进算法，依赖于在采样点处计算隐式函数及其梯度。通过隐函数微分计算出的梯度，能够实现高效的球体追踪和光照计算。

Thus, implicit differentiation bridges the gap between a compact geometric representation and the differential quantities needed for photorealistic rendering.

因此，隐函数微分在紧凑的几何表示与真实感渲染所需的微分量之间架起了桥梁。

---

7. Implicit Differentiation in Machine Learning and Optimization | 机器学习与优化中的隐函数微分

Modern deep learning often involves layers or loss functions that are defined implicitly. For instance, in neural ODEs or equilibrium models, the output is defined as the solution to an implicit equation f(x, y) = 0.

现代深度学习经常涉及隐式定义的层或损失函数。例如，在神经常微分方程或平衡模型中，输出被定义为隐式方程 f(x, y) = 0 的解。

To backpropagate gradients through such layers, we cannot use standard explicit derivatives. Instead, we apply the implicit function theorem at the solution point, computing dy/dx = −[∂f/∂y]⁻¹ ∂f/∂x. This requires solving a linear system for each backward pass.

要通过此类层反向传播梯度，我们不能使用标准的显式导数。相反，我们在解点处应用隐函数定理，计算 dy/dx = −[∂f/∂y]⁻¹ ∂f/∂x。这需要在每次反向传播中求解一个线性系统。

This technique is central to implicit deep learning frameworks and meta-learning, where the inner loop solves an optimization problem that can be differentiated through via implicit differentiation.

该技术是隐式深度学习框架和元学习中的核心，其中内循环求解一个优化问题，并可通过隐函数微分进行求导。

Automatic differentiation libraries such as JAX and PyTorch now offer tools to set up implicit differentiation using custom VJP rules, making it practical for large-scale models.

像 JAX 和 PyTorch 这样的自动微分库，现在提供了使用自定义 VJP 规则来建立隐函数微分的工具，使其在大规模模型中切实可行。

---

8. A Practical Python Implementation Using SymPy | 使用 SymPy 的 Python 实现

To solidify the concepts, let us implement implicit differentiation using SymPy. The following code symbolically differentiates the unit circle equation and evaluates the derivative at a given point.

为了巩固概念，让我们使用 SymPy 实现隐函数微分。以下代码对单位圆方程进行符号微分，并在给定点处计算导数。

import sympy as sp

x, y = sp.symbols('x y')
F = x**2 + y**2 - 1

partial_x = sp.diff(F, x)
partial_y = sp.diff(F, y)
dydx = - partial_x / partial_y
dydx_simplified = sp.simplify(dydx)

print("dy/dx =", dydx_simplified)

# Evaluate at (√2/2, √2/2)
from sympy import sqrt
pt = {x: sqrt(2)/2, y: sqrt(2)/2}
slope = dydx_simplified.subs(pt)
print("Slope at point:", slope)

The output dy/dx = −x/y yields −1 at the given point, confirming the tangent slope. SymPy handles the algebraic manipulation automatically, giving an exact result.

输出 dy/dx = −x/y 在给定点处得到 −1，验证了切线斜率。SymPy 自动处理代数运算，给出精确结果。

For more complex equations like the folium, the same pattern applies: define F, compute partial derivatives, and simplify. Symbolic engines are particularly useful in educational and rapid prototyping settings.

对于像笛卡尔叶形线这样更复杂的方程，同样适用这一模式：定义 F，计算偏导数，然后化简。符号引擎在教育和快速原型环境中特别有用。

---

9. Challenges: Expression Swell and Numerical Stability | 挑战：表达式膨胀与数值稳定性

When symbolic differentiation is pushed to deep function compositions, the derivative expressions can explode in size—a phenomenon known as expression swell. This drastically increases memory usage and slows computation.

当符号微分被应用于深层函数复合时，导数表达式的大小可能急剧膨胀——这种现象称为表达式膨胀。这会大幅增加内存使用并减慢计算速度。

Numerical differentiation, while simple, is vulnerable to instability near discontinuities or when the denominator ∂F/∂y approaches zero. At such points, the implicit function theorem breaks down, and the derivative may be undefined or infinite.

数值微分虽然简单，但在不连续点附近或分母 ∂F/∂y 趋近于零时容易不稳定。在这些点上，隐函数定理失效，导数可能无定义或无穷大。

Automatic differentiation mitigates expression swell by keeping intermediate values as numbers, not formulas. However, solving the implicit step may require iterative methods that also demand careful regularisation for stability.

自动微分通过将中间值保存为数值而非公式，减轻了表达式膨胀。然而，求解隐式步骤可能需要迭代方法，这也需要仔细正则化以保证稳定性。

Understanding these trade-offs helps developers choose the right differentiation technique for their application, balancing expressiveness, speed, and numerical robustness.

理解这些权衡有助于开发者为其应用选择合适的微分技术，在表达性、速度和数值健壮性之间取得平衡。

---

10. Conclusion and Future Perspectives | 结论与展望

Implicit differentiation provides a powerful lens to compute slopes and gradients for equations that cannot be conveniently rewritten in explicit form. From the core formula dy/dx = −(∂F/∂x)/(∂F/∂y) to modern automatic differentiation libraries, the technique underpins a wide array of computer science applications.

隐函数微分提供了一个强大的视角，用于为不易改写成显式形式的方程计算斜率和梯度。从核心公式 dy/dx = −(∂F/∂x)/(∂F/∂y) 到现代自动微分库，该技术支撑着广泛的计算机科学应用。

As fields like differentiable simulation, neural implicit representations, and probabilistic programming advance, implicit differentiation will continue to evolve. Future tools may seamlessly blend symbolic, automatic, and numerical methods to offer reliable derivatives with minimal user intervention.

随着可微分仿真、神经隐式表示和概率编程等领域的发展，隐函数微分也将不断演进。未来的工具可能无缝地融合符号、自动和数值方法，以最小的用户干预提供可靠的导数。

Whether you are rendering a soft-edged metaball or optimising a deep equilibrium model, mastering implicit differentiation is a key step toward robust computational geometry and machine learning.

无论你是在渲染一个软边元球，还是在优化一个深度平衡模型，掌握隐函数微分都是迈向稳健计算几何和机器学习的关键一步。

---

Published by TutorHao | Computer Science Revision Series | aleveler.com

更多咨询请联系16621398022（同微信）