1 Newton-Raphson Method
1.1 Introduction
Solving an analog circuit involves determining the voltage at all nodes in the circuit. This is usually done by using Kirchoff’s current law, which is based on the principle of conservation of charge, which states that the algebraic sum of the currents at any node in a circuit is equal to zero.
By applying Kirchoff’s current laws to every node in the circuit, we get a collection of equations of the form $f_{i} (V_{1}, \ldots, V_{d}) = 0$, where $f_{i}$ is some scalar function. Assuming there are $k$ scalar functions of $d$ variables, we can combine them into a single column vector to obtain the following system of equations:
\begin{equation} F = \left[\begin{array}{c} f_{1} (\boldsymbol{V})\\\ \vdots \\\ f_{k} (\boldsymbol{V}) \end{array}\right] =\boldsymbol{0}, \quad \text{where } \boldsymbol{V} \in \mathbb{R}^d\tag{1} \end{equation}
Solving the system above involves finding the unknown voltages $\boldsymbol{V}$ such that $F =\boldsymbol{0}$. When $f_{i} (\boldsymbol{V})$ is an affine function, the system can be solved exactly using standard linear algebra methods. However, in the presence of non-linearities, we have to resort to approximate methods. One such method is the Newton-Raphson method, which is based on the intuition that a function $F$ can be well-approximated by its tangent plane $T$ near its root.
Algorithm: Newton-Raphson Method
- Assume that the root of $F$ is located at the point $\boldsymbol{V}^{(k)}$.
- Approximate $F$ by its tangent plane $T$ at the point $\boldsymbol{V}^{(k)}$.
- Update the guess for the root of $F$ with the root of $T$ at $\boldsymbol{V}^{(k)}$. (The root of $T$ is easy to find since $T =\boldsymbol{0}$ is an affine system.)
- Repeat steps 1-3 until the change in $\boldsymbol{V}$ between iterations is below a certain threshold.
1.2 Dertivation from Taylor Series
The Newton-Raphson method can be deduced from the Taylor series approximation of a function. Suppose $F : \mathbb{R}^d \rightarrow \mathbb{R}^k$ is differentiable at the point $\boldsymbol{V}^{(k)}$. Then, its Taylor series expansion at the point $\boldsymbol{V}^{(k)}$ is given by:
\begin{equation} F (\boldsymbol{V}^{(k + 1)}) = F (\boldsymbol{V}^{(k)}) + J (\boldsymbol{V}^{(k)}) (\boldsymbol{V}^{(k + 1)} -\boldsymbol{V}^{(k)}) + \cdots \tag{2} \end{equation}
If we consider only the terms up to the first-order derivative, then:
\begin{equation} T (\boldsymbol{V}^{(k + 1)}) = F (\boldsymbol{V}^{(k)}) + J (\boldsymbol{V}^{(k)}) (\boldsymbol{V}^{(k + 1)} -\boldsymbol{V}^{(k)}) \tag{3} \end{equation}
which is the equation of the multivariate tangent plane of $F$ at the point $\boldsymbol{V}^{(k)}$.
Similarly, $J (\boldsymbol{V})$, called the Jacobian matrix, is multivariate form of the gradient and is calculated as shown below.
\begin{equation} J (\boldsymbol{x}) = \left[\begin{array}{c} \nabla _{\boldsymbol{V}} f_{1} (\boldsymbol{V})\\\ \vdots \\\ \nabla _{\boldsymbol{V}} f_{k} (\boldsymbol{V}) \end{array}\right] = \left[\begin{array}{ccc} \frac{\partial f_{1}}{\partial V_{1}} & \ldots & \frac{\partial f_{1}}{\partial V_{d}}\\\ \vdots & \ddots & \vdots \\\ \frac{\partial f_{k}}{\partial V_{1}} & \cdots & \frac{\partial f_{k}}{\partial V_{d}} \end{array}\right] \tag{4} \end{equation}
Setting $T (\boldsymbol{V}^{(k + 1)}) =\boldsymbol{0}$ and solving the resulting linear system, we get the roots of $T (\boldsymbol{V}^{(k + 1)}) =\boldsymbol{0}$, which can be used as an approximation for the roots of $F (\boldsymbol{V}) = \boldsymbol{0}$.
\begin{equation} \underbrace{\boldsymbol{V}^{(k + 1)}}_{\text{updated guess}} = \underbrace{\boldsymbol{V}^{(k)}}_{\text{old guess}} - (J (\boldsymbol{V}^{(k)}))^{- 1} F (\boldsymbol{V}^{(k)}) \tag{5} \end{equation}
1.3 Circuit Intrepretation
The system given by (3) is a linearized version of system given by (1) evaluated at $\boldsymbol{V}^{(k)}$. In terms of devices, the tangent is realized by the parallel combination of an independent current source and a conductance.
As an example, consider a diode whose current-voltage characteristics is given by $I = I_{S} (e^{\alpha \cdot V} - 1)$. The Taylor approximation of the current at the point $V^{(k)}$ is given by:
\[ I \approx \underbrace{I_{S} (e^{\alpha \cdot V^{(k)}} - 1)}_{\operatorname{constant}} + \underbrace{\alpha I_{S} e^{\alpha \cdot V^{(k)}}}_{\operatorname{constant}} \cdot (\Delta V) \]
\begin{equation} I \approx \beta _{1} + \beta _{2} \cdot \Delta V \tag{6} \end{equation}
Recall that for a resistor, we have $I = \frac{1}{R} \cdot V$. Therefore, a linearized diode can be represented by a conductance $\beta _{2}$ in parallel with an independent current source $\beta _{1}$.
Figure 1. Linearized diode
2 Limitations
There are several factors that can lead to convergence issues in the Newton-Raphson method:
- Initial guess: If the initial guess is far from the true root or near a point where the function’s derivative is zero, the algorithm may not converge, or it may converge very slowly.
- Ill-conditioned systems of equations: if the function’s derivative is too large in one direction and too small in another direction, the algorithm might overshoot or undershoot the true root, causing instability in the iteration process.
To solve these issues and improve the convergence properties of the Newton-Raphson algorithm, several techniques can be used.
2.1 Gmin Stepping
Gmin stepping is a technique that adds a small conductance (Gmin) in parallel with the non-linear components, effectively reducing their non-linearity. This modification simplifies the system of equations and improves the likelihood of convergence. The value of Gmin is gradually decreased in multiple steps until the original non-linear behavior is restored. At each step, the simulator attempts to find a converged solution, leveraging the solution from the previous step as the initial guess for the next step.
2.2 Source Stepping
Source stepping works by gradually increasing the magnitude of independent voltage or current sources in the circuit, allowing the simulator to find the final solution in smaller, more manageable increments. The rationale behind this technique is that at lower source magnitudes, the non-linear components may operate in regions where their behavior is more linear or more predictable. This simplification can make it easier for the Newton-Raphson method to converge, as the linearization around the operating point becomes more accurate.