In control system engineering, the Nyquist plot remains a cornerstone technique for assessing closed-loop stability and frequency-domain performance. By integrating Nyquist plot analysis directly into MATLAB, engineers can automate repetitive calculations, visualize complex system behaviors, and make data-driven design decisions with far greater speed and accuracy than manual methods allow. This article provides an authoritative, expanded guide to combining Nyquist theory with MATLAB’s numerical and graphical capabilities, enabling efficient, production-ready control system design.

Theoretical Foundations of the Nyquist Plot

The Nyquist plot, developed by Harry Nyquist in 1932, maps the open-loop frequency response of a system as a polar plot of G(jω)H(jω) for frequencies ranging from –∞ to +∞. The critical insight is that the number of encirclements of the –1 point in the Nyquist plot directly determines closed-loop stability via the Nyquist stability criterion. In modern practice, this criterion is indispensable for both continuous-time and discrete-time systems.

While Bode plots show magnitude and phase separately, the Nyquist plot preserves the complete complex vector information, making it particularly valuable for systems with non-minimum phase, delays, or resonant peaks. Engineers rely on it to compute gain margin and phase margin—two key robustness metrics—and to identify the frequency at which instability first appears.

For a deeper theoretical primer, refer to the MathWorks Nyquist plot overview.

MATLAB’s Ecosystem for Nyquist Analysis

MATLAB provides the Control System Toolbox as the primary environment for Nyquist analysis. The toolbox includes the core nyquist function, which accepts tf (transfer function), zpk (zero-pole-gain), or ss (state-space) models. Additionally, interactive apps like the Control System Designer allow engineers to drag frequency points on Nyquist plots and automatically adjust compensators.

Beyond the basic function, MATLAB supports:

  • Parameterized Nyquist plotting – Using nyquist with an array of systems to compare variations.
  • NyquistGrid options – Customizing frequency vector, plot style, and stability margins.
  • Discrete-time Nyquist – Using nyquist directly on sysd objects for Z-domain models.
  • MIMO Nyquist analysis – Using sigma and nyquist on individual input-output channels.

The official MathWorks documentation for the nyquist function provides syntax details and examples.

Step-by-Step Integration Workflow

Integrating Nyquist plot analysis into a MATLAB-based design process follows a consistent workflow. The steps below assume you have the Control System Toolbox installed and a system model in hand.

1. Model Definition

Begin by defining the open-loop transfer function. For a continuous-time SISO system, use tf:

s = tf('s');
G = 1 / (s^2 + 0.5*s + 1);

% Alternatively, using zero-pole-gain format:
Gzpk = zpk([], [-0.25+0.9682j, -0.25-0.9682j], 1);

2. Basic Nyquist Plot Generation

Generate the Nyquist plot with a single command:

nyquist(G);
grid on;
title('Nyquist Plot of G(s) = 1/(s^2 + 0.5s + 1)');

The plot displays the complex frequency response from ω = –∞ to +∞. MATLAB automatically scales the axes based on the system dynamics.

3. Extracting Stability Margins

To compute gain margin and phase margin directly from the Nyquist plot, use the margin function:

[Gm, Pm, Wcg, Wcp] = margin(G);
disp(['Gain Margin: ', num2str(Gm), ' dB']);
disp(['Phase Margin: ', num2str(Pm), ' deg']);

These values correspond to the nearest approach of the Nyquist curve to the –1 point. A gain margin above 6 dB and phase margin between 30° and 60° generally indicate robust stability.

4. Customizing the Frequency Range

For systems with widely separated poles and zeros, the automatic frequency vector may miss critical regions. Manually specify the frequency range using logspace or a custom vector:

w = logspace(-3, 3, 500);
[re, im] = nyquist(G, w);
plot(squeeze(re), squeeze(im));
grid on;
xlabel('Real Axis');
ylabel('Imaginary Axis');
title('Customized Nyquist Plot');

5. Stability Check via Encirclements

For open-loop unstable systems, the Nyquist criterion requires counting encirclements. MATLAB’s nyquist function does not automatically count encirclements, but you can programmatically determine the number using the phase characteristic or by directly evaluating the number of crossings of the negative real axis. A simple script to count crossings between –∞ and –1 is:

function N = count_encirclements(re, im)
    % Find indices where imaginary part changes sign
    sign_changes = find(diff(sign(im)) ~= 0);
    % For each sign change, check if real part < -1
    crossings = 0;
    for i = 1:length(sign_changes)
        j = sign_changes(i);
        if re(j) < -1
            crossings = crossings + 1;
        end
    end
    % For a stable closed-loop with no RHP poles, N should be 0
    N = crossings;
end

Advanced Nyquist Techniques in MATLAB

Nyquist for Discrete-Time Systems

Discrete-time Nyquist plots require careful handling of the unit circle. The nyquist function works directly on tf objects with sample time defined. However, the plot wraps the frequency axis from 0 to π radians/sample. For mixed-signal systems, use c2d to discretize the continuous plant at the design sample rate:

Ts = 0.01;
Gd = c2d(G, Ts, 'zoh');
nyquist(Gd);

Nyquist for MIMO Systems

For multiple-input multiple-output (MIMO) systems, the Nyquist plot loses visual clarity because you must plot each input-output pair. MATLAB offers the sigma function (singular values) for MIMO robustness. However, you can still examine individual channels:

% Create a two-input, two-output system
sys_mimo = rss(3,2,2);
% Nyquist for channel (1,1)
nyquist(sys_mimo(1,1));
hold on;
% Overlay other channels as needed

For rigorous MIMO Nyquist analysis, consider using the ν-gap metric or structured singular value (μ).

Parametric Sweeps and Monte Carlo Analysis

To evaluate robustness against parameter uncertainty, sweep over a range of values:

k_range = logspace(-1, 1, 50);
figure;
hold on;
for k = k_range
    sys_k = k * G;
    [re, im] = nyquist(sys_k, {0.1, 100});
    plot(squeeze(re), squeeze(im), 'b');
end
plot(-1, 0, 'ro', 'MarkerSize', 8, 'LineWidth', 2);
axis equal;
grid on;
title('Nyquist Plot Family for Gain Variation');

This approach reveals the region of instability as the gain varies—critical for worst-case analysis in aerospace and automotive applications. The University of Michigan CTMS Nyquist tutorial offers additional visual examples.

Integrating Nyquist with Controller Design

MATLAB’s sisotool (or controlSystemDesigner) provides a graphical environment where you can adjust compensator poles and zeros while watching the Nyquist plot update in real time. This is particularly efficient for cascading lead‑lag compensators or PID tuning. An example session:

sisotool(G);
% Inside the tool: select 'Open-Loop Nyquist' as the plot
% Drag the gain slider to observe margin changes.

Interpreting Non-Standard Nyquist Shapes

Not all Nyquist plots look like simple circles. Common variations include:

  • Infinite gain margin – When the Nyquist curve never crosses the negative real axis (common for first‑order systems). This indicates unconditional stability, but phase margin should still be checked.
  • Multiple encirclements – Systems with right‑half‑plane poles or zeros cause the plot to loop around –1. Count encirclements carefully; MATLAB can be used to trace the unwrapped phase.
  • Time delays – A delay adds a linear phase shift that spirals the Nyquist plot inward. Use the pade approximation or the exact exp(-s*T) delay model in MATLAB’s tf to study stability margins.

When the Nyquist curve approaches –1, the system exhibits near‑oscillatory behavior. The distance from the curve to –1 is proportional to the sensitivity peak—a critical quantity for robust design.

Optimization and Automation Scripts

For large‑scale design studies, wrap Nyquist analysis into a function that automatically computes margins and flags instability. Below is a production‑grade snippet:

function [stable, Gm, Pm] = analyze_nyquist(sys)
    [Gm, Pm] = margin(sys);
    if isinf(Gm)
        stable = true;
    else
        % For systems with finite gain margin, check if gain margin > 0 dB
        stable = (Gm > 1);  % Gm in absolute units
    end
    % Visual check: Nyquist plot with margins
    figure;
    margin(sys);
end

Batch analysis of multiple system variants (e.g., from a parameter sweep) can then be parallelized using MATLAB’s parfor or run in a loop.

Common Pitfalls and How to Avoid Them

Experienced engineers know that Nyquist integration is not without traps:

  • Frequency range too coarse – If the frequency vector is too sparse, sharp peaks near the –1 point are missed. Always verify with a dense logarithmic sweep around the crossover frequency.
  • Scaling issues – For systems with very large or very small gains, MATLAB may clip the plot. Use axis equal and manually set limits.
  • Misunderstanding encirclements – The Nyquist stability criterion requires knowing the number of open‑loop right‑half‑plane poles (P). Failure to provide this information leads to incorrect conclusions.
  • Discrete‑time warping – For discrete systems, the frequency response only spans 0 to π/T. Aliasing can cause plot distortion; use nyquist(Gd, 'phase') to inspect the closed‑loop phase.

Case Study: Compensating a Third‑Order System

Consider a plant with transfer function G(s) = 1/(s^3 + 3s^2 + 3s + 1)—a triple pole at –1. Without compensation, the Nyquist plot passes near –1, yielding a phase margin of only 12°. Using the sisotool approach, a lead compensator C(s) = 10 * (s + 1)/(s + 10) is inserted. The updated Nyquist plot opens a larger phase margin of 48°, and the gain margin exceeds 10 dB. The complete script:

s = tf('s');
G = 1/(s^3 + 3*s^2 + 3*s + 1);
C = 10*(s + 1)/(s + 10);
L = C * G;
nyquist(L);
grid on;
[Gm, Pm] = margin(L)

This demonstrates how integration accelerates iteration: within seconds, you can test multiple compensator structures and immediately visualize stability margins.

Linking Nyquist Results to Time‑Domain Performance

Nyquist plots directly relate to time‑domain metrics. A small phase margin corresponds to high overshoot in the step response, while a gain margin close to 0 dB indicates near‑persistent oscillation. Use MATLAB’s step function after Nyquist analysis to correlate margins with transient behavior:

T = feedback(L, 1);
step(T);
grid on;
title('Closed-Loop Step Response');

This pairing ensures that frequency‑domain stability margins translate to acceptable time‑domain performance—essential for real‑world systems like motor drives, actuator controls, and active damping.

Benefits of Full Integration into MATLAB

Integrating Nyquist analysis into a broader MATLAB workflow offers tangible advantages over using standalone tools:

  • Reproducibility – Script‑based analysis eliminates manual plotting errors and allows version control.
  • Multidomain coupling – Nyquist data can feed into Simulink models for nonlinear validation or hardware‑in‑the‑loop testing.
  • Parameter optimization – Use MATLAB’s fmincon or the Optimization Toolbox to tune controller gains such that Nyquist‑derived margins meet specifications.
  • Reporting automation – Generate publication‑quality plots with print or exportgraphics, and embed margin results in a live script or dashboard.
  • Educational value – Students can interactively experiment with Nyquist plots, building deeper intuition for the stability criterion.

Conclusion

Nyquist plot analysis, when integrated into MATLAB, becomes a powerful accelerator for control system design. By combining a solid theoretical understanding of the Nyquist stability criterion with MATLAB’s numerical and graphical environment, engineers can quickly evaluate stability, optimize margins, and validate designs across a wide range of operating conditions. The workflow presented here—model definition, plot generation, margin extraction, advanced customization, and integration with controller design tools—provides a complete, production‑ready approach. For further reading, the MathWorks video tutorial on Nyquist stability margins offers additional hands‑on insights. Embrace this integration to reduce development time, minimize errors, and deliver robust control systems with confidence.