Introduction to Constrained Optimal Control

Constrained optimal control problems lie at the heart of modern engineering, economics, and robotics. The essential challenge is to determine a control input that minimizes (or maximizes) a performance criterion—such as energy consumption, time, or cost—while respecting physical, safety, or regulatory constraints. These constraints often take the form of inequalities: bounds on control signals, limits on state variables, or non-negativity of certain quantities. When dynamic equations are nonlinear and constraints are non-smooth, classical calculus of variations and Pontryagin’s maximum principle may become difficult to apply directly. This is where variational inequalities (VIs) provide a powerful alternative. By reformulating optimality conditions as a VI, researchers and practitioners can systematically handle inequality constraints, non-differentiable cost functions, and multi-valued operators. The result is a unified framework that connects control theory, optimization, and equilibrium analysis. This article explores the theory behind VIs, their application to constrained optimal control, key numerical methods, and emerging research directions.

What Are Variational Inequalities?

Formal Definition and Basic Properties

A variational inequality problem, denoted VI(K, F), consists of finding a vector u in a closed convex set K ⊆ ℝn such that

⟨F(u), v − u⟩ ≥ 0 for all v ∈ K,

where F: ℝn → ℝn is a continuous operator. This seemingly simple inequality captures a vast class of equilibrium and optimization problems. When K is the whole space, the condition reduces to F(u) = 0. When K is the nonnegative orthant and F is the gradient of a convex function, VI becomes the Karush–Kuhn–Tucker (KKT) conditions of convex optimization. The core idea is that the solution lies in the feasible set, and any feasible perturbation v − u must point in a direction that does not reduce the value of F (or, geometrically, F(u) is in the normal cone of K at u). Existence and uniqueness results follow from monotonicity and coercivity assumptions on F. For a deeper mathematical treatment, see the classic text by Kinderlehrer and Stampacchia or the Wikipedia entry on variational inequalities.

Connection to Optimization and Complementarity

Variational inequalities are intimately related to complementarity problems. In the special case where K = ℝ+n, the VI condition becomes: u ≥ 0, F(u) ≥ 0, uTF(u) = 0 — a standard complementarity problem. This connection makes VIs a natural tool for problems with inequality constraints in optimal control, where multipliers (costates) are often nonnegative. Many game-theoretic and economic equilibrium models also fit directly into the VI framework. In optimal control, the KKT conditions for a constrained problem can be expressed as a differential variational inequality (DVI), where the state evolves over time with a VI in each time step. This viewpoint unifies the analysis of continuous-time and discrete-time control systems.

Variational Inequalities in Optimal Control

Formulating Constrained Optimal Control Problems

A typical constrained optimal control problem seeks to minimize a cost functional

J(x, u) = φ(x(T)) + ∫0T L(x(t), u(t)) dt

subject to the dynamics

ẋ(t) = f(x(t), u(t)),   x(0) = x0

and pointwise constraints

  • u(t) ∈ U (a closed convex set, e.g., [umin, umax])
  • g(x(t)) ≤ 0 (state inequality constraints)

Classical approaches use Pontryagin’s maximum principle, which yields necessary conditions involving the Hamiltonian H = L + λTf. The optimal control u* minimizes the Hamiltonian pointwise over U. However, when U is complex or the problem includes state constraints, the minimized Hamiltonian condition can be expressed as a VI: find u*(t) ∈ U such that

⟨∂uH(x*(t), u*(t), λ(t)), v − u*(t)⟩ ≥ 0 for all v ∈ U.

This turns the dynamic optimization into a differential algebraic system with a VI. Similarly, state constraints introduce additional multipliers (measures) that turn the adjoint dynamics into a differential variational inequality. The book by Facchinei and Pang on finite-dimensional variational inequalities provides a rigorous foundation, while this SIAM Review article surveys VI methods for optimal control.

Optimality Conditions and VI Reformulation

The advantage of the VI reformulation is twofold. First, it eliminates the need to differentiate nonsmooth functions (like the indicator of U). Second, it enables the use of powerful numerical solvers designed for VIs, such as semismooth Newton methods and interior-point algorithms. For discrete-time problems, the optimality conditions become a finite-dimensional VI at each time step, often solved via complementarity. In continuous time, one obtains a differential variational inequality (DVI). The theory of DVIs extends standard results (existence, uniqueness, sensitivity) to constrained control. For instance, if the Hamiltonian is strongly convex in u and the constraints are convex, the DVI is well-posed and the control can be expressed as a projection onto the feasible set: u* = PU(·), where PU is the Euclidean projection. This projection formula leads to efficient numerical schemes.

Applications

Robotics and Mechanical Systems

Robotic manipulators and autonomous vehicles frequently face inequality constraints: torque limits, collision avoidance, joint angle limits. Variational inequalities naturally model contact forces, friction, and unilateral constraints. For example, when a robot hand grasps an object, the normal forces must be nonnegative (no pulling), and the tangential forces must satisfy Coulomb friction — a second-order cone complementarity condition. In trajectory optimization, VIs combine the dynamics with contact constraints to produce a DVI that can be solved by direct transcription methods (e.g., multiple shooting with VI solver). Recent work in ICRA 2015 showcases how VI-based approaches outperform traditional penalty methods in legged locomotion.

Economics and Game Theory

Dynamic economic models involve budget constraints, production limits, and nonnegativity of consumption. The Nash equilibrium of a differential game can be formulated as a coupled system of VIs. In optimal consumption–savings problems, the Euler equation becomes a VI when borrowing constraints are imposed. The framework also handles environmental and resource economics, where state constraints (like pollution caps) produce optimal policies that are piecewise smooth and can be computed via VI algorithms.

Engineering Design

In process control and chemical engineering, reactors and separation units operate under safety limits (temperature, pressure). Constrained optimal control using VIs allows for rigorous enforcement of these limits while optimizing yield. The VI approach also appears in the control of electrical power systems, where generator limits and line capacities are inequality constraints. The unit commitment problem, a mixed-integer optimization in power systems, has been tackled with VI and complementarity techniques to handle thermal limits and transmission congestion.

Numerical Methods for VI-Constrained Optimal Control

Projection Methods

The simplest numerical approach applies a projected gradient descent to the Hamiltonian minimization. In discrete time, the update for the control is

uk+1 = PU(uk − α ∇uH)

where α is a step size. This method is easy to code but converges slowly for stiff problems. Acceleration techniques such as Nesterov’s method or quasi-Newton updates with projection can improve performance. For convex problems, projection methods are guaranteed to converge to the global optimum.

Interior Point and Penalty Methods

Penalty methods relax constraints by adding a penalty term to the cost. The augmented Lagrangian method (also known as the method of multipliers) is particularly effective. It transforms the constrained optimal control problem into an unconstrained one by introducing Lagrange multiplier updates and a penalty parameter. The resulting unconstrained subproblems can be solved with standard gradient-based optimizers. For state constraints, an interior point approach uses barrier functions to keep the state strictly inside the feasible region. These methods are widely implemented in toolboxes such as Ipopt and CasADi.

Operator Splitting and ADMM

Operator splitting methods, especially the Alternating Direction Method of Multipliers (ADMM), decompose the problem into simpler subproblems: a dynamics propagation step and a constraint projection step. ADMM is attractive for large-scale problems because it can handle non-smoothness and promote sparsity. For VIs, the classical forward–backward splitting method (FBS) applies to monotone operators: one step forward on the dynamics and one backward (proximal) step on the constraint. Douglas–Rachford splitting is another powerful technique for problems where F is the sum of two monotone operators. These methods are well suited for real-time control applications because they can be implemented efficiently on embedded hardware.

Current Research and Future Directions

Recent advances in machine learning have led to the development of learning-based control methods that incorporate VIs. For example, reinforcement learning with constraints (safe RL) formulates the policy optimization as a VI to enforce safety bounds during exploration. Another frontier is the application of VIs to stochastic optimal control, where inequality constraints must hold in probability or expectation. The theory of stochastic variational inequalities is still evolving, with promising results in portfolio optimization and supply chain management.

In the context of hybrid systems (where dynamics switch between modes), VIs elegantly capture the transition conditions. For instance, a bouncing ball can be modeled as a DVI with a complementarity condition at impact. This approach unifies impact mechanics with control design. Meshfree discretization methods, such as the use of neural networks to represent the value function and control, also benefit from VI formulations that handle constraints without explicit grid generation.

Finally, parallel and distributed computing architectures are being exploited to solve large-scale DVIs in real time. The combination of fast solvers (e.g., semismooth Newton) with GPUs has enabled model predictive control (MPC) for systems with thousands of states and inequality constraints, such as building energy management or autonomous driving fleets.

Conclusion

Variational inequalities offer a mathematically elegant and computationally effective framework for solving constrained optimal control problems. By reformulating inequality constraints as VI conditions, one gains access to a wealth of theoretical results (existence, uniqueness, stability) and a suite of robust numerical algorithms. From robotics and economics to process control and power systems, the use of VIs continues to grow as the demands for safe, constraint-aware autonomous systems increase. As research expands into stochastic, learning-based, and hybrid domains, the role of VIs will only become more central. Practitioners and researchers alike should consider adding this versatile tool to their optimization armamentarium.