Disciplined Saddle Programming

Studying saddle problems is a long-standing area of research, resulting in many theoretical insights, numerous algorithms for specific classes of problems, and a large number of applications.

Saddle problems are often studied in the context of minimax or maximin optimization [DM90, DP95], which, while dating back to the 1920s and the work of von Neumann and Morgenstern on game theory [MVN53], continue to be active areas of research, with many recent advancements for example in machine learning [Goo+14]. A variety of methods have been developed for solving saddle point problems, including interior point methods [HT03, Nem99], first-order methods [Kor76, Nem04, Nes07, NO09, CLO13], and second-order methods [NP06, Nes08], where many of these methods are specialized to specific classes of saddle problems. Depending on the class of saddle problem, the methods differ in convergence rate. For example, for the subset of smooth minimax problems, an overview of rates for different curvature assumptions is given in [The+19]. Due to their close relation to Lagrange duality, saddle problems are commonly studied in the context of convex analysis (see, for example, [BV04, §5.4], [Roc70, §33–37], [RW09, §11.J], [BL06, §4.3]), with an analysis via monotone operators given in [RY22].

The practical usefulness of saddle programming in many applications is also increasingly well known. Many applications of saddle programming are robust optimization problems [BBC11, BTEGN09]. For example, in statistics, distributionally robust models can be used when the true distribution of the data generating process is not known [DA19]. Another common area of application is in finance, with [CPT18, §19.3–4] describing a range of financial applications that can be characterized as saddle problems. Similarly, [Boy+17, GI03, LB00] describe variations of the classical portfolio optimization problem as saddle problems.

DCP is a grammar for constructing optimization problems that are provably convex, meaning that they can be solved globally, efficiently and accurately. It is based on the rule that the convexity of a function $f$ is preserved under composition if all inner expressions in arguments where $f$ is nondecreasing are convex, and all expressions where $f$ is nonincreasing are concave, and all other expressions are affine. A detailed description of the composition rule is given in [BV04, §3.2.4]. Using this rule, functions can be composed from a small set of primitives, called atoms, where each atom has known curvature, sign, and monotonicity. Every function that can be constructed from these atoms according to the composition rule is convex, but the converse is not true. The DCP framework has been implemented in many programming languages, including MATLAB [GB14, Lof04], Python [DB16], R [FNB20], and Julia [Ude+14], and is used by researchers and practitioners in a wide range of fields.

As mentioned earlier, disciplined saddle programming is based on Juditsky and Nemirovski’s recent work on well-structured convex-concave saddle point problems [JN22].

• •

  Functions of $x$ or $y$ alone. A convex function of $x$, or a concave function of $y$, are trivial examples of saddle functions.

• •

  Lagrangian of a convex optimization problem. The convex optimization problem

  $$\begin{array}[]{ll}\mbox{minimize}&f_{0}(x)\\ \mbox{subject to}&Ax=b,\quad f_{i}(x)\leq 0,\quad i=1,\ldots,m,\end{array}$$

  with variable $x\in{\mbox{\bf R}}^{n}$, where $f_{0},\ldots,f_{m}$ are convex and $A\in{\mbox{\bf R}}^{p\times n}$, has Lagrangian

  $$L(x,\nu,\lambda)=f(x)+\nu^{T}(Ax-b)+\lambda_{1}f_{1}(x)+\cdots+\lambda_{m}f_{m}(x),$$

  for $\lambda\geq 0$ (elementwise). It is convex in $x$ and affine (and therefore also concave) in $y=(\nu,\lambda)$, so it is a saddle function with

  $$\mathcal{X}=\bigcap_{i=0,\ldots,m}\mathop{\bf dom}f_{i},\qquad\mathcal{Y}={\mbox{\bf R}}^{p}\times{\mbox{\bf R}}_{+}^{m},$$

• •

  Bi-affine function. The function $f(x,y)=(Ax+b)^{T}(Cy+d)$, with $\mathcal{X}={\mbox{\bf R}}^{p}$ and $\mathcal{Y}={\mbox{\bf R}}^{q}$, is evidently a saddle function. The inner product $x^{T}y$ is a special case of a bi-affine function. For a bi-affine function, either variable can serve as the convex variable, with the other serving as the concave variable.

• •

  Convex-concave inner product. The function $f(x,y)=F(x)^{T}G(y)$, where $F:{\mbox{\bf R}}^{p}\to{\mbox{\bf R}}^{n}$ is a nonnegative elementwise convex function and $G:{\mbox{\bf R}}^{q}\to{\mbox{\bf R}}^{n}$ is a nonnegative elementwise concave function.

• •

  Weighted $\ell_{2}$ norm. The function

  $$f(x,y)=\left(\sum_{i=1}^{n}y_{i}x_{i}^{2}\right)^{1/2},$$

  with $\mathcal{X}={\mbox{\bf R}}^{n}$ and $\mathcal{Y}={\mbox{\bf R}}^{n}_{+}$, is a saddle function.

• •

  Weighted log-sum-exp. The function

  $$f(x,y)=\log\left(\sum_{i=1}^{n}y_{i}\exp x_{i}\right),$$

  with $\mathcal{X}={\mbox{\bf R}}^{n}$ and $\mathcal{Y}={\mbox{\bf R}}^{n}_{+}$, is a saddle function.

• •

  Weighted geometric mean. The function $f(x,y)=\prod_{i=1}^{n}y_{i}^{x_{i}}$, with $\mathcal{X}={\mbox{\bf R}}_{+}^{n}$ and $\mathcal{Y}={\mbox{\bf R}}^{n}_{+}$, is a saddle function.

• •

  Quadratic form with quasi-semidefinite matrix. The function

  $$f(x,y)=\left[\begin{array}[]{c}x\\ y\end{array}\right]^{T}\left[\begin{array}[]{cc}P&S\\ S^{T}&Q\end{array}\right]\left[\begin{array}[]{c}x\\ y\end{array}\right],$$

  where the matrix is quasi-semidefinite, i.e., $P\in{\mbox{\bf S}}_{+}^{n}$ (the set of symmetric positive semidefinite matrices) and $-Q\in{\mbox{\bf S}}_{+}^{n}$.

• •

  Quadratic form. The function $f(x,Y)=x^{T}Yx$, with $\mathcal{X}={\mbox{\bf R}}^{n}$ and $\mathcal{Y}={\mbox{\bf S}}_{+}^{n}$ (the set of symmetric positive semidefinite $n\times n$ matrices), is a saddle function.

• •

  As a more esoteric example, the function $f(x,Y)=x^{T}Y^{1/2}x$, with $\mathcal{X}={\mbox{\bf R}}^{n}$ and $\mathcal{Y}={\mbox{\bf S}}_{+}^{n}$, is a saddle function.

Saddle functions can be combined in several ways to yield saddle functions. For example the sum of two saddle functions is a saddle function, provided the domains have nonempty intersection. A saddle function scaled by a nonnegative scalar is a saddle function. Scaling a saddle function with a nonpositive scalar, and swapping its arguments, yields a saddle function: $g(x,y)=-f(y,x)$ is a saddle function provided $f$ is. Saddle functions are preserved by pre-composition of the convex and concave variables with an affine function, i.e., if $f$ is a saddle function, so is $f(Ax+b,Cx+d)$. Indeed, the bi-affine function is just the inner product with an affine pre-composition for each of the convex and concave variables.

• •

  Matrix game. In a matrix game, player one chooses $i\in\{1,\ldots,m\}$, and player two chooses $j\in\{1,\ldots,n\}$, resulting in player one paying player two the amount $C_{ij}$. Player one wants to minimize this payment, while player two wishes to maximize it. In a mixed strategy, player one makes choices at random, from probabilities given by $x$ and player two makes independent choices with probabilities given by $y$. The expected payment from player one to player two is then $f(x,y)=x^{T}Cy$. With $\mathcal{X}=\{x\mid x\geq 0,\;\mathbf{1}^{T}x=1\}$, and similarly for $\mathcal{Y}$, a saddle point corresponds to an equilibrium, where no player can improve her position by changing (mixed) strategy. The saddle point problem consists of finding a stable equilibrium, i.e., an optimal mixed strategy for each player.

• •

  Lagrangian. A saddle point of a Lagrangian of a convex optimization problem is a primal-dual optimal pair for the convex optimization problem.

• •

  Dual function. Minimizing a Lagrangian $L(x,\nu,\lambda)$ over $x$ gives the dual function of the original convex optimization problem.

• •

  Maximizing a Lagrangian $L(x,\nu,\lambda)$ over $y=(\nu,\lambda)$ gives the objective function restricted to the feasible set.

• •

  Conjugate of a convex function. Suppose $f$ is convex. Then $g(x,y)=f(x)-x^{T}y$ is a saddle function, the Lagrangian of the problem of minimizing $f$ subject to $x=0$. Its saddle min is the negative conjugate function: $\inf_{x}g(x,y)=-f^{*}(y)$.

• •

  Sum of $k$ largest entries. Consider $f(x,y)=x^{T}y$, with $\mathcal{Y}=\{y\mid 0\leq y\leq 1,\;\mathbf{1}^{T}y=k\}$. The associated saddle max function $G$ is the sum of the $k$ largest entries of $x$.

A pair $(x^{\star},y^{\star})$ is a saddle point of a saddle function $f$ if and only if $x^{\star}$ minimizes the convex SE function $G$ in (3) over $x\in\mathcal{X}$, and $y^{\star}$ maximizes the concave SE function $H$ defined in (4) over $y\in\mathcal{Y}$. This means that we can find saddle points, i.e., solve the saddle point problem (2), by solving the convex optimization problem

with variable $x$, and the convex optimization problem

with variable $y$. The problem (5) is called a minimax problem, since we are minimizing a function defined as the maximum over another variable. The problem (6) is called a maximin problem.

While the minimax problem (5) and maximin problem (6) are convex, they cannot be directly solved by conventional methods, since the objectives themselves are defined by maximization and minimization, respectively. There are solution methods specifically designed for minimax and maximin problems [LJJ20, MB09], but as we will see minimax problems involving SE functions can be transformed to equivalent forms that can be directly solved using conventional methods.

As a more specific example of a saddle problem consider the linear program with robust cost,

with variable $x\in{\mbox{\bf R}}^{n}$, with $\mathcal{C}=\{c\mid Fc\leq g\}$. This is an LP with worst case cost over the polyhedron $\mathcal{C}$ [BBC11, BTEGN09]. This is a saddle problem with convex variable $x$, concave variable $y$, and an objective which is a saddle max function.

There are cases where an SE function can be worked out analytically. An example is the max of a linear function over a box,

where the absolute value is elementwise. We will see other cases in our examples.

We can readily compute a subgradient of a saddle max function (or a supergradient of a saddle min function) at a given input, by simply maximizing over the concave variable (minimizing over the convex variable), which is itself a convex optimization problem, and then obtaining a subgradient (supergradient) at that maximizer (minimizer). We can then use any method to solve the saddle problem using these subgradients, e.g., subgradient-type methods, ellipsoid method, or localization methods such as the analytic center cutting plane method. In [MB09] such an approach is used for general minimax problems.

Many methods have been developed for finding saddle points of saddle functions with the special form

where $\phi$ is convex, $\psi$ is concave, and $K$ is a matrix [BS15, Con13, CP11, Nes05, Nes05a, CP16]. Beyond this example, there are many other special forms of saddle functions, with different methods adapted to properties such as smoothness, separability, and strong-convex-strong-concavity.

We illustrate the dualization method for the robust cost LP (8). The key is to express the robust cost or saddle max function $\sup_{Fc\leq g}c^{T}x$ as an infimum. We first observe that this saddle max function is the optimal value of the LP

with variable $c$. Its dual is

with variable $\lambda$. With $\mathcal{C}=\{c\mid Fc\leq g\}$, and assuming $\mathcal{C}$ is nonempty, this dual problem has the same optimal value as the primal, i.e.,

Substituting this into (8) we obtain the problem

with variables $x$ and $\lambda$. This simple LP is equivalent to the original robust LP (8), in the sense that if $(x^{\star},\lambda^{\star})$ is a solution of (9), then $x^{\star}$ is a solution of the robust LP (8).

We will see this dualization trick in a far more general setting in §4.

We use the notation $\phi(x,y):\mathcal{X}\times\mathcal{Y}\subseteq{\mbox{\bf R}}^{n\times m}\to{\mbox{\bf R}}$ to denote a saddle function which is convex in $x$ and concave in $y$. Let $K_{x}$, $K_{y}$ and $K$ be members of a collection $\mathcal{K}$ of closed, convex, and pointed cones with nonempty interiors in Euclidean spaces such that $\mathcal{K}$ contains a nonnegative ray, is closed with respect to taking finite direct products of its members, and is closed with respect to passing from a cone to its dual. We denote conic membership $z\in K$ by $z\succeq_{K}0$. We call a set $\mathcal{X}\subseteq{\mbox{\bf R}}^{n}$ $\mathcal{K}$-representable if there exist constant matrices $A$ and $B$, a constant vector $c$, and a cone $K\in\mathcal{K}$ such that

CVXPY [DB16] can implement a function $f$ exactly when its epigraph $\{(x,u)\mid f(x)\leq u\}$ is $\mathcal{K}$-representable.

Let $\mathcal{X}$ and $\mathcal{Y}$ be nonempty and possessing $\mathcal{K}$-representations

A saddle function $\phi(x,y):\mathcal{X}\times\mathcal{Y}\to{\mbox{\bf R}}$ is $\mathcal{K}$-representable if there exist constant matrices $P$, $Q$, $R$, constant vectors $p$ and $s$ and a cone $K\in\mathcal{K}$ such that for each $x\in\mathcal{X}$ and $y\in\mathcal{Y}$,

Here $f$ is a vector of the same dimension as $y$, $t$ is a scalar, and $u$ is a vector. This definition generalizes simple class of bilinear saddle functions. See [JN22] for much more detail.

Suppose we have a $\mathcal{K}$-representable saddle function $\phi$ as above. The conic form allows us to derive a dualized representation of the saddle extremum function

which again admits a tractable conic form, meaning that it can be represented in a DSL like CVXPY. Specifically,

where in (4.2) we use Sion’s minimax theorem [Sio58] to reverse the inf and sup, and in (15) we invoke strong duality to replace the supremum over $y$ with an infimum over $\lambda$. Concretely, strong duality and the conic structure allow us to equate

where $K^{*}$ is the dual cone of $K$. This is exactly the automated dualization made possible by the conic representable form of $\phi$ (which DSP provides). Given the conic representation of $\phi$, the dualized form is obtained via the explicit formula given in (15).

The final line implies a conic representation of the epigraph of $\Phi(x)$,

which is tractable and can be implemented in a DSL like CVXPY. This transformation is exact, and so there is no notion of approximation error or optimality gap arising from the dualization procedure.

Switching the $\inf$ and $\sup$ in (4.2) requires Sion’s theorem to hold. A sufficient condition for Sion’s theorem to hold is that the set $\mathcal{Y}$ is compact. However, the min and max can be exchanged even if $\mathcal{Y}$ is not compact. Then, due to the max-min inequality

the equality in (15) is replaced with a less than or equal to, and we obtain a convex restriction. Thus, if a user creates a problem involving an SE function (as opposed to a saddle point problem only containing saddle functions in the objective), then DSP guarantees that the problem generated is a restriction. This means that the variables returned are feasible and the returned optimal value is an upper bound on the optimal value for the user’s problem.

One challenge that arises in transforming the mathematical concept of conic representable saddle functions into a practical implementation is that the automated dualization removes the concave variable from the problem. Additionally, the procedure relies on the technical conditions such as compactness, which we believe should not be exposed in a user interface. We now address these points.

In our implementation, a saddle problem with an SE function in the objective is solved by applying the above automatic dualization to both the objective $\phi$ and $-\phi$ and then solving each resulting convex problem. Note that $\phi(x,y)$ is convex in $x$ and concave in $y$, while $-\phi(x,y)$ is concave in $x$ and convex in $y$. We do so in order to obtain both the convex and concave components of the saddle point, since the dualization removes the concave variable. To see this, note that (15) contains $x$ but not $y$ (and the opposite holds for the negated problem). The saddle problem is only reported as solved if the optimal value of the problem with objective $\phi$, $u$, is within a numerical tolerance of the negated optimal value of the problem with objective $-\phi$, $-l$. If this holds, this actually implies that

i.e., (4.2) was valid, even if for example $\mathcal{Y}$ is not compact. To see this, note that solving for $\phi$ as well as $-\phi$ results in an upper and a lower bound on the optimal value of the saddle point problem,

Using symmetry and combining the above inequalities, we obtain

Suppose now that $l=u$. Note that since

we have that $\phi(x^{\star},y^{\star})=l=u$. That is, the pair $(x^{\star},y^{\star})$ obtains the optimal value of the saddle point problem. All that remains is to verify that this pair satisfies the saddle point property. We have that $\phi(x^{\star},y)\leq\phi(x^{\star},y^{\star})$ for all $y\in\mathcal{Y}$, since otherwise $u=\phi(x^{\star},y^{\star})<\max_{y\in\mathcal{Y}}\phi(x^{\star},y)=u$, a contradiction. Similarly, $\phi(x,y^{\star})\geq\phi(x^{\star},y^{\star})$ for all $x\in\mathcal{X}$. Taken together, these inequalities state that $(x^{\star},y^{\star})$ is a saddle point, since

Thus, a user need not concern themselves with the compactness of $\mathcal{Y}$ (or any other sufficient condition for Sion’s theorem) when using DSP to find a saddle point; if a saddle point problem is solved, then the saddle point property is guaranteed to hold. This mathematical insight extends the work of [JN22], which assumes compactness of $\mathcal{Y}$, allowing users who might be unfamiliar with this technical restriction to use DSP.

The atom inner(x,y) represents the inner product $x^{T}y$. Since either $x$ or $y$ could represent the convex variable, we adopt the convention in DSP that the first argument of inner is the convex variable. According to the DSP rules, both arguments to inner must be affine, and the variables they depend on must be disjoint.

The atom saddle_inner(F, G) corresponds to the function $F(x)^{T}G(y)$, where $F$ and $G$ are vectors of nonnegative and respectively elementwise convex and concave functions. It is DSP-compliant if $F$ is DCP convex and nonnegative and $G$ is DCP concave. If the function $G$ is not DCP nonnegative, then the DCP constraint G >= 0 is attached to the expression. This is analogous to how the DCP constraint x >= 0 is added to the expression cp.log(x). As an example consider

f = saddle_inner(cp.square(x), cp.log(y)).

This represents the saddle function

where $I$ is the $\{0,\infty\}$ indicator function of its argument.

The weighted_norm2(x, y) atom represents the saddle function $\left(\sum_{i=1}^{n}y_{i}x_{i}^{2}\right)^{1/2}$, with $y\geq 0$. It is DSP-compliant if x is either DCP affine or both convex and nonnegative, and $y$ is DCP concave. Here too, the constraint y >= 0 is added if $y$ is not DCP nonnegative.

The weighted_log_sum_exp(x, y) atom represents the saddle function $\log\left(\sum_{i=1}^{n}y_{i}\exp x_{i}\right)$, with $y\geq 0$. It is DSP-compliant if x is DCP convex, and $y$ is DCP concave. The constraint y >= 0 is added if $y$ is not DCP nonnegative.

The quasidef_quad_form(x, y, P, Q, S) atom represents the function

where the matrix is quasi-semidefinite, i.e., $P\in{\mbox{\bf S}}_{+}^{n}$ and $-Q\in{\mbox{\bf S}}_{+}^{n}$. It is DSP-compliant if $x$ is DCP affine and $y$ is DCP affine.

The saddle_quad_form(x, Y) atom represents the function $x^{T}Yx$, where $Y$ is a PSD matrix. It is DSP-compliant if $x$ is DCP affine, and $Y$ is DCP PSD.

The DSP rules prohibit mixing of convex and concave variables. For example if we add two saddle expressions, no variable can appear in both its convex and concave variable lists.

Recall that if an expression is DCP convex (concave), then it is convex (concave), but the converse is false. For example, the expression cp.sqrt(1 + cp.square(x)) represents the convex function $\sqrt{1+x^{2}}$, but is not DCP. But we can express the same function as cp.norm2(cp.hstack([1, x])), which is DCP. The same holds for DSP and saddle function: If an expression is DSP-compliant, then it represents a saddle function; but it can represent a saddle function and not be DSP-compliant. As with DCP, such an expression would need to be rewritten in DSP-compliant form, to use any of the other features of DSP (such as a solution method). As an example, the expression x.T @ C @ y represents the saddle function $x^{T}Cy$, but is not DSP-compliant. The same function can be expressed as inner(x, C @ y), which is DSP-compliant. While this restrictive syntax is an inherent limitation of disciplined convex programming in general, it is required for any parser based on the DSP composition rules.

When there are affine variables in a DSP-compliant expression, it means that those variables could be considered either convex or concave; either way, the function is a saddle function.

The code below defines the bi-linear saddle function $f(x,y)=x^{T}Cy$, the objective of a matrix game, with $x$ the convex variable and $y$ the concave variable.

Lines 1–3 import the necessary packages (which we will use but not show in the sequel). In lines 5–7, we create two CVXPY variables and a constant matrix. In line 9 we construct the saddle function f using the DSP atom inner. Both its arguments are affine, so this matches the DSP rules. In line 11 we check if saddle_function is DSP-compliant, which it is. In lines 13–15 we call functions that return lists of the convex, concave, and affine variables, respectively. The results of lines 13–15 might seem odd, but recall that inner marks its first argument as convex and its second as concave.

To construct a saddle point problem, we first create an objective using

obj = MinimizeMaximize(f),

where f is a CVXPY expression. The objective obj is DSP-compliant if the expression f is DSP-compliant. This is analogous to the CVXPY contructors cp.Minimize(f) and cp.Maximize(f), which create objectives from expressions.

A saddle point problem is constructed using

prob = SaddlePointProblem(obj, constraints, cvx_vars, ccv_vars)

Here, obj is a MinimizeMaximize objective, constraints is a list of constraints, cvx_vars is a list of convex variables and ccv_vars is a list of concave variables. The objective must be DSP-compliant for the problem to be DSP-compliant. We now describe the remaining conditions under which the constructed problem is DSP-compliant.

Each constraint in the list must be DCP, and can only involve convex variables or concave variables; convex and concave variables cannot both appear in any one constraint. The list of convex and concave variables partitions all the variables that appear in the objective or the constraints. In cases where the role of a variable is unambiguous, it is inferred, and does not need to be in either list. For example with the objective

MinimizeMaximize(weighted_log_sum_exp(x, y) + cp.exp(u) + cp.log(v) + z),

x and u must be convex variables, and y and v must be concave variables, and so do not need to appear in the lists used to construct a saddle point problem. The variable z, however, could be either a convex or concave variable, and so must appear in one of the lists.

The role of a variable can also be inferred from the constraints: Any variable that appears in a constraint with convex (concave) variables must also be convex (concave). With the objective above, the constraint z + v <= 1 would serve to classify z as a concave variable. With this constraint, we could pass empty variable lists to the saddle point constructor, since the roles of all variables can be inferred. When the roles of all variables are unambiguous, the lists are optional.

The roles of the variables in a saddle point problem prob can be found by calling prob.convex_variables() and prob.concave_variables(), which return lists of variables, and is a partition of all the variables appearing in the objective or constraints. This is useful for debugging, to be sure that DSP agrees with you about the roles of all variables. A DSP-compliant saddle point problem must have an empty list of affine variables. (If it did not, the problem would be ambiguous.)

The solve() method of a SaddlePointProblem object canonicalizes and solves the problem. This involves checking the objective and constraints for DSP-compliance. The conic representation of the problem is obtained, which involves setting up an auxiliary problem and compiling it using CVXPY. Then, the dualization is carried out, which results in another CVXPY problem which is then solved to yield the objective value. This has the side effect of setting all convex variables’ .value attribute. To also obtain the values of the concave variables, the saddle point problem is solved again with a negated objective and the roles of the minimization and maximization variables reversed. We emphasize that as DSP acts as a compiler, it does not implement any optimization algorithms itself, but rather relies on the solvers accessible through CVXPY.

Here we create and solve a matrix game, continuing the example above where f was defined. We do not need to pass in lists of variables since their roles can be inferred.

An SE function has one of the forms

where $f$ is a saddle function. Note that $y$ in the definition of $G$, and $x$ in the definition of $H$, are local or dummy variables, understood to have no connection to any other variable. Their scope extends only to the definition, and not beyond.

To express this subtlety in DSP, we use the class LocalVariable to represent these dummy variables. The variables that are maximized over (in a saddle max function) or minimized over (in a saddle min function) must be declared using the LocalVariable() constructor. Any LocalVariable in an SE function cannot appear in any other SE function.

We construct SE functions in DSP using

saddle_max(f, constraints)  or  saddle_min(f, constraints).

Here, f is a CVXPY scalar expression, and constraints is a list of constraints. We now describe the rules for constructing a DSP-compliant SE function.

If a saddle_max is being constructed, f must be DSP-compliant, and the function’s concave variables, and all variables appearing in the list of constraints, must be LocalVariables, while the function’s convex variables must all be regular Variables. A similar rule applies for saddle_min.

The list of constraints is used to specify the set over which the sup or inf is taken. Each constraint must be DCP-compliant, and can only contain LocalVariables.

With x a Variable, y_loc a LocalVariable, z_loc a LocalVariable, and z a Variable, consider the following two SE functions:

Both are DSP-compliant. For the first, calling f_1.convex_variables() would return [x, z], and calling f_1.concave_variables() would return [y_loc]. For the second, calling f_2.convex_variables() would return [x], and f_2.concave_variables() return [y_loc, z_loc].

Let y be a Variable. Both of the following are not DSP-compliant:

The first is not DSP-compliant because z is not a LocalVariable, but appears in the constraints. The second is not DSP-compliant because y is not a LocalVariable, but appears as a concave variable in the saddle function.

When they are DSP-compliant, a saddle_max is a convex function, and a saddle_min is a concave function. They can be used anywhere in CVXPY that a convex or concave function is appropriate. You can add them, compose them (in appropriate ways), use them in the objective or either side of constraints (in appropriate ways).

Now we provide full examples demonstrating construction of a saddle_max, which we can use to solve the matrix game described in §5.3 as a saddle problem involving an SE function.

Note that G is a CVXPY expression. Constructing a saddle_min works exactly the same way.

We continue our example from §5.4 and solve the matrix game using either a saddle max.