Sum-of-squares hierarchy lower bounds
for symmetric formulations^††thanks: Supported by the Swiss National Science Foundation project 200020-144491/1 “Approximation Algorithms for Machine Scheduling Through Theory and Experiments” and by Sciex Project 12.311.

Proving lower bounds for the Sum-of-Squares (SoS)/Lasserre hierarchy [25, 31] has attracted notable attention in the theoretical computer science community during the last decade, see e.g. [7, 12, 13, 18, 19, 27, 28, 29, 34, 36]. This is partly because the hierarchy captures many of the best known approximation algorithms based on semidefinite programming (SDP) for several natural 0/1 optimization problems (see [28] for a recent result). Indeed, it can be argued that the SoS hierarchy is the strongest candidate to be the “optimal” meta-algorithm predicted by the Unique Games Conjecture (UGC) [22, 32]. On the other hand, the hierarchy is also one of the best known candidates for refuting the conjecture since it is still conceivable that one could show that the SoS hierarchy achieves better approximation guarantees than the UGC predicts (see [6] for discussion). Despite the interest in the algorithm and due to the many technical challenges presented by semidefinite programming, only relatively few techniques are known for proving lower bounds for the hierarchy. In particular, several integrality gap results follow from applying gadget reductions to the few known original lower bound constructions.

Indeed, many of the known lower bounds for the SoS hierarchy originated in the works of Grigoriev [18, 19].¹¹1More precisely, Grigoriev considers the positivstellensatz proof system, which is the dual of the SoS hierarchy considered in this paper. For brevity, we will use SoS hierarchy/proof system interchangeably. We defer the formal definition of the hierarchy for later and only point out that solving the hierarchy after $t$ rounds takes $n^{O(t)}$ time. In [19] Grigoriev showed that random 3Xor or 3Sat instances cannot be solved even by $\Omega(n)$ rounds of the SoS hierarchy (some of these results were later independently rediscovered by Schoenebeck [34]). Lower bounds, such as those of [7, 36] rely on [19, 34] combined with gadget reductions. Another important lower bound was also given by Grigoriev [18] for the Knapsack problem (a simplified proof can be found in [20]), showing that the SoS hierarchy cannot prove within $\lfloor n/2\rfloor$ rounds that the polytope $\{x\in[0,1]^{n}:\sum_{i=1}^{n}x_{i}=n/2\}$ contains no integer point when $n$ is odd. Using essentially the same construction as in [20], Laurent [27] independently showed that $\lfloor\frac{n}{2}\rfloor$ rounds are not enough for finding the integer cut polytope of the complete graph with $n$ nodes, where $n$ is odd (this result was recently shown to be tight in [15]).²²2The two problems, Knapsack and Max-Cut in complete graphs, considered respectively in [18, 20] and in [27], are essentially the same and we will use Max-Cut to refer to both. By using several new ideas and techniques, but a similar starting point as in [20, 27], Meka, Potechin and Wigderson [29] were able to show a lower bound of $\Omega(\log^{1/2}n)$ for the Planted-Clique problem. Common to the works [20, 27] and [29] is that the matrix involved in the analysis has a large kernel, and they prove that a principal submatrix is positive definite by applying the theory of association schemes [16]. It is also interesting to point out that for the class of Max-CSPs, Lee, Raghavendra and Steurer [28] proved that the SoS relaxation yields the “optimal” approximation, meaning that SDPs of polynomial-size are equivalent in power to those arising from $O(1)$ rounds of the SoS relaxations. Then, by appealing to the result by Grigoriev/Laurent [18, 27] they showed an exponential lower bound on the size of SDP formulations for the integer cut polytope. For different techniques to obtain lower bounds, we refer for example to the recent papers [5, 23, 24] (see also Section 5.4) and the survey [13] for an overview of previous results.

In this paper we introduce a method for proving SoS hierarchy lower bounds when the initial problem formulation exhibits a high degree of symmetry. Our main technical theorem (Theorem 1) allows us to reduce the study of the positive semidefiniteness to the analysis of “well-behaved” univariate polynomial inequalities. The theorem applies whenever the solution and constraints are symmetric, informally meaning that all subsets of the variables of equal cardinality play the same role in the formulation (see Section 3 for the formal definition). For example, the solution in [18, 20, 27] for Max-Cut is symmetric in this sense.

We note that exploiting symmetry reduces the number of variables involved in the analysis, and different ways of utilizing symmetry have been widely used in the past for proving integrality gaps for different hierarchies, see for example [8, 17, 19, 21, 24, 35]. An interesting difference of our approach from others is that we establish several lower bounds without fully identifying the formula of eigenvectors. More specifically, the common task in this context is to identify the spectral structure to get a simple diagonalized form. In the previous papers the moment matrices belong to the Bose-Mesner algebra of a well-studied association scheme, and hence one can use the existing theory. In this paper, instead of identifying the spectral structure completely, we identify only possible forms and propose to test all the possible candidates. This is in fact an important point, since the approach may be extended even if the underlying symmetry is imperfect or its spectral property is not well understood.

The proof of Theorem 1 is obtained by a sequence of elementary operations, as opposed to notions such as big kernel in the matrix form, the use of interlacing eigenvalues, the machinery of association schemes and various results about hyper-geometric series as in [18, 20, 27]. Thus Theorem 1 applies to the whole class of symmetric solutions, even when several conditions and machinery exploited in [18, 20, 27] cannot be directly applied. For example the kernel dimension, which was one of the important key property used to prove the results in  [18, 20, 27], depends on the particular solution that is used and it is not a general property of the class of symmetric solutions. The solutions for two problems considered in this paper have completely different kernel sizes of the analyzed matrices, one large and the other zero.

We demonstrate the technique with two illustrative and complementary applications. First, we show that the analysis of the lower bound for Max-Cut in [18, 20, 27] simplifies to few elementary calculations once the main theorem is in place. This result is partially motivated by the open question posed by O’Donnell [30] of finding a simpler proof for Grigoriev’s lower bound for the Knapsack problem.

As a second application we consider a constrained problem. We show that after $\Omega(\log^{1-\epsilon}n)$ levels the SoS hierarchy does not improve the integrality gap of $2$ for the Min-knapsack linear program formulation strengthened with cover inequalities [10] introduced by Wolsey [37]. Adding cover inequalities is currently the most successful approach for capacitated covering problems of this type [1, 2, 3, 9, 11].

Our result is the first SoS lower bound for formulations with cover inequalities. In this application we demonstrate that our technique can also be used for suggesting the solution and for analyzing its feasibility.

Finally we point it out that the same analysis can be used to provide a non trivial lower bound to an open question raised by Laurent [26] regarding the Lasserre rank of the knapsack problem (see Section 5.4 for a discussion).

Consider a $0/1$ optimization problem with $m\geq 0$ linear constraints $g_{\ell}(x)\geq 0$, for $\ell\in[m]$ and $x\in\mathbb{R}^{n}$. We are interested in approximating the convex hull of the integral points of the set $K=\left\{x\in\mathbb{R}^{n}~|~g_{\ell}(x)\geq 0,\forall\ell\in[m]\right\}$ with the SoS hierarchy defined in the following.

The form of the SoS hierarchy we use in this paper (Definition 1) is equivalent to the one used in literature (see e.g. [4, 25, 26]). It follows from applying a change of basis to the dual certificate of the refutation of the proof system [26] (see also [29] for discussion on the connection to the proof system). We use this change of basis in order to obtain a useful decomposition of the moment matrices as a sum of rank one matrices of special kind. This will play an important role in our analysis. We refer the reader to Appendix A for more details and for a mapping between the different forms.

For any $I\subseteq N=\{1,\ldots,n\}$, let $x_{I}$ denote the $0/1$ solution obtained by setting $x_{i}=1$ for $i\in I$, and $x_{i}=0$ for $i\in N\setminus I$. We denote by $g_{\ell}(x_{I})$ the value of the constraint evaluated at $x_{I}$. For each integral solution $x_{I}$, where $I\subseteq N$, in the SoS hierarchy defined below there is a variable $y^{N}_{I}$ that can be interpreted as the “relaxed” indicator variable for the solution $x_{I}$. We point out that in this formulation of the hierarchy the number of variables $\{y^{N}_{I}:I\subseteq N\}$ is exponential in $n$, but this is not a problem in our context since we are interested in proving lower bounds rather than solving an optimization problem.

Let $\mathcal{P}_{t}(N)$ be the collection of subsets of $N$ of size at most $t\in\mathbb{N}$. For every $I\subseteq N$, the $q$-zeta vector $Z_{I}\in\mathbb{R}^{\mathcal{P}_{q}(N)}$ is a $0/1$ vector with $J$-th entry ($|J|\leq q$) equal to $1$ if and only if $J\subseteq I$.³³3In order to keep the notation simple, we do not emphasize the parameter $q$ as the dimension of the vectors should be clear from the context. Note that $Z_{I}Z_{I}^{\top}$ is a rank one matrix and the matrices considered in Definition 1 are linear combinations of these rank one matrices.

It is straightforward to see that the SoS hierarchy formulation given in Definition 1 is a relaxation of the integral polytope. Indeed consider any feasible integral solution $x_{I}\in K$ and set $y^{N}_{I}=1$ and the other variables to zero. This solution clearly satisfies Condition (1), Condition (2) because the rank one matrix $Z_{I}Z_{I}^{\top}$ is positive semidefinite (PSD), and Condition (3) since $x_{I}\in K$.

The main result of this paper (see Theorem 1 below) allows us to reduce the study of the positive semidefiniteness for matrices (2) and (3) to the analysis of “well-behaved” univariate polynomial inequalities. It can be applied whenever the solutions and constraints are symmetric, namely they are invariant under all permutations $\pi$ of the set $N$: $z_{I}^{N}=z^{N}_{\pi(I)}$ for all $I\subseteq N$ (equivalently when $z_{I}^{N}=z^{N}_{J}$ whenever $|I|=|J|$),⁴⁴4We define the set-valued permutation by $\pi(I)=\left\{\pi(i)~|~i\in I\right\}$. where $z_{I}^{N}$ is understood to denote either $y^{N}_{I}$ or $g_{\ell}(x_{I})y^{N}_{I}$. For example, the solution for Max-Cut considered by Grigoriev [18] and Laurent [27] belongs to this class.

Note that polynomial $G_{h}(k)$ in (6) is nonnegative in a real interval, and in (5) it is zero for a finite set of integers. Moreover, constraints (7) are trivially satisfied for $h>\lfloor n/2\rfloor$.

Theorem 1 is a actually a corollary of a technical theorem that is not strictly necessary for the applications of this paper, and therefore deferred to a later section (see Theorem 7 in Section 6). The proof (given in Section 6) is obtained by exploiting the high symmetry of the eigenvectors of the matrix appearing in (8). Condition (7) corresponds to the requirement that the Rayleigh quotient being non-negative restricted to some highly symmetric vectors (which we show are the only ones we need to consider).

In the Max-Cut problem, we are given an undirected graph and we wish to find a partition of the vertices (a cut) which maximizes the number of edges whose endpoints are on different sides of the partition (cut value). For the complete graph with $n$ vertices, consider any solution with $\omega$ vertices on one side and the remaining $n-\omega$ on the other side of the partition. This gives a cut of value $\omega(n-\omega)$. When $n$ is odd and for any $\omega\leq n/2$, Grigoriev [18] and Laurent [27] considered the following solution (reformulated in the basis considered in Definition 1, see Appendix B):

It is shown [18, 27] that (9) is a feasible solution for the SoS hierarchy of value $\omega(n-\omega)$, for any $\omega\leq n/2$ up to round $t\leq\lfloor\omega\rfloor$. In particular for $\omega=n/2$ the cut value of the SoS relaxation is strictly larger than the value of the optimal integral cut (i.e. $\lfloor\frac{n}{2}\rfloor(\lfloor\frac{n}{2}\rfloor+1)$), showing therefore an integrality gap at round $\lfloor n/2\rfloor$.

We note that the formula for the solution (9) is essentially implied by the requirement of having exactly $\omega$ vertices on one side of the partition (see [18, 27] and [29] for more details) and the core of the analysis in [18, 27] is in showing that (9) is a feasible solution for the SoS hierarchy. By taking advantage of Theorem 1, the proof that (9) is a feasible solution for the SoS relaxation follows by observing the fact below.

Now by Lemma 2 we have $\sum_{k=0}^{n}y^{N}_{k}\binom{n}{k}G_{h}(k)=G_{h}(\omega)$ and the feasibility of (9) follows by Theorem 1, since we have that $G_{h}(\omega)\geq 0$ whenever $t\leq\omega$ for $\omega\leq n/2$.

The Min-Knapsack problem is defined as follows: we have $n$ items with costs $c_{i}$ and profits $p_{i}$, and we want to choose a subset of items such that the sum of the costs of the selected items is minimized and the sum of the profits is at least a given demand $P$. Formally, this can be formulated as an integer program $(IP)~\min\left\{\sum_{j=1}^{n}c_{j}x_{j}:\\ \sum_{j=1}^{n}p_{j}x_{j}\geq P,x\in\{0,1\}^{n}\right\}$. It is easy to see that the natural linear program $(LP)$, obtained by relaxing $x\in\{0,1\}^{n}$ to $x\in[0,1]^{n}$ in $(IP)$, has an unbounded integrality gap.

By adding the Knapsack Cover (KC) inequalities introduced by Wolsey [37] (see also [10]), the arbitrarily large integrality gap of the natural LP can be reduced to 2 (and it is tight [10]). The KC constraints are as follows: $\sum_{j\not\in A}p^{A}_{j}x_{j}\geq P-p(A)$ for all $A\subseteq N$, where $p(A)=\sum_{i\in A}p_{i}$ and $p^{A}_{j}=\min\left\{p_{j},P-p(A)\right\}$. Note that these constraints are valid constraints for integral solutions. Indeed, in the “integral world” if a set $A$ of items is picked we still need to cover $P-p(A)$; the remaining profits are “trimmed” to be at most $P-p(A)$ and this again does not remove any feasible integral solution.

The following instance [10] shows that the integrality gap implied by KC inequalities is 2: we have $n$ items of unit costs and profits. We are asked to select a set of items in order to obtain a profit of at least $1+1/(n-1)$. The resulting linear program formulation with KC inequalities is as follows (for $x_{i}\in[0,1]$, $i=1,\ldots,n$)

Note that the solution $x_{i}=1/(n-1)$ is a valid fractional solution of value $1+1/(n-1)$ whereas the optimal integral solution has value $2$. In the following we show that $\text{\sc{SoS}}_{t}(LP^{+})$, with $t$ arbitrarily close to a logarithmic function of $n$, admits the same integrality gap as the initial linear program ($LP^{+}$) relaxation.