On the Exactness of SDP Relaxation for Quadratic Assignment Problem

Shuyang Ling Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning, New York University Shanghai, Shanghai, China. S.L. and Z.S.Z. are (partially) financially supported by the National Key R&D Program of China, Project Number 2021YFA1002800, National Natural Science Foundation of China (NSFC) No.12001372, Shanghai Municipal Education Commission (SMEC) via Grant 0920000112, and NYU Shanghai Boost Fund.

Abstract

Quadratic assignment problem (QAP) is a fundamental problem in combinatorial optimization and finds numerous applications in operation research, computer vision, and pattern recognition. However, it is a very well-known NP-hard problem to find the global minimizer to the QAP. In this work, we study the semidefinite relaxation (SDR) of the QAP and investigate when the SDR recovers the global minimizer. In particular, we consider the two input matrices satisfy a simple signal-plus-noise model, and show that when the noise is sufficiently smaller than the signal, then the SDR is exact, i.e., it recovers the global minimizer to the QAP. It is worth noting that this sufficient condition is purely algebraic and does not depend on any statistical assumption of the input data. We apply our bound to several statistical models such as correlated Gaussian Wigner model. Despite the sub-optimality in theory under those models, empirical studies show the remarkable performance of the SDR. Our work could be the first step towards a deeper understanding of the SDR exactness for the QAP.

1 Introduction

Given two matrices $\bm{A}$ and $\bm{C}$ , how to find a simultaneous row and column permutation of $\bm{C}$ such that the resulting two matrices are well aligned? This problem, known as the quadratic assignment (QAP), is one of most challenging problems in optimization [5, 6, 32, 34, 38]. Moreover, it has found numerous applications including graph matching [3, 13], de-anonymization and privacy [43], and protein network [47], and traveling salesman [16, 29].

One of the most common approaches to find the optimal permutation is to minimize the least squares objective:

\min_{\bm{\Pi}\in{\cal P}(n)}~{}\|\bm{A}\bm{\Pi}-\bm{\Pi}\bm{C}\|_{F}^{2}~{}~{}\Longleftrightarrow~{}~{}\max~{}_{\bm{\Pi}\in{\cal P}(n)}\langle\bm{A}\bm{\Pi},\bm{\Pi}\bm{C}\rangle

(1.1)

where ${\cal P}(n)$ is the set of all $n\times n$ permutation matrices. In general, it is well-known NP-hard problem to find its global minimizer [32]. Numerous works have been done to either approximate or exactly solve the QAP problem [5, 6, 27, 34, 38]. Among various algorithms, convex relaxation is a popular approach to solve the QAP [18, 24, 39, 44, 51, 29]. In [27], Gilmore proposed a famous relaxation of quadratic assignment via linear programming and [27, 34] derived a lower bound for the QAP problem. Another straightforward convex relaxation is to relax the permutation matrix to the set of doubly stochastic matrix ${\cal D}(n)$ , which leads to a quadratic program:

\min_{\bm{X}\in{\cal D}(n)}~{}\|\bm{A}\bm{X}-\bm{X}\bm{C}\|_{F}^{2}.

(1.2)

One can also consider spectral relaxation of the QAP [46, 5, 7], and moreover an estimation of the optimal value of the QAP can be also characterized by the spectra of input data matrices.

In this work, we will focus on the semidefinite relaxation (SQR) of QAP. One well-known SDR was proposed in [51]. After that, many variants of SDR have been proposed [44, 15, 16] to improve the formulation in [51], and [10, 42] have studied efficient algorithms to solve the SDR. Our work will be more on the theoretical sides of the SDR for QAP. In particular, we are interested in the following question:

When does the SDR recover the global minimizer to (1.1)?

Without any more constraints, there is certainly almost no hope to find the exact solution to the QAP for general input data due to its NP-hardness. We consider a signal-plus-noise model for the quadratic assignment. More precisely, let $\bm{A}$ be an $n\times n$ symmetric matrix (e.g. adjacency matrix), and $\bm{C}$ is a perturbed matrix of $\bm{A}$ :

\bm{C}=\bm{\Pi}^{\top}(\bm{A}+\bm{\Delta})\bm{\Pi}

(1.3)

where $\bm{\Pi}\in{\cal P}(n)$ is an unknown permutation matrix. Our goal is to recover $\bm{\Pi}$ from the two matrices $\bm{A}$ and $\bm{C}$ in an efficient way.

In the noise-free case, i.e., $\bm{\Delta}=0$ which corresponds to the graph isomorphism, the ground true permutation can be exactly recovered by solving (1.2) or using spectral methods [23, 33] under some regularity conditions on the spectra of $\bm{A}$ . Several other convex programs have been studied [3] to recover the ground true permutation for $\bm{\Delta}=0$ or in presence of very weak noise. Recently, the quadratic assignment has been extensively studied under various statistical models (average-case analysis), especially in the context of graph matching or graph alignment such as correlated Erdös-Rényi graph model [41, 50, 20, 30] and correlated Wigner model [14, 25, 19]. A series of works fully exploit the statistical properties of random weight matrix, and use spectral methods [21, 19, 26] or extracts the feature of vertices [17, 40] to efficiently align two correlated random matrices. The core question is: under what noise levels, one can design an efficient algorithm to find the permutation [19, 20, 21] and whether the algorithm can achieve the information-theoretical threshold [14, 25].

On the other hand, the study on the optimization approaches to solve these random instances is quite limited compared with spectral methods or feature-extraction-based approaches. But optimization-based approach often enjoys more robustness [12]. Therefore, we are interested in studying the performance of optimization methods, especially SDR, in solving the random instances of QAP. In the work [39], the authors studied (1.2) for the correlated Erdös-Rényi model and proved that (1.2) will never produce the exact permutation matrix even if the noise level is extremely small. The work [19, 20] proposed a spectral method to estimate the true permutation under both correlated Gaussian Wigner and Erdös-Rényi models. The spectral method can be viewed as a convex relaxation of (1.1). Although that the global minimizer is not a permutation, it produces the true permutation after a rounding procedure.

SDR (semidefinite relaxation) has proven itself to be powerful in tacking many challenging nonconvex problems in signal processing and data science [48]. The exactness or tightness of SDR has been studied in various problems in signal processing and data science. k-means and data clustering [8, 31, 35, 37], community detection [1, 2], synchronization [9, 52, 36], phase retrieval [11], matrix completion [45], and blind deconvolution [4]. These works show that the SDR can recover the ground truth as long as the SNR (signal-to-noise ratio) is sufficiently large, such as the sample size is large enough or the noise in the data is smaller than certain threshold. Inspired by these observations, we will study when the exactness holds for the SDR of QAP, i.e., the SDR produces a permutation matrix which is also the global minimizer to (1.1).

In this work, we focus on two variants of SDR for the QAP, and study its exactness under the signal-plus-noise model. We provide a sufficient condition that is based on the spectral gap of $\bm{A}$ and also the noise strength $\|\bm{\Delta}\|$ to guarantee the exactness of the SDR. It is worth noting that this sufficient condition is deterministic, and can be applied to several statistical models. Despite the theoretical sub-optimality in the statistical examples, the SDR has shown powerful numerical performance and this could be the first step towards understanding the exactness of SDR for QAP.

1.1 Notation

Before proceeding, we go over some notations that will be used. We denote boldface $\bm{x}$ and $\bm{X}$ as a vector and matrix respectively, and $\bm{x}^{\top}$ and $\bm{X}^{\top}$ are their corresponding transpose. The $\bm{I}_{n}$ and $\bm{J}_{n}$ stands the $n\times n$ identity and constant “1” matrix of size $n\times n$ . For a vector $\bm{x}$ , $\operatorname{diag}(\bm{x})$ is a diagonal matrix whose diagonal entries are given by $\bm{x}.$ For two matrices $\bm{X}$ and $\bm{Y}$ of the same size, $\bm{X}\circ\bm{Y}$ is the Hadamard product, $\langle\bm{X},\bm{Y}\rangle=\sum_{i,j}X_{ij}Y_{ij}=\operatorname{Tr}(\bm{X}\bm{Y}^{\top})$ is their inner product, and $\bm{X}\otimes\bm{Y}$ is their Kronecker product. For any matrix $\bm{X}$ , the Frobenius, operator norms, and the maximum absolute value among all entries are denoted by $\|\bm{X}\|_{F}$ , $\|\bm{X}\|$ , and $\|\bm{X}\|_{\max}$ respectively. Here ${\cal P}(n)$ is the set of $n\times n$ permutation matrix, ${\cal D}(n)$ is the set of all $n\times n$ doubly stochastic matrix, $\bm{e}_{i}$ is a one-hot vector, and $\delta_{ij}=1$ if $i=j$ , and $\delta_{ij}=0$ if $i\neq j.$ We let $\operatorname{vec}(\bm{X})$ be the vectorization of $\bm{X}$ by stacking the columns of $\bm{X}$ , and for a given vector $\bm{x}\in\hbox{\msbm{R}}^{n^{2}}$ , we denote $\operatorname{mat}(\bm{x})\in\hbox{\msbm{R}}^{n\times n}$ as the matrization of $\bm{x}$ .

1.2 Organization

The derivation of SDRs for the QAP is discussed in Section 2; and we will present the main results and numerics in Section 2 as well. The theoretical justification of the main theorems will be given in Section 3 and 4.

2 Preliminaries and main results

Without loss of generality, we assume the hidden ground true permutation is $\bm{I}_{n}$ in (1.3). Note that we can rewrite this least squares objective by vectorizing $\bm{A}\bm{\Pi}$ and $\bm{\Pi}\bm{C}$ :

	$\displaystyle\\|\bm{A}\bm{\Pi}-\bm{\Pi}\bm{C}\\|_{F}^{2}$	$\displaystyle=\\|(\bm{I}_{n}\otimes\bm{A}-\bm{C}\otimes\bm{I}_{n})\bm{x}\\|^{2}$
		$\displaystyle=\bm{x}^{\top}(\bm{I}_{n}\otimes\bm{A}-\bm{C}\otimes\bm{I}_{n})^{2}\bm{x}$

where $\bm{A}$ and $\bm{C}$ are symmetric, and $\bm{x}$ is the vectorization of $\bm{\Pi}$ , i.e., for any permutation $\sigma(\cdot)$ , which is bijective on $\{1,\cdots,n\}$ ,

\bm{x}=\begin{bmatrix}\bm{e}_{\sigma(1)}\\ \vdots\\ \bm{e}_{\sigma(n)}\end{bmatrix}

(2.1)

Therefore, by letting

\bm{M}:=(\bm{I}_{n}\otimes\bm{A}-\bm{C}\otimes\bm{I}_{n})^{2}\succeq 0,

(2.2)

the quadratic assignment problem is equivalent to

\min~{}\operatorname{Tr}(\bm{M}\bm{x}\bm{x}^{\top})\quad\text{ s.t. }\quad\bm{x}=\operatorname{vec}(\bm{\Pi}),~{}~{}\bm{\Pi}\in{\cal P}(n).

(2.3)

Another equivalent form of (1.1) follows from

	$\displaystyle\\|\bm{A}\bm{\Pi}-\bm{\Pi}\bm{C}\\|_{F}^{2}$	$\displaystyle=\\|\bm{A}\\|_{F}^{2}-2\langle\bm{A}\bm{\Pi},\bm{\Pi}\bm{C}\rangle+\\|\bm{C}\\|_{F}^{2}=-2\langle(\bm{I}_{n}\otimes\bm{A})\bm{x},(\bm{C}\otimes\bm{I}_{n})\bm{x}\rangle+\\|\bm{A}\\|_{F}^{2}+\\|\bm{C}\\|_{F}^{2}$
		$\displaystyle=-2\langle\bm{C}\otimes\bm{A},\bm{x}\bm{x}^{\top}\rangle+\\|\bm{A}\\|_{F}^{2}+\\|\bm{C}\\|_{F}^{2}$

By letting

\bm{M}:=-\bm{C}\otimes\bm{A},

(2.4)

the least squares objective is equal to (2.3). Throughout the discussion, we will mainly focus on (2.3) with $\bm{M}=(\bm{I}_{n}\otimes\bm{A}-\bm{C}\otimes\bm{I}_{n})^{2}$ in (2.2), and all the theoretical analysis also applies to (2.4) with minor modifications.

2.1 Convex relaxation

By letting $\bm{X}=\bm{x}\bm{x}^{\top},$ we note that (2.3) is a linear function in $\bm{X}$ . Therefore, the idea of convex relaxation of (2.3) is to find a proper convex set that includes all rank-1 matrix $\bm{X}=\bm{x}\bm{x}^{\top}$ where $\bm{x}=\operatorname{vec}(\bm{\Pi})$ and $\bm{\Pi}\in{\cal P}(n).$ By (2.1), $\bm{X}=\bm{x}\bm{x}^{\top}$ is highly structured:

\bm{X}=\begin{bmatrix}\bm{e}_{\sigma(1)}\bm{e}_{\sigma(1)}^{\top}&\bm{e}_{\sigma(1)}\bm{e}_{\sigma(2)}^{\top}&\cdots&\bm{e}_{\sigma(1)}\bm{e}_{\sigma(n)}^{\top}\\ \bm{e}_{\sigma(2)}\bm{e}_{\sigma(1)}^{\top}&\bm{e}_{\sigma(2)}\bm{e}_{\sigma(2)}^{\top}&\cdots&\bm{e}_{\sigma(2)}\bm{e}_{\sigma(n)}^{\top}\\ \vdots&\vdots&\ddots&\vdots\\ \bm{e}_{\sigma(n)}\bm{e}_{\sigma(1)}^{\top}&\bm{e}_{\sigma(n)}\bm{e}_{\sigma(2)}^{\top}&\cdots&\bm{e}_{\sigma(n)}\bm{e}_{\sigma(n)}^{\top}\\ \end{bmatrix}

which consists of $n^{2}$ blocks with each block exactly rank-1 and only containing one non-zero entry.

Now we try to find a proper convex set which contains

\{\bm{X}:\bm{X}=\bm{x}\bm{x}^{\top},\quad\bm{x}=\operatorname{vec}(\bm{\Pi}),\quad\bm{\Pi}\in{\cal P}(n)\}.

It is obvious that $\bm{X}\succeq 0$ and $\bm{X}\geq 0$ , which will be incorporated into the constraints.

Convex relaxation I.

Note that for each $n\times n$ block, we have

\bm{X}_{ij}=\bm{e}_{\sigma(i)}\bm{e}_{\sigma(j)}^{\top}

is exactly rank-1 and only contains one nonzero entry. Moreover, for any permutation $\sigma$ , it holds that

\operatorname{Tr}(\bm{X}_{ij})=\begin{cases}1,&i=j,\\ 0,&i\neq j,\end{cases},~{}~{}~{}\langle\bm{X}_{ij},\bm{J}_{n}\rangle=1

and also

	$\displaystyle\sum_{i=1}^{n}\bm{X}_{ii}$	$\displaystyle=\sum_{i=1}^{n}\bm{e}_{\sigma(i)}\bm{e}_{\sigma(i)}^{\top}=\bm{I}_{n},$
	$\displaystyle\sum_{i,j}\bm{X}_{ij}$	$\displaystyle=\sum_{i,j}\bm{e}_{\sigma(i)}\bm{e}_{\sigma(j)}^{\top}=\sum_{i=1}^{n}\bm{e}_{\sigma(i)}\left(\sum_{j=1}^{n}\bm{e}_{\sigma(j)}\right)^{\top}=\bm{J}_{n}.$

Therefore, combining these constraints leads to the following convex relaxation:

$\displaystyle\min$	$\displaystyle\langle\bm{M},\bm{X}\rangle$	(2.5)
s.t.	$\displaystyle\bm{X}\succeq 0,~{}~{}~{}\bm{X}\geq 0,$
	$\displaystyle\operatorname{Tr}(\bm{X}_{ij})=\delta_{ij},~{}~{}~{}\langle\bm{X}_{ij},\bm{J}_{n}\rangle=1,~{}~{}\forall 1\leq i,j\leq n,$
	$\displaystyle\sum_{i=1}^{n}\bm{X}_{ii}=\bm{I}_{n},~{}~{}~{}\sum_{i,j}\bm{X}_{ij}=\bm{J}_{n}.$

Convex relaxation II.

Note that for $\bm{X}=\bm{x}\bm{x}^{\top}$ , it holds $\operatorname{diag}(\bm{X})=\bm{x}.$ Using the fact that

\begin{bmatrix}\bm{x}\bm{x}^{\top}&\bm{x}\\ \bm{x}^{\top}&1\end{bmatrix}\succeq 0,

we have a few new constraints:

\begin{bmatrix}\bm{X}&\operatorname{diag}(\bm{X})\\ \operatorname{diag}(\bm{X})^{\top}&1\end{bmatrix}\succeq 0,~{}~{}~{}\operatorname{mat}(\operatorname{diag}(\bm{X}))\in{\cal D}(n)

where ${\cal D}(n)$ is the set of $n\times n$ doubly stochastic matrix. Combining the constraints above with $\operatorname{Tr}(\bm{X}_{ij})=\delta_{ij}$ and nonnegativity, we get the SDR similar to [51]:

$\displaystyle\min$	$\displaystyle\langle\bm{M},\bm{X}\rangle$	(2.6)
s.t.	$\displaystyle\begin{bmatrix}\bm{X}&\operatorname{diag}(\bm{X})\\ \operatorname{diag}(\bm{X})^{\top}&1\end{bmatrix}\succeq 0,~{}~{}~{}\bm{X}\geq 0,$
	$\displaystyle\langle\bm{X}_{ij},\bm{J}_{n}\rangle=1,~{}~{}~{}\operatorname{Tr}(\bm{X}_{ij})=\delta_{ij},~{}~{}\forall 1\leq i,j\leq n,$
	$\displaystyle\sum_{i=1}^{n}\bm{X}_{ii}=\bm{I}_{n},~{}~{}~{}\sum_{i,j}\bm{X}_{ij}=\bm{J}_{n},~{}~{}\forall 1\leq i,j\leq n,$
	$\displaystyle\operatorname{mat}(\operatorname{diag}(\bm{X}))\bm{1}_{n}=\operatorname{mat}(\operatorname{diag}(\bm{X}))^{\top}\bm{1}_{n}=\bm{1}_{n}.$

For the last constraints in (2.6), the explicit form of $\operatorname{mat}(\operatorname{diag}(\bm{X}))$ is given by

\operatorname{mat}(\operatorname{diag}(\bm{X}))=\begin{bmatrix}\bm{X}_{11,11}&\bm{X}_{11,22}&\bm{X}_{11,33}&\cdots&\bm{X}_{11,nn}\\ \bm{X}_{22,11}&\bm{X}_{22,22}&\bm{X}_{22,33}&\cdots&\bm{X}_{22,nn}\\ \bm{X}_{33,11}&\bm{X}_{33,22}&\bm{X}_{33,33}&\cdots&\bm{X}_{33,nn}\\ \vdots&\vdots&\vdots&\ddots&\vdots\\ \bm{X}_{nn,11}&\bm{X}_{nn,22}&\bm{X}_{nn,33}&\cdots&\bm{X}_{nn,nn}\end{bmatrix}

which reshapes the diagonal elements of $\bm{X}$ into an $n\times n$ matrix. The relaxation (2.6) is tighter than (2.5) as it imposes a few more constraints on $\bm{X}.$

2.2 Main theorems

With the introduction of two SDRs in (2.5) and (2.6), we are ready to present our main theorems. Note that if $\bm{A}$ has distinct eigenvalues and all eigenvectors are not orthogonal to $\bm{1}_{n}$ , then finding the optimal permutation is possible via simple convex relaxations [3, 33]. We interested in whether the exactness holds in presence of the noise, and below is our main theorem.

Theorem 2.1.

Let $(\lambda_{i},\bm{u}_{i})$ be the $i$ -th eigenvalue and eigenvector of $\bm{A}$ , then $\bm{X}=\bm{x}\bm{x}^{\top}$ is the unique global minimizer to SDR (2.5) if

\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}\min|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}\geq n(\|\bm{\Delta}\|\|\bm{A}\|+\|\bm{A}\bm{\Delta}\|_{\max})

where $\bm{A}_{i}$ and $\bm{\Delta}_{i}$ are the $i$ -th columns of $\bm{A}$ and $\bm{\Delta}$ respectively.

There are remarks regarding Theorem 2.1. Suppose $\bm{\Delta}=0$ , $\min_{i\neq j}|\lambda_{i}-\lambda_{j}|>0$ and $\langle\bm{u}_{i},\bm{1}_{n}\rangle\neq 0$ , then the exactness always holds. This is aligned with the results for graph isomorphism in [3, 33]. The interesting point is that even if the noise exists, as long as the noise strength is sufficiently small compared with minimum spectral gap and the alignment of $\bm{u}_{i}$ and $\bm{1}_{n}$ , then SDR is still exact. In other words, our theorem provides a deterministic condition that guarantees the exactness of the SDR (2.5) in presence of noise. A result of similar flavor was derived in [3, Lemma 2], which provides an error estimation between the solution from a quadratic program relaxation and the true permutation in terms of spectral gap and $\min_{i}|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}$ . However, the exactness was not obtained in [3].

For the SDR in (2.6), we can derive a similar deterministic sufficient condition for its exactness under the same assumptions.

Theorem 2.2.

Let $(\lambda_{i},\bm{u}_{i})$ be the $i$ -th eigenvalue and eigenvector of $\bm{A}$ , then $\bm{X}=\bm{x}\bm{x}^{\top}$ is the unique global minimizer to SDR (2.6) if

\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}\min|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}\geq n(\|\bm{\Delta}\|\|\bm{A}\|+\|\bm{A}\bm{\Delta}\|_{\max})

where $\bm{A}_{i}$ and $\bm{\Delta}_{i}$ are the $i$ -th columns of $\bm{A}$ and $\bm{\Delta}$ respectively.

The proof of both theorems can be reduced to constructing a dual certificate (finding the dual variables) such that it can ensure the global optimality of the solution $\bm{X}=\bm{x}\bm{x}^{\top}.$ This routine is well-established but the actual construction is highly problem dependent. For the SDR of QAP, the construction is not simple as the SDRs have complicated constraints. One may expect a sharper theoretical bound for the SDR (2.6) than (2.5), as (2.6) has more constraints. However, due to the complication of constructing a proper dual certificate that ensures $\bm{X}=\bm{x}\bm{x}^{\top}$ , we employ a similar construction of the dual variables and this leads to the same deterministic condition.

As both Theorem 2.1 and 2.2 are general, we present three special examples and see how our theorem works, and also make a comparison with the numerical experiments and the best known theoretical bounds. The experiments under these models imply that our theoretical bound is sub-optimal, although the SDR is remarkable in numerics. As briefly mentioned before, this sub-optimality results from the construction of dual certificate which is quite challenging for the SDR of QAP with general input data.

2.3 Examples and numerics

Example: diagonal matrix plus Gaussian noise.

Suppose $\bm{A}$ is a diagonal matrix, i.e.,

\bm{A}=\operatorname{diag}(\lambda_{1},\cdots,\lambda_{n})

where the eigenvalues are in a descending order and the eigenvector is $\bm{u}_{i}=\bm{e}_{i}$ . Suppose $\bm{\Delta}=\sigma\operatorname{diag}(\bm{w})$ where $\bm{w}$ is a standard Gaussian random vector, and $\bm{C}=\bm{A}+\sigma\operatorname{diag}(\bm{w})$ is always a diagonal matrix. Then applying Theorem 2.1 and 2.2 implies that the SDR is exact if

\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}\geq n\left(4\sigma_{\operatorname{SDR}}\sqrt{\log n}\cdot\max_{1\leq i\leq n}|\lambda_{i}|\right)

where $\|\bm{\Delta}\|\leq 2\sqrt{\log n}$ holds with high probability and $\langle\bm{e}_{i},\bm{1}_{n}\rangle=1$ . Then the noise level $\sigma_{\operatorname{SDR}}$ should satisfy

\sigma_{\operatorname{SDR}}\leq\frac{\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}}{4n\sqrt{\log n}\max_{i}|\lambda_{i}|}.

On the other hand, given two diagonal matrices $\bm{A}$ and $\bm{C}$ , the global minimizer to (1.1) should still be $\bm{I}_{n}$ if and only if the ordering of the eigenvalues remains unchanged, which holds if

\sigma\leq\frac{\min_{i\neq j}|\lambda_{i}-\lambda_{j}|}{2\sqrt{\log n}}.

For a specific example $\lambda_{k}=k$ , $1\leq k\leq n$ , then

\sigma_{\operatorname{SDR}}\leq\frac{1}{4n^{2}\sqrt{\log n}},~{}~{}~{}\sigma\leq\frac{1}{2\sqrt{\log n}}.

This indicates that the SDR is sub-optimal by a factor of $n^{2}.$

We also look into the performance of (2.5) in numerical simulations. We choose $n=10$ due to the high computational complexity for larger $n$ , and let $\lambda_{k}=k$ and $0\leq\sigma\leq 2$ . For each $\sigma$ , we run 20 experiments and compute the correlation of $\widehat{\bm{X}}$ and $\bm{x}\bm{x}^{\top}:$

\operatorname{corr}(\widehat{\bm{X}},\bm{x}\bm{x}^{\top})=\frac{\bm{x}^{\top}\widehat{\bm{X}}\bm{x}}{n^{2}}

(2.7)

where $\widehat{\bm{X}}$ is the solution to (2.5). This correlation is between 0 and 1: the higher the correlation, the better the recovery. In particular, if $\operatorname{corr}(\cdot,\cdot)$ is 1, then $\widehat{\bm{X}}=\bm{x}\bm{x}^{\top}$ . We count one instance as exactness if $\operatorname{corr}(\widehat{\bm{X}},\bm{x}\bm{x}^{\top})\geq 1-10^{-3}.$ All simulations are done by using CVX [28] and the numerical result is presented below in Figure 1.

Refer to caption — Figure 1: Exactness of SDR (2.5) and (2.6) for QAP under diagonal matrix model.

Figure 1 indicates that for $\sigma\leq 0.3$ , then the SDR is exact and its performance nearly matches the optimal bound. The two types of SDRs perform similarly under this random setting.

Example: diagonal-plus-Wigner model

Consider $\bm{A}=\operatorname{diag}(\lambda_{1},\cdots,\lambda_{n})$ and $\bm{W}$ is a Gaussian random matrix with $\operatorname{diag}(\bm{W})=0$ , and $\bm{C}=\bm{A}+\sigma\bm{W}$ . Then applying Theorem 2.1 implies that

\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}\min|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}\geq n\sigma_{\operatorname{SDR}}\max_{i}|\lambda_{i}|\cdot 2\sqrt{n}~{}~{}\Longleftrightarrow~{}~{}\sigma_{\operatorname{SDR}}\leq\frac{\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}}{2n^{3/2}\max_{i}|\lambda_{i}|}

is needed to ensures the exactness where $\bm{u}_{i}=\bm{e}_{i}$ and $\|\bm{W}\|\leq(2+o(1))\sqrt{n}$ holds with high probability [49]. In particular, for $\lambda_{k}=k$ , $1\leq k\leq n$ , then the exactness of the SDR holds if $\sigma_{\operatorname{SDR}}\leq 1/(2n^{5/2}).$

The numerical experiment is given in Figure 2, which shows the exactness holds if $\sigma\leq 0.15$ . For $n=10$ , our bound requires $\sigma_{\operatorname{SDR}}\leq 0.0016$ in theory. This implies that there is gap between the actual performance of the SDR and our results.

Example: correlated Wigner model.

The correlated Wigner model assumes $\bm{A}$ and $\bm{C}$ are two Gaussian random matrices satisfying $\bm{C}=\bm{A}+\sigma\bm{W}$ where $\bm{W}$ is a Gaussian random matrix independent of $\bm{A}$ . To apply Theorem 2.1 and 2.2, it suffices to obtain a lower bound on the spectral gap and also an upper bound on $\|\bm{A}\|.$

For the spectral gap, [22, Corollary 1] implies for $n$ sufficiently large, it holds that

\hbox{\msbm{P}}(2^{-3/2}n\delta_{n}\geq t)\approx 2\int_{t}^{\infty}xe^{-x^{2}}\mathop{}\!\mathrm{d}x=e^{-t^{2}}\geq 1-t^{2}

where $\delta_{n}$ denotes the smallest gap. Therefore, it holds with probability at least $1-O(\log^{-1}n)$ that

\delta_{n}=\max_{i\neq j}|\lambda_{i}-\lambda_{j}|\geq\frac{1}{n\sqrt{\log n}}.

As a result, our result implies that the exactness holds if

\frac{1}{2}\left(\frac{1}{n\sqrt{\log n}}\right)^{2}\geq n(2\sigma\sqrt{n}\cdot 2\sqrt{n})\Longrightarrow\sigma_{\operatorname{SDR}}\lesssim\frac{1}{n^{4}\log n}

where $\|\bm{W}\|\leq 2(1+o(1))\sqrt{n}$ and $\min_{i}|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}\geq 1/2$ holds with high probability. As shown in the Figure 3, it implies that $\sigma=0.6$ suffices to have the exactness numerically which is far better than the theoretic bound. Under the correlated Wigner model, (2.6) performs better than (2.5) when $\sigma\geq 0.6.$

Let’s conclude this section by commenting on the numerics and also future research problems. In all three examples, we can see the SDRs perform much better in numerics than the theoretical bound by a factor of $n^{2}$ or even more. In fact, for the correlated Gaussian Wigner model, the SDRs are able to recover the true permutation even if $\sigma$ is of constant order, which is comparable with the state-of-the-art works on the same model such as [19]. Therefore, there is still big room for the improvement. In particular, it will be very interesting to show theoretically that SDR achieves exactness on random instances that is consistent with the numerical observation. This asks for a new technique to construct dual certificates that could exploit the statistical property of noise. More importantly, it is unclear that for the QAP problem, what is the best way to characterize the SNR (signal-to-noise ratio) that could be useful in analyzing the SDR of QAP. For example, in the correlated Gaussian Wigner model, using the spectral gap for the analysis does not seem to match the numerical performance as the spectral gap of a Gaussian random matrix can be very small for larger $n$ . All of these will be worthwhile to consider in the future.

3 Proof of Theorem 2.1

The proof of Theorem 2.1 essentially follows from the well-established technique in convex relaxation. Without loss of generality, we assume $\bm{\Pi}=\bm{I}_{n}$ and thus

\bm{x}^{\top}=[\bm{e}_{1}^{\top},\cdots,\bm{e}_{n}^{\top}]\in\hbox{\msbm{R}}^{n^{2}},~{}~{}\|\bm{x}\|^{2}=n.

Our goal is to establish sufficient conditions that ensure the global minimizer to (2.5) is $\bm{x}\bm{x}^{\top}$ by constructing a dual certificate.

3.1 Dual program and optimality condition of (2.5)

For each constraint in (2.5), we assign a dual variable:

	$\displaystyle\bm{Q}\succeq 0$	$\displaystyle:\bm{X}\succeq 0,$
	$\displaystyle\bm{B}\geq 0$	$\displaystyle:\bm{X}\geq 0,$
	$\displaystyle\bm{T}\in\hbox{\msbm{R}}^{n\times n}$	$\displaystyle:\operatorname{Tr}(\bm{X}_{ij})=\delta_{ij},$
	$\displaystyle\bm{Z}\in\hbox{\msbm{R}}^{n\times n}$	$\displaystyle:\langle\bm{X}_{ij},\bm{J}_{n}\rangle=1,$
	$\displaystyle\bm{K}\in\hbox{\msbm{R}}^{n\times n}$	$\displaystyle:\sum_{i=1}^{n}\bm{X}_{ii}=\bm{I}_{n},$
	$\displaystyle\bm{H}\in\hbox{\msbm{R}}^{n\times n}$	$\displaystyle:\sum_{i,j}\bm{X}_{ij}=\bm{J}_{n}.$

The Lagrangian function is

	$\displaystyle{\cal L}(\bm{X},\bm{Q},\bm{B},\bm{T},\bm{Z},\bm{K},\bm{H})$
	$\displaystyle\qquad:=\langle\bm{X},\bm{M}-\bm{Q}-\bm{B})\rangle-\sum_{i,j}t_{ij}(\operatorname{Tr}(\bm{X}_{ij})-\delta_{ij})-\sum_{i,j}z_{ij}(\langle\bm{X}_{ij},\bm{J}_{n}\rangle-1)$
	$\displaystyle\qquad\qquad-\left\langle\sum_{i=1}^{n}\bm{X}_{ii}-\bm{I}_{n},\bm{K}\right\rangle-\left\langle\sum_{i,j}\bm{X}_{ij}-\bm{J}_{n},\bm{H}\right\rangle$
	$\displaystyle\qquad=\langle\bm{X},\bm{M}-\bm{Q}-\bm{B}-\bm{T}\otimes\bm{I}_{n}-\bm{Z}\otimes\bm{J}_{n}-\bm{I}_{n}\otimes\bm{K}-\bm{J}_{n}\otimes\bm{H}\rangle+\operatorname{Tr}(\bm{T}+\bm{K})+\langle\bm{J}_{n},\bm{Z}+\bm{H}\rangle.$

As a result, the dual program of (2.5) is

$\displaystyle\max$	$\displaystyle\operatorname{Tr}(\bm{T}+\bm{K})+\langle\bm{J}_{n},\bm{H}+\bm{Z}\rangle$	(3.1)
s.t.	$\displaystyle\bm{M}=\bm{Q}+\bm{B}+\bm{T}\otimes\bm{I}_{n}+\bm{Z}\otimes\bm{J}_{n}+\bm{I}_{n}\otimes\bm{K}+\bm{J}_{n}\otimes\bm{H},$
	$\displaystyle\bm{Q}\succeq 0,~{}~{}\bm{B}\geq 0.$

Suppose $\bm{X}=\bm{x}\bm{x}^{\top}$ is the global minimizer, then corresponding the complementary slackness:

\langle\bm{B},\bm{X}\rangle=0,\quad\bm{Q}\bm{X}=0.

(3.2)

Note that $\bm{Q}\succeq 0$ and $\bm{X}=\bm{x}\bm{x}^{\top}\succeq 0$ , and then $\bm{Q}\bm{X}=0$ is equivalent to $\bm{Q}\bm{x}=0.$ Therefore the KKT condition becomes:

1.

Stationarity:

$\bm{M}=\bm{Q}+\bm{B}+\bm{T}\otimes\bm{I}_{n}+\bm{I}_{n}\otimes\bm{K}+\bm{J}_{n}\otimes\bm{H}+\bm{Z}\otimes\bm{J}_{n},$ (3.3)
2.

Dual feasibility:

$\bm{B}\geq 0,\qquad\bm{Q}\succeq 0,$ (3.4)
3.

Complementary slackness:

$\langle\bm{B},\bm{X}\rangle=0,\qquad\bm{Q}\bm{x}=0,$ (3.5)

where $\bm{M}=(\bm{I}_{n}\otimes\bm{A}-\bm{C}\otimes\bm{I}_{n})^{2}$ is the data matrix. Due to the nonnegativity of $\bm{B}$ , $\langle\bm{B},\bm{X}\rangle=0$ is equivalent to $\bm{B}\circ\bm{x}\bm{x}^{\top}=0$ where “ $\circ$ ” is the Hadamard product of two matrices.

Now we compute $\bm{Q}\bm{x}=0$ to further make the KKT condition explicit. Instead of computing $\bm{Q}\bm{x}$ , we will use the property of Kronecker product to simplify the expression. Note that

	$\displaystyle\operatorname{mat}(\bm{M}\bm{x})=\operatorname{mat}((\bm{I}_{n}\otimes\bm{A}-\bm{C}\otimes\bm{I}_{n})^{2}\bm{x})$
	$\displaystyle\qquad\qquad~{}~{}=\bm{A}^{2}-2\bm{A}\bm{C}+\bm{C}^{2}=\bm{\Delta}^{2}+\bm{\Delta}\bm{A}-\bm{A}\bm{\Delta},$
	$\displaystyle\operatorname{mat}((\bm{T}\otimes\bm{I}_{n})\bm{x})=\bm{T},$
	$\displaystyle\operatorname{mat}((\bm{Z}\otimes\bm{J}_{n})\bm{x})=\bm{J}_{n}\bm{Z},$
	$\displaystyle\operatorname{mat}((\bm{I}_{n}\otimes\bm{K})\bm{x})=\bm{K},$
	$\displaystyle\operatorname{mat}((\bm{J}_{n}\otimes\bm{H})\bm{x})=\bm{H}\bm{J}_{n},$

where $\bm{x}=\operatorname{vec}(\bm{I}_{n}).$ We define

\bm{S}=\bm{T}\otimes\bm{I}_{n}+\bm{I}_{n}\otimes\bm{K}+\bm{J}_{n}\otimes\bm{H}+\bm{Z}\otimes\bm{J}_{n}

(3.6)

as we will use it quite often, and it holds

	$\displaystyle\operatorname{mat}(\bm{S}\bm{x})$	$\displaystyle=\bm{T}+\bm{J}_{n}\bm{Z}+\bm{K}+\bm{H}\bm{J}_{n},$
	$\displaystyle\operatorname{mat}((\bm{M}-\bm{S})\bm{x})$	$\displaystyle=\bm{\Delta}^{2}+\bm{\Delta}\bm{A}-\bm{A}\bm{\Delta}-(\bm{T}+\bm{J}_{n}\bm{Z}+\bm{K}+\bm{H}\bm{J}_{n}).$

Therefore, (3.3), (3.4) and (3.5) imply

$\displaystyle\operatorname{mat}((\bm{Q}+\bm{B})\bm{x})$	$\displaystyle=\operatorname{mat}((\bm{M}-\bm{S})\bm{x})$	(3.7)
	$\displaystyle=\operatorname{mat}((\bm{M}-(\bm{T}\otimes\bm{I}_{n}+\bm{I}_{n}\otimes\bm{K}+\bm{J}_{n}\otimes\bm{H}+\bm{Z}\otimes\bm{J}_{n}))\bm{x})$
	$\displaystyle=\bm{\Delta}^{2}+\bm{\Delta}\bm{A}-\bm{A}\bm{\Delta}-(\bm{T}+\bm{J}_{n}\bm{Z}+\bm{K}+\bm{H}\bm{J}_{n})=\operatorname{mat}(\bm{B}\bm{x})$

where $\bm{Q}\bm{x}=0$ is in (3.4). Therefore, the KKT condition has a simplified form:

		$\displaystyle\bm{Q}=\bm{M}-\left(\bm{B}+\bm{T}\otimes\bm{I}_{n}+\bm{Z}\otimes\bm{J}_{n}+\bm{I}_{n}\otimes\bm{K}+\bm{J}_{n}\otimes\bm{H}\right)\succeq 0,$		(3.8)
		$\displaystyle\bm{\Delta}^{2}+\bm{\Delta}\bm{A}-\bm{A}\bm{\Delta}=\operatorname{mat}(\bm{B}\bm{x})+(\bm{T}+\bm{J}_{n}\bm{Z}+\bm{K}+\bm{H}\bm{J}_{n}),$
		$\displaystyle\langle\bm{B},\bm{X}\rangle=0,~{}~{}\bm{B}\geq 0.$

In the following section, we will construct an explicit dual certificate, i.e., $\bm{T},\bm{Z},\bm{K},\bm{H}$ , and $\bm{B}\succeq 0$ such that (3.8) holds. Moreover, we can also certify $\bm{X}=\bm{x}\bm{x}^{\top}$ as the unique global minimizer under mild conditions.

Theorem 3.1.

Suppose there exist $\bm{B}\geq 0$ with $\bm{B}\circ\bm{x}\bm{x}^{\top}=0$ , and $\bm{T},\bm{K},\bm{Z}$ and $\bm{H}$ such that $\bm{Q}\succeq c\left(\bm{I}_{n^{2}}-n^{-1}\bm{x}\bm{x}^{\top}\right)$ for some $c\geq 0$ , then $\bm{X}=\bm{x}\bm{x}^{\top}$ is a global minimizer. In addition, if $c>0$ , then $\bm{X}$ is the unique global minimizer.

Proof: .

Let $\widehat{\bm{X}}$ be any feasible solution that is not equal to $\bm{x}\bm{x}^{\top}$ . Then

	$\displaystyle\langle\bm{Q},\widehat{\bm{X}}\rangle$	$\displaystyle=\langle\bm{Q},\widehat{\bm{X}}-\bm{x}\bm{x}^{\top}\rangle=\langle\bm{M}-\bm{S}-\bm{B},\widehat{\bm{X}}-\bm{x}\bm{x}^{\top}\rangle$
		$\displaystyle=\langle\bm{M}-\bm{B},\widehat{\bm{X}}-\bm{x}\bm{x}^{\top}\rangle$
		$\displaystyle=\langle\bm{M},\widehat{\bm{X}}-\bm{x}\bm{x}^{\top}\rangle-\langle\bm{B},\widehat{\bm{X}}\rangle$

where $\bm{Q}=\bm{M}-\bm{S}-\bm{B}$ and $\langle\bm{S},\widehat{\bm{X}}-\bm{x}\bm{x}^{\top}\rangle=0$ holds due to the primal feasibility of $\widehat{\bm{X}}$ and $\bm{x}\bm{x}^{\top}$ . Now we have

\displaystyle\langle\bm{M},\widehat{\bm{X}}-\bm{x}\bm{x}^{\top}\rangle

\displaystyle=\langle\bm{Q}+\bm{B},\widehat{\bm{X}}\rangle\geq\langle\bm{Q},\widehat{\bm{X}}\rangle=\left\langle\bm{Q},\left(\bm{I}_{n^{2}}-\frac{\bm{x}\bm{x}^{\top}}{n}\right)\widehat{\bm{X}}\left(\bm{I}_{n^{2}}-\frac{\bm{x}\bm{x}^{\top}}{n}\right)\right\rangle\geq 0

where $\left(\bm{I}_{n^{2}}-n^{-1}\bm{x}\bm{x}^{\top}\right)\widehat{\bm{X}}\left(\bm{I}_{n^{2}}-n^{-1}\bm{x}\bm{x}^{\top}\right)$ is nonzero. In particular, if $c>0$ , then the inequality above is strict, implying the uniqueness of the global minimizer. ∎

3.2 Construction of a dual certificate

From the first equation in (3.8) and $\bm{Q}\bm{x}=0$ , it holds

\bm{B}\bm{x}=(\bm{M}-\bm{S})\bm{x}\geq 0

where $\bm{B}\geq 0$ and $\bm{S}=\bm{T}\otimes\bm{I}_{n}+\bm{Z}\otimes\bm{J}_{n}+\bm{I}_{n}\otimes\bm{K}+\bm{J}_{n}\otimes\bm{H}.$ Note that $\bm{B}\circ\bm{x}\bm{x}^{\top}=0$ , and thus $\operatorname{diag}(\bm{x})\bm{B}\bm{x}=0$ , i.e., the diagonal elements of $\operatorname{mat}(\bm{B}\bm{x})$ equal 0. Therefore, the second equation in (3.8) gives $\operatorname{diag}(\bm{\Delta}^{2})=\operatorname{diag}(\bm{T}+\bm{K})+(\bm{Z}+\bm{H})\bm{1}_{n}$ .

With the discussion above, we try to first determine $\bm{B}$ via solving the linear equation $\bm{B}\bm{x}=(\bm{M}-\bm{S})\bm{x}$ , and we will find that the construction of a dual certificate finally reduce to searching for proper $\bm{T},\bm{K},\bm{Z}$ and $\bm{H}$ .

Proposition 3.2 (Construction of $\bm{B}$ and $\bm{Q}$ ).

Suppose

\operatorname{diag}(\bm{\Delta}^{2})=\operatorname{diag}(\bm{T}+\bm{K})+(\bm{Z}+\bm{H})\bm{1}_{n}

(3.9)

for some $\bm{T},\bm{K}$ , $\bm{Z}$ , and $\bm{H}$ . Let

\bm{B}=\frac{1}{n}\left(\bm{M}-\bm{S}\right)\bm{x}\bm{x}^{\top}+\frac{1}{n}\bm{x}\bm{x}^{\top}\left(\bm{M}-\bm{S}\right),

(3.10)

then $\bm{B}\bm{x}=(\bm{M}-\bm{S})\bm{x}$ and $\bm{B}\circ\bm{x}\bm{x}^{\top}=0$ hold automatically. Moreover, $\bm{B}\geq 0$ if

(\bm{M}-\bm{S})\bm{x}\geq 0\Longleftrightarrow\bm{\Delta}^{2}+\bm{\Delta}\bm{A}-\bm{A}\bm{\Delta}-(\bm{T}+\bm{J}_{n}\bm{Z}+\bm{K}+\bm{H}\bm{J}_{n})\geq 0.

(3.11)

for some $\bm{T},\bm{Z},\bm{K}$ and $\bm{H}.$ As a result, it holds

\bm{Q}=\bm{M}-\bm{S}-\bm{B}=\left(\bm{I}_{n^{2}}-\frac{\bm{x}\bm{x}^{\top}}{n}\right)\left(\bm{M}-\bm{S}\right)\left(\bm{I}_{n^{2}}-\frac{\bm{x}\bm{x}^{\top}}{n}\right)

(3.12)

with the construction of $\bm{B}$ in (3.10).

Proof: .

Consider $\bm{B}$ in (3.10), it holds that

\displaystyle\bm{B}\bm{x}

\displaystyle=\frac{1}{n}\left(\bm{M}-\bm{S}\right)\bm{x}\bm{x}^{\top}\bm{x}+\frac{1}{n}\bm{x}\cdot\bm{x}^{\top}\left(\bm{M}-\bm{S}\right)\bm{x}=\left(\bm{M}-\bm{S}\right)\bm{x}

where

	$\displaystyle\bm{x}^{\top}(\bm{M}-\bm{S})\bm{x}$	$\displaystyle=\langle\bm{I}_{n},\operatorname{mat}((\bm{M}-\bm{S})\bm{x})\rangle$
		$\displaystyle=\langle\bm{I}_{n},(\bm{\Delta}^{2}+\bm{\Delta}\bm{A}-\bm{A}\bm{\Delta}-\operatorname{mat}(\bm{S}\bm{x}))\rangle$
		$\displaystyle=\operatorname{Tr}\left(\operatorname{diag}(\bm{\Delta}^{2})-(\operatorname{diag}(\bm{T}+\bm{K})+(\bm{H}+\bm{Z})\bm{1}_{n})\right)=0$

and the expression of $(\bm{M}-\bm{S})\bm{x}$ follows from (3.7) and (3.9).

We proceed to verify that $\bm{B}\circ\bm{x}\bm{x}^{\top}=0$ :

	$\displaystyle\operatorname{mat}(\operatorname{diag}(\bm{x})\bm{B}\bm{x})$	$\displaystyle=\operatorname{mat}(\operatorname{diag}(\bm{x})(\bm{M}-\bm{S})\bm{x})$
		$\displaystyle=\bm{I}_{n}\circ\operatorname{mat}((\bm{M}-\bm{S})\bm{x})$
		$\displaystyle=\bm{I}_{n}\circ(\bm{\Delta}^{2}+\bm{\Delta}\bm{A}-\bm{A}\bm{\Delta}-(\bm{T}+\bm{J}_{n}\bm{Z}+\bm{K}+\bm{H}\bm{J}_{n}))$
		$\displaystyle=\operatorname{diag}(\bm{\Delta}^{2})-(\operatorname{diag}(\bm{T}+\bm{K})+(\bm{Z}+\bm{H})\bm{1}_{n})=0$

follows from (3.9). This implies that the diagonal entries of $\operatorname{mat}(\bm{B}\bm{x})$ are zero, i.e., the supports of $\bm{B}\bm{x}$ and $\bm{x}$ are disjoint. Now using $\bm{B}$ in (3.10) leads to

	$\displaystyle\bm{B}\circ\bm{x}\bm{x}^{\top}$	$\displaystyle=\operatorname{diag}(\bm{x})\bm{B}\operatorname{diag}(\bm{x})=\frac{1}{n}\operatorname{diag}(\bm{x})\left(\bm{M}-\bm{S}\right)\bm{x}\bm{x}^{\top}+\frac{1}{n}\bm{x}\bm{x}^{\top}\left(\bm{M}-\bm{S}\right)\operatorname{diag}(\bm{x})$
		$\displaystyle=\frac{1}{n}\operatorname{diag}(\bm{x})\bm{B}\bm{x}\bm{x}^{\top}+\frac{1}{n}\bm{x}\bm{x}^{\top}\bm{B}\operatorname{diag}(\bm{x})=0$

where $\operatorname{diag}(\bm{x})\bm{x}=\bm{x}$ and $\operatorname{diag}(\bm{x})\bm{B}\bm{x}=0.$

Finally, to ensure $\bm{B}\geq 0$ , it suffices to have

(\bm{M}-\bm{S})\bm{x}=\bm{\Delta}^{2}+\bm{\Delta}\bm{A}-\bm{A}\bm{\Delta}-(\bm{T}+\bm{J}_{n}\bm{Z}+\bm{K}+\bm{H}\bm{J}_{n})\geq 0

due to the construction of $\bm{B}$ in (3.10).

Then it holds that

	$\displaystyle\bm{Q}$	$\displaystyle=\bm{M}-\bm{S}-\bm{B}$
		$\displaystyle=\bm{M}-\bm{S}-\frac{1}{n}\left(\bm{M}-\bm{S}\right)\bm{x}\bm{x}^{\top}-\frac{1}{n}\bm{x}\bm{x}^{\top}\left(\bm{M}-\bm{S}\right)\left(\bm{I}_{n^{2}}-\frac{\bm{x}\bm{x}^{\top}}{n}\right)$
		$\displaystyle=\left(\bm{I}_{n^{2}}-\frac{\bm{x}\bm{x}^{\top}}{n}\right)\left(\bm{M}-\bm{S}\right)\left(\bm{I}_{n^{2}}-\frac{\bm{x}\bm{x}^{\top}}{n}\right)$

where $\bm{x}^{\top}(\bm{M}-\bm{S})\bm{x}=0.$ ∎

Finally, we summarize our findings: by choosing $\bm{B}$ in the form of (3.10), it suffices to find $\bm{T}$ , $\bm{K}$ , $\bm{Z}$ and $\bm{H}$ such that

		$\displaystyle\operatorname{diag}(\bm{T}+\bm{K})+(\bm{Z}+\bm{H})\bm{1}_{n}=\operatorname{diag}(\bm{\Delta}^{2}),$		(3.13)
		$\displaystyle\bm{\Delta}^{2}+\bm{\Delta}\bm{A}-\bm{A}\bm{\Delta}-(\bm{T}+\bm{J}_{n}\bm{Z}+\bm{K}+\bm{H}\bm{J}_{n})\geq 0,$
		$\displaystyle\left(\bm{I}_{n^{2}}-\frac{\bm{x}\bm{x}^{\top}}{n}\right)\left(\bm{M}-(\bm{T}\otimes\bm{I}_{n}+\bm{Z}\otimes\bm{J}_{n}+\bm{I}_{n}\otimes\bm{K}+\bm{J}_{n}\otimes\bm{H})\right)\left(\bm{I}_{n^{2}}-\frac{\bm{x}\bm{x}^{\top}}{n}\right)\succeq 0.$

where the first two constraints ensure $\bm{B}\circ\bm{x}\bm{x}^{\top}=0$ and $\bm{B}\geq 0$ respectively, and third one corresponds to $\bm{Q}\succeq 0.$

Now we proceed to prove Theorem 2.1. The proof relies on the following proposition that assume $\bm{\Delta}=0$ . In the noiseless case, i.e., $\bm{C}=\bm{A}$ with $\bm{\Delta}=0$ :

\displaystyle\bm{M}

\displaystyle=(\bm{I}_{n}\otimes\bm{A}-\bm{A}\otimes\bm{I}_{n})^{2}=\sum_{i=1}^{n}\sum_{j=1}^{n}(\lambda_{i}-\lambda_{j})^{2}\bm{u}_{i}\bm{u}_{i}^{\top}\otimes\bm{u}_{j}\bm{u}_{j}^{\top}

where $(\lambda_{i},\bm{u}_{i})$ is the $i$ th eigenvalue and eigenvector of $\bm{A}.$ The smallest eigenvectors of $\bm{M}$ are 0 with corresponding eigenvectors $\{\bm{u}_{i}\otimes\bm{u}_{i},1\leq i\leq n\}.$

Theorem 3.3 (Noise-free version of Theorem 2.1).

Suppose $|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}>0$ for all $1\leq i\leq n$ and the eigenvalues of $\bm{A}$ are distinct. Moreover, by letting

\bm{T}=-t\bm{J}_{n},~{}\bm{K}=-t\bm{J}_{n},~{}\bm{H}=t\bm{J}_{n}/n,~{}\bm{Z}=t\bm{J}_{n}/n,

(3.14)

then the second smallest eigenvalue of

\bm{Q}=\bm{M}-\bm{S}=\sum_{i=1}^{n}\sum_{j=1}^{n}(\lambda_{i}-\lambda_{j})^{2}\bm{u}_{i}\bm{u}_{i}^{\top}\otimes\bm{u}_{j}\bm{u}_{j}^{\top}+t\bm{J}_{n}\otimes(\bm{I}_{n}-\bm{J}_{n}/n)+t(\bm{I}_{n}-\bm{J}_{n}/n)\otimes\bm{J}_{n}.

is bounded below by

\lambda_{2}(\bm{Q})\geq\frac{2}{n}\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}\cdot\min_{1\leq i\leq n}|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}

for sufficiently large $t>0$ , and $\bm{Q}\bm{x}=0.$

Proof: .

Under (3.14), it holds

	$\displaystyle\bm{S}$	$\displaystyle=-t\left(\bm{J}_{n}\otimes(\bm{I}_{n}-\bm{J}_{n}/n)+t(\bm{I}_{n}-\bm{J}_{n}/n)\otimes\bm{J}_{n}\right),$
	$\displaystyle\bm{Q}$	$\displaystyle=\bm{M}-\bm{S}=\bm{M}+t\bm{J}_{n}\otimes(\bm{I}_{n}-\bm{J}_{n}/n)+t(\bm{I}_{n}-\bm{J}_{n}/n)\otimes\bm{J}_{n}.$

It is easy to verify that $\bm{M}\bm{x}=\bm{S}\bm{x}=0$ , and thus $\bm{Q}\bm{x}=0$ and $\bm{B}=0.$

Next, we will show $\lambda_{2}(\bm{M}-\bm{S})>0$ under the assumption of this theorem. Before proceeding, we introduce a few notations. Let $\bm{U}=[\bm{u}_{1},\cdots,\bm{u}_{n}]$ consist of $n$ eigenvectors of $\bm{A}$ with corresponding eigenvalues $\bm{\Lambda}=\operatorname{diag}(\lambda_{1},\cdots,\lambda_{n})$ , i.e., $\bm{A}=\bm{U}\bm{\Sigma}\bm{U}^{\top}.$ Let

\bm{\Phi}=[\bm{u}_{1}\otimes\bm{u}_{1},\cdots,\bm{u}_{n}\otimes\bm{u}_{n}]\in\hbox{\msbm{R}}^{n^{2}\times n}

be the eigenvectors of $\bm{M}$ w.r.t. eigenvalue $0$ . Then it holds that

\bm{\Phi}\bm{\Phi}^{\top}\bm{x}=\sum_{i=1}^{n}(\bm{u}_{i}\bm{u}_{i}^{\top}\otimes\bm{u}_{i}\bm{u}_{i}^{\top})\bm{x}=\bm{x},~{}~{}~{}~{}\bm{\Phi}^{\top}\bm{x}=\bm{1}_{n}.

(3.15)

Let $\bm{P}=\bm{J}_{n}/n$ , and

\displaystyle\bar{\bm{P}}

\displaystyle=(\bm{P}\otimes\bm{I}_{n}-\bm{I}_{n}\otimes\bm{P})^{2}=\frac{1}{n}\left(\bm{J}_{n}\otimes\left(\bm{I}_{n}-\frac{\bm{J}_{n}}{n}\right)+\left(\bm{I}_{n}-\frac{\bm{J}_{n}}{n}\right)\otimes\bm{J}_{n}\right)~{}~{}~{}\bm{\Pi}=\bm{I}_{n^{2}}-\frac{\bm{x}\bm{x}^{\top}}{n},

where $\bar{\bm{P}}$ , $\bm{P}$ , $\bm{\Pi},$ and $\bm{\Pi}-\bar{\bm{P}}$ are all projection matrices satisfying $\bm{\Pi}\bar{\bm{P}}=\bar{\bm{P}}$ and $\bar{\bm{P}}\bm{x}=0.$

Our goal is to show that for some sufficiently large $t>0$ , it holds

\displaystyle\bm{M}-\bm{S}

\displaystyle=\bm{M}+t\bm{J}_{n}\otimes(\bm{I}_{n}-\bm{J}_{n}/n)+t(\bm{I}_{n}-\bm{J}_{n}/n)\otimes\bm{J}_{n}=\bm{M}+nt\bar{\bm{P}}\succeq c\bm{\Pi}

for some $c>0$ . Lemma 4.2 implies that it suffices to prove that

(\bm{\Pi}-\bar{\bm{P}})\bm{M}(\bm{\Pi}-\bar{\bm{P}})\succ c(\bm{\Pi}-\bar{\bm{P}}).

Note that

\bm{M}\succeq\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}\cdot(\bm{I}_{n^{2}}-\bm{\Phi}\bm{\Phi}^{\top})

and thus it remains to show that $(\bm{\Pi}-\bar{\bm{P}})(\bm{I}_{n^{2}}-\bm{\Phi}\bm{\Phi}^{\top})(\bm{\Pi}-\bar{\bm{P}})\succeq c^{\prime}(\bm{\Pi}-\bar{\bm{P}})$ for some $c^{\prime}>0$ . Since

\displaystyle(\bm{\Pi}-\bar{\bm{P}})(\bm{I}_{n^{2}}-\bm{\Phi}\bm{\Phi}^{\top})(\bm{\Pi}-\bar{\bm{P}})=(\bm{\Pi}-\bar{\bm{P}})-(\bm{\Pi}-\bar{\bm{P}})\bm{\Phi}\bm{\Phi}^{\top}(\bm{\Pi}-\bar{\bm{P}}),

it remains to show that $\lambda_{\max}((\bm{\Pi}-\bar{\bm{P}})\bm{\Phi}\bm{\Phi}^{\top}(\bm{\Pi}-\bar{\bm{P}}))\leq 1-c^{\prime}$ and equivalently

\lambda_{\max}(\bm{\Phi}^{\top}(\bm{\Pi}-\bar{\bm{P}})\bm{\Phi})\leq 1-c^{\prime}\Longleftrightarrow\lambda_{\min}\left(\bm{\Phi}^{\top}\left(\frac{\bm{x}\bm{x}^{\top}}{n}+\bar{\bm{P}}\right)\bm{\Phi}\right)\geq c^{\prime}

which follows from

\bm{\Phi}^{\top}(\bm{\Pi}-\bar{\bm{P}})\bm{\Phi}=\bm{I}_{n}-\bm{\Phi}^{\top}\left(\frac{\bm{x}\bm{x}^{\top}}{n}+\bar{\bm{P}}\right)\bm{\Phi}.

Since $\bar{\bm{P}}=n^{-2}(\bm{J}_{n}\otimes\bm{I}_{n}-\bm{I}_{n}\otimes\bm{J}_{n})^{2}$ , we first compute

	$\displaystyle(\bm{J}_{n}\otimes\bm{I}_{n}-\bm{I}_{n}\otimes\bm{J}_{n})\bm{\Phi}$	$\displaystyle=(\bm{J}_{n}\otimes\bm{I}_{n}-\bm{I}_{n}\otimes\bm{J}_{n})[\bm{u}_{1}\otimes\bm{u}_{1},\cdots,\bm{u}_{n}\otimes\bm{u}_{n}]$
		$\displaystyle=\left([\bm{1}_{n}\otimes\bm{u}_{1},\cdots,\bm{1}_{n}\otimes\bm{u}_{n}]-[\bm{u}_{1}\otimes\bm{1}_{n},\cdots,\bm{u}_{n}\otimes\bm{1}_{n}]\right)\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})$
		$\displaystyle=(\bm{1}_{n}\otimes\bm{U}-\bm{U}\otimes\bm{1}_{n})\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})$

and then

	$\displaystyle\bm{\Phi}^{\top}\bar{\bm{P}}\bm{\Phi}$	$\displaystyle=\frac{1}{n^{2}}\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})(\bm{1}_{n}^{\top}\otimes\bm{U}^{\top}-\bm{U}^{\top}\otimes\bm{1}_{n}^{\top})(\bm{1}_{n}\otimes\bm{U}-\bm{U}\otimes\bm{1}_{n})\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})$
		$\displaystyle=\frac{1}{n^{2}}\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})(2n\bm{I}_{n}-\bm{U}^{\top}\bm{1}_{n}\otimes\bm{1}_{n}^{\top}\bm{U}-\bm{1}_{n}^{\top}\bm{U}\otimes\bm{U}^{\top}\bm{1}_{n})\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})$
		$\displaystyle=\frac{2}{n}\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})(\bm{I}_{n}-\bm{U}^{\top}\bm{P}\bm{U})\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n}).$

where $\bm{U}^{\top}\bm{1}_{n}\otimes\bm{1}_{n}^{\top}\bm{U}=\bm{U}^{\top}\bm{J}_{n}\bm{U}.$

Using (3.15), we have

\displaystyle\bm{\Phi}^{\top}(\bar{\bm{P}}+n^{-1}\bm{x}\bm{x}^{\top})\bm{\Phi}

\displaystyle=\frac{2}{n}\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})(\bm{I}_{n}-\bm{U}^{\top}\bm{P}\bm{U})\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})+\frac{\bm{J}_{n}}{n}

where $(\bm{I}_{n}-\bm{U}^{\top}\bm{P}\bm{U})\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})\bm{1}_{n}=(\bm{I}_{n}-\bm{U}^{\top}\bm{P}\bm{U})\bm{U}^{\top}\bm{1}_{n}=0$ . Therefore, we only need to control the second smallest eigenvalue of the first term.

Since $\bm{U}^{\top}\bm{P}\bm{U}$ is rank-1 and $\operatorname{Tr}(\bm{U}^{\top}\bm{P}\bm{U})=1$ , we have $\bm{I}_{n}-\bm{U}^{\top}\bm{P}\bm{U}$ is a projection matrix. The second smallest eigenvalue of $\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})(\bm{I}_{n}-\bm{U}^{\top}\bm{P}\bm{U})\operatorname{diag}(\bm{U}^{\top}\bm{1}_{n})$ is lower bounded by $\min|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}.$ Therefore, we have

\lambda_{\min}(\bm{\Phi}^{\top}(\bar{\bm{P}}+n^{-1}\bm{x}\bm{x}^{\top})\bm{\Phi})\geq\min\left\{\frac{2|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}}{n},1\right\}\geq\frac{2\min|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}}{n}=:c^{\prime}.

(3.16)

where $\min_{i}|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}\leq 1$ and it holds $(\bm{\Pi}-\bar{\bm{P}})(\bm{I}_{n^{2}}-\bm{\Phi}\bm{\Phi}^{\top})(\bm{\Pi}-\bar{\bm{P}})\succeq c^{\prime}(\bm{\Pi}-\bar{\bm{P}}).$ Finally, we have

	$\displaystyle(\bm{\Pi}-\bar{\bm{P}})\bm{M}(\bm{\Pi}-\bar{\bm{P}})$	$\displaystyle\succeq\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}(\bm{\Pi}-\bar{\bm{P}})(\bm{I}_{n^{2}}-\bm{\Phi}\bm{\Phi}^{\top})(\bm{\Pi}-\bar{\bm{P}})$
		$\displaystyle\succeq c^{\prime}\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}(\bm{\Pi}-\bar{\bm{P}}).$

Lemma 4.2 implies that

\bm{M}+nt\bar{\bm{P}}\succeq\left(\frac{2}{n}\min|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}\cdot\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}-\epsilon\right)(\bm{I}_{n^{2}}-\bm{\Pi})

for a sufficiently large $t$ where $\epsilon>0$ can be arbitrarily small for a sufficiently large $t$ . ∎

Proof of Theorem 2.1.

Theorem 3.1 indicates that if the second smallest eigenvalue of $\bm{Q}$ is nonnegative, then the exactness holds, i.e., $\bm{X}=\bm{x}\bm{x}^{\top}$ is a global minimizer to (2.5).

In presence of noise, $\bm{M}$ equals

	$\displaystyle\bm{M}$	$\displaystyle=(\bm{I}_{n}\otimes\bm{A}-\bm{A}\otimes\bm{I}_{n}-\bm{\Delta}\otimes\bm{I}_{n})^{2}$
		$\displaystyle=(\bm{I}_{n}\otimes\bm{A}-\bm{A}\otimes\bm{I}_{n})^{2}-2\bm{\Delta}\otimes\bm{A}+(\bm{\Delta}\bm{A}+\bm{A}\bm{\Delta}+\bm{\Delta}^{2})\otimes\bm{I}_{n}.$

Let

	$\displaystyle\bm{T}$	$\displaystyle=-t\bm{J}_{n}+\bm{\Delta}^{2}+\bm{\Delta}\bm{A}+\bm{A}\bm{\Delta}-2\operatorname{ddiag}(\bm{A}\bm{\Delta})-c(\bm{J}_{n}-\bm{I}_{n}),$		(3.17)
	$\displaystyle\bm{K}$	$\displaystyle=-t\bm{J}_{n},~{}~{}\bm{H}=t\bm{J}_{n}/n,~{}~{}\bm{Z}=t\bm{J}_{n}/n,$		(3.17)

for some $c$ and $t>0$ . Now we verify (3.9) in Proposition 3.2:

\displaystyle\operatorname{diag}(\bm{T}+\bm{K})+(\bm{Z}+\bm{H})\bm{1}_{n}

\displaystyle=-t\bm{1}_{n}+\operatorname{diag}(\bm{\Delta}^{2})-t\bm{1}_{n}+2t\bm{1}_{n}=\operatorname{diag}(\bm{\Delta}^{2})

where $\operatorname{diag}(\bm{T})=-t\bm{1}_{n}+\operatorname{diag}(\bm{\Delta}^{2}).$ Therefore, we can choose $\bm{B}$ in the form of (3.10). To ensure the KKT condition (3.13), it suffices to have (3.11) holds so that $\bm{B}\geq 0$ and $\bm{Q}$ in (3.12) is positive semidefintie. For the first requirement, we have

\displaystyle\bm{T}+\bm{J}_{n}\bm{Z}+\bm{K}+\bm{H}\bm{J}_{n}

\displaystyle=-c(\bm{J}_{n}-\bm{I}_{n})+\bm{\Delta}^{2}+\bm{\Delta}\bm{A}+\bm{A}\bm{\Delta}-2\operatorname{ddiag}(\bm{A}\bm{\Delta})

and thus $\bm{B}\geq 0$ is guaranteed if

\bm{\Delta}^{2}+\bm{\Delta}\bm{A}-\bm{A}\bm{\Delta}-(\bm{T}+\bm{J}_{n}\bm{Z}+\bm{K}+\bm{H}\bm{J}_{n})=c(\bm{J}_{n}-\bm{I}_{n})+2\operatorname{ddiag}(\bm{A}\bm{\Delta})-2\bm{A}\bm{\Delta}\geq 0

provided that $c+2\min_{i\neq j}\langle\bm{A}_{i},\bm{\Delta}_{j}\rangle\geq 0$ .

For $\bm{Q}\succeq 0$ , we note

\displaystyle\bm{S}

\displaystyle=(\bm{\Delta}^{2}+\bm{\Delta}\bm{A}+\bm{A}\bm{\Delta}-2\operatorname{ddiag}(\bm{A}\bm{\Delta})-c(\bm{J}_{n}-\bm{I}_{n}))\otimes\bm{I}_{n}-nt\bar{\bm{P}}

where $\bar{\bm{P}}:=n^{-2}(\bm{J}_{n}\otimes\bm{I}_{n}-\bm{I}_{n}\otimes\bm{J}_{n})^{2}.$ Then

	$\displaystyle\bm{M}-\bm{S}$	$\displaystyle=(\bm{I}_{n}\otimes\bm{A}-\bm{A}\otimes\bm{I}_{n})^{2}-2\bm{\Delta}\otimes\bm{A}+(\bm{\Delta}\bm{A}+\bm{A}\bm{\Delta}+\bm{\Delta}^{2})\otimes\bm{I}_{n}$
		$\displaystyle\qquad-(\bm{\Delta}^{2}+\bm{\Delta}\bm{A}+\bm{A}\bm{\Delta}-2\operatorname{ddiag}(\bm{A}\bm{\Delta})-c(\bm{J}_{n}-\bm{I}_{n}))\otimes\bm{I}_{n}+nt\bar{\bm{P}}$
		$\displaystyle=(\bm{I}_{n}\otimes\bm{A}-\bm{A}\otimes\bm{I}_{n})^{2}+nt\bar{\bm{P}}-2\bm{\Delta}\otimes\bm{A}+(2\operatorname{ddiag}(\bm{A}\bm{\Delta})+c(\bm{J}_{n}-\bm{I}_{n}))\otimes\bm{I}_{n}.$

By choosing $\bm{B}$ in the form of (3.10), we have $\bm{Q}=(\bm{I}_{n^{2}}-n^{-1}\bm{x}\bm{x}^{\top})(\bm{M}-\bm{S})(\bm{I}_{n^{2}}-n^{-1}\bm{x}\bm{x}^{\top})$ and the second smallest eigenvalue satisfies

	$\displaystyle\lambda_{2}\left(\bm{Q}\right)$	$\displaystyle\geq\lambda_{2}((\bm{I}_{n}\otimes\bm{A}-\bm{A}\otimes\bm{I}_{n})^{2}+nt\bar{\bm{P}})-2\\|\bm{\Delta}\otimes\bm{A}\\|+2\min_{1\leq i\leq n}\langle\bm{A}_{i},\bm{\Delta}_{i}\rangle-c$		(3.18)
	(Theorem 3.3)	$\displaystyle\geq\frac{2}{n}\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}\min\|\langle\bm{u}_{i},\bm{1}_{n}\rangle\|^{2}-2\\|\bm{\Delta}\\|\\|\bm{A}\\|-2\\|\bm{A}\bm{\Delta}\\|_{\max}$		(3.18)

where $\|\bm{\Delta}\otimes\bm{A}\|=\|\bm{A}\|\|\bm{\Delta}\|$ , $c=-2\min_{i\neq j}\langle\bm{A}_{i},\bm{\Delta}_{j}\rangle$ is chosen, and $t$ is sufficiently large. Then by Theorem 3.1, the exactness holds if $\lambda_{2}(\bm{Q})>0$ , i.e., $\bm{X}=\bm{x}\bm{x}^{\top}$ is the unique global minimizer to (2.5). ∎

4 Proof of Theorem 2.2

4.1 Dual program and optimality condition of (2.6)

We start with deriving the dual form of (2.6). For each constraint, we assign a dual variable:

	$\displaystyle\begin{bmatrix}\bm{Q}&\bm{q}\\ \bm{q}&z\end{bmatrix}\succeq 0$	$\displaystyle:\begin{bmatrix}\bm{X}&\operatorname{diag}(\bm{X})\\ \operatorname{diag}(\bm{X})^{\top}&1\end{bmatrix}\succeq 0,$
	$\displaystyle\bm{B}\geq 0$	$\displaystyle:\bm{X}\geq 0,$
	$\displaystyle\bm{T}\in\hbox{\msbm{R}}^{n\times n}$	$\displaystyle:\operatorname{Tr}(\bm{X}_{ij})=\delta_{ij},~{}~{}1\leq i,j\leq n,$
	$\displaystyle\bm{Z}\in\hbox{\msbm{R}}^{n\times n}$	$\displaystyle:\langle\bm{X}_{ij},\bm{J}_{n}\rangle=1,~{}~{}1\leq i,j\leq n,$
	$\displaystyle\bm{K}\in\hbox{\msbm{R}}^{n\times n}$	$\displaystyle:\sum_{i=1}^{n}\bm{X}_{ii}=\bm{I}_{n},$
	$\displaystyle\bm{H}\in\hbox{\msbm{R}}^{n\times n}$	$\displaystyle:\sum_{i,j}\bm{X}_{ij}=\bm{J}_{n},$
	$\displaystyle\bm{\mu}\in\hbox{\msbm{R}}^{n}$	$\displaystyle:\operatorname{mat}(\operatorname{diag}(\bm{X}))\bm{1}_{n}=\bm{1}_{n},$
	$\displaystyle\bm{\lambda}\in\hbox{\msbm{R}}^{n}$	$\displaystyle:\operatorname{mat}(\operatorname{diag}(\bm{X}))^{\top}\bm{1}_{n}=\bm{1}_{n},$

where $\operatorname{diag}(\bm{X})$ takes the diagonal entries of $\bm{X}$ and forms them into a column vector.

For the constraint $\operatorname{mat}(\operatorname{diag}(\bm{X}))$ ,

	$\displaystyle\bm{\mu}^{\top}(\operatorname{mat}(\operatorname{diag}(\bm{X}))\bm{1}_{n}-\bm{1}_{n})$	$\displaystyle=\langle\operatorname{mat}(\operatorname{diag}(\bm{X})),\bm{\mu}\bm{1}_{n}^{\top}\rangle-\langle\bm{\mu},\bm{1}_{n}\rangle$
		$\displaystyle=\langle\operatorname{diag}(\bm{X}),\bm{1}\otimes\bm{\mu}\rangle-\langle\bm{\mu},\bm{1}_{n}\rangle=\langle\bm{I}_{n}\otimes\operatorname{diag}(\bm{\mu}),\bm{X}\rangle-\langle\bm{\mu},\bm{1}_{n}\rangle$
	$\displaystyle\bm{\lambda}^{\top}(\operatorname{mat}(\operatorname{diag}(\bm{X}))^{\top}\bm{1}_{n}-\bm{1}_{n})$	$\displaystyle=\langle\operatorname{mat}(\operatorname{diag}(\bm{X})),\bm{1}_{n}\bm{\lambda}^{\top}\rangle-\langle\bm{\lambda},\bm{1}_{n}\rangle$
		$\displaystyle=\langle\operatorname{diag}(\bm{X}),\bm{\lambda}\otimes\bm{1}_{n}\rangle-\langle\bm{\lambda},\bm{1}_{n}\rangle=\langle\operatorname{diag}(\bm{\lambda})\otimes\bm{I}_{n},\bm{X}\rangle-\langle\bm{\lambda},\bm{1}_{n}\rangle$

where $\operatorname{vec}(\bm{u}\bm{v}^{\top})=\bm{v}\otimes\bm{u}.$ Also

\left\langle\begin{bmatrix}\bm{X}&\operatorname{diag}(\bm{X})\\ \operatorname{diag}(\bm{X})^{\top}&1\end{bmatrix},\begin{bmatrix}\bm{Q}&\bm{q}\\ \bm{q}&z\end{bmatrix}\right\rangle=\langle\bm{X},\bm{Q}\rangle+2\langle\bm{X},\operatorname{diag}(\bm{q})\rangle+z

where $\langle\operatorname{diag}(\bm{X}),\bm{q}\rangle=\langle\bm{X},\operatorname{diag}(\bm{q})\rangle.$

Now the Lagrangian function is

	$\displaystyle{\cal L}(\bm{X},\bm{Q},\bm{B},\bm{T},\bm{Z},\bm{K},\bm{H},\bm{\mu},\bm{\lambda},\bm{q},z)$
	$\displaystyle\qquad:=\langle\bm{X},\bm{M}-\bm{Q}-2\operatorname{diag}(\bm{q})-\bm{B}-(\bm{I}_{n}\otimes\operatorname{diag}(\bm{\mu})+\operatorname{diag}(\bm{\lambda})\otimes\bm{I}_{n})\rangle+\langle\bm{\lambda}+\bm{\mu},\bm{1}_{n}\rangle$
	$\displaystyle\qquad\quad-\sum_{i,j}t_{ij}(\operatorname{Tr}(\bm{X}_{ij})-\delta_{ij})-\sum_{i,j}z_{ij}(\langle\bm{X}_{ij},\bm{J}_{n}\rangle-1)-\left\langle\sum_{i=1}^{n}\bm{X}_{ii}-\bm{I}_{n},\bm{K}\right\rangle-\left\langle\sum_{i,j}\bm{X}_{ij}-\bm{J}_{n},\bm{H}\right\rangle-z$
	$\displaystyle\qquad=\left\langle\bm{X},\bm{M}-\bm{Q}-2\operatorname{diag}(\bm{q})-\bm{B}-\operatorname{diag}(\bm{1}\otimes\bm{\mu}+\bm{\lambda}\otimes\bm{1}_{n})-\bm{T}\otimes\bm{I}_{n}-\bm{Z}\otimes\bm{J}_{n}-\bm{I}_{n}\otimes\bm{K}-\bm{J}_{n}\otimes\bm{H}\right\rangle$
	$\displaystyle\qquad\quad+\operatorname{Tr}(\bm{T}+\bm{K})+\langle\bm{J}_{n},\bm{Z}+\bm{H}\rangle+\langle\bm{\lambda}+\bm{\mu}\rangle-z.$

We define

\bm{\Lambda}=\bm{I}_{n}\otimes\operatorname{diag}(\bm{\mu})+\operatorname{diag}(\bm{\lambda})\otimes\bm{I}_{n},~{}~{}~{}~{}\bm{S}=\bm{T}\otimes\bm{I}_{n}+\bm{I}_{n}\otimes\bm{K}+\bm{J}_{n}\otimes\bm{H}+\bm{Z}\otimes\bm{J}_{n}.

(4.1)

Then the Lagrangian equals

{\cal L}(\cdot)=\langle\bm{X},\bm{M}-\bm{Q}-2\operatorname{diag}(\bm{q})-\bm{B}-\bm{\Lambda}-\bm{S}\rangle+\operatorname{Tr}(\bm{T}+\bm{K})+\langle\bm{J}_{n},\bm{Z}+\bm{H}\rangle+\langle\bm{\lambda}+\bm{\mu},\bm{1}_{n}\rangle-z.

The resulting dual program becomes

$\displaystyle\max{}$	$\displaystyle\operatorname{Tr}(\bm{T}+\bm{K})+\langle\bm{J}_{n},\bm{Z}+\bm{H}\rangle+\langle\bm{\lambda}+\bm{\mu},\bm{1}_{n}\rangle-z,$	(4.2)
s.t.	$\displaystyle\bm{M}=\bm{Q}+2\operatorname{diag}(\bm{q})+\bm{B}+\bm{\Lambda}+\bm{S},$
	$\displaystyle\begin{bmatrix}\bm{Q}&\bm{q}\\ \bm{q}&z\end{bmatrix}\succeq 0,~{}~{}\bm{B}\geq 0.$

As a result, the KKT conditions are

1.

Stationarity:

$\bm{M}=\bm{Q}+2\operatorname{diag}(\bm{q})+\bm{B}+\bm{\Lambda}+\bm{S},$
2.

Dual feasibility:

$\begin{bmatrix}\bm{Q}&\bm{q}\\ \bm{q}^{\top}&z\end{bmatrix}\succeq 0,~{}~{}~{}~{}\bm{B}\geq 0,$

Complementary slackness:

\left\langle\begin{bmatrix}\bm{X}&\operatorname{diag}(\bm{X})\\ \operatorname{diag}(\bm{X})^{\top}&1\end{bmatrix},\begin{bmatrix}\bm{Q}&\bm{q}\\ \bm{q}&z\end{bmatrix}\right\rangle=0,~{}~{}~{}\langle\bm{B},\bm{X}\rangle=0.

Suppose $\bm{X}=\bm{x}\bm{x}^{\top}$ is a global minimizer, and then we can try to simplify the KKT optimality condition by determining some of the dual variables. Using the complementary slackness condition gives rise to

\displaystyle\begin{bmatrix}\bm{Q}&\bm{q}\\ \bm{q}^{\top}&z\end{bmatrix}\begin{bmatrix}\bm{x}\\ 1\end{bmatrix}=0,~{}~{}~{}~{}\langle\bm{B},\bm{x}\bm{x}^{\top}\rangle=0,

which leads to $\bm{Q}\bm{x}+\bm{q}=0,z=-\langle\bm{q},\bm{x}\rangle,$ and $\bm{B}\circ\bm{x}\bm{x}^{\top}=0.$ Now we look into the KKT conditions again: for the dual feasibility, it holds that

\begin{bmatrix}\bm{Q}&-\bm{Q}\bm{x}\\ -(\bm{Q}\bm{x})^{\top}&\bm{x}^{\top}\bm{Q}\bm{x}\end{bmatrix}\succeq 0~{}\Longleftrightarrow~{}\bm{Q}\succeq 0.

For the stationarity, we have

\bm{M}=\bm{Q}-2\operatorname{diag}(\bm{Q}\bm{x})+\bm{B}+\bm{\Lambda}+\bm{S}.

This linear system $\bm{Q}-2\operatorname{diag}(\bm{Q}\bm{x})=\bm{M}-(\bm{B}+\bm{\Lambda}+\bm{S})$ has a unique solution for $\bm{Q}$ :

\bm{Q}\bm{x}-2\operatorname{diag}(\bm{x})\bm{Q}\bm{x}=(\bm{M}-\bm{B}-\bm{\Lambda}-\bm{S})\bm{x}\Longrightarrow\bm{Q}\bm{x}=(\bm{I}-2\operatorname{diag}(\bm{x}))(\bm{M}-\bm{B}-\bm{\Lambda}-\bm{S})\bm{x}

and then

	$\displaystyle\bm{Q}$	$\displaystyle=\bm{M}-(\bm{B}+\bm{\Lambda}+\bm{S})+2\operatorname{diag}(\bm{Q}\bm{x})$
		$\displaystyle=\bm{M}-(\bm{B}+\bm{\Lambda}+\bm{S})+2(\bm{I}-2\operatorname{diag}(\bm{x}))\operatorname{diag}((\bm{M}-\bm{B}-\bm{\Lambda}-\bm{S})\bm{x})$

Therefore, the KKT condition becomes

		$\displaystyle\bm{Q}=\bm{M}-\bm{B}-\bm{\Lambda}-\bm{S}+2(\bm{I}-2\operatorname{diag}(\bm{x}))\operatorname{diag}((\bm{M}-\bm{B}-\bm{\Lambda}-\bm{S})\bm{x})\succeq 0,$		(4.3)
		$\displaystyle\bm{B}\geq 0,~{}\bm{B}\circ\bm{x}\bm{x}^{\top}=0,$		(4.3)

where $\bm{S}$ is defined in (4.1). Next, we show that (4.3) implies the (unique) global optimality of $\bm{X}.$

Theorem 4.1.

Suppose that there exist $\bm{B}$ , $\bm{S}$ , and $\bm{\Lambda}$ such that (4.3) holds, then $\bm{X}=\bm{x}\bm{x}^{\top}$ is a global minimizer. Moreover, if $\bm{Q}\succ 0$ , then $\bm{X}$ is the unique global minimizer.

Proof of Theorem 4.1.

Suppose that

\bm{Q}=\bm{M}-\bm{B}-\bm{\Lambda}-\bm{S}+2\operatorname{diag}(\bm{Q}\bm{x})\succeq 0

which gives

\bm{x}^{\top}\bm{Q}\bm{x}=-\bm{x}^{\top}(\bm{M}-\bm{B}-\bm{\Lambda}-\bm{S})\bm{x}.

For any feasible solution $\widehat{\bm{X}}$ , we have

		$\displaystyle\left\langle\begin{bmatrix}\bm{Q}&-\bm{Q}\bm{x}\\ -(\bm{Q}\bm{x})^{\top}&\bm{x}^{\top}\bm{Q}\bm{x}\end{bmatrix},\begin{bmatrix}\widehat{\bm{X}}&\operatorname{diag}(\widehat{\bm{X}})\\ \operatorname{diag}(\widehat{\bm{X}})^{\top}&1\end{bmatrix}\right\rangle=\langle\widehat{\bm{X}},\bm{Q}-2\operatorname{diag}(\bm{Q}\bm{x})\rangle+\bm{x}^{\top}\bm{Q}\bm{x}$		(4.4)
		$\displaystyle\qquad=\langle\widehat{\bm{X}}-\bm{x}\bm{x}^{\top},\bm{M}-\bm{B}-\bm{\Lambda}-\bm{S}\rangle=\langle\widehat{\bm{X}}-\bm{x}\bm{x}^{\top},\bm{M}\rangle-\langle\widehat{\bm{X}},\bm{B}\rangle\geq 0$		(4.4)

where $\langle\widehat{\bm{X}}-\bm{x}\bm{x}^{\top},\bm{\Lambda}+\bm{S}\rangle=0$ follows from the feasibility of $\widehat{\bm{X}}$ and $\bm{x}\bm{x}^{\top}$ , and $\langle\bm{x}\bm{x}^{\top},\bm{B}\rangle=0.$ Therefore, it holds $\langle\widehat{\bm{X}}-\bm{x}\bm{x}^{\top},\bm{M}\rangle\geq\langle\widehat{\bm{X}},\bm{B}\rangle\geq 0$ which implies $\bm{X}=\bm{x}\bm{x}^{\top}$ is a global minimizer to (2.6). In particular, if $\bm{Q}\succ 0$ , then $\bm{X}$ is the unique global minimizer since (4.4) becomes strictly positive for any $\widehat{\bm{X}}\neq\bm{x}\bm{x}^{\top}.$ ∎

Proof of Theorem 2.2.

Consider the dual certificate $\bm{T},\bm{K},\bm{Z}$ and $\bm{H}$ in (3.17), and then it holds that

\operatorname{diag}(\bm{T}+\bm{K})+(\bm{Z}+\bm{H})\bm{1}_{n}=\operatorname{diag}(\bm{\Delta}^{2}),~{}~{}~{}\bm{B}\bm{x}=(\bm{M}-\bm{S})\bm{x}

where $\bm{B}$ is chosen in the form of (3.10). As a result, $\bm{Q}$ satisfies

	$\displaystyle\bm{Q}$	$\displaystyle=\bm{M}-\bm{B}-\bm{\Lambda}-\bm{S}+2(\bm{I}-2\operatorname{diag}(\bm{x}))\operatorname{diag}((\bm{M}-\bm{B}-\bm{\Lambda}-\bm{S})\bm{x})$
		$\displaystyle=\bm{\Pi}(\bm{M}-\bm{S})\bm{\Pi}-\left(\bm{\Lambda}+2(\bm{I}-2\operatorname{diag}(\bm{x}))\operatorname{diag}(\bm{\Lambda}\bm{x})\right)$

where $\bm{B}\bm{x}=(\bm{M}-\bm{S})\bm{x}$ and $\bm{\Pi}:=\bm{I}-\bm{x}\bm{x}^{\top}/n.$ Note that (3.18) gives

\lambda_{2}\left(\bm{\Pi}(\bm{M}-\bm{S})\bm{\Pi}\right)\geq\frac{2}{n}\min_{i\neq j}(\lambda_{i}-\lambda_{j})^{2}\min|\langle\bm{u}_{i},\bm{1}_{n}\rangle|^{2}-2\|\bm{\Delta}\|\|\bm{A}\|-2\|\bm{A}\bm{\Delta}\|_{\max}.

Note that $\operatorname{mat}(\bm{\Lambda}\bm{x})=\operatorname{diag}(\bm{\lambda}+\bm{\mu})$ where $\bm{\Lambda}$ is defined in (4.1). Also we have

\operatorname{mat}(\operatorname{diag}(\bm{\Lambda})+2(\bm{I}-2\operatorname{diag}(\bm{x}))\bm{\Lambda}\bm{x})=\bm{1}_{n}\bm{\lambda}^{\top}+\bm{\mu}\bm{1}_{n}^{\top}-2\operatorname{diag}(\bm{\lambda}+\bm{\mu}).

In particular, if we choose $\bm{\lambda}$ and $\bm{\mu}$ as constant vectors, i.e., $\bm{\lambda}=\bm{\mu}=t\bm{1}_{n}$ , then

\operatorname{mat}(\operatorname{diag}(\bm{\Lambda})+2(\bm{I}-2\operatorname{diag}(\bm{x}))\bm{\Lambda}\bm{x})=2t(\bm{J}_{n}-2\bm{I}_{n})

and thus

	$\displaystyle\bm{Q}$	$\displaystyle=\bm{\Pi}(\bm{M}-\bm{S})\bm{\Pi}-2t(\bm{I}-\operatorname{diag}(\bm{x}))+2t\operatorname{diag}(\bm{x})$
		$\displaystyle\succeq\lambda_{2}(\bm{\Pi}(\bm{M}-\bm{S})\bm{\Pi})\left(\bm{I}-\frac{\bm{x}\bm{x}^{\top}}{n}\right)-2t(\bm{I}-\operatorname{diag}(\bm{x}))+2t\operatorname{diag}(\bm{x})$

For any $0<2t<\lambda_{2}(\bm{\Pi}(\bm{M}-\bm{S})\bm{\Pi})$ , then $\bm{Q}\succ 0$ holds and therefore $\bm{X}=\bm{x}\bm{x}^{\top}$ is the unique global minimizer to (2.6), following from Theorem 4.1. ∎

Appendix

Lemma 4.2.

Let $\bm{\Pi}$ and $\bm{V}$ be two matrices of same size and $\bm{P}_{\Pi}$ and $\bm{P}_{V}$ be their orthogonal projection matrices respectively that satisfy $\bm{P}_{V}(\bm{I}-\bm{P}_{\Pi})=0$ , i.e., $\operatorname{Ran}(\bm{V})\subseteq\operatorname{Ran}(\bm{\Pi})$ . Suppose

(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})\succeq c(\bm{P}_{\Pi}-\bm{P}_{V})

for some $c>0$ . Then for any $0<c^{\prime\prime}<c$ , we have

\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V}\succeq c^{\prime\prime}\bm{P}_{\Pi}

for sufficiently large $t$ .

Proof of Lemma 4.2.

We decompose $\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V}$ into

\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V}=(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})+(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}\bm{P}_{V}+\bm{P}_{V}\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})+(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V}).

Let $\bm{R}:=\bm{P}_{\Pi}-(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}$ where $\operatorname{Ran}(\bm{R})\subseteq\operatorname{Ran}(\bm{\Pi})$ , “ $\dagger$ ” stands for the Moore-Penrose pseudo-inverse and

(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}=\bm{P}_{V}.

(4.5)

Note that

(\bm{R}-\bm{P}_{\Pi})^{2}=(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}=0

where $(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}(\bm{P}_{\Pi}-\bm{P}_{V})=0$ . Using $\bm{P}_{\Pi}\bm{R}=\bm{R}\bm{P}_{\Pi}=\bm{R}$ and $(\bm{R}-\bm{P}_{\Pi})^{2}=0$ gives

\bm{R}^{2}-2\bm{R}+\bm{P}_{\Pi}=0

which implies $\bm{R}$ is invertible on $\operatorname{Ran}(\bm{\Pi})$ as 1 is an eigenvalue of $\bm{R}$ with multiplicity equal to the rank of $\bm{P}_{\Pi}.$ Therefore, $\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V}\succ_{\Pi}0$ is equivalent to $\bm{R}(\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V})\bm{R}^{\top}\succ_{\Pi}0$ . Now

	$\displaystyle\bm{R}(\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V})$
	$\displaystyle=(\bm{P}_{\Pi}-(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger})(\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V})$
	$\displaystyle=\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V}-(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}(\bm{P}_{V}\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})+(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V}))$
	$\displaystyle=\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V}-(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})-(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}\bm{P}_{V}$
	$\displaystyle=(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})+\bm{P}_{V}\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})+(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})$
	$\displaystyle\qquad-(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})$

which follows from $\bm{P}_{\Pi}\bm{P}_{V}=\bm{P}_{V}$ and (4.5). Then

	$\displaystyle\bm{R}(\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V})\bm{R}^{\top}$
	$\displaystyle\qquad=(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})+\bm{P}_{V}\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})+(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})$
	$\displaystyle\qquad~{}~{}~{}-(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})-\bm{P}_{V}\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})$
	$\displaystyle\qquad=(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})+(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})-(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})$

where $(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}=\bm{P}_{V}$ for sufficiently large $t$ .

Next, we will show that $\bm{R}(\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V})\bm{R}^{\top}$ is strictly positive semidefinite when restricted to $\bm{P}_{V}.$ Suppose $(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})\succeq c(\bm{P}_{\Pi}-\bm{P}_{V})$ , and then for a sufficiently large $t>0$ , it holds for $0<c^{\prime}<c$ that

\displaystyle(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})-(\bm{P}_{\Pi}-\bm{P}_{V})\bm{A}(\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V})^{\dagger}\bm{A}(\bm{P}_{\Pi}-\bm{P}_{V})\succeq c^{\prime}(\bm{P}_{\Pi}-\bm{P}_{V})

as the second term can be arbitrarily small for a sufficiently large $t>0.$ Then

\bm{R}(\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V})\bm{R}^{\top}\succeq c^{\prime}\bm{P}_{V}+c^{\prime}(\bm{P}_{\Pi}-\bm{P}_{V})\succeq c^{\prime}\bm{P}_{\Pi}

where $\bm{P}_{V}\bm{A}\bm{P}_{V}+t\bm{P}_{V}\succeq c^{\prime}\bm{P}_{V}$ for sufficiently large $t$ .

Note that $\bm{R}$ is invertible on $\operatorname{Ran}(\bm{\Pi})$ and also the range of $\bm{R}$ belongs to $\operatorname{Ran}(\bm{\Pi}).$ Therefore, we have

\bm{P}_{\Pi}\bm{A}\bm{P}_{\Pi}+t\bm{P}_{V}\succeq c^{\prime}(\bm{R}\bm{R}^{\top})^{\dagger}\succeq c^{\prime\prime}\bm{P}_{\Pi}

for any $c^{\prime\prime}<c$ where $\|\bm{R}\bm{R}^{\top}-\bm{P}_{\Pi}\|$ goes to 0 at the rate of $1/t$ and thus $\bm{R}\bm{R}^{\top}$ is close to $\bm{P}_{\Pi}$ for sufficiently large $t$ . ∎

References

[1] E. Abbe. Community detection and stochastic block models: recent developments. The Journal of Machine Learning Research, 18(1):6446–6531, 2017.
[2] E. Abbe, A. S. Bandeira, and G. Hall. Exact recovery in the stochastic block model. IEEE Transactions on information theory, 62(1):471–487, 2015.
[3] Y. Aflalo, A. Bronstein, and R. Kimmel. On convex relaxation of graph isomorphism. Proceedings of the National Academy of Sciences, 112(10):2942–2947, 2015.
[4] A. Ahmed, B. Recht, and J. Romberg. Blind deconvolution using convex programming. IEEE Transactions on Information Theory, 60(3):1711–1732, 2013.
[5] K. M. Anstreicher. Eigenvalue bounds versus semidefinite relaxations for the quadratic assignment problem. SIAM Journal on Optimization, 11(1):254–265, 2000.
[6] K. M. Anstreicher. Recent advances in the solution of quadratic assignment problems. Mathematical Programming, 97:27–42, 2003.
[7] K. M. Anstreicher and N. W. Brixius. A new bound for the quadratic assignment problem based on convex quadratic programming. Mathematical Programming, Series A, 89(341-357), 2001.
[8] P. Awasthi, A. S. Bandeira, M. Charikar, R. Krishnaswamy, S. Villar, and R. Ward. Relax, no need to round: Integrality of clustering formulations. In Proceedings of the 2015 Conference on Innovations in Theoretical Computer Science, pages 191–200. ACM, 2015.
[9] A. S. Bandeira. Random Laplacian matrices and convex relaxations. Foundations of Computational Mathematics, 18(2):345–379, Apr 2018.
[10] J. F. Bravo Ferreira, Y. Khoo, and A. Singer. Semidefinite programming approach for the quadratic assignment problem with a sparse graph. Computational Optimization and Applications, 69:677–712, 2018.
[11] E. J. Candes, T. Strohmer, and V. Voroninski. Phaselift: Exact and stable signal recovery from magnitude measurements via convex programming. Communications on Pure and Applied Mathematics, 66(8):1241–1274, 2013.
[12] Y. Cheng and R. Ge. Non-convex matrix completion against a semi-random adversary. In Conference On Learning Theory, pages 1362–1394. PMLR, 2018.
[13] D. Conte, P. Foggia, C. Sansone, and M. Vento. Thirty years of graph matching in pattern recognition. International journal of pattern recognition and artificial intelligence, 18(03):265–298, 2004.
[14] D. Cullina and N. Kiyavash. Improved achievability and converse bounds for erdos-rényi graph matching. ACM SIGMETRICS performance evaluation review, 44(1):63–72, 2016.
[15] E. De Klerk and R. Sotirov. Exploiting group symmetry in semidefinite programming relaxations of the quadratic assignment problem. Mathematical Programming, 122:225–246, 2010.
[16] E. de Klerk and R. Sotirov. Improved semidefinite programming bounds for quadratic assignment problems with suitable symmetry. Mathematical programming, 133:75–91, 2012.
[17] J. Ding, Z. Ma, Y. Wu, and J. Xu. Efficient random graph matching via degree profiles. Probability Theory and Related Fields, 179:29–115, 2021.
[18] Y. Ding and H. Wolkowicz. A low-dimensional semidefinite relaxation for the quadratic assignment problem. Mathematics of Operations Research, 34(4):1008–1022, 2009.
[19] Z. Fan, C. Mao, Y. Wu, and J. Xu. Spectral graph matching and regularized quadratic relaxations i algorithm and gaussian analysis. Foundations of Computational Mathematics, 23(5):1511–1565, 2023.
[20] Z. Fan, C. Mao, Y. Wu, and J. Xu. Spectral graph matching and regularized quadratic relaxations ii: Erdős-rényi graphs and universality. Foundations of Computational Mathematics, 23(5):1567–1617, 2023.
[21] S. Feizi, G. Quon, M. Recamonde-Mendoza, M. Medard, M. Kellis, and A. Jadbabaie. Spectral alignment of graphs. IEEE Transactions on Network Science and Engineering, 7(3):1182–1197, 2019.
[22] R. Feng, G. Tian, and D. Wei. Small gaps of GOE. Geometric and Functional Analysis, 29(6):1794–1827, 2019.
[23] M. Fiori and G. Sapiro. On spectral properties for graph matching and graph isomorphism problems. Information and Inference: A Journal of the IMA, 4:63–76, 2015.
[24] F. Fogel, R. Jenatton, F. Bach, and A. d’Aspremont. Convex relaxations for permutation problems. Advances in Neural Information Processing Systems, 26, 2013.
[25] L. Ganassali. Sharp threshold for alignment of graph databases with gaussian weights. In Mathematical and Scientific Machine Learning, pages 314–335. PMLR, 2022.
[26] L. Ganassali, M. Lelarge, and L. Massoulié. Spectral alignment of correlated gaussian matrices. Advances in Applied Probability, 54(1):279–310, 2022.
[27] P. C. Gilmore. Optimal and suboptimal algorithms for the quadratic assignment problem. Journal of the Society for Industrial and Applied Mathematics, 10(2):305–313, 1962.
[28] M. Grant, S. Boyd, and Y. Ye. Cvx: Matlab software for disciplined convex programming, 2008.
[29] S. C. Gutekunst and D. P. Williamson. Semidefinite programming relaxations of the traveling salesman problem and their integrality gaps. Mathematics of Operations Research, 47(1):1–28, 2022.
[30] G. Hall and L. Massoulié. Partial recovery in the graph alignment problem. Operations Research, 71(1):259–272, 2023.
[31] T. Iguchi, D. G. Mixon, J. Peterson, and S. Villar. Probably certifiably correct k-means clustering. Mathematical Programming, 165:605–642, 2017.
[32] R. M. Karp. Reducibility among combinatorial problems. Complexity of Computer Computations, pages 85–103, 1972.
[33] S. Klus and T. Sahai. A spectral assignment approach for the graph isomorphism problem. Information and Inference: A Journal of the IMA, 7:689–706, 2018.
[34] E. L. Lawler. The quadratic assignment problem. Management Science, 9(4):586–599, 1963.
[35] X. Li, Y. Li, S. Ling, T. Strohmer, and K. Wei. When do birds of a feather flock together? k-means, proximity, and conic programming. Mathematical Programming, 179:295–341, 2020.
[36] S. Ling. Improved performance guarantees for orthogonal group synchronization via generalized power method. SIAM Journal on Optimization, 32(2):1018–1048, 2022.
[37] S. Ling and T. Strohmer. Certifying global optimality of graph cuts via semidefinite relaxation: A performance guarantee for spectral clustering. Foundations of Computational Mathematics, 20(3):367–421, 2020.
[38] E. M. Loiola, N. M. M. De Abreu, P. O. Boaventura-Netto, P. Hahn, and T. Querido. A survey for the quadratic assignment problem. European Journal of Operational Research, 176(2):657–690, 2007.
[39] V. Lyzinski, D. E. Fishkind, M. Fiori, J. T. Vogelstein, C. E. Priebe, and G. Sapiro. Graph matching: Relax at your own risk. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(01):60–73, 2016.
[40] C. Mao, M. Rudelson, and K. Tikhomirov. Random graph matching with improved noise robustness. In 34th Annual Conference on Learning Theory, volume 134, pages 1–34, 2021.
[41] C. Mao, M. Rudelson, and K. Tikhomirov. Exact matching of random graphs with constant correlation. Probability Theory and Related Fields, 186(1):327–389, 2023.
[42] D. E. Oliveira, H. Wolkowicz, and Y. Xu. Admm for the sdp relaxation of the qap. Mathematical Programming Computation, 10(4):631–658, 2018.
[43] E. Onaran, S. Garg, and E. Erkip. Optimal de-anonymization in random graphs with community structure. In 2016 50th Asilomar Conference on Signals, Systems and Computers, pages 709–713. IEEE, 2016.
[44] J. Povh and F. Rendl. Copositive and semidefinite relaxations of the quadratic assignment problem. Discrete Optimization, 6(3):231–241, 2009.
[45] B. Recht. A simpler approach to matrix completion. Journal of Machine Learning Research, 12(12), 2011.
[46] F. Rendl and H. Wolkowicz. Applications of parametric programming and eigenvalue maximization to the quadratic assignment problem. Mathematical Programming, 53(1):63–78, 1992.
[47] R. Singh, J. Xu, and B. Berger. Global alignment of multiple protein interaction networks with application to functional orthology detection. Proceedings of the National Academy of Sciences, 105(35):12763–12768, 2008.
[48] L. Vandenberghe and S. Boyd. Semidefinite programming. SIAM Review, 38(1):49–95, 1996.
[49] R. Vershynin. High-dimensional Probability: An Introduction with Applications in Data Science, volume 47. Cambridge University Press, 2018.
[50] Y. Wu, J. Xu, and H. Y. Sophie. Settling the sharp reconstruction thresholds of random graph matching. IEEE Transactions on Information Theory, 68(8):5391–5417, 2022.
[51] Q. Zhao, S. E. Karisch, F. Rendl, and H. Wolkowicz. Semidefinite programming relaxations for the quadratic assignment problem. Journal of Combinatorial Optimization, 2:71–109, 1998.
[52] Y. Zhong and N. Boumal. Near-optimal bounds for phase synchronization. SIAM Journal on Optimization, 28(2):989–1016, 2018.

On the Exactness of SDP Relaxation for Quadratic Assignment Problem

Abstract

1 Introduction

1.1 Notation

1.2 Organization

2 Preliminaries and main results

2.1 Convex relaxation

Convex relaxation I.

Convex relaxation II.

2.2 Main theorems

Theorem 2.1.

Theorem 2.2.

2.3 Examples and numerics

Example: diagonal matrix plus Gaussian noise.

Example: diagonal-plus-Wigner model

Example: correlated Wigner model.

3 Proof of Theorem 2.1

3.1 Dual program and optimality condition of (2.5)

Theorem 3.1.

Proof: .

3.2 Construction of a dual certificate

Proposition 3.2 (Construction of B\bm{B} and Q\bm{Q}).

Proof: .

Theorem 3.3 (Noise-free version of Theorem 2.1).

Proof: .

Proof of Theorem 2.1.

4 Proof of Theorem 2.2

4.1 Dual program and optimality condition of (2.6)

Theorem 4.1.

Proof of Theorem 4.1.

Proof of Theorem 2.2.

Appendix

Lemma 4.2.

Proof of Lemma 4.2.

References

Proposition 3.2 (Construction of $\bm{B}$ and $\bm{Q}$ ).