\jyear

2021

[2]\fnmPeng-fei \surHuang

1]\orgdivSchool of Mathematics and Statistics, \orgnameKashi University, \orgaddress\cityKashi, \postcode844008, \countryP.R. China

2]\orgdivSchool of Mathematical Sciences and LPMC, \orgnameNankai University, \orgaddress\cityTianjin, \postcode300071, \countryP.R. China

The low-rank approximation of fourth-order partial-symmetric and conjugate partial-symmetric tensor

\fnmAmina \surSabir amina_sabir1@126.com huangpf@mail.nankai.edu.cn \fnmQing-zhi \surYang qz-yang@nankai.edu.cn [ [

Abstract

We present an orthogonal matrix outer product decomposition for the fourth-order conjugate partial-symmetric (CPS) tensor and show that the greedy successive rank-one approximation (SROA) algorithm can recover this decomposition exactly. Based on this matrix decomposition, the CP rank of CPS tensor can be bounded by the matrix rank, which can be applied to low rank tensor completion. Additionally, we give the rank-one equivalence property for the CPS tensor based on the SVD of matrix, which can be applied on the rank-one approximation for CPS tensors.

keywords:

Conjugate partial-symmetric tensor, approximation algorithm, rank-one equivalence property, convex relaxation

pacs:

[

MSC Classification]15A69, 15B57, 90C26, 41A50

1 Introduction

Tensor decomposition and approximation have significant applications in computer vision, data mining, statistical estimation and so on. We refer to kolda2009tensor for the survey. Moreover, it is general that tensor from real applications with special symmetric structure. For instance, the symmetric outer product decomposition is particularly important in the process of blind identification of under-determined mixtures comon2008symmetric.

Jiang et al.jiang2016characterizing studied the functions in multivariate complex variables which always take real values. They proposed the conjugate partially symmetric (CPS) tensor to characterize such polynomial functions, which is the generalization of Hermitian matrix. Various examples of conjugate partial-symmetric tensors can be encountered in engineering applications arising from singal processing, electrical engineering, and control theoryaubry2013ambiguity; de2007fourth. Ni et al.Ni2019Hermitian and Nie et al.Nie2019Hermitian researched on the Hermitian tensor decomposition. Motivated by Lieven et al.de2007fourth, we proposed a new orthogonal matrix outer product decomposition model for CPS tensors, which explore the orthogonality of these matrices.

It is well known that unlike the matrix case, the best rank-r ( $r>1$ ) approximation of a general tensor may not exist, and even if it admits a solution, it is NP-hard to solvede2008tensor. The greedy successive rank-one approximation (SROA) algorithm can be applied to compute the rank-r ( $r>1$ ) approximation of tensor. However, the theoretical guarantee for obtaining the best rank-r approximation is less developed. Zhang et al.zhang2001rank first proved the successive algorithm exactly recovers the symmetric and orthogonal decomposition of the underlying real symmetrically and orthogonally decomposable tensors. Fu et al.2018Successive showed that SROA algorithm can exactly recover unitarily decomposable CPS tensors. We offer the theoretical guarantee of SROA algorithm for our matrix decomposition model of CPS tensor tensors.

Many multi-dimensional data from real practice are fourth-order tensors and can be formulated as low-CP-rank tensor. However it is very difficult to compute CP-rank of a tensor. Jiang et al.Jiang2018Low showed that CP-rank can be bounded by the corresponding rank of square unfolding matrix of tensors. Following the idea we research on the low rank tensor completion for fourth order partial-symmetric tensor in particular.

Recently, Jiang et al. 2015Tensor proposed convex relaxations for solving a tensor optimization problem closely related to the best rank-one approximation problem for symmetric tensors. They proved an equivalence property between a rank-one symmetric tensor and its unfolding matrix. Yang et al. Yuning2016Rank studied the rank-one equivalence property for general real tensors. Based on these rank-one equivalence properties, the above mentioned tensor optimization problem can be casted into a matrix optimization problem, which alleviates the difficulty of solving the tensor problem. In line with this idea, we study the rank-one equivalence property for the fourth-order CPS tensor and transform the best rank-one tensor approximation problem into a matrix optimization problem.

In Section 2, we give some notations and definitions. The outer product approximation model based on matrix is proposed and the successive rank-one approximation (SMROA) algorithm is given to solve it in Section 3. We show that the SMROA algorithm can exactly recover the matrix outer product decomposition or approximation of the CPS tensor in Section 4. Section LABEL:sec:application discusses applications of our model simply. In Section LABEL:sec:rank1Equ, we present the rank-one equivalence property of fourth-order CPS tensor, and based on it an application is proposed. Numerical examples are in Section LABEL:sec:numerical.

2 Preliminary

All tensors in this paper are fourth-order. For any complex number $z=a+ib\in C$ , $\bar{z}=a-ib$ denotes the conjugate of $z$ . " $\circ$ " denotes the outer product of matrices, namely $\mathcal{A}=X\circ Y$ means that

\mathcal{A}_{ijkl}=X_{ij}Y_{kl}.

$S^{n}$ denotes the set of $n$ by $n$ symmetric matrices, the entries of these matrices can be complex or only real according to the context, without causing ambiguity. The inner product between $\mathcal{A},~{}\mathcal{B}\in C^{n^{4}}$ is defined as

\left<\mathcal{A},~{}\mathcal{B}\right>=\sum\limits_{i,j,k,l=1}^{n}\mathcal{A}_{ijkl}\bar{\mathcal{B}}_{ijkl}.

Definition 1.

A fourth-order tensor $\mathcal{A}$ is called symmetric if $\mathcal{A}$ is invariant under all permutations of its indices, i.e.,

\mathcal{A}_{ijkl}=\mathcal{A}_{\pi(ijkl)},\quad i,~{}j,~{}k,~{}l=1,\cdots n.

Definition 2.

Ni2019Hermitian A fourth-order complex tensor $\mathcal{A}\in C^{n_{1}\times n_{2}\times n_{1}\times n_{2}}$ is called a Hermitian tensor if

\mathcal{A}_{i_{1}i_{2}j_{1}j_{2}}=\bar{\mathcal{A}}_{j_{1}j_{2}i_{1}i_{2}}.

Jiang et al.jiang2016characterizing introduced the concept of conjugate partial-symmetric tensors as following.

Definition 3.

A fourth-order complex tensor $\mathcal{A}\in C^{n^{4}}$ is called conjugate partial-symmetric (CPS) if

	$\displaystyle\mathcal{A}_{ijkl}$	$\displaystyle=\mathcal{A}_{\pi(ij)\pi(kl)}$
	$\displaystyle\mathcal{A}_{ijkl}$	$\displaystyle=\overline{\mathcal{A}}_{klij},\quad i,~{}j,~{}k,~{}l=1,\cdots n.$

Definition 4.

A fourth-order tensor $\mathcal{A}\in R^{n^{4}}$ is called partial-symmetric if

\mathcal{A}_{ijkl}=\mathcal{A}_{\pi(ij)\pi(kl)}=\mathcal{A}_{\pi(kl)\pi(ij)},\quad i,~{}j,~{}k,~{}l=1,\cdots n.

Example 1.

de2007fourth In the blind source separation problem, the cumulant tensor is computed as

\mathcal{C}=\sum\limits_{r=1}^{R}k_{r}a_{r}\circ\bar{a}_{r}\circ\bar{a}_{r}\circ a_{r}.

By a permutation of the indices, it is in fact a conjugate partial-symmetric tensor.

Definition 5.

The square unfolding form $M(\mathcal{A})$ of a fourth-order tensor $\mathcal{A}\in C^{n^{4}}$ is defined as

M(\mathcal{A})_{(j-1)n+i,(l-1)n+k}=\mathcal{A}_{ijkl}.

3 Matrix outer product approximation model

Jiang et al.Jiang2018Low introduced the new notions of M-decomposition for an even-order tensor $\mathcal{A}$ , which is exactly the rank-one decomposition of $M(\mathcal{A})$ , followed by the notion of tensor M-rank.

For each $\mathcal{A}\in R^{n^{4}}$ , let $M(\mathcal{A})=U\Sigma V^{T}=\sum\limits_{i=1}^{n^{2}}\sigma_{i}u_{i}\circ v_{i}$ be the SVD of $M(\mathcal{A})$ , then $\mathcal{A}$ has the following decomposition form

\mathcal{A}=\sum\limits_{i=1}^{n^{2}}\sigma_{i}U_{i}\circ V_{i},

(1)

where $(U_{i})_{st}=(u_{i})_{(t-1)n+s}$ , $(V_{i})_{st}=(v_{i})_{(t-1)n+s}$ , for $i=1,2,\cdots,n^{2}$ . $\left<U_{i},U_{j}\right>=\delta_{ij},~{}\left<V_{i},V_{j}\right>=\delta_{ij}$ , $i,j=1,2,\cdots,n^{2}$ .

We are particularly interested in the tensor with some symmetric properties. And analogous to Lieven et al.de2007fourth, we prove that the CPS tensor has a decomposition based on matrix as following.

Theorem 1.

If $\mathcal{A}\in C^{n^{4}}$ is a conjugate partial-symmetric tensor, then it can be decomposed as,

\mathcal{A}=\sum\limits_{i=1}^{r}\lambda_{i}E_{i}\circ\overline{E_{i}},

(2)

where $\lambda_{i}\in R$ , $E_{i}\in C^{n^{2}\times n^{2}}$ are symmetric matrices and $\left<E_{i},E_{j}\right>=\delta_{ij}$ , for $i,~{}j=1,2,\cdots,r$ . And the decomposition is unique when $\lambda_{i}$ are different from each other.

Proof: Since $\mathcal{A}$ is conjugate partial-symmetric, then the unfold matrix $M(\mathcal{A})$ is Hermitian, and can be decomposed as

M(\mathcal{A})=\sum_{i=1}^{r}\lambda_{i}e_{i}e_{i}^{*},

where $\lambda_{i}\in R$ , $e_{i}\in C^{n^{2}}$ , $i=1,\ldots,r$ are mutually orthogonal. Folding $e_{i}$ into matrix $E_{i}$ via $(E_{i})_{ij}=(e_{i})_{(j-1)\times n+i}$ , thus $E_{i}$ , $i=1,\ldots,p$ are mutually orthogonal, that is $\langle E_{i},E_{j}\rangle=\delta_{ij}$ . In this case, we have $\mathcal{A}=\sum\limits_{i=1}^{r}\lambda_{i}E_{i}\circ\bar{E_{i}}$ .

From the eigen-decomposition of $M(\mathcal{A})$ , we have $M(\mathcal{A})e_{\tau}=\lambda_{\tau}e_{\tau}$ , for $\tau=1,\ldots,r$ i.e., $\sum_{k,l}a_{ijkl}(e_{\tau})_{(l-1)\times n+k}=\lambda_{\tau}(e_{\tau})_{(j-1)\times n+i}$ , for any $i,j=1,\ldots,n$ . Since $a_{ijkl}=a_{jikl}$ , for all $i,j=1,\ldots,n$ , then $(e_{\tau})_{(j-1)\times n+i}=(e_{\tau})_{(i-1)\times n+j}$ , thus $E_{\tau}$ is symmetric. The uniqueness of the decomposition follows the property of eigen-decomposition of Hermitian matrix naturally.

Remark 1.

Jiang et al.jiang2016characterizing gave the decomposition theorem for CPS tensor like Theorem 3. However, they established this theorem in the view of polynomial decomposition and did not explore the mutually orthogonality of matrices in the decomposition model.

Definition 6.

$\mathcal{A}\in C^{n^{4}}$ is a CPS tensor,

rank_{M}(\mathcal{A})=min\{r\mid\mathcal{A}=\sum\limits_{i=1}^{r}\lambda_{i}E_{i}\circ\bar{E}_{i},\lambda_{i}\in R,E_{i}\in S^{n^{2}}\}.

The $rank_{M}(\mathcal{A})$ is actually the strongly symmetric M-rank $rank_{ssm}(\mathcal{A})$ defined by JiangJiang2018Low. For symmetric tensor $\mathcal{A}$ , they also proved the equivalence between $rank_{ssm}(\mathcal{A})$ and

rank_{sm}(\mathcal{A})=min\{r\mid\mathcal{A}=\sum\limits_{i=1}^{r}\lambda_{i}E_{i}\circ E_{i},\lambda_{i}\in R,E_{i}\in C^{n^{2}}\}.

This is also true for CPS fourth-order tensor.

Theorem 2.

Let $\mathcal{A}\in C^{n^{4}}$ be a CPS tensor, then $rank_{M}(\mathcal{A})=rank_{sm}(\mathcal{A}).$

Proof: It is obvious that $rank_{M}(\mathcal{A})\geq rank_{SM}(\mathcal{A}).$ On the other hand, if $rank_{SM}(\mathcal{A})=r$ , we have $rank(M(\mathcal{A}))\leq r$ . Since $rank(M(\mathcal{A}))=rank_{M}(\mathcal{A})$ , we obtain the desired conclusion.

Corollary 3.

Let $\mathcal{A}\in R^{n^{4}}$ be a partial-symmetric tensor, then one has,

\mathcal{A}=\sum\limits_{i=1}^{r}\lambda_{i}E_{i}\circ E_{i},

(3)

where $\lambda_{i}\in R$ , $E_{i}$ are symmetric matrices and $\left<E_{i},E_{j}\right>=\delta_{ij}$ , for $i,~{}j=1,2,\cdots,r$ . $rank_{M}(\mathcal{A})\leq\frac{n(n+1)}{2}$ .

Proof: The first part is obvious according to Theorem 3. Since all matrices belonging to $S^{n}$ form a $\frac{n(n+1)}{2}$ -dimensional vector space, we have $rank_{M}(\mathcal{A})\leq\frac{n(n+1)}{2}$ . Fu et al. gave a rank-one decomposition of vector form for the CPS tensor based on Theorem 1 as follows,

Theorem 4.

(fu2018decompositions, Theorem 3.2) $\mathcal{A}\in C^{n^{4}}$ is CPS if and only if it has the following partial-symmetric decomposition

\mathcal{A}=\sum\limits_{i}\lambda_{i}\bar{a}_{i}\circ\bar{a}_{i}\circ a_{i}\circ a_{i},

where $\lambda_{j}\in R$ and $a_{i}\in C^{n}$ . That is, a CPS tensor can be decomposed as the sum of rank-one CPS tensors.

However, when we restricted the decomposition on real domain, the decomposition does not seem to hold, since $\sum\limits_{i}\lambda_{i}a_{i}\circ a_{i}\circ a_{i}\circ a_{i}$ , where $\lambda_{i}\in R$ , $a_{i}\in R^{n}$ , can only represent symmetric tensor. Thus, an extended rank-one approximation model for the partial-symmetric tensor can be proposed based on Corollary 3.

Corollary 5.

Let $\mathcal{A}\in R^{n^{4}}$ be a partial-symmetric tensor, then it can be decomposed as the sum of simple low rank partial-symmetric tensor,

\mathcal{A}=\sum\limits_{i}\lambda_{i}(p_{i}\circ p_{i}\circ q_{i}\circ q_{i}+q_{i}\circ q_{i}\circ p_{i}\circ p_{i}).

(4)

Proof: From Corollary 3, partial-symmetric tensor $\mathcal{A}=\sum\limits_{i=1}^{r}\lambda_{i}E_{i}\circ E_{i}$ , where $E_{i}$ are symmetric. So it can be decomposed as $\sum_{j=1}^{r_{i}}\beta_{i}^{j}u_{i}^{j}(u_{i}^{j})^{\top}$ , thus

	$\displaystyle\mathcal{A}$	$\displaystyle=$		$\displaystyle\sum\limits_{i=1}^{r}\lambda_{i}(\sum_{j=1}^{r_{i}}\beta_{i}^{j}u_{i}^{j}(u_{i}^{j})^{\top})\circ(\sum_{k=1}^{r_{i}}\beta_{i}^{k}u_{i}^{k}(u_{i}^{k})^{\top})$
		$\displaystyle=$		$\displaystyle\sum_{i=1}^{r}\lambda_{i}(\sum_{j=1}^{r_{i}}\sum_{k=j}^{r_{i}}\beta_{i}^{j}\beta_{i}^{k}(u_{i}^{j}\circ u_{i}^{j}\circ u_{i}^{k}\circ u_{i}^{k}+u_{i}^{k}\circ u_{i}^{k}\circ u_{i}^{j}\circ u_{i}^{j}).$

The desired decomposition form follows.

Remark 2.

From the proof of Corollary 5, we can see that if $p_{i}\neq q_{i}$ , $p_{i}^{T}q_{i}=0$ . Whether this decomposition form is the compactest will be one of our future work.

We can discuss the case of skew partial-symmetric tensor in parallel.

Theorem 6.

We call $\mathcal{A}\in R^{n^{4}}$ skew partial-symmetric tensor if

\mathcal{A}_{ijkl}=\mathcal{A}_{\pi(ij)\pi(kl)}=-\mathcal{A}_{\pi(kl)\pi(ij)},\quad i,~{}j,~{}k,~{}l=1,2,\cdots,n.

Then one has

\mathcal{A}=\sum\limits_{i}\lambda_{i}(U_{i}\circ V_{i}-V_{i}\circ U_{i}),

and

\mathcal{A}=\sum\limits_{i}\lambda_{i}(p_{i}\circ p_{i}\circ q_{i}\circ q_{i}-q_{i}\circ q_{i}\circ p_{i}\circ p_{i}).

Proof: $M(\mathcal{A})$ is skew-symmetric according to the definition of the skew partial-symmetric tensor. Then $M(\mathcal{A})=\sum_{i}\lambda_{i}(u_{i}v_{i}^{T}-v_{i}u_{i}^{T})$ . The rest of the proof is similar to that for partial-symmetric tensor, here we omit it.

Based on Theorem 1, we propose a matrix outer product approximation model for the CPS tensor as following.

	$\displaystyle\underset{\lambda_{i}\in R,~{}X_{i}\in S^{n}}{min}$	$\displaystyle\\|\mathcal{A}-\sum\limits_{i=1}^{r}\lambda_{i}X_{i}\circ\bar{X_{i}}\\|^{2}_{F}$		(5)
	$\displaystyle s.t.$	$\displaystyle\left<X_{i},X_{j}\right>=\delta_{ij}.$		(5)

The successive rank-one approximation algorithm can also be applied to the conjugate partial symmetric tensors to find the matrix outer product decompositions or approximations, as shown in Algorithm 1.

Algorithm 1 Successive Matrix Outer Product Rank-One Approximation (SMROA) Algorithm

Given a CPS tensor $\mathcal{A}\in C^{n^{4}}$ . Initialize $\mathcal{A}_{0}=\mathcal{A}$ .

for

j=1

r

(\lambda_{j},X_{j})\in\underset{\|X\|_{F}=1,X\in S^{n},\lambda\in R}{argmin}\|\mathcal{A}_{j-1}-\lambda X\circ\bar{X}\|_{F}

\mathcal{A}_{j}=\mathcal{A}_{j-1}-\lambda_{j}X_{j}\circ\bar{X_{j}}

end for

Return

\{\lambda_{j},X_{j}\}_{j=1}^{r}

The main optimization problem in Algorithm 1 could be expressed as

(\lambda_{*},X_{*})\in\underset{\|X\|_{F}=1,X\in S^{n},\lambda\in R}{argmin}\|\mathcal{A}-\lambda X\circ\bar{X}\|_{F}^{2},

(6)

The objective function of (6) can be rewritten as

\begin{split}&\|\mathcal{A}-\lambda X\circ\bar{X}\|_{F}^{2}\\ =&\|\mathcal{A}\|_{F}^{2}+\lambda^{2}-2\lambda\left<\mathcal{A},X\circ\bar{X}\right>\end{split}

From which we can derive that problem (6) is equivalent to

X_{*}\in\underset{\|X\|_{F}=1,X\in S^{n}}{argmax}\mid\left<\mathcal{A},X\circ\bar{X}\right>\mid,

(7)

and $\lambda_{*}=\left<\mathcal{A},X_{*}\circ\bar{X_{*}}\right>$ . We can solve (7) by transforming it into matrix eigenproblem as follows,

x_{*}\in\underset{\|x\|=1,x\in C^{n^{2}}}{argmax}\mid x^{*}M(\mathcal{A})x\mid.

(8)

Remark 3.

Zhang et al.zhang proved that if $\mathcal{A}\in R^{n^{4}}$ is symmetric,

\underset{\begin{subarray}{c}\|x_{i}\|=1\\ i=1,2,3,4\end{subarray}}{min}\|\mathcal{A}-\lambda x_{1}\circ x_{2}\circ x_{3}\circ x_{4}\|_{F}=\underset{\|x\|=1}{min}\|\mathcal{A}-\lambda x\circ x\circ x\circ x\|_{F}\mathchar 24635\relax\;

if $\mathcal{A}$ is symmetric about the first two and the last two mode respectively,

\underset{\begin{subarray}{c}\|x_{i}\|=1\\ i=1,2,3,4\end{subarray}}{min}\|\mathcal{A}-\lambda x_{1}\circ x_{2}\circ x_{3}\circ x_{4}\|_{F}=\underset{\|x\|=\|y\|=1}{min}\|\mathcal{A}-\lambda x\circ x\circ y\circ y\|_{F}.

It is obvious that for partial-symmetric tensor, we also have

\underset{\|X_{i}\|_{F}=1,X_{i}\in R^{n^{2}},\lambda\in R}{min}\|\mathcal{A}-\lambda X_{1}\circ X_{2}\|_{F}=\underset{\|X\|_{F}=1,X\in S^{n},\lambda\in R}{min}\|\mathcal{A}-\lambda X\circ X\|_{F}.

Remark 4.

It is well-known that (6) is equivalent to the nearset Kronecker product problem golub2013matrix as below

(\lambda_{*},X_{*})\in\underset{\|X\|_{F}=1,X\in S^{n},\lambda\in R}{argmin}\|A-\lambda X\otimes\bar{X}\|_{F}^{2},

where $A_{(i-1)n+k,(j-1)n+l}=\mathcal{A}_{ijkl}$ , " $\otimes$ " denotes the kronecker product of matrices.

4 Exact Recovery for CPS tensors

In this section, we give the theoretical analysis of exact recovery for CPS tensors by the SMROA algorithm.

Theorem 7.

Let $\mathcal{A}$ be a CPS tensor with $rank_{M}(\mathcal{A})=r$ , that is

\mathcal{A}=\sum\limits_{i=1}^{r}\lambda_{i}E_{i}\circ\bar{E_{i}}.

If $\lambda_{i}$ are different from each other, then the SMROA algorithm will obtain the exact decomposition of $\mathcal{A}$ after $r$ iterations.

We first claim the following lemma before proving the above theorem.

Lemma 8.

Let $\mathcal{A}$ be a CPS tensor with $rank_{M}(\mathcal{A})=r$ , that is

\mathcal{A}=\sum\limits_{i=1}^{r}\lambda_{i}E_{i}\circ\bar{E_{i}}.

$\lambda_{i}$ are different from each other. Suppose

\hat{X}_{1}\in\underset{X\in S^{n},\|X\|_{F}=1}{argmax}\mid\left<\mathcal{A},X\circ\bar{X}\right>\mid,~{}\hat{\lambda}_{1}=\left<\mathcal{A},X\circ\bar{X}\right>.

Then, there exists $j\in\{1,2,\cdots,r\}$ such that

\hat{\lambda}_{1}=\lambda_{j},~{}\hat{X}_{1}=E_{j}.

Proof: According to Theorem 1, $E_{i}$ are mutually orthogonal, thus $\{E_{1},\cdots,E_{r}\}$ is a subset of an orthonormal basis $\{E_{1},\cdots,E_{\frac{n(n+1)}{2}}\}$ of $S^{n}$ and $0\neq\lambda_{i}\in R$ . Let $\hat{X}_{1}=\sum\limits_{i=1}x_{i}E_{i}$ , where $x_{i}=\left<\hat{X}_{1},E_{i}\right>$ for $i=1,2,\cdots,\frac{n(n+1)}{2}$ . Since $\|\hat{X}_{1}\|_{F}$ =1, we have $\sum_{i = 1} x_i^2=1 . R e o r d e r t h e i n d i c e s s u c h t h a t (9)Equation 99∣λ1∣≥∣λ2∣≥⋯≥∣λr∣. T h e n w e o b t a i n ∣⟨A,^X1∘¯^X1⟩∣=∣∑i=1rλi∣xi∣2∣∣⟨A,^X1∘¯^X1⟩∣=∣∑i=1rλi∣xi∣2∣≤∣λ1∣.≤∣λ1∣. O n t h e o t h e r h a n d, t h e o p t i m a l i t y l e a d s t o ∣⟨A,^X1∘¯^X1⟩∣≥∣⟨A,E1∘¯E1⟩∣∣⟨A,^X1∘¯^X1⟩∣≥∣⟨A,E1∘¯E1⟩∣=∣λ1∣.=∣λ1∣. H e n c e, ∣λ1∣≤∣⟨A,^X1∘¯^X1⟩∣=∣^λ1∣≤∣λ1∣. S o, ∣ \hat{λ}_1∣=∣ λ_1∣, ∣x_1∣=1 . T h e r e f o r e, \hat{X}_1=e^i θ E_1, f o r a n y θ ∈[0,2 π], a n d^λ1=⟨A,^X1∘¯^X1⟩=⟨A,E1∘¯E1⟩=λ1. T h e n l e t x_1=1, w e h a v e \hat{X}_1=E_1 . N o w, w e p r o v e T h e o r e m 7 . 𝐏𝐫𝐨𝐨𝐟 : ByLemma 8, thereexistsj \in 1, 2, \dots, rsuchthat \hat{X}_{1} = E_{j}, \hat{λ}_{1} = λ_{j} . Let A1=A-^λ1^X1∘¯^X1=∑i≠jλiEi∘¯Ei, and^X2∈X∈Sn,∥X∥F=1argmax∣⟨A1,X∘¯X⟩∣,^λ2=⟨A,^X2∘¯^X2⟩. BythesimilarproofofLemma 8, weknowthatthereexistsak \in {1, 2, \dots, n} \ {j} suchthat \hat{λ}_{2} = λ_{k}, \hat{X}_{2} = E_{k} . Repeatedly, Wecaninduceapermutation π on {1, 2, \dots, r} suchthat^λj=λπ(j),^Xj=Eπ(j),j=1,2,⋯,r. Corollary 999Corollary 9Corollary 9. 𝐿𝑒𝑡 A=∑i=1rλiEi∘¯Ei, 𝑤ℎ𝑒𝑟𝑒 {E_{1}, \dots, E_{r}} 𝑖𝑠𝑎𝑠𝑢𝑏𝑠𝑒𝑡𝑜𝑓𝑎𝑛𝑜𝑟𝑡ℎ𝑜𝑛𝑜𝑟𝑚𝑎𝑙𝑏𝑎𝑠𝑖𝑠 {E_{1}, \dots, E_{\frac{n (n + 1)}{2}}} 𝑜𝑓𝑆^{n} .0 \neq λ_{i} \in 𝑅𝑎𝑟𝑒𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡𝑓𝑟𝑜𝑚𝑒𝑎𝑐ℎ𝑜𝑡ℎ𝑒𝑟, 𝑎𝑛𝑑𝑎𝑟𝑒𝑜𝑟𝑑𝑒𝑟𝑒𝑑𝑎𝑠 ∣λ1∣≥∣λ2∣≥⋯≥∣λr∣. 𝑆𝑢𝑝𝑝𝑜𝑠𝑒 {(\hat{λ}_{i}, \hat{X}_{i})}_{i = 1}^{n} 𝑖𝑠𝑡ℎ𝑒𝑜𝑢𝑡𝑝𝑢𝑡𝑜𝑓𝑡ℎ𝑒𝑆𝑀𝑅𝑂𝐴𝑎𝑙𝑔𝑜𝑟𝑖𝑡ℎ𝑚𝑓𝑜𝑟𝑖𝑛𝑝𝑢𝑡 𝒜 . 𝑇ℎ𝑒𝑛, \hat{λ}_{i} = λ_{i} . \hat{X}_{i} = E_{i}, 𝑓𝑜𝑟𝑖 = 1, 2, \dots, r . 𝐼𝑛𝑝𝑎𝑟𝑡𝑖𝑐𝑢𝑙𝑎𝑟, 𝑖𝑓 λ_{i} > 0, 𝑤𝑒ℎ𝑎𝑣𝑒 \hat{λ}_{1} > \hat{λ}_{2} > \dots > \hat{λ}_{r}; 𝑖𝑓 λ_{i} < 0, 𝑤𝑒ℎ𝑎𝑣𝑒 \hat{λ}_{1} < \hat{λ}_{2} < \dots < \hat{λ}_{r} . T h i s p r o p o s i t i o n d i r e c t l y f o l l o w s f r o m t h e p r o o f o f L e m m a 8 . Remark 555Remark 5Remark 5$