Optimizing Gershgorin for Symmetric Matrices

One of the best known bounds for the eigenvalues of a matrix is the classical Gershgorin Circle Theorem [1, 2], which allows us to determine an inclusion domain of the eigenvalues of a matrix that can be determined solely from the entries of this matrix. Specifically, given an $n\times n$ matrix $A$ whose entries are denoted by $a_{ij}$, we define

Then the spectrum of $A$ is contained inside the union of the $D_{i}$.

If $A$ is a symmetric matrix, then the spectrum is real. In general, we might like to determine whether $A$ is positive definite, or even to determine a bound on the spectral gap of $A$, i.e. a number $\alpha>0$ such that all eigenvalues are greater than $\alpha$. Let us refer to the diagonal dominance of the $i$th row of the matrix by the quantity $a_{ii}-R_{i}(A)$. A restatement of Gershgorin is that the smallest eigenvalue is at least as large as the smallest diagonal dominance of any row.

As an example, let us consider the matrix

and we see the quantity $a_{ii}-R_{i}(A)=-4$ for all rows. From this and Gershgorin we know that all eigenvalues are at least $-4$, but we cannot assert that $A$ is positive definite. However, we can compute directly that ${\mathrm{Spec}}(A)=\{1,1,16\}$; not only is $A$ positive definite, it has a spectral gap of $1$.

We then note that $A=I+5{\bf 1}$ (here and below we use ${\bf 1}$ to denote the matrix with all entries equal to $1$). Since $I$ is positive definite and ${\bf 1}$ is positive semidefinite, then it follows that $A$ is positive definite as well. In fact, we know that the eigenvalues of $A$ are at least as large as those of $I$, so that $A$ has a spectral gap of at least $1$, which gives us the exact spectral gap.

Let us consider the family of matrices $A-x{\bf 1}$ with $x\geq 0$, and consider how the diagonal dominance changes. Subtracting off the value of $x$ from each entry moves the diagonal to the left by $x$, but as long as the off-diagonal terms are positive, it decreases the off-diagonal sum by a factor of $2x$. This has the effect of moving the center of each Gershgorin circle to the left by $x$, but decrease the radius by $2x$, effectively moving the left-hand edge of each circle to the right by $x$. This gives an improved estimate for the lowest eigenvalue. It is not hard to see in this case that choosing $x=5$ gives the optimal diagonal dominance of 1. Generalizing this idea: let $A$ be a symmetric $n\times n$ matrix with strictly positive entries. Since ${\bf 1}$ is positive semidefinite, whenever $x\geq 0$, $A=B+x{\bf 1}$ is positive definite whenever $B$ is. However, subtracting $x$ from each entry, at least until some of the entries become negative, will only increase the diagonal dominance of each row: each center of each Gershgorin circle moves to the left by $x$, but the radius decreases by $(N-1)x$. From this it follows that we can always improve the Gershgorin bound for such a matrix with positive entries by choosing $x$ to be the smallest off-diagonal term.

However, now consider the matrix

We see directly that the bottom row has the smallest diagonal dominance of $-2$. Subtracting off a 1 from each entry gives

and now the bottom row has a diagonal dominance of $-1$. From here we are not able to deduce that $A$ is positive definite. However, as it turns out, for all $x\in(2,10/3)$, the matrix $A-x{\bf 1}$ has a positive diagonal dominance for each row, and by the techniques described in this paper we can compute that the optimal choice is $x=3$:

and each row has a diagonal dominance of $1$. Since $3\cdot{\bf 1}$ is positive semi-definite, this guarantees that $A$ is positive definite and, in fact, has a spectral gap of at least $1$. In fact, we can compute directly that the eigenvalues of $A$ are $(11.4704,5.39938,1.13026)$, so this it not too bad of a bound for this case.

At first it might be counter-intuitive that we should subtract off entries and cause some off-diagonal entries to become negative, since the $R_{i}(A)$ is a function of the absolute value of the off-diagonal terms and making these entries more negative can actually make the bound from the $i$th row worse. However, there is a coupling between the rows of the matrix, since we are looking for the minimal diagonal dominance across all of the rows: it might help us to make some rows worse, as long as we make the worst row better. In this example, increasing $x$ past $1$ does worsen the diagonal dominance of the first row, but it continues to increase the dominance of the third row, and thus the minimum is increasing as we increase $x$. Only until the rows pass each other should we stop, and this happens at $x=3$. From this argument it is also clear that the optimal value of $x$ must occur when two rows have an equal diagonal dominance.

The topic of this paper is to study the possibility of using the idea of shifting the entries of the matrix uniformly to obtaining better eigenvalue bounds. Note that this technique is potentially applicable to any eigenvalue bounding method that is a function of the entries of the matrix, and we consider examples other than Gershgorin as well. We show that using Gershgorin, which is piecewise linear in the coefficients of the matrix, the technique boils down to the simultaneous optimization of families of piecewise linear functions, so can always be solved, e.g. by conversion to a linear program. We will also consider “nonlinear” versions of Gershgorin theorems and shift these as well, but we see that for these cases we can lose convexity and in general this problems are difficult to solve.

We show below that for many classes of matrices, the “shifted Gershgorin” gives significantly better bounds than many of the unshifted nonlinear methods that are currently known.

Throughout the remainder of this paper, all matrices are symmetric.

Here we review several known methods. There are many methods known that we do not mention here; a very comprehensive and engaging review of many methods and their histories is contained in [2].

1. 1.

   Gershgorin’s Circles. [1] As mentioned in the introduction, define

   $$R_{i}(A)=\sum_{j\neq i}\left|{a_{ij}}\right|,\quad D_{i}=\{z\in\mathbb{C}:\left|{z-a_{ii}}\right|<R_{i}(A)\}.$$

   Then ${\mathrm{Spec}}(A)\subseteq\cup_{i}D_{i}$. For this method, we obtain the bounds

   $$\lambda_{{\mathsf{{{{G}}}}}}({A})=\min_{i}(a_{ii}-R_{i}(A)),\quad\rho_{{\mathsf{{{{G}}}}}}({A})=\max_{i}(a_{ii}+R_{i}(A)).$$

   (3)

2. 2.

   Brauer’s Ovals of Cassini. [3], [4, eqn. (21)] Using the definitions of $R_{i}(A)$ as above, let us define

   $$B_{ij}=\{z\in\mathbb{C}:\left|{z-a_{ii}}\right|\left|{z-a_{jj}}\right|\leq R_{i}(A)R_{j}(A)\}.$$

   Then ${\mathrm{Spec}}(A)\subseteq\cup_{i\neq j}B_{i,j}$. It can be shown [2, Theorem 2.3] that this method always gives a nonworse bound than Gershgorin (note for example that choosing $i=j$ gives a Gershgorin disk). The bounds come from the left- or right-hand edges of this domain, so that $z-a_{ii},z-a_{jj}$ have the same sign. This gives

   $$a_{ii}a_{jj}-R_{i}(A)R_{j}(A)-(a_{ii}+a_{jj})z+z^{2}\leq 0.$$

   The roots of this quadratic are

   $$\frac{a_{ii}+a_{jj}}{2}\pm\sqrt{(a_{ii}-a_{jj})^{2}+R_{i}(A)R_{j}(A)}.$$

   Therefore we have

   	$\displaystyle\lambda_{{\mathsf{{{{B}}}}}}({A})$	$\displaystyle=\min_{i\neq j}\left(\frac{a_{ii}+a_{jj}}{2}-\sqrt{(a_{ii}-a_{jj})^{2}+R_{i}(A)R_{j}(A)}\right),$		(4)
   	$\displaystyle\rho_{{\mathsf{{{{B}}}}}}({A})$	$\displaystyle=\max_{i\neq j}\left(\frac{a_{ii}+a_{jj}}{2}+\sqrt{(a_{ii}-a_{jj})^{2}+R_{i}(A)R_{j}(A)}\right),$		(5)

3. 3.

   Melman method. According to Theorem 2 of [5], if we define

   $$\Omega_{ij}(A)=\{z\in\mathbb{C}:\left|{(z-a_{ii})(z-a_{jj})-a_{ij}a_{ji}}\right|\leq\left|{z-a_{jj}}\right|\sum_{k\neq i,j}\left|{a_{ik}}\right|+\left|{a_{ij}}\right|\sum_{k\neq i,j}\left|{a_{jk}}\right|\},$$

   then

   $${\mathrm{Spec}}(A)\subseteq\bigcup_{i=1}^{n}\bigcap_{j\neq i}\Omega_{ij}(A).$$

   From this we can obtain bounds $\lambda^{\mathsf{S}}_{{\mathsf{{{{M}}}}}}({A}),\rho^{\mathsf{S}}_{{\mathsf{{{{M}}}}}}({A})$ similar to those in (4, 5).

4. 4.

   Cvetkovic–Kostic–Varga method. Let $S$ be a nonempty set of $[n]$, and $\overline{S}=[n]\setminus S$, and let

   $$R_{i}^{S}(A)=\sum_{j\in S,j\neq i}\left|{a_{ij}}\right|,\quad R_{i}^{\overline{S}}(A)=\sum_{j\not\in S,j\neq i}\left|{a_{ij}}\right|.$$

   Further define

   	$\displaystyle\Gamma_{i}^{S}(A)$	$\displaystyle=\left\{z\in\mathbb{C}:\left|{z-a_{ii}}\right|\leq R_{i}^{S}(A)\right\},$
   	$\displaystyle V_{ij}^{S}(A)$	$\displaystyle=\left\{z\in C:(\left|{z-a_{ii}}\right|-R_{i}^{S}(A))(\left|{z-a_{jj}}\right|-R_{j}^{\overline{S}}(A))\leq R_{i}^{\overline{S}}(A)R_{j}^{S}(A)\right\},$

   According to Theorem 6 of [6],

   $${\mathrm{Spec}}(A)\subseteq\left(\bigcup_{i\in S}\Gamma_{i}^{S}\right)\cup\left(\bigcup_{i\in S,j\in\overline{S}}V_{ij}^{S}(A)\right).$$

It is clear from the definition that $\lambda^{\mathsf{S}}_{{\mathsf{{{{Q}}}}}}({A})\geq\lambda_{{\mathsf{{{{Q}}}}}}({A})$ and $\rho^{\mathsf{S}}_{{\mathsf{{{{Q}}}}}}({A})\leq\rho_{{\mathsf{{{{Q}}}}}}({A})$ for any method ${\mathsf{{{Q}}}}$ and any matrix $A$. In short, shifting cannot give a worse estimate, and by Lemma 2.4 this improved estimate is still a good bound. Moreover, as long as

then $\lambda^{\mathsf{S}}_{{\mathsf{{{{Q}}}}}}({A})>\lambda_{{\mathsf{{{{Q}}}}}}({A})$, and similarly for the upper bounds. Moreover, we show in many contexts that the bounding functions are convex in $x$, so the converse is also true in many cases.

In this section, we concentrate on the shifted Gershgorin method. Recall that the standard Gershgorin bounds are given by

Let us define

then

First we prove the “global bounds” part of the theorem. From Lemma 3.4,

and we are done. The proof is the same for $\rho^{\mathsf{S}}_{{\mathsf{{{{G}}}}}}({A})$.

The argument for convexity is similar. We have

As a minimizer of linear functions, it follows that $\lambda_{{\mathsf{{{{G}}}}}}({A-x{\bf 1}})$ is concave down, and similarly, as $\rho_{{\mathsf{{{{G}}}}}}({A-x{\bf 1}})$ is the maximizer of linear functions, it is concave up.

Finally, we consider the “local improvement” statement. Choose and fix a row $i$, and define $y$ as in Definition 3.3. If all of the entries of $y_{k}$ are positive, then for $x<y_{1}$, $d_{i}(A,x)=(n-2)x+\sum_{k}y_{k}$, and thus has a positive slope at $x=0$. Otherwise, let us write $y_{k-1}\leq 0\leq y_{k}$. If $n-2k>0$, then $d_{i}(A,x)$ has a positive slope at $x=0$. Thus, as long as $y_{\lceil{n/2}\rceil}>0$, then $d_{i}(A,x)$ has a positive slope at $0$. In short, $d_{i}(A,x)$ has a positive slope at zero iff at least $n-1-\lceil{n/2}\rceil=\lfloor{n/2-1}\rfloor$ of the entries of $y$ are positive. Since $\lambda_{{\mathsf{{{{G}}}}}}({A-x{\bf 1}})$ is the minimum of the $d_{i}$, as long as this is true for every row which minimizes the quantity in a neighborhood of zero, then $\lambda_{{\mathsf{{{{G}}}}}}({A-x{\bf 1}})$ is itself increasing at $x=0$.

∎

In this section we consider the shifted Brauer estimator. The Brauer estimate is nonlinear and, moreover, we show that the convexity properties of shifted Gershgorin no longer apply. We show that this adds considerable technical difficulty to the problem, except in certain cases. Although we do not study the shift of some of the other nonlinear estimators from Section 2, we see from the difficulties that arise for Brauer that it is unlikely for us to obtain a tractable optimization problem without further assumptions.

We recall from (4) that

and thus

Noting that

and writing

this simplifies to

This last expression is deceptively complicated, because in fact the functions $f_{ij}(A,x)$ are not convex, or even very simple.

We could, in theory, write out the expression (11) as a single minimization over a large number of functions. Consider the expression for $f_{ij}(A,x)$: the expression under the radical is piecewise quadratic on any interval between distinct off-diagonal terms of the two rows, and since $\left|{x}\right|\geq x,-x$, this means that $f_{ij}(A,x)$ could be written as the minimum of such terms. However, this approach will not be as fruitful as it was in the previous section, since these functions can be both concave up and concave down. Moreover, since the bounding functions are nonlinear with a square root, it is possible that they have a domain strictly smaller than $x\geq 0$. In any case, extrema do not have to occur at the places where the domain of definition shifts.

However, there is one case where an approximation gives a tractable minimization problem, which we state below.

In Section 5.1 we use the main results of this paper to give a bound on the lowest eigenvalue of the adjacency matrix of an undirected graph, which is sharp for some circulant graphs. In Section 5.2 we perform some numerics to compare the shifted Gershgorin method to other methods on a class of random matrices with nonnegative coefficients. Finally in Sections 5.3 and 5.4 we consider the efficacy of shifting as a function of many of the entries of the matrix.