Network construction: A learning framework through localizing principal eigenvector

Networks furnish a mathematical framework to model and decipher the collective behavior of complex real-world systems. Scrutiny of principal eigenvector (PEV) and the corresponding eigenvalue of the networks are known to provide an understanding of various local and global structural properties as well as the time evolution of dynamics on those networks [1], [2], [3], [4], [5], [6]. Further, different networks’ eigenvector-based centrality measures have been proposed to understand the importance of the nodes forming those networks. For example, eigenvector centrality or Katz centrality provides a ranking to nodes of networks based on the entries of the PEV [7], [8]. Similarly, the PageRank algorithm which is based on the PEV of Google matrices predicts the importance of the web pages [8]. Also, variants of principal component analysis and independent component analysis have led to radical developments in machine learning approaches [9]. Furthermore, conditions under which the degree vector of a network and PEV are correlated have been derived leading to the usage of degree vector instead of PEV to approximate various network analysis results [10]. Furthermore, a network construction for which the message passing equations are exact have been explained and solutions near the critical point in terms of the PEV components have been analyzed [11]. Recently, sensitivity in the network dynamics has been explored using networks’ eigenvectors [12]. Particularly, development of community detection techniques based on the localization properties of eigenvector [13],[14],[15] have been another significant contribution to the complex network analysis. Further, localization has an important role in quantum physics [16], mathematics [17], [18], [19], approximate algorithm development [20], machine learning [21], numerical linear algebra, matrix gluing, structural engineering, computational quantum chemistry [22], [23], and in quantum information theory [24]. From real-world systems, it is also evident that underlying interaction patterns form the infrastructure for their different emerging dynamical responses. For instance, information or rumor propagates through the Facebook-Twitter networks; in brain, the neurons interact to perform specific functions over the underlying network. Moreover, reconfiguration or rewiring of the functional brain networks are required during the learning phases [25]. Therefore a scrutiny of the network architecture is important as ‘underlying structure has crucial impact on its function’ and vice-versa [26].

Due to the versatile applications of the eigenvector properties, we analyze the network architecture which optimizes a specific behavior of its PEV. In this article, we study the network structure from a different point of view. Instead of analyzing the properties of a network, we construct a sequence of networks, $\{\mathcal{G}_{1},\mathcal{G}_{2},\ldots,\mathcal{G}_{final}\}$ optimizing a few specific behaviors of the PEV. The primary aim of this framework is to examine the sequence of networks and learn the network properties collectively when optimizing a function $\zeta:\Re^{n}\rightarrow\Re$ on the eigenvectors. In other words, we can represent an undirected network by an adjacency matrix which encodes the interactions or relations among $n$ objects (nodes) of a real-world complex system. We consider that the adjacency matrix is symmetric, and hence from the eigenvalue equation, we have $n$ number of eigenvectors which represent $n$ different solutions of the system. Each eigenvector has a different meaning corresponding to the underlying system. We have an interest in network architecture that satisfies few particular behaviors of the eigenvector (solution). The entry of an eigenvector for a symmetric matrix may contain negative, zero, or positive values. Can we tune the eigenvector entries and construct the network structure accordingly$?$ The question one can ask that how values of the eigenvector and the corresponding network structure are related. Reversely, to get a particular behavior for the eigenvector what will be the interaction matrix which in our case is the network’s adjacency matrix. The tuning can be performed based on a particular function $\zeta$, and for our purpose here it is the inverse participation ratio (IPR) of the PEV (Eq. (2)).

It has been demonstrated that few structural features of networks, such as the presence of a hub node, the existence of dense subgraphs, or a power-law degree distribution may lead to the localization of the PEV [32],[33]. However, the questions which still need investigation and dealt with in this article are;

• a.

  Starting with a network having a delocalized PEV, how can one gradually localize the PEV behavior and construct the corresponding network structures?

• b.

  What is the particular architecture of the optimized network corresponding to a highly localized PEV?

Here, we develop a general framework to evolve a network structure based on rewiring its edges as the PEV of the corresponding adjacency matrix goes from the delocalized to the highly localized state. We devise three different algorithms which use random edge sampling (Hub-based, Monte-Carlo-based, and simulated annealing based) to find out the final network structure. Moreover, we show that the optimized network concerning the highly localized PEV has a distinctive architecture.

The article is organized as follows: Section 2 motivates to study PEV localization. Section 3, contains the notations and definitions used in the later discussion. Also, it includes a brief explanation and formulation of our work. Section 4, illustrates various algorithms on edge rewiring-based optimization in detail. Finally, section 5, summarizes the current study and discusses the open problems for further investigations.

We know relations or interactions are everywhere either interactions of power grid generators to provide proper functioning of the power supply over a country, or interactions of bio-molecules inside cell to proper functioning of cellular activity or interactions of neurons inside brains to perform specific functions or interactions among satellites to provide accurate GPS services or interactions among quantum particle to form quantum communications or recent coronavirus spread [26], [25], [27], [28], [29]. For all of them there are two things that are common –- network structure and interactions mechanism among the components or dynamics running on top of that. For instance, for a linear dynamical model (Fig. 1)

where ${\bf M}=a{\bf I}+b{\bf A}$ is the transition matrix, $\bm{x(0)}$ is the initial state of the system, $\bm{x}^{*}$ is the stationary state and $\bm{v}_{1}^{\bf M}$ is the PEV of M. To find the pattern or understand the behavior of the linear dynamics on networks, we iterate the matrix multiplication process for a large number of times or we can find out the PEV ($\bm{v}_{1}^{\bf M}$) of M (Eq. (1)). There is another way, we can also solve the problem. We can see that eigenvectors of M are the same as eigenvectors of A [1].

where I is the identity matrix and $\lambda_{i}^{\bf M}=a+b\lambda_{i}^{\bf A}$ and. Thus, $\bm{x}^{*}\sim\bm{v}_{1}^{\bf M}=\bm{v}_{1}^{\bf A}$. Hence, understanding the behavior of the PEV of the adjacency matrix is enough to understand the information flow pattern for linear dynamical systems. Therefore, the network properties which enhance PEV localization of adjacency matrices can implicitly restrict the linear-dynamics in a smaller section of the network. A fundamental question at the core of the structural-dynamical relation is that: How can we find network structure having highly localized PEV? Here, we provide random-sampling based algorithm to construct localized network structure.

We represent a graph, $\mathcal{G}=\{V,E\}$, where $V=\{v_{1},v_{2},\ldots,v_{n}\}$ is the set of vertices and $E=\{e_{1},e_{2},\ldots,e_{m}|e_{m}=(v_{i},v_{j})\}\subseteq U$ is the set of edges. We define the universal set $U=V\times V=\{(v_{i},v_{j})|v_{i},v_{j}\in V\;\text{and}\;i\neq j\}$ which contains all possible ordered pairs of vertices excluding the self-loops and the complementary set can be defined as $E^{c}=U-E=\{(v_{i},v_{j})|(v_{i},v_{j})\in U\;\text{and}\;(v_{i},v_{j})\notin E\}$ i.e., $E\cap E^{c}=\varnothing$ and $E\cup E^{c}=U$. We denote the adjacency matrices corresponding to $\mathcal{G}$ as ${\bf A}\in\Re^{n\times n}$ and which can be defined as $(a)_{ij}=1$, if $v_{i}\sim v_{j}$ and $0$ otherwise. The degree of a node can be represented as $d(v_{i})=\sum_{j=1}^{n}a_{ij}$, and the average degree of $\mathcal{G}$ can be defined as $\langle k\rangle=\frac{1}{n}\sum_{i=1}^{n}d(v_{i})$. Here, we consider $|V|=n$, $|E|=m$, and $|E^{c}|=\frac{n(n-1)}{2}-m$. The spectrum of $\mathcal{G}$ is the set of the eigenvalues $\{\lambda_{1},\lambda_{2},\ldots,\lambda_{n}\}$ of ${\bf A}$. Without loss of generality we can order the eigenvalues of ${\bf A}$ as $\lambda_{1}>\lambda_{2}\geq\cdots\geq\lambda_{n}$ and corresponding eigenvectors as $\bm{x}_{1},\bm{x}_{2},\cdots,\bm{x}_{n}$ respectively. Here, ${\bf A}$ is a real symmetric matrix, and each has real eigenvalues. In addition, the networks are connected. Hence, we know from the Perron-Frobenius theorem [30] that all the entries in the PEV ($\bm{x}_{1}$) of ${\bf A}$ are positive. We quantify localization of the PEV through the inverse participation ratio, which is the sum of fourth power of the eigenvector entries and calculate [33], [3] as follows:

where $(x_{1})_{i}$ is the $ith$ component of $\bm{x}_{1}$ and $||\bm{x}_{1}||_{2}^{2}=1$. A delocalized eigenvector with component $(\frac{1}{\sqrt{n}},\frac{1}{\sqrt{n}},\ldots,\frac{1}{\sqrt{n}})$ has $Y_{\bm{x}_{1}}=\frac{1}{n}$, whereas the most localized eigenvector with components $(1,0,\ldots,0)$ yields an IPR value equal to $Y_{\bm{x}_{1}}=1$. A network is said to be regular if each node has the same degree [30]. It also turns out that for any regular graph (Theorem 6 [30]), we get PEV, $\bm{x}_{1}=(\frac{1}{\sqrt{n}},\frac{1}{\sqrt{n}},\ldots,\frac{1}{\sqrt{n}})$. Hence, $Y_{\bm{x}_{1}}=\frac{1}{n}$, corresponds to the most delocalized PEV. Therefore, for any regular network IPR value of the PEV provides the lower bound. Hence, a sparse as well as a dense regular network contains delocalized PEV. Now, we can consider a disconnected graphs where each node is isolated from each other and each node has a self-loop. The adjacency matrix can be represented with the $n\times n$ identity matrix. For this disconnected networks, $Y_{\bm{x}_{1}}=1$. In another situation if we consider only $n$ number of isolated nodes with $|E|=0$. We have a zero matrix and for which we can choose $\bm{x}_{1}=(1,0,\ldots,0)$ and $Y_{\bm{x}_{1}}=1$. These are the special cases. Additionally, for any disconnected network with less than $n$ number of components, the PEV entries might be zeros. Hence, for a connected network, IPR value lies between $1/n\leq Y_{\bm{x}_{1}}<1$. Therefore, it is evident that finding out a network architecture for a given $n$ with delocalized PEV is easier than searching for a connected network structure with highly localized PEV. For a given $n$ and $m$, our aim is to get a connected network which has the most localized PEV. In other words, we can state the problem as, we search for a binary symmetric matrix (${\bf A}_{final}$) which is irreducible and which has the PEV with maximum IPR value. Also we have an interest to know the properties of sequence of the adjacency matrices $\{{\bf A}_{1},{\bf A}_{2},\ldots,{\bf A}_{final}\}$ during the searching process which can maximize the IPR value. The search space for a given $n$ and $m$ is of the order $O(n^{2m})$ [31]. Therefore, we use random-sampling to formulate this problem.

We use randomized algorithms based on edge rewiring to construct a network architecture that corresponds to a highly localized PEV in an iterative manner. It is very natural that modification in the entries of an adjacency matrix leads to a change in the spectral properties (eigenvalues and eigenvectors) which also simultaneously change the network architecture. We use this fact to develop randomized algorithms. The modification in the adjacency matrix can be performed by removing or adding edges as well as nodes or rearrangements of the edges in $\mathcal{G}$ [3]. Here, we devise algorithms by rearrangement of the edges to get highly localized PEV when $\mathcal{G}$ remains connected, and the number of nodes and edges remain fixed. In the following, we discuss the algorithms in detail.

We explore different network construction algorithms which optimizing the behavior of PEV. We construct the network structure through the optimization process that possesses highly localized PEV quantified by the IPR value. This approach provides a comprehensive way to investigate not only the optimized network but also intermediate networks before an optimized structure is found. Furthermore, we restrict our study to the adjacency matrix of the network. It is also interesting to examine for other matrices related to graphs such as Laplacian matrix, modularity matrix. Here, eigenvector behavior has been regulated based on the particular function which is IPR. It is good to define another function that can tune the negative or zeros entries of the eigenvectors or some part of the eigenvector and based on that one can construct a network structure. This framework also helps to construct a network for other lower-order eigenvectors. Low order eigenvectors have also been studied to develop machine learning tools. They used IPR as well as another kind of measure for the eigenvector localization, called statistical leverage scores [34], [35] which has an impact on statistics and modern big data analysis.

It is also interesting to distribute weights in a weighted network so that PEV becomes localized [33]. Moreover, removing edges make the PEV delocalized, and also adding edges to the network produces delocalized PEV, but it may be possible to add a proper number of edges to the network which forms the highly localized PEV. We have not included the complexity analysis of the algorithms, and it is an exciting part to do in the future. Finally, we devise edge rewiring-based optimization algorithms which allow us to learn about the network structure from the PEV and it may be relevant to be used later to develop machine learning tools.

Next, we discuss in details about the objective function and constraints.