Tests for principal eigenvalues and eigenvectors

Factor models have been widely adopted in many disciplines, most notably, economics and finance. Some of the most famous examples include the capital asset pricing model (CAPM, Sharpe (1964)), arbitrage pricing theory (Ross (1976)), approximate factor model (Chamberlain and Rothschild (1983)), Fama-French three factor model (Fama and French (1992)) and the more recent five-factor model (Fama and French (2015)).

Statistically, the analysis of factor models is closely related to principal component analysis (PCA). For example, finding the number of factors is equivalent to determining the number of principal eigenvalues (Bai and Ng (2002); Onatski (2010); Ahn and Horenstein (2013)); estimating factor loadings as well as factors relies on principal eigenvectors (Stock and Watson (1998, 2002); Bai (2003); Bai and Ng (2006); Fan et al. (2011, 2013); Wang and Fan (2017)).

A factor model typically reads as follows:

$$y_{it}={\bf b}_{i}^{\mathrm{T}}{\bf f}_{t}+\varepsilon_{it},~{}~{}~{}~{}i=1,2,\ldots,N,~{}t=1,2,\ldots,T,$$

(1)

where $y_{it}$ is the observation from the $i$th subject at time $t$, ${\bf f}_{t}$ is a set of factors, and $\varepsilon_{it}$ is the idiosyncratic component. The number of factors, $r=\dim(f_{t})$, is small compared with the dimension $N$, and is assumed to be fixed throughout the paper. The factor model (1) can be put in a matrix form as

$${\bf y}_{t}={\bf B}{\bf f}_{t}+{\boldsymbol{\varepsilon}}_{t},t=1,2,\ldots,T,$$

where ${\bf y}_{t}=(y_{1t},\ldots,y_{Nt})^{\mathrm{T}}$, ${\bf B}=({\bf b}_{1},\ldots,{\bf b}_{N})^{\mathrm{T}}$ and ${\boldsymbol{\varepsilon}}_{t}=(\varepsilon_{1t},\ldots,\varepsilon_{Nt})^{\mathrm{T}}$. If follows that the covariance matrix ${\boldsymbol{\Sigma}}$ of ${\bf y}_{t}$ satisfies

$${\boldsymbol{\Sigma}}={\bf B}\operatorname{Cov}({\bf f}_{t}){\bf B}^{\mathrm{T}}+{\boldsymbol{\Sigma}}_{\varepsilon},$$

where ${\boldsymbol{\Sigma}}_{\varepsilon}$ is the covariance matrix of $({\boldsymbol{\varepsilon}}_{t})$.

The factors $({\bf f}_{t})$ in some situations are taken to be observable. Examples include the market factor in CAPM and the Fama-French three factors. In some other situations, factors are latent and hence unobservable. In this paper, we focus on the latent factor case.

Factor models provide a parsimonious way to describe the dynamics of large dimensional variables. In the study of factor models, time invariance of factor loadings is a standard assumption. For example, in order to apply PCA, the loadings need to be time invariant or at least roughly so, otherwise the estimation will be inconsistent. However, parameter instability has been a pervasive phenomenon in time series data. Such instability could be due to policy regime switches, changes in economic/finanncial fundamentals, etc. Because of this reason, caution has to be exercised about potential structural changes in real data. Statistical analysis of structural change in large factor model is challenging because the factors are unobserved and factor loadings have to be estimated.

There are some existing work on detecting structural breaks. Typically, the setup is as follows: suppose there are two time periods, one from time 1 to $T_{1}$, the second from $T_{1}+1$ to $T_{1}+T_{2}$, where $T_{1}$ and $T_{2}$ do not necessarily equal. The first period has loading ${\bf B}_{1}$, and the second period has loading ${\bf B}_{2}$. One then tests whether ${\bf B}_{1}$ equals ${\bf B}_{2}$. Specifically, one considers the following model:

	$\displaystyle{\bf y}_{t}$	$\displaystyle=$	$\displaystyle{\bf B}_{1}{\bf F}_{t}+{\boldsymbol{\varepsilon}}_{t},~{}~{}~{}t=1,2,\ldots,T_{1},$
	$\displaystyle{\bf y}_{t}$	$\displaystyle=$	$\displaystyle{\bf B}_{2}{\bf F}_{t}+{\boldsymbol{\varepsilon}}_{t},~{}~{}~{}t=T_{1}+1,\ldots,T_{1}+T_{2},$

and tests the following hypothesis for detecting structural breaks

$$H_{0}:~{}{\bf B}_{1}={\bf B}_{2}~{}~{}~{}~{}\textrm{vs.}~{}~{}~{}~{}H_{a}:~{}{\bf B}_{1}\neq{\bf B}_{2}.$$

Existing works include Stock and Watson (2009); Breitung and Eickmeier (2011); Chen et al. (2020); Han and Inoue (2015), among others.

Let us connect the factor loadings with principal eigenvalues and eigenvectors. Recall that ${\boldsymbol{\Sigma}}$ stands for the covariance matrix of $({\bf y}_{t})$. Write its spectral decomposition as

$${\boldsymbol{\Sigma}}={\bf V}{\boldsymbol{\Lambda}}{{\bf V}}^{T},$$

where

$${\bf V}=({\bf v}_{1},\ldots,{\bf v}_{N}),\mbox{ and }{\boldsymbol{\Lambda}}=\operatorname{diag}(\lambda_{1},\ldots,\lambda_{N}).$$

The diagonal matrix ${\boldsymbol{\Lambda}}$ consists of eigenvalues in descending order, and ${\bf V}$ consists of corresponding eigenvectors. Under the convention that $\operatorname{Cov}({\bf f}_{t})={\bf I}$, the factor loading matrix

$${\bf B}=\left(\sqrt{\lambda_{1}}{\bf v}_{1},\ldots,\sqrt{\lambda_{r}}{\bf v}_{r}\right).$$

Therefore structural breaks can be due to changes in

1. (i)

   one or more $\lambda_{i}$, or

2. (ii)

   one or more ${\bf v}_{i}$, or

3. (iii)

   both.

The economic and/or financial implications of these possibilities are, however, completely different. If a structural break is only due to change in eigenvalues, then in many applications, the structural break has no essential impact. For example, from dimension reduction point of view, if the principal eigenvalues change while the principal eigenvectors do not change, then projecting onto the principal eigenvectors is still valid. In contrast, if a structural break is caused by eigenvectors, then it may indicate a much more fundamental change, possibly associated with important economic or market condition changes, to which one should be alerted.

Such observations bring up the aim of this paper: instead of testing whether the whole matrix ${\bf B}$ is the same during two periods, we want to detect changes in individual principal eigenvalues and eigenvectors. By doing so, we can pinpoint the source of structural changes. Specifically, when a structural break occurs, we can determine whether it is caused by a change in a principal eigenvalue, a change in a principal eigenvector, or perhaps changes in both principal eigenvalues and eigenvectors.

To be more specific, we consider the the following three tests. Let ${\boldsymbol{\Sigma}}^{(1)}$ and ${\boldsymbol{\Sigma}}^{(2)}$ be the population covariance matrices for the two periods under study. For any symmetric matrix $A$ and any integer $k$, we let $\lambda_{k}(A)$ denote the $k$th largest eigenvalue of $A$, ${\bf v}_{k}(A)$ the corresponding eigenvector, and $\operatorname{tr}({\bf A})$ its trace.

1. (i)

   Test equality of principal eigenvalues: for each $k=1,\ldots,r$, we test

   $$H_{0}^{(I,k)}:~{}\lambda_{k}^{(1)}=\lambda_{k}^{(2)}~{}~{}~{}~{}~{}\textrm{vs}~{}~{}~{}~{}H_{a}^{(I,k)}:~{}\lambda_{k}^{(1)}\neq\lambda_{k}^{(2)},$$

   where $\lambda_{k}^{(1)}:=\lambda_{k}({\boldsymbol{\Sigma}}^{(i)}),\ i=1,2$.

2. (ii)

   Considering that the total variation may vary, we test about equality of the ratio of principal eigenvalues: for each $k=1,\ldots,r$, test

   $$H_{0}^{(II,k)}:~{}\dfrac{\lambda_{k}^{(1)}}{\operatorname{tr}({\boldsymbol{\Sigma}}^{(1)})}=\dfrac{\lambda_{k}^{(2)}}{\operatorname{tr}({\boldsymbol{\Sigma}}^{(2)})}~{}~{}~{}~{}~{}\textrm{vs}~{}~{}~{}~{}H_{a}^{(II,k)}:~{}\dfrac{\lambda_{k}^{(1)}}{\operatorname{tr}({\boldsymbol{\Sigma}}^{(1)})}\neq\dfrac{\lambda_{k}^{(2)}}{\operatorname{tr}({\boldsymbol{\Sigma}}^{(2)})}.$$

3. (iii)

   Most importantly, we test equality of principal eigenvectors: for each $k=1,2,\ldots,r$, test

   $$H_{0}^{(III,k)}:~{}|\langle{\bf v}_{k}^{(1)},{\bf v}_{k}^{(2)}\rangle|=1,~{}~{}~{}~{}~{}\textrm{vs}~{}~{}~{}~{}H_{a}^{(III,k)}:~{}|\langle{\bf v}_{k}^{(1)},{\bf v}_{k}^{(2)}\rangle|<1,$$

where ${\bf v}_{k}^{(i)}:={\bf v}_{k}({\boldsymbol{\Sigma}}^{(i)}),\ i=1,2$, and $\langle{\bf a},{\bf b}\rangle$ denotes the inner product of two vectors ${\bf a}$ and ${\bf b}$.

In this paper, we establish central limit theorems (CLT) for principal eigenvalues, eigenvalue ratios, as well as eigenvectors. We then develop two-sample tests based on these CLTs.

Due to the wide application of PCA, a lot of work has been devoted to investigating principal eigenvalues. However, the study of principal eigenvectors is very limited. This paper represents a significant advancement in this direction.

We remark that there is an independent work, Bao et al. (2022), that study similar questions. Nevertheless, there are several significant differences between Bao et al. (2022) and our paper. First, the non-principal eigenvalues are assumed to be equal in Bao et al. (2022); see equation (1.2) therein. This is an unrealistic assumption in many applications. In our paper, we allow the non-principal eigenvalues to follow an arbitrary distribution, rendering our results readily applicable in practice. Second, in Bao et al. (2022), the dimension to the sample size ratio needs to be away from one; see Assumption 2.4 therein. We do not impose such a restriction in our paper. Third, Bao et al. (2022) only consider the one-sample situation and study the projection of sample leading eigenvectors onto a given direction. In our paper, we establish two-sample CLT, where the projection of a principal eigenvector onto a random direction is considered. Establishing such a result presents a significant challenge. In summary, the setting of our paper is practically appropriate, and the results are of significant importance.

The organization of the paper is as follows. Theoretical results are presented in Sections 2-4. Simulation and Empirical studies are presented in Section 5 and 6, respectively. Proofs are collected in the Appendix.

Notation: we use the following notation in addition to what have been introduced above. For a symmetric $N\times N$ matrix ${\bf A}$, its empirical spectral distribution (ESD) is defined as

$$F^{{\bf A}}(x)=\dfrac{1}{N}\ \sum_{j=1}^{N}\ {\mathbf{1}}(\lambda_{j}({\bf A})\leq x),~{}~{}~{}~{}~{}x\in{\mathbb{R}},$$

where ${\mathbf{1}}(\cdot)$ is the indicator function. The limit of ESD as $N\rightarrow\infty$, if it exists, is referred to as the limiting spectral distribution, or LSD for short. For any vector ${\bf a}$, let ${{\bf a}}[k]$ be its $k$th entry. We use $``\stackrel{{\scriptstyle\mathcal{D}}}{{\rightarrow}}"$ to denote weak convergence.

We assume that $({\bf y}_{t})_{t=1}^{T}$ is a sequence of i.i.d. $N-$dimensional random vectors with mean zero and covariance matrix ${\boldsymbol{\Sigma}}$. Let $\lambda_{1},\ldots,\lambda_{N}$ be the eigenvalues of ${\boldsymbol{\Sigma}}$ in descending order, and ${\bf v}_{1},\ldots,{\bf v}_{N}$ be the corresponding eigenvectors. Write ${\boldsymbol{\Lambda}}=\operatorname{diag}(\lambda_{1},\ldots,\lambda_{N})$ and ${\bf V}=({\bf v}_{1},\ldots,{\bf v}_{N})$. Then the spectral decomposition of ${\boldsymbol{\Sigma}}$ is given by ${\boldsymbol{\Sigma}}={\bf V}{\boldsymbol{\Lambda}}{\bf V}^{\mathrm{T}}$.

We make the following assumptions.

Let $\widehat{{\boldsymbol{\Sigma}}}_{N}$ be the sample covariance matrix defined as

$$\widehat{{\boldsymbol{\Sigma}}}_{N}=\dfrac{1}{T}\sum_{t=1}^{\mathrm{T}}{\bf y}_{t}{\bf y}_{t}^{\mathrm{T}}.$$

Denote its eigenvalues by $\widehat{\lambda}_{1}\geq\cdots\geq\widehat{\lambda}_{N}$, and let $\widehat{{\bf v}}_{1},\ldots,\widehat{{\bf v}}_{N}$ be the corresponding eigenvectors.

The proofs of Theorems 1, 2 and 3 are given in the supplementary material.

We now discuss how to conduct the three tests mentioned in the Introduction.

Suppose that we have two groups of observations of the same dimension $N$:

$${\bf y}_{1}^{(1)},\ldots,{\bf y}_{T_{1}}^{(1)},~{}~{}~{}~{}\textrm{and}~{}~{}~{}{\bf y}_{1}^{(2)},\ldots,{\bf y}_{T_{2}}^{(2)},$$

which are drawn independently from two populations with mean zero and covariance matrices ${\boldsymbol{\Sigma}}^{(1)}$ and ${\boldsymbol{\Sigma}}^{(2)}$. We assume that Assumption B holds for each group of observations. Moreover, analogous to Assumption C, we have

$$\lim_{N\rightarrow\infty}\frac{N}{T_{i}}=\rho_{i}\in(0,+\infty),\quad i=1,2.$$

Finally, analogous to Assumption A, with the spectral decompositions of ${\boldsymbol{\Sigma}}^{(i)},i=1,2$, given by

$\displaystyle{\boldsymbol{\Sigma}}^{(i)}$

$\displaystyle=$

$\displaystyle{{\bf V}}^{(i)}{\boldsymbol{\Lambda}}^{(i)}{{{\bf V}}^{(i)}}^{\mathrm{T}},$

where ${\boldsymbol{\Lambda}}^{(i)}=\operatorname{diag}(\lambda_{1}^{(i)},\ldots,\lambda_{r}^{(i)},\lambda_{r+1}^{(i)},\ldots,\lambda_{N}^{(i)}),$ and ${{\bf V}}^{(i)}=({{\bf v}}_{1}^{(i)},\ldots,{{\bf v}}_{r}^{(i)},{{\bf v}}_{r+1}^{(i)},\ldots{{\bf v}}_{N}^{(i)}),$ we assume, for each population, there are $r$ principal eigenvalues, which satisfy

$$\lim_{N\rightarrow\infty}\dfrac{\lambda_{k}^{(i)}}{N}\ =\ \theta_{k}^{(i)}\in(0,+\infty),~{}~{}~{}~{}~{}~{}~{}\textrm{for }~{}1\leq k\leq r,~{}i=1,2;$$

while the remaining eigenvalues $\lambda_{j}^{(i)},r+1\leq j\leq N$, are uniformly bounded and have a limiting empirical distribution. The two liming empirical distributions for the two populations can be different.

Naturally, our tests will be based on the sample covariance matrices

$$\widehat{{\boldsymbol{\Sigma}}}_{N}^{(i)}=\frac{1}{T_{i}}\sum_{j=1}^{T_{i}}{\bf y}_{j}^{(i)}{{\bf y}^{(i)}_{j}}^{\mathrm{T}},\quad i=1,2.$$

Write their spectral decompositions as

$\displaystyle\widehat{{\boldsymbol{\Sigma}}}_{N}^{(i)}=\widehat{{\bf V}}^{(i)}\widehat{{\boldsymbol{\Lambda}}}^{(i)}{(\widehat{{\bf V}}^{(i)})}^{\mathrm{T}}$

where

$$\widehat{{\boldsymbol{\Lambda}}}^{(i)}=\operatorname{diag}(\widehat{\lambda}_{1}^{(i)},\ldots,\widehat{\lambda}_{N}^{(i)}),~{}~{}\widehat{{\bf V}}^{(i)}=(\widehat{{\bf v}}_{1}^{(i)},\ldots,\widehat{{\bf v}}_{N}^{(i)}).$$

In this section, we conduct empirical studies based on daily returns of S&P500 Index constituents from January 2000 to December 2020. The objective is to test, between two consecutive years, whether the principal eigenvalues, eigenvalue ratios and principal eigenvectors are equal or not.

We establish both one-sample and two-sample central limit theorems for principal eigenvalues and eigenvectors under large factor models. Based on these CLTs, we develop three tests to detect structural changes in large factor models. Our tests can reveal whether the change is in principal eigenvalues or eigenvectors or both. Numerically, these tests are found to have good finite sample performance. Applying these tests to daily returns of the S&P500 Index constituent stocks, we find that, between two consecutive years, the principal eigenvalues, eigenvalue ratios and principal eigenvectors all exhibit frequent changes.

Recall the spectral decomposition of ${\boldsymbol{\Sigma}}={\bf V}{\boldsymbol{\Lambda}}{\bf V}^{\mathrm{T}}$, where the orthogonal matrix ${\bf V}=({\bf v}_{1},\ldots,{\bf v}_{N})$ consists of the eigenvectors of ${\boldsymbol{\Sigma}}$, and ${\boldsymbol{\Lambda}}=\operatorname{diag}(\lambda_{1},\ldots,\lambda_{N})$ with eigenvalues $\lambda_{1}\geq\ldots\geq\lambda_{N}$. Write ${\boldsymbol{\Lambda}}=\operatorname{diag}({\boldsymbol{\Lambda}}_{A},{\boldsymbol{\Lambda}}_{B})$, where

$${\boldsymbol{\Lambda}}_{A}=\operatorname{diag}(\lambda_{1},\ldots,\lambda_{r})~{}~{}~{}~{}~{}\textrm{and}~{}~{}~{}~{}{\boldsymbol{\Lambda}}_{B}=\operatorname{diag}(\lambda_{r+1},\ldots,\lambda_{N}).$$

Define ${\bf x}_{t}={\bf V}^{\mathrm{T}}{\bf y}_{t}$. Then $\operatorname{Cov}({\bf x}_{t})={\boldsymbol{\Lambda}}$, and the eigenvectors of ${\boldsymbol{\Lambda}}$ are the unit vectors ${\bf e}_{1},\ldots,{\bf e}_{N}$, where ${\bf e}_{k}$ is the unit vector with 1 in the $k$th entry and zeros elsewhere. Let ${\bf S}_{N}=1/T\cdot\sum_{t=1}^{\mathrm{T}}{\bf x}_{t}{\bf x}_{t}^{\mathrm{T}}$, whose eigenvalues are denoted by $\widehat{\lambda}_{1}\geq\widehat{\lambda}_{2}\geq\ldots\geq\widehat{\lambda}_{N}$ with corresponding eigenvectors ${\bf u}_{1},{\bf u}_{2},\ldots,{\bf u}_{N}$. To resolve the ambiguity in the direction of an eigenvector, we specify the direction such that ${\bf u}_{k}[k]\geq 0$ for all $1\leq k\leq N$, namey, the $k$th coordinate of the $k$th eigenvector is nonnegative (although when the $k$th coordinate is zero, the direction is still not specified, in which case we take an arbitrary direction.) Note that the eigenvalues of $\widehat{{\boldsymbol{\Sigma}}}_{N}=\dfrac{1}{T}\sum_{t=1}^{T}{\bf y}_{t}{\bf y}_{t}^{\mathrm{T}}$ are the same as ${\bf S}_{N}$, and the eigenvectors are $\widehat{{\bf v}}_{k}={\bf V}{\bf u}_{k}$. It follows that

$$\langle{\bf v}_{k},\widehat{{\bf v}}_{k}\rangle=\langle{\bf v}_{k},{\bf V}{\bf u}_{k}\rangle={\bf v}_{k}^{\mathrm{T}}{\bf V}{\bf u}_{k}=\langle{\bf e}_{k},{\bf u}_{k}\rangle.$$

In the below, we focus on the analysis of principal eigenvalues and eigenvectors of ${\bf S}_{N}$.