A Geometric Reduction Approach for
Identity Testing of Reversible Markov Chains

The scientist is given the full description of a reference transition matrix $\overline{P}$ and a single Markov chain (MC) $X_{1}^{n}$ sampled with respect to some unknown transition operator $P$ and arbitrary initial distribution. For fixed proximity parameter $\varepsilon>0$, the goal is to design an algorithm that distinguishes between the two cases $P=\overline{P}$ and $K(P,\overline{P})>\varepsilon$, with high probability, where $K$ is a contrast function between stochastic matrices.

Under the contrast function (1) described in Section 2, and the hypothesis that $P$ and $\overline{P}$ are both irreducible and symmetric over a finite space $\mathcal{X}$, Daskalakis et al. (2018) constructed a tester with sample complexity $\tilde{\mathcal{O}}(\left|\mathcal{X}\right|/\varepsilon+h)$, where $h$ (Daskalakis et al., 2018, Definition 3) corresponds to some hitting property of the chain, and a lower bound in $\Omega(\left|\mathcal{X}\right|/\varepsilon)$. In Cherapanamjeri and Bartlett (2019), a graph partitioning algorithm delivers, under the same symmetry assumption, a testing procedure with sample complexity $\mathcal{O}(\left|\mathcal{X}\right|/\varepsilon^{4})$, i.e. independent on hitting properties. More recently, Fried and Wolfer (2022) relaxed the symmetry requirement, replacing it with a more natural reversibility assumption. Their algorithm has a sample complexity of $\mathcal{O}(1/(\overline{\pi}_{\star}\varepsilon^{4}))$, where $\overline{\pi}_{\star}$ is the minimum stationary probability of $\overline{P}$, gracefully recovering Cherapanamjeri and Bartlett (2019) under symmetry. In parallel, Wolfer and Kontorovich (2020) started the research program of inspecting the problem under the infinity norm for matrices, and derived nearly minimax-optimal bounds.

We show how to mostly recover Fried and Wolfer (2022) under additional assumptions (see Section 3), with a technique based on a geometry preserving embedding. We obtain a more economical proof than Fried and Wolfer (2022) who went through the process of re-deriving a graph partitioning algorithm for the reversible case. Furthermore, our approach, by its generality, is also applicable to related inference problems.

We denote $\mathcal{W}(\mathcal{X},\mathcal{D})$ the set of irreducible stochastic matrices over a strongly connected digraph $(\mathcal{X},\mathcal{D})$. When $P\in\mathcal{W}(\mathcal{X},\mathcal{D})$, $\pi$ is unique and we denote $\pi_{\star}=\min_{x\in\mathcal{X}}\pi(x)>0$. When $P$ verifies the detailed-balance equation $\pi(x)P(x,x^{\prime})=\pi(x^{\prime})P(x^{\prime},x)$ for any $(x,x^{\prime})\in\mathcal{D}$, we say that $P$ is reversible.

In contradistinction with the distribution setting, merging symbols in a Markov chain may break the Markov property. For $P\in\mathcal{W}(\mathcal{Y},\mathcal{E})$ and a surjective map $\kappa\colon\mathcal{Y}\to\mathcal{X}$, merging elements of $\mathcal{Y}$ together, we say that $P$ is $\kappa$-lumpable Kemeny and Snell (1983) when the resulting process still defines a MC. If so, the resulting transition matrix can be found in (Kemeny and Snell, 1983, Theorem 6.3.2), which we denote as $\kappa_{\star}P\in\mathcal{W}(\mathcal{X},\kappa_{2}(\mathcal{D}))$, with

We consider the following notion of discrepancy between two stochastic matrices $P,P^{\prime}\in\mathcal{W}(\mathcal{X})$,

Although $K$ first appeared in Daskalakis et al. (2018) in the context of MC testing, its inception can be traced back to Kazakos Kazakos (1978). It is instructive to observe that $K$ vanishes on chains that share an identical component and does not satisfy the triangle inequality for reducible matrices, hence is not a proper metric on $\mathcal{W}(\mathcal{X})$ (Daskalakis et al., 2018, p.10, footnote 13). Some additional properties of $K$ of possible interest are listed in (Fried and Wolfer, 2022, Section 7).

Problem reduction is ubiquitous in the property testing literature. Our work takes inspiration from Goldreich (2016), who introduced two so-called “stochastic filters” in order to show how in the distribution setting, identity testing was reducible to uniformity testing, thereby recovering the known complexity of $\mathcal{O}(\sqrt{\left|\mathcal{X}\right|}/\varepsilon^{2})$ obtained more directly by Valiant and Valiant (2017). Other notable works include Diakonikolas and Kane (2016), who reduced a collection of distribution testing problems to $\ell_{2}$-identity testing.

Our construction and notation follow Nagaoka (2005), who established the dually-flat structure $(\mathcal{W}(\mathcal{X},\mathcal{D}),\mathfrak{g},\nabla^{(e)},\nabla^{(m)})$ on the space of irreducible stochastic matrices. Writing $P_{\theta}\in\mathcal{W}(\mathcal{X},\mathcal{D})$ for the transition matrix at coordinates $\theta\in\Theta\subset\mathbb{R}^{d}$, with $d$ the dimension of the manifold, and with the shorthand $\partial_{i}\doteq\partial/\partial\theta^{i}$, recall that the Fisher metric is expressed in the chart induced basis $(\partial_{i})_{i\in[d]}$ as

In Wolfer and Watanabe (2022), the following notion of an embedding for stochastic matrices is proposed.

The above embeddings are characterized as the linear maps over lumpable matrices that satisfy some monotonicity requirements and are congruent with respect to the lumping operation (Wolfer and Watanabe, 2022, Theorem 3.1). When for any $y,y^{\prime}\in\mathcal{Y}$, it additionally holds that $\Lambda(y,y^{\prime})=\Lambda(y^{\prime})\delta\left[(\kappa(y),\kappa(y^{\prime}))\in\mathcal{D}\right]$, the embedding $\Lambda_{\star}$ is called memoryless (Wolfer and Watanabe, 2022, Section 3.4.2) and is e/m-geodesic affine (Wolfer and Watanabe, 2022, Th. 3.2, Lemma 3.6), preserving both exponential and mixture families of MC.

Given $\overline{\pi}$ and $\Delta$ as defined in Section 3, from (Wolfer and Watanabe, 2022, Corollary 3.3), there exists a lumping function $\kappa\colon[\Delta]\to\mathcal{X}$, and a memoryless embedding $\sigma^{\overline{\pi}}_{\star}\colon\mathcal{W}(\mathcal{X},\mathcal{D})\to\mathcal{W}([\Delta],\mathcal{E})$ with $\mathcal{E}=\left\{(y,y^{\prime})\in[\Delta]^{2}\colon(\kappa(y),\kappa(y^{\prime}))\in\mathcal{D}\right\}$, such that $\sigma^{\overline{\pi}}_{\star}\overline{P}$ is symmetric. Furthermore, identifying $\mathcal{X}=\left\{1,2,\dots,\left|\mathcal{X}\right|\right\}$, its existence is given constructively by

As a consequence, we have both,

1. 1.

   The expression of $\sigma^{\overline{\pi}}_{\star}\overline{P}$ following the algebraic manipulations in Definition 4.1.

2. 2.

   A random algorithm Wolfer and Watanabe (2022) to simulate trajectories from $\sigma^{\overline{\pi}}_{\star}P$ out of trajectories from $P$ (see (Wolfer and Watanabe, 2022, Section 3.1)).

It is immediate that $P=\overline{P}\implies L_{\star}P=L_{\star}\overline{P}$.

From Lemma 5.2, $K(P,\overline{P})>\varepsilon\implies K(\sigma^{\overline{\pi}}P,\sigma^{\overline{\pi}}\overline{P})>\varepsilon$.

As a consequence of (Cherapanamjeri and Bartlett, 2019, Theorem 10), the sample complexity of testing is upper bounded by $\mathcal{O}(\Delta/\varepsilon^{4})$. With $\overline{\pi}_{\star}=p_{1}/\Delta$ and treating $p_{1}$ as a small constant, we recover the known sample complexity.