Moderate deviation principles for kernel estimator of invariant density in bifurcating Markov chains models.

We denote by $\mathbb{N}$ (resp. $\mathbb{N}^{*}$) the space of (resp. positive) natural integers. We set $\mathbb{T}_{0}=\mathbb{G}_{0}=\{\emptyset\}$, $\mathbb{G}_{k}=\{0,1\}^{k}$ and $\mathbb{T}_{k}=\bigcup_{0\leq r\leq k}\mathbb{G}_{r}$ for $k\in{\mathbb{N}}^{*}$, and $\mathbb{T}=\bigcup_{r\in{\mathbb{N}}}\mathbb{G}_{r}$. The set $\mathbb{G}_{k}$ corresponds to the $k$-th generation, $\mathbb{T}_{k}$ to the tree up to the $k$-th generation, and $\mathbb{T}$ the complete binary tree. One can see that the genealogy of the cells is entirely described by $\mathbb{T}$ (each vertex of the tree designates an individual). For $i\in\mathbb{T}$, we denote by $|i|$ the generation of $i$ ($|i|=k$ if and only if $i\in\mathbb{G}_{k}$) and $iA=\{ij;j\in A\}$ for $A\subset\mathbb{T}$, where $ij$ is the concatenation of the two sequences $i,j\in\mathbb{T}$, with the convention that $\emptyset i=i\emptyset=i$. For $A\subset\mathbb{T}$, we denote by $|A|$ the number of elements of $A$. Note that for all $n\in\mathbb{N},$ $|\mathbb{G}_{n}|=2^{n}$ and $|\mathbb{T}_{n}|=2^{n+1}-1.$

For our convenience, we set $S=\mathbb{R}^{d}$, $d\geq 1$ and $S$ is equipped with the Borel sigma-algebra ${\mathscr{S}}$. For any $q\in\mathbb{N}^{*}$, we denote by ${\mathcal{B}}(S^{q})$ (resp. ${\mathcal{B}}_{b}(S^{q})$, resp. ${\mathcal{C}}_{b}(S^{q})$) the space of (resp. bounded, resp. bounded continuous ) $\mathbb{R}\text{-}$valued measurable functions defined on $S^{q}$. For all $q\in\mathbb{N}^{*}$, we set ${\mathscr{S}}^{\otimes q}={\mathscr{S}}\otimes\ldots\otimes{\mathscr{S}}$. Let ${\mathcal{P}}$ be a probability kernel on $(S,{\mathscr{S}}^{\otimes 2})$, that is: ${\mathcal{P}}(\cdot,A)$ is measurable for all $A\in{\mathscr{S}}^{\otimes 2}$, and ${\mathcal{P}}(x,\cdot)$ is a probability measure on $(S^{2},{\mathscr{S}}^{\otimes 2})$ for all $x\in S$. For any $g\in{\mathcal{B}}_{b}(S^{3})$ and $h\in{\mathcal{B}}_{b}(S^{2})$, we set for $x\in S$:

(1)

$$({\mathcal{P}}g)(x)=\int_{S^{2}}g(x,y,z)\;{\mathcal{P}}(x,{\rm d}y,{\rm d}z)\quad\text{and}\quad({\mathcal{P}}h)(x)=\int_{S^{2}}h(y,z)\;{\mathcal{P}}(x,{\rm d}y,{\rm d}z).$$

We define $({\mathcal{P}}g)$ (resp. $({\mathcal{P}}h)$), or simply $Pg$ for $g\in{\mathcal{B}}(S^{3})$(resp. ${\mathcal{P}}h$ for $h\in{\mathcal{B}}(S^{2})$), as soon as the corresponding integral (1) is well defined, and we have that ${\mathcal{P}}g$ and ${\mathcal{P}}h$ belong to ${\mathcal{B}}(S)$. We denote by ${\mathcal{P}}_{0}$, ${\mathcal{P}}_{1}$ and ${\mathcal{Q}}$ respectively the first and the second marginal of ${\mathcal{P}}$, and the mean of ${\mathcal{P}}_{0}$ and ${\mathcal{P}}_{1}$, that is, for all $x\in S$ and $B\in\mathcal{S}$

$${\mathcal{P}}_{0}(x,B)={\mathcal{P}}(x,B\times S),\quad{\mathcal{P}}_{1}(x,B)={\mathcal{P}}(x,S\times B)\quad\text{ and}\quad{\mathcal{Q}}=\frac{({\mathcal{P}}_{0}+{\mathcal{P}}_{1})}{2}.$$

Now let us give a precise definition of bifurcating Markov chain.

Following [16], we introduce an auxiliary Markov chain $Y=(Y_{n},n\in{\mathbb{N}})$ on $(S,{\mathscr{S}})$ with $Y_{0}=X_{1}$ and transition probability ${\mathcal{Q}}$. The chain $(Y_{n},n\in\mathbb{N})$ corresponds to a random lineage taken in the population. We shall write ${\mathbb{E}}_{x}$ when $X_{\emptyset}=x$ (i.e. the initial distribution $\nu$ is the Dirac mass at $x\in S$). We will assume that the Markov chain $Y$ is ergodic and we denote by $\mu$ its invariant probability measure. Asymptotic and non-asymptotic behaviour of BMCs are strongly related to the knowledge of $\mu$. In particular, Guyon has proved that if $Y$ is ergodic, then for all $f\in{\mathcal{C}}_{b}(S)$,

$$|{\mathbb{A}}_{n}|^{-1}\sum_{u\in{\mathbb{A}}_{n}}f(X_{u})\underset{n\rightarrow+\infty}{\xrightarrow{\hskip 21.33955pt}}\langle\mu,f\rangle\quad\text{in probability,}\quad\text{where ${\mathbb{A}}_{n}\in\{\mathbb{G}_{n},\mathbb{T}_{n}\}.$}$$

But in most cases, the invariant probability $\mu$ is unknown, so its estimation from the data is of great interest. For that purpose, we do the following assumption.

Recall that ${\mathbb{A}}_{n}\in\{\mathbb{G}_{n},\mathbb{T}_{n}\}$ and $S=\mathbb{R}^{d}$, $d\geq 1$. Assume we observe $\mathbb{X}^{n}=(X_{u},u\in{\mathbb{A}}_{n})$. Let $(h_{n},n\in\mathbb{N})$ be a sequence of positive numbers which converges to $0$ as $n$ goes to infinity. We will simply write $h$ for $h_{n}$ if there is no ambiguity. Let the kernel function $K:S\rightarrow\mathbb{R}$ such that $\int_{S}K(x)dx=1.$ Then, for all $x\in S,$ we propose to estimate $\mu(x)$ by

(2)

$$\widehat{\mu}_{{\mathbb{A}}_{n}}(x)=|{\mathbb{A}}_{n}|^{-1}h_{n}^{-d/2}\sum_{u\in{\mathbb{A}}_{n}}K_{h_{n}}(x-X_{u}),$$

where $K_{h_{n}}(\cdot)=h_{n}^{-d/2}K(h_{n}\cdot).$ These estimators are strongly inspired from [18, 21, 22]. They have been studied in [13, 8] (non asymptotic studies) and in [2] (central limit theorem).

Our aim is to study moderate deviation principles for the estimators defined in (2). Before we proceed, let us introduce the notion of moderate deviation principle. We give the definition in a general setting. Let $(Z_{n})_{n\geq 0}$ be a sequence of random variables with values in $S$ endowed with its Borel $\sigma$-field ${\mathscr{S}}$ and let $(s_{n})_{n\geq 0}$ be a positive sequence that converges to $+\infty$. We assume that $Z_{n}/s_{n}$ converges in probability to 0 and that $Z_{n}/\sqrt{s_{n}}$ converges in distribution to a centered Gaussian law. Let $I:S\rightarrow\mathbb{R}^{+}$ be a lower semicontinuous function, that is for all $c>0$ the sub-level set $\{x\in S,I(x)\leq c\}$ is a closed set. Such a function $I$ is called rate function and it is called good rate function if all its sub-level sets are compact sets. Let $(b_{n})_{n\geq 0}$ be a positive sequence such that $b_{n}\rightarrow+\infty$ and $b_{n}/\sqrt{s_{n}}\rightarrow 0$ as $n$ goes to $+\infty$.

The following two concepts are closely related to the theory of MDP: super-exponential convergence and exponential equivalence. Let $(Z_{n},n\in\mathbb{N})$, $(W_{n},n\in\mathbb{N})$ be sequences of random variables and $Z$ a random variable with value in a metric space $(S,d)$.

The following result give a sufficient condition for super-exponential convergence of a sequence of random variables.

The moderate deviation principle has been proved in the i.i.d. setting for kernel density estimator, see for e.g. Gao [15], Mokkadem & al. [20]. We refer also to [19] where Mokkaddem and Pelletier have constructed confidence bands for probability densities based on moderate deviation principles. In this paper, we will establish moderate deviation principle for $\widehat{\mu}_{{\mathbb{A}}_{n}}(x)$ following the martingale approach developed in [2]. We will need the following assumption.

The others assumptions we will need are based on the following bias-variance type decomposition of the estimator $\widehat{\mu}_{{\mathbb{A}}_{n}}(x)$:

(4)

$$\widehat{\mu}_{{\mathbb{A}}_{n}}(x)-\mu(x)=B_{h_{n}}(x)+V_{{\mathbb{A}}_{n},h_{n}}(x),$$

where for $h>0$ and ${\mathbb{A}}\subset\mathbb{T}$ finite:

$$B_{h}(x)=h^{-d/2}K_{h}\star\mu(x)-\mu(x)\quad\text{and}\quad V_{{\mathbb{A}},h}(x)=|{\mathbb{A}}|^{-1}h^{-d/2}\sum_{u\in{\mathbb{A}}}\Big{(}K_{h}(x-X_{u})-K_{h}\star\mu(x)\Big{)},$$

and for $h>0$ and $u\in\mathbb{T}$, we set:

$$K_{h}\star\mu(x)=\mathbb{E}_{\mu}[K_{h}(x-X_{u})]=\int_{S}K_{h}(x-y)\mu(y)\,dy.$$

To study the variance term $V_{{\mathbb{A}}_{n},h_{n}}(x)$, we will introduce a more general sequence of functions (see Section 3.2).

The following assumptions on the kernel, the bandwidth and the regularity of the unknown density function are usual. Recall $S={\mathbb{R}}^{d}$ with $d\geq 1$.

For $\alpha>1/2$, we shall also assume the following.

In the sequel, we will consider the positive sequence $(b_{n},n\in\mathbb{N})$ such that:

(6)

$$\lim_{n\rightarrow+\infty}b_{n}=+\infty;\quad\lim_{n\rightarrow+\infty}\frac{n^{3/2}\,b_{n}}{\sqrt{|\mathbb{G}_{n}|h_{n}^{d}}}=0;\quad\lim_{n\rightarrow+\infty}\frac{b_{n}}{\sqrt{|\mathbb{G}_{n}|h_{n}^{2s+d}}}=+\infty,$$

where $s$ is the regularity parameter given in Assumption 2.12.

The paper is organised as follows. In Section 3.1 we state the main result for the moderate deviation principles of the estimators $\widehat{\mu}_{{\mathbb{A}}_{n}}(x)$ for $x$ in the set continuity of $\mu$ and ${\mathbb{A}}_{n}\in\{\mathbb{T}_{n},\mathbb{G}_{n}\}$. In Section 3.2, directly linked to the study of variance term $V_{{\mathbb{A}},h}(x)$ defined in (4), we study the moderate deviation principle for general additive functionals of BMCs. Sections 4 and 5 are devoted to the proofs of results. In Section 6, we recall some useful results.

First, we state a strong consistency result for the estimators $\widehat{\mu}_{{\mathbb{A}}_{n}}(x)$ for $x$ in the set of continuity of $\mu$. Its proof is given in Section 4.1.

The main result of this Section is the following theorem which state the moderate deviation principle for $\widehat{\mu}_{{\mathbb{A}}_{n}}(x)-\mu(x)$ for $x$ in the set of continuity of the function $\mu$.

In order to obtain confidence intervals for $\mu(x)$, it would be interesting to replace $\mu(x)$ in the expression of the rate function $I(\cdot)$ by an estimator. In that direction, we have the following. Let ${\mathbb{A}}_{n}^{*}\in\{\mathbb{G}_{n},\mathbb{T}_{n}\}.$ Obviously, ${\mathbb{A}}_{n}^{*}$ and ${\mathbb{A}}_{n}$ can be the same. We consider the estimator $\widehat{\mu}_{{\mathbb{A}}_{n}^{*}}(x)$ of $\mu(x)$ defined with ${\mathbb{A}}_{n}^{*}$ instead of ${\mathbb{A}}_{n}$. Let $(\varpi_{n},n\in\mathbb{N})$ be a sequence of real numbers such that $\varpi_{n}\rightarrow 0$ as $n\rightarrow+\infty.$ Then, we have the following result which the proof is given in Section 4.3.

In particular, using the contraction principle (see for e.g Dembo and Zeitouni [11], Chap 4), we have the following corollary of Theorem 3.3.

Using the structure of the asymptotic variance $\sigma^{2}$ in (7), we can prove the following multidimensional result which the proof is given in Section 4.4

 

In order to study the variance term $V_{{\mathbb{A}}_{n},h_{n}}(x)$, we give here a moderate deviation principle for a general additive functionals of BMCs. For that purpose, we introduce the following assumption.

We will use the following notations. For a finite set ${\mathbb{A}}\subset\mathbb{T}$ and a function $f\in{\mathcal{B}}(S)$, we set:

$$M_{\mathbb{A}}(f)=\sum_{i\in{\mathbb{A}}}f(X_{i}).$$

In this paper, we are interested in the cases ${\mathbb{A}}=\mathbb{G}_{n}$ and ${\mathbb{A}}=\mathbb{T}_{n}$, that is the $n\text{-}$th generation and the first $n$ generation of the tree. Recall $\mu$ the invariant probability of ${\mathcal{Q}}$, transition probability of the auxiliary Markov chain $(Y_{n},n\in\mathbb{N})$. For $f\in L^{1}(\mu)$, we set:

$$\tilde{f}=f-\langle\mu,f\rangle.$$

Recall the sequence ${\mathfrak{f}}_{n}$ defined in Assumption 3.8. For $n\in\mathbb{N}$, we set:

(8)

$$N_{n,\emptyset}({\mathfrak{f}}_{n})=|\mathbb{G}_{n}|^{-1/2}\sum_{\ell=0}^{n}M_{\mathbb{G}_{n-\ell}}(\tilde{f}_{\ell,n}).$$

The notation $N_{n,\emptyset}$ means that we consider the average from the root $\emptyset$ to the $n\text{-}$th generation.

For our convenience, we assume that the quantity $\gamma$ which appears in Assumptions 2.11 and 3.8 is the same. The main result of this section is the following.

We begin the proof with ${\mathbb{A}}_{n}=\mathbb{T}_{n}.$ Recall the decomposition (4) with $\mathbb{T}_{n}$ instead of ${\mathbb{A}}$. Using Lemma 6.3, we have $\lim_{n\rightarrow+\infty}|B_{h_{n}}(x)|=0.$ From Remark 2.7, this implies that $B_{h_{n}}(x)\xRightarrow[b_{n}^{2}]{\rm superexp}0.$ Next, we set $f_{n}(\cdot)=K_{h_{n}}(x-\cdot)$ in such a way that we have

$$|\mathbb{T}_{n}|^{-1}h_{n}^{d-/2}\sum_{u\in\mathbb{T}_{n}}\Big{(}K_{h}(x-X_{u})-K_{h}\star\mu(x)\Big{)}=|\mathbb{T}_{n}|^{-1}h_{n}^{d-/2}\sum_{\ell=0}^{n}M_{\mathbb{G}_{\ell}}(\tilde{f}_{n}).$$

Following line by line the proof of (32) (where we take $f_{\ell,n}=f_{n}$ for all $\ell\leq n$), we get

$$\mathbb{P}\Big{(}|\mathbb{T}_{n}|^{-1}h_{n}^{d-/2}\Big{|}\sum_{\ell=0}^{n}M_{\mathbb{G}_{\ell}}(\tilde{f}_{n})\Big{|}>\delta\Big{)}\leq 2\exp\Big{(}\frac{3\delta}{c_{1}+c_{2}\delta}\Big{)}\exp\Big{(}-\frac{3\delta^{2}|\mathbb{T}_{n}|h_{n}^{d}}{c_{1}+c_{2}\delta}\Big{)}.$$

Taking the $\log$, dividing by $b_{n}^{2}$ and letting $n$ goes to the infinity in the latter inequality, we get

$$|\mathbb{T}_{n}|^{-1}h_{n}^{d-/2}\sum_{u\in\mathbb{T}_{n}}\Big{(}K_{h}(x-X_{u})-K_{h}\star\mu(x)\Big{)}\xRightarrow[b_{n}^{2}]{\rm superexp}0.$$

It then follows from the decomposition (4) that $\widehat{\mu}_{\mathbb{T}_{n}}(x)\xRightarrow[b_{n}^{2}]{\rm superexp}\mu(x).$ We similarly get the result for ${\mathbb{A}}_{n}=\mathbb{G}_{n}$ and this ends the proof of the lemma.

We begin the proof with ${\mathbb{A}}_{n}=\mathbb{T}_{n}$. We have the following decomposition:

$$b_{n}^{-1}\sqrt{|\mathbb{T}_{n}|h_{n}^{d}}\big{(}\widehat{\mu}_{\mathbb{T}_{n}}(x)-\mu(x)\big{)}=\sqrt{\frac{|\mathbb{G}_{n}|}{|\mathbb{T}_{n}|}}b_{n}^{-1}N_{n,\emptyset}({\mathfrak{f}}_{n})+\frac{\sqrt{|\mathbb{T}_{n}|h_{n}^{d}}}{b_{n}}B_{h_{n}}(x),$$

where ${\mathfrak{f}}_{n}=(f_{\ell,n},\,n\geq\ell\geq 0)$ with the functions $f_{\ell,n}=f_{\ell,n}^{\text{id}}$ defined in (9) for $n\geq\ell\geq 0$ and $f_{\ell,n}=0$ otherwise; $N_{n,\emptyset}({\mathfrak{f}}_{n})$ is defined in (8) and the bias term $B_{h_{n}}(x)$ is defined in (4). Thanks to Theorem 3.10 applied to the sequence $(f_{\ell,n}^{id},n\geq\ell\geq 0)$ and using that $\lim_{n\rightarrow+\infty}|\mathbb{G}_{n}|/|\mathbb{T}_{n}|=1/2,$ we get that $\sqrt{|\mathbb{G}_{n}||\mathbb{T}_{n}|^{-1}}b_{n}^{-1}N_{n,\emptyset}({\mathfrak{f}}_{n})$ satisfies a moderate deviation principle in $\mathbb{R}$ with speed $b_{n}^{2}$ and rate function $I$ defined by: $I(y)=y^{2}/(2\|K\|_{2}^{2}\,\mu(x))$ for all $y\in\mathbb{R}.$ To complete the proof of Theorem 3.2, it suffices to prove that

(11)

$$\lim_{n\rightarrow+\infty}\frac{\sqrt{|\mathbb{T}_{n}|h_{n}^{d}}}{b_{n}}B_{h_{n}}(x)=0.$$

Next, using that

$$\mu(x-h_{n}y)-\mu(x)=\sum_{j=1}^{d}(\mu(x_{1}-h_{n}y_{1},\ldots,x_{j}-h_{n}y_{j},x_{j+1},\ldots,x_{d})\\ -\mu(x_{1}-h_{n}y_{1},\ldots,x_{j-1}-h_{n}y_{j-1},x_{j},x_{j+1},\ldots,x_{d})),$$

the Taylor expansion and Assumption 2.12, we get that, for some finite constant $C>0$,

	$\displaystyle|\mathbb{T}_{n}|^{1/2}h_{n}^{d/2}B_{h_{n}}(x)$	$\displaystyle=\sqrt{|\mathbb{T}_{n}|h_{n}^{d}}\,\,\Big{|}\int_{\mathbb{R}^{d}}h_{n}^{-d}K(h_{n}^{-1}(x-y))\mu(y)dy-\mu(x)\Big{|}$
		$\displaystyle=\sqrt{|\mathbb{T}_{n}|h_{n}^{d}}\,\,\Big{|}\int_{\mathbb{R}^{d}}K(y)(\mu(x-h_{n}y)-\mu(x))\,dy\Big{|}$
		$\displaystyle\leq C\sqrt{|\mathbb{T}_{n}|h_{n}^{d}}\,\,\sum_{j=1}^{d}\,\,\int_{\mathbb{R}^{d}}K(y)\frac{(h_{n}|y_{j}|)^{s}}{\lfloor s\rfloor!}dy$
		$\displaystyle\leq C\sqrt{|\mathbb{T}_{n}|h_{n}^{2s+d}}.$

Now, (11) follows using the latter inequality and (6). This ends the proof of Theorem 3.2 for ${\mathbb{A}}_{n}=\mathbb{T}_{n}$. The proof is similar for ${\mathbb{A}}_{n}=\mathbb{G}_{n}$ using $f_{\ell,n}=f^{0}_{\ell,n}.$