Propagation of chaos in path spaces via information theory

The interacting particle system, mostly built upon basic physical laws including Newton’s second law, has received growing popularity recent years in the study of both natural and social sciences. Practical application of such large-scale interacting particle systems includes groups of birds [11], consensus clusters in opinion dynamics [41], chemotaxis of bacteria [23], etc. Despite its strong applicability, the theoretical analysis and practical computation for the interacting particle system is rather complicated, mainly due to the fact that the particle number $N$ is very large in many practical settings. One classical strategy to reduce this complexity is to study instead the “mean-field” regime. The limiting partial differential equation (mean-field equation) is used to describe the behavior of the particle system as $N\rightarrow\infty$. This approximation allows one to obtain a one-body model instead of the original many-body one. For instance, Jeans proposed a mean-field equation to study the galactic dynamics in 1915 [28]. Much work has been done to study the mean-field behaviors of various kinds of interacting particle systems [15, 33, 39, 18, 43] in the past decades.

Here, let us take the second order system as the example to explain the concepts of mean field limit and propagation of chaos. The second-order system is described by Newton’s second law for $N$ point particles driven by 2-body interaction forces and Brownian motions, satisfying the following system of stochastic differential equations (SDE):

where $m$ and $\gamma$ represent the mass $m$ and friction coefficient respectively, $X_{i}(t),V_{i}(t)\in\mathbb{R}^{d}$. The processes $W_{i}(t)$ ($1\leq i\leq N$) are independent Brownian motions in $\mathbb{R}^{d^{\prime}}$, and $K$: $\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ is the interaction kernel. We assume that the initial data $\{(X_{i}(0),V_{i}(0))\}$ are $i.i.d$ drawn from some initial law $F_{0}^{N}$ independent of the Brownian motions. Denote $Z_{i}(t):=(X_{i}(t),V_{i}(t))$, and the corresponding joint law

where $\mathcal{P}(\mathbb{R}^{2Nd})$ denotes the probability measure space on $\mathbb{R}^{2Nd}.$ Then, the evolution of the density $F_{t}^{N}$ satisfies a Liouville’s equation [16, 17]:

with $F_{t}^{N}|_{t=0}=F_{0}^{N}.$ Note that the matrix $\Lambda$ is defined by $\Lambda:=\sigma\sigma^{T}$. Here, “$:$” means the Hilbert-Schmidt inner product so that $\nabla^{2}_{v_{i}}:(\Lambda F_{t}^{N})=\sum_{j,k}\partial_{v_{j}v_{k}}^{2}(\Lambda_{jk}F_{t}^{N})$. As the particle number $N$ tends to infinity, the correlation between any two focused particles through the weak interaction is expected to vanish. Hence, if two particles are initially independent, then they are expected to be independent as $N\to\infty$ at any fixed time point $t>0$. This is the so-called propagation of chaos. Due to the asymptotic independence, a fixed particle with position and velocity $\bar{Z}_{i}(t):=(\bar{X}_{i}(t),\bar{V}_{i}(t))$ is then expected to satisfy the following mean field Mckean SDE system:

where $\bar{F}_{t}\in\mathcal{P}(\mathbb{R}^{2d})$ is the law, and $\bar{\rho}_{t}(x):=\int_{\mathbb{R}^{d}}\bar{F}_{t}(x,v)dv$ is its marginal. The law $\bar{F}_{t}$ is then expected to satisfy the following mean field kinetic Fokker-Planck equation [24, 25]:

Rigorous justification of this mean limit, or the propagation of chaos, has then become an active research topic.

The prevalent method in analyzing mean-field limits is based on Dobrushin’s Estimate, which is proposed in 1979 by Dobrushin etc. [13], to study the stability of the mean-field characteristic flow in terms of Wasserstein distances. Dobrushin-type analysis has now been a classical tool in mean-field limits for Valsov-type equations during these decades. Based on Dobrushin-type analysis, one can then prove the mean-field limit for the deterministic system in a finite time interval $[0,T]$ in terms of Wasserstein distances [3, 44, 18]. Another way is to compare the stochastic trajectories through certain coupling technique. By considering trajectory controls, the mean-field limit for stochastic systems with Lipschitz kernel $K$ has been established [50, 19, 21].

Another class of methods is to compare the laws directly. What has become popular recently on chaos qualification is given by the analysis of relative entropy (also called Kullback-Leibler divergence, KL-divergence) between $F_{t}^{N:k}=\int_{(\mathbb{R}^{2d})^{N-k}}F_{t}^{N}dz_{k+1}\cdots dz_{N}$ and $k$ tensorized product of $\bar{F}_{t}$, $\bar{F}_{t}^{\otimes k}:=\prod_{i=1}^{k}\bar{F}_{t}(z_{i})$ for $1\leq k\leq N$. The analysis could also be performed on the laws on path space with $F_{t}^{N:k}$ and $\bar{F}_{t}^{\otimes k}$ being their time marginals. Some early results in path space using the relative entropy have been achieved in the last century (e.g. [2, 1]). For time marginal distributions, Jabin et. al. proved the propagation of chaos for Vlasov-type systems with $\mathcal{O}(k/N)$ bound, assuming the interaction kernel $K$ is bounded, and the propagation of chaos for first order systems with singular kernels [26]. For results in path space, Lacker obtained the propagation of chaos relying on Girsanov’s and Sanov’s theorem [30] and the BBGKY hierarchy [31, 32]. The approach in [31, 32] yields an $\mathcal{O}((k/N)^{2})$ bound of the relative entropy between the marginal law of $k$ particles and its limiting product measure. For singular $L^{p}$-interactions, Tomašević et. al. used the the partial Girsanov transform to derive the propagation of chaos in [27, 51]. Recently, Hao et. al. further showed the strong convergence of the propagation of chaos with singular $L^{p}$-interactions in [22]. Also, based on Lacker’s approach, Cattiaux gave an $\mathcal{O}(k/N)$ estimate on the path space in [6], by using the invariance of relative entropy under time reversal [5]. The results in [12] and [20] are uniform in time for the Coulomb and the Biot-Savart kernel, respectively. There is a vast literature on this topic, and we provide recent review articles [7, 8] for the convenience of readers.

In this work, we propose to use the information theory to study the propagation of chaos by comparing the discrepancy between the joint law of the particle system and the corresponding mean-field equation in terms of KL-divergence defined by

where $P$ and $Q$ are two probability measures over some appropriate space $E$. In our framework, the propagation of chaos is reduced to the space for driving processes with possible lower dimension. We will mainly take the second-order systems as the example, which avoids using the usual hypocoercivity technique or taking the pseudo-inverse of the diffusion matrix. We remark that the bounds under relative entropy for the second order system can be obtained by direct Girsanov transform if one takes the pseudo-inverse of the degenerate diffusion matrix as mentioned in [31, Remark 4.5]. Nevertheless, we believe our approach is still of significance as there is no degeneracy in diffusion if we look at the measures in the space for driving processes, which could be more stable for numerical computation. We will also look at the application of our framework to other illustrating examples.

We focus an estimate for the KL-divergence between the laws in path space, in particular $D_{KL}(F^{N}_{[0,T]}\|\bar{F}^{\otimes N}_{[0,T]})$. Here $F^{N}_{[0,T]}$ and $\bar{F}^{\otimes N}_{[0,T]}$ are probability distributions in the path space $\mathcal{X}:=C([0,T];\mathbb{R}^{2Nd})$ (for fixed time interval $[0,T]$) corresponding to the SDE systems (1.1) and $N$ independent copies of (1.4) respectively. Denoting $\mathcal{Z}_{[0,T]}:=(Z_{1},\dots,Z_{N})_{[0,T]}$, $\bar{\mathcal{Z}}_{[0,T]}:=(\bar{Z}_{1},\dots,\bar{Z}_{N})_{[0,T]}$ in the path space, the path measures satisfy $F^{N}_{[0,T]}=\mathcal{Z}_{[0,T]}{\#}\mathbb{P}$, and $\bar{F}^{\otimes N}_{[0,T]}=\bar{\mathcal{Z}}_{[0,T]}{\#}\mathbb{P}$ ($\mathbb{P}$ is the original probability measure such that $W$ is a Brownian motion). With this setting, $F_{t}^{N}$ is the time marginal of $F^{N}_{[0,T]}$, and $\bar{F}_{t}^{\otimes N}$ is then the time marginal of $\bar{F}^{\otimes N}_{[0,T]}$. We then regard the process of the mean-field McKean SDEs and the interacting particle systems as the same dynamical system with different driving processes (input signals). Then, applying the data processing inequality, we can work on probability measures in the space for the input signals instead of the space for the particles. The former space is sometimes easier to deal with than the latter as one may avoid the degeneracy of the diffusion. Moreover, the dimension could be lower. This has similarity with the so-called latent space in machine learning [38]. Moreover, we will also present the applications of the framework onto neural networks and numerical analysis to illustrate this point.

The rest of the paper is organized as follows: In Section 2, we present our main ideas. The result (Theorem 3.6) on the propagation of chaos for the second-order system in path space is shown in Section 3 for both bounded kernels (not necessarily smooth) or Lipschitz kernels (not necessarily bounded) with the necessary assumptions and auxiliary lemmas. In Section 4, we provide two applications of our approach on numerical analysis and neural networks. Lastly in Section 5, we perform a discussion on the reversed relative entropy and mass-independence.

In this section, taking the second order system as the example, we present the main ideas without rigorous proof. The rigorous mathematical setup, assumptions and proof will be given in the next section.

For fixed $[0,T]$, let $\bar{F}_{[0,T]}$ be the law of the trajectories of the following Mckean SDE system (1.4). Then the tensorized distribution $\bar{F}^{\otimes N}_{[0,T]}$ is the law of trajectories of the following system:

and the particles $\bar{Z}_{i}:=(\bar{X}_{i},\bar{V}_{i})$, $1\leq i\leq N$ are independent.

The key idea of this work is rewriting (1.1) above into:

where the process $\theta_{i}^{(1)}(t)$ is defined by

Here,

We also denote

Based on (2.8) and (2.7), formally, we write the generalized dynamics

Here, $\theta:=(\theta_{1},\cdots,\theta_{N})$ is a driving process. In (2.8), the driving process is taken as the noise process $\theta^{(2)},$ while in (2.7) is taken as $\theta^{(1)}.$ For fixed initial data, as shown in (2.13), the driving process $\theta$ can be viewed as an input, then through the equation (2.12), the particle trajectory is obtained as an output.

From this perspective, a natural guess is that, if there is only slight difference between two driving processes, the difference between the outputs might be not large. Luckily, if the mean field McKean SDE (1.4) has pathwise uniqueness, the following well-known data processing inequality [9] can help to establish such intuition.

Now, by the data processing inequality, we can control the KL-divergence between the output into that between the input. In this respect, we change our problem from the trajectory space into the space for the driving process $\theta$. Exactly, we find that

where we recall $F^{N}_{[0,T]}$ and $\bar{F}^{\otimes N}_{[0,T]}$ are path measures introduced in Section 1 and we denote $Q^{j}$ to be the path measures for

To compute the latter relative entropy, we rewrite the equation for $\theta^{(1)}$ by

Then, $\theta^{(1)}$ satisfies an SDE in the space of the driving process, with a dimension smaller than that of $(X,V)$. Then, by Girsanov’s transform, it holds

Note that this reduction avoids the degeneracy of the diffusion coefficient. Though the degeneracy can be treated by using the pseudo-inverse as remarked in [31], such a reduction could be helpful for practical estimates using numerical computations. We will give more details in the next sections.

Let us discuss the choice of the noise and dynamical system. One may be tempted to rewrite the mean-field McKean SDE into

with

Then, the $N$-body interacting particle system is given by

with $\eta_{i}^{(1)}(t):=\sigma\cdot W_{i}(t)$ $(1\leq i\leq N)$.

The two systems are also the same dynamical system with difference driving noises

At first glance, this formulation seems good since the drift in $\eta^{(2)}$ involves only the solution to the mean-field McKean SDE. Then, one may apply the law of large numbers. However, this is not the case. In fact, applying the data processing inequality, one has

where $\bar{Q}^{j}$ is the law for $\eta^{(j)}$. We consider

Here, the mapping $\hat{\Phi}_{s}:\eta\mapsto(X,V)$ is the solution map for the $N$-body interacting dynamical system and $\pi_{s}f=f(s)$ is the time marginal. This is again an SDE in the space for the driving process. Then,

The point is that the Radon-Nykodym derivative is integrated against $\bar{Q}^{1}$. The Girsanov’s transform then gives that

where the inside is changed from $\eta^{(2)}$ to $\eta^{(1)}$! The eventual result is the same as (2.17).

In this section, we establish the propagation of chaos in path space for the second order systems using the framework of information theory, in particular the data processing inequality.

We first present our assumptions on the kernels and coefficients. The first set of assumptions requires that $K$ is bounded. {assumption}

• (a)

  The kernel $K$ has finite essential bound, namely, $\|K\|_{L^{\infty}(\mathbb{R}^{d})}<+\infty$.

• (b)

  The matrix $\Lambda=\sigma\sigma^{T}$ is non-degenerate with minimum eigenvalue $\lambda>0.$

The boundedness condition for the interaction kernel $K$ (condition (a) in Assumption 3 above) sometimes is strong in practice. Here, if we assume that the initial distribution has a fast decaying tail, we can allow a Lipschitz kernel. In fact, we will assume also alternatively the followings: {assumption}

• (a)

  The initial space-marginal distribution of the Mckean SDE (1.4) is sub-Gaussian, namely, there exists $C>0$ such that for any $a\geq 0$, $P(|\bar{X}_{1}(0)|>a)\leq 2\exp(-a^{2}/C^{2})$.

• (b)

  The interaction kernel $K(\cdot)$ is $L_{K}$-Lipschitz, namely, $\forall x,y\in\mathbb{R}^{d}$, $|K(x)-K(y)|\leq L_{K}|x-y|$.

• (c)

  The matrix $\Lambda=\sigma\sigma^{T}$ is non-degenerate with minimum eigenvalue $\lambda>0.$