Improved Discretization Analysis for Underdamped
Langevin Monte Carlo

We address the two questions above by providing a new Girsanov discretization bound for ULMC. Our bound holds in the strong Rényi divergence metric and applies under general assumptions (in particular, it does not require strong log-concavity of the target $\pi$, and it allows for weakly smooth potentials). Consequently, it leads to the following new state-of-the-art results for ULMC:

• •

  We obtain an $\epsilon^{2}$-guarantee in KL divergence with iteration complexity $\widetilde{\mathcal{O}}(\kappa^{3/2}d^{1/2}\epsilon^{-1})$ for strongly log-concave and log-smooth distributions, which removes the Lipschitz Hessian assumption of [Ma+21]; here, $\kappa$ is the condition number of the distribution.

• •

  We obtain an $\epsilon$-guarantee in TV distance with iteration complexity $\widetilde{\mathcal{O}}(C_{\mathsf{LSI}}^{3/2}L^{3/2}d^{1/2}\epsilon^{-1})$ under a log-Sobolev inequality (LSI) and $L$-smooth potential, again without assuming a Lipschitz Hessian. This is the state-of-the-art guarantee for this class of distributions with regards to dimension dependence.

• •

  We obtain $\epsilon^{2}$-guarantees in the stronger Rényi divergence metric of any order in $[1,2)$ with iteration complexity $\widetilde{\mathcal{O}}(C_{\mathsf{PI}}^{3/2}L^{3/2}\,d^{2}\epsilon^{-1})$ under a Poincaré inequality and a $L$-smooth potential, which improves to $\widetilde{\mathcal{O}}(C_{\mathsf{PI}}Ld^{2}\epsilon^{-1})$ under log-concavity. These are the first guarantees for ULMC known in these settings, and they substantially improve upon the corresponding results for LMC in these settings [Che+21].

• •

  In the Poincaré case, we also consider weakly smooth potentials (i.e., Hölder continuous gradients with coefficient $s\in(0,1]$), which more realistically reflect the delicate smoothness properties of distributions satisfying a Poincaré inequality.

We now discuss our results in more detail in the context of the existing literature.

The remainder of this paper will be organized as follows. In Section 2, we will review the required definitions and assumptions. In Section 3, we will state our main results and briefly sketch their proofs. In Section 4, we highlight several implications of our theorems through some examples. In Section 5, we briefly sketch the proofs of our main results, before concluding in Section 6 with a discussion of future directions.

Hereafter, we will use $\lVert\cdot\rVert$ to denote the $2$-norm on vectors. In general, we will only work with measures that admit densities on $\mathbb{R}^{d}$, and we will abuse notation slightly to conflate a measure with its density for convenience. The notation $a=\mathcal{O}(b)$ signifies that there exists an absolute constant $C>0$ such that $a\leq Cb$, and $\widetilde{\mathcal{O}}(\cdot)$ hides logarithmic factors. Similarly we write $a=\Theta(b)$ if there exist constants $c,C>0$ such that $cb\leq a\leq Cb$, and $\widetilde{\Theta}(\cdot)$ hides logarithmic factors. The stationary measure (in the position coordinate) is $\pi\propto\exp(-U)$, and $U$ will be referred to as the potential. We will use $L^{2}(\pi)$ to denote test functions $f$ where $\mathbb{E}_{\pi}\,f^{2}<\infty$, and $\mathcal{H}^{1}(\pi)$ to denote weakly differentiable $L^{2}(\pi)$ functions where $\partial_{x_{i}}f\in L^{2}(\pi)$. Finally, the notations $\lesssim$, $\gtrsim$, $\asymp$ represent $\leq$, $\geq$, $=$ up to absolute constants. Further notations are introduced in subsequent sections.

In this subsection, we will define the relevant processes, divergences, and isoperimetric inequalities. Firstly, we define the ULMC algorithm by the following stochastic differential equation (SDE):

$\displaystyle\begin{aligned} \mathrm{d}x_{t}&=v_{t}\,\mathrm{d}t\,,\\ \mathrm{d}v_{t}&=-\gamma v_{t}\,\mathrm{d}t+\nabla U(x_{kh})\,\mathrm{d}t+\sqrt{2\gamma}\,\mathrm{d}B_{t}\,,\end{aligned}$

(ULMC)

where $t\in[kh,(k+1)h)$ for some step size $h>0$. We note this formulation of ULMC can be integrated in closed form (see Appendix A).

Next, we define a few measures of distance between two probability distributions $\mu$ and $\pi$ on $\mathbb{R}^{d}$. We define the total variation distance as

$\displaystyle\lVert\mu-\pi\rVert_{\mathsf{TV}}\coloneqq\sup|\mu(A)-\pi(A)|\,,$

(2.1)

where the $\sup$ is taken over Borel measurable sets $A\subset\mathbb{R}^{d}$. We further define the $\mathsf{KL}$ divergence as

$\displaystyle\mathsf{KL}(\mu\mathbin{\|}\pi)\coloneqq\int\frac{\mathrm{d}\mu}{\mathrm{d}\pi}\log\frac{\mathrm{d}\mu}{\mathrm{d}\pi}\,\mathrm{d}\pi\,,$

(2.2)

and $\mathsf{KL}(\mu\mathbin{\|}\pi)\coloneqq+\infty$ if $\mu$ is not absolutely continuous with respect to $\pi$. Finally, we define the Rényi divergence with order $q>1$ as

$\displaystyle\mathcal{R}_{q}(\mu\mathbin{\|}\pi)\coloneqq\frac{1}{q-1}\log\int\Bigl{|}\frac{\mathrm{d}\mu}{\mathrm{d}\pi}\Bigr{|}^{q}\,\mathrm{d}\pi\,,$

and similarly $\mathcal{R}_{q}(\mu\mathbin{\|}\pi)\coloneqq+\infty$ if $\mu\not\ll\pi$. The Rényi divergence upper bounds $\mathsf{KL}$ for all orders, i.e., $\mathsf{KL}(\mu\mathbin{\|}\pi)\leq\mathcal{R}_{q}(\mu\mathbin{\|}\pi)$ for any order $q>1$, and $\mathcal{R}_{q}$ is monotonic in $q$. In particular, when $q=2$, we also get $\chi_{2}$ divergence, i.e., $\chi_{2}(\mu\mathbin{\|}\pi)=\exp(\mathcal{R}_{2}(\mu\mathbin{\|}\pi))-1$.

Our primary results are provided under the following smoothness conditions.

We conduct three lines of analysis. The first assumes strong convexity of the potential, i.e.:

In the case $m=0$ above, we say that $U$ is convex. If a potential function $U$ is (strongly) convex, then we say the distribution $\pi\propto\exp(-U)$ is (strongly) log-concave.

A second, strictly more general assumption is the log-Sobolev inequality.

An $m$-strongly convex potential is known to satisfy (LSI) with constant $m^{-1}$ [BGL14]. More generally, we can consider the following weaker isoperimetric inequality, which corresponds to a linearization of (LSI).

Conditions (LSI) and (PI) are standard assumptions made on the stationary distribution in the theory of Markov diffusions as well as sampling [BGL14, VW19, Che+21, Che23]. They are known to be satisfied by a broad class of targets such as log-concave distributions or certain mixture distributions [Che21, CCN21].

We define the condition number for an $m$-strongly log-concave target with $(L,s)$-weakly smooth potential as $\kappa\triangleq L/m$. In the case where instead of strong convexity, the target only satisfies (LSI) (respectively (PI)), the condition number is instead $\kappa\triangleq C_{\mathsf{LSI}}L$ (respectively $\kappa\triangleq C_{\mathsf{PI}}L$).

Finally, we collect several mild assumptions to simplify computing the bounds below, which have also appeared in prior work; see in particular the discussion in [Che+21, Appendix A].

In order to state our results for ULMC in $\mathsf{KL}$ and $\mathsf{TV}$, we leverage the following result in continuous-time from [Ma+21], which relies on an entropic hypocoercivity argument, after a time-change of the coordinates (see Appendix B.1 for a proof).

We now proceed to state our main results more precisely. First, we obtain the following KL divergence guarantee under strong log-concavity and smoothness.

Here, we justify the choice of error tolerance for $\mathsf{KL}$ to be $\epsilon^{2}$. Based on Pinsker’s and Talagrand’s transport inequalities, we know $\mathsf{KL}$ is on the order of $\mathsf{TV}^{2},\mathcal{W}_{2}^{2}$. Hence, this allows for a fair comparison of convergence guarantees in terms of $\mathsf{KL}$ with $\mathsf{TV}$ and $\mathcal{W}_{2}$. Weakening the strong convexity assumption to $\eqref{eq:LSI}$, we obtain a result in $\mathsf{TV}$.

To state our convergence results in $\mathcal{R}_{q}$, we additionally inherit the following technical assumption from [CLW20].

In the light of the above discussion, we emphasize that this additional assumption largely does not hinder the applicability of our results. Under this assumption, [CLW20] established the following guarantee on (1) in continuous time.

Our final result leverages the above accelerated convergence guarantees of ULD, and establishes the first bound for ULMC in Rényi divergence with an improved condition number dependence.

For results under both the Poincaré and log-Sobolev inequalities, we leverage the existing results as stated in [CLW20, Ma+21], which we present in Lemmas 5 and 8. These allow us to bound $\chi_{2}(\mu_{t}\mathbin{\|}\mu)$, $\mathsf{KL}(\mu_{t}\mathbin{\|}\mu)$ with exponentially decaying quantities.

With the additional assumption of strong convexity, we can obtain a contraction in an alternate system of coordinates $(\phi,\psi)\triangleq\mathcal{M}(x,v)\triangleq(x,x+\frac{2}{\gamma}\,v)$ (see Appendix B). This allows us to consider the distributions of the continuous time iterates and the target in these alternate coordinates $\mu_{t}^{\mathcal{M}},\mu^{\mathcal{M}}$ respectively. From this, we obtain the following proposition.

The main idea behind the proof of this proposition is to analyze the discretization (ULMC) of the underdamped Langevin diffusion in the coordinates $(\phi,\psi)$. Note that this can be written in the following form, for some matrix $\overline{\Sigma}\in\mathbb{R}^{2d\times 2d}$ and function $\bar{F}:\mathbb{R}^{d}\times\mathbb{R}^{d}\to\mathbb{R}^{d}\times\mathbb{R}^{d}$,

$\displaystyle(\phi_{(k+1)h},\psi_{(k+1)h})$

$\displaystyle\stackrel{{\scriptstyle d}}{{=}}\bar{F}(\phi_{kh},\psi_{kh})+\mathcal{N}(0,\overline{\Sigma})\,.$

This is the composition of a deterministic function $\bar{F}$ giving the mean of the next iterate of ULMC started at $(\phi,\psi)$, followed by addition with a Gaussian distribution giving the variance of the resulting iterate. In particular, we show that for coordinates $(\phi(x,v),\psi(x,v))\triangleq(x,x+\frac{2}{\gamma}v)$, we can find an almost sure strict contraction under $\bar{F}$ in the sense that

$\displaystyle\lVert\bar{F}\rVert_{\text{Lip}}\leq 1-\frac{m}{\sqrt{2L}}\,h+\mathcal{O}(Lh^{2})\,,$

where by abuse of notation $\bar{F}:\mathbb{R}^{2d}\to\mathbb{R}^{2d}$, and the seminorm $\lVert g\rVert_{\text{Lip}}$ of a function $g:\mathbb{R}^{2d}\to\mathbb{R}^{2d}$ refers to the Lipschitz constant of the function.

Since $\bar{F}$ is a contraction for small enough $h$, each push forward improves the log-Sobolev constant by a multiplicative factor [VW19, Lemma 19]. At the same time, a Gaussian convolution can only worsen the log-Sobolev constant by an additive constant [Cha04, Corollary 3.1]. Subsequently, the log-Sobolev constant at each iterate forms a (truncated) geometric sum, and therefore can be bounded by the infinite series. This incidentally can be used to bound the log-Sobolev constant of the ULMC iterates. Taking an appropriate limit of $h\to 0$ while keeping $Nh=t$, we arrive at the stated bound in the proposition. Consequently, considering the decomposition of the $\mathsf{KL}$, a simple application of Cauchy–Schwarz tells us that

	$\displaystyle\mathsf{KL}(\hat{\mu}_{t}^{\mathcal{M}}\mathbin{\|}\mu^{\mathcal{M}})$	$\displaystyle=\int\log\frac{\hat{\mu}_{t}^{\mathcal{M}}}{\mu^{\mathcal{M}}}\,\mathrm{d}\hat{\mu}_{t}^{\mathcal{M}}=\mathsf{KL}(\hat{\mu}_{t}^{\mathcal{M}}\mathbin{\|}\mu_{t}^{\mathcal{M}})+\int\log\frac{\mu_{t}^{\mathcal{M}}}{\mu^{\mathcal{M}}}\,\mathrm{d}\hat{\mu}_{t}^{\mathcal{M}}$
		$\displaystyle\leq\mathsf{KL}(\hat{\mu}_{t}^{\mathcal{M}}\mathbin{\|}\mu_{t}^{\mathcal{M}})+\mathsf{KL}(\mu_{t}^{\mathcal{M}}\mathbin{\|}\mu^{\mathcal{M}})+\sqrt{\chi^{2}(\hat{\mu}_{t}^{\mathcal{M}}\mathbin{\|}\mu_{t}^{\mathcal{M}})\times\mathsf{var}_{\mu_{t}^{\mathcal{M}}}\Bigl{(}\log\frac{\mu_{t}^{\mathcal{M}}}{\mu^{\mathcal{M}}}\Bigr{)}}\,.$

The log-Sobolev inequality for $\mu_{t}^{\mathcal{M}}$ implies a Poincaré inequality, which allows us to bound the variance term by the Fisher information $\mathsf{FI}(\mu_{t}^{\mathcal{M}}\mathbin{\|}\mu^{\mathcal{M}})=\operatorname{\mathbb{E}}_{\mu_{t}^{\mathcal{M}}}\lVert\nabla\log(\mu_{t}^{\mathcal{M}}/\mu^{\mathcal{M}})\rVert^{2}$. This can be bounded by the same entropic hypocoercivity argument from [Ma+21] that is used to generate our $\mathsf{TV}$ bounds, while the remaining two terms are handled respectively via the discretization analysis and again the entropic hypocoercivity argument.

The main result we use to control the discretization error can be found below.

We outline the proof of this result below. Similar to the work of [Che+21], we first invoke the data processing inequality, allowing us to bound the Rényi between the time marginal distributions of the iterates with Rényi between the path measures

$\displaystyle\mathcal{R}_{q}(\hat{\mu}_{T}\mathbin{\|}\mu_{T})\leq\mathcal{R}_{q}(P_{T}\mathbin{\|}Q_{T})\,,$

where $P_{T},Q_{T}$ are probability measures of (ULMC), (1) respectively on the space of paths $C([0,T],\mathbb{R}^{2d})$. Subsequently, we invoke Girsanov’s theorem, which allows us to exactly bound the pathwise divergence by the difference between the drifts of the two processes:

$\displaystyle\mathcal{R}_{2q}(P_{T}\mathbin{\|}Q_{T})\lesssim\log\operatorname{\mathbb{E}}\exp\Bigl{(}\frac{4q^{2}}{\gamma}\int_{0}^{T}\lVert\nabla U(x_{t})-\nabla U(x_{\lfloor t/h\rfloor h})\rVert^{2}\,\mathrm{d}t\Bigr{)}\,.$

It remains to bound the term inside the expectation. We achieve this by conditioning on the event that $\sup_{t\in[0,T]}\lVert x_{t}-x_{\lfloor t/h\rfloor h}\rVert^{2}$ is bounded by a vanishing quantity as $h\to 0$, which we must demonstrate occurs with sufficiently high probability. To show this, we begin with a single-step analysis, i.e., we bound the above for $T\leq h$. Compared to LMC, the main gain in this analysis is that the SDEs (1) and (ULMC) match exactly in the position coordinate, while the difference between the drifts manifests solely in the momentum. After integration of the momentum, the order of error is better in the position coordinate (the dominant term is $\mathcal{O}(dh^{2})$ compared to $\mathcal{O}(dh)$ seen in [Che+21, Lemma 24]).

The technique for extending this analysis from a single step to the full time interval follows closely that seen in [Che+21]. In particular, we obtain a dependence for $\lVert x_{t}\rVert$ on $\lVert x_{kh}\rVert$ in the interval $t\in[kh,(k+1)h)$. Controlling the latter is quite complicated when the potential satisfies only a Poincaré inequality, since it is equivalent to showing sub-Gaussian tail bounds on the iterates, while the target itself is not sub-Gaussian in the position coordinate. By comparing against an auxiliary potential, we can show that for our choice of initialization, the iterates remain sub-Gaussian for all iterations up to $N$ (albeit with a growing constant). Finally, this allows us to recover our discretization result in the proposition above.