Geometric Inference on Kernel Density Estimates

A kernel is a non-negative similarity measure $K:{{\mathbb{R}}}^{d}\times{{\mathbb{R}}}^{d}\to{{\mathbb{R}}}^{+}$; more similar points have higher value. For any fixed $p\in{{\mathbb{R}}}^{d}$, a kernel $K(p,\cdot)$ can be normalized to be a probability distribution; that is $\int_{x\in{{\mathbb{R}}}^{d}}K(p,x){\rm d}{x}=1$. For the purposes of this article we focus on the Gaussian kernel defined as $K(p,x)=\sigma^{2}\exp(-\|p-x\|^{2}/2\sigma^{2})$. ¹¹1 $K(p,x)$ is normalized so that $K(x,x)=1$ for $\sigma=1$. The choice of coefficient $\sigma^{2}$ is not the standard normalization, but it is perfectly valid as it scales everything by a constant. It has the property that $\sigma^{2}-K(p,x)\approx\|p-x\|^{2}/2$ for $\|p-x\|$ small.

A kernel density estimate [65, 61, 26, 27] is a way to estimate a continuous distribution function over ${{\mathbb{R}}}^{d}$ for a finite point set $P\subset{{\mathbb{R}}}^{d}$; they have been studied and applied in a variety of contexts, for instance, under subsampling [57, 71, 3], motion planning [59], multimodality [64, 33], and surveillance [37], road reconstruction [4]. Specifically,

The kernel distance [47, 42, 48, 58] (also called current distance or maximum mean discrepancy) is a metric [56, 66] between two point sets $P$, $Q$ (as long as the kernel used is characteristic [66], a slight restriction of being positive definite [2, 70], this includes the Gaussian and Laplace kernels). Define a similarity between the two point sets as

Then the kernel distance between two point sets is defined as

When we let point set $Q$ be a single point $x$, then $\kappa(P,x)=\textsc{kde}_{P}(x)$.

Kernel density estimates can apply to any measure $\mu$ (on ${{\mathbb{R}}}^{d}$) as $\textsc{kde}_{\mu}(x)=\int_{p\in{{\mathbb{R}}}^{d}}K(p,x){\rm d}{\mu(p)}.$ The similarity between two measures is $\kappa(\mu,\nu)=\int_{(p,q)\in{{\mathbb{R}}}^{d}\times{{\mathbb{R}}}^{d}}K(p,q){\rm d}{\textsf{m}_{\mu,\nu}(p,q)},$ where $\textsf{m}_{\mu,\nu}$ is the product measure of $\mu$ and $\nu$ ($\textsf{m}_{\mu,\nu}:=\mu\otimes\nu$), and then the kernel distance between two measures $\mu$ and $\nu$ is still a metric, defined as $D_{K}(\mu,\nu)=\sqrt{\kappa(\mu,\mu)+\kappa(\nu,\nu)-2\kappa(\mu,\nu)}.$ When the measure $\nu$ is a Dirac measure at $x$ ($\nu(q)=0$ for $x\neq q$, but integrates to $1$), then $\kappa(\mu,x)=\textsc{kde}_{\mu}(x)$. Given a finite point set $P\subset{{\mathbb{R}}}^{d}$, we can work with the empirical measure $\mu_{P}$ defined as $\mu_{P}=\frac{1}{|P|}\sum_{p\in P}\delta_{p}$, where $\delta_{p}$ is the Dirac measure on $p$, and $D_{K}(\mu_{P},\mu_{Q})=D_{K}(P,Q)$.

If $K$ is positive definite, it is said to have the reproducing property [2, 70]. This implies that $K(p,x)$ is an inner product in some reproducing kernel Hilbert space (RKHS) $\mathcal{H}_{K}$. Specifically, there is a lifting map $\phi:{{\mathbb{R}}}^{d}\to\mathcal{H}_{K}$ so that $K(p,x)=\langle\phi(p),\phi(x)\rangle_{\mathcal{H}_{K}}$, and moreover the entire set $P$ can be represented as $\Phi(P)=\sum_{p\in P}\phi(p)$, which is a single element of $\mathcal{H}_{K}$ and has a norm $\|\Phi(P)\|_{\mathcal{H}_{K}}=\sqrt{\kappa(P,P)}$. A single point $x\in{{\mathbb{R}}}^{d}$ also has a norm $\|\phi(x)\|_{\mathcal{H}_{K}}=\sqrt{K(x,x)}$ in this space.

Given an unknown compact set $S\subset{{\mathbb{R}}}^{d}$ and a finite point cloud $P\subset{{\mathbb{R}}}^{d}$ that comes from $S$ under some process, geometric inference aims to recover topological and geometric properties of $S$ from $P$. The offset-based (and more generally, the distance function-based) approach for geometric inference reconstructs a geometric and topological approximation of $S$ by offsets from $P$ (e.g. [13, 14, 15, 20, 21]).

Given a compact set $S\subset{{\mathbb{R}}}^{d}$, we can define a distance function $f_{S}$ to $S$; a common example is $f_{S}(x)=\inf_{y\in S}\|x-y\|$ (i.e. $\alpha$-shapes). The offsets of $S$ are the sublevel sets of $f_{S}$, denoted $(S)^{r}=f_{S}^{-1}([0,r])$. Now an approximation of $S$ by another compact set $P\subset{{\mathbb{R}}}^{d}$ (e.g. a finite point cloud) can be quantified by the Hausdorff distance $d_{H}(S,P):=\|f_{S}-f_{P}\|_{\infty}=\inf_{x\in{{\mathbb{R}}}^{d}}|f_{S}(x)-f_{P}(x)|$ of their distance functions. The intuition behind the inference of topology is that if $d_{H}(S,P)$ is small, thus $f_{S}$ and $f_{P}$ are close, and subsequently, $S$, ${(S)}^{r}$ and ${(P)}^{r}$ carry the same topology for an appropriate scale $r$. In other words, to compare the topology of offsets ${(S)}^{r}$ and ${(P)}^{r}$, we require Hausdorff stability with respect to their distance functions $f_{S}$ and $f_{P}$. An example of an offset-based topological inference result is formally stated as follows (as a particular version of the reconstruction Theorem 4.6 in [14]), where the reach of a compact set $S$, ${\sf reach}(S)$, is defined as the minimum distance between $S$ and its medial axis [54].

Here $\eta<R$ ensures that the topological properties of $(S)^{\eta}$ and $(S)^{r}$ are the same, and the $\varepsilon$ parameter ensures $(S)^{r}$ and $(P)^{r}$ are close. Typically $\varepsilon$ is tied to the density with which a point cloud $P$ is sampled from $S$.

For function $\phi:\mathbb{R}^{d}\to\mathbb{R}^{+}$ to be distance-like it should satisfy the following properties:

• •

  (D1) $\phi$ is $1$-Lipschitz: For all $x,y\in{{\mathbb{R}}}^{d}$, $|\phi(x)-\phi(y)|\leq\|x-y\|$.

• •

  (D2) $\phi^{2}$ is $1$-semiconcave: The map $x\in{{\mathbb{R}}}^{d}\mapsto(\phi(x))^{2}-\|x\|^{2}$ is concave.

• •

  (D3) $\phi$ is proper: $\phi(x)$ tends to the infimum of its domain (e.g., $\infty$) as $x$ tends to infinity.

In addition to the Hausdorff stability property stated above, as explained in [15], $f_{S}$ is distance-like. These three properties are paramount for geometric inference (e.g. [14, 53]). (D1) ensures that $f_{S}$ is differentiable almost everywhere and the medial axis of $S$ has zero $d$-volume [15]; and (D2) is a crucial technical tool, e.g., in proving the existence of the flow of the gradient of the distance function for topological inference [14].

We show how to estimate the topology (e.g., approximate persistence diagrams, infer homotopy of compact sets) using superlevel sets of the kernel density estimate of a point set $P$. We accomplish this by showing that a similar set of properties hold for the kernel distance with respect to a measure $\mu$, (in place of distance to a measure $d^{\textsf{{ccm}}}_{\mu,m_{0}}$), defined as

This treats $x$ as a probability measure represented by a Dirac mass at $x$. Specifically, we show $d^{K}_{\mu}$ is distance-like (it satisfies (D1), (D2), and (D3)), so it inherits reconstruction properties of $d^{\textsf{{ccm}}}_{\mu,m_{0}}$. Moreover, it is stable with respect to the kernel distance:

• •

  (K4) [Stability] If $\mu$ and $\nu$ are two measures on ${{\mathbb{R}}}^{d}$, then $\|d^{K}_{\mu}-d^{K}_{\nu}\|_{\infty}\leq D_{K}(\mu,\nu)$.

In addition, we show how to construct these topological estimates for $d^{K}_{\mu}$ using weighted Rips complexes, following power distance machinery introduced in [8]. That is, a particular form of power distance permits a multiplicative approximation with the kernel distance.

We also describe further advantages of the kernel distance. (i) Its sublevel sets conveniently map to the superlevel sets of a kernel density estimate. (ii) It is Lipschitz with respect to the smoothing parameter $\sigma$ when the input $x$ is fixed. (iii) As $\sigma$ tends to $\infty$ for any two probability measures $\mu,\nu$, the kernel distance is bounded by the Wasserstein distance: $\lim_{\sigma\to\infty}D_{K}(\mu,\nu)\leq W_{2}(\mu,\nu)$. (iv) It has a small coreset representation, which allows for sparse representation and efficient, scalable computation. In particular, an $\varepsilon$-kernel sample [48, 57, 71] $Q$ of $\mu$ is a finite point set whose size only depends on $\varepsilon>0$ and such that $\max_{x\in{{\mathbb{R}}}^{d}}|\textsc{kde}_{\mu}(x)-\textsc{kde}_{\mu_{Q}}(x)|=\max_{x\in{{\mathbb{R}}}^{d}}|\kappa(\mu,x)-\kappa(\mu_{Q},x)|\leq\varepsilon$. These coresets preserve inference results and persistence diagrams.

We also note in Appendix A that semiconcavity follows trivially in the RKHS $\mathcal{H}_{K}$.

We generalize a (folklore, see [15]) relation between semiconcave and Lipschitz functions and prove it for completeness. A function $f$ is $\ell$-semiconcave if the function $T(x)=(f(x))^{2}-\ell\|x\|^{2}$ is concave.

We can now state the following lemma as a corollary of Lemma 2.3 and Lemma 2.1.

Finally, for $d^{K}_{\mu}$ to be distance-like, we need to show it is proper when its range is restricted to be less than $c_{\mu}:=\sqrt{\kappa(\mu,\mu)+\kappa(x,x)}$. Here, the value of $c_{\mu}$ depends on $\mu$ not on $x$ since $\kappa(x,x)=K(x,x)=\sigma^{2}$. This is required for a distance-like version [15], Proposition 4.2) of the Isotopy Lemma ([44], Proposition 1.8).

We delay this technical proof to Appendix A. The main technical difficulty comes in mapping standard definitions and approaches for distance functions to our function $d^{K}_{\mu}$ with a restricted range.

Also by properness (see discussion in Appendix A), Lemma 2.6 also implies that $d^{K}_{\mu}$ is a closed map and its levelset at any value $a\in[0,c_{\mu})$ is compact. This also means that the sublevel set of $d^{K}_{\mu}$ (for ranges $[0,a)\subset[0,c_{\mu})$) is compact. Since the levelset (sublevel set) of $d^{K}_{\mu}$ corresponds to the levelset (superlevel set) of $\textsc{kde}_{\mu}$, we have the following corollary.

The result in [33] shows that given a measure ${\mu_{P}}$ defined by a point set $P$ of size $n$, the $\textsc{kde}_{{\mu_{P}}}$ has polynomial in $n$ modes; hence the superlevel sets of $\textsc{kde}_{{\mu_{P}}}$ are compact in this setting. The above corollary is a more general statement as it holds for any measure.

First consider the case for a kernel power distance ${f^{\textsc{k}}_{P}}(\mu,x)$ where $\mu$ is an arbitrary measure.

Recall the definition of a point $p_{+}=\arg\max_{q\in{{\mathbb{R}}}^{d}}\kappa(\mu,q)$.

We now need two properties of the point set $P$ to reach our bound, namely, the spread $\beta_{P}$ and the median concentration $\Lambda_{P}$. Typically $\log(\beta_{P})$ is not too large, and it makes sense to choose $\sigma$ so $\sigma/\Lambda_{P}\leq 1$, or at least $\sigma/\Lambda_{P}=O(1)$.

We have shown that the kernel distance function $d^{K}_{\mu}$ is a distance-like function. Therefore the reconstruction theory for a distance-like function [15] (which is an extension of results for compact sets [14]) holds in the setting of $d^{K}_{\mu}$. We state the following two corollaries for completeness, whose proofs follow from the proofs of Proposition 4.2 and Theorem 4.6 in [15]. Before their formal statement, we need some notation adapted from [15] to make these statements precise. Let $\phi:{{\mathbb{R}}}^{d}\to{{\mathbb{R}}}^{+}$ be a distance-like function. A point $x\in{{\mathbb{R}}}^{d}$ is an $\alpha$-critical point if $\phi^{2}(x+h)\leq\phi^{2}(x)+2\alpha\|h\|\phi(x)+\|h\|^{2}$ with $\alpha\in[0,1]$, $\forall h\in{{\mathbb{R}}}^{d}$. Let $(\phi)^{r}=\{x\in{{\mathbb{R}}}^{d}\mid\phi(x)\leq r\}$ denote the sublevel set of $\phi$, and let $(\phi)^{[r_{1},r_{2}]}=\{x\in{{\mathbb{R}}}^{d}\mid r_{1}\leq\phi(x)\leq r_{2}\}$ denote all points at levels in the range $[r_{1},r_{2}]$. For $\alpha\in[0,1]$, the $\alpha$-reach of $\phi$ is the maximum $r$ such that $(\phi)^{r}$ has no $\alpha$-critical point, denoted as ${\sf reach}_{\alpha}(\phi)$. When $\alpha=1$, ${\sf reach}_{1}$ coincides with reach introduced in [40].

In order to actually construct a topological estimate using the kernel distance $d^{K}_{\mu}$, one needs to be able to compute quantities related to its sublevel sets, in particular, to compute the persistence diagram of the sub-level sets filtration of $d^{K}_{\mu}$. Now we describe such tools needed for the kernel distance based on machinery recently developed by Buchet et al. [8], which shows how to approximate the persistent homology of distance-to-a-measure for any metric space via a power distance construction. Then using similar constructions, we can use the weighted Rips filtration to approximate the persistence diagram of the kernel distance.

To state our results, first we require some technical notions and assume basic knowledge on persistent homology (see [34, 35] for a readable background). Given a metric space ${{\mathbb{X}}}$ with the distance $d_{{{\mathbb{X}}}}(\cdot,\cdot)$, a set $P\subseteq{{\mathbb{X}}}$ and a function $w:P\to{{\mathbb{R}}}$, the (general) power distance $f$ associated with $(P,w)$ is defined as $f(x)=\sqrt{\min_{p\in P}\left(d_{{{\mathbb{X}}}}(p,x)^{2}+w(p)^{2}\right)}.$ Now given the set $(P,w)$ and its corresponding power distance $f$, one could use the weighted Rips filtration to approximate the persistence diagram of $w$, under certain restrictive conditions proven in Appendix D.1. Consider the sublevel set of $f$, $f^{-1}((-\infty,\alpha])$. It is the union of balls centered at points $p\in P$ with radius $r_{p}(\alpha)=\sqrt{\alpha^{2}-w(p)^{2}}$ for each $p$. The weighted Čech complex $C_{\alpha}(P,w)$ for parameter $\alpha$ is the union of simplices $s$ such that $\bigcap_{p\in s}B(p,r_{p}(\alpha))\neq 0$. The weighted Rips complex $R_{\alpha}(P,w)$ for parameter $\alpha$ is the maximal complex whose $1$-skeleton is the same as $C_{\alpha}(P,w)$. The corresponding weighted Rips filtration is denoted as $\{R_{\alpha}(P,w)\}$.

Setting $w:=d^{K}_{\mu_{P}}$ and given point set $\hat{P}_{+}$ described in Section 3, consider the weighted Rips filtration $\{R_{\alpha}(\hat{P}_{+},d^{K}_{\mu})\}$ based on the kernel power distance, ${f^{\textsc{k}}_{\hat{P}_{+}}}$. We view the persistence diagrams on a logarithmic scale, that is, we change coordinates of points following the mapping $(x,y)\mapsto(\ln x,\ln y)$. $d_{B}^{\ln}$ denotes the corresponding bottleneck distance between persistence diagrams. We now state a corollary of Theorem 3.1.