Generalization of the energy distance by Bernstein functions

A popular method to compare two probabilities is done by embedding the space (or a subset) of probabilities into a Hilbert space and use the metric provided by the embedding. Currently, there are two main approaches for this task:

1. $(I)$

   The maximum mean discrepancy on a bounded, continuous, positive definite kernel $K:X\times X\to\mathbb{R}$ that is characteristic [7], [4]. The distance between two Radon regular probabilities $P$ and $Q$ is defined by

   $$MMD(P,Q):=\sqrt{\int_{X}\int_{X}K(x,y)d[P-Q](x)d[P-Q](y)}.$$

2. $(II)$

   The use of a continuous conditionally negative definite kernel $\gamma:X\times X\to\mathbb{R}$ with $\gamma(x,x)=0$ for every $x\in X$, [19]. The kernel $\gamma$ must additionally satisfy the equality

   (1.1)

   $$\int_{X}\int_{X}-\gamma(x,y)d[P-Q](x)d[P-Q](y)=0$$

   for two Radon regular probabilities $P$ and $Q$ that integrates the function $x\to\gamma(x,z)$ for every $z\in X$ only when $P=Q$. It can be proved that the above double integral is always a nonnegative number and when this property occurs

   $$D_{\gamma}(P,Q):=\sqrt{\int_{X}\int_{X}-\gamma(x,y)d[P-Q](x)d[P-Q](y)},$$

   is a metric on the mentioned subspace of probabilities on $X$.

On this paper, we focus on the second method.

The most popular example of this method is the energy distance, initially defined as $X=\mathbb{R}^{m}$, $\gamma(x,y)=\|x-y\|^{\theta}$, where $0<\theta<2$ and the set of probabilities are those that integrates $\|x\|^{\theta}$, [24], [23]. When $\theta=2$, the kernel is conditionally negative definite but do not satisfy the additional property of Equation 1.1.

A more geometrical approach is when $\gamma$ is a metric on $X$ that satisfy Equation 1.1 (the topology is the one from the metric), hence $(X,\gamma)$ is a metric space of strong negative type. Examples of such spaces include:

$\bullet$ Hilbert spaces: Proved on [12] as a generalization of the energy distance.

$\bullet$ Hyperbolic spaces (finite dimensional): Proved on [13]

In some cases, the conditionally negative definite kernel $\gamma$ may define a metric on the set $X$, but $\gamma$ is not of strong type. A metric space where we only know that the distance is a conditionally negative definite kernel is called a metric space of negative type. An example of such space is the real sphere, proved on [5], where it is also proved that the real, complex and quaternionic projective spaces and the Cayley projective plane are not metric spaces of negative type.

In [12], it is also proved that if $(X,\gamma)$ is a metric space of negative type then $\gamma^{\theta}$, $0<\theta<1$ is a conditionally negative definite kernel that satisfies Equation 1.1, with the topology of the metric $\gamma$. Interestingly, the kernel $\gamma^{\theta}$ is a metric on $X$, with the same topology as $\gamma$, so we can rephrase the result of Lyon as $(X,\gamma^{\theta})$ being a metric space of strong negative type. We provide more details and generalizations of this property on Corollary 4.3.

The major aim of this paper is to provide a large amount of examples of conditionally negative definite kernels that satisfy Equation 1.1, by using Bernstein functions on Theorem 4.1. Our method encompasses all of the above mentioned kernels that satisfy $(II)$. We also provide a new proof that hyperbolic spaces (any dimension) are metric paces of strong negative type on Theorem on 4.2.

In [15], Mattner analysed the behaviour of the kernel $\|x-y\|^{\alpha}$, for $\alpha>2$, defined on $\mathbb{R}^{m}$. What occurs is that we can still provide a metric structure on the space of probabilities with certain integrability assumptions, but we can only compare them if they have the same vector mean ( $2<\alpha<4$), the same same vector mean and the same covariance matrix ($4<\alpha<6$), and so on. It also provided the same analysis for others radial kernels, that we generalize on Theorem 4.4 to a broader setting.

Section 3 is focused on the integrability conditions of a conditionally negative definite kernel (and its generalizations). Section 4 contains the most important results of this paper, mentioned before. On Section 5 we analyse the space of functions

where $\psi$ is a continuous function that is the difference of two derivatives (same order) of a completely monotone function. More precisely, we analyse when they are uniquely defined by the measure $\mu$. Section 2 is entirely focused on definitions that we use. The proofs are presented on Section 6.

We recall that a nonnegative measure $\lambda$ on a Hausdorff space $X$ is Radon regular (which we simply refer as Radon) when it is a Borel measure such that is finite on every compact set of $X$ and

1. (i)

   (Inner regular)$\lambda(E)=\sup\{\lambda(K),\ \ K\text{ is compact },K\subset E\}$ for every Borel set $E$.

2. (ii)

   (Outer regular) $\lambda(E)=\inf\{\lambda(U),\ \ U\text{ is open },E\subset U\}$ for every Borel set $E$.

We then said that a complex valued measure $\lambda$ of bounded variation is Radon if its variation is a Radon measure. The vector space of such measures is denoted by $\mathfrak{M}(X)$. Recall that every Borel measure of finite variation (in particular, probability measures) on a separable complete metric space is necessarily Radon.

An semi-inner product on a real (complex) vector space $V$ is a bilinear real (sesquilinear complex) valued function $(\cdot,\cdot)_{V}$ defined on $V\times V$ such that $(u,u)_{V}\geq 0$ for every $u\in V$. When this inequality is an equality only for $u=0$, we say that $(\cdot,\cdot)_{V}$ is an inner-product. Similarly, a pseudometric on a set $X$ is a symmetric function $d:X\times X\to[0,\infty)$, such that $d(x,x)=0$ that satisfies the triangle inequality. If $d(x,y)=0$ only when $x=y$, $d$ is a metric on $X$.

A kernel $K:X\times X\to\mathbb{C}$ is called positive definite if for every finite quantity of distinct points $x_{1},\ldots,x_{n}\in X$ and scalars $c_{1},\ldots,c_{n}\in\mathbb{C}$, we have that

where $\lambda=\sum_{i=1}^{n}c_{i}\delta_{x_{i}}$. The set of measures on $X$ used before are denoted by the symbol $\mathcal{M}_{\delta}(X)$.

The reproducing kernel Hilbert space (RKHS) of a positive definite kernel $K:X\times X\to\mathbb{C}$ is the Hilbert space $\mathcal{H}_{K}\subset\mathcal{F}(X,\mathbb{C})$, and it satisfies [22]

1. $(i)$

   $x\in X\to K_{y}(x):=K(x,y)\in\mathcal{H}_{K}$;

2. $(ii)$

   $\langle K_{y},K_{x}\rangle=K(x,y)$

3. $(iii)$

   $\overline{span\{K_{y},\quad y\in X\}}=\mathcal{H}_{K}$.

When $X$ is a Hausdorff space and $K$ is continuous it holds that $\mathcal{H}_{K}\subset C(X)$.

The following widely known result describes how it is possible to define a semi-inner product structure on a subspace of $\mathfrak{M}(X)$ using a continuous positive definite kernel.

We present a generalization of this result to a larger class of measures in Lemma 3.6. Usually, the kernel $K$ is bounded, so $\mathfrak{M}_{\sqrt{K}}(X)=\mathfrak{M}(X)$. On this case, if the semi-inner product is in fact an inner product we say that $K$ is integrally strictly positive definite (ISPD), and when is an inner product on the vector space of measures in $\mathfrak{M}(X)$ that $\mu(X)=0$, we say that $K$ is characteristic. If the kernel $K$ is real valued, it is sufficient to analyse the ISPD and characteristic property on real valued measures.

When the kernel is characteristic we define the maximum mean discrepancy (MMD) as the metric on the space of probability measures in $\mathfrak{M}(X)$ by

As mentioned at the introduction, the focused of this paper is to analyse metrics on the space of probabilities using conditionally negative definite kernels. We present a more general definition which will be useful to the analysis of the energy distance through the kernel $\|x-y\|^{\alpha}$, $\alpha>2$, defined on a Hilbert space.

This definition generalize the concepts of positive definite kernels ($P$ is the zero space) and CPD kernels ($P$ as the set of constant functions). The most important example is when $X$ is a finite dimensional Euclidean space and $P$ is the set of multivariable polynomials on $X$ with degree less than or equal to a constant $k\in\mathbb{N}$, [25] [9], [6]. Sometimes it might be more convenient to work with the opposite sign on Definition 2.2, on this case we say that the kernel is $P$-conditionally negative definite ($P$-CND).

In [9], [16], it is proved that a characterization for the continuous functions $\psi:[0,\infty)\to\mathbb{R}$, such that the kernel

is CPD for $P$ as the family of multivariable polynomials of degree less than a fixed $\ell\in\mathbb{Z}_{+}$ (we denote this family by $\pi_{\ell-1}(\mathbb{R}^{m})$, where $\pi_{-1}(\mathbb{R}^{m})=\{0\}$ and $\pi_{0}(\mathbb{R}^{m})=\{\text{constant functions}\}$) for every $m\in\mathbb{N}$. A function $\psi$ satisfy this property if and only if $\psi\in C^{\infty}(0,\infty)$ and $(-1)^{\ell}\psi^{(\ell)}$ is a completely monotone function on $(0,\infty)$. A function with this property can be uniquely written as

where $\lambda$ is a nonnegative Radon measure on $(0,\infty)$ (not necessarily with finite variation) with

and $a_{k}\in\mathbb{R}$, $(-1)^{\ell}a_{\ell}\geq 0$ and $\omega_{0,\infty}$ is the zero function. For instance, the functions

1. $i)$

   $(-1)^{\ell}t^{a}+p(t)$;

2. $ii)$

   $(-1)^{\ell+1}t^{\ell}\log(t)+p(t)$;

3. $iii)$

   $(-1)^{\ell}(c+t)^{a}+p(t)$;

4. $iv)$

   $e^{-rt}+p(t)$,

are elements of $CM_{\ell}$, for $\ell-1<a\leq\ell$ , $c>0$ and $p\in\pi_{\ell-1}$. Those functions are not only in $CM_{\ell}$, but they are $\ell-1$ continuously differentiable on $[0,\infty)$ and we have a similar and simpler characterization compared to Equation 2.3 for them.

In general, a function $\psi\in CM_{\ell}$ is such that $\psi\in C^{\ell-1}([0,\infty))$ if and only if

where $\eta$ is a nonnegative Radon measure on $(0,\infty)$ (not necessarily with finite variation) with

$b_{k}=\psi^{(k)}(0)/k!$ for $k<\ell$ and $(-1)^{\ell}b_{\ell}\geq 0$.

Note that if a function $\psi\in CM_{\ell}$ then $\psi(\cdot+c)\in CM_{\ell}\cap C^{\ell-1}([0,\infty))$. On this case, the measure $\eta_{c}$ relative to the decomposition given on Equation 2.4 has finite variation and satisfy $d\eta_{c+s}(r)=e^{-sr}d\eta_{c}(r)$ for every $c,s>0$. This property and the decomposition given on Equation 2.4 are implicitly proved on Theorem $2.1$ on [16] and can also be found on Theorem $8.19$ of [25]. We remark that a polynomial $p\in CM_{\ell}$ if and only if $p\in\pi_{\ell}(\mathbb{R})$ and the constant $(-1)^{\ell}p^{(\ell)}\geq 0$.

By Lemma $2.4$ in [9], a function $\psi\in CM_{\ell}$ satisfies $|\psi(t)|\lesssim 1+t^{\ell}$ (this notation means that $|\psi(t)|/1+t^{\ell}$ is a bounded function).

The following known result states a connection between positive definite kernels and $P$-CPD kernels [25]. A Lagrange basis for $P$ is a basis $\{p_{1},\ldots,p_{m}\}$ of $P$ and points $\xi_{1},\ldots,\xi_{m}\in X$, such that $p_{i}(\xi_{j})=\delta_{i,j}$. A set of points $\xi_{1},\ldots,\xi_{m}\in X$ is unisolvent with respect to a $m$-dimensional space $P$ if the only function $p\in P$ such that $p(\xi_{i})=0$ for every $i$ is the zero function.

This result can be easily seen by the fact that if $x_{1},\ldots,x_{n}\in X$ and $c_{1},\ldots,c_{n}\in\mathbb{C}$ are such that $\sum_{i=1}^{n}c_{i}p(x_{i})=0$ for every $p\in P$, then

and conversely, if $z_{1},\ldots,z_{m+n}\in X$ (with $z_{n+k}=\xi_{k}$) and $d_{1},\ldots,d_{m+n}\in\mathbb{C}$, then

where $e_{i}=d_{i}$, for $i\leq n$ and $e_{i}=-\sum_{i=1}^{n}d_{i}p_{i-n}(z_{i})$, for $i>n$.

Similar to continuous positive definite kernels, continuous $P$-CPD kernels can be analyzed by its behaviour on a certain type of space of measures.

If we restrict the measures on Theorem 3.3 to those that $\gamma(x,\xi)\in L^{1}(|\mu|)$ for every $\xi\in X$, then the kernel $\gamma$ defines a semi-inner product on this vector space.

When $P$ is the space generated by a single function $p$, we can simplify the assumptions of Theorem 3.3.

As a direct consequence of the previous Lemma we obtain that if the function $\gamma(x,x)/p^{2}(x)$ is bounded, the set of measures on $\mathfrak{M}_{P}(X)$ that integrates $\gamma(x,y)$ is a vector space and the double integral defines a semi inner product on it. We focus on the CPD case and when $\gamma$ is real valued due to its relevance.

On the next lemma we improve the condition $\sqrt{K(x,x)}\in L^{1}(|\mu|)$ and the set of measures analysed on Lemma 2.1, at the cost of describing the function $K_{\mu}$ at the exception of a $|\mu|$ measure zero set.

Since all kernels that we deal on this Section are real valued, we simplify the writing by only focusing on real valued measures (which we still use the notation $\mathfrak{M}(X)$). As mentioned on Section $2$, this is not a restriction.

In [12], it is proved that on a separable real Hilbert space $\mathcal{H}$, the bilinear function $I_{1/2}$ defined as

defines a inner product on the vector space

The function $t\in[0,\infty)\to\psi(t):=\sqrt{t}\in\mathbb{R}$ is an example of a Bernstein function, [18]. It is continuous, $\psi\in C^{\infty}((0,\infty))$ and $\psi^{\prime}$ is a completely monotone function on $(0,\infty)$ (we do not need to assume on our context that Bernstein functions are nonnegative). In other words, a function $\psi$ is a Bernstein function if and only if $-\psi\in CM_{1}$, and then it can be written, by Equation 2.4 for $\ell=1$, as

So,

and this kernel is CPD. The Gaussian kernels $e^{-r\|x-y\|^{2}}$, $r>0$, are ISPD for every Hilbert space [8], being so, by Fubini-Tonelli Theorem we have that if $\mu\in\mathfrak{M}(\mathcal{H})$ with $\mu(\mathcal{H})=0$ and $\|x\|\in L^{1}(|\mu|)$, then

Further, the double inner integral is positive whenever $\mu$ is not the zero measure, implying that the final result is a positive number, which is the key argument in order to verify that $I_{1/2}$ is an inner product, thus reobtaining the main result of [12] by a complete different argument. More generally, we have the following result.

We emphasize that by Lemma 3.4 ($p$ is the constant $1$ function) $\psi(\gamma(x,y))\in L^{1}(|\eta|\times|\eta|)$ if and only if $x\to\psi(\gamma(x,z))\in L^{1}(|\eta|)$ for some (or every) $z\in X$.

For instance, if $X$ is a real Hilbert space $\mathcal{H}$, $\gamma(x,y)=\|x-y\|^{2}$ and $\psi(t)=t^{a/2}$, $0<a<2$, then

defines an inner product on

It is relevant to say that usually the inner product on Theorem 4.1 is not complete (hence, $\mathfrak{M}_{1}(X;\gamma,\psi)$ is not a Hilbert space). For instance, on [20] it is proved that the Gaussian kernel can be used to define an inner product on the space of tempered distributions on Euclidean spaces.

Another example occurs on the generalized real hyperbolic space. Let $\mathcal{H}$ be a Hilbert space and define $\mathbb{H}:=\{(x,t_{x})\in\mathcal{H}\times(0,\infty),\quad t_{x}^{2}-\|x\|^{2}=1\}$ be the real hyperbolic space relative to $\mathcal{H}$ and consider the kernel

which satisfies the relation

where $d_{\mathbb{H}}$ is a metric in $\mathbb{H}$. On [3] or chapter $5$ in [1], it is proved that the metric $d_{\mathbb{H}}$ on $\mathbb{H}$ is a CND kernel, being so we can apply Theorem 4.1 for the kernel $\gamma=d_{\mathbb{H}}$ and $\psi=t^{a/2}$, $0<a<2$, then

defines a inner product on

We can also include the case $a=2$. A proof when $\mathbb{H}$ is finite dimensional was provided on [13] using geometric properties of hyperbolic spaces. Our proof relies on a Laurent type of approximation for the function $\operatorname{arcCosh}(t)$.

A different behaviour occurs on the generalized real spheres. Let $\mathcal{H}$ be a Hilbert space and define $S^{\mathcal{H}}:=\{x\in\mathcal{H},\quad\|x\|=1\}$ be the real sphere relative to $\mathcal{H}$. The kernel $d_{S^{\mathcal{H}}}$ defined on $S^{\mathcal{H}}$ by the relation

is a metric and defines a CND kernel as shown on [5]. However, unlikely the Hilbert space and the real hyperbolic space, $d_{S^{\mathcal{H}}}$ is not a metric space of strong negative type, [14]. Gangolli also proved on [5] that the metric on the other compact two-point homogeneous spaces (real/complex/quaternionic projective spaces and the Cayley projective plane) does not define a CND kernel.

The following Corollary of Theorem 4.1, connects the setting of metric spaces of strong negative type and the kernels on Theorem 4.1.

As an example of Corollary 4.3, the Bersntein function $\psi(t)=\log(t+1)$, satisfies $\psi(0)=0$ and $\lim_{t\to\infty}\psi(t)/t=0$. In particular, on a Hilbert space $\mathcal{H}$ , $\log(\|x-y\|+1)$ is a metric on $\mathcal{H}$ that is homeomorphic with the Hilbertian topology and this metric is of strong negative type. Interestingly we can apply Corollary 4.3 again in order to obtain that the same occurs with the metric $\log(\log(\|x-y\|+1)+1)$.

Returning to the kernel $(x,y)\in\mathcal{H}\times\mathcal{H}\to\|x-y\|^{a}$, we may ask ourselves what occurs when $a\geq 2$. The case $a=2$ is simpler, because

for every $\mu,\nu\in\mathfrak{M}_{1}(\mathcal{H};t):=\{\eta\in\mathfrak{M}(\mathcal{H}),\quad\|x\|^{2}\in L^{1}(|\eta|)\text{ and }\eta(X)=0\}$. This still defines a semi-inner product on $\mathfrak{M}_{1}(\mathcal{H};t)$, but the vector space

is equivalent to the zero measure on this inner product. For an arbitrary measure $\eta\in\mathfrak{M}(\mathcal{H})$ such that $\|x\|^{2}\in L^{1}(|\eta|)$, the linear functional

is continuous, so there exists a vector $v_{\eta}$, which we call the vector mean of $\eta$, which represents the above continuous linear functional.

On the case $a>2$, a different behaviour emerges. The double integral kernel does not define a semi-inner product on $\mathfrak{M}_{1}(\mathcal{H},t^{a/2})$, however, if we restrict ourselves to the vector space space