Functional Covering of Point Processes

The classical theory of compression [2] focuses on discrete-time, sequential sources. The theory is thus well-suited to text, audio, speech, genomic data, and the like. Continuous-time signals are typically handled by reducing to discrete-time via projection onto a countable basis. Multi-dimensional extensions enable application to images and video.

Point processes model a distinct data type that appears in diverse domains such as neuroscience [3, 4, 5, 6, 7, 8], communication networks [9, 10, 11], imaging [12, 13], blockchains [14, 15, 16, 17], and photonics [18, 19, 20, 21, 22]. Formally, a point process can be viewed as a random counting measure on some space of interest [23], or if the space is a real line, a random counting function; we shall adopt the latter view. Informally, it may be viewed as simply a random collection of points representing epochs in time or points in space.

Compression of point processes emerges naturally in several of the above domains. Sub-cranial implants need to communicate the timing of neural firings to a monitoring station over a wireless link that is low-rate because it must traverse the skull [24, 25]. In network flow correlation analysis, one cross-correlates packet timings from different links in the network [11]; this requires communication of the packet timings from one place to another. Compressing point process realizations in 2-D (also known as point clouds) arises in computer vision [26, 27, 28], and so on.

Various specialized approaches have been developed for compressing point processes, and in particular for measuring distortion. One natural approach is for the compressed representation to be itself a point-process realization. In this case, the distortion can be the sum of the absolute value of the differences between the actual and reconstructed epochs, with the constraint that the two processes must have the same number of points. For the Poisson point process, Gallager [29] obtained a lower bound on the rate-distortion function by insisting on the causal reconstruction of the points but allowing for their reorder. Bedekar [30] determined the rate-distortion function with the additional constraint of exact orders of epochs in reconstruction. Verdú [31] allowed the reconstruction to be non-causal. Coleman et al. [32] introduced the queueing distortion function, where the reproduced epochs lead the actual epochs. Rubin [33] used the $L_{1}$ distance between the counting functions as a distortion measure. In a more general setting, Koliander et al. [34] gave upper and lower bounds on the rate-distortion function under a more generic distortion defined between pair of point processes.

Most relevant to the present paper, Lapidoth et al. [35] introduced a covering distortion measure, where the reconstruction of a point process on $[0,T]$ is a subset $Y$ of $[0,T]$ that must contain all the points, and the distortion is the Lebesgue measure of the covering set (see also Shen et al. [36]).

If we encode the subset $Y$ as an indicator function

then $Y_{t}=0$ guarantees that no point occurred at time $t$, while $Y_{t}=1$ indicates that a point may occur at $t$. More generally, $Y_{t}$ could encode the relative belief that there is a point at $t$. Inspired by this observation, and the notion of logarithmic-loss distortion [37, 38], we consider the following formulation. For a realization of a counting (or point) process $y_{0}^{T}=(y_{t}:t\in[0,T])$ (i.e., $y_{t}$ is integer-valued, non-decreasing, and has unit jumps) and a non-negative reconstruction $\hat{y}_{0}^{T}$, we define the functional-covering distortion as

This is related to the covering distortion measure in the following sense. If we impose that $\hat{y}_{t}\in\{0,1\}$, then (1) reduces to the covering distortion measure. Yet it is natural to consider the distortion in (1) without such a restriction, or with a more general set of allowable values for $\hat{y}_{t}$. In fact, there are advantages to not restricting $\hat{y}_{0}^{T}$ to the set $\{0,1\}$. Consider a remote source setting where the encoder cannot access the point-process source directly, but instead observes a thinned version where some of the points in the source point process are deleted randomly. Then, in case of the covering distortion the reconstruction can only be the entire interval $[0,T]$ (i.e. $\hat{y}_{t}=1,t\in[0,T]$). On the other hand, under functional covering distortion the problem has a nontrivial solution.

The relation functional covering distortion measure to logarithmic-loss is as follows. If we constrain $\hat{y}_{0}^{T}$ to be bounded, then we can use a Girsanov-type transformation [39, Chapter VI, Theorems T2-T4] to define a probability measure on the set of all counting processes using $\hat{y}_{0}^{T}$, and the distortion can be defined as the expectation of the negative logarithm of the Radon-Nikodym derivative between this probability measure and an appropriately chosen reference measure, evaluated at the source realization, which is equivalent to (1). However, we will allow $\hat{y}_{0}^{T}$ to be unbounded but integrable $\mathbb{E}[\int_{0}^{T}\hat{Y}_{t}\,dt]<\infty$.

The relation to intensity theory is as follows. Heuristically, given a random variable $M$, the intensity of a point process represented by a counting function $Y_{0}^{T}$ is a non-negative process $\Gamma_{0}^{T}$ such that $P(Y_{t+\Delta}-Y_{t}=1|M,Y_{0}^{t})\approx\Gamma_{t}\Delta$ (see Definition 2 for the precise statement). From (1), we expect any optimal $\hat{Y}_{0}^{T}$ (in the rate-distortion trade-off sense) to be related to the intensity of $Y_{0}^{T}$. In fact, we will see in the proof of Theorem 4 that an optimal reconstruction $\hat{Y}_{0}^{T}$ is the intensity of $Y_{0}^{T}$ given the encoder’s output.

Beyond the introduction of the functional covering distortion measure and the accompanying coding theorems, the paper provides a collection of results for the information-theoretic analysis of point processes, which may be of independent use. One such contribution is Theorem 1, where we derive the mutual information between point-processes with intensities and arbitrary random variables. This is the most general expression available for mutual informations involving point process with intensities. Theorem 1 subsumes the existing formulae for mutual informations involving doubly stochastic Poisson processes [40, 41, 42] and queuing processes [43] as special cases. The other theorems proved in this paper are: we obtain the rate-distortion trade-off with feedforward for the functional-covering distortion measure for point processes which admit intensities (see Theorem 4). For Poisson processes, we obtain the rate-distortion region when the reconstruction function $\hat{y}_{0}^{T}$ is constrained to take value in a subset of reals (Theorem 5). The covering distortion in [35, Theorem 1] is a special case of this constrained functional-covering distortion, hence the rate-distortion function in [35] can be obtained as the special case of this theorem. We characterize the rate-distortion region for a two-encoder Poisson CEO problem (see Figure 1) under functional-covering distortion in Theorem 6. To prove the converse of the CEO problem, we derive a strong data processing inequality for Poisson processes under superposition (see Theorem 2), which complements the strong data processing inequality for Poisson processes under thinning due to Wang [44]. We also provide a self-contained proof of Wang’s theorem in Theorem 3. The solution to the CEO problem gives the rate-distortion trade-off for remote Poisson sources as an immediate corollary.

We will consider a probability space $(\Omega,\mathcal{F},P)$ on which all stochastic processes considered here are defined. For a finite $T>0$, let $(\mathcal{F}_{t}:t\in[0,T])$ be an increasing family of $\sigma$-fields with $\mathcal{F}_{T}\in\mathcal{F}$. We will assume that the given filtration $(\mathcal{F}_{t}:t\in[0,T])$, $P$, and $\mathcal{F}$ satisfy the “usual conditions”[39, Chapter III, p. 75]: $\mathcal{F}$ is complete with respect to $P$, $\mathcal{F}_{t}$ is right continuous, and $\mathcal{F}_{0}$ contains all the $P$-null sets of $\mathcal{F}_{t}$. Stochastic processes are denoted as $\hat{Y}_{0}^{T}=\{\hat{Y}_{t}:0\leq t\leq T\}$. The process $X_{0}^{T}$ is said to be adapted to the history $(\mathcal{F}_{t}:t\in[0,T])$ if $X_{t}$ is $\mathcal{F}_{t}$ measurable for all $t\in[0,T]$. The internal history recorded by the process $X_{0}^{T}$ is denoted by $\mathcal{F}^{X}_{t}=(\sigma(X_{s}):s\in[0,t])$, where $\sigma(A)$ denotes the $\sigma$-field generated by $A$.

A process $X_{0}^{T}$ is called $(\mathcal{F}_{t}:t\in[0,T])$-predictable if $X_{0}$ is $\mathcal{F}_{0}$ measurable and the mapping $(t,\omega)\to X_{t}(\omega)$ defined from $(0,T)\times\Omega$ into $\mathbb{R}$ (the set of real numbers) is measurable with respect to the $\sigma$-field over $(0,T)\times\Omega$ generated by rectangles of the form

For two measurable spaces $(\Omega_{1},\mathcal{F}_{1})$ and $(\Omega_{2},\mathcal{F}_{2})$, the product space is denoted by $(\Omega_{1}\times\Omega_{2},\mathcal{F}_{1}\otimes\mathcal{F}_{2})$. We say that $A\rightleftarrows B\rightleftarrows C$ forms a Markov chain under measure $P$ if $A$ and $C$ are conditionally independent given $B$ under $P$. $P\ll Q$ denotes that the probability measure $P$ is absolutely continuous with respect to the measure $Q$. $\textbf{1}\{\mathsf{E}\}$ denotes the indicator function for an event $\mathsf{E}$. $\log(x)$ is the natural logarithm of $x$. $(x)^{+}$ and $(x)^{-}$ denote the positive ($\max(x,0)$) and the negative part ($-\min(x,0)$) of $x$ respectively. $\lceil x\rceil$ denotes the ceiling of $x$. Throughout this paper we will adopt the convention that $0\log(0)=0$, $\exp(\log(0))=0$, and $0^{0}=1$.

We note that $\phi(x)$ is convex.

We will use the following form of Jensen’s inequality [45, Theorem 7.9, p. 149] and [45, Theorem 8.20, p. 177].

We now recall the definition of mutual information for general ensembles and its properties. Let $A$, $B$, and $C$ be measurable mappings defined on a given probability space $(\Omega,\mathcal{F},P)$, taking values in $(\mathcal{A},\mathfrak{F}^{A})$, $(\mathcal{B},\mathfrak{F}^{B})$, and $(\mathcal{C},\mathfrak{F}^{C})$ respectively. Consider partitions of $\Omega$, $\mathfrak{Q}_{A}=\left\{\mathtt{A}_{i},1\leq i\leq N_{A}\right\}\subseteq\sigma(A)$ and $\mathfrak{Q}_{B}=\left\{\mathtt{B}_{j},1\leq j\leq N_{B}\right\}\subseteq\sigma(B)$. Wyner defined the conditional mutual information $I(A;B|C)$ as [46]

where the supremum is over all such partitions of $\Omega$. Wyner showed that $I(A;B|C)\geq 0$ with equality if and only if $A\rightleftarrows C\rightleftarrows B$ forms a Markov chain [46, Lemma 3.1], and that (what is generally referred to as) Kolmogrov’s formula holds [46, Lemma 3.2]

Hence if $I(A;B)<\infty$, then $I(C;B|A)=I(A,C;B)-I(A;B)$. The data processing inequality can be obtained from (4) as well: if $A\rightleftarrows C\rightleftarrows B$ forms a Markov chain, then $I(A;B)\leq I(C;B)$.

Denote by $P^{A,B}$, the joint distribution of $A$ and $B$ on the space ($\mathcal{A}\times\mathcal{B},\mathfrak{F}^{A}\otimes\mathfrak{F}^{B}$ ), i.e.,

Similarly, $P^{A}$ and $P^{B}$ denote the marginal distributions. Gelfand and Yaglom [47] proved that if $P^{A,B}\ll P^{A}\times P^{B}$, then the mutual information $I(A;B)$ (defined via (3) by taking $\sigma(C)$ to be the trivial $\sigma$-field) can be computed as:

A sufficient condition for $P^{A,B}\ll P^{A}\times P^{B}$ is that $I(A;B)<\infty$ [48, Lemma 5.2.3, p. 92]. We will also require the following result [46, Lemma 2.1]:

and $H(M)$ is the entropy of $M$.

Let $\mathcal{N}_{0}^{T}$ denote the set of counting realizations (or point-process realizations) on $[0,T]$, i.e., if ${N}^{T}_{0}\in\mathcal{N}_{0}^{T}$, then for $t\in[0,T]$, ${N}_{t}\in\mathbf{N}$ (the set of non-negative integers), is right continuous, and has unit increasing jumps with ${N}_{0}=0$. Let $\mathfrak{F}^{N}$ be the restriction of the $\sigma$-field generated by the Skorohod topology on $D[0,1]$ to $\mathcal{N}_{0}^{T}$.

When it is clear from the context, we will drop the probability measure $P$ from the notation and say $N_{0}^{T}$ has $(\mathcal{F}_{t}:t\in[0,T])$-intensity $\Gamma_{0}^{T}$.

The above definition can be shown to imply the usual definition of Poisson process [39, Theorem T4, Chapter II, p. 25] and vice versa [39, Section 2, Chapter II, p. 23].

A point processes with stochastic intensity and a Poisson process with unit rate are linked via the following result.

The following theorem allows us to express the mutual information involving a point processes with intensity and other random variables in terms of the intensity functions. The proof of the theorem is similar to the proof of Theorem 1 in [42].

We shall require several strong data processing inequalities, for which purpose we now derive some ancillary results regarding the intensity of a point process. Combining [39, T8 Theorem, Chapter II, p. 27] and [39, T9 Theorem, Chapter II, p. 28], we can conclude the following result.

If we impose the stricter condition of finite expectation $\mathbb{E}[\int_{0}^{T}\Gamma_{t}\,dt]<\infty$, the local martingale condition in the above statement can be replaced by the martingale condition.

The following theorem was first proven by Wang in [44] using a property of certain “contraction coefficient” used in strong data processing inequalities [51]. Here, we provide a self-contained proof which uses Theorem 1 and Lemma 10.