Robust Proximity Search for Balls using Sublinear Space¹¹1Work on this paper was partially support by NSF AF awards CCF-0915984 and CCF-1217462

The nearest neighbor problem is a fundamental problem in Computer Science [17, 1]. Here, one is given a set of points $\mathsf{P}$, and given a query point $\mathtt{q}$ one needs to output the nearest point in $\mathsf{P}$ to $\mathtt{q}$. There is a trivial $O(n)$ algorithm for this problem. Typically the set of data points is fixed, while different queries keep arriving. Thus, one can use preprocessing to facilitate a faster query. There are several applications of nearest neighbor search in computer science including pattern recognition, information retrieval, vector compression, computational statistics, clustering, data mining and learning among many others, see for instance the survey by Clarkson [10] for references. If one is interested in guaranteed performance and near linear space, there is no known way to solve this problem efficiently (i.e., logarithmic query time) for dimension $d>2$, while using near linear space for the data structure.

In light of the above, major effort has been devoted to develop approximation algorithms for nearest neighbor search [6, 16, 10, 13]. In the $(1+{\varepsilon})$-approximate nearest neighbor problem, one is additionally given an approximation parameter ${\varepsilon}>0$ and one is required to find a point $\mathsf{u}\in\mathsf{P}$ such that $\mathsf{d}\!\left({\mathtt{q},\mathsf{u}}\right)\leq(1+{\varepsilon})\mathsf{d}\!\left({\mathtt{q},\mathsf{P}}\right)$. In $d$ dimensional Euclidean space, one can answer ANN queries in $O(\log n+1/{\varepsilon}^{d-1})$ time using linear space [6, 12]. Unfortunately, the constant hidden in the $O$ notation is exponential in the dimension (and this is true for all bounds mentioned in this paper), and specifically because of the $1/{\varepsilon}^{d-1}$ in the query time, this approach is only efficient in low dimensions. Interestingly, for this data structure, the approximation parameter ${\varepsilon}$ need not be specified during the construction, and one can provide it during the query. An alternative approach is to use Approximate Voronoi Diagrams (AVD), introduced by Har-Peled [11], which is a partition of space into regions of low total complexity, with a representative point for each region, that is an ANN for any point in the region. In particular, Har-Peled showed that there is such a decomposition of size $O\!\left({(n/{\varepsilon}^{d})\log^{2}n}\right)$, see also [13]. This allows ANN queries to be answered in $O(\log n)$ time. Arya and Malamatos [2] showed how to build AVDs of linear complexity (i.e., $O(n/{\varepsilon}^{d})$). Their construction uses WSPD (Well Separated Pairs Decomposition) [8]. Further trade-offs between query time and space usage for AVDs were studied by Arya et al. [4].

A more general problem is the $k$-nearest neighbors problem where one is interested in finding the $k$ points in $\mathsf{P}$ nearest to the query point $\mathtt{q}$. This is widely used in classification, where the majority label is used to label the query point. A restricted version is to find only the $k$th-nearest neighbor. This problem and its approximate version have been considered in [3, 14].

Recently, the authors [14] showed that one can compute a $(k,{\varepsilon})$-AVD that $(1+{\varepsilon})$-approximates the distance to the $k$th nearest neighbor, and surprisingly, requires $O(n/k)$ space; that is, sublinear space if $k$ is sufficiently large. For example, for the case $k=\Omega(\sqrt{n})$, which is of interest in practice, the space required is only $O\!\left({\sqrt{n}}\right)$. Such ANN is of interest when one is worried that there is noise in the data, and thus one is interested in the distance to the $k$th NN which is more robust and noise resistant. Alternatively, one can think about such data structures as enabling one to summarize the data in a way that still facilitates meaningful proximity queries.

In this paper we consider a generalization of the $k$th-nearest neighbor problem. Here, we are given a set of $n$ disjoint balls in ${\rm I\!\hskip-0.24994ptR}^{d}$ and we want to preprocess them, so that given a query point we can find approximately the $k$th closest ball. The distance of a query point to a ball is defined as the distance to its boundary if the point is outside the ball or $0$ otherwise. Clearly, this problem is a generalization of the $k$th-nearest neighbor problem by viewing points as balls of radius $0$. Algorithms for the $k$th-nearest neighbor for points, do not extend in a straightforward manner to this problem because the distance function is no longer a metric. Indeed, there can be two very far off points both very close to a single ball, and thus the triangle inequality does not hold. The problem of finding the closest ball can also be modeled as a problem of approximating the minimization diagram of a set of functions; here, a function would correspond to the distance from one of the given balls. There has been some recent work by the authors on this topic, see [15], where a fairly general class of functions admits a near-linear sized data structure permitting a logarithmic time query for the problem of approximating the minimization diagram. However, the problem that we consider in this paper does not fall under the framework of [15]. The technical assumptions of [15] mandate that the set of points which form the $0$-sublevel set of a distance function, i.e., the set of points at which the distance function is $0$ is a single point (or an empty set). This is not the case for the problem we consider here. Also, we are interested in the more general $k$th-nearest neighbor problem, while [15] only considers the nearest-neighbor problem, i.e., $k=1$.

We first show how to preprocess the set of balls into a data structure requiring space $O(n)$, in $O(n\log n)$ time, so that given a query point $\mathtt{q}$, a number $1\leq k\leq n$ and ${\varepsilon}>0$, one can compute a $(1\pm{\varepsilon})$-approximate $k$th closest ball in time $O(\log n+{\varepsilon}^{-d})$. If both $k$ and ${\varepsilon}$ are available during preprocessing, one can preprocess the balls into a $(k,{\varepsilon})$-AVD, using $O(\frac{n}{k{\varepsilon}^{d}}\log(1/{\varepsilon}))$ space, so that given a query point $\mathtt{q}$, a $(k,{\varepsilon})$-ANN closest ball can be computed, in $O(\log(n/k)+\log(1/{\varepsilon}))$ time.

In Section 2, we define the problem, list some assumptions, and introduce notations. In Section 3, we set up some basic data structures to answer approximate range counting queries for balls. In Section 4, we present the data structure, query algorithm and proof of correctness for our data structure which can compute $(1\pm{\varepsilon})$-approximate $k$th-nearest neighbors of a query point when $k,{\varepsilon}$ are only provided during query time. In Section 5 we present approximate quorum clustering, see [9, 14], for a set of disjoint balls. Using this, in Section 6, we present the $(k,{\varepsilon})$-AVD construction. We conclude in Section 7.

We are given a set of disjoint²²2Our data structure and algorithm work for the more general case where the balls are interior disjoint, where we define the interior of a “point ball”, i.e., a ball of radius $0$, as the point itself. This is not the usual topological definition. balls $\mathcal{B}=\left\{{b_{1},\dots,b_{n}}\right\}$, where $b_{i}=\mathsf{b}(\mathsf{c}_{i},\mathsf{r}_{i})$, for $i=1,\ldots,n$. Here $\mathsf{b}(\mathsf{c},\mathsf{r})\subseteq{\rm I\!\hskip-0.24994ptR}^{d}$ denotes the (closed) ball with center $\mathsf{c}$ and radius $\mathsf{r}\geq 0$. Additionally, we are given an approximation parameter ${\varepsilon}\in(0,1)$. For a point $\mathtt{q}\in{\rm I\!\hskip-0.24994ptR}^{d}$, the distance of $\mathtt{q}$ to a ball $b=\mathsf{b}(\mathsf{c},\mathsf{r})$ is $\mathsf{d}\!\left({\mathtt{q},b}\right)=\max\!\left({\rule[0.0pt]{0.0pt}{9.95863pt}\left\lVert{\mathtt{q}-\mathsf{c}}\right\rVert-\mathsf{r},\,0}\right).$

The $k$th-nearest neighbor distance of $\mathtt{q}$ to $\mathcal{B}$, denoted by $\mathsf{d}_{k}\!\left({\mathtt{q},\mathcal{B}}\right)$, is the $k$th smallest number in $\mathsf{d}\!\left({\mathtt{q},b_{1}}\right),\dots,\mathsf{d}\!\left({\mathtt{q},b_{n}}\right)$. Similarly, for a given set of points $\mathsf{P}$, $\mathsf{d}_{k}\!\left({\mathtt{q},\mathsf{P}}\right)$ denotes the $k$th-nearest neighbor distance of $\mathtt{q}$ to $\mathsf{P}$.

We aim to build a data structure to answer $(1\pm{\varepsilon})$-approximate $k$th-nearest neighbor (i.e., $(k,{\varepsilon})$-ANN) queries, where for any query point $\mathtt{q}\in{\rm I\!\hskip-0.24994ptR}^{d}$ one needs to output a ball $b\in\mathcal{B}$ such that, $(1-{\varepsilon})\mathsf{d}_{k}\!\left({\mathtt{q},\mathcal{B}}\right)\leq\mathsf{d}\!\left({\mathtt{q},b}\right)\leq(1+{\varepsilon})\mathsf{d}_{k}\!\left({\mathtt{q},\mathcal{B}}\right)$. There are different variants depending on whether ${\varepsilon}$ and $k$ are provided with the query or in advance.

We use cube to denote a set of the form $[a_{1},a_{1}+\ell]\times[a_{2},a_{2}+\ell]\times\ldots\times[a_{d},a_{d}+\ell]\subseteq{\rm I\!\hskip-0.24994ptR}^{d}$, where $a_{1},\ldots,a_{d}\in{\rm I\!\hskip-0.24994ptR}$ and $\ell\geq 0$ is the side length of the cube.

For a real positive number $x$ and a point $\mathsf{p}=(\mathsf{p}_{1},\ldots,\mathsf{p}_{d})\in{\rm I\!\hskip-0.24994ptR}^{d}$, define $\mathsf{G}_{x}(\mathsf{p})$ to be the grid point $\left({\left\lfloor{\mathsf{p}_{1}/x}\right\rfloor x,\ldots,\left\lfloor{\mathsf{p}_{d}/x}\right\rfloor x}\right)$. The number $x$ is the width or side length of the grid $\mathsf{G}_{x}$. The mapping $\mathsf{G}_{x}$ partitions ${\rm I\!\hskip-0.24994ptR}^{d}$ into cubes that are called grid cells.

Let $\mathcal{B}$ be a family of balls in ${\rm I\!\hskip-0.24994ptR}^{d}$. Given a set $X\subseteq{\rm I\!\hskip-0.24994ptR}^{d}$, let

denote the set of all balls in $\mathcal{B}$ that intersect $X$.

For two compact sets $X,Y\subseteq{\rm I\!\hskip-0.24994ptR}^{d}$, $X\preceq Y$ if and only if $\mathrm{diam}\!\left({X}\right)\leq\mathrm{diam}\!\left({Y}\right)$. For a set $X$ and a set of balls $\mathcal{B}$, let ${\mathcal{B}}_{\succeq}\!\left({X}\right)=\left\{{b\in\mathcal{B}\,\left|\,{b\cap X\neq\emptyset\text{ and }b\succeq X}\rule[-5.69046pt]{0.0pt}{11.38092pt}\right.}\right\}$. Let $\mathsf{c}_{d}$ denote the maximum number of pairwise disjoint balls of radius at least $\mathsf{r}$, that may intersect a given ball of radius $\mathsf{r}$ in ${\rm I\!\hskip-0.24994ptR}^{d}$. Clearly, we have $\left\lvert{{\mathcal{B}}_{\succeq}\!\left({b}\right)}\right\rvert\leq\mathsf{c}_{d}$ for any ball $b$. We have the following bounds,

We denote the collection of data structures above by $\mathcal{DS}_{\ref{d:s:everything}}$ and where necessary, specific functionality it provides, say for finding the large balls intersecting a cell, by $\mathcal{DS}_{\ref{d:s:everything}}$ (B).

We are given a set $\mathcal{B}$ of $n$ disjoint balls in ${\rm I\!\hskip-0.24994ptR}^{d}$, and we describe how to compute quorum clustering for them quickly.

Let $\xi$ be some constant. Let $\mathcal{B}_{0}=\emptyset$. For $i=1,\ldots,m$, let $\mathcal{R}_{i}=\mathcal{B}\setminus(\bigcup_{j=0}^{i-1}\mathcal{B}_{j})$, and let $\Lambda_{i}=\mathsf{b}(\mathsf{w}_{i},\mathsf{x}_{i})$ be any ball that satisfies,

1.   (A)

   $\Lambda_{i}$ contains $\min(k-\mathsf{c}_{d},\left\lvert{\mathcal{R}_{i}}\right\rvert)$ balls of $\mathcal{R}_{i}$ completely inside it,

2.   (B)

   $\Lambda_{i}$ intersects at least $k$ balls of $\mathcal{B}$, and

3.   (C)

   the radius of $\Lambda_{i}$ is at most $\xi$ times the radius of the smallest ball satisfying the above conditions.

Next, we remove any $k-\mathsf{c}_{d}$ balls that are contained in $\Lambda_{i}$ from $\mathcal{R}_{i}$ to get the set $\mathcal{R}_{i+1}$. We call the removed set of balls $\mathcal{B}_{i}$. We repeat this process till all balls are extracted. Notice that at each step $i$, we only require that the $\Lambda_{i}$ intersects $k$ balls of $\mathcal{B}$ (and not $\mathcal{R}_{i}$), but that it must contain $k-\mathsf{c}_{d}$ balls from $\mathcal{R}_{i}$. Also, the last quorum ball may contain fewer balls. The balls $\Lambda_{1},\ldots,\Lambda_{m}$, are the resulting $\xi$-approximate quorum clustering.