Streaming Euclidean Max-Cut: Dimension vs Data Reduction

We present an algorithm that is based on the data-reduction approach, namely, it uses the dataset $X$ to construct a small instance $X^{\prime}$ that has a similar Max-Cut value, then solve Max-Cut on it optimally and report this value $\textsc{Max-Cut}(X^{\prime})$. Following a common paradigm, $X^{\prime}$ is actually a re-weighted subset of $X$, that is picked by non-uniform sampling from $X$, known as importance sampling.

This data-reduction approach was previously used for several clustering problems. For $k$-median and related problems, such an instance $X^{\prime}$ is often called a coreset, and there are many constructions, see e.g. [HM04, FS05, FL11, BFL⁺17, HSYZ18, FSS20]. Earlier work [Sch00] has designed importance-sampling based algorithms for a problem closely related to Max-Cut, but did not provide a guarantee that $X$ and $X^{\prime}$ have a similar Max-Cut value (see Section 1.3 for a more detailed discussion). Recently, importance sampling was used to design streaming algorithms for facility location in high dimension [CJK⁺22], although their sample $X^{\prime}$ is not an instance of facility location. Overall, this prior work is not useful for us, and we have to devise our own sampling distribution, prove that it preserves the Max-Cut value, and design a streaming algorithm that samples from this distribution.

The approximation ratio $1+\varepsilon$ in Theorem 1.1 is essentially the best one can hope for using small space, because finding the Max-Cut value exactly, even in one dimension, requires $\Omega(\Delta)$ space, as shown in A.3. Compared with the dimension-reduction approach (based on [FS05]), our Theorem 1.1 has the advantage that it works for all $\ell_{p}$ norms ($p\geq 1$) and not only $\ell_{2}$. The result of [FS05] is stronger in another aspect, of providing a “cut oracle”, i.e., an implicit representation of the approximately optimal cut that can answer the side of the cut that each data point $x\in X$ (given as a query) belongs to. This feature extends also to high dimension, as it is easy to combine with the dimension reduction. For completeness, we give a (somewhat simplified) proof of the dimension reduction in Appendix B, followed by a formal statement of this “cut oracle” in Corollary B.3. It remains open to design a streaming algorithm that computes such an implicit representation using space $\operatorname{poly}(\varepsilon^{-1}d\log\Delta)$.

One indication that geometric Max-Cut admits data reduction by sampling comes from the setting of dense unweighted graphs, where it is known that Max-Cut can be $(1+\varepsilon)$-approximated using a uniform sample of $O(\varepsilon^{-4})$ vertices, namely, by taking the Max-Cut value in the induced subgraph and scaling appropriately [AdlVKK03, RV07] (improving over [GGR96]). However, sampling points uniformly clearly cannot work in the metric case – if a point set $X$ has many co-located points and a few distant points that are also far from each other, then uniform sampling from $X$ is unlikely to pick the distant points, which have a crucial contribution to the Max-Cut value. It is therefore natural to employ importance sampling, i.e., sample each point with probability proportional to its contribution. The contribution of a point $x$ to the Max-Cut value is difficult to gauge, but we can use instead a simple proxy – its total distance to all other points $q(x):=\sum_{y\in X}\operatorname{dist}(x,y)$, which is just its contribution to twice the total edge weight $\sum_{x,y\in X}\operatorname{dist}(x,y)$. For any fixed cut in $X$, this sampling works well and the estimate will (likely) have additive error $\varepsilon\sum_{x,y\in X}\operatorname{dist}(x,y)=\Theta(\varepsilon)\cdot\textsc{Max-Cut}(X)$. While the analysis is straightforward for a fixed cut, say a maximum one, proving that the sampling preserves the Max-Cut value is much more challenging, as one has to argue about all possible cuts.

We show in Theorem 3.1 that $O(\varepsilon^{-4})$ independent samples generated with probability proportional to $q(x)$ preserve the Max-Cut value. This holds even if the sampling probabilities are dampened by a factor $\lambda\geq 1$, at the cost of increasing the number of samples by a $\operatorname{poly}(\lambda)$ factor. To be more precise, it suffices to sample from any probability distribution $\{p_{x}:\ x\in X\}$, where $p(x)\geq\frac{1}{\lambda}\frac{q(x)}{\sum_{y\in X}q(y)}$ for all $x\in X$. A small technicality is that we require the sampling procedure to report a random sample $x^{*}$ together with its corresponding $p(x^{*})$, in order to re-weight the sample $x^{*}$ by factor $1/p(x^{*})$, but this extra information can often be obtained using the same methods.

This sampling distribution, i.e., probabilities proportional to $\{q(x):\ x\in X\}$, can be traced to two prior works. Schulman [Sch00] used essentially the same probabilities, but his analysis works only for one fixed cut, and extends to a multiple cuts by a union bound.²²2We gloss over slight technical differences, e.g., he deals with squared Euclidean distances, and his sampling and re-weighting processes are slightly different. The exact same probabilities were also used by [dlVK01] as weights to convert a metric instance to a dense unweighted graph, and thereby obtain a PTAS (without sampling or any data reduction).

In fact, our proof combines several observations from [dlVK01] about a uniform sample of $O(\varepsilon^{-4})$ vertices in unweighted graphs [AdlVKK03, RV07]. In a nutshell, we relate sampling from $X$ proportionally to $\{q(x):\ x\in X\}$ with sampling uniformly from a certain dense unweighted graph, whose cut values corresponds to those in $X$, and we thereby derive a bound on the Max-Cut value of the sample $X^{\prime}$.

Designing a procedure to sample proportionally to $\{q(x):\ x\in X\}$, when $X$ is presented as a stream, is much more challenging and is our main technical contribution (Lemma 4.2). The main difficulty is that standard tools for sampling from a stream, such as $\ell_{p}$-samplers [MW10, JST11, JW21], are based on the frequency vector and oblivious to the geometric structure of $X$. Indeed, the literature lacks geometric samplers, which can be very useful when designing geometric streaming algorithms. Our goal of sampling proportionally to $q(x)$, which is the total distance to all other points in $X$, seems like a fundamental geometric primitive, and therefore our sampler (Lemma 4.2) is a significant addition to the geometric-streaming toolbox, and may be of independent interest.

At a high level, our sampling procedure is based on a randomly-shifted quadtree $T$ that is defined on the entire input domain $[\Delta]^{d}$ and captures its geometry. This randomly-shifted quadtree $T$ is data oblivious, and thus can be picked (constructed implicitly) even before the stream starts (as an initialization step), using space $O(\operatorname{poly}(d\log\Delta))$. This technique was introduced by Indyk [Ind04], who noted that the quadtree essentially defines a tree embedding with expected distortion of distances $O(d\log\Delta)$. Unfortunately, the distortion is fundamental to this approach, and directly affects the accuracy (namely, goes into the approximation ratio) of streaming algorithms that use this technique [Ind04, CJLW22].³³3A randomly-shifted quadtree was used also in Arora’s approximation algorithm for TSP [Aro98], but as the basis for dynamic programming rather than as a tree embedding, and similarly in streaming applications of this technique [CJKV22]. While our approach still suffers this distortion, we can avoid its effect on the accuracy; instead, it affects the importance-sampling distribution, namely, the dampening factor $\lambda$ grows by factor $O(d\log\Delta)$, which can be compensated by drawing more samples. This increases the space complexity moderately, which we can afford, and overall leads to $(1+\varepsilon)$-approximation using space $\operatorname{poly}(\varepsilon^{-1}d\log\Delta)$.

A different importance sampling method was recently designed in [CJK⁺22] for facility location in high dimension. Their sampling procedure relies on a geometric hashing (space partitioning), and completely avoids a quadtree.⁴⁴4A key parameter in that hashing, called consistency, turns out to be $\operatorname{poly}(d)$, and unfortunately affects also the final approximation ratio for facility location, and not only the importance-sampling parameter $\lambda$ (and thus the space). Our sampling technique may be more easily applicable to other problems, as it uses a more standard and general tool of a randomly-shifted quadtree. Another importance-sampling method was recently designed in [MRWZ20] in the context of matrix streams. Geometrically, their importance-sampling can be viewed as picking a point (row in the matrix) proportionally to its length, but after the points are projected, at query time, onto a subspace. This sampling procedure is very effective for linear-algebraic problems, like column-subset selection and subspace approximation, which can be viewed also as geometric problems.

Previously, quadtrees were employed in geometric sampling results, but mostly a fixed quadtree and not a randomly-shifted one, and to perform uniform sampling over its non-empty squares at each level, as opposed to our importance sampling [FS05, FIS08, CJKV22]. Furthermore, in [FIS08], uniform sampling was augmented to report also all the points nearby the sampled points, and this was used to estimate the number of connected components in a metric threshold graph.

Finally, we discuss how to perform the importance sampling, based on a randomly-shifted quadtree $T$, but without the $O(d\log\Delta)$ distortion going into the approximation ratio. As explained above, our importance sampling (Theorem 3.1) requires that every point $x\in X$ is sampled with some probability $p(x)\geq\frac{1}{\lambda}\frac{q(x)}{Q}$ for $Q:=\sum_{y\in X}q(y)$, and the dampening factor $\lambda$ can be up to $O(\operatorname{poly}(d\log\Delta))$. We will exploit the fact that there is no upper bound on the probability $p(x)$. As usual, a randomly-shifted quadtree $T$ is obtained by recursively partitioning of the grid $[2\Delta]^{d}$, each time into $2^{d}$ equal-size grids, which we call squares, and building a tree whose nodes are all these squares, and every square is connected to its parent by an edge whose weight is equal to that square’s Euclidean diameter. Furthermore, this entire partitioning is shifted by a uniformly random $v_{\textrm{shift}}\in[\Delta]^{d}$. (See Section 4.1 for details.) The key is that every grid point $x\in[\Delta]^{d}$ is represented by a tree leaf, and thus $T$ defines distances on $[\Delta]^{d}$, also called a tree embedding. Define $q_{T}$ and $Q_{T}$ analogously to $q$ and $Q$, but using the tree distance, that is, $q_{T}(x)$ is the total tree distance from $x$ to all other points in $X$, and $Q_{T}:=\sum_{y\in X}q_{T}(y)$.

The tree-embedding guarantees, for all $x\in[\Delta]^{d}$, that $q_{T}(x)$ overestimates $q(x)$ (with probability $1$), and that $\operatorname*{\mathbb{E}}_{T}[q_{T}(x)]\leq O(d\log\Delta)\cdot q(x)$. It thus suffices to have one event, $Q_{T}\leq O(d\log\Delta)\cdot Q$, which happens with constant probability by Markov’s inequality, and then we would have $\frac{q_{T}(x)}{Q_{T}}\geq\frac{1}{O(d\log\Delta)}\cdot\frac{q(x)}{Q}$ simultaneously for all $x\in X$. As mentioned earlier, the algorithm picks (constructs implicitly) $T$ even before the stream starts, and the analysis assumes the event mentioned above happens. In fact, the rest of the sampling procedure works correctly for arbitrary $T$, i.e., regardless of that event. We stress, however, that our final algorithm actually needs to generate multiple samples that are picked independently from the same distribution, and this is easily achieved by parallel executions that share the same random tree $T$ but use independent coins in all later steps. We thus view this $T$ as a preprocessing step that is run only once, particularly when we refer to samples as independent.

The remaining challenge is to sample (in a streaming fashion) with probability proportional to $q_{T}$ for a fixed quadtree $T$. To this end, we make heavy use of the structure of the quadtree. In particular, the quadtree $T$ has $O(d\log\Delta)$ levels, and every data point $x\in X$ forms a distinct leaf in $T$. Clearly, each internal node in $T$ represents a subset of $X$ (all its descendants in $T$, that is, the points of $X$ inside its square). At each level $i$ (the root node has level $1$), we identify a level-$i$ heavy node $h_{i}$ that contains the maximum number of points from $X$ (breaking ties arbitrarily). We further identify a critical level $k$, such that $h_{1},\ldots,h_{k}$ (viewed as subsets of $X$), all contain more than $0.5|X|$ points, but $h_{k+1}$ does not. This clearly ensures that $h_{1},\ldots,h_{k}$ forms a path. Let $X(h_{i})\subseteq X$ denote the subset of $X$ represented by node $h_{i}$. The tree structure and the path property of $(h_{1},\ldots,h_{k})$ guarantee that, for each $i<k$, all points $x\in X(h_{i})\setminus X(h_{i+1})$ have roughly the same $q_{T}(x)$ values. For the boundary case $i=k$, a similar claim holds for $x\in X(h_{k})$. Hence, a natural algorithm is to employ two-level sampling: first draw a level $i^{*}\leq k$, then draw a uniform sample from $X(h_{i^{*}})\setminus X(h_{i^{*}+1})$. The distribution of sampling $i^{*}$ also requires a careful design, but we omit this from the current overview, and focus instead on how to draw a uniform sample from $X(h_{i})\setminus X(h_{i+1})$.

Unfortunately, sampling from $X(h_{i})\setminus X(h_{i+1})$ still requires $\Omega(\Delta)$ space in the streaming model, even for $d=1$. (We prove this reduction from INDEX in A.2.) In fact, it is even hard to determine whether $X(h_{i})\setminus X(h_{i+1})=\emptyset$; the difficulty is that $h_{i}$ is not known in advance, otherwise it would be easy. We therefore relax the two-level sampling, and replace sampling from $X(h_{i})\setminus X(h_{i+1})$, with its superset $X\setminus X(h_{i+1})$. This certainly biases the sampling probability, in fact some probabilities might change by an unbounded factor (Remark 4.12), nevertheless we prove that the increase in the dampening parameter $\lambda$ is bounded by an $O(\log\Delta)$ factor (Lemma 4.13), which we can still afford. This part crucially uses the tree and the path property of $(h_{1},\ldots,h_{k})$.

The final remaining step is to sample from $X\setminus X(h_{i})$ for a given $i\leq k$ (Lemma 4.14). We can assume here $X(h_{i})$ contains more than $0.5|X|$ points, and thus $X\setminus X(h_{i})$ is indeed the “light” part, containing few (and possibly no) points. To let the light points “stand out” in a sampling, we hash the tree nodes (instead of the points) randomly into two buckets. Since $|X(h(i))|>0.5|X|$, the heavy node $h_{i}$ always lies in the bucket that contains more points, and we therefore sample only from the light bucket. (To implement this in streaming, we actually generate two samples, one from each bucket, and in parallel estimate the two buckets’ sizes to know which of the two samples to use.) Typically, the light bucket contains at least half of the points from $X\setminus X(h_{i})$, which is enough. Overall, this yields a sampling procedure that uses space $\operatorname{poly}(\varepsilon^{-1}d\log\Delta)$.

Since we focus on approximate solutions, we can assume that $p_{x}$’s ($x\in X$) are of finite precision. Then, let $N$ be a sufficiently large number such that for all $x\in X$, $Np_{x}$ is an integer. We define an auxiliary graph $G^{\prime}(X^{\prime},E^{\prime}:=X^{\prime}\times X^{\prime})$, such that $X^{\prime}$ is formed by copying each point $x\in X$ for $Np_{x}$ times, and edge weight $\mathrm{len}_{G^{\prime}}(x,y):=\frac{1}{4\lambda^{2}Q}\cdot\frac{\operatorname{dist}(x,y)}{\hat{p}_{x}\hat{p}_{y}}$. Clearly, if we let $x^{*}$ be a uniform sample from $X^{\prime}$, then for every $x\in X$, $\Pr[x^{*}=x]=p_{x}$. Hence, this uniform sample $x^{*}$ is identically distributed as an importance-sample from distribution $\mathcal{D}$ on $X$. Furthermore, for $x,y\in S$, it holds that

Hence, we conclude the following fact.

Therefore, it suffices to show the $S^{\prime}$ from 3.3 satisfies that $4\lambda^{2}Q\cdot\frac{\textsc{Max-Cut}(S^{\prime})}{|S^{\prime}|^{2}}$ is a $(1+\varepsilon)$-approximation to $\textsc{Max-Cut}(X)$, with constant probability. Our plan is to apply Lemma 3.2, but we first need to show, in Lemma 3.4, that the edge weights of $G^{\prime}$ are in $[0,1]$, and in Lemma 3.6 that $\textsc{Max-Cut}(X^{\prime})$ is a $(1+\varepsilon)$-approximation to $\textsc{Max-Cut}(X)$ up to a scaling.

Now, we are ready to apply Lemma 3.2. Let $S^{\prime}\subseteq X^{\prime}$ such that $|S^{\prime}|=O(\varepsilon^{-4})$ be the resultant set by applying Lemma 3.2 with $G=G^{\prime}$ (recalling that the promise of $[0,1]$ edge weights is proved in Lemma 3.4). Then

Applying Lemma 3.6, and observe that $|X^{\prime}|=N$, the above equivalents to

where the last inequality follows from $\textsc{Max-Cut}(X)\geq\Omega(Q)$. We finish the proof by rescaling $\varepsilon$.

Assume without loss of generality that $\Delta\geq 1$ is an integral power of $2$, and let $L:=1+d\log\Delta$. Let $\{\mathcal{G}_{i}\}_{i=0}^{L}$ be a recursive partitioning of the grid $[2\Delta]^{d}$ into squares,⁷⁷7Strictly speaking, these squares are actually hypercubes (sometimes called cells or grids), but we call them squares for intuition. as follows. Start with $\mathcal{G}_{0}$ being a trivial partitioning that has one part corresponding to the entire grid $[2\Delta]^{d}$, and for each $i\geq 0$, subdivide every square in $\mathcal{G}_{i}$ into $2^{d}$ squares of half the side-length, to obtain a partition $\mathcal{G}_{i+1}$ of the entire grid $[2\Delta]^{d}$. Thus, every $\mathcal{G}_{i}$ is a partition into squares of side-length $2^{i}$. The recursive partitioning $\{\mathcal{G}_{i}\}_{i}$ naturally defines a rooted tree $T$, whose nodes are the squares inside all the $\mathcal{G}_{i}$’s, that if often called a quadtree decomposition (even though every tree node has $2^{d}$ children rather than $4$). Finally, make the quadtree $T$ random by shifting the entire recursive partitioning by a vector $-v_{\textrm{shift}}$, where $v_{\textrm{shift}}$ is chosen uniformly at random from $[\Delta]^{d}$. (This is equivalent to shifting the dataset $[\Delta]^{d}$ by $v_{\textrm{shift}}$, which explains why we defined the recursive partitioning over an extended grid $[2\Delta]^{d}$.) Every node in $T$ has a level (or equivalently, depth), where the root is at level $1$, and the level of every other node is one bigger than that of its parent node. The scale of a tree node is the side-length of the corresponding square. Observe that leaves of $T$ have scale $2^{0}$ and thus correspond to squares that contain a single grid point; moreover, points $x\in[\Delta]^{d}$ correspond to distinct leaves in $T$. Define the weight of an edge in $T$ between a node $u$ at scale $2^{i}$ and its parent as $d^{\frac{1}{p}}\cdot 2^{i}$ (i.e., the diameter of $u$’s square). Define a tree embedding of $[\Delta]^{d}$ by mapping every point $x\in[\Delta]^{d}$ to its corresponding leaf in $T$, and let the tree distance between two points $x,y\in[\Delta]^{d}$, denoted $\operatorname{dist}_{T}(x,y)$, be the distance in $T$ between their corresponding leaves. The following lemma bounds the distortion of this randomized tree embedding. We remark that a better distortion of $O(d^{\max\{\frac{1}{p},1-\frac{1}{p}\}}\log\Delta)$ may be obtained via a different technique that is less suitable for streaming [CCG⁺98].

We emphasize that in our definition of the quadtree $T$ is non-standard as it contains the entire grid $[\Delta]^{d}$ as leaves (the standard approach is to recursively partition only squares that contain at least one point from the dataset $X$). The advantage of our approach is that the tree is defined obliviously of the dataset $X$ (e.g., of updates to $X$). In particular, the leaf-to-root path from a point $x\in[\Delta]^{d}$ is well-defined regardless of $X$ and can be computed on-the-fly (without constructing the entire tree $T$) using time and space $\operatorname{poly}(d\log\Delta)$, providing sufficient information for evaluating the tree distance.

Our streaming algorithm samples such a tree $T$ as an initialization step, i.e., before the stream starts, which requires small space because it can be done implicitly by picking $\operatorname{poly}(d\log\Delta)$ random bits that describe the random shift vector $v$. Next, we show in Lemma 4.4 that this initialization step succeeds with $0.99$ probability, and on success, every distance $\operatorname{dist}(x,y)$ for $x,y\in X$ is well-approximated by its corresponding $\operatorname{dist}_{T}(x,y)$. In this case, the sampling of points $x$ with probability proportional to $q(x)$ can be replaced by sampling with probabilities that are derived from the tree metric. More specifically, the probability of sampling each $x\in X$ deviates from the desired probability $\frac{q(x)}{Q}$ by at most a factor of $\operatorname{poly}(d\log\Delta)$. We remark that the event of success does depend on the input $X$, but the algorithm does not need to know whether the initialization succeeded.

In the remainder of the proof, we assume that the random tree $T$ was already picked and condition on its success as formulated in Lemma 4.4. This lemma shows that it actually suffices to sample each $x$ with probability proportional to $q_{T}(x)$. Next, we provide in 4.5 a different formula for $q_{T}(x)$ that is based on $x$’s ancestors in the tree $T$, namely, on counting how many data points (i.e., from $X$) are contained in the squares that correspond to these ancestors. To this end, we need to set up some basic notation regarding $X$ and $T$.

Let $n:=|X|$ be the number of input points at the end of the stream. For a tree node $v\in T$, let $X(v)\subseteq X$ be the set of points from $X$ that are contained in the square corresponding to $v$. For $x\in X$ and $i\geq 1$, let $\operatorname{anc}_{i}(x)$ be the level-$i$ ancestor of $x$ in $T$ (recalling that $x$ corresponds to a leaf). By definition, $\operatorname{anc}_{L+1}(x):=x$. For $0\leq i\leq L$, let $\beta_{i}:=d^{\frac{1}{p}}\cdot 2^{L+1-i}$, which is the edge-length between a level-$i$ node $u$ and its parent (since the scale of a level-$i$ node is $2^{L+1-i}$). Due to the tree structure, we have the following representation of $q_{T}(x)$.

For each level $i$, let $h_{i}$ be a level-$i$ node whose corresponding square contains the most points from $X$, breaking ties arbitrarily. Next, we wish to identify a critical level $k$; ideally, this is the last level going down from the root, i.e., largest $i$, such that $|X(h_{i})|\geq 0.6n$ (the constant $0.6$ is somewhat arbitrary). However, it is difficult to find this $k$ exactly in a streaming algorithm, and thus we use instead a level $\tilde{k}$ that satisfies a relaxed guarantee that only requires estimates on different $|X(h_{i})|$, as follows. Let us fix henceforth two constants $0.5<\sigma^{-}\leq\sigma^{+}\leq 1$.

Suppose henceforth that $\tilde{k}$ is a $(\sigma^{-},\sigma^{+})$-critical level. (Such a critical level clearly exists, although its value need not be unique.) Since $|X(h_{i})|\geq|X(h_{i+1})|$ for every $i<\tilde{k}$ (because $h_{i}$ contains the most points from $X$ at level $i$), we know that $|X(h_{i})|\geq\sigma^{-}n$ for every $i\leq\tilde{k}$ (not only for $i=\tilde{k}$), and $|X(h_{i})|\leq\sigma^{+}n$ for every $i>\tilde{k}$.