Clustering What Matters in Constrained Settings

We give a general definition of a constrained clustering problem. For a positive integer $k$, we shall use $[k]$ to denote the set $\{1,\ldots,k\}$. Let $(\mathcal{X},D)$ denote the metric space with distance function $D$. For a point $x$ and a subset $S$ of points, we shall use $D(x,S)$ to denote $\min_{y\in S}D(x,y)$.

The set $\mathcal{X}$ contains subsets $F$ and $X$: here $X$ denotes the set of input points and $F$ the set of points where a center can be located. An outlier constrained clustering problem is specified by the following parameters and functions:

• •

  $k$: the number of clusters.

• •

  $m$: the number of points which can be left out from the clusters.

• •

  a function check: given a partitioning $X_{0},X_{1},\ldots,X_{k}$ of $X$ (here $X_{0}$ is the set of outliers) and centers $f_{1},\ldots,f_{k}$, each lying in the set $F$, the function ${\textsf{check}}(X_{1},\ldots,X_{k},f_{1},\ldots,f_{k})$ outputs 1 iff this is a feasible clustering. For example, in the $r$-gathering problem, the ${\textsf{check}}(X_{0},X_{1},\ldots,X_{k},f_{1},\ldots,f_{k})$ outputs 1 iff $|X_{i}|\geq r$ for each $i\in[k]$. The check function depends only on the cardinality of the sets $X_{1},\ldots,X_{k}$ and the locations $f_{1},\ldots,f_{k}$. This already captures many of the constrained clustering problems. Our framework also applies to the more general labelled version (see details below).

• •

  a cost function cost: given a partitioning $X_{0},X_{1},\ldots,X_{k}$ of $X$ and centers $f_{1},\ldots,f_{k}$,

  $${\textsf{cost}}(X_{1},\ldots,X_{k},f_{1},\ldots,f_{k}):=\sum_{i\in[k]}\sum_{x\in X_{i}}D^{z}(x,f_{i}),$$

  where $z$ is either 1 (the outlier constrained $k$-median problem) or 2 (the outlier constrained $k$-means problem).

Given an instance ${\mathcal{I}}=(X,F,k,m,{\textsf{check}},{\textsf{cost}})$ of an outlier constrained clustering problem as above, the goal is to find a partitioning $X_{0},X_{1},\ldots,X_{k}$ of $X$ and centers $f_{1},\ldots,f_{k}\in F$ such that $|X_{0}|\leq m$,
${\textsf{check}}(X_{1},\ldots,X_{k},f_{1},\ldots,f_{k})$ is 1 and ${\textsf{cost}}(X_{0},X_{1},\ldots,X_{k},f_{1},\ldots,f_{k})$ is minimized. The outlier-free constrained clustering problem is specified as above, except that the parameter $m$ is 0. For sake of brevity, we leave out the parameter $m$ and the set $X_{0}$ while defining the instance ${\mathcal{I}}$, and functions check and cost.

We shall also consider a more general class of constrained clustering problems, where each input point is assigned a label. In other words, an instance ${\mathcal{I}}$ of such a problem is specified by a tuple $(X,F,k,m,\sigma,{\textsf{check}},{\textsf{cost}})$, where $\sigma:X\rightarrow L$ for a finite set $L$. Note that the check function may depend on the function $\sigma$. For example, $\sigma$ could assign a label “red” or “blue” to each point in $X$ and the check function would require that each cluster $X_{i}$ should have an equal number of red and blue points. In addition to the locations $f_{1},\ldots,f_{k}$, the ${\textsf{check}}(X_{1},\ldots,X_{k},f_{1},\ldots,f_{k},\sigma)$ function also depends on $|\sigma^{-1}(l)\cap X_{j}|$ for each $l\in L,j\in[k]$, i.e., the number of points with a particular label in each of the clusters. Indirectly, this also implies that the check function can impose conditions on the labels of the outliers points. For example, the colourful $k$-median problem discussed in [AISX23] has the constraint that $m_{i}$ clients from the label type $i$ should be designated as outliers, given that every client has a unique label.  Table 1 gives a description of some of these problems.

We shall use the approximate triangle inequality, which states that for $z\in\{1,2\}$ and any three points $x_{1},x_{2},x_{3}\in\mathcal{X}$,

Our main result reduces the outlier constrained clustering problem to the outlier-free version. In our reduction, we shall also use approximation algorithms for the (unconstrained) $k$-median and $k$-means problems. We assume we have a constant factor approximation algorithm for these problems¹¹1Several such constant factor approximation algorithms exist [ANSW17, AGK⁺04, CGETS02].: let $\mathcal{C}$ denote such an algorithm with running time $T_{\mathcal{C}}(n)$ on an input of size $n$. Note that $\mathcal{C}$ would be an algorithm for the $k$-means or the $k$-median problem depending on whether $z=1$ or $2$ in the definition of the cost function.

The above theorem implies that as long as there is an FPT or polynomial-time approximation algorithm for the constrained, outlier-free $k$-median or $k$-means clustering problem, there is an FPT approximation algorithm (with almost the same approximation ratio) for the corresponding outlier version. We prove this result by creating $q$ instances of the outlier-free version of ${\mathcal{I}}$ and picking the best solution on these instances using the algorithm $\mathcal{A}$. We also extend the above result to the labelled version:

The consequences of our results for specific constrained clustering problems is summarized in Table 2. We give the results of related works [BGJK20, GJK20, AISX23] in the same table to see the contributions of this work. Our contributions can be divided into two main categories:

1. 1.

   Matching the best-known result: This can be further divided into two categories:

   1. (a)

      Matching results of [AISX23]: [AISX23] gives an outlier to outlier-free reduction. We also give such a reduction using a different technique with slightly better parameters. This means that we match all the results of [AISX23], which includes problems such as the classical $k$-median/means problems, the Matroid $k$-median problem, the colorful $k$-median problem, and $k$-median in certain special metrics. See rows 2-6 in Table 2.

   2. (b)

      Matching results of [GJK20]: [GJK20] gives FPT approximation algorithms for certain constrained problems on which the coreset-based approach of [AISX23] is not known to work. See the last row of Table 2. [GJK20] gives algorithms for outlier and outlier-free versions with the same approximation guarantee. Since the best outlier-free approximation is also from [GJK20], our results currently only match the approximation guarantees of [GJK20]. However, if there is an improvement in any of these problems, our results will immediately beat the known outlier results of [GJK20].

2. 2.

   Best known results: Since our results hold for a larger class of constrained problems than earlier works, there are certain problems for which our results give the best-known FPT approximation algorithm. The list includes capacitated $k$-median/$k$-means with hard capacities in general metric and Euclidean spaces. It also includes the $k$-median problem in the Ulam metric. A recent development in the Ulam $k$-median problem [CDK23] has broken the $2$-approximation barrier. Our reduction allows us to take this development to the outlier setting as well. The outlier-free results from which our best results are derived using our reduction are given in Table 2 (see rows 7-9).

As discussed earlier, the idea of a reduction from a outlier clustering problem to the corresponding outlier-free version in the context of the Euclidean $k$-means problem was suggested by [BGJK20] using a $D^{2}$-sampling based idea. [GJK20] used the sampling ideas to design approximation algorithms for the outlier versions of various constrained clustering problems. However, the approximation guarantee obtained by [GJK20] was limited to $(3+\varepsilon)$ for a large class of constrained $k$-median and $(9+\varepsilon)$ for the constrained $k$-means problems, and it was not clear how to extend these techniques to get improved guarantees. As a result, their techniques could not exploit the recent developments by [CAGK⁺19] in the design of $(1+2/e+\varepsilon)$ and $(1+8/e+\varepsilon)$ FPT approximation algorithms for the classical outlier-free $k$-median and $k$-means problems respectively in general metric spaces. [AISX23] gave an outlier-to-outlier-free reduction, making it possible to extend the above-mentioned FPT approximation guarantees for the outlier-free setting to the outlier setting.

The reduction of [AISX23] is based on the coreset construction by [Che09] using uniform sampling. A coreset for a dataset is a weighted set of points such that the clustering of the coreset points with respect to any set of $k$ centers is the same (within a $1\pm\varepsilon$ factor) as that of the original set points. The coreset construction in [Che09] starts with a set $C$ of centers that give constant factor approximation. They consider $O(\log{n})$ “rings” around these centers, uniformly sample points from each of these rings, and set the weight of the sampled points appropriately. The number of sampled points, and hence the size of the coreset, is $\left(\frac{|C|\log{n}}{\varepsilon}\right)^{2}$. [AISX23] showed that when starting with $(k+m)$ centers that give a constant approximation to the classical $(k+m)$-median problem, the coreset obtained as above has the following additional property: for any set of $k$ centers, the clustering cost of the original set of points excluding $m$ outliers is same (again, within $1\pm\varepsilon$ factor) as that of the coreset, again allowing for exclusion of a subset of $m$ points from it. This means that by trying out all $m$ subsets from the coreset, we ensure that at least one subset acts as a good outlier set. Since the coreset size is $\left(\frac{(k+m)\log{n}}{\varepsilon}\right)^{2}$, the number of outlier-free instances that we construct is $\left(\frac{(k+m)\log{n}}{\varepsilon}\right)^{O(m)}$. Using $(\log{n})^{O(m)}=\max\{m^{O(m)},n^{O(1)}\}$, this is of the form $f(k,m,\varepsilon)\cdot n^{O(1)}$ for a suitable function $f$. At this point, we note the first quantitative difference from our result. In our algorithm, we save the $(\log{n})^{O(m)}$ factor, which also means that the number of instances does not depend on the problem size $n$. Further, a coreset based construction restricts the kind of problems it can be applied to. The coreset property that the cost of original points is the same as that of the weighted cost of coreset points holds when points are assigned to the closest center (i.e., the entire weight of the coreset goes to the closest center).²²2The reason is how Haussler’s lemma is applied to bound the cost difference. This works for the classical unconstrained $k$-median and $k$-means problems (as well as the few other settings considered in [AISX23]). However, for several constrained clustering problems, it may not hold that every point is assigned to the closest center. There have been some recent developments [BFS21, BCAJ⁺22] in designing coresets for constrained clustering settings. However, they have not been shown to apply to the outlier setting. Another recent work [HJLW22] designs coresets for the outlier setting, but like [AISX23], it has limited scope and has not been shown to extend for most constrained settings. Our $D^{z}$-sampling-based technique has the advantage that instead of running the outlier-free algorithm on a coreset as in [AISX23], it works directly with the dataset. That is, we run the outlier-free algorithm on the dataset (after removing outlier candidates). This also makes our results helpful in weighted settings (e.g., see [CDK23]) where the outlier-free algorithm is known to work only for unweighted datasets – note that a coreset is a weighted set).

In this section, we give a high-level description of our algorithm. Let ${\mathcal{I}}$ denote an instance of outlier constrained clustering on a set of points $X$ and ${\cal O}$ denote an optimal solution to ${\mathcal{I}}$. The first observation is that the optimal cost of the outlier-free and unconstrained clustering with $k+m$ centers on $X$ is a lower bound on the cost of ${\cal O}$ (Claim 1). ³³3This observation was used in both [BGJK20] and [AISX23]. Let $C$ denote the set of these $(k+m)$ centers (we can use any constant factor approximation for the unconstrained version to find $C$). The intuition behind choosing $C$ is that the centers in ${\cal O}$ should be close to $C$.

Now we divide the set of $m$ outliers in ${\cal O}$ into two subsets: those which are far from $C$ and the remaining ones close to $C$ (“near” outliers). Our first idea is to randomly sample a subset $S$ of $O(m\log m)$ points from $X$ with sampling probability proportional to distance (or square of distance) from the set $C$. This sampling ensures that $S$ contains the far outliers with high probability (Claim 2). We can then cycle through all subsets of $S$ to guess the exact subset of far outliers. Handling the near outliers is more challenging and forms the heart of the technical contribution of this paper.

We “assign” each near outlier to its closest point in $C$ – let $X^{{\textsf{opt}}}_{N,j}$ be the set of outliers assigned to $c_{j}$. By cycling over all choices, we can guess the cardinality $t_{j}$ of each of the sets $X^{{\textsf{opt}}}_{N,j}$. We now set up a suitable minimum cost bipartite $b$-matching instance which assigns a set of $t_{j}$ points to each center $c_{j}$. Let ${\widehat{X}}_{j}$ be the set of points assigned to $c_{j}$. Our algorithm uses $\cup_{j}{\widehat{X}}_{j}$ as the set of near outliers. In the analysis, we need to argue that there is a way of matching the points in $X^{{\textsf{opt}}}_{N,j}$ to ${\widehat{X}}_{j}$ whose total cost (sum of distances or squared distances between matched points) is small (Lemma 1). The hope is that we can go from the optimal set of outliers in ${\cal O}$ to the ones in the algorithm and argue that the increase in cost is small. Since we are dealing with constrained clustering, we need to ensure that this process does not change the size of each of the clusters. To achieve this, we need to further modify the matching between the two sets of outliers (Lemma 2). Finally, with this modified matching, we are able to argue that the cost of the solution produced by the algorithm is close to that of the optimal solution. The extension to the labelled version follows along similar lines.

In the remaining paper, we prove our two main results, Theorem 1.1 and Theorem 1.2. The main discussion will be for Theorem 1.1 since Theorem 1.2 is an extension of Theorem 1.1 that uses the same proof ideas. In the following sections, we give the details of our algorithm (Section 2) and its analysis (Section 3). In Section 3.1, we discuss the extension to the labelled version.