Deterministic Approximation for Submodular Maximization over a Matroid in Nearly Linear Time

Submodular function maximization has aroused great interests from both academic and industrial societies due to its wide applications such as crowdsourcing [47], information gathering [35], sensor placement [33], influence maximization [37, 48] and exemplar-based clustering [30]. Due to the large volume of data and heterogeneous application scenarios in practice, there is a growing demand for designing accurate and efficient submodular maximization algorithms subject to various constraints.

Matroid is an important structure in combinatorial optimization that abstracts and generalizes the notion of linear independence in vector spaces [8]. The problem of submodular maximization subject to a matroid constraint (SMM) has attracted considerable attention since the 1970s. When the considered submodular function $f(\cdot)$ is monotone, the classical work of Fisher et al. [27] presents a deterministic approximation ratio of $1/2$, which keeps as the best deterministic ratio during decades until Buchbinder et al. [14] improve this ratio to $1/2+\epsilon$ very recently.

When the submodular function $f(\cdot)$ is non-monotone, the best-known deterministic ratio for the SMM problem is ${1}/{4}-\epsilon$, proposed by Lee et al. [38], but with a high time complexity of $\mathcal{O}((n^{4}\log n)/\epsilon)$. Recently, there appear studies aiming at designing more efficient and practical algorithms for this problem. In this line of studies, the elegant work of Mirzasoleiman et al. [44] and Feldman et al. [25] proposes the best deterministic ratio of $1/6-\epsilon$ and the fastest implementation with an expected ratio of $1/4$, and their algorithms can also handle more general constraints such as a $p$-set system constraint. For clarity, we list the performance bounds of these work in Table 1. However, it is still unclear whether the ${1}/{4}-\epsilon$ deterministic ratio for a single matroid constraint can be further improved, or whether there exist faster algorithms achieving the same ${1}/{4}-\epsilon$ deterministic ratio.

In this paper, we propose an approximation algorithm TwinGreedy (Alg. 1) with a deterministic ${1}/{4}$ ratio and $\mathcal{O}(nr)$ running time for maximizing a non-monotone, non-negative submodular function subject to a matroid constraint, thus improving the best-known $1/4-\epsilon$ deterministic ratio of Lee et al. [38]. Furthermore, we show that the solution framework of TwinGreedy can be implemented in a more efficient way, and present a new algorithm dubbed TwinGreedyFast with $1/4-\epsilon$ deterministic ratio and nearly-linear $\mathcal{O}(\frac{n}{\epsilon}\log\frac{r}{\epsilon})$ running time. To the best of our knowledge, TwinGreedyFast is the fastest algorithm achieving the $\frac{1}{4}-\epsilon$ deterministic ratio for our problem in the literature. As a byproduct, we also show that TwinGreedyFast can be used to address a more general $p$-set system constraint and achieves a $\frac{1}{2p+2}-\epsilon$ approximation ratio with the same time complexity.

It is noted that most of the current deterministic algorithms for non-monontone submodular maximization (e.g., [44, 25, 45]) leverage the “repeated greedy-search” framework proposed by Gupta et al. [31], where two or more candidate solution sets are constructed successively and then an unconstrained submodular maximization (USM) algorithm (e.g., [12]) is called to find a good solution among the candidate sets and their subsets. Our approach is based on a novel “simultaneous greedy-search” framework different from theirs, where two disjoint candidate solution sets $S_{1}$ and $S_{2}$ are built simultaneously with only single-pass greedy searching, without calling a USM algorithm. We call these two solution sets $S_{1}$ and $S_{2}$ as “twin sets” because they “grow up” simultaneously. Thanks to this framework, we are able to bound the “utility loss” caused by greedy searching using $S_{1}$ and $S_{2}$ themselves, through a careful classification of the elements in an optimal solution $O$ and mapping them to the elements in $S_{1}\cup S_{2}$. Furthermore, by incorporating a thresholding method inspired by Badanidiyuru and Vondrák [2] into our framework, the TwinGreedyFast algorithm achieves nearly-linear time complexity by only accepting elements whose marginal gains are no smaller than given thresholds.

We evaluate the performance of TwinGreedyFast on two applications: social network monitoring and multi-product viral marketing. The experimental results show that TwinGreedyFast runs more than an order of magnitude faster than the state-of-the-art efficient algorithms for our problem, and also achieves better solution quality than the currently fastest randomized algorithms in the literature.

Given a ground set $\mathcal{N}$ with $|\mathcal{N}|=n$, a function $f:2^{\mathcal{N}}\mapsto\mathbb{R}$ is submodular if for all $X,Y\subseteq\mathcal{N}$, $f(X)+f(Y)\geq f(X\cup Y)+f(X\cap Y)$. The function $f(\cdot)$ is called non-negative if $f(X)\geq 0$ for all $X\subseteq\mathcal{N}$, and $f(\cdot)$ is called non-monotone if $\exists X\subset Y\subseteq\mathcal{N}:f(X)>f(Y)$. For brevity, we use $f(X\mid Y)$ to denote $f(X\cup Y)-f(Y)$ for any $X,Y\subseteq\mathcal{N}$, and write $f(\{x\}\mid Y)$ as $f(x\mid Y)$ for any $x\in\mathcal{N}$. We call $f(X\mid Y)$ as the “marginal gain” of $X$ with respect to $Y$.

An independence system $(\mathcal{N},\mathcal{I})$ consists of a finite ground set $\mathcal{N}$ and a family of independent sets $\mathcal{I}\subseteq 2^{\mathcal{N}}$ satisfying: (1) $\emptyset\in\mathcal{I}$; (2) If $A\subseteq B\in\mathcal{I}$, then $A\in\mathcal{I}$ (called hereditary property). An independence system $(\mathcal{N},\mathcal{I})$ is called a matroid if it satisfies: for any $A\in\mathcal{I},B\in\mathcal{I}$ and $|A|<|B|$, there exists $x\in B\backslash A$ such that $A\cup\{x\}\in\mathcal{I}$ (called exchange property).

Given an independence system $(\mathcal{N},\mathcal{I})$ and any $X\subseteq Y\subseteq\mathcal{N}$, $X$ is called a base of $Y$ if: (1) $X\in\mathcal{I}$; (2) $\forall x\in Y\backslash X:X\cup\{x\}\notin\mathcal{I}$. The independence system $(\mathcal{N},\mathcal{I})$ is called a $p$-set system if for every $Y\subseteq\mathcal{N}$ and any two basis $X_{1},X_{2}$ of $Y$, we have $|X_{1}|\leq p|X_{2}|$ ($p\geq 1$). It is known that $p$-set system is a generalization of several structures on independence systems including matroid, $p$-matchoid and $p$-extendible system, and an inclusion hierarchy of these structures can be found in [32].

In this paper, we consider a non-monotone, non-negative submodular function $f(\cdot)$. Given $f(\cdot)$ and a matroid $(\mathcal{N},\mathcal{I})$, our optimization problem is $\max\{f(S):S\in\mathcal{I}\}$. In the end of Section 4, we will also consider a more general case where $(\mathcal{N},\mathcal{I})$ is a $p$-set system.

We introduce some frequently used properties of submodular functions. For any $X\subseteq Y\subseteq\mathcal{N}$ and any $Z\subseteq\mathcal{N}\backslash Y$, we have $f(Z\mid Y)\leq f(Z\mid X)$; this property can be derived by the definition of submodular functions. For any $X,Y\subseteq\mathcal{N}$ and a partition $Z_{1},Z_{2},\cdots,Z_{t}$ of $Y\backslash X$, we have

For convenience, we use $[h]$ to denote $\{1,\cdots,h\}$ for any positive integer $h$, and use $r$ to denote the rank of $(\mathcal{N},\mathcal{I})$, i.e., $r=\max\{|S|:S\in\mathcal{I}\}$, and denote an optimal solution to our problem by $O$.

In this section, we consider a matroid constraint and introduce the TwinGreedy algorithm (Alg. 1) that achieves $1/4$ approximation ratio. The TwinGreedy algorithm maintains two solution sets $S_{1}$ and $S_{2}$ that are initialized to empty sets. At each iteration, it considers all the candidate elements in $\mathcal{N}\backslash(S_{1}\cup S_{2})$ that can be added into $S_{1}$ or $S_{2}$ without violating the feasibility of $\mathcal{I}$. If there exists such an element, then it greedily selects $(e,S_{i})$ where $i\in\{1,2\}$ such that adding $e$ into $S_{i}$ can bring the maximal marginal gain $f(e\mid S_{i})$ without violating the feasibility of $\mathcal{I}$. TwinGreedy terminates when no more elements can be added into $S_{1}$ or $S_{2}$ with a positive marginal gain while still keeping the feasibility of $\mathcal{I}$. Then it returns the one between $S_{1}$ and $S_{2}$ with the larger objective function value.

Although the TwinGreedy algorithm is simple, its performance analysis is highly non-trivial. Roughly speaking, as TwinGreedy adopts a greedy strategy to select elements, we try to find some “competitive relationships” between the elements in $O$ and those in $S_{1}\cup S_{2}$, such that the total marginal gains of the elements in $O$ with respect to $S_{1}$ and $S_{2}$ can be upper-bounded by $S_{1}$ and $S_{2}$’s own objective function values. However, this is non-trivial due to the correlation between the elements in $S_{1}$ and $S_{2}$. To overcome this hurdle, we first classify the elements in $O$, as shown in Definition 1:

Intuitively, each element $e\in O_{1}^{+}\subseteq S_{1}$ can also be added into $S_{2}$ without violating the feasibility of $\mathcal{I}$ when $e$ is added into $S_{1}$, while the elements in $O_{1}^{-}\subseteq S_{1}$ do not have this nice property. The sets $O_{2}^{+}$ and $O_{2}^{-}$ can be understood similarly. With the above classification of the elements in $O$, we further consider two groups of elements: $O_{1}^{+}\cup O_{1}^{-}\cup O_{2}^{-}\cup O_{3}$ and $O_{1}^{-}\cup O_{2}^{+}\cup O_{2}^{-}\cup O_{4}$. By leveraging the properties of independence systems, we can map the first group to $S_{1}$ and the second group to $S_{2}$, as shown by Lemma 1 (the proof can be found in the supplementary file). Intuitively, Lemma 1 holds due to the exchange property of matroids.

The first property shown in Lemma 1 implies that, at the moment that $\pi_{1}(e)$ is added into $S_{1}$, $e$ can also be added into $S_{1}$ without violating the feasibility of $\mathcal{I}$ (for any $e\in O_{1}^{+}\cup O_{1}^{-}\cup O_{2}^{-}\cup O_{3}$). This makes it possible to compare the marginal gain of $e$ with respect to $S_{1}$ with that of $\pi_{1}(e)$. The construction of $\pi_{2}(\cdot)$ is also based on this intuition. With the two injections $\pi_{1}(\cdot)$ and $\pi_{2}(\cdot)$, we can bound the marginal gains of $O_{1}^{+}$ to $O_{4}$ with respect to $S_{1}$ and $S_{2}$, as shown by the Lemma 2. Lemma 2 can be proved by using Definition 1, Lemma 1 and the submodularity of $f(\cdot)$.

The proof of Lemma 2 is deferred to the supplementary file. In the next section, we will provide a proof sketch for a similar lemma (i.e., Lemma 3), which can also be used to understand Lemma 2. Now we can prove the performance bounds of TwinGreedy:

Meanwhile, as $f(\cdot)$ is a non-negative submodular function and $S_{1}\cap S_{2}=\emptyset$, we have

By summing up Eqn. (6)-Eqn. (10) and simplifying, we get

which completes the proof on the $1/4$ ratio. Finally, the $\mathcal{O}(nr)$ time complexity is evident, as $|S_{1}|+|S_{2}|\leq 2r$ and adding one element into $S_{1}$ or $S_{2}$ has at most $\mathcal{O}(n)$ time complexity. $\square$

Interestingly, when the objective function $f(\cdot)$ is monotone, the proof of Theorem 1 shows that the TwinGreedy algorithm achieves a $1/2$ approximation ratio due to $f(O\cup S_{1})+f(O\cup S_{2})\geq 2f(O)$. Therefore, we rediscover the $1/2$ deterministic ratio proposed by Fisher et al. [27] for monotone submodular maximization over a matroid.

As the TwinGreedy algorithm still has quadratic running time, we present a more efficient algorithm TwinGreedyFast (Alg. 2). As that in TwinGreedy, the TwinGreedyFast algorithm also maintains two solution sets $S_{1}$ and $S_{2}$, but it uses a threshold to control the quality of elements added into $S_{1}$ or $S_{2}$. More specifically, given a threshold $\tau$, TwinGreedyFast checks every unselected element $e\in\mathcal{N}\backslash(S_{1}\cup S_{2})$ in an arbitrarily order. Then it chooses $S_{i}\in\{S_{1},S_{2}\}$ such that adding $e$ into $S_{i}$ does not violate the feasibility of $\mathcal{I}$ while $f(e\mid S_{i})$ is maximized. If the marginal gain $f(e\mid S_{i})$ is no less than $\tau$, then $e$ is added into $S_{i}$, otherwise the algorithm simply neglects $e$. The algorithm repeats the above process starting from $\tau=\tau_{max}$ and decreases $\tau$ by a factor $(1+\epsilon)$ at each iteration until $\tau$ is sufficiently small, then it returns the one between $S_{1}$ and $S_{2}$ with the larger objective value.

The performance analysis of TwinGreedyFast is similar to that of TwinGreedy. Let us consider the two solution sets $S_{1}$ and $S_{2}$ when TwinGreedyFast returns. We can define $O_{1}^{+}$, $O_{1}^{-}$, $O_{2}^{+}$, $O_{2}^{-}$, $O_{3}$, $O_{4}$, $\mathrm{Pre}(e,S_{i})$ and $\delta(e)$ in exact same way as Definition 1, and Lemma 1 still holds as the construction of $\pi_{1}(\cdot)$ and $\pi_{2}(\cdot)$ only depends on the insertion order of the elements in $S_{1}\cup S_{2}$, which is fixed when TwinGreedyFast returns. We also try to bound the marginal gains of $O_{1}^{+}$-$O_{4}$ with respect to $S_{1}$ and $S_{2}$ in a way similar to Lemma 2. However, due to the thresholds introduced in TwinGreedyFast, these bounds are slightly different from those in Lemma 2, as shown in Lemma 3:

With Lemma 3, we can use similar reasoning as that in Theorem 1 to prove the performance bounds of TwinGreedyFast, as shown in Theorem 2. The full proofs of Lemma 3 and Theorem 2 can be found in the supplementary file.

In this section, we evaluate the performance of TwinGreedyFast under two social network applications, using both synthetic and real networks. We implement the following algorithms for comparison:

1. 1.

   SampleGreedy: This algorithm is proposed in [25] which has $1/4$ expected ratio and $\mathcal{O}(nr)$ running time. To the best of our knowledge, it is currently the fastest randomized algorithm for non-monotone submodular maximization over a matroid.

2. 2.

   Fantom: Excluding the $1/4-\epsilon$ ratio proposed in [38], the Fantom algorithm from [44] has the best deterministic ratio for our problem. As it needs to call an unconstrained submodular maximization (USM) algorithm, we use the USM algorithm proposed in [12] with $1/3$ deterministic ratio and linear running time. As such, Fantom achieves $1/7$ deterministic ratio in the experiments.²²2We have also tested Fantom using the randomized USM algorithm in [12] with $1/2$ expected ratio. The experimental results are almost identical, although Fantom has a larger ${1}/{6}$ ratio (in expectation) in such a case.

3. 3.

   ResidualRandomGreedy: This algorithm is from [11] which has $1/4$ expected ratio and $\mathcal{O}(nr\log n)$ running time. We denote it as “RRG” for brevity.

4. 4.

   TwinGreedyFast: We implement our Algorithm 2 by setting $\epsilon=0.1$. As such, it achieves 0.15 deterministic approximation ratio in the experiments.

We have proposed the first deterministic algorithm to achieve an approximation ratio of $1/4$ for maximizing a non-monotone, non-negative submodular function subject to a matroid constraint, and our algorithm can also be accelerated to achieve nearly-linear running time. In contrast to the existing algorithms adopting the “repeated greedy-search” framework proposed by [31], our algorithms are designed based on a novel “simultaneous greedy-search” framework, where two candidate solutions are constructed simultaneously, and a pair of an element and a candidate solution is greedily selected at each step to maximize the marginal gain. Moreover, our algoirthms can also be directly used to handle a more general $p$-set system constraint or monotone submodular functions, while still achieving nice performance bounds. For example, by similar reasoning with that in Sections 3-4, it can be seen that our algorithms can achieve $\frac{1}{p+1}$ ratio for monotone $f(\cdot)$ under a $p$-set system constraint, which is almost best possible [2]. We have evaluated the performance of our algorithms in two concrete applications for social network monitoring and multi-product viral marketing, and the extensive experimental results demonstrate that our algorithms runs in orders of magnitude faster than the state-of-the-art algorithms, while achieving approximately the same utility.

This work was supported by the National Key R$\&$D Program of China under Grant No. 2018AAA0101204, the National Natural Science Foundation of China (NSFC) under Grant No. 61772491 and Grant No. U1709217, Anhui Initiative in Quantum Information Technologies under Grant No. AHY150300, and the Fundamental Research Funds for the Central Universities.

Submodular optimization is an important research topic in data mining, machine learning and optimization theory, as it has numerous applications such as crowdsourcing [47], viral marketing [34], feature selection [28], network monitoring [40], document summarization [41, 20], online advertising [49], crowd teaching [46] and blogosphere mining [21]. Matroid is a fundamental structure in combinatorics that captures the essence of a notion of "independence" that generalizes linear independence in vector spaces. The matroid structure has been pervasively found in various areas such as geometry, network theory, coding theory and graph theory [8]. The study on submodular maximization subject to matroid constraints dates back to the 1970s (e.g., [27]), and it is still a hot research topic today [44, 25, 45, 5, 16, 3]. A lot of practical problems can be cast as the problem of submodular maximziation over a matroid constraint (or more general $p$-set system constraints), such as diversity maximization [1], video summarization [52], clustering [42], multi-robot allocation [51] and planning sensor networks [19]. Therefore, our study has addressed a general and fundamental theoretical problem with many potential applications.

Due to the massive datasets used everywhere nowadays, it is very important that submodular optimization algorithms should achieve accuracy and efficiency simultaneously. Recently, there emerge great interests on designing more practical and efficient algorithms for submodular optimization (e.g., [2, 13, 4, 43, 36, 23, 11]), and our work advances the state of the art in this area by proposing a new efficient algorithm with improved performance bounds. Moreover, our algorithms are based on a novel “simultaneous greedy-search” framework, which is different from the classical “repeated greedy-search” and “local search” frameworks adopted by the state-of-the-art algorithms (e.g., [31, 38, 25, 44]). We believe that our “simultaneous greedy-search” framework has the potential to be extended to address other problems on submodular maximization with more complex constraints, which is the topic of our ongoing research.