A Crowding Distance That Provably Solves the Difficulties of the NSGA-II in Many-Objective Optimization

In many practical applications, the problems to be solved have several, often conflicting objectives. Since such problems often do not have a single optimal solution, one resorts to computing a set of diverse good solutions (ideally so-called Pareto optima) and lets a human decision maker take the final decision among these.

One of the most successful algorithm for computing such a set of solutions for a multi-objective optimization problem is the Non-dominated Sorting Genetic Algorithm II (NSGA-II) by Deb et al. [DPAM02], currently cited more than 50,000 times according to Google scholar.

While it was always known that the performance of this algorithm becomes weaker with increasing numbers of objectives – this was the main motivation for Deb and Jain [DJ14] to propose the NSGA-III –, very recent mathematical analyses of multi-objective evolutionary algorithms (MOEAs) could quantify and obtain a deeper understanding of this shortcoming. In [ZD23b], it was proven that the NSGA-II with any population size linear in the Pareto front size cannot optimize the simplistic OneMinMax benchmark in subexponential time when the number of objectives is at least three (for two objectives, a small polynomial runtime guarantee was proven by Zheng and Doerr [ZD23a]). In contrast, for the NSGA-III and the SMS-EMOA, two algorithms differing from the NSGA-II only in that the crowding distance is replaced by a different secondary selection criterion, polynomial runtime guarantees could be proven for the OneMinMax and several other benchmarks in any (constant) number of objectives [WD23, ODNS24, ZD24b, WD24]. This different optimization behavior suggests that it is the crowding distance which is the root for the problems of the NSGA-II in higher numbers of objectives.

Given that the NSGA-II is the by far dominant MOEA in practice, clearly beating the NSGA-III (cited less than 6,000 times according to Google scholar) and the SMS-EMOA (cited less than 2,200 times), and speculating that practitioners prefer working with a variant of the NSGA-II rather than switching to a different algorithm (and also noting that the NSGA-III and SMS-EMOA have some known shortcomings the NSGA-II does not have), in this work we propose to use the NSGA-II unchanged apart from a mild modification to the crowding distance. This change will again build on insights from Zheng and Doerr [ZD23b]. We defer the technical details to a separate section below and state here only that we call our crowding distance truthful crowding distance since we feel that it better reflects how close a solution is to others.

Given that we build mostly on previous works of mathematical nature, and strongly profited from the precision of such results, we analyze the NSGA-II with truthful crowding distance also via mathematical means. Our main results are the following. (i) For the standard many-objective benchmarks mOneMinMax, mCOCZ, mLOTZ, and mOJZJ, the NSGA-II with truthful crowding distance computed the whole Pareto front efficiently, in asymptotically the same time as the NSGA-III or the SMS-EMOA. This demonstrates clearly that it was in fact a weakness of the original crowding distance that led to the drastic problems observed for the NSGA-II in many-objective optimization. (ii) For the bi-objective versions of these benchmarks, for which the NSGA-II was efficient, we show that the NSGA-II with truthful crowding distance is equally efficient, and this already from population sizes on that equal the Pareto front of the problem, whereas the previous results needed a population size some constant factor larger. (iii) We also regard the problem of approximating the Pareto front when the population size is too small to cover the full Pareto front. Here we show that our NSGA-II with sequential selection, in an analogous way as the sequential version of the classic NSGA-II, computes good approximations to the Pareto front of the bi-objective OneMinMax problem (the approximation quality is minimally better for our algorithm).

In summary, these results show that the NSGA-II with truthful crowding distance overcomes the difficulties of the classic NSGA-II in many-objective optimization, but preserves its good performance in bi-objective optimization.

In this work, we discuss variants of the NSGA-II, the most prominent multi-objective evolutionary algorithm and one of the most successful approaches to solve multi-objective optimization problems.

A multi-objective optimization problem is a tuple $f=(f_{1},\dots,f_{m})$ of functions defined on a common search space. As common in discrete evolutionary optimization, we always consider the search space $\{0,1\}^{n}$. We call $n$ the problem size. When using asymptotic notation, this will be with respect to $n$ tending to infinity.

Also always, our goal will be to maximize $f$. Since the individual objective $f_{i}$ might be conflicting, we usually do not have a single solution maximizing all $f_{i}$. In this case, the best we can hope for are solutions that are not strictly dominate by others. We say that a solution $x$ dominates a solution $y$, written as $x\succeq y$, if $f_{i}(x)\geq f_{i}(y)$ for all $i\in[1..m]$. If in addition one of these inequalities is strict, we speak of strict domination, denoted by $\succ$. The Pareto set of a problem is the set of solutions that are not strictly dominated by another solution; the set of their objective values is called the Pareto front.

A common solution concept to multi-objective problems is to compute a small set $S$ of solutions such that $f(S)$ is the Pareto front or approximates it in some sense. The idea is that a human decision maker, based on preferences not included in the problem formulation, can then select from $S$ the final solution.

For this problem, evolutionary algorithms have been employed with great success [CLvV07, ZQL⁺11]. The by far dominant algorithm among these multi-objective evolutionary algorithms (MOEAs) is the non-dominated sorting genetic algorithms II (NSGA-II) proposed by Deb et al. [DPAM02]. This algorithm works with a population $P$, initialized randomly, of fixed size $N$. Each iteration of the main optimization loop consist of creating $N$ new solutions from these parents (“offspring”) and selecting from the combined parent and offspring population the next population of $N$ individuals.

Various ways of creating the offspring have been used. We shall regard random parent selection (each offspring is generated from randomly chosen parents) and fair parent selection (only with mutation, here from each parent and offspring is generated via mutation), 1-bit and bit-wise mutation with mutation rate $1/n$, and uniform crossover. When using crossover, we assume that there is a positive constant $p<1$ (crossover rate) and in each iteration, with probability $p$ and offspring is created via crossover, else via mutation. Binary tournament parent selection has also been studied, but the existing mathematical results, see, e.g., [ZD23a] suggest that it does not lead to substantially different results, but only to more complicated analyses.

More important and characteristic for the NSGA-II is the selection of the next parent population. The most important selection criterion is non-dominated sorting, that is, the combined parent and offspring population $R$ is partitioned into fronts $F_{1},F_{2},\dots$ such that $F_{i}$ consist of all non-dominated elements (that is, elements not strictly dominated by another one) of $R\setminus(F_{1}\cup\dots\cup F_{i-1})$. Individuals in an earlier front are preferred in the selection of the next population, that is, for the maximum $i^{*}$ such that $F_{1}\cup\dots\cup F_{i^{*}-1}$ contains less than $N$ elements, these fronts all fully go into the next population. The remaining elements are selected from the critical front $F_{i^{*}}$ using a secondary criterion, which is the crowding distance for the NSGA-II. We defer the precise definition of the crowding distance to the subsequent section. We note that the non-dominated-sorting partition is uniquely defined and can be computed in time $O(m|R|^{2})$. The crowding distance depends on how ties in sortings are broken, the crowding distance of all individuals in $F_{i^{*}}$ can be computed very efficiently in time $O(m|F_{i^{*}}|\log|F_{i^{*}}|)$.

The pseudocode of the NSGA-II can be found in Algorithms 1, where we note that the presentation here is optimized for a uniform treatment of this NSGA-II and the variant with sequential selection to be discussed now.

Noting that the removal of an individual changes the crowding distance of the remaining individuals, Kukkonen and Deb [KD06] proposed to take into account this change, that is, to sequentially remove individuals with smallest crowding distance and update the crowding distance of the remaining individuals. This was shown to give superior results in the empirical study [KD06] and the mathematical analysis [ZD24a].

In this section, we first describe the original crowding distance used in the NSGA-II of Deb et al. [DPAM02] and compare it with other ways to select a subset of individuals from the critical front of a non-dominated sorting (secondary selection criterion). This comparison motivates the development of a modification of the crowding distance, called truthful crowding distance, done in the second half of this section.

Having introduced the truthful crowding distance and the truthful (sequential) NSGA-II, denoted by NSGA-II-T, in this and the subsequent section we will conduct several runtime analyses of this algorithm. The results in this section will in particular show that the NSGA-II-T can efficiently optimize the many-objective problems, in contrast to the exponential runtime of the original NSGA-II on mOneMinMax [ZD23b].

In Section 4, we proved that the standard and sequential NSGA-II-T can efficiently optimize the many-objective mOJZJ, mOneMinMax, mCOCZ, and mLOTZ benchmarks as well as the popular bi-objective OneJumpZeroJump, OneMinMax, COCZ, LOTZ, and DLTB benchmarks. In this section, we will consider the approximation ability when the population size is too small to cover the full Pareto front. We will prove that the sequential NSGA-II-T has a slightly better approximation performance than the sequential NSGA-II and the steady-state NSGA-II for OneMinMax [ZD24a].

We note that there are no proven approximation guarantees for non-sequential variants of the NSGA-II (except for the steady-state version) so far and the mathematical results in [ZD24a] suggest that such results might be difficult to obtain. For that reason, we do not aim at such results for the truthful NSGA-II. We also note that so far there is no theoretical study on the approximation ability of the NSGA-II other than for the (bi-objective) OneMinMax benchmark [ZD24a]. We shall therefore also only consider this problem. We expect that results for larger numbers of objectives or other benchmarks need considerably new methods as already the approximation measure MEI is may not be suitable then.

The following lemma gives a useful criterion for individuals surviving into the next generation.

The following lemma shows that once the two extreme points are in the population, a linear runtime suffices to obtain a good approximation of the Pareto front of OneMinMax.

Noting that the maximal function value of $f_{1}(P_{t})$ and $f_{2}(P_{t})$ cannot decrease (there always is one individual witnessing this value and having infinite truthful crowding distance), we easily obtain that in expected $O(n\log n)$ iterations both extreme points $0^{n}$ and $1^{n}$ are reached for the first time, and also for all future iterations. This can be shown with a proof analogous to the one of [ZD24a, Lemma 15]. Therefore, we have the following main result on the approximation ability and runtime of the sequential NSGA-II-T.

Note that the best possible value for the MEI is $\lceil\frac{n}{N-1}\rceil$ [ZD24a]. Theorem 13 shows that the sequential NSGA-II-T can reach a good approximation guarantee of MEI of the optimal value multiplied by at most only a factor of two. We also note that for the NSGA-II with sequential survival selection using the classic crowding distance, an approximation guarantee of $\textsc{MEI}\leq\max\{\frac{2n}{N-3},1\}$ (also within $O(n\log n)$ iterations) was shown in [ZD24a]. The slightly better approximation ability shown above (within the same runtime), as the proof of Lemma 12 shows, stems from the fact that our definition of the crowding distance admits at most one individual with infinite crowding distance contribution per objective, whereas the classic crowding distance admits two.

To complement our theoretical findings, we now show a few experimental results. These meant as illustration of our main (mathematical) results, not as substantial stand-alone results.

Computing the full Pareto front, many objectives: Our main theoretical result was a proof that the NSGA-II-T can efficiently solve many-objective problems, different from the classic NSGA-II, where an exponential lower bound was shown for the OneMinMax problem. To illustrate how the NSGA-II-T solves many-objective problems, we regard the 4-objective OneMinMax problem. That the NSGA-II cannot solve this problem efficiently was shown, also experimentally, in [ZD23b]. We hence study only the (sequential) NSGA-II-T as algorithm, using random selection, bit-wise mutation, no crossover, and the minimal possible population size $N=M$ (the Pareto front size) and $N=2M$. We also use the GSEMO toy algorithm. In Figure 1, we display the median (over 20 runs) number of function evaluations these algorithms took to cover the full Pareto front of $4$-OneMinMax for different problem sizes $n$.

We observe that all algorithms efficiently find the full Pareto front, in drastic contrast to the results for the classic NSGA-II in [ZD23b, Figure 1 and 2]. Not surprisingly, a larger population size is not helpful, which shows that it is good that the NSGA-II-T admits smaller population sizes than the classic NSGA-II. Also not surprisingly, the sequential versions give slightly better results. The minimal inferiority of the (sequential) NSGA-II-T (with $N=M$) to the toy GSEMO does not mean a lot given that this algorithm is rarely used in practice.

Computing the full Pareto front, two objectives: We conducted analoguous experiments for two objectives, where a comparison with the NSGA-II is interesting. The proven guarantees for the NSGA-II require a population size of $N\geq 4M$, where this algorithm is clearly slower than all others regarded by us. We therefore did some preliminary experiments showing that already for $N=1.5M$ the NSGA-II consistently is able to solve our problem instances. The results for this NSGA-II, the NSGA-II-T with optimal population size $N=M$ and with $N=1.5M$, and the GSEMO are shown in Figure 2. With this optimized population size for the NSGA-II, all algorithms show a roughly similar performance on the $2$-objective OneMinMax problem, with the NSGA-II slightly ahead.

Approximation results: To analyze who well the different NSGA-II variants with small population size approximate the Pareto front, we conduct the following experiments. Note that the GSEMO cannot be used for approximative solution and is therefore not included. Following experimental settings in the only previous theoretical work [ZD24a] on the approximation topic, we regard the bi-objective OneMinMax problem with problem size $n=601$. We use the same algorithms as above (except for the GSEMO), with population sizes $N=\lceil(n+1)/2\rceil(=301),\lceil(n+1)/4\rceil(=151),\lceil(n+1)/8\rceil(=76)$. As before, we measure the approximation quality via the MEI. We note that the best possible MEI values are $3,5,9$ for $N=301,151,76$ respectively.

As in [ZD24a], we regard the approximation quality in two time intervals, namely in iterations $[1..100]$ and $[3001..3100]$ after the two extreme points of the Pareto front have entered the population.

Figure 3 shows the MEI values for the different algorithms in a single run, for reasons of space only for $N=151$ (but the other population sizes gave a similar picture). We clearly see a much better performance of the sequential algorithms, with no significant differences between the classic and the truthful sequential NSGA-II.

To overcome the difficulties the NSGA-II was found to have in many-objective optimization, we used the insights from several previous theoretical works, most profoundly [ZD23b], to design a truthful crowding distance for the NSGA-II. Different from the original crowding distance, this new measure has the natural and desirable property that solutions with objective vector far from all others receive a large crowding distance value. The truthful crowding distances are slightly more complex to compute, but asymptotically not more complex than the non-dominated sorting step of the NSGA-II.

Via mathematical runtime analyses on several classical benchmark problems, we prove that the NSGA-II with the truthful crowding distance indeed is effective in more than two objectives, admitting the same performance guarantees as previously shown for the harder to use NSGA-III and the SMS-EMOA, which is computationally demanding due to the use of the hypervolume contribution. For the bi-objective benchmarks, for which the classic NSGA-II has been analyzed, we prove the same runtime guarantees (however, only requiring the population size to be at least the size of the Pareto front). Similarly, the truthful NSGA-II admits essentially the same (that is, minimally stronger) approximation guarantees as previously shown for the classic NSGA-II.

Consequently, the NSGA-II with truthful crowding distance overcomes the difficulties of the classic NSGA-II in many objective without that we observe disadvantages in two objectives, where the classic NSGA-II has shown a very good performance.