Divide-and-Conquer Large Scale Capacitated Arc Routing Problems with Route Cutting Off Decomposition

Given an undirected graph $G(V,E)$, where $V$ and $E$ represent the vertex and edge sets. Each edge $e\in E$ has a demand $d(e)\geq 0$, a non-negative service cost $sc(e)\geq 0$ and a non-negative deadheading cost $dc(e)\geq 0$. An edge with a positive demand is called a required edge or a task. The set of all the tasks is denoted as $T=\{e\in E|d(e)>0\}\subseteq E$. A fleet of vehicles with a limited capacity $Q$ is located at the depot $v_{0}\in V$ to serve the tasks. CARP is to design a set of least-cost routes for the vehicles to serve all the tasks subject to the following constraints:

• 1.

  each route starts and ends at the depot (i.e. is a cycle);

• 2.

  each task is served exactly once by a vehicle;

• 3.

  (capacity constraint) the total demand served by each vehicle cannot exceed its capacity.

Under the task representation [36], each task $(u,v)$ can be represented by two IDs, each representing one of its directions. In addition to the demand, service cost and deadheading cost of the corresponding task, each ID $t$ is associated with a head vertex $hv(t)$, a tail vertex $tv(t)$ and a inverse ID $inv(t)$. Specifically, for task $(u,v)$ and its two IDs $t_{1}$ and $t_{2}$, we have $hv(t_{1})=tv(t_{2})=u$, $tv(t_{1})=hv(t_{2})=v$, $t_{2}=inv(t_{1})$ and $t_{1}=inv(t_{2})$.

A CARP solution $S$ can be represented as a set of routes, i.e. $S=\{S_{1},\dots,S_{|S|}\}$, where $S_{k}$ ($k=1,\dots,|S|$) is the $k$th route. Each route $S_{k}$ is represented as a sequence of task IDs $S_{k}=(t_{k1},\dots,t_{k|S_{k}|})$.

The total cost of $S$ is calculated as follows:

where $S_{k}[i]$ stands for the $i$th element (task ID) in $S_{k}$. $\delta(u,v)$ indicates the cost of the shortest path from vertices $u$ to $v$, which can be calculated by Dijkstra’s algorithm [10] beforehand.

Then, CARP can be formulated as follows:

where in Eq. (3), $t_{0}$ is a dummy task ID representing the depot loop, i.e. $hv(t_{0})=tv(t_{0})=v_{0}$, $d(t_{0})=sc(t_{0})=dc(t_{0})=0$ and $inv(t_{0})=t_{0}$. Eq. (2) is the objective function, which is to minimize the total cost calculated by Eq. (1). Eq. (3) specifies that in each route $S_{k}$, the first and last element must be the depot loop, indicating that each route starts and ends at the depot. Eq. (4) means that the total number of task IDs served by all the routes (excluding the first and last elements which are the depot loop) equals the total number of tasks. Eqs. (5) and (6) implies that any two task IDs served at different positions of the routes belong to different tasks. Therefore, Eqs. (4)–(6) guarantee that each task is served exactly once. Eq. (7) indicates that the total demand served by each route is no greater than its capacity. Eq. (8) defines the domain of the elements of each route, i.e. except the first and last element, all the other elements must belong to the task set.

Since presented in 1981 [15], CARP has received a lot of research interests over the past decades, and a variety of competitive algorithms ranging from mathematical programming (e.g. [3, 1, 6, 2]) and heuristic solution search approaches (e.g. [18, 5, 26, 19, 36, 13, 44, 34]) have been proposed for solving it. However, most of the early studies focused on small and medium scaled problems, and only tested on small and medium-sized benchmark instances. For example, the most commonly used gdb [9], val [4], egl [11] and Beullens’ benchmark sets [5] have no more than 190 tasks.

In 2008, Brandão and Eglese [7] generated a large scale dataset named EGL-G, in which the number of tasks was increased to 375. Since then, the research interests gradually shifted to the scalability of the approach, and more and more studies were conducted specifically to tackle LSCARP. For example, Brandão and Eglese [7] proposed a tabu search and achieved promising results on the EGL-G instances. Mei et al. [23] proposed a tabu search with a global repair operator, and Martinelli et al. [20] proposed an Iterative Local search based on Random Variable Neighbourhood Descent (ILS-RVND), both of which improved the upper bounds of the EGL-G dataset. Vidal [39] proposed an unified hybrid genetic search (UHGS) which outperformed all the existing algorithms on the EGL-G dataset.

As the problem size grows, solving the problem as a whole becomes much less effective, and divide-and-conquer strategy can be a promising technique in this case. The divide-and-conquer strategy has achieved great success in a range of problems, such as continuous optimization [43, 35, 14], vehicle routing [29, 28] and job shop scheduling [46, 27]. Also for LSCARP, there have been a variety of divide-and-conquer approaches [21, 22, 31, 33, 32] based on decomposing the problem into smaller sub-problems by grouping the routes of the best-so-far solution. Specifically, in these algorithms, the entire search process is divided into cycles. At the beginning of each cycle, the routes of the best-so-far solution are grouped together based on different strategies (e.g. randomly or by clustering methods). They have achieved much better results than the methods without divide-and-conquer.

In 2017, based on two major Chinese cities, i.e. Beijing and Hefei, Tang et al. [37] created two real-world LSCARP datasets, and extended the number of tasks to over 3000. The Beijing and Hefei datasets are much larger than the EGL-G dataset. The existing divide-and-conquer approaches are not effective enough to solve them. Tang et al. [37] proposed a new hierarchical decomposition based on the concept of “virtual task”. A virtual task is a sequence of tasks. Starting from elementary tasks, the hierarchical decomposition recursively concatenates (virtual) tasks together to form new higher-level virtual tasks. The resultant algorithm, named SAHiD, managed to obtain much better results than other divide-and-conquer approaches (e.g. RDG-MAENS [22]) within a limited time budget.

In 2017, Kiilerich and Wøhlk [17] generated a very large scale dataset (denoted as the KW set) based on the five countries in Denmark. In the KW dataset, the number of tasks is further enlarged to over 8000. The KW set is the largest dataset by far. Wøhlk et al. [41] proposed a fast heuristic named Fast-CARP to solve the LSCARP, in which the whole problem is partitioned into a number of districts and each district is then optimized independently. Fast-CARP can be regarded as a divide-and-conquer approach.

In summary, divide-and-conquer approaches have achieved great success in solving LSCARP effectively. However, the current approaches still have limitations. They focused on the interactions between the tasks in different routes of the best-so-far solution during the decomposition, while the interactions between tasks within each route were neglected. In this paper, we aim to consider the interactions both in different routes and within the same route, and propose the RCO operator for this purpose.

Based on the above intuition, we define the task rank matrix ($\bm{\Gamma}$) to indicate the quality of the links from each task. Given a set of tasks $T$, the entry $\bm{\Gamma}_{t_{1},t_{2}}$ represents the rank of the link from task $t_{1}$ to task $t_{2}$, which is calculated based on the links from $t_{1}$ to all the other tasks.

An example is given in Fig. 1 to show how to calculate the task rank matrix. In the figure, $v_{0}$ is the depot. There are 8 tasks (represented by the solid lines) require to be served, and the shortest paths between the tasks are represented as dashed lines. The service cost and deadheading cost of each task are 1, and the cost of the shortest paths between the tasks are given next to the dashed lines.

For the graph shown in Fig. 1, we first calculate the matrix of the link costs in Eq. (10).

Then for each row of Eq. (10), we assign the ranks to the links based on their costs, e.g. the link with the lowest cost is given rank 1. If multiple links have the same cost, then they share the same rank. Eq. (11) shows the task rank matrix obtained from Eq.(10).

From the first row of Eq. (11), one can see that the links from $(v_{0},v_{1})$ to $(v_{0},v_{8})$ and $(v_{0},v_{10})$ are both of rank 1 among all the links from $(v_{0},v_{1})$, and the link to $(v_{2},v_{3})$ is of rank 7, since $(v_{2},v_{3})$ is the farthest task from $(v_{0},v_{1})$. Note that the task rank matrix is not symmetric. For $(v_{0},v_{1})$, the task $(v_{2},v_{3})$ has a very low priority (rank 7), since there are many other closer tasks for $(v_{0},v_{1})$ to consider. On the contrary, for $(v_{2},v_{3})$, $(v_{0},v_{1})$ is a very promising task to go to (rank 2).

Based on the task rank matrix, we propose the new RCO operator for decomposition. Briefly speaking, given a best-so-far solution, the RCO operator splits the routes of the solution based on the task rank matrix, to generate a pool of sub-routes. Clustering these sub-routes is expected to obtain more promising decomposition than the existing decomposition methods.

The pseudo code of the RCO operator is described in Algorithm 1. Given the best-so-far solution $S$, the average task rank $\overline{\gamma}(S)$ of the links in $S$ is first calculated based on the task rank matrix $\bm{\Gamma}$ (line 2). For each route $S_{k}$, we categorize the links into “good links” and “poor links” (lines 3–22). Here we adopt a simple rule for the categorization. A link is considered to be good if its rank is smaller than $\overline{\gamma}(S)$, and poor otherwise. Then, we randomly cut off a good link with probability $\lambda$, and a poor link with probability $\theta$ (lines 11–20). The corresponding sub-routes are finally inserted into $\Omega$.

Fig. 2 shows an example solution to the graph shown in Fig. 1, where the tasks are denoted as $x_{1}$ to $x_{8}$. Only the links between the tasks in the same route are considered (e.g. there is no link from $x_{3}$ to $x_{4}$). Based on Eq. (11), the ranks of the 5 links are $\bm{\Gamma}_{x_{1},x_{2}}=7$, $\bm{\Gamma}_{x_{2},x_{3}}=1$, $\bm{\Gamma}_{x_{4},x_{5}}=5$, $\bm{\Gamma}_{x_{5},x_{6}}=1$, and $\bm{\Gamma}_{x_{7},x_{8}}=1$. In this case, the average task rank of the solution is $(7+1+5+1+1)/5=3$. $\langle x_{2},x_{3}\rangle$, $\langle x_{5},x_{6}\rangle$ and $\langle x_{7},x_{8}\rangle$ are good links, while $\langle x_{1},x_{2}\rangle$ and $\langle x_{4},x_{5}\rangle$ are poor links.

Given the best-so-far solution, the proposed RCO operator identifies promising cutting points to split the routes of the solution into promising sub-routes for clustering. In other words, the RCO operator is a generic decomposition operator, and can be embedded into any divide-and-conquer algorithm based on clustering the routes/sub-routes. In this paper, we embed the RCO operator into two state-of-the-art algorithms, i.e. RDG-MAENS [22] and SAHiD [37] to verify the effectiveness of the RCO operator. The resultant algorithms are named RCO-RDG-MAENS and RCO-SAHiD, respectively.

The pseudo code of RCO-RDG-MAENS is described in Algorithm 2. The algorithm is the same as the original one [22], except that instead of clustering the routes directly, the RCO operator is used to split the best-so-far solution before clustering (line 6).

Likewise, the pseudo code of RCO-SAHiD is described in Algorithm 3, where the key difference from SAHiD [37] is that the routes of the current solution $S$ are split by RCO (line 6), while they are randomly split in SAHiD.

Since our work focuses on LSCARP, we select the four existing LSCARP datasets, i.e. the EGL-G [7], Hefei [37], Beijing [37] and KW [17] datasets. The EGL-G dataset consists of 10 instances, which are derived from a real-world road network of Lancashire, UK, with 255 nodes and 375 edges. The dataset contains two groups G1 and G2, each with 5 instances. The instances belonging to the same group have the same task set, and different vehicle capacities. The Hefei dataset contains 10 instances based on a road network in Hefei, China, with 850 nodes and 1212 edges. The instances have the same vehicle capacity, but vary in their task sets. Similarly, the Beijing dataset consists of 10 instances sharing the same road network in Beijing, China (2820 nodes and 3584 edges) and vehicle capacity. The KW dataset consists of 264 CARP benchmark instances generated from 88 graphs by varying the vehicle capacity. In the KW dataset, the largest instance contains 11640 nodes, 12675 edges, and 8581 required edges. Overall, EGL-G is the smallest LSCARP dataset. Hefei and Beijing are much larger than EGL-G, and KW is the largest dataset.

Since different datasets have different available results, we designed three experimental comparisons as follows.

• 1.

  Experiment 1: compare RCO-RDG-MAENS with RDG-MAENS [22], ILS-RVND [20], IRDG-MAENS [31], QICA-CARP [33] and ESMAENS [32] on the EGL-G dataset.

• 2.

  Experiment 2: compare RCO-SAHiD with SAHiD [37], UHGS [39], RDG-MAENS [22], VNS [30] and TSA1 [7] on the Hefei and Beijing datasets.

• 3.

  Experiment 3: compare RCO-SAHiD with SAHiD [37], UHGS [39], Fast-CARP [41] and PS [45] on some large KW instances (task number ranging 7831 to 8581).

In each experiment, we tried our best to compare with all the algorithms whose results are available in literature for the corresponding datasets. For each instance, each algorithm was run multiple times independently, and the Wilcoxon rank sum test [40] was conducted to test the results statistically.

To make fair comparisons, we set the algorithm parameters consistent with the settings in the literature. Specifically, in Experiment 1, following the parameter settings in [22], in RCO-RDG-MAENS, we set the population size to $30$, offspring population size to $180$, maximum generation number to $500$, number of cycles to $50$, and probability of local search to $0.2$. The RDG operator has two parameters: the number of groups $g$ and degree of fuziness $\alpha$ (used by the fuzzy $k$-medoid clustering). We set $g=2$ and $\alpha=5$ for both RCO-RDG-MAENS and RDG-MAENS, as they showed the best performance [22].

In Experiment 2, following the parameter settings in [37], in RCO-SAHiD, the scale parameter in HD is $0.1$, the threshold for accepting a worse solution is $110\%$, and the maximum number of idle iterations for accepting an ascending move is $10000$. Note that in Experiment 2, the stopping criterion of RCO-SAHiD and UHGS is the runtime i.e. after 30 minutes. The runtime depends on a variety of factors such as CPU frequency, RAM, operating system, coding language and compiler. In our experiments, we implemented RCO-SAHiD based on the original SAHiD source code to make sure they share the same programming language and compiler. To improve fairness, a common approach used by previous studies (e.g.[24, 47, 36, 23, 20, 22]) is to scale the runtime based on the CPU frequency. In this paper, we adopt the same scaling approach. RCO-SAHiD and UHGS were run on Intel(R) Xeon(R) E5-2650 v2 with 2.6 GHz, 64GBs RAM, and the other compared algorithms were run on Intel Core i7-4790 with 3.6 GHz. Therefore, the maximum runtime for RCO-SAHiD and UHGS was set to $3.6/2.6\times 30\times 60=2492$ seconds.

In Experiment 3, the parameters of RCO-SAHiD are the same as in Experiment 2. The runtime follows the configuration of the original literature of Fast-CARP [41], which is one minute per 1000 nodes. Fast-CARP was run on an Intel Xeon CPU with 3.5 GHz, while RCO-SAHiD, SAHiD and UHGS were run on Intel(R) Xeon(R) E5-2650 v2 with 2.6 GHz, 64GBs RAM. The runtime of these three algorithms were scaled to $3.5/2.6\times 60=81$ seconds per 1000 nodes.

The RCO operator has two important parameters, namely the probability of cutting good links $\lambda$ and the probability of cutting poor links $\theta$. Intuitively, it is more promising to cut poor links than good links. On the other hand, cutting good links can increase the exploration capability of the algorithm, and help the search jump out of the current local optimum. Based on the above consideration, we should set $\lambda$ to a small value, and $\theta$ to a relatively large value. To analyze the sensitivity of $\lambda$ and $\theta$, we conducted some pilot experiments by running RCO-RDG-MAENS with $\lambda=0.05$ and $0.1$, and $\theta=0.1$, $0.2$ and $0.3$ (six combinations in total) on the EGL-G dataset. Each algorithm was run 30 times independently on each instance.

Fig. 3 and Fig. 4 show the average total cost and computational time of RCO-RDG-MEANS with different values of $\lambda$ and $\theta$ over the 10 EGL-G instances. From the figures, one can see that for all the instances, there is no significant difference among the different $\lambda$ and $\theta$ values in terms of the average total cost. However, the $\lambda$ and $\theta$ values have some impact on the computational time. The algorithm with $\lambda=0.1$ and $\theta=0.3$ usually had the longest computational time. All the other five versions had similar computation time. Overall, the algorithm with $(\lambda,\theta)=(0.05,0.2)$ seems to have a short computational time over all the instances.

In summary, the tested $\lambda$ and $\theta$ values have little impact on the performance, and affect the computational time only slightly. This is a good sign, as it means that the effectiveness and efficiency of the algorithm are not sensitive to the $\lambda$ and $\theta$ values within a considerable range.

Fig. 4 shows that $(\lambda,\theta)=(0.05,0.2)$ achieved the overall lowest computation time. Therefore, in the subsequent experiments, we select $(\lambda,\theta)=(0.05,0.2)$ for the RCO operator in both RCO-RDG-MAENS and RCO-SAHiD.

In Experiment 1, RCO-RDG-MAENS is compared with RDG-MAENS [22], ILS-RVND [20], IRDG-MAENS [31], QICA-CARP [33], and ESMEANS [32]. For RDG-MAENS, we downloaded the code from online ¹¹1The C code of RDG-MAENS is available from http://homepages.ecs.vuw.ac.nz/~yimei/codes/RDG-MAENS.zip and reran it for 30 times independently on each instance. However, for all the other compared algorithms, no code is available for rerunning the experiments. Therefore, we directly copied the results of these algorithms from their original literature. Note that we configured RCO-RDG-MAENS in exactly the same way as RDG-MAENS. Therefore, we can guarantee a fair comparison with RDG-MAENS and other algorithms (as they compared with RDG-MAENS under the same configuration).

Table 1 shows the average performance of the compared algorithms on the 10 EGL-G instances. In the table, the columns “$|V|$”, “$|E|$” and “$|T|$” stand for the number of vertices, edges and tasks, respectively. $\zeta$ indicates the minimum number of vehicles required to serve all the routes, which can be computed as $\zeta=\left\lceil\frac{\sum_{t\in T}{d(t)}}{Q}\right\rceil$, where $d(t)$ is the demand of task $t$ and $Q$ refers to the capacity of the vehicles. In general, with the same problem size, a larger $\zeta$ value implies a more complex problem instance. For each algorithm, the columns “Mean” and “Std” are the mean and standard deviation of the total costs obtained by 30 independent runs. Note that there is no “Std” column for ILS-RVND and IRDG-MAENS, since only the mean value was reported in their original literature.

For each instance, the minimal mean total cost among all the algorithms is marked with “$\dagger$”. In addition, we conduct statistical test between RCO-RDG-MAENS and each compared algorithm using Wilcoxon rank sum test under the significance level of $0.05$. If an algorithm is significantly worse (better) than RCO-RDG-MAENS, then it is marked with underline (in bold). In addition, the last row “W-D-L” stands for the number of instances on which RCO-RDG-MAENS performed significantly better than (“W”), statistically comparable with (“D”), and significantly worse than (“L”) the corresponding algorithm. For example, “1-9-0” under RDG-MAENS indicates that RCO-RDG-MAENS performed significantly better than RDG-MAENS on 1 instance, and statistically comparable with it on the remaining 9 instances.

From Table 1, RCO-RDG-MAENS performed statistically comparable with RDG-MAENS on 9 out of 10 EGL-G instances, and significantly outperformed it on G2-B. In comparison with the other algorithms, RCO-RDG-MAENS performed much better. It significantly outperformed ILS-RVND and QICA-CARP on all the instances, and IRDG-MAENS and ESMAENS on 7 and 8 instances, respectively. RCO-RDG-MAENS never performed significantly worse than any compared algorithm on any instance.

Table 2 shows the best total cost obtained from the 30 independent runs of the compared algorithms on each instance.²²2The full detail of the best solutions can be found from https://meiyi1986.github.io/files/data/carp/results.zip. The minimal total cost is marked with “$\dagger$” and result is marked with undeline (in bold) if it is larger (smaller) than that of RCO-RDG-MAENS. In the last two rows, the “Mean” row stands for the mean values of the best total costs obtained by each compared algorithm over all the instances, and the last row “G-E-S” stands for the number of instances on which results obtained by RCO-RDG-MAENS was greater than (“G”), equal to (“E’), and smaller than (“S”) the corresponding algorithm. Note that the table does not include IRDG-MAENS [31], since the best performance of IRDG-MAENS was not reported in the original literature.

From Table 2, one can see that RCO-RDG-MAENS reached the minimal total cost on 6 out of 10 instances, which is much more than that of the other algorithms (i.e., 3 instances for RDG-MAENS, 0 for ILS-RVND, 1 for QICA-CARP and 2 for ESMAENS). In terms of the mean of the best total cost over the 10 instances, RCO-RDG-MAENS performed much better than the other compared algorithms (e.g. $1300592.9$ versus $1301236.6$ in comparison with RDG-MAENS).

To make more intuitive comparisons between RCO-RDG-MAENS and RDG-MAENS, two representative instances (i.e., G1-A and G2-E) are selected from EGL-G, and the convergence curves of RCO-RDG-MAENS and RDG-MAENS on them are shown in Fig. 5. In the figure, the $x$-axis is the computational time, and the $y$-axis is the mean total cost of the best-so-far solutions obtained by the two algorithms. From the figure, one can see that the convergence curves of the two algorithms are very close to each other. RCO-RDG-MAENS tend to converge slightly slower than RDG-MAENS in the early stage of the search, and then catch up and achieve better results than RDG-MAENS in the later stage (e.g. the two curves crossed each other after around 500 seconds for both instances).

In summary, the comparison between RCO-RDG-MAENS and other algorithms including RDG-MAENS shows that the RCO operator can improve the performance of RDG-MAENS on the EGL-G instances. In terms of average performance, RCO-RDG-MAENS significantly outperformend RDG-MAENS on 1 instance, and was never beaten by RDG-MAENS. In terms of best performance, RCO-RDG-MAENS managed to outperform RDG-MAENS (and other compared algorithms) on most EGL-G instances.

The advantage of the RCO operator looks marginal in Experiment 1. This is partially because RDG-MAENS already has a high decomposition accuracy for medium-sized instances such as the EGL-G instances [37], and there is not much space for improvement by the RCO operator. However, the Hefei, Beijing and KW datasets are much larger and more complex. Thus, we expect the RCO operator to show more advantage in Experiments 2 and 3.

In Experiment 2, RCO-SAHiD is compared with SAHiD [37], UHGS [39], RDG-MAENS [22], VNS [30] and TSA1 [7] on the Hefei and Beijing datasets, which are much larger than the EGL-G dataset. Following the same practice in [37], both RCO-SAHiD and SAHiD were ran 25 times independently. The maximum runtime of the compared algorithms is set to $2492$ seconds after the scaling.

Note that the configuration of Experiment 2 is very different from Experiment 1. The instances in Experiment 2 is much larger than the instances in Experiment 1, and the time budget is much tighter. Therefore, the search efficiency of the algorithm within a very limited time budget becomes much more important.

Table 3 shows the average performance of the compared algorithm on the 10 Hefei instances. For each instance, the minimal mean total cost is marked with “$\dagger$”. Under Wilcoxon rank sum test with significance level of $0.05$, if an algorithm is significantly worse (better) than RCO-SAHiD, then its result is marked with underline (in bold).

From Table 3, one can see that RCO-SAHiD significantly outperforms all the other compared algorithms except UHGS with respect to the average performance. In particular, RCO-SAHiD significantly outperformed SAHiD on 9 out of the 10 instances, with much smaller mean and standard deviation. This indicates that embedding RCO into SAHiD can greatly improve its effectiveness and stability.

Table 4 shows the best total cost obtained by the compared algorithms over the 25 independent runs on the Hefei dataset. The table shows consistent patterns in terms of the best performance with the average performance. UHGS performed the best on all the 10 Hefei instances. It is followed by RCO-SAHiD, which is much better than all the other algorithms.

Tables 5 and 6 show the average and best performance of the compared algorithms on the Beijing dataset. From the tables, we can observe consistent patterns with those on the Hefei dataset. UHGS performed the best on all the Beijing instances in terms of both average and best performance. RCO-SAHiD was the second best algorithm, showing significantly better performance than all the other compared algorithms (including SAHiD) on all the Beijing instances.

To further demonstrate the efficacy of embedding RCO, Fig. 6 shows the convergence curves of RCO-SAHiD and SAHiD over the 25 independent runs on the Hefei-1, Hefei-10, Beijing-1 and Beijing-10 instances, where the $x$-axis is the computational time in seconds, and the $y$-axis is the average total cost of the best-so-far solution. These four instances are selected as the representative instances of the corresponding datasets, and similar patterns are observed on other instances.

From the figure, one can see that the convergence curves of RCO-SAHiD are almost always below that of SAHiD, and there is a decent gap between the two convergence curves. The only exception is the Hefei-10 instance, for which SAHiD converged slightly better than RCO-SAHiD. This is also consistent with Table 3, which shows that Hefei-10 is the only instance where there is no statistical difference between RCO-SAHiD and SAHiD.

In summary, the results in Experiment 2 clearly demonstrate the effectiveness of the RCO operator in improving the performance of SAHiD. Although RCO-SAHiD did not perform so well as UHGS, this is mainly due to the superiority of UHGS over SAHiD in terms of search capability, rather than the problem decomposition. Since UHGS is not a divide-and-conquer approach, we expect that embedding RCO into UHGS can further improve its performance.

Experiment 3 is to compare RCO-SAHiD with SAHiD [37], UHGS [39], Fast-CARP [41] and PS [45] on 12 largest KW instances. RCO-SAHiD, SAHiD and UHGS were ran 25 times independently. The runtime of $81$ seconds per $1000$ nodes is set for the compared algorithms on each instance. The results of Fast-CARP and PS are copied from the original literatures ([41] and [45]), as their codes are not available.

Compared with Experiment 2, Experiment 3 has a much larger problem size and much tighter time budget. For example, in K1_g-2, there are 8556 tasks, while only $81\times 11640/1000=943$ seconds is allowed. In other words, Experiment 3 has a strong requirement for an algorithm to search effectively in a huge search space within a very limited time budget.

Table 7 shows the average performance of the compared algorithms on the 12 KW instances. For each instance, the minimal mean total cost is marked with “$\dagger$”. Under Wilcoxon rank sum test with significance level of $0.05$, if an algorithm is significantly worse (better) than RCO-SAHiD, then its result is marked with underline (in bold).

From Table 7, one can see that RCO-SAHiD statistically significantly outperformed SAHiD on all the 12 KW instances. It is consistent with the results on the Hefei and Beijing datasets, which demonstrates the effectiveness of the proposed RCO operator. Again, UHGS performed the best on the KW datasets. However, we observed that it is much slower than our algorithm. Specifically, even the initialisation stage can take much longer time than the given time budget (e.g. $3774.5$ seconds for the O1_p-4 instance, while the given time budget is $804.2$ seconds).

Table 8 shows the best performance of the compared algorithms on the KW dataset, where the minimal total cost of each instance is marked with “$\dagger$”, and is marked with underline (in bold) if it is greater (smaller) than RCO-SAHiD. For Fast-CARP and PS, the results are obtained from [41] directly. Since their results were obtained by a single run after sophisticated parameter tuning (i.e. Fast-CARP), or the best ones obtained by 105 runs with different evaluation criteria and degrees of randomization (i.e. PS), we treated them as the best performance in the comparison. The patterns shown in Table 8 are consistent with those in Table 7. In addition, Fast-CARP performed slightly better than RCO-SAHiD on some KW instances. However, it was very carefully tuned, by testing a large number of combinations of parameters (about 200 combinations). On the other hand, SAHiD is almost parameter-free, and is easier to use in practice.

Fig. 7 shows the convergence curves of RCO-SAHiD and SAHiD over 25 independent runs on four representative KW instances (K1_g-2, K1_g-6, O6_g-2 and O6_g-6), where the $x$-axis is the computational time in seconds, and the $y$-axis is the average total cost of the best-so-far solution. These four instances are selected as the representative instances of the corresponding datasets, and similar patterns are observed on other instances.

From the figure, one can see that the convergence curves of RCO-SAHiD are always significantly below that of SAHiD, and there is an apparent gap between the two convergence curves. This indicates that no matter when to stop the search, RCO-SAHiD will always provide a better solution than SAHiD.

In summary, the results in Experiment 3 clearly demonstrates the effectiveness of the proposed RCO operator in improving the SAHiD algorithm on large instances. Although RCO-SAHiD did not manage to outperform the state-of-the-art UHGS and Fast-CARP, due to the limited performance of the embedded SAHiD, it was still faster and less space demanding than UHGS, and less parameter sentitive than Fast-CARP.

To investigate how the effectiveness of the RCO operator is affected by the problem size, we plot the relative performance of RCO-SAHiD to SAHiD on different problem sizes. Fig. 8 shows the relative performance of RCO-SAHiD to SAHiD (ratio between their performances) versus the number of tasks on all the tested instances, where the $x$-axis is the number of tasks, and the $y$-axis is relative performance.

From Fig. 8, one can see that embedding RCO can improve the performance of SAHiD (the ratio is smaller than 1) on almost all the instances except one Hefei instance. For the Beijing instances, the advantage of RCO becomes more obvious as the problem size increases. For the Hefei instances, on the other hand, RCO becomes less effective with the increase of the problem size. For the EGLG and KW instances, there is no particular trend, since they have very similar problem sizes. Overall, RCO can almost always lead to improvement.

To give a further analysis on the poor performance of RCO on the Hefei dataset, we investigated the topology of the original network on Hefei-10 by computing the link rank of each pair of tasks. We observed that Hefei-10 contains less high-rank links than other instances. Such link distribution decreases the effectiveness of RCO in catching poor links. As a result, RCO-SAHiD can hardly obtain high-quality solution to Hefei-10, and showed poor performance.