Improved Swarm Engineering: Aligning Intuition and Analysis

Within SR, no widely accepted theory of self-organizing systems exists [48, 11], in part because emergent, self-organizing swarms cannot be easily understood by studying individual parts in isolation [13]. In SR systems, interactions between components can overtake outside interactions and give rise to new emergent behaviors not predictable from component study; that is, directly from the swarm control algorithm [15, 49, 12, 13]. However, self-organization can be related to the latency of information propagation through a swarm and swarm sparseness [50], and is often synergistically coupled to emergent behaviors [13].

While many papers show evidence that their algorithms exhibit emergent behavior ([51, 47, 16]), or provide a framework for logical reasoning about emergent properties ([10, 52, 53]), a general theory of emergence remains elusive. Nevertheless, some common preconditions for self-organizing emergent behaviors have been observed [54, 13]. Let $\mathbb{S}$ be a swarm of $\mathcal{N}$ robots, then the preconditions are:

1. 1.

   Positive feedback via heuristics which promote organization at the collective level, such as reactive recruitment and reinforcement. In constrained physical environments it can lead to emergent spatial self-organization [55].

2. 2.

   Negative feedback via pheromone evaporation, interference, etc., stabilizes the behavior of $\mathbb{S}$  [56, 47].

3. 3.

   Random fluctuations via random walks, random task switching, etc., are necessary for collective creativity and problem solving, though they can also lead to sub-optimal collective solutions if not moderated by other properties.

4. 4.

   Multiple interactions between robots. Robots use information from other robots, possibly through joint environment manipulation, to choose how to act. More interactions can lead to more informed decisions.

Early analyses of emergent behavior posited that a swarm of $\mathcal{N}$ units only exhibits emergent behavior when $\mathcal{N}$ exceeds a critical size $\mathcal{N}_{c}$ [57]. Below this threshold, swarm attempts at self organization can negatively affect performance (Fig. 1, light blue shaded region). A more generalized definition defines emergence as the positive difference $P(\mathcal{N})-P_{0}(\mathcal{N})$ [58] (Fig. 1, shaded region), where $P(\mathcal{N})$ is the amount of performance achieved by a swarm of $\mathcal{N}$ robots, and $P_{0}(\mathcal{N})$ is the amount of performance achieved by an equivalent swarm of independent agents; this intuition is also mentioned in [35, 12, 13]. In essence, emergent systems exhibit non-linearities in behavior; self-organizing systems also exhibit non-linearities as they acquire and maintain a spatial, temporal, or functional structure [13].

From these observations we develop our definition of emergent self-organization. We reject the argument that the ability of $\mathbb{S}$ to accomplish a complex task is evidence of emergent self-organization. Consider “programmed” emergence in which robot behaviors “fit” together and directly “sum” to the desired outcome—clearly the collective behavior is easily predictable from component study, and therefore not emergent. Instead, we define emergent self-organization as the second order effects of the robot control algorithm; that is, how a complex sequence of robot-environment and robot-robot interactions can give rise to superlinear behaviors (e.g., performance gains) [12]. As an example, consider a control algorithm for cooperatively building 3D structures [59, 60]: the swarm’s ability to collectively build such structures does not in itself indicate self-organization (a single robot can also do it). Nor does it unilaterally indicate emergence: if a single robot takes $T$ seconds to build a structure, and $\mathcal{N}$ robots take $\geq T/\mathcal{N}$ seconds to build the same structure, that is predictable from component study, since increased interference might cause a slowdown. However, if $\mathcal{N}$ robots can build the same structure in less than $T/\mathcal{N}$ seconds, then that is indicative of emergent self-organization (superlinear performance gain).

Our two metric axes (see Table II) are based on characteristics of self-organizing systems, in which the systems are generally far from equilibrium and show an increase in order [13]. We can infer from [15, 13] that SR systems with a high degree of local interactivity should exhibit higher levels of emergent self-organization than systems with a low degree of interactivity [14, 11, 10]. From Fig. 1, we know that emergent behavior can be quantitatively measured as a super-linear increase in swarm performance as $\mathcal{N}$ increases linearly. We argue that sub-linear increases in inter-robot interference across linearly increasing $\mathcal{N}$ are also indicative of emergent behavior, since there may be a correlation between reduced interference and increased performance with self-organization.

We measure spatial self-organization, defined as collective optimization of inter-robot interference and movement patterns, by measuring the sub-linearity of inter-robot interference as $\mathcal{N}$ increases. If the proportional increase in the amount of performance lost due to inter-robot interference between swarms of size $\mathcal{N}_{1}$ and $\mathcal{N}_{2}$, with $\mathcal{N}_{1}<\mathcal{N}_{2}$, is sub-linear, then self-organization has occurred. That is, $\mathbb{S}$ has learned to leverage inter-robot interactions to become more spatially efficient with increasing $\mathcal{N}$. In the absence of self-organizing, random robot motions would produce approximately linear increases in inter-robot interference as the size of $\mathcal{N}$ increases, and be in equilibrium regarding interference levels. In order for spatial self-organization to reliably arise, $\mathbb{S}$ must be operating in a bounded space; without this restriction, a swarm will not necessarily be motivated to manage space efficiently. This restriction manifests itself in applications such as warehouses [41].

We measure task-based self-organization, defined as collective optimization of task allocation, by measuring the super-linearity of the marginal performance gain as $\mathcal{N}$ increases [61]. If a super-linear increase in performance gained due to optimization of higher level swarm functions is observed between two swarms of size $\mathcal{N}_{1}$ and $\mathcal{N}_{2}$, then emergent self-organization has occurred. That is, the swarm has learned to leverage inter-robot interactions and relationships between tasks to become more efficient with larger $\mathcal{N}$. In applications without a bounded operating arena, task-based self-organization is more likely to emerge than spatial self-organization, as it does not directly depend on environmental factors.

Formalizing our intuition, we define our task emergent self-organization measure $E_{S}(\mathcal{N}_{1},\mathcal{N}_{2},\kappa)$ as follows. We calculate (post-hoc) the number of robots experiencing inter-robot interference, instead of doing useful work, within $\mathbb{S}$ on each timestep $t$, denoted as $t_{lost}(\mathcal{N},\kappa)$. This can then be used to compute the per-timestep fraction of overall performance loss $P_{lost}(\mathcal{N},\kappa,t)$ as follows:

Eq. 1 quantifies a swarm’s ability to detect and eliminate non-cooperative situations between agents (e.g., frequent inter-robot interference) as $\mathcal{N}$ increases [49]. For $\mathcal{N}>1$ we subtract the interference that would have occurred in a swarm of $\mathcal{N}$ independent robots which never interfered with each other. If we compute Eq. 1 for two swarms of size $\mathcal{N}_{1}<\mathcal{N}_{2}$, sub-linear increases in the computed value indicate that a swarm of $\mathcal{N}_{1}$ robots is capable of emergent self-organization. We can now define our spatial emergent self-organization measure $E_{S}(\mathcal{N}_{1},\mathcal{N}_{2},\kappa)$, where positive values indicate self-organization, as:

We define our task emergent self-organization measure $E_{T}(\mathcal{N}_{1},\mathcal{N}_{2},\kappa)$ as:

Eq. 3 quantifies the marginal performance gain [61] of the swarm as superlinear increases in self-organization. Positive values indicate self-organization.

In recent years, swarm engineering has produced many design tools and methods for rigorous algorithm analysis and derivation of analytical, rather than weakly inductive proofs of correctness [22, 16, 31, 62, 63, 32]. Despite this, there has not been a corresponding increase in the number of robots used, which is important for tackling problems of real-world scale. Investigation of simple behaviors such as pattern formation, localization, or collective motion is generally evaluated at small scales, with 20–40 robots [64, 23, 22], and more complex behaviors such as foraging and task allocation with 20–30 robots [65, 66, 31]. Notable exceptions to this trend include [62, 6, 48, 20], which used 600, 768, 375, and 16,384 robots respectively.

While these swarm sizes may technically meet the criteria to be a considered a swarm (10–20 robots [1]), from a swarm engineering perspective they may not provide enough confidence in algorithm correctness, since studies at small scales may suffer from unmanifested theoretical and empirical scalability issues. Furthermore, reliably analyzing emergent and self-organizing behaviors during the design process is difficult at small scales since they only reliably arise at larger scales, where “larger” is problem-dependent [52].

Previous work calculated swarm scalability as $S(\mathcal{N})=P(\mathcal{N})/\mathcal{N}$ (i.e., per-robot efficiency) [6]. While this measure provides insight into scalability, it is not predictive (e.g. given $P(\mathcal{N})$, we cannot plausibly estimate $P({2}\mathcal{N})$ without retroactively charting $S(\mathcal{N})$ across a range of values for $\mathcal{N}$).

Our metric axis (see Table II) is based on the intuition that $\mathcal{N}$ robots in $\mathbb{S}$ are analogous to nodes in a supercomputing system: if a job takes $T$ seconds with $\mathcal{N}$ robots, ideally it will take $T/2$ seconds with $2\mathcal{N}$ robots in a perfectly parallelizable system. We adapt the Karp-Flatt metric [67] which measures the level of parallelization of a program and the plausibility of speedups if more computational resources are added, to swarm engineering, where it exposes the portion of observed performance rooted in inter-robot cooperation. Using such a metric, higher values will indicate cooperative swarms and plausible predictions of efficiency increases for larger $\mathcal{N}$.

Formalizing our intuition, we define our scalability measure $\mathcal{C}(\mathcal{N}_{1},\mathcal{N}_{2},\kappa)$, where $\mathcal{N}_{1}<\mathcal{N}_{2}$, using the Karp-Flatt metric:

In Eq. 4, the term in $[~{}]$ is the serial fraction $\mathbf{e}$ from the original Karp—Flatt paper. Our formulation defines “speedup” as the performance gain with $\mathcal{N}_{2}$ robots over the performance with $\mathcal{N}_{1}$ robots, rather than over the performance with 1 robot, per the original paper. This “marginal performance gain” provides comparable results between $\kappa$ which require a critical $\mathcal{N}$ to perform well (i.e., sub-linear increases in performance across small $\mathcal{N}$), and $\kappa$ which have more linear increases in performance across small $\mathcal{N}$. Negative values for $\mathcal{C}(\mathcal{N}_{1},\mathcal{N}_{2},\kappa)$ are possible, due to the fact that the original Karp-Flatt metric assumes that $P(\mathcal{N},\kappa,t)$ is a monotonically increasing function (i.e., adding more processors always gives some marginal gain), and this assumption is not valid in general for SR systems when adding more robots.

Flexibility among agents within a swarm in response to environmental changes is an important aspect of their collective utility: they should be able to react to factors in the environment, such as the quality or safety of locations within it, while also not changing their behavior in response to every fluctuation [12]. Some works on flexibility compare the raw performance of swarm control algorithms under dynamic environmental conditions [21, 18] or conditions from those they were developed in [68, 69, 6]. However, the resultant claims of flexibility due to performance similarity across scenarios is strictly qualitative, due to a lack of mathematical quantification of “difference” in scenario conditions as a factor in performance comparisons.

A swarm $\mathbb{S}$ is considered flexible if it can self-organize to cope with a variety of non-ideal environmental conditions (defined by $V_{dev}(t)$) which change over time and are different than the conditions it was developed in. These changes can include the cost of performing a particular action (e.g., picking up/dropping an object in foraging), maximum robot speed (e.g., wheeled robots buffeted in variable winds in outdoor environments) or the performance of some tasks more slowly than others (e.g., carrying objects of differing weights).

We measure swarm flexibility along two opposing axes (see Table II) in order to encompass a wide range of possible design constraints. First, its reactivity: how quickly and proportionally $P(\mathcal{N},\kappa,t)$ changes according to changes in $V_{dev}(t)$, as shown in Fig. 2. Second, its adaptability: $P(\mathcal{N},\kappa,t)$ does not change, regardless of $V_{dev}(t)$ (i.e., consistent performance). That is, if non-ideal environmental conditions change over time, performance does not change in ideal environmental conditions, as shown in Fig. 2. Adaptive swarms minimize performance losses under adverse conditions ($V_{dev}(t)>0$), and resist performance increases under beneficial deviations from $I_{ec}(t)$, (i.e., $V_{dev}(t)<0$).

Using the results of [70], we derive temporally-aware measures quantifying the difference between expected and observed performance in arbitrary environmental conditions, and use it to calculate swarm reactivity and adaptability.

To derive the performance curve of a swarm with optimal reactivity, $R^{*}(\mathcal{N},\kappa)$, we note that such a swarm will react instantaneously and proportionally whenever $V_{dev}(t)$ is non-zero. Define $c_{t}$ as a per-timestep constant representing the signed proportional difference between ideal and non-ideal conditions:

Then, $P_{R^{*}}(\mathcal{N},\kappa,t)=c_{t}P_{ideal}(\mathcal{N},\kappa,t)$ for all $t\in{T}$, and we can now define our reactivity measure $R(\mathcal{N},\kappa)$:

where $DTW(X,Y)$ is a Dynamic Time Warp similarity measure between two discrete curves $X$ and $Y$, based on the conclusions in [70]. We note that DTW correctly matches temporally shifted sequences of applied variance and observed performance (achieving $R^{*}(\mathcal{N},\kappa)$ is not possible for realistic swarms), and exhibits robustness to signal noise.

It is clear from our intuition above that in a swarm with optimal adaptability, $A^{*}(\mathcal{N},\kappa)$ $P_{A^{*}}(\mathcal{N},\kappa,t)=P_{ideal}(\mathcal{N},\kappa,t)$ for all $t$. We can now define our adaptability measure $A(\mathcal{N},\kappa)$:

where $DTW(X,Y)$ is a Dynamic Time Warp similarity measure, for the same reasons described above for $R(\mathcal{N},\kappa)$.

Bjkernes et al. [45] show that contrary to the common assumption in SR literature, larger swarms can be less robust than smaller swarms, and argue that fundamental changes to SR system design, such as the inclusion of an artificial immune system, are required to detect and self-heal from faults. A generic framework for fault detection in robot swarms based on immunological response and peer voting was presented in [71], and showed a high level of accuracy in identifying a faulty robot in real-robot swarms, even if swarm behavior collectively changed over time. Other works in formation control [72] and decision making [73] treat robustness from the perspective of the ability of $\mathbb{S}$ to meet its goals in the presence of malicious robots, and provide provable collective behavior guarantees, which are by their nature guarantees of emergent behavior. Availability analysis, i.e., determining what fraction of a swarm will be available on average in statistical equilibrium to work on tasks, given robot malfunction and repair rates, has investigated both independent and coupled robot malfunction rates [74].

Swarm robustness in the presence of single robot sensor malfunction was demonstrated in [4], allowing failing robots to share data with functioning robots. Swarm robustness to Gaussian noise was measured and analyzed by [26, 24] on collective swarm motion, and on obstacle detection [25]. Gaussian noise was applied to robot sensor readings in [9], and their effects studied for different variants of a foraging algorithm. The Fokker-Planck equation was derived from statistical physics for a given Langevin equation governing robot motion, including noise models, and applied to the analysis of simple phototaxis (i.e., motion towards light) behaviors [48]. Other works inject a small level of uniform or Gaussian noise (0.5–10%) to help narrow the simulation-reality gap, without performing statistical analysis [6, 66]. Many theoretic works do not incorporate noise into their models [18, 30]; exceptions include [26]. Similarly, many application oriented works do not systematically evaluate algorithms in the presence of noise; exceptions include [9, 24].

We motivate our robustness axes by considering real world examples such as underwater, mining, and warehouses [68, 41]. In such environments, swarms may have to deal with one or more of: (1) high levels of sensor and actuator noise, (2) adversarial task re-allocation in which robots can leave a sub-swarm working on a certain task at any time, and (3) frequent robot mechanical failures. We therefore define a swarm’s robustness along two axes.

First, sensor and actuator noise robustness, or its ability to collectively adapt to temporally varying levels of sensor and actuator noise. Swarm sensor and actuator noise robustness should measure how close $P(\mathcal{N},\kappa,t)$ of $\mathbb{S}$ remains to $P_{ideal}(\mathcal{N},\kappa,t)$ when $V_{dev}(t)$ affects the internal state of $\mathbb{S}$ via sensor and actuator noise, where $P_{ideal}(\mathcal{N},\kappa,t)$ is the performance of $\mathbb{S}$ with idealized, noise-free sensors.

Second, population dynamics robustness, or its ability to collectively adapt to temporal changes in the swarm size $\mathcal{N}$. We emphasize the collective adaptation, as our robustness definition is an emergent property. Swarm population dynamics robustness of $\mathbb{S}$ should measure the ability to maintain the same level of performance as population size changes. As robots enter and leave ${S}$ over time (permanently or temporarily), they will have an inaccurate world model. Different $\kappa$ will cope with these robots in different ways and will ultimately affect $P(N(t),\kappa,t)$ differently. We want to correlate $N(t)$ with changes in $P(N(t),\kappa,t)$. Building on [74], we derive expressions for the average number of robots in the system and how long a robot stays in the system, and use this to define our population dynamics robustness measure.

It is clear from our intuition above that in a swarm with optimal sensor and actuator noise robustness $B_{sa}^{*}(\mathcal{N},\kappa)$, $P_{{B_{sa}}^{*}}(\mathcal{N},\kappa,t)=P_{ideal}(\mathcal{N},\kappa,t)$ for all $t$. We can now define our sensor and actuator noise robustness measure $B_{sa}(\mathcal{N},\kappa)$:

where $DTW(X,Y)$ is a Dynamic Time Warp similarity measure, as described in Section VI for $R(\mathcal{N},\kappa)$ and $A(\mathcal{N},\kappa)$. We compare performance curves, rather than cumulative performance values, in order to account for swarm emergent (partial) mitigation of the noise via changing behaviors.

To compute the variability of $N(t)$, we first derive the average amount of time a robot spends in ${S}$, $T_{{S}}$.

In queueing theory, queues are described by a tuple $A/S/c$ [75]. $A$ is the arrival process, which is typically Markovian, $S$ is the service time distribution, which is also typically Markovian, $c$ is the number of service channels available to service items as they leave the queue. We study the $M/M/1$ queue, in which items arrive at rate $\sim{Poisson(\lambda)}$, remain in the queue for a period of time, and are serviced at rate $\sim{Exp(\mu)}$ [75]. We define three queues to model the possible sources of a temporally varying $N(t)$ resulting from robots leaving ${S}$, extending [74], as follows:

1. 1.

   Birth queue, $Q_{b}$, is an M/M/1 queue defined by $(\lambda_{b}=0,\mu_{b})$, which models the periodic addition of robots to $\mathbb{S}$ at rate $\mu_{b}$ that have been released from other tasks to join ${S}$ to work on task $\mathcal{T}$. $Q_{b}$ begins with $(\mathcal{N}-N(0))$ robots in it and has a finite population; that is, every robot not part of ${S}$ at $t=0$ is a member of this queue.

2. 2.

   Death queue, $Q_{d}$, is an M/M/0 queue defined by $(\lambda_{d},\mu_{d}=0)$ modeling the permanent removal of robots from ${S}$ as they critically malfunction or are permanently reallocated to other tasks and are removed from the swarm at rate $\lambda_{d}$. We treat the two cases identically. There are 0 servers in this queue, and hence $\mu_{d}=0$. $Q_{d}$ begins with 0 robots in it.

3. 3.

   Repair/reallocation queue, $Q_{r}$, is a birth-death M/M/1 queue defined by $(\lambda_{bd},\mu_{bd})$. $Q_{r}$ models the temporary removal of robots from ${S}$ as they (1) non-critically malfunction, or (2) are temporarily reallocated to other tasks at rate $\lambda_{bd}$ (we treat the two cases identically), and return to ${S}$ to work on $\mathcal{T}$ at rate $\mu_{bd}$. $Q_{r}$ begins with 0 robots in it.

Assuming that (1) the service/arrival rates are constant, (2) $Q_{b},Q_{d},Q_{r}$ are all independent, and (3) a robot can be at most in one queue at a time, we leverage the additive property of Poisson random variables and combine $Q_{b},Q_{d},Q_{r}$ into a single M/M/1/$N(t)$/$\mathcal{N}$ queue, $Q_{\bar{S}}$, which models the number of robots not in ${S}$ at $t$.

To have a stable system, the utilization $\rho_{\bar{{S}}}$ must be $<1$:

Depending on the relative arrival and service rates of $Q_{b},Q_{d},Q_{r}$, stability can be achieved in many different ways. Similarly, the variance of the size of $Q_{\bar{S}}$ can also vary greatly as robots stay in the swarm for different amounts of time on average. Using Little’s law [75] to calculate the average amount of time each robot spends in $Q_{\bar{S}}$ we have:

and the total amount of time $T_{\bar{S}}$ that a robot spends not in ${S}$:

We can now compute the amount of time a robot is not in $Q_{\bar{S}}$ (and therefore in ${S}$), by taking $T_{{S}}=T-T_{\bar{S}}$. Higher values of $T_{{S}}$ indicate more stable swarms, and lower values indicate more volatile swarms in which more “experienced” robots will have to work with the sub-optimal actions of less experienced robots which are newly added, have been repaired, or have finished a different task. We now define our population dynamics robustness measure $B_{pd}(\kappa)$ by correlating $T_{{S}}$ with $P(N(t),\kappa,t)$:

Eq. 12 measures the difference between $P_{ideal}(N(t),\kappa,t)$ and $P(N(t),\kappa,t)$, weighted by swarm population variance, $T_{{S}}/T_{{S}_{ideal}}$.

We motivate the need for insightful metrics by considering the design constraints shown below, each of which may be present or absent in specific scenarios such as the two we study in this work. Without precise numerical quantification of the effect of constraints on swarm behavior, then any evaluation of control algorithms will be strictly qualitative, and based solely on iterative refinement and engineer intuition. We show that our metrics encompass all the constraints shown below, and therefore solve the problem of modeling real-world scenario characteristics in a comprehensive way.

1. 1.

   Environmental Disturbances. In many cases, it may not be feasible to develop and test SR systems in the exact environmental conditions in which they will operate (e.g., space, military, disaster cleanup applications) due to cost or locality constraints [76]. Therefore, flexibility in reacting and adapting to unknown conditions is critical.

2. 2.

   Noisy environments. In many environments, sensor and actuator readings may not be accurate beyond the usual level of noise. Levels of noise can vary (1) depending on the robot’s current location in the operating arena, and (2) the current time (e.g., temporally varying sources). In this work, we consider a noise model in which the exact noise level is not known a priori, but its distribution is known or can be reasonably inferred. We further assume that the effects of sensor and actuator noise are spatially homogeneous and affect all sensors and actuators equally.

3. 3.

   Unreliable robots. All robots are fallible, due to the nature of robotic hardware. It is only a question of how often they fail and if the failure is permanent or repairable. Some prior work has considered temporary failures [74, 4]; we expand the scope by considering the effect of temporary and permanent robot failures.

4. 4.

   Semi-adversarial task reallocation. Given the inherent flexibility to adapt to perform different tasks efficiently, it is expected that not all robots within a swarm will be working on the same task at the same time, as different system operators dynamically reallocate robots to tasks in real-time. Previous work in swarm robotics in general, and foraging in particular, assumed that system operators will have exclusive use of the entire swarm, whatever its size is, for the task at hand (i.e., $\mathcal{N}=N(t)$ for all $t$) ([47, 20, 27, 66, 6]). For simplicity of analysis, we assume that all robot reallocations are independent and can happen at any time.

5. 5.

   Operating arena size. If the size of the arena in which $\mathbb{S}$ will operate is known beforehand, variable swarm density is an appropriate model, in which swarms of increasing size $\mathcal{N}$ are placed in the same arena. If the size of the arena is not known beforehand, constant swarm density is more appropriate, in which the size of the operating arena scales linearly with increasing $\mathcal{N}$. Our definition of swarm density loosely falls under the Eulerian continuum for swarm dynamics described in [77]. The choice of variable or constant swarm density is a critical factor in the design process. With variable density, robots in swarms with higher densities will experience more inter-robot interactions than robots in smaller swarms, thus skewing performance results. With constant density, as swarm sizes increase, the arena size also increases proportionally, greatly reducing skewing artifacts. An appropriate model needs to be chosen in either case, as swarm density, not size, is the critical factor in determining swarm emergent self-organization [58, 63].

6. 6.

   Object Distribution. The distribution of the objects to be gathered in the operating arena can be modeled in various ways. The most common distribution is random (Fig. 3a), in which objects are scattered uniformly in a square or circular environment [30, 58, 9, 6]. Other studied distributions include power law (Fig. 3b), in which objects are clustered in groups of various sizes [6]. When the object distribution is not known or cannot be inferred a priori, random or power law are appropriate, following empirical observations [6]. The single source distribution (Fig. 3c) has also been extensively studied [47, 20, 65, 27, 66].

   

   

   

7. 7.

   Minimum collective performance. In SR systems with task reallocation and robotic failures, a common design constraint is a minimum performance that ${S}\in\mathbb{S}$ must meet in statistical equilibrium, as $N(t)$ fluctuates. We use the notion of swarm availability, and define $p_{v}$ as the probability that $N_{min}$ robots from a swarm of size $\mathcal{N}$ are allocated to task $\mathcal{T}$ at time $t$ (for a derivation, see [74]):

   $\displaystyle p_{v}=$

   $\displaystyle\pi_{\mathcal{N}}\big{(}1+\sum_{k=N_{min}}^{\mathcal{N}-1}\prod_{i=k+1}^{\mathcal{N}}\frac{1}{\rho}\big{)}$

   (13)

   Using Eq. 13 we can now solve for either:

   • •

     $\rho$, obtaining a ratio that the rates of robots entering/leaving $\mathbb{S}$ must satisfy if $(p_{v},N_{min})$ are specified.

   • •

     A range of achievable $p_{v}$, given the rates of robots entering/leaving $\mathbb{S}$, and a range of $N_{min}$ values (see Fig. 10a for examples of this analysis).

We have selected four state-of-the-art foraging swarm control algorithms, which in our notation we indicate with $\kappa$. Videos for each $\kappa$ with different spatial distributions of the swarm are included in the multimedia accompaniment to this paper, to provide intuitive grounding of the behavior of each algorithm¹¹1https://www-users.cs.umn.edu/~harwe006/showcase/tro-2021. We selected these algorithms because they use common paradigms for foraging algorithms, from simple reactive random walks to more complex methods with theoretical bases which do task allocation. Comparative analysis across such a wide spectrum of complexity using our metrics will constitute a significant “stress test.” If they provide predictive insights, we will have strong empirical evidence that they capture underlying elements of swarm behavior.

• •

  Correlated Random Walk (CRW). Robots do a correlated random walk, which is a random walk in which the direction of the motion at each step depends on the direction of the previous step [78], until they acquire an object, which they then transport to the nest using phototaxis, i.e., motion in response to light. Robots do not reallocate tasks, i.e., they always do the same task, and have no memory (reactive algorithm). Similar controllers can be found in [35, 11].

• •

  Decaying Pheromone Object (DPO). Robots track objects they have seen using exponentially decaying pheromones [6, 27], and determine the “best” object to acquire using derived information relevance. Robots do not perform dynamic task allocation (i.e., always perform the same task). Similar controllers can be found in [33].

  

• •

  STOCHM. Robots stochastically allocate tasks using the STOCH-N1 algorithm [27], a stochastic choice depending on the location of the most recently executed task within the graph of tasks available to each robot (Fig. 4). From Fig. 4, STOCHM robots can allocate any of the red or blue tasks, which are: (1) the entire task, Forager, retrieving an object and bringing it to the nest, (2) the first half of the task, Harvester, retrieving an object and bringing it to a static intermediate drop site (cache), or (3) the second half of the task, Collector, picking up a block from a cache and bringing it to the nest.

• •

  STOCHX. Robots also use the STOCH-N1 algorithm on the graph in Fig. 4. However, they can now choose red, blue, or green tasks, i.e., they can exploit existing caches and perform tasks related to dynamically creating new caches. The new available tasks are: placing a block somewhere to start a new cache, Cache Starter, placing a block next to a started cache to finish creating it, Cache Finisher, transferring a block between two caches, making progress towards the nest, Cache Transferer, and picking up a block from a cache and bringing it to the nest, Cache Collector. A similar algorithm for mobile caches was developed in [19].

The experiments described in this paper have been carried out in the ARGoS simulator [79]. We employ a dynamical physics model of the robots in a three dimensional space for maximum fidelity, even if robots are restricted to motion in the XY plane, using a model of the marXbot developed by [80]. For all the experiments we average the results of 192 experimental runs of $T=10,000$ seconds. We utilize SIERRA²²2https://github.com/swarm-robotics/sierra.git to automatically generate simulation inputs, run experiments, and generate the graphs seen in this paper.

This scenario, or variants of it, has been studied by [47, 27, 66, 9]. Given the company’s description of their scenario, we define the following design constraints:

• •

  $P(\mathcal{N},\kappa,t)$: Measures the rate of object collection; more efficient $\kappa$ gather more objects per $t$ on average.

• •

  Operating arena: Fixed sizes ($500~{}m^{2}$, $2,000~{}m^{2}$) with variable swarm density.

• •

  Object distribution: Single source for the small warehouse, and random for the medium warehouse.

• •

  Environmental disturbances: Periodic events such as shift changes and deliveries may occur, which we model as square waves, applying throttling functions to the maximum robot speed while carrying an object. This models increased congestion or additional computational time needed to plan safe trajectories in the presence of unknown dynamic obstacles while transporting goods.

  Given that the frequency and disruption of the disturbances is unknown, we test throttling levels $0-0.40$, and set the frequency $f=5000$, modeling periods of intense disruption that appear roughly every 1.4 hours.

• •

  Sensor and actuator noise: Given the tightly controlled indoor environment, there are no unexpected sources of sensor or actuator noise; precise indoor localization methods may also be available. However, no SR system is noise free, so we will investigate the effect of minor levels of Gaussian noise $G(\mu,\sigma)$ with $\mu=0,\sigma=0-0.03$.

• •

  Task reallocations: Given that robots will work alongside humans in both warehouses, we assume that human employees (or a centralized warehouse AI) can reallocate robots at any time as workloads evolve throughout the day. We do not have an estimate of how often this will occur from the company’s problem description, but given that $N_{min}=5$, and 50 robots are available, we can compute the reallocation rates using Eqs. 13, 9 and 10 for the small warehouse, and extrapolate for the medium warehouse as needed.

• •

  Minimum collective performance: Achieved by $N_{min}=5$ robots for all $\kappa$ in the small warehouse, which we extrapolate for the medium warehouse, $N_{min}=5*4=20$.

• •

  Robot reliability: Given the tightly controlled indoor environment and assumed regular maintenance of robots, the overall robotic temporary failure, repair, and permanent failure rates should be negligible, and we set $\lambda_{d}=0$ and $\mu_{b}=0$, as $\mathcal{N}$ is fixed throughout operation.

We use a $32\times 16=512~{}m^{2}$ arena with a single-source object distribution for the small warehouse, and a $48\times 48=2,048~{}m^{2}$ arena with a random object distribution for the medium warehouse, modeling smaller and larger scale order fulfillment operations, respectively. We use the intuitions in Table IV and perform a comparative analysis of the swarm properties, i.e., emergent self-organization, scalability, flexibility, and robustness of the candidate $\kappa$ algorithms described in Section VIII-B to determine if our intuitions are supported by our numerical metric calculations. We omit results where no statistically significant differences are observed, or where the observed trends are the same as those for the shown warehouse.

In Fig. 5a, the level of spatial self-organization remains relatively constant across all swarm sizes, indicating that each $\kappa$ is able to manage increasingly restricted space reasonably well. The small arena size restricts the ability of swarms to learn task parameters by conflating beneficial inter-robot interactions with excessive collision avoidance, and there are no statistically significant differences in the spatial self-organization among the more complex STOCHM and STOCHX algorithms. We see statistically significant differences between CRW and the other algorithms in the small warehouse, however, as its purely reactive nature is relatively unaffected by the conflation.

Spatial self-organization emerges due to sufficiently large perturbations of robot exploratory random walks via interference by robots returning to the nest with objects and the otherwise random motion of the swarm for CRW. In more spatially constrained situations, DPO robots are less random, but compensate for this with memory. STOCHM and STOCHX robots have additional fluctuations due to their stochastic task allocation policies. Overall, these observations demonstrate the importance of the random fluctuations system property necessary for self-organization to occur, as discussed in Section IV-A, and that in spatially constrained environments self-organization can arise even in less “intelligent” algorithms due to the high levels of inter-robot interaction. This suggests that algorithmic randomness is the most important factor in emergence, together with swarm density, which was shown in previous work to be critical to develop self-organization [58].

From Fig. 5b we see few statistically significant differences between the level of spatial self-organization for CRW, DPO, and STOCHM. In this less spatially constrained environment, the swarm must collectively learn more from fewer interactions (i.e., there is less conflation between different types of inter-robot interactions), demonstrating the importance of the multiple interactions system property necessary for self-organization to occur, as discussed in Section IV-A. As expected, the complex STOCHX algorithm is the most adept at doing this via its dynamic use of caches, and obtains a near linear scaling in self-organization as $\mathcal{N}$ is increased.

From Fig. 6, we see much tighter confidence intervals for the simpler CRW and DPO algorithms, and broader intervals for STOCHM and STOCHX, indicating that while more “intelligent” algorithms have the potential to be highly scalable, their scalability is also more stochastic than simpler algorithms. This is in agreement with the intuitions laid out in Table IV: more complex algorithms balance exploring the solution space with exploiting the current solution, resulting in reduced performance and scalability, while simpler algorithms perform more exploitation of the current solution than exploration, resulting in increased performance and scalability.

Between Fig. 5 and Fig. 6 we see strong orthogonality between self-organization and scalability/performance: algorithms which are more “intelligent” are less reliably scalable, though they can be more scalable than less intelligent algorithms. Previous work [27] has shown that STOCHX outperforms the other algorithms in foraging scenarios more conducive to the emergence of strong traffic patterns. The random foraging environment of the medium warehouse is much more adverse, leading to the observed lower performance and scalability. This difference demonstrates the interaction between scenario characteristics and swarm dynamics, the necessity of accurate modeling and thorough analysis when designing SR systems, and the importance of matching algorithms to environments to which they are naturally well-suited.

In Figs. 7 and 8, we see more statistically significant differences between algorithms in reactivity and adaptability for the medium warehouse than for the small one, which is likely due to the conflation of the experimental variables for inter-robot interference and environmental adversity in the small warehouse, as discussed in Section IX-B. We also see further evidence of synergy between our measures in the correlation between the relative levels of spatial and task-based self-organization and flexibility.

STOCHM and STOCHX swarms have the highest levels of self-organization (Fig. 5), and the highest reactivity and adaptability as well for more disruptive $V_{dev}(t)$. The DPO algorithm, while more complex than CRW, is not as reactive or adaptive, demonstrating that adding cognitive capacity to a given algorithm is not guaranteed to make the swarm more flexible. DPO swarms are unable to substantially react to changing environmental conditions because they are fundamentally unable to alter their behavior, which is compounded with increased congestion at shared resources, which manifests as a lack of “intelligence” in changing environmental conditions.

STOCHM performs better due to the more complex task allocation distributions, and STOCHX swarms, as expected, are generally the most reactive and adaptive of all evaluated algorithms. Both STOCHM and STOCHX are also cognitive, suggesting that it is possible to design a complex, cognitive algorithm which is just as flexible as a reactive algorithm—an important insight during the design process. More work is needed to further explore this relationship, but our results suggest that there is a threshold of emergent self-organization for cognitive algorithms which must be met in order to obtain collective performance which consistently outperforms that of simpler biomimetic algorithms.

In the medium warehouse (Figs. 7b and 8b), because the relative levels of inter-robot interference are lower, we see much more consistent trendlines of approximately the same slope for STOCHX swarms, indicating that such swarms are sufficiently “intelligent” to maintain nearly identical marginal reactivity to increasingly adverse environmental conditions (i.e., very small slopes). Other algorithms, having less “intelligence,” show decreasing marginal reactivity (i.e., increasing slopes).

For the small warehouse in Fig. 9a, the CRW and STOCHX algorithms generally show statistically equivalent robustness. Of all the candidate algorithms, CRW is the least reliant on the accuracy of sensors/actuators of all algorithms, while STOCHX has a high level of self-organization—both equivalent means of achieving robustness to sensor and actuator noise. DPO and STOCHM likewise are not statistically different in terms of sensor and actuator noise robustness, except at higher $\sigma$. We see a clearer separation between algorithms in the medium warehouse in Fig. 9b, again likely due to the reduced conflation between different types of inter-robot interactions. In both Figs. 9a and 9b, we also see strong evidence of “phase transitions” of the behavior of all algorithms with $\sigma\geq 0.017$. The transition is much more pronounced for DPO and STOCHM, in which the ability of these algorithms to handle injected noise begins to break down—a crucial insight in the design process.

To analyze population dynamics robustness, we use $\mathcal{N}=50$ for the small warehouse, per the company’s specification, and Eq. 13 to solve for $p_{v}$, given desired steady state $N(t)=\{5,10,20\}$, shown in Fig. 10a. Given that sufficient performance can be achieved (per scenario specification) with $N_{min}=5$ robots for any algorithm, we see that this corresponds to $p_{v}\sim{}0.40$, meaning that approximately 40% of the time all 5 robots of a given team were available to work on filling an order. For the medium warehouse, we use $\mathcal{N}=200$ and $N_{min}=20$, extrapolating from the company’s specification, and solve for $p_{v}$ in the same manner, shown in Fig. 10b. With $\mathcal{N}=200$, the availability curves drop off much more steeply, and we must have larger team sizes to achieve the same level of availability as the small warehouse.

We again see the same dichotomy between STOCHM/DPO and STOCHX/CRW in the small warehouse in Fig. 11a for higher population variances: the emergent self-organization of the STOCHX algorithm makes it the most robust to fluctuating swarm sizes, while other algorithms struggle. We speculate that the dynamic use of caches by STOCHX allows additional information about the current swarm task distribution to be encoded into the environment, which can then be partially recovered by returning/new robots, per our intuition. In Fig. 11b, we see much stronger evidence of synergy between our metrics: the relative ordering mirrors the level of emergent self-organization, with the caveat of a possible lack of statistically significant differences between STOCHM and CRW.

We have discussed swarm self-organization, scalability, flexibility, and robustness for the warehouse scenario for all $\kappa$. It would now be up to the company to take our results and determine which of the four main swarm properties are most important to them to make a final algorithm decision.

Overall, we see strong evidence in this section’s analyses of the synergistic nature of our metrics for SR system properties: trends within and between curves on graphs of each can be intuitively explained, and easily mapped back to the technical specifications of a given method in ways that are not evident from raw performance curves. We have observed strong correlations between emergent self-organization, scalability, flexibility, and robustness, and our intuitions in Table IV.

This scenario, or variants of it, has been studied by [81, 7]. We define the following design constraints:

• •

  $P(\mathcal{N},\kappa,t)$: measures the rate at which objects are picked up for the first time; more efficient $\kappa$ will discover more objects faster.

• •

  Operating arena: Variable size with constant swarm density; size of the arena in which targets can be found is not known beforehand. Given the limited sensing capabilities of the robots, and the anticipated robot losses due to malfunction or environmental issues, relatively large swarm sizes should be tested.

• •

  Object distribution: Not known a priori, so random or power law are appropriate models; we choose power law. Objects can move randomly over time; it is not known how often this happens so we evaluate performance with $p_{rw}=0-0.051$.

• •

  Environmental disturbances: Potentially large gradually evolving environmental disturbances during swarm operation, such that the swarm operating conditions will change from favorable to unfavorable or vice versa, and back again throughout swarm operation (e.g., a passing storm, windy conditions, change in visibility, etc). We apply throttling functions to the maximum robot speed during operation, and use a sinusoidal disturbance model with throttling amplitudes $0-0.4$ and frequency $f=10000$, modeling cyclically shifting environmental conditions roughly every 2.75 hours.

• •

  Sensor and actuator noise: Environmental disturbances may occur, such as clouds interfering with localization, fluctuating lighting conditions interfering with perception, etc. Given our hardened robotic hardware, we select a Gaussian noise model $G(\mu,\sigma)$ with $\mu=0$ as an appropriate model, and test with $\sigma=0-0.1$.

• •

  Robot reliability: Environmental disturbances and the general unpredictability of operating in an outdoor environment may give rise to software or hardware artifacts which will cause robots to fail permanently, so we test $\lambda_{d}=0-0.0007$.

• •

  Task reallocations: No unexpected task reallocations, due to the time sensitivity of the application, so we set $\lambda_{bd}=\mu_{bd}=0$.

We develop the intuitions shown in Table V, and perform a comparative analysis of the emergent self-organization, scalability, flexibility, and robustness of the candidate $\kappa$ described in Section VIII-B to determine if our intuitions are supported by our numerical measurements. We omit results where no statistically significant differences are observed.

In Fig. 12, we see a strong dichotomy: STOCHX and CRW swarms show consistent levels of self-organization across swarm sizes for both high and low rates of target motion, while the self-organization of DPO and STOCHM swarms is much more sensitive to swarm sizes across both target motion rates. As swarm sizes increase, it naturally becomes easier to find moving targets through sheer numbers rather than intelligent algorithm design. At smaller swarm sizes, STOCHX exhibited the strongest trends (marginally), and was therefore able to overcome some of the world model errors introduced by target motion through the emergence of specialized task allocation distributions, dynamic cache usage, and bucket brigading, while the (partially) random walk used in CRW also proves an effective strategy, despite its simplicity.

DPO relies heavily on static target locations, while STOCHM relies on both static target and static cache locations. As a result, much of the learning of STOCHM swarms to utilize caches is wasted due to the unconstrained nature of the environment and target motion. Overall, the strategies used by these algorithms are much less effective when compared with STOCHX and CRW, as we would intuitively expect.

From Fig. 13, we observe a negative correlation between emergent self-organization and performance, in contrast to [27]. This suggests that the correlation between self-organization and performance is highly dependent on the nature of the environment and that in dynamic or extremely adversarial environments, less “intelligent” algorithms may perform better, as we would expect.

The extremely adverse power law object distribution in this scenario is not conducive to self-organization because of its irregular block clusters. In combination with a constant swarm density and target motion this leads to no statistically significant differences in scalability between the algorithms. This result is in contrast to [27], in which differences in emergent self-organization were positively correlated with statistically significant differences in scalability. This again suggests that positive correlation only holds in static environments, and that in dynamic and adversarial environments, “intelligence” plays a minimal role in determining scalability.

Having analyzed the effect of target movement rate, we set a moderate rate of $0.001$ to simplify the rest of our analyses.

In Fig. 14a, we see the same dichotomy observed in Sections X-B and X-C: CRW and STOCHX are the most reactive algorithms, and DPO and STOCHM are the least. However, this dichotomy only emerges for higher levels of environmental disturbances; for smaller levels, the algorithms are largely statistically equivalent. Overall, as expected, the CRW and STOCHX algorithms are the most reactive, due to their memory-less nature and ability to dynamically utilize caches, respectively. DPO and STOCHM, which rely on static object and cache locations, are much less reactive.

In Fig. 14b, we see fewer significant differences between algorithm adaptability, regardless of the level of environmental disturbance, with the exception of CRW showing significant differences with DPO and STOCHM for higher levels of environmental disturbance. In the presence of randomized target motion, we expect it to be difficult for algorithms to also adapt to changing environmental conditions. In this instance, the memory-less CRW algorithm is the only algorithm able to statistically differentiate itself from the other evaluated algorithms.

In Fig. 15b, we see identical trends for each individual $\kappa$, as well as identical trends between $\kappa$. The robustness of CRW and STOCHX swarms to sensor and actuator noise is statistically identical in most cases for increasing $\sigma$. For CRW swarms this is likely due to their low reliance on the accuracy of sensors and actuators during operation: the more reliant a swarm control algorithm is on accurate sensor and actuator information, the less robust it will be to adverse environmental conditions which negatively affect sensors and actuators. For STOCHX swarms this is likely due to their ability to be flexible and switch tasks dynamically to attempt to mitigate the noise (e.g., less transporting of objects over long distances, which is more prone to localization errors).

DPO, STOCHM, and STOCHX swarms have noticeably lower robustness for larger values of $\sigma$, and undergo phase changes at $\sim{\sigma=0.01-0.05}$, with the overall robustness dramatically dropping for larger $\sigma$. This is the inflection point at which the combination of increasing noise and moving objects overwhelms the swarm’s cognitive ability to compensate; it occurs at much lower $\sigma$ for STOCHM swarms than those for DPO and STOCHX, due to their usage of static caches as they attempt to adapt.

In Fig. 15a, CRW swarms exhibit a high level of population dynamics robustness, due to their memory-less nature; similarly for STOCHX swarms with their ability to encode the current task distribution into environmental state for inexperienced robots to find via caches. DPO and STOCHM, lacking such abilities, have much lower robustness. For the omitted graph with $\mathcal{N}=320$, statistically identical responses to pure death dynamics follow directly from our intuition: with large swarm sizes, the loss of an individual robot affects the swarm’s overall progress on its task much less. However, this observation is likely tied to the extremely adversarial nature of the scenario, and we would reasonably expect differences to emerge in a more favorable environment.

Our results in this section suggest strong synergy between our robustness and flexibility measures, similar to what was observed for the first scenario. We have again seen quantitative confirmation of our intuitions for each $\kappa$ through our derived metrics for each of the swarm properties of self-organization, scalability, flexibility, and robustness. It would now be up to project managers to take our recommendations and make a final decision. However, our results suggest that the STOCHX algorithm is, on the whole, best suited to the outdoor search and rescue application.