Multi-level Adaptation of Distributed Decision-Making Agents in Complex Task Environments

Since agents face binary choices, there are $S^{2}$ possible solutions for each subtask. Agents are limited as they do not know the entire solution space within their subtask. Initially, they only know a subset of $Q\in\mathbb{N}<S^{2}$ solutions. Agents are homogeneous concerning the capacity $Q$ and heterogeneous with respect to the $Q$ solutions they actually know. We endow agents with the capability of adapting to their subtask environment that is defined by their decision problem $\mathbf{d}_{m}$ (see Fig. 1). With probability $p$, agents randomly learn an unknown solution to $\mathbf{d}_{m}$ that differs in the value of only one decision ${d}_{n}$ from the set of currently known solutions. Since capabilities are limited, with probability $p$, agents also erase from their memory (i.e., forget) solutions they do not frequently use due to the low level of associated utility. The implemented adaptation follows an autonomous learning approach that can be found in previous implementations of the NK-framework [10].

We assume that the population of $P$ agents is equally distributed across $M$ subtasks so that for each subtask, there are $J\in\mathbb{N}=P/M$ agents who can find a solution to this subtask. Consequently, $J$ agents compete for a slot in the team; and all $P$ agents autonomously organize themselves in a team finally composed of $M$ agents. The self-organizing process follows the concept of a second-price auction and always occurs at timestep $t=1$. Afterwards, $\tau-1$ other auctions are held over the remaining $T-1$ time steps in regular intervals, so for $t>1$, auctions are held each time $t\mod\frac{T}{\tau}=0$. This means that $\tau$ auctions take place over the considered time horizon $T$ each $\frac{T}{\tau}$ time steps.

We implement this second-price auction process in the following way: To become a team member, agents place bids. These bids represent what agents intend to contribute to team performance. Since the selection mechanism follows a second-price auction, agents have incentives to reveal their true contributions [19]. Agents, however, cannot observe the other agents’ bids and this is why they assume the residual decisions $\mathbf{D}_{mt-1}$ will remain constant from implemented solution at $t-1$. Agents compute the expected utilities for all their solutions to their subtasks $\mathbf{d}_{m}$ available in time step $t$ according to $U(\mathbf{d}_{mt},\mathbf{D}_{mt-1})$ (see Eq. 4). Each agent submits the highest attainable expected utility among the different solutions as their bid in $t$. Consequently, there are $J$ bids for each slot in the team. Following the logic of second-price auctions, the team comprises those agents $M$ who submit the highest proposals per slot. For being a part of the team, agents are charged the second-highest bid per slot.²²2Since agents only experience utility if they are team members, they always have incentives to participate in this process.

In each time step $t$, the $M$ team members are tasked with finding a solution $\mathbf{d_{t}}$ to the complex task $\mathbf{d}$. Agents are autonomous in their decisions, and there is no communication or coordination between agents. The decision-making process is a two-stage process. First, each agent $m$ focuses on their subtask $\mathbf{d}_{mt}$ and computes the expected utility for each solution available in this particular time step. Since we do not allow for communication, the agents assume that the residual decisions do not change compared to the previous period $\mathbf{D}_{mt-1}$. The expected utility is computed by $U(\mathbf{d}_{mt},\mathbf{D}_{mt-1})$ (see Eq. 4) for each solution known. The agent chooses the solution that promises the highest utility. In the first step, each agent comes up with an $S$ binary values vector that represents the solution to their subtask. Second, once all agents have submitted their solutions, the overall solution is computed by concatenating each of the $M$ solutions chosen by the agents. The solution to the complex problem in $t$, $\mathbf{d}_{t},$ is thus represented by a vector of $N$ binary values. We compute the performance of the overall solution according to Eq. 3, and agents receive the resulting utility according to Eq. 4.

An overview of the model’s variables and the values they can take is given in Tab. 1. With the values provided in Tab. 1 for the number of auctions that occur at each simulation, we identify three different scenarios. First, for $\tau=1$, there is initial collective adaptation: The auction process occurs only at the first period, never to be repeated. Second, for $\tau=20$, there is moderate collective adaptation, with the team self-organizing each $\frac{T}{\tau}=10$ time steps. Finally, for $\tau=200$, there is high collective adaptation, with the team self-organizing at each time step.

Fig. 3 provides an overview of the events of the model in order.

During each simulation run $r$ and timestep $t$, the team performance $\Phi(\mathbf{d}_{t})$ is normalized by the highest possible performance in that simulation run, $max(\Phi_{r})$. Based on this measure, we compute the average performance of the scenario for each timestep $t$ as follows:

In Fig. 4, we plot the Manhattan Distance for each studied scenario. The x-axis of each subplot represents the probability of individual adaptation. On the y-axis, we plot the three scenarios considered for collective adaptation (see Sec. 2, Scenarios, variables and sequencing of the model). The three subplots represent the results for different complexity levels (see Fig. 1). Each point of the subplots represents a total Manhattan Distance MD for a particular configuration of individual and collective adaptation. A point located in a lighter (darker) area implies lower (higher) MD, and thus a better (worse) performance.

The results suggest that the probability of individual adaptation $p$ has a considerable effect on task performance. In particular, we observe a substantial decrease in the total Manhattan Distance MD, when agents are capable of learning, i.e., when $p>0$. This pattern is robust across all considered interdependence structures (i.e., for each level of $K$ considered), as shown in all three contour plots of Fig. 4. The effect of agents being capable of learning on performance is highly significant, suggesting that promoting learning, even if it is just minimal, is always beneficial for task performance.

Regarding $p>0.1$, it can be observed in low complexity scenarios (i.e., the left plot of Fig. 4) that there is a decreasing marginal positive effect of individual adaptation on performance: Each successive increase in $p$ increases task performance less than the previous increase did. For example, the effect on the average task performance of increasing the probability of individual adaptation from $p=0.1$ to $p=0.5$ is lower in magnitude than the effect of increasing the probability from $p=0$ to $p=0.1$ for all three collective adaptation scenarios considered. Starting to learn about a task has huge implications on performance. However, each increase that occurs in individual adaptation reduces the overall impact of individual learning on performance, even if this effect is still positive.

However, for higher levels of complexity (see the central and right panels of Fig. 4), this positive effect eventually turns negative, and the overall performance declines as a result. For example, in the central plot of Fig. 4 (i.e., for $K=5$) we observe how a team with high collective adaptation reaches its lowest total distance of $MD=20.00$ at a learning probability of $p=0.1$. Further increases in individual adaptation appear to affect performance negatively, surpassing $MD=25.00$ for $p=0.5$. Eventual decreases in performance can also be observed for moderate collective adaptation and for initial collective adaptation (both for $p>0.4$). This pattern of initial growth but with an eventual decline in performance is even more pronounced in scenarios of high complexity (i.e., $K=11$, see the right panel of Fig. 4). Results imply that, for moderate or high levels of complexity, individual adaptation and task performance are related in the form of an inverted-U: Initial increases in individual adaptation are positive for task performance, although the positive effect decreases in magnitude with each successive increase. Eventually, the relationship turns negative and performance is harmed as a consequence of increasing individual adaptation.

The results suggest that whether collective adaptation has a positive or negative impact on performance depends on the complexity of the scenario studied. For low levels of complexity (represented by the left panel of Fig. 4), we can observe that high collective adaptation is associated with higher performance as compared to scenarios with either moderate or initial collective adaptation (except for $p=0$, in which all three teams perform very similarly, and $p=0.1$, in which high and moderate collective adaptation perform similarly). Moreover, the highest possible performance in low complexity scenarios considered is attained by a team exhibiting high collective adaptation, reaching a total distance below $MD=1.00$ for $p=0.5$. When facing a task that is not highly complex, it is beneficial in terms of performance to increase the effort dedicated to finding new solutions (i.e., increasing exploration), at both the individual and collective level.

However, in scenarios with medium complexity (represented by the central panel of Fig. 4), for most values of $p$, moderate collective adaptation leads to higher performances compared to both high or initial collective adaptation scenarios. There are two exceptions to this, which are for the extreme values of $p=0$ (in which moderate and high collective adaptation perform very similarly) and $p=0.5$ (in which moderate and initial collective adaptation perform very similarly). For $0<p<0.5$, moderate collective adaptation is associated with a higher performance than any other alternative. Moreover, the highest attainable performance in all medium complexity scenarios considered occurs for moderate collective adaptation and $p=0.4$ and is around $MD=15.00$. In contrast to low complexity scenarios (except for the extreme case of $p=0$, in which agents do not learn at all), high collective adaptation is always associated with the lowest performance compared to the other two alternatives. The results suggest that increasing exploitation (exploration) excessively via decreasing (increasing) collective adaptation can be harmful to performance. Moderate collective adaptation, in turn, benefits performance in the majority of cases.

Results in scenarios of high complexity, as shown in the right panel of Fig. 4, suggest that increasing collective adaptation is detrimental to performance when individual learning occurs (i.e., for $p>0$). An exception to this occurs in the extreme case of $p=0.5$, in which initial and moderate collective adaptation are associated with a similar performance around $MD=43.00$. For $0<p<0.5$, initial collective adaptation is associated with a higher performance than any other of the alternatives. Furthermore, the highest attainable performance for high complexity scenarios is attained by a team with initial collective adaptation and $p=0.2$, for approximately $MD=38.00$. When tasks are highly complex, stability in team composition is favoured, since the recurrent reorganization of a team eventually impairs task performance.

This research is concerned with multi-level adaptation and its relationship with task performance in teams. Results indicate that, in low complexity environments, putting a lot of effort into adaptation at both the collective and the individual level is not detrimental in terms of performance. This extends the insights given by [11] about adaptation in complex environments: When facing a task of low complexity, high exploration does not decrease the overall task performance, since there are lower chances of ending in sub-optimal, long-term solutions. Since only a single level at which adaptation occurs was considered in [11], our results contribute to previous research by showing that this is the case also for multi-level settings.

For moderate levels of complexity, our results show that the exploration vs exploitation dilemma can also be applied in a multi-level adaptation setting; and that a proper balance between exploration and exploitation is the key to improving task performance in teams. Thus, these insights are in line with previous research [11, 13, 15, 21] regarding the exploration vs exploitation dilemma. The main difference to the previous literature is that, in our research, exploration can be increased either by increasing individual or collective adaptation. In particular, when considering moderately complex tasks, the results suggest that for performance to improve, moderate individual learning has to be combined with moderate collective adaptation. The reported findings are consistent with previous results, which indicate that recurrent self-organization of teams combined with individual learning can be associated with higher task performance [8]. In real-life settings this moderate adaptation at both levels could be achieved, for example, by allowing employees of a firm to dedicate a limited amount of working hours to self-education and training; combined with holding regular team meetings with employees outside the team to discuss the state of the task and what other members could contribute to it.

Given these results for low and moderate complexity, it can be concluded that recurrent self-organization can be beneficial to task performance. These insights are in contrast with views in the field of management science that characterize team reorganization as harmful to performance [9]. In particular, team stability has been associated with high degrees of team-level learning and, eventually, higher performance [9]. In this context, adaptation occurs just at the team level, following a learning process for the collective. This highlights the importance of considering multi-level adaptation, as done in our research. By considering multiple, simultaneous adaptation processes (in this case, by endowing agents with the capabilities of learning and combined with recurrent team self-organization), emerging insights into topics such as team reorganization differ from those found in the previous literature. Since adaptation has been considered as occurring at multiple levels in the previous literature [13], we believe that these findings and similar implementations help us to understand better the relationship between adaptation and task solving in teams and to relate it to real-life problems.

Our results also contribute to the literature on multi-level adaptation and task performance when considering highly complex tasks. Previous research on multi-level adaptation and task performance (which is, in itself, not extensive) has not considered highly complex tasks (see, for example, [8]). Thus, we can consider our results as a first step in the matter of studying multi-level adaptation under high complexity. We find that, in highly complex tasks, collective adaptation is negative for performance, and that the positive effect of individual adaptation is limited considerably. As the previous literature shows, highly complex environments are associated with lower benefits for exploration, since the probability of finding sub-optimal, long-term solutions is very high [11]. Again, our results can be understood as extensions to previous insights on the exploration vs exploitation dilemma by considering multi-level adaptation.

This set of results provides an extension to approaches that consider a single level of adaptation [11, 13, 15, 21]. The value of our research lies in the fact that it provides a discussion on individual and collective adaptation, both separately and simultaneously. To our knowledge, there is a lack of an extensive implementation of a multi-level adaptation approach that considers autonomous individual learning and team self-organization in management science. By considering these two aspects, we believe that this research provides a novel approach to multi-level adaptation in this field. Our results should then be understood as both complementary to previous findings in the literature and as an extension to them [11, 21].