A Computationally Efficient Bi-level Coordination Framework for CAVs at Unsignalized Intersections

In this paper, we consider a typical single-lane intersection scenario which consists of 4 roads, i.e., down road, right road, up road, and left road, as shown in Fig. 2. On each road, there are three possible paths that vehicles can follow: go straight, turn left and turn right, and there is a total of 12 different paths within the intersection area. The central area where multiple paths cross and merge is called the conflict area (CA). A centralized coordinator is deployed at the RSU to store road structure information (e.g., intersection’s geometric parameters, pre-defined paths, etc), collect vehicles’ status information (e.g., positions, velocities, accelerations, and routes, etc), and give instructions to vehicles to cross the intersection safely and efficiently. $\mathrm{C}_{\mathrm{DL,RD}}$ represents the collision region of two conflicting paths: ego vehicle’s path $\mathrm{D}\rightarrow\mathrm{L}$ and high-priority vehicle’s $\mathrm{R}\rightarrow\mathrm{D}$. The calculation of all collision regions will be discussed in the next subsection.

Without loss of generality, we assume that vehicles must follow one of the 12 pre-defined paths according to their driving directions and only the longitudinal movements of vehicles are controlled. Hence, the dynamics of each vehicle $i\in\mathcal{N}$ can be modeled as a double integrator, i.e.,

where $s_{i}(t)$ denotes the position along its path, $v_{i}(t)$ and $u_{i}(t)$ are the longitudinal speed and acceleration. Considering the practical constraints in the vehicle dynamics, we do have:

here $[s_{i}^{0},v_{i}^{0},u_{i}^{0}]$ is the initial state of vehicle $i$, which should be collected by the coordination center at the beginning.

Collision region modeling is essential for coordination safety and efficiency. To fully utilize the spatial resources of the intersection area, we propose a novel collision region model by leveraging collision-checking algorithms. Fig. 3 demonstrates the calculation of collision region between two conflicting routes $\mathrm{D}\rightarrow\mathrm{L}$ and $\mathrm{R}\rightarrow\mathrm{D}$. It is worth noting that $\mathrm{C}_{\mathrm{DL,RD}}$ and $\mathrm{C}_{\mathrm{RD,DL}}$ are two different collision regions since the ego vehicle is on different paths. Take $\mathrm{C}_{\mathrm{DL,RD}}$ as an example, we can locate the ego vehicle at a given point $s_{\mathrm{ego}}$ on path DL, and let conflicting vehicle travel at a constant speed along path RD. We then use collision-checking algorithms such as Separating Axis Theorem (SAT)[34] to check if a collision occurs. By iteratively locating the ego vehicle from one side to another along path DL, we can get the exact collision boundary $s_{\mathrm{DL,RD}}^{\mathrm{in}}$ that the ego vehicle cannot enter before the conflicting vehicle passes through. In this case, the ego vehicle can enter the collision region $\mathrm{C}_{\mathrm{DL,RD}}=[s_{\mathrm{DL,RD}}^{\mathrm{in}},s_{\mathrm{DL,RD}}^{\mathrm{out}}]$ if and only if the high-priority vehicle pass its collision region $\mathrm{C}_{\mathrm{RD,DL}}=[s_{\mathrm{RD,DL}}^{\mathrm{in}},s_{\mathrm{RD,DL}}^{\mathrm{out}}]$. This calculation process is time-consuming but can be pre-computed since road structures and paths are fixed. The collision table $\mathcal{T}_{\mathrm{collision}}$ consisting of all collision regions can be obtained from Algorithm 1. Herein, each pair of paths intersect with each other at most once and we do not consider the self-collision of two identical paths.

In this section, we first introduce a classical centralized coordination strategy based on the CS model[16], as shown in Fig. 1(a). Considering that any two conflicting vehicles must cross the collision set one after another, the lateral collision-free constraints can be written as:

where $\mathcal{N}_{\mathrm{con}}^{i}$ denotes the set of all conflicting vehicles of vehicle $i$, $s^{\mathrm{in}}$ and $s^{\mathrm{out}}$ are the entry and exit point of the collision set, respectively. For a given initial configuration, the exact time that a vehicle occupies the collision set is determined by the passing order and trajectory planning results. However, the optimal ordering for a given initial configuration can only be found by evaluating all feasible passing order alternatives in a structured manner. Consequently, constraint (6) is essentially NP-hard, and it is necessary to seek low-complexity relaxations. One common approach is introducing auxiliary binary variables, i.e., $\delta_{i}(t),\gamma_{i}(t)\in\{0,1\}$, to indicate the priority of each vehicle. For instance, $\delta_{i}(t)=1,\gamma_{i}(t)=1$ indicates that vehicle $i$ is inside the CA, while $\delta_{i}(t)=0,\gamma_{i}(t)=1$ or $\delta_{i}(t)=1,\gamma_{i}(t)=0$ means that vehicle $i$ has not yet reached the CA or has already passed the CA. $\delta_{i}(t)=0,\gamma_{i}(t)=0$ is invalid state. Thus, we can replace constraint (6) using the following linear inequalities:

where $M$ is a sufficiently large number.

To guarantee no rear-end collision occurs between vehicles on the same lane, we impose the following rear-end safety constraint:

where $j\in\mathcal{N}_{\mathrm{pre}}^{i}$ is the preceding vehicles in front of vehicle $i$, $L_{\mathrm{safe}}$ is the longitudinal safety gap.

Thus, the centralized-CS problem with the aim of maximizing traffic throughput can be formulated as:

Similarly, we can formulated a centralized optimal coordination problem (OCP) based on our collision region model described in Section II-B. The lateral collision avoidance constraints are as follows,

where $c\in\mathcal{C}_{\mathrm{con}}^{i,j}$ is the corresponding collision region between vehicle $i$ and vehicle $j$, $s_{i,c}^{\mathrm{in}}$ and $s_{i,c}^{\mathrm{out}}$ are the entry and exit point of collision region $c$. As a result, the centralized optimal coordination problem (OCP) can be formulated as follows,

Problems $\mathscr{P}_{\mathrm{centralized-CS}}$ and $\mathscr{P}_{\mathrm{centralized-OCP}}$ can be cast into Mixed-integer Quadratic Programming (MIQP) problems, which can return optimal inputs $u$ and priority sequence $\gamma,\delta$ simultaneously. However, MIQPs are generally time-consuming and the problem size may grow exponentially with the number of vehicles.

Although the aforementioned centralized strategies are able to find global optima by jointly optimizing cooperative trajectories and priority sequence, they scale poorly and are somehow not directly applicable in industrial practice. In this work, we propose an efficient bi-level coordination framework, as shown in Fig. 4.

The centralized node (high-level planner) is deployed to collect system-wide state information of all involved vehicles. Based on the initial configuration, the high-level planner only solves the priority scheduling problem and then assigns priority to vehicles. Note that adjacent non-conflicting vehicles may be assigned with the same priority, while conflicting vehicles must be sequentially arranged. The low-level planner of each vehicle thereafter solves its local trajectory planning problem in sequence to avoid collisions with high-priority vehicles. Then, the optimized trajectory is transmitted to low-priority neighbors for their planning problems.

The key issue to be addressed is decomposing the optimal coordination problem into the priority scheduling and distributed trajectory planning. The quality (i.e., traffic efficiency) of a given priority sequence can only be evaluated after all vehicles have planned their trajectories, which means priority scheduling is closely intertwined with trajectory planning. Rule-base approaches[25, 26, 27, 28] greedily find a feasible priority sequence without evaluating the quality of the solution which may, however, greatly sacrifice traffic efficiency. Enumeration-based approaches[22] find the globally optimal solution by evaluating all possible alternatives. However, they are not applicable in practice since the priority scheduling itself is a combinatorial optimization problem and iterative trajectory planning is also time-consuming.

In the following two sections, we will proposed a promising scheme to address these challenges. We start our exposition with the low-level trajectory planner in Section III and then we discuss the high-level priority scheduler in Section IV.

The high-level planner aims at finding an optimal priority sequence from a systematic point of view. The priority assignment problem can be mathematically written as the following combinatorial optimization problem:

where $t_{\mathrm{leave}}(\mathbf{o})$ denotes the total passing time the vehicles follow a given priority sequence $\mathbf{o}$, $\mathcal{O}$ is the set of all possible priority sequences. However, the calculation of $t_{\mathrm{leave}}(o)$ is time-consuming. Specifically, for a given priority sequence $o$, $t_{\mathrm{leave}}(o)$ can only be obtained after all vehicles have planned their trajectories. This direct evaluation approach takes over-long problem-solving time in real-world implementations. For example, if a vehicle needs $\tau_{c}$ seconds to plan its local trajectory, the evaluation of a given priority sequence of $N$ vehicles consumes $N\tau_{c}$ seconds. To find the optimal priority sequence, we need to traverse all $N!$ solutions, which requires about $N!N\tau_{c}$ seconds. To address this issue, we present an efficient evaluation method, as shown in Fig. 6.

The speed profiles in Fig. 6 are approximated by straight lines, i.e.,

where $t_{\mathrm{acc}}^{i}$ and $t_{\mathrm{end}}^{i}$ are the starting time and existing time, respectively. The straight line with slop $v_{\mathrm{max}}$ can be denoted as $s=v_{\mathrm{max}}t+b$ and must below all the blocked regions, such that

To avoid rear-end collisions with preceding vehicle on the same lane, we have

where $b_{\mathrm{pre}}^{*}$ is the optimal parameter of preceding vehicle, $L_{\mathrm{safe}}$ is the safety distance. Hence, $t_{\mathrm{acc}}^{i}$ and $t_{\mathrm{end}}^{i}$ can be calculated as follows,

Also, the occupation time of the collision regions along its path can be calculated in the same way. For a given priority sequence $\mathbf{o}$, $t_{\mathrm{leave}}(\mathbf{o})$ can be calculated as:

Though the simplified speed profiles do not satisfy kinematic constraints (1) - (2), they can evaluate the quality of a priority sequence simply using simple calculations.

Problem $\mathscr{P}_{\mathrm{priority}}$ is still challenging and it is typically impossible to traverse all priority sequences within the limited computational budget, especially when a large number of vehicles within the coordination area. In the following, we present an MCTS based approach which could efficiently find high-quality sequences within a short planning time. The effectiveness of MCTS has been shown in many sequential decision-making problems such as games, planning, and control[35, 36, 37].

The key idea of MCTS is to iteratively build a search tree by estimating the values of tree nodes through random simulations [35]. The computational budget (e.g., planning time constraint or iteration constraint) is pre-defined, such that MCTS can always be stopped and return a solution, though might not be optimal. Four steps are applied per search iteration: selection, expansion, simulation, and backpropagation, as shown in Fig. 7. Firstly, select the most valuable and expandable node, i.e., vehicle $\mathrm{R}_{1}$, according to a certain tree search policy. Secondly, expand a new node that can be selected in the next round, i.e., $\mathrm{U}_{1}$. Thirdly, simulate by randomly sampling a feasible priority sequence, e.g., $(\mathrm{L}_{1},\mathrm{R}_{1},\mathrm{U}_{1},\mathrm{D}_{1},\mathrm{D}_{2},\mathrm{U}_{2})$ shown in Fig. 2. Calculate the cost $t_{\mathrm{leave}}$ according to (15) - (18) and convert it into reward $r$ using a certain reward function. Then, backpropagate the current reward $r$ to the selected node $\mathrm{U}_{1}$ and its ancestors $(\mathrm{root},\mathrm{L}_{1},\mathrm{R}_{1})$.

In this paper, we adopt the commonly used tree search policy called Upper Confidence Bounds for Trees (UCT), which can be expressed as:

where $p$ is a parent node, $\mathcal{A}_{p}$ is the set of possible children (the first uncoordinated vehicle on each lane), and $c$ is a child of parent node $p$. $R_{c}$, $n_{c}$, $n_{p}$ represent the cumulative rewards of child $c$, the number of visits of child $r$, and the number of visits of parent $p$, respectively. The first term of (19) is called the exploitation term, which means a child with higher cumulative rewards is more likely to be selected. While the second term is called the exploration term, which encourages agent to select less visited nodes that may result in better results. The constant $\epsilon>0$ makes a trade-off between exploitation and exploration.

The reward function is essential for the MCTS algorithm. Generally, the first term of (19) should be within $[0,1]$. This can be easily guaranteed in problems such as games since the reward is set as $\{\mathrm{win=1},\mathrm{draw=0.5},\mathrm{loss=0}\}$. In our priority scheduling problem, we normalize the reward of each node $c$ whose parent is node $p$ to $[0,1]$ with the following formulation,

where $f_{p}^{c}=-t_{\mathrm{leave}}^{c}$ is the instant reward obtained after each simulation step, $f_{p,\mathrm{min}}$ and $f_{p,\mathrm{max}}$ are the minimal and maximal reward among all children nodes of node $p$ respectively. Note that $f_{p,\mathrm{max}}$ and $f_{p,\mathrm{min}}$ are dynamically determined as the MCTS algorithm progresses. Specifically, at the current simulation step, if the parent node $p$ has no child nodes, then we set $f_{p,\mathrm{min}}=f_{p,\mathrm{max}}=f_{\mathrm{sim}}$, where $f_{\mathrm{sim}}$ is a temporal reward obtained by randomly sampling a feasible sequence from parent node $p$. On the other hand, if $p$ already has existing children, then $f_{p,\mathrm{min}}$ and $f_{p,\mathrm{max}}$ are set to be the minimal and maximal reward among these children. The proposed MCTS-based priority scheduler is summarized in Algorithm 2.

In this section, we conduct simulations to evaluate the efficiency and adaptability of the proposed bi-level coordination strategy. To compare the coordination efficiency in sub-static scenarios, both the centralized-CS and centralized-OCP strategies described in Section II-C are considered as benchmarks. In addition, we implement traditional traffic signal control and a FIFO-based distributed strategy to validate traffic throughput in continuous traffic flows with different arrival rates. In traffic signal control, signal timing is predetermined using historical traffic data, and only non-conflicting vehicles are allowed to enter the CA during each green light phase.

All experiments are conducted with Python 3.8 on a personal computer with Intel i7-10700F 2.9 GHz CPU and 16 GB of RAM. We construct a signalized intersection using the Simulation of Urban MObility (SUMO) platform and use Python to interact with SUMO via the TraCI library. The centralized-CS problem $\mathscr{P}_{\mathrm{centralized}}$, the centralized-OCP problem $\mathscr{P}_{\mathrm{centralized-OCP}}$, and the distributed trajectory planning problem $\{\mathscr{P}_{\mathrm{distributed}}^{i}|i\in\mathcal{N}\}$ are solved using the MOSEK solver. We also use Python API to access the MOSEK optimizer. The chosen parameters for the vehicles and the road structure are shown in Table I. Videos of our simulation results can be found at the supplementary materials, or online at https://youtu.be/WYAKFMNnQfs.

We first consider the coordination scenario illustrated in Fig. 2 for comparison of the centralized-CS and our bi-level strategies. Fig. 8 shows a capture of coordination animation under the centralized-CS strategy. We observe that vehicle $\mathrm{L}_{1}$ crosses the coordination area first, followed by high-priority vehicles $\mathrm{D}_{1}$ and $\mathrm{D}_{2}$, causing the remaining vehicles to decelerate or stop before the entry point. Once $\mathrm{D}_{1}$ and $\mathrm{D}_{2}$ exit the coordination area, vehicles $\mathrm{U}_{1}$ and $\mathrm{U}_{2}$ enter, with vehicle $\mathrm{R}_{1}$ being the last to cross. The resulting priority sequence is $(\mathrm{L}_{1},\mathrm{D}_{1},\mathrm{D}_{2},\mathrm{U}_{1},\mathrm{U}_{2},\mathrm{R}_{1})$, and the optimal total travel time (i.e., the time when the last vehicle leaves the coordination area) is $t_{\mathrm{leave}}=18.1$s.

Fig. 9 shows an animation of the coordination process under the proposed bi-level strategy. The priority sequence obtained by the high-level planner is $(\mathrm{L}_{1},\mathrm{U}_{1},\mathrm{D}_{1},\mathrm{R}_{1},\mathrm{U}_{2},\mathrm{D}_{1})$. The vehicles then plan their trajectories sequentially based on this priority configuration. The collision set can accommodate up to 4 vehicles simultaneously, suggesting that the proposed collision region model can fully utilize time-spatial resources. The resulting total travel time is $t_{\mathrm{leave}}=14.9$s, which represents a 17.7% increase in traffic efficiency compared to the centralized-CS strategy. We also evaluate the priority sequence $(\mathrm{L}_{1},\mathrm{R}_{1},\mathrm{U}_{1},\mathrm{D}_{1},\mathrm{D}_{2},\mathrm{U}_{2})$ shown in Fig. 2, which results in a total travel time of approximately 16.7s. This highlights the significance of assigning priorities for coordination efficiency.

Fig. 11 demonstrates the coordination efficiency of these three strategies with respect to the number of vehicles. We observe that our strategy significantly outperforms the centralized-CS strategy, as the collision model we propose allows vehicles to share the CA simultaneously. As the number of vehicles increases, the efficiency of the centralized-CS strategy decreases because more vehicles have to yield to conflicting vehicles. The centralized-OCP strategy, which integrates our collision region model, can always achieve optimal coordination performance. Notably, our bi-level strategy achieves near-optimal coordination performance, with a small gap observed between our approach and the centralized-OCP strategy. This gap may be attributed to (i) sub-optimal priority sequences caused by MCTS and (ii) sub-optimal trajectories caused by sequential optimization.

Fig. 11 shows the maximum computation time with respect to the number of vehicles for these strategies. Our results show that when solving a local trajectory planning problem (i.e., $\mathscr{P}_{\mathrm{distributed}}^{i},i\in\mathcal{N}$) over the total planing horizon $T_{\mathrm{total}}=40s$ (i.e., $T_{\mathrm{total}}/\Delta t=400$ slots), the MOSEK solver typically takes about 0-1.5s, and the computation time increases linearly with the number of vehicles. It is worth noting that in real-world deployments, receding horizon control methods (i.e., model predictive control) can be employed in our framework to solve the coordination problem at each time slot over a short planning horizon, thus reducing computation time. Additionally, the MCTS-based high-level planner always returns a priority sequence within 0.1s. In contrast, the centralized strategies suffer from high computational complexity, with computation time increasing exponentially with the number of vehicles. This is because $\mathscr{P}_{\mathrm{centralized-CS}}$ and $\mathscr{P}_{\mathrm{centralized-OCP}}$ are MIQP problems, which are computationally prohibitive in large problem sizes. For instance, when dealing with 12 vehicles, the centralized-CS strategy takes approximately 13.2 hours to find a solution, while our approach achieves near-optimal results in only 14.1 seconds.

The proposed bi-level coordination strategy is also applied to continuous traffic flow. The vehicles arrival is assumed to be a Poisson process. In Fig. 12, we vary the arrival rates from 100 to 3000 vehicles/h/lane to compare the traffic throughput of the traffic signal control and our strategy under different traffic loads. For each arrival rate, the traffic throughput is averaged over the number of coordinated vehicles of 100 independent trials, and each trial simulates a 1-hour traffic process. To present the upper bound of throughput under a certain strategy, Fig. 12 does not consider left-turning vehicles. It can be seen that, in low traffic loads (100 to 800 veh/h/lane), both traffic signal control and the proposed strategy can provide enough coordination capacity for incoming traffic volumes. However, the traffic signal control soon reaches its capacity (about 800 veh/h/lane) when the arrival rate is 1200 veh/h/lane. In contrast, the proposed strategy reaches its limit when the arrival rate is about 2800 veh/h/lane and can provide a much higher serving rate of up to 1400 veh/h/lane.

Fig. 13 further examines the influence of different left-turning probabilities on traffic throughput. For each traffic pattern, we simulate a 1-hour traffic process with a arrival rate of 3000 veh/h/lane. It is evident that compared to traffic signal control, our strategy can significantly improve traffic throughput, regardless of the traffic patterns. We observe that an increase in left-turning vehicles results in a decline in throughput. This is because left-turning paths occupy more collision regions and intersect with many paths, leading to more vehicles giving way to left-turning ones. Furthermore, we compare the efficiency of our MCTS-based priority scheduler with the FIFO method. For fairness, the distributed-FIFO strategy uses the same trajectory planner as ours. We observe that our MCTS-based scheduler has an approximately 12% improvement in traffic throughput compared to the FIFO method.