Game Theoretic Application to Intersection Management: A Literature Review

Ziye Qin, Ang Ji, Zhanbo Sun, Guoyuan Wu, Peng Hao, and Xishun Liao Ziye Qin, Ang Ji, and Zhanbo Sun (Corresponding author) are with the School of Transportation and Logistics, Southwest Jiaotong University, Chengdu, Sichuan, 611756, China, and the National Engineering Laboratory of Integrated Transportation Big Data Application Technology, Southwest Jiaotong University, Chengdu, Sichuan, 611756, China. (Email: ziye.qin@my.swjtu.edu.cn, ang.ji@swjtu.edu.cn, zhanbo.sun@home.swjtu.edu.cn; Tel: 86-28-87600156; Fax: 86-28-87600156).Ziye Qin, Guoyuan Wu, and Peng Hao are with the College of Engineering, Center for Environmental Research and Technology, University of California at Riverside, Riverside, CA 92507 USA. (Email: gywu@cert.ucr.edu, haop@cert.ucr.edu).Xishun Liao is with the University of California at Los Angeles, Los Angeles, CA 90095 USA. (Email: liaoxishun@gmail.com).

Abstract

The emergence of vehicle-to-everything (V2X) technology offers new insights into intersection management. This, however, has also presented new challenges, such as the need to understand and model the interactions of traffic participants, including their competition and cooperation behaviors. Game theory has been widely adopted to study rationally selfish or cooperative behaviors during interactions and has been applied to advanced intersection management. In this paper, we review the application of game theory to intersection management and sort out relevant studies under various levels of intelligence and connectivity. First, the problem of urban intersection management and its challenges are briefly introduced. The basic elements of game theory specifically for intersection applications are then summarized. Next, we present the game-theoretic models and solutions that have been applied to intersection management. Finally, the limitations and potential opportunities for subsequent studies within the game-theoretic application to intersection management are discussed.

Index Terms:

Intersection management, Game theory, Cooperative and non-cooperative behavior, Decision-making, Multi-agent reinforcement learning.

I Introduction

Intersections within urban traffic networks are subject to traffic conflicts, leading to prolonged vehicle delay and elevated energy consumption [1, 2, 3]. Innovative technologies such as human-machine interface (HMI), and vehicle-to-everything (V2X) communications facilitate the collection and sharing of vehicular and trip information, such as origin and destination, vehicle trajectories, and personal attributes [4, 5, 6]. These advancements unlock a significant number of opportunities for more efficient intersection management. For example, traffic signal controllers can utilize more comprehensive vehicle information to optimize signal phase and timing, and minimize overall traffic delays. Since the introduction of the autonomous intersection management (AIM) system by Dresner and Stone, there have been increasing efforts to explore the applications of emerging technologies in this area [7]. Simultaneously, management strategies have shifted from focusing only on macroscopic measurements (e.g., traffic efficiency) to microscopic considerations such as experiences and intentions of traffic participants [8].

In the context of management and control, intersections can be classified as signalized and un-signalized intersections [9]. The optimization of signalized intersections typically involves the allocation of the signal phase and timing (SPaT). Through the optimization of SPaT, bus rapid transit (BRT) priority strategies can be effectively executed [10]. With the advancement of cooperative adaptive cruise control (CACC) technology, a compatible SPaT plan has been demonstrated to improve intersection capacity [11]. In the partially/fully connected automated vehicle (CAV) environment, signal optimization can be coupled with vehicle trajectory planning [12, 13]. The co-optimization of SPaT information and vehicle speed has the potential to concurrently enhance eco-driving practices and traffic efficiency [14, 15]. Even given SPAT information, optimizing vehicle trajectories can balance traffic efficiency and fuel economy [16, 17]. Meanwhile, the management of signalized intersections, considering the stochastic behavior inherent to human-operated vehicles (HVs), is attracting heightened interest [18, 19]. In an ideal environment characterized by a full penetration of CAVs, the utilization of traffic signals may become redundant (i.e., un-signalized intersections). The main issues in managing un-signalized intersections pertain to right-of-way allocation and collision-free vehicle trajectory planning [20, 21, 22, 23]. The trajectory planning for cyclists, a vulnerable demographic at un-signalized intersections, is garnering attention. This concern arises from the conflicts that cyclists may encounter both within their group and with motorized vehicles [24]. For all types of interactions, the optimization cannot be solely considered from the perspective of traffic efficiency; factors such as the value of time, time-to-collision (TTC), and fuel consumption are also frequently investigated [25, 26, 27]. Several review articles have summarized the existing methodologies on intersection management and proposed promising future directions. For example, Zhao et al. discussed the application of computational intelligence in traffic signal control that can address the large complex nonlinear problems in urban transportation systems [28]. Araghi et al. also reviewed computational intelligence methods for traffic signal control and compared the performance of various learning methods [29]. Wang et al. surveyed the history of traffic self-adaptive control systems, pointed out the deficiencies, and expressed the expectations for multi-agent reinforcement learning in this field [30]. Yau et al. surveyed the application of reinforcement learning algorithms to traffic signal control [31]. Florin and Olariu reviewed three adaptive traffic signal control strategies for various levels of vehicular communications, including: (i) no vehicle communication; (ii) wireless transmission of vehicle status data; and (iii) on-board computation to optimize traffic signals [32]. Chen and Englund surveyed cooperative intersection management strategies under both signalized and un-signalized intersections and summarized related research projects worldwide [33]. Studies related to CAV-enabled intelligent intersection management were summarized in [34, 35, 36]. Namazi et al. provided a systematic review of intelligent intersection management in mixed traffic including autonomous vehicle (AV) and human-operated vehicle (HV), also with high expectations for artificial intelligence (AI)-enabled traffic management in the future [37]. Al-Turki et al. concluded signalized intersection control methods in mixed traffic and proposed an alternative machine learning (ML)-based solution to the design of intelligent adaptive traffic signal controllers [38]. Additionally, the existing challenges and potential future directions were outlined. Shirazi and Morris reviewed the behavior at intersections and the safety analysis for vehicles, drivers, and pedestrians, and made suggestions for the safety management of intersections [39]. Zhong et al. provided a summary of AIM research in terms of the corridor coordination layer, the intersection management layer, and the vehicle control layer [40]. Iliopoulou et al. summarized the market-inspired allocation of spatial-temporal resources at intersections within the context of the connected vehicle (CV) environment [41]. The aforementioned reviews highlighted the significant accomplishments in the development of technical tools for intersection management. Nevertheless, most traffic management strategies, when scrutinized from an optimization standpoint, have been developed in an altruistic manner, that is, any player might have to sacrifice his or her own interest to achieve the global optimum. For example, vehicles are often obliged to endure waiting times for the larger collective benefits, without any compensation. Therefore, enhancing our understanding of the needs of traffic participants, the intricacies of their interactive behaviors, and the delicate equilibrium between individual gains and collective benefits are worthy subjects for continued exploration. Toward this end, a comprehensive summary of interaction analysis between decision-makers at intersections holds promise in facilitating the research of more user-friendly management strategies.

Consequently, there is an urgent demand to identify a suitable tool for analyzing individual preferences and interaction behaviors to guarantee fair decision-making in future intersection management. Game theory – known as a classic mathematical method to model conflicts and cooperation among decision-makers, is commonly used to study the behaviors of traffic participants [42]. A game consists of three basic elements: players, strategies for players, and payoff functions that describe how much payoff or utility each player can obtain under various environment states and joint actions chosen by the players [43, 44]. Game theory has demonstrated its potential for managing traffic in various scenarios such as traffic congestion management [45], lane-changing [46, 47, 48, 49, 50], ramp-merging [51, 52, 53, 54], and intersection management [55, 56]. In lane-changing and ramp-merging games, the number of participants may be much less than that at a standard four-arm intersection where the interactions of traffic participants are more complex. In terms of the operational strategies, discrete actions such as acceleration/deceleration during lane-changing and yield/not yield during ramp-merging may be sufficient in decision-making modeling. However, as the diversity and number of participants grow at intersections, lateral and longitudinal joint strategies are needed for adapting to complex situations. An example of this complexity is the coordination of the green phase of traffic signals. Factors such as TTC and travel time are widely used to construct the payoff function from the perspective of players’ safety and driving efficiency [57]. In scenarios where players are traffic operators, traffic signal delays and queue lengths are frequently considered [58, 59]. In addition, heterogeneous trip characteristics, such as trip purpose and distance, result in diversified preferences of time, safety, and other measures [51, 60]. For example, catching a flight generally has a higher value of time compared to shopping. Consequently, the payoff functions should be more specific. Another concern is that the continuous flow approaching the intersection makes the problem of passing sequence optimization even more complex, which can be NP-hard [61, 62]. Game theory serves as a valuable instrument for decomposing complex interactions, modeling decision-making processes, analyzing the behavior of traffic participants, and helping take reasonable actions to manage the aforementioned issues. Specifically, game theory recognizes and accommodates user heterogeneity in the first category of issues, allowing for tailored personalization of payoff function compositions in line with individual characteristics. For the second category, a synergistic combination of game theory and multi-agent reinforcement learning (MARL) offers a potent solution, effectively resolving the intricate challenges presented.

Great efforts have been made to apply game-theoretic models for intersection management. The diversity of models makes it challenging to outline the existing studies, so a clear lineage of game theoretic approaches for intersection management needs to be identified. Additionally, most studies simplified intersection games by assuming a complete information environment. In reality, self-interested players may exploit misreported private information to increase their payoff, making it more practical to address the problem in an incomplete information environment. In a nutshell, despite the well-established conventional game models for intersection management, there are several challenges to be tackled in practical applications, such as how to find an appropriate game model and how to incentivize traffic participants to take part in the games.

Based on the author’s understanding, there have been few studies that provide a comprehensive summary of recent developments in game theory applications to intersections, especially on the behavior analysis and the modeling of traffic participants to enhance intersection management. Moreover, as connected and automated technologies gradually integrate into transportation systems, the emphasis on individual travel needs grows correspondingly. In this context, there is an evident demand for methods capable of balancing the requirements of traffic participants, guaranteeing their respective interests. Game theory provides a fitting solution to this requirement. Employing game-theoretic methods to manage intersection areas with multiple conflict points holds significant promise. To fill research gaps and facilitate knowledge and development for future relevant research, the paper provides a review on the decision-making problems of intersection management and aims to investigate the applications of game theory to intersections. The main contributions of this paper include: (i) summarizing the existing applications of game theory for intersections, including the intersection circumstances for deploying game theory, the basic elements of game theory, and the various types of game theory models and solutions; and (ii) highlighting existing issues and future directions in combining game theory and intersection management.

II Applications of Game Theory to Intersection Management

To collect relevant literature on game-theoretic applications to intersections, we presented a workflow for systematic literature retrieval and analysis, illustrated in Fig. 1. This methodology is versatile and can be adapted to trace advancements in diverse research fields. We began by identifying critical keywords associated with our research focus, incorporating terms such as “game theory”, “game theoretic”, “auction”, and “mechanism design”, combined with “intersection”, to retrieve relevant studies. This was followed by a keyword search in prominent scientific repositories like Google Scholar, Web of Science, and Scopus. We also followed up on articles citing the selected publications. Subsequently, we undertook a preliminary screening step to offset potential inaccuracies stemming from keyword searches. This involved a brief review of the topics, abstracts, highlights, and the prestige of the journals in which these studies were disseminated. Finally, we provide an in-depth review and summary of the game theory framework as applied to intersection management. This analysis is methodically presented, encompassing the types of intersections examined, the game theoretic modeling procedures, and both the innovations and limitations observed. An extensive search using specific keywords in databases yielded a significant number of pertinent studies. However, initial scrutiny and subsequent in-depth reviews identified instances of duplicative content. Finally, guided by the criteria presented in Fig. 1, we selected 88 articles, published between 2005 and 2023, for inclusion in this review.

Refer to caption — Figure 1: Literature retrieval and analysis workflow for collecting relevant papers

II-A An Overview of Intersection Management

Intersections serve as the critical points in urban transportation networks where multiple roadways converge horizontally; they also represent concentrated conflict areas, resulting in a higher incidence of collisions compared to the other parts of urban transportation networks [63]. Fig. 2 presents a standard four-leg intersection with three movements along each approach, including through traffic, left-turning traffic, and right-turning traffic. When traffic flows from different directions pass through the same section of the intersection, there are some potential collision areas (defined as “conflict points”) [64]. Conflict points at intersections can be categorized as merging conflict points, diverging conflict points, and crossing conflict points [65]. Merging conflict points arise when two paths converge or come together into a single path. Conversely, diverging conflict points emerge when one path splits into two or more paths. Lastly, crossing conflict points are observed when two paths intersect each other perpendicularly or at some angle, resulting in the potential for one road user to cross directly into the path of another. There exist thirty-two (32) conflict points among motorized traffic (MT) in the given scenario. This features eight (8) merging and eight (8) diverging conflict points, where rear-ended and sideswipe collisions typically occur. Additionally, there are sixteen (16) crossing conflict points inside the intersection, with twelve (12) of them associated with left-turn movements. The remaining four (4) crossing conflict points involve the through movements on two adjacent approaches, where angle collisions may occur [66]. Non-motorized traffic (NmT), as the vulnerable group, encounters twenty-four (24) conflict points interacting with motorized traffic. Conventional intersection management relies on the separation of time-space conflicts among approaching vehicles, using traffic signal control and lane channelization, which also limits intersection capacity. To balance efficiency and safety, many studies have explored game-theoretic approaches for intersection management, which have proven to be a valuable tool for characterizing such interactions, particularly in the decision-making of CAVs [67, 68, 69].

To better understand the operation mechanism of intersections and the interactions among traffic participants, we revisited the intersection management solutions with the application of game theory. The intersection circumstances in game-theoretic studies can be classified based on several criteria: (i) the presence or absence of traffic signals: signalized and un-signalized intersections; (ii) the number of intersections under consideration: single, isolated intersections and/or multiple intersections; (iii) the type of traffic composition: traffic flow with HVs only, CVs only, CAVs only, mixed traffic with pedestrians and vehicles. A brief overview of the studies conducted in the various circumstances is presented in Fig. 3. More than half of the game theory-related studies (63.5%) have focused on the interactive behaviors among vehicles at un-signalized intersections. The multi-intersection coordination problem accounts for only 16.5% of studies, which still needs further attention. A major chunk of studies focuses on the traffic flow with CVs only or CAVs only since CVs/CAVs are more conducive to obtaining traffic information and implementing vehicle control strategies.

II-B Game Theory-based Modeling

The key to modeling with game theory is to identify three basic elements in decision-making problems, i.e., players, strategies, and payoff functions. However, identifying these elements can be challenging due to the complex nature of intersections in urban environments. The set of players can include a variety of traffic participants, such as HVs, CAVs, and pedestrians. The set of strategies can be represented by the various choices each participant can make, such as selecting a movement direction or a speed profile. The payoff function can be defined based on various factors, i.e., vehicle delay, fuel consumption, and safety. Additionally, various game theory models, such as non-cooperative and cooperative games, can be employed to analyze interactions among traffic participants at the intersection. Identifying and applying these elements and models can provide valuable insights for intersection management and can improve traffic flow efficiency and safety.

II-B1 Players

Existing studies usually make two common assumptions related to the rationality and intelligence of players [42]. Rational players adopt strategies that maximize their expected payoff. If an intelligent player knows everything about the game, the player can make informed decisions regarding his or her next moves. Table I summarizes the literature concerning the categories and number of players involved during intersection management. The number of players in a game directly affects the choices of game theory models. Although the two-player game has been studied more extensively than other games with multiple players [44], the two-player game appears to be insufficient in behavioral modeling at intersections where multiple players, such as vehicles and pedestrians, compete for the right-of-way. Therefore, in addition to the two-player matrix game, it is worthwhile to investigate multi-player games to better represent the diverse interactions at intersections. Note that individual vehicles are widely applied as players in game-theoretic modeling at intersections, and it is generally assumed that each vehicle takes the optimal action to maximize its expected payoff. In situations where pedestrian signals are not present, pedestrians and motor vehicles also compete for right-of-way. Some researchers opt to consider players who represent a collective, such as phase, intersection, and platoon of vehicles, whose objective is to maximize the collective payoff for all the vehicles encompassed within the aggregation.

TABLE I: Players at the intersection games

Players	Number of players
Players	Two-player	Multiple ( $2^{+}$ ) players
Vehicle/Driver	[70, 61, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103]	[9, 20, 25, 55, 57, 92, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 102]
Pedestrian	[80, 83, 97, 98]	-
Phase/Barrier group	[59, 124, 67, 68, 125, 69]	[58, 59, 126]
Intersection/Traffic operator/Signal controller	[86, 127, 128, 129, 130, 131]	[105, 131, 132, 133, 134]
Incoming link/Path/Movement/Flow/Direction/Vehicle group/Platoon	[27, 135, 136, 137, 138]	[139, 140, 141, 142, 143]

II-B2 Strategies

In game-theoretic modeling at intersections, players choose different strategies to maximize their expected payoffs from a finite (e.g., yield or not yield) or infinite (e.g., acceleration rate) set of strategies. The strategies can also be categorized into two main types: non-cooperative and cooperative strategies. With cooperative strategies, players try to achieve mutual benefits. In non-cooperative games, on the other hand, players take actions to maximize their own payoffs without considering the payoffs of others or the system. The most common strategies are summarized below and outlined in Table II.

(i)

Yielding: This strategy involves a player giving up the right-of-way to others, which is usually adopted by traffic participants to avoid collision and to maintain safety at intersections.
(ii)

Accelerating: The strategy involves a player increasing his/her speed to approach and/or depart from an intersection, which is commonly used by traffic participants who want to avoid waiting at red lights or to pass through the intersection quickly.
(iii)

Decelerating: This strategy involves players reducing their speed to stay safe or to comply with traffic regulations, which is commonly adopted by drivers who want to avoid accidents or obey traffic lights.
(iv)

Signal control: This strategy pertains to the traffic signal controller that controls the phase and signal timing to manage traffic flow and reduce congestion at intersections.

In summary, players at intersections adopt a variety of strategies to optimize their payoffs. The choice of strategy depends on the players’ type, the intersection circumstances, and the available information for decision-making. Game-theoretic modeling can help to understand how players interact and how their choices of strategies would affect intersection operations such as traffic flow and safety.

TABLE II: Players’ strategies

Players	Strategies	References
Vehicle/Driver	Vehicle operation (accelerate/maintain current speed/decelerate; speed profile; go through/turn left/turn right; heading angle; steering)	[9, 20, 70, 57, 61, 71, 72, 73, 74, 75, 76, 77, 81, 82, 85, 87, 88, 89, 90, 92, 99, 107, 108, 109, 111, 115, 118, 119, 120, 121, 122, 130, 102, 103]
	Yield/Not yield	[80, 83, 95, 96, 97, 98, 100, 101]
	Cooperation/Defection, comply/disobey	[78, 79, 84, 86]
	Bid for the right-of-way	[25, 55, 91, 93, 94, 104, 106, 110, 112, 113, 114, 116, 117, 123]
	Route choice	[105]
Pedestrian	Wait/Cross	[80, 83, 97, 98]
Phase/Barrier group	Signal control (green light time/red light time, keep current phase/change to next phase)	[58, 59, 124, 67, 68, 125, 69, 126]
Intersection/Traffic operator/Signal controller	Signal control (green light time/red light time, keep current phase/change to next phase)	[105, 127, 129, 131, 132, 133, 134]
	Bid for the signal time	[128]
	Passing permitted/Passing forbidden	[86]
	Independent/Cooperative	[130]
Incoming link/Path/Movement/Flow/Direction /Vehicle group/Platoon	Signal control (green light time/red light time, keep current phase/change to next phase)	[27, 136, 139, 138, 141, 142, 143]
	Passing permitted/Passing forbidden	[137]
	Bid for the right-of-way	[140]
	Speed profile	[135]

II-B3 Payoff functions

The payoff function is a real-valued function defined on the set of all outcomes and strategy profiles. The payoff function of each player maps the multidimensional strategy profiles into real values to capture preferences. Importantly, the player’s payoff depends not only on his or her own strategy but also on the strategies of other players [144]. The factors considered in the payoff functions are also diverse among traffic scenarios. For example, traffic operators and participants may be interested in safety, efficiency, comfort, and energy efficiency at intersections.

For safety, metrics such as TTC [82, 111], time difference to collision (TDTC) [57, 77, 107], post-encroachment time (PET) [97], and other surrogate safety measures (SSMs) are usually adopted, which are listed as follows:

(i)

Time-to-collision (TTC) was introduced by Hayward [145] to indicate the time it takes for two vehicles to collide when the following vehicle is moving faster than the leading vehicle in the same lane. TTC can be written as

$\displaystyle TTC(t)=\frac{x_{l}(t)-x_{f}(t)}{v_{l}(t)-v_{f}(t)},\forall v_{l}(t)<v_{f}(t)$ (1)

where $x_{l}(t)$ and $v_{l}(t)$ are the location and velocity of the leading vehicle, respectively, and $x_{f}(t)$ and $v_{f}(t)$ pertain to the following vehicle. $t$ is the current timestamp.

(ii)

Time different to collision (TDTC) in (2) was proposed by [146] representing the time difference between the two vehicles arriving at the conflict point.

\displaystyle\begin{split}TDTC(t)_{i,j}=|(\sqrt{(\frac{v_{i}(t)}{a_{i}(t)})^{2}+\frac{2L_{i}(t)}{a_{i}(t)}}-\frac{v_{i}(t)}{a_{i}(t)})-\\ (\sqrt{(\frac{v_{j}(t)}{a_{j}(t)})^{2}+\frac{2L_{j}(t)}{a_{j}(t)}}-\frac{v_{j}(t)}{a_{j}(t)})|,\\ v_{min}\leq v_{i}(t),v_{j}(t)\leq v_{max}\end{split}

(2)

where $v_{i}(t)$ and $a_{i}(t)$ are the velocity and acceleration at time $t$ , respectively, and $L_{i}(t)$ is the distance from the current position of vehicle $i$ to the conflict point. Similarly, $v_{j}(t)$ , $a_{j}(t)$ , and $L_{j}(t)$ pertain to vehicle $j$ . $v_{min}$ and $v_{max}$ are the minimum velocity and maximum velocity, respectively.

(iii)

Post-encroachment time (PET) is defined by [147], which is the time elapsed between the moment that the first vehicle leaves the virtual conflict zone and the moment the second vehicle reaches it.

$\displaystyle PET=|\frac{L_{i}}{v_{i}}-\frac{L_{j}}{v_{j}}|$ (3)

where $i$ and $j$ are the indexes of vehicles. $L_{j}$ and $L_{j}$ represent the distance from their position to the conflict point. $v_{i}$ and $v_{j}$ are the velocity of vehicle $i$ and vehicle $j$ , respectively.
(iv)

Time advantage (TAdv) describes situations where two road users pass a common spatial area at different times to avoid a collision [72, 148]. TAdv serves as a measure to evaluate PET, with the calculation method delineated as follows,

$\displaystyle TAdv(t)=|\frac{L_{i}(t)}{v_{i}(t)}-\frac{L_{j}(t)}{v_{j}(t)}|<T_{M}$ (4)

where $i$ and $j$ are the vehicle indexes, and $L_{i}(t)$ , $L_{j}(t)$ denote the current distance from their position to the point of conflict. $v_{i}(t)$ and $v_{j}(t)$ are the current velocities of vehicle $i$ and vehicle $j$ , respectively. $T_{M}$ is the time threshold of safety.

For traffic efficiency, individual vehicles focus on microscopic information, such as velocity, delay, and waiting time; traffic operators, on the other hand, concern more about macro-level metrics such as queue length, and throughput from an aggregated/systemic perspective.

Velocity-based metrics are usually expressed in two ways. As shown in (5), the first way involves quantifying the change of speed $\Delta v_{i}(t)$ (i.e., acceleration or deceleration rate) of vehicle $i$ , [77]. The second way in (6) is the absolute value or square of the difference between the vehicle’s current speed and the speed limit or desired speed [9].

\displaystyle\Delta v_{i}(t)=v_{i}(t+1)-v_{i}(t)

(5)

\displaystyle\varphi_{i}(t)=\left\{\begin{matrix}v_{i}(t+1)-v_{i}(t)\\ |v_{i}(t)-v^{e}|\end{matrix}\right.

(6)

where $v_{i}(t+1)$ and $v_{i}(t)$ are the velocities of vehicle $i$ at time $t$ and time $t+1$ , respectively, and $v^{e}$ is the speed limit or desired speed.

Delay is considered as the difference between normal travel time on a roadway and the estimated travel time through a work zone. Vehicle delay may be accumulated by the increased travel distance and/or reduced speed, insufficient capacity, and temporary stoppage of the traffic flow [63]. Many metrics such as the average total delay and the average stopped delay are widely used in payoff functions [27, 58, 68, 148].

Queue length is a common measure used to evaluate the efficiency of intersections. Average queue length is a conventional indicator, which is the quantities of vehicles in the queue per lane per time interval [149, 68]. Estimated queue length is another frequent metric that has two different calculation methods. The first method calculates queue length by accumulating arrivals and departures at intersections, following input-output models. The second one analyzes the formation and dissipation of queues using the shockwave theory [150].

With respect to driving/riding comfort, the focus is typically on the smoothness of vehicle control. Efforts to enhance comfort often focus on achieving a smooth trajectory. Techniques include penalizing impractical acceleration/deceleration [104], applying filters to omit abrupt changes in acceleration/deceleration [88], and incorporating jerk minimization in optimization models [73, 111]. The commonly-used metric is jerk $jerk_{i}(t)$ in (7), which is the first-order derivative of acceleration.

\displaystyle jerk_{i}(t)=\frac{\mathrm{d}a_{i}(t)}{\mathrm{d}t}

(7)

where $a_{i}(t)$ is the acceleration of vehicle $i$ at time $t$ .

Fuel (energy) consumption and emissions are crucial for both individual drivers and traffic operators, as they are directly related to travel costs and environmental sustainability. Models such as the Comprehensive Modal Emissions Model (CMEM) [151], Virginia Tech Microscopic (VT-Micro) model [152] and Motor Vehicle Emission Simulator (MOVES) model [153] are generally adopted in game-theoretic modeling. These models provide fine-grained estimates of fuel consumption and emissions, enabling decision-makers to manage traffic flow while minimizing environmental impacts.

Fig. 4 visualizes the frequency of various factors used in constructing payoff functions. We employed both color and the range of circles to differentiate the frequency of various factors in determining the payoff function. Specifically, a larger range circle represents that the factor is prevalently adopted, and more detailed frequencies can be observed through different colors. According to the statistics, traffic efficiency is the most concerned, which is usually characterized by delay, queue length, and velocity. For safety, conflict-related factors such as $TTC$ , and $TDTC$ are commonly used, mostly in the cases of multi-vehicle cooperation and/or un-signalized intersections. Note that only a few studies focus on comfort and energy consumption, implying that passing through intersections safely and as fast as possible is still the most urgent demand.

II-C Classifications of Game-theoretic Models

Game theory is a powerful tool for modeling and analyzing complex interactions among traffic participants in traffic systems. There are various ways to classify game theory models according to different criteria, such as whether players will negotiate a binding contract and whether all information is common knowledge for players. In general, game theory models can be classified into non-cooperative games and cooperative games, which are distinguished by considering whether a coalition is formed. Non-cooperative games investigate the decision-making of individual players, while cooperative games focus on the joint actions of groups of players [144]. In addition, based on the amount and/or type of available information, games can also be classified into complete information games and incomplete information games. The subsequent section presents the game theory models commonly adopted in intersection management. Table III illustrates the applications of these game theory models and showcases their relevance and usefulness in the context of intersection management.

TABLE III: Game theory models and solutions

Game type		Solutions	References
Cooperative games	Transferable utility game	Pareto efficiency	[27, 134, 58]
	Transferable utility game	Shapley value	[125]
	Bargaining game	Pareto efficiency	[57]
	Bargaining game	Nash bargaining solution	[59, 124]
	Coalition formation game	Merge and split rule	[126]
	Fuzzy coalitional game	Fuzzy Shapley method	[111]
Non-cooperative games	Auction-based game	-	[25, 104, 106, 108, 110, 114, 117, 112, 113, 123]
		First-price auction	[128, 116, 93, 140]
		Second-price auction	[93, 109]
		Vickrey-Clarke-Groove mechanism	[55]
	N-player normal form game	Pareto improvement solution of Nash equilibrium	[127]
		Nash equilibrium	[73, 76, 83, 86, 90, 91, 94, 95, 96, 100, 101, 132, 136, 71, 99, 129, 143, 107]
		Pareto efficiency	[68, 130, 138]
		-	[79, 84]
	Sequential game	Nash equilibrium	[20, 103, 9]
	Sequential game	Stackelberg-Nash equilibrium	[67, 120, 131, 142, 102]
	Level-k model	-	[70, 74, 75, 115, 118, 122]
	Level-k model	Nash equilibrium	[121]
	Differential game	Nash equilibrium and Stackelberg equilibrium	[82]
	Others	Nash equilibrium/Pareto efficiency	[69, 61, 77, 67, 72, 105, 87, 88]

Non-cooperative V.S. Cooperative:

II-C1 Non-cooperative games

Non-cooperative (strategic) game is a type of game in which individual players take strategies and in which the outcome of the game is described by the strategy taken by each player, along with the corresponding payoff for each player [154]. In non-cooperative games, players do not form coalitions or cooperate directly with one another. Instead, they take strategy independently to maximize their individual payoffs. Note that “non-cooperative” does not mean there is no cooperation among players. Instead, the game focuses on the individual strategies and payoffs, and it analyzes the joint strategies without considering the possibility of some players forming coalitions and transferring their payoff within the coalition [44]. Non-cooperative games are widely used in modeling right-of-way competition at intersections, such as signal control, pedestrian crossing, and passing sequence decision-making of CAVs. Some commonly used non-cooperative games for intersection management include and are not limited to $N$ -player normal form games, Level- $k$ thinking, and Stackelberg games.

(i)

$N$ -player normal form games: The game describes all possible combinations of strategies and the corresponding payoffs in a matrix for two players or multiple matrices for more than two players. Players aiming to maximize their expected utilities may take a pure strategy, identified as a deterministic single strategy. Alternatively, they may resort to a mixed strategy, which assigns a probability distribution to the possible pure strategies [44]. A finite $N$ -player normal form game can be formulated by using a tuple $\langle\mathcal{N},\mathcal{A},a\rangle$ , where:

$\bullet\mathcal{N}=\{1,2,3,\ldots,N\}$ is the finite set of $N$ -players, indexed by $i$ .

$\bullet\mathcal{A}=\mathcal{A}_{1}\times\mathcal{A}_{2}\times\ldots\times\mathcal{A}_{N}$ represents the joint strategy set of players, where $\mathcal{A}_{i}$ is the finite strategy set available to player $i$ and $a=(a_{1},a_{2},\ldots,a_{N})\in\mathcal{A}$ is a strategy profile with $a_{i}\in\mathcal{A}_{i}$ .

$\bullet u=(u_{1},u_{2},\ldots,u_{N})$ , $u_{i}:\mathcal{A\to\mathbb{R}}$ is the payoff function for player $i$ , which maps the strategy profile $a$ to a real value. The payoff of player $i$ is dependent not only on his or her own strategy but also on strategies taken by others. Therefore, the utility function is defined over the space of $\mathcal{A}$ instead of $\mathcal{A}_{i}$ .
(ii)

Level- $k$ thinking: Level- $k$ thinking assumes that players take their strategies based on predictions about the strategies of others, thus we can categorize players by the “depth” of their strategic thought, a.k.a., “Levels” [115]. “Level-zero” players take their strategies without considering the strategies of others. “Level- $k$ ” behave under the assumption that their fellow players are at “Level- $k-$ 1”. They then adopt the optimal response to the strategies of the “Level- $k-$ 1” players. In [115], driving aggressiveness, encompassing both conservative vehicles and aggressive vehicles, was utilized as a criterion to demonstrate the “Level”.
(iii)

Stackelberg games: The Stackelberg game depicts the process in perfect information where one player is the leader who takes a strategy first. The other players are followers who observe the leader’s strategy and then choose their strategies accordingly [44]. In the context of intersection signal control, the signal phase with emergency vehicles can be deemed as a leader who first determines its phase timing and the other phases may be considered as followers [67].
(iv)

Auction-based game: An auction is a mechanism that allocates a set of goods to bidders based on their announced bids. Today, auctions are pervasive in e-commerce, e-business transactions, and many other web-based applications [144]. With the evolution of V2X technology, they have also found applications in intersection management [110, 113, 123]. Common types of auctions include the English auction, Dutch auction, first-price auction, and second-price auction. In subsequent sections on incomplete information games, we will delve into those auction types frequently used in intersection management that are incentive-compatible.
(v)

Differential game: Differential games are grounded in a mathematical theory that addresses conflict problems modeled as games where players’ states continuously vary over time [155]. In such games, the potential actions of players are modeled and analyzed using differential equations that encompass control vectors manipulated by players [156]. The inherent ability of differential games to handle continuous time challenges, combined with their resemblance to control problems featuring multiple controllers with different objectives, makes them aptly suited for intersection management, where traffic states are continuously evolving.

II-C2 Cooperative games

The cooperative (coalitional) game is a type of game in which players can negotiate binding contracts that allow them to plan joint strategies, and no players can do worse by cooperating[154, 126]. In cooperative games, a group of rational players cooperates to reach a shared goal, in which the most crucial step is the allocation of the payoff among the players. The mathematical definition of a cooperative game can be represented using an ordered pair $\langle N,v\rangle$ , where $N$ is a finite set of players and $v=2^{N}\to\mathbb{R}$ is the characteristic function, which describes the value created by the group of players. It conforms to the $v(\phi)=0$ , signifying that the collective payoff of an empty coalition is zero (0) across all subsets of $N$ [157]. Typical cooperative games used in intersection management are transferable utility games that allow a group of vehicles to “purchase” the travel time from others [27], and bargaining games to optimize the signal timing by modeling signal phases as players [59].

(i)

Transferable utility games: In a transferable utility game, players can make unlimited side payments to others. This allows them to redistribute the value of a coalition among the players in the coalition in any way as they deem appropriate [158]. The characteristic function in transferable utility games assigns a value of $v(S)$ to each coalition $S\subseteq N$ , where $v(S)$ is the value of the coalition $S$ depicting the total amount of transferable utility that the members of $S$ can earn without any help from the player outside of $S$ [144].
(ii)

Bargaining games: Bargaining games describe the process by which two or more players bargain toward an agreed-upon outcome [159]. The two-player bargaining game was first presented by [160], where each player demands a portion of some good, if the sum of the proposals is no more than the total good, then both players obtain their demand. Otherwise, both players receive nothing [156]. A two-player bargaining game can be represented by using the ordered pair $(S,d)$ , where $S\subseteq\mathbb{R}^{2}$ is the set of alternatives, which is nonempty, compact, and convex; $d=(d_{1}.d_{2})\in S$ is the conflict (disagreement) point. There is an alternative $x=(x_{1},x_{2})\in S$ satisfying $x\gg d$ , which means that if players agree on the alternative $x$ , their payoffs are $x_{1}$ and $x_{2}$ , respectively, if not, their payoffs are $d_{1}$ and $d_{2}$ [159].

Complete information V.S. Incomplete information:

II-C3 Complete information games

A game is regarded as a complete information game if the players share common knowledge of the game being played, including each player’s strategy sets and payoff functions. This indicates that each player is aware of these elements, each player knows that all other players are aware, and this cycle of knowledge continues indefinitely [144, 44]. In traffic scenarios, particularly at intersections, complete information gaming is an overly idealized method of modeling, as traffic participants typically prefer to maintain the privacy of their trip information. Furthermore, obtaining information about all users poses a significant challenge due to communication constraints.

II-C4 Incomplete information games

A game is regarded as an incomplete information (also known as asymmetric information) game when, as the players are ready to take a strategy, at least one player possesses private information about the game that the other players may not know [144]. In such scenarios, players must make decisions based on the available information and their beliefs about the undisclosed information. To address the information asymmetry in incomplete information games, mechanism design is commonly used to reveal the real information of the players [144]. Mechanism design involves designing a game that incentivizes players to reveal their private information truthfully, leading to a more efficient outcome. In mechanism design, the designer chooses a mechanism that maximizes the social welfare subject to some constraints, such as incentive compatibility, individual rationality, and budget balance. Incentive compatibility is an effective tool for revealing the real information players possess, with two main types: (1) For dominant strategy incentive compatibility (DSIC), truth revelation is the optimal response for each player, regardless of the information reported by others; (2) for Bayesian incentive compatibility (BIC). truth revelation is the optimal response for each player in expectation of the information of other players [144]. Incentive compatibility can be achieved by the following methods.

(i)

Second-price auction (Vickrey auction): A second-price auction is a sealed-bid auction where the highest bidder emerges as the winner but only pays a price equivalent to the second-highest bid [144].
(ii)

Vickrey-Clarke-Groves (VCG) mechanism: The Vickrey-Clarke-Groves (VCG) mechanism is designed to eliminate incentives for misreporting by imposing on each player the cost of any distortion they cause. In the VCG mechanism, the payment for a player is set in such a way that their report cannot influence the total payoff for the rest of the players [161]. This approach encourages truthful reporting by aligning the individual player’s interests with the overall welfare of the group.

Research related to incomplete information environments at intersections is comparatively limited, with key studies outlined below: Initially, the second-price auction was applied to design a reservation policy for managing the passing sequence of conflicting zone movements at intersections [110, 109]. Subsequently, the VCG mechanism was designed to reveal true private information from players, such as the urgency level, to schedule the vehicle passing sequence at intersections [55]. More recently, an online incentive-compatible mechanism was proposed to prevent players from misreporting their delay costs, notably, it can be implemented in dynamic traffic environments [123]. Sponsored search auction was employed as a solution for encouraging players to report truthful information to prevent conflicts and deadlocks in the mixed traffic flow comprising HVs and CAVs [113]. Following this, a sponsored search auction was extended for application in multi-vehicle dynamic traffic environments to mitigate overflow issues [112].

II-D Game solutions

The solutions of a game represent the combinations of strategies that players may adopt, and in turn predict the possible outcomes of the game, which may correspond to multiple different solutions. The general solutions of games are summarized below, and the applications for intersections are listed in Table III.

II-D1 Nash equilibrium

Nash equilibrium is one of the solution concepts prevalent in non-cooperative games, and players have no incentives to deviate from their current strategies, even if they know others’ strategies at Nash equilibrium [162].

Given a non-cooperative game $\langle\mathcal{N},\mathcal{S}_{i},u_{i}\rangle$ , let $\mathcal{N}={\{1,2,3,\ldots,N\}}$ be the set of players, $\mathcal{S}_{i}$ be the possible strategy set of player $i$ and $u_{i}$ be the payoff function of player $i$ . The strategy profile $s^{*}=(s_{i}^{*},s_{-i}^{*})$ where $s_{-i}^{*}$ is strategies of all players except player $i$ is the Nash equilibrium if

\displaystyle u_{i}(s_{i}^{*},s_{-i}^{*})\geq u_{i}(s_{i},s_{-i}^{*}),\forall s_{i}\in S_{i},i\in\mathcal{N}

(8)

which can also be formulated as

\displaystyle u_{i}(s_{i}^{*},s_{-i}^{*})=\max u_{i}(s_{i},s_{-i}^{*}),\forall i\in\mathcal{N}

(9)

In short, each player’s Nash equilibrium is the best response to the other players’ Nash equilibrium [144].

II-D2 Pareto efficiency

Pareto efficiency (Pareto optimality) is an economic state where resources cannot be reallocated to make one individual better off without making at least one individual worse off [144]. Pareto efficiency can be expressed mathematically as

The strategy profile $s$ is Pareto efficient if there is no other strategy profile $s^{\prime}$ such that the payoff $u_{i}(s^{\prime})\geq u_{i}(s)$ for every player $i$ and $u_{j}(s^{\prime})\geq u_{j}(s^{\prime})$ for some player $j$ .

II-D3 Shapley value

The Shapley value is a popular single-valued solution concept for cooperative games, which provides a way to allocate total benefits to players in a coalition [159]. The Shapley value gives player $i$ in the coalition a share of

\displaystyle\phi_{i}(N,v)=\sum_{S\subseteq N\setminus{i}}\frac{\left\lvert S\right\rvert!(n-\left\lvert S\right\rvert-1)!}{n!}(v(S\cup{i})-v(S))

(10)

in which $n$ is the total number of players, $\left\lvert S\right\rvert$ is the cardinality of coalition $S$ , and $v(S)$ is the characteristic function indicating the worth of coalition $S$ .

II-D4 Core

In a transferable utility cooperative game, the core is the set of payoff allocations that are stable against deviations by any coalition of players, which ensures that no coalition has an incentive to break away and form a new coalition to obtain a more favorable payoff. Given a cooperative game $\langle N,v\rangle$ , the core $C(N,v)$ can be represented as

\displaystyle\begin{split}C(N,v)=\{\textbf{x}={x_{1},x_{2},x_{3},\ldots,x_{n}}\mid\sum_{i\in N}{x_{i}=v(N)},\\ and\;\sum_{i\in S}{x_{i}\geq v(S)}\;for\;all\;S\subset N\}\end{split}

(11)

where $N$ is a finite set of players, $v$ is the characteristic function, $S$ represents a coalition and $\textbf{x}=\{x_{1},x_{2},x_{3},\ldots,x_{n}\}$ is a payoff vector [163].

III MULTI-AGENT REINFORCEMENT LEARNING AT INTERSECTIONS

MARL has a significant connection to game theory, where players select actions to maximize payoffs in the presence of other payoff-maximizing players [164]. A multi-agent system (MAS) is an extension of agent technology where a group of loosely connected autonomous agents act in an environment to achieve a common goal, which can be done either by cooperating or competing, sharing or not sharing knowledge with each other [165]. Recently, the requirement for adaptive MAS, combined with the imperative to examine complicated interactions among agents, has spurred the advancement of the MARL field. This is built on two fundamental pillars: the RL performed within AI and the interdisciplinary work on game theory [166]. Game theory models MAS as a multi-player game and can provide a rational strategy for each player in a game [167].

III-A A Brief Overview of Multi-agent Reinforcement Learning

The single-agent reinforcement learning process can be formulated using a Markov decision process (MDP) [168]. The single agent will find the optimal policy through a trial-and-error process, which can maximize a possibly delayed reward in a stochastic stationary environment [166]. In the single-agent scenario, each agent solves the sequential decision-making problem by trial and error, while this comes with the difficulty of defining an optimal learning goal for multiple agents [169]. Agents have to simultaneously interact with the environment and others, and the reward function is formulated by all agent’s joint actions, making this problem even more challenging in this dynamic case. With the development in recent decades, the MARL has achieved remarkable accomplishments in many fields, such as StarCraft II [170], Atari [171], and Robotics [172], and it has also made ripples in transportation [173].

The decision-making process in multi-agent cases is usually modeled by the stochastic game in Definition 1 [174, 168].

Definition 1 (Stochastic game) A stochastic game is the extension of MDP in the multi-agent case, which can be defined by a tuple $\langle\mathcal{N},\mathcal{S},\{\mathcal{A}^{i}\}_{i\in\mathcal{N}},\mathcal{P},\{\mathcal{R}^{i}\}_{i\in\mathcal{N}},\gamma\rangle$ .
Where

•

$\mathcal{N}$ : the number of agents, $\mathcal{N}=\{1,2,\cdots,N\}$ , $\mathcal{N}>1$
•

$\mathcal{S}$ : the state space shared by all agents
•

$\mathcal{A}^{i}$ : the action space of agent $i$ . Let $\mathbb{A}$ := $\mathcal{A}^{1}\times\mathcal{A}^{2}\times\mathcal{A}^{3}\times\cdots\times\mathcal{A}^{N}$
•

$\mathcal{P}:\mathcal{S}\times\mathcal{A}^{i}\to\Delta(\mathcal{S})$ : the transition probability from any state $s\in\mathcal{S}$ to state $s^{\prime}\in\mathcal{S}$ for any joint actions $a\in\mathcal{A}$
•

$\mathcal{R}^{i}:\mathcal{S}\times\mathcal{A}\times\mathcal{S}\to\mathbb{R}$ : the reward function that returns a scalar value to the agent $i$ for a transition from $(s,a)$ to $s^{\prime}$
•

$\gamma\in[0,1]$ is the discount factor

At each time step $t$ , given the state $s_{t}$ , the agent $\in\mathcal{N}$ will execute the action $a_{t}^{i}$ , simultaneously with others. The joint actions of the agents make the system transit to the next state $s_{t+1}\sim P(\cdot|s_{t},a_{t})$ , and rewards of the agent $i$ by $R^{i}(s_{t},a_{t},s_{t+1})$ . The agent $i$ aims at optimizing its long-term reward in (12) by finding the policy $\pi^{i}:\mathcal{S}\to\Delta(\mathcal{A}^{i})$ such that $a^{i}\sim\pi^{i}(\cdot|s_{t})$ . The common shorthand notation $-i=\mathcal{N}\setminus{i}$ is used to represent the set of other agents. Therefore, the value function $V^{i}:\mathcal{S}\to\mathbb{R}$ of agent $i$ is transformed into the joint policy $\pi:\mathcal{S}\to\Delta(\mathcal{A})$ defined as $\pi(a|s):=\prod_{i\in\mathcal{N}}\pi^{i}(a^{i}|s)$ . Given the specific joint policy $\pi$ and state $s\in\mathcal{S}$ .

\displaystyle V_{\pi^{i},\pi^{-i}}^{i}(s):=\mathbb{E}[\sum_{t\geq 0}\gamma^{t}R^{i}(s_{t},a_{t},s_{t+1})|a_{t}^{i}\sim\pi^{i}(\cdot|s_{t}),s_{0}=s]

(12)

The optimal policy is

\displaystyle\pi_{i}^{*}(a_{i}|s,\pi_{-i})=\underset{\pi_{i}}{{\arg\max}}\,V_{i}^{\pi_{i},\pi_{-i}}(s)

(13)

We recommend that interested readers refer to [168] and [172] for a detailed introduction to the MARL.

III-B Benchmarks and Simulation Platforms

The opportunities of multi-agent learning in intersection management pointed out by Dresner and Stone have attracted and encouraged an increasing number of scholars to address the challenges of MARL in intersections [175]. Especially, MARL is widely considered to be promising for traffic signal control (TSC) in large-scale road networks. The topic of MARL applications for intersection management has attracted increasing attention from researchers.

In the domain of intersection management, the application of MARL often entails performance comparisons with conventionally recognized methods to underscore their advantages. Notably, several high-performing AI-based techniques have also gained prominence. Hence, we provide a list of potential benchmarks here to facilitate related research.

Conventional methods

•

Fixed time [176]
•

MaxPressure [177]
•

GreenWave [178]
•

Longest queue first [179]
•

SCOOT [180]
•

SOTL [181]

AI-based methods

•

PressLight [182]
•

CoLight [183]
•

MA2C [184]
•

MetaLight [185]
•

IG-RL [186]
•

MARLIN-ATSC [167]
•

MPLight [187]
•

AttendLight [188]

MARL requires considerable training before it can be applied in practice, so a simulation platform that provides a more realistic picture of traffic operations is critical. With the rapid development of MARL technology and its widespread application at intersections, some platforms are becoming compatible with multi-agent training. This section lists some prevailing platforms.

•

SUMO [189]
•

Paramics [190]
•

Aimsun [191]
•

VISSIM [192]
•

CityFlow [193]
•

Flow [194]
•

Traffic3D [195]
•

SMARTS [196]
•

highway-env [197]
•

Movsim [198]
•

Carla [199]

III-C Pros and Cons of Game Theory V.S. MARL

Both game theory and MARL can tackle decision-making in multi-agent environments. Game theory offers an analytical method, shedding light on players’ strategies and producing explainable outcomes like the Nash equilibrium and the Pareto optimality. However, its efficacy diminishes with continuous actions or an extensive number of agents while MARL is adept at generating continuous actions for numerous agents. Despite its strengths, MARL’s challenges include time-consuming training, transferability issues, and lack of explainability. Recognizing these possible shortcomings, hybrid algorithms like Minimax-Q and Nash-Q have emerged, integrating game solutions with MARL to derive learning-based equilibrium [200]. These hybrids aim to balance the rigor of game-theoretic solutions with the dynamic learning capabilities of MARL, addressing their individual limitations while leveraging their collective strengths.

IV Limitations and future research directions

To facilitate the adoption of game theory in intersection management, it is essential to address the technical challenges associated with these applications and to identify potential future research directions. This section discusses some of the main challenges in using game theory for intersection management, as well as promising approaches and future research directions in this field.

IV-A Limitations of existing related studies

In the previous sections, the applications of game theory for intersections were reviewed and summarized. However, despite numerous studies that have investigated various intersection environments, there are still many limitations, as listed below.

(i)

One significant challenge is the development of game-theoretic models in dynamic traffic environments. Traffic states at intersections can exhibit marked changes dynamically, such as the arrival of vehicles and vehicle states. Therefore, the game theory model should be able to adapt to such changes and to make real-time decisions. Additionally, the computational complexity of game-theoretic models can make them challenging to implement in real-time traffic management systems [20]. Thus, there is a need to develop more efficient algorithms and techniques to calculate with these models efficiently.
(ii)

Communication plays a crucial role in micro-level interactions among traffic participants, but some studies do not specify the degree of connectivity and intelligence required for effective communication. In general, vehicular communications rely on robust link and channel access protocols, such as IEEE 802.11p wireless access in vehicular environments (WAVE) [201]. It is worth noting several inherent communication issues: (a) frequent topology partitioning resulting from the high-speed mobility of traffic participants, (b) instability of long-range communication due to increased delay spread and diminished channel capacity, (c) complications arising from the use of conventional routing protocols, and (d) broadcast storm issue in high dense scenarios. Real-time performance is crucial for intersection management. Without a reliable communication network or accurate adherence to control strategies by vehicles, intersections can become highly disorganized and inefficient. Consequently, potential communication challenges need further discussion and investigation.
(iii)

NmT, who are highly involved in traffic and interact frequently still receive insufficient attention in related studies, and more research is needed to address their specific needs and challenges.
(iv)

The assumption of a complete information environment may be too idealistic in real-world scenarios, and traffic participants may misreport their private information to gain more benefits, resulting in unfair resource allocation. Thus, more research is needed to develop mechanisms to detect and to mitigate the effects of information asymmetry.
(v)

Existing studies have skillfully applied game theory to upper-level decision-making for autonomous driving. However, the upper-level decisions made by game theory models are not consistently well-integrated with lower-level vehicle motion planning and control.
(vi)

Game theory is formulated under the assumption of rational players, but players’ strategies can also be influenced by different characteristics. In existing studies, many utilitarian factors, such as total delay or throughput, are considered, but the real needs and preferences of traffic participants are often not adequately addressed. Heterogeneous users, with their diverse preferences and objectives, need further investigation in addition to the mere differentiation in the categories of players.

IV-B Future research directions

Based on the limitations of the available literature, several gaps and opportunities for future research in game theory-based intersection management can be identified. These include:

(i)

For micro-level interactions between vehicles, it is crucial to address the communication requirements of traffic participants in real-time scenarios. Simulating a connected environment by adding communication systems to traffic simulation is one way to improve the realism of the simulations. Additionally, field experiments under connected conditions can provide valuable insights.
(ii)

The interaction among MT and NmT should receive more attention in future research. These groups are highly involved in traffic and interact frequently, but their interactions are often overlooked in existing studies.
(iii)

Incomplete information environments are common in real-world scenarios, and mechanism design under incomplete information in a dynamic traffic environment should be highlighted. Future research can explore the interaction of traffic participants by considering incentive compatibility and other constraints.
(iv)

Combining optimization algorithms and game theory in decision-making is a promising research direction. For instance, game theory models can be used to decide whether to adopt the planned vehicle trajectory or speed profiles obtained from system-level optimization, thereby improving the efficiency of intersections.
(v)

Existing models assume rational players, but drivers’ and passengers’ strategies can also be influenced by their emotions and experience. Future research should consider the individual characteristics and bounded rationality of traffic participants. In addition, including more personal travel information in constructing the payoff function can make intersection management acceptable to all individuals.
(vi)

Future research directions in this field also include the exploration of new game-theoretic models and techniques for managing CAVs at intersections, the development of multi-agent reinforcement learning approaches for adaptive intersection management, and the investigation of game-theoretic approaches for managing interactions between MT and NmT.

Overall, game theory has the potential to revolutionize intersection management by providing more efficient and adaptive solutions to traffic control. However, significant technical challenges must be addressed, and innovative approaches must be developed to fully realize the benefits of game theory in this field.

V Conclusion

In conjunction with the advancement of intelligent transportation systems, incorporating knowledge of game theory and mechanism design becomes imperative for ensuring the ground truth collected individual travel information (such as the value of time), thereby fostering fair decision-making processes. Moreover, rather than the conventional collective approach to intersection management considering all vehicles in an approach as a unified group, there is a growing emphasis on personalized guidance. Game theory becomes instrumental in this shift, offering insights into individual behaviors and their interactions. In this paper, we provided a summary of the applications of game theory for urban intersections. First, we proposed and applied a workflow for systematic literature retrieval and analysis, yielding high-quality relevant literature. Then, numerous game-theoretic modeling approaches and corresponding possible solutions applied to intersections were reviewed and summarized. Concurrently, the widely discussed multi-agent reinforcement learning (MARL), which is inextricably linked to game theory, is briefly summarized. Finally, limitations and possible future directions in this area were pointed out. In conclusion, numerous innovative solutions for intersection management have been proposed by leveraging the advantages of game theory. Game theory excels in resolving traffic conflicts and harmonizing the interests of multiple traffic participants, making it an invaluable tool in the field of transportation management. As the intelligent transportation system continues to evolve, the strategic application of game theory for analyzing individual traffic participants’ interactions can significantly contribute to establishing a harmonious transportation ecosystem.

Acknowledgments

The work is supported by the National Natural Science Foundation of China via grant 52072316 and 52302418, the Postdoctoral International Exchange Program via grant YJ20220311, the Fundamental Research Funds for the Central Universities via grant 2682023CX047, and the Tencent-SWJTU Joint Laboratory of Intelligent Transportation Program via grant R113623H01015. Any opinions, findings, conclusions, or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the sponsors.

References

[1] A. Mirheli, M. Tajalli, L. Hajibabai, and A. Hajbabaie, “A consensus-based distributed trajectory control in a signal-free intersection,” Transp. Res. Part C, Emerg. Technol., vol. 100, pp. 161–176, 2019, doi: 10.1016/j.trc.2019.01.004.
[2] C. Yu, Y. Feng, H. X. Liu, W. Ma, and X. Yang, “Integrated optimization of traffic signals and vehicle trajectories at isolated urban intersections,” Transp. Res. Part B, Methodol., vol. 112, pp. 89–112, 2018, doi: 10.1016/j.trb.2018.04.007.
[3] B. Qian, H. Zhou, F. Lyu, J. Li, T. Ma, and F. Hou, “Toward collision-free and efficient coordination for automated vehicles at unsignalized intersection,” IEEE Internet Things J., vol. 6, no. 6, pp. 10 408–10 420, 2019, doi: 10.1109/JIOT.2019.2939180.
[4] K. C. Dey, A. Rayamajhi, M. Chowdhury, P. Bhavsar, and J. Martin, “Vehicle-to-vehicle (v2v) and vehicle-to-infrastructure (v2i) communication in a heterogeneous wireless network–performance evaluation,” Transp. Res. Part C, Emerg. Technol., vol. 68, pp. 168–184, 2016, doi: 10.1016/j.trc.2016.03.008.
[5] Z. Sun, T. Huang, and P. Zhang, “Cooperative decision-making for mixed traffic: A ramp merging example,” Transp. Res. Part C, Emerg. Technol., vol. 120, p. 102764, 2020, doi: 10.1016/j.trc.2020.102764.
[6] L. Chen, Y. Li, C. Huang, B. Li, Y. Xing, D. Tian, L. Li, Z. Hu, X. Na, Z. Li, S. Teng, C. Lv, J. Wang, D. Cao, N. Zheng, and F.-Y. Wang, “Milestones in autonomous driving and intelligent vehicles: Survey of surveys,” IEEE Trans. Intell. Veh., vol. 8, no. 2, pp. 1046–1056, 2023, doi: 10.1109/TIV.2022.3223131.
[7] K. Dresner and P. Stone, “A multiagent approach to autonomous intersection management,” J. Artif. Intell. Res., vol. 31, pp. 591–656, 2008, doi: 10.1613/jair.2502.
[8] W. Wang, L. Wang, C. Zhang, C. Liu, L. Sun et al., “Social interactions for autonomous driving: A review and perspectives,” Found. Trends Robot., vol. 10, no. 3-4, pp. 198–376, 2022, doi: 10.1561/2300000078.
[9] S. Pruekprasert, X. Zhang, J. Dubut, C. Huang, and M. Kishida, “Decision making for autonomous vehicles at unsignalized intersection in presence of malicious vehicles,” in Proc. IEEE Intell. Transp. Syst. Conf. (ITSC). IEEE, 2019, pp. 2299–2304, doi: 10.1109/ITSC.2019.8917132.
[10] J. Liu, P. Lin, and B. Ran, “A reservation-based coordinated transit signal priority method for bus rapid transit system with connected vehicle technologies,” IEEE Intell. Transp. Syst. Mag., vol. 13, no. 4, pp. 17–30, 2020, doi: 10.1109/MITS.2020.3014151.
[11] H. Liu, X.-Y. Lu, and S. E. Shladover, “Traffic signal control by leveraging cooperative adaptive cruise control (cacc) vehicle platooning capabilities,” Transp. Res. Part C, Emerg. Technol., vol. 104, pp. 390–407, 2019, doi: 10.1016/j.trc.2019.05.027.
[12] Y. Guo, J. Ma, C. Xiong, X. Li, F. Zhou, and W. Hao, “Joint optimization of vehicle trajectories and intersection controllers with connected automated vehicles: Combined dynamic programming and shooting heuristic approach,” Transp. Res. Part C, Emerg. Technol., vol. 98, pp. 54–72, 2019, doi: 10.1016/j.trc.2018.11.010.
[13] Y. Feng, C. Yu, and H. X. Liu, “Spatiotemporal intersection control in a connected and automated vehicle environment,” Transp. Res. Part C, Emerg. Technol., vol. 89, pp. 364–383, 2018, doi: 10.1016/j.trc.2018.02.001.
[14] P. Hao, G. Wu, K. Boriboonsomsin, and M. J. Barth, “Eco-approach and departure (ead) application for actuated signals in real-world traffic,” IEEE Trans. Intell. Transp. Syst., vol. 20, no. 1, pp. 30–40, 2018, doi: 10.1109/TITS.2018.2794509.
[15] P. Sun, D. Nam, R. Jayakrishnan, and W. Jin, “An eco-driving algorithm based on vehicle to infrastructure (v2i) communications for signalized intersections,” Transp. Res. Part C, Emerg. Technol., vol. 144, p. 103876, 2022, doi: 10.1016/j.trc.2022.103876.
[16] P. Lin, J. Liu, P. J. Jin, and B. Ran, “Autonomous vehicle-intersection coordination method in a connected vehicle environment,” IEEE Intell. Transp. Syst. Mag., vol. 9, no. 4, pp. 37–47, 2017, doi: 10.1109/MITS.2017.2743167.
[17] H. Yang, F. Almutairi, and H. Rakha, “Eco-driving at signalized intersections: A multiple signal optimization approach,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 5, pp. 2943–2955, 2020, doi: 10.1109/TITS.2020.2978184.
[18] B.-K. Xiong and R. Jiang, “Speed advice for connected vehicles at an isolated signalized intersection in a mixed traffic flow considering stochasticity of human driven vehicles,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 8, pp. 11 261–11 272, 2021, doi: 10.1109/TITS.2021.3102430.
[19] C. Chen, J. Wang, Q. Xu, J. Wang, and K. Li, “Mixed platoon control of automated and human-driven vehicles at a signalized intersection: dynamical analysis and optimal control,” Transp. Res. Part C, Emerg. Technol., vol. 127, p. 103138, 2021, doi: 10.1016/j.trc.2021.103138.
[20] N. Li, Y. Yao, I. Kolmanovsky, E. Atkins, and A. R. Girard, “Game-theoretic modeling of multi-vehicle interactions at uncontrolled intersections,” IEEE Trans. Intell. Transp. Syst., 2020, doi: 10.1109/TITS.2020.3026160.
[21] Z. Hu, J. Huang, D. Yang, and Z. Zhong, “Constraint-tree-driven modeling and distributed robust control for multi-vehicle cooperation at unsignalized intersections,” Transp. Res. Part C, Emerg. Technol., vol. 131, p. 103353, 2021, doi: 10.1016/j.trc.2021.103353.
[22] C. Chen, Q. Xu, M. Cai, J. Wang, J. Wang, and K. Li, “Conflict-free cooperation method for connected and automated vehicles at unsignalized intersections: Graph-based modeling and optimality analysis,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 11, pp. 21 897–21 914, 2022, doi: 10.1109/TITS.2022.3182403.
[23] J. Luo, T. Zhang, R. Hao, D. Li, C. Chen, Z. Na, and Q. Zhang, “Real-time cooperative vehicle coordination at unsignalized road intersections,” IEEE Trans. Intell. Transp. Syst., 2023, doi: 10.1109/TITS.2023.3243940.
[24] L. Huang and J. Wu, “Cyclists’ path planning behavioral model at unsignalized mixed traffic intersections in china,” IEEE Intell. Transp. Syst. Mag., vol. 1, no. 2, pp. 13–19, 2009, doi: 10.1109/MITS.2009.933859.
[25] I. K. Isukapati and S. F. Smith, “Accommodating high value-of-time drivers in market-driven traffic signal control,” in Proc. 2017 IEEE Intell. Veh. Symp. (IV). Los Angeles, CA, USA: IEEE, Jul. 2017, pp. 1280–1286, doi: 10.1109/IVS.2017.7995888.
[26] R. Niroumand, M. Tajalli, L. Hajibabai, and A. Hajbabaie, “Joint optimization of vehicle-group trajectory and signal timing: Introducing the white phase for mixed-autonomy traffic stream,” Transp. Res. Part C, Emerg. Technol., vol. 116, p. 102659, 2020, doi: 10.1016/j.trc.2020.102659.
[27] D. Lin and S. E. Jabari, “Pay for intersection priority: A free market mechanism for connected vehicles,” IEEE Trans. Intell. Transp. Syst., 2021, doi: 10.1109/TITS.2020.3048475.
[28] D. Zhao, Y. Dai, and Z. Zhang, “Computational intelligence in urban traffic signal control: A survey,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 42, no. 4, pp. 485–494, 2011, doi: 10.1109/TSMCC.2011.2161577.
[29] S. Araghi, A. Khosravi, and D. Creighton, “A review on computational intelligence methods for controlling traffic signal timing,” Expert Syst. Appl., vol. 42, no. 3, pp. 1538–1550, 2015, doi: 10.1016/j.eswa.2014.09.003.
[30] Y. Wang, X. Yang, H. Liang, and Y. Liu, “A review of the self-adaptive traffic signal control system based on future traffic environment,” J. Adv. Transp., vol. 2018, 2018, doi: 10.1155/2018/10961237.
[31] K.-L. A. Yau, J. Qadir, H. L. Khoo, M. H. Ling, and P. Komisarczuk, “A survey on reinforcement learning models and algorithms for traffic signal control,” ACM Comput. Surv., vol. 50, no. 3, pp. 1–38, 2017, doi: 10.1145/3068287.
[32] R. Florin and S. Olariu, “A survey of vehicular communications for traffic signal optimization,” Veh. Commun., vol. 2, no. 2, pp. 70–79, 2015, doi: 10.1016/j.vehcom.2015.03.002.
[33] L. Chen and C. Englund, “Cooperative intersection management: A survey,” IEEE Trans. Intell. Transp. Syst., vol. 17, no. 2, pp. 570–586, 2015, doi: 10.1109/TITS.2015.2471812.
[34] J. Rios-Torres and A. A. Malikopoulos, “A survey on the coordination of connected and automated vehicles at intersections and merging at highway on-ramps,” IEEE Trans. Intell. Transp. Syst., vol. 18, no. 5, pp. 1066–1077, 2016, doi: 10.1109/TITS.2016.2600504.
[35] Q. Guo, L. Li, and X. J. Ban, “Urban traffic signal control with connected and automated vehicles: A survey,” Transp. Res. Part C, Emerg. Technol., vol. 101, pp. 313–334, 2019, doi: 10.1016/j.trc.2019.01.026.
[36] M. Khayatian, M. Mehrabian, E. Andert, R. Dedinsky, S. Choudhary, Y. Lou, and A. Shirvastava, “A survey on intersection management of connected autonomous vehicles,” ACM Trans. Cyber-Phys. Syst, vol. 4, no. 4, pp. 1–27, 2020, doi: 10.1145/3407903.
[37] E. Namazi, J. Li, and C. Lu, “Intelligent intersection management systems considering autonomous vehicles: A systematic literature review,” IEEE Access, vol. 7, pp. 91 946–91 965, 2019, doi: 10.1109/ACCESS.2019.2927412.
[38] M. Al-Turki, N. T. Ratrout, S. M. Rahman, and K. J. Assi, “Signalized intersection control in mixed autonomous and regular vehicles traffic environment–a critical review focusing on future control,” IEEE Access, 2022, doi: 10.1109/ACCESS.2022.3148706.
[39] M. S. Shirazi and B. T. Morris, “Looking at intersections: a survey of intersection monitoring, behavior and safety analysis of recent studies,” IEEE Trans. Intell. Transp. Syst., vol. 18, no. 1, pp. 4–24, 2016, doi: 10.1109/TITS.2016.2568920.
[40] Z. Zhong, M. Nejad, and E. E. Lee, “Autonomous and semiautonomous intersection management: A survey,” IEEE Intell. Transp. Syst. Mag., vol. 13, no. 2, pp. 53–70, 2020, doi: 10.1109/MITS.2020.3014074.
[41] C. Iliopoulou, K. Kepaptsoglou, and E. I. Vlahogianni, “A survey on market-inspired intersection control methods for connected vehicles,” IEEE Intell. Transp. Syst. Mag., vol. 15, no. 2, pp. 162–176, 2022, doi: 10.1109/MITS.2022.3203573.
[42] R. B. Myerson, Game theory: Analysis of conflict. Harvard University Press, 1997.
[43] A. Ji and D. Levinson, “A review of game theory models of lane changing,” Transportmetrica A, Transport Sci., vol. 16, no. 3, pp. 1628–1647, 2020, doi: 10.1080/23249935.2020.1770368.
[44] C. A. Kamhoua, C. D. Kiekintveld, F. Fang, and Q. Zhu, Game theory and machine learning for cyber security. John Wiley & Sons, 2021.
[45] F. Ahmad, O. Almarri, Z. Shah, and L. Al-Fagih, “Game theory applications in traffic management: A review of authority-based travel modelling,” Travel Behav. Soc., vol. 32, p. 100585, 2023, doi: 10.1016/j.tbs.2023.100585.
[46] Q. Zhang, R. Langari, H. E. Tseng, D. Filev, S. Szwabowski, and S. Coskun, “A game theoretic model predictive controller with aggressiveness estimation for mandatory lane change,” IEEE Trans. Intell. Veh., vol. 5, no. 1, pp. 75–89, 2020, doi: 10.1109/TIV.2019.2955367.
[47] A. Ji and D. Levinson, “Estimating the social gap with a game theory model of lane changing,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 10, pp. 6320–6329, 2020, doi: 10.1109/TITS.2020.2991242.
[48] A. Ji, M. Ramezani, and D. Levinson, “Pricing lane changes,” Transp. Res. Part C, Emerg. Technol., vol. 149, p. 104062, 2023, doi: 10.1016/j.trc.2023.104062.
[49] B. Wang, Z. Li, S. Wang, M. Li, and A. Ji, “Modeling bounded rationality in discretionary lane change with the quantal response equilibrium of game theory,” Transp. Res. Part B, Methodol., vol. 164, pp. 145–161, 2022, doi: 10.1016/j.trb.2022.08.008.
[50] S. Karimi, A. Karimi, and A. Vahidi, “Level- $k$ reasoning, deep reinforcement learning, and monte carlo decision process for fast and safe automated lane change and speed management,” IEEE Trans. Intell. Veh., vol. 8, no. 6, pp. 3556–3571, 2023, doi: 10.1109/TIV.2023.3265311.
[51] Z. Sun, Z. Qin, R. Ma, T. Huang, Z. Gao, and A. Ji, “Microscopic right-of-way trading mechanism for cooperative decision-making: Theories and preliminary results,” SSRN Electron. J., 2023, doi: 10.2139/ssrn.4571173.
[52] X. Liao, X. Zhao, Z. Wang, K. Han, P. Tiwari, M. J. Barth, and G. Wu, “Game theory-based ramp merging for mixed traffic with unity-sumo co-simulation,” IEEE Trans. Syst. Man Cybern.: Syst., vol. 52, no. 9, pp. 5746–5757, 2021, doi: 10.1109/TSMC.2021.3131431.
[53] W. Li, F. Qiu, L. Li, Y. Zhang, and K. Wang, “Simulation of vehicle interaction behavior in merging scenarios: A deep maximum entropy- inverse reinforcement learning method combined with game theory,” IEEE Trans. Intell. Veh., pp. 1–15, 2023, doi: 10.1109/TIV.2023.3323138.
[54] Z. Sun, T. Huang, Z. Qin, and Z. Gao, “Microscopic right-of-way trading mechanism for cooperative decision-making: A ramp merging example,” in Proc. Transp. Res. Board 99th Annu. Meeting (TRB), Washington, DC, 2020.
[55] M. O. Sayin, C.-W. Lin, S. Shiraishi, J. Shen, and T. Başar, “Information-driven autonomous intersection control via incentive compatible mechanisms,” IEEE Trans. Intell. Transp. Syst., vol. 20, no. 3, pp. 912–924, 2018, doi: 10.1109/TITS.2018.2838049.
[56] Y. Wang, Y. Ren, S. Elliott, and W. Zhang, “Enabling courteous vehicle interactions through game-based and dynamics-aware intent inference,” IEEE Trans. Intell. Veh., vol. 5, no. 2, pp. 217–228, 2020, doi: 10.1109/TIV.2019.2955897.
[57] K. Wang, Y. Wang, H. Du, and K. Nam, “Game-theory-inspired hierarchical distributed control strategy for cooperative intersection considering priority negotiation,” IEEE Trans. Veh. Technol., vol. 70, no. 7, pp. 6438–6449, 2021, doi: 10.1109/TVT.2021.3086563.
[58] R. Lloret-Batlle and R. Jayakrishnan, “Envy-minimizing pareto efficient intersection control with brokered utility exchanges under user heterogeneity,” Transp. Res. Part B, Methodol., vol. 94, pp. 22–42, 2016, doi: 10.1016/j.trb.2016.08.014.
[59] H. M. Abdelghaffar and H. A. Rakha, “Development and testing of a novel game theoretic de-centralized traffic signal controller,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 1, pp. 231–242, 2019, doi: 10.1109/TITS.2019.2955918.
[60] Z. Sun, X. Yao, Z. Qin, P. Zhang, and Z. Yang, “Modeling car-following heterogeneities by considering leader–follower compositions and driving style differences,” Transp. Res. Rec., vol. 2675, no. 11, pp. 851–864, 2021, doi: 10.1177/03611981211020006.
[61] L. Zhang, X. W. huang, and W. M. Wu, “The analysis of driver’s behavior in non-signalized intersection based on the game,” Appl. Mech. Mater., vol. 505, pp. 1157–1162, 2014, doi: 10.4028/www.scientific.net/AMM.505-506.1157.
[62] Y. Zhang, R. Hao, T. Zhang, X. Chang, Z. Xie, and Q. Zhang, “A trajectory optimization-based intersection coordination framework for cooperative autonomous vehicles,” IEEE Trans. Intell. Transp. Syst., 2021, doi: 10.1109/TITS.2021.3131570.
[63] B. Wolshon, A. Pande et al., Traffic engineering handbook. John Wiley & Sons, 2016.
[64] F. Zhu and S. V. Ukkusuri, “A linear programming formulation for autonomous intersection control within a dynamic traffic assignment and connected vehicle environment,” Transp. Res. Part C, Emerg. Technol., vol. 55, pp. 363–378, 2015, doi: 10.1016/j.trc.2015.01.006.
[65] M. W. Hancock and B. Wright, “A policy on geometric design of highways and streets,” American Association of State Highway and Transportation Officials: Washington, DC, USA, vol. 3, 2013.
[66] L. A. Rodegerdts, B. L. Nevers, B. Robinson, J. Ringert, P. Koonce, J. Bansen, T. Nguyen, J. McGill, D. Stewart, J. Suggett et al., “Signalized intersections: informational guide,” United States. Federal Highway Administration, Tech. Rep., 2004.
[67] K. H. N. Bui, J. E. Jung, and D. Camacho, “Game theoretic approach on Real-time decision making for IoT-based traffic light control,” Concurrency Comput. Pract. Exper., vol. 29, no. 11, 2017, doi: 10.1002/cpe.4077.
[68] Y. Zhao, Y. Liang, J. Hu, and Z. Zhang, “Traffic Signal Control for Isolated Intersection Based on Coordination Game and Pareto Efficiency,” in Proc. 2019 IEEE Intell. Transp. Syst. Conf., Auckland, New Zealand, 2019, pp. 3508–3513, doi: 10.1109/ITSC.2019.8917165.
[69] M. Khanjary, “Using game theory to optimize traffic light of an intersection,” in Proc. 14th Int. Symp. Comput. Intell. Inform., Budapest, Hungary, 2013, pp. 249–253, doi: 10.1109/CINTI.2013.6705201.
[70] N. Li, I. Kolmanovsky, A. Girard, and Y. Yildiz, “Game theoretic modeling of vehicle interactions at unsignalized intersections and application to autonomous vehicle control,” in Proc. Annu. Amer. Control Conf. (ACC), 2018, pp. 3215–3220, doi: 10.23919/ACC.2018.8430842.
[71] M. Elhenawy, A. A. Elbery, A. A. Hassan, and H. A. Rakha, “An intersection game-theory-based traffic control algorithm in a connected vehicle environment,” in Proc. IEEE 18th Int. Conf. Intell. Transp. Syst, Gran Canaria, Spain, 2015, pp. 343–347, doi: 10.1109/ITSC.2015.65.
[72] C. Cheng, Z. Yang, and D. Yao, “A speed guide model for collision avoidance in non-signalized intersections based on reduplicate game theory,” in Proc. IEEE Intell. Vehicles Symp. (IV), Changshu, China, 2018, pp. 1614–1619, doi: 10.1109/IVS.2018.8500502.
[73] X. Chen, Y. Sun, Y. Ou, X. Zheng, Z. Wang, and M. Li, “A conflict decision model based on game theory for intelligent vehicles at urban unsignalized intersections,” IEEE Access, vol. 8, pp. 189 546–189 555, 2020, doi: 10.1109/ACCESS.2020.3031674.
[74] X. Jin, K. Li, Q.-S. Jia, H. Xia, Y. Bai, and D. Ren, “A game-theoretic reinforcement learning approach for adaptive interaction at intersections,” in Proc. Chin. Autom. Congr. (CAC), Shanghai, China, 2020, pp. 4451–4456, doi: 10.1109/CAC51589.2020.9327245.
[75] R. Tian, S. Li, N. Li, I. Kolmanovsky, A. Girard, and Y. Yildiz, “Adaptive game-theoretic decision making for autonomous vehicle control at roundabouts,” in Proc. IEEE Conf. Decis. Control (CDC), Miami, FL, USA, Dec. 2018, pp. 321–326, doi: 10.1109/CDC.2018.8619275.
[76] I. H. Zohdy and H. Rakha, “Game theory algorithm for intersection-based cooperative adaptive cruise control (cacc) systems,” in Proc. IEEE 15th Int. Conf. Intell. Transp. Syst., Anchorage, AK, USA, Sep. 2012, pp. 1097–1102, doi: 10.1109/ITSC.2012.6338644.
[77] Z. Yang, H. Huang, D. Yao, and Y. Zhang, “Cooperative driving model for non-signalized intersections based on reduplicate dynamic game,” in Proc. IEEE 19th Int. Conf. Intell. Transp. Syst. (ITSC), Rio de Janeiro, Nov. 2016, pp. 1366–1371, doi: 10.1109/ITSC.2016.7795735.
[78] H. Fan, B. Jia, J. Tian, and L. Yun, “Characteristics of traffic flow at a non-signalized intersection in the framework of game theory,” Phys. A, Statist. Mech. Appl., vol. 415, pp. 172–180, 2014, doi: 10.1016/j.physa.2014.07.031.
[79] W. Zhang and W. Chen, “Dilemma game in a cellular automaton model with a non-signalized intersection,” Eur. Phys. J. B, vol. 85, no. 2, pp. 1–8, 2012, doi: 10.1140/epjb/e2012-20904-x.
[80] L. Wang, N. G. Xie, and R. Meng, “Dirty-face game analysis on mixed traffic flow at unsignalized intersection,” Adv. Mat. Res., vol. 201, pp. 2119–2125, 2011, doi: 10.4028/www.scientific.net/AMR.201-203.2119.
[81] M. Lemmer, S. Schwab, and S. Hohmann, “Driver interaction at intersections: A hybrid dynamic game based model,” in Proc. IEEE Int. Conf. Syst., Man, Cybern. (SMC), Toronto, ON, Canada, Oct. 2020, pp. 2269–2276, doi: 10.1109/SMC42975.2020.9283079.
[82] P. Hang, C. Huang, Z. Hu, and C. Lv, “Driving conflict resolution of autonomous vehicles at unsignalized intersections: A differential game approach,” IEEE/ASME Trans. Mechatron., 2022, doi: 10.1109/TMECH.2022.3174273.
[83] X. Qiu, L. Yang, L. Zhang, and Z. Huang, “The non-signal intersection traffic behavior analysis based on game theory,” in Proc. 4th Int. Symp. Comput. Intell. Design (ISCID), vol. 2, Hangzhou, China, Oct. 2011, pp. 122–124, doi: 10.1109/ISCID.2011.132.
[84] S. I. Bouderba and N. Moussa, “Evolutionary dilemma game for conflict resolution at unsignalized traffic intersection,” Int. J. Mod. Phys. C, vol. 30, no. 02n03, p. 1950018, 2019, doi: 10.1142/S0129183119500189.
[85] J. Cai, P. Hang, and C. Lv, “Game theoretic modeling and decision making for connected vehicle interactions at urban intersections,” in Proc. IEEE Int. Conf. Adv. Robot. Mech (ICARM), Chongqing, China, Jul. 2021, pp. 874–880, doi: 10.1109/ICARM52023.2021.9536147.
[86] W. Qi, H. Wen, C. Fu, and M. Song, “Game theory model of traffic participants within amber time at signalized intersection,” Comput. Intell. Neurosci., vol. 2014, pp. 56–56, 2014, doi: 10.1155/2014/756235.
[87] G. Liu, B. Xiao, and D. Li, “Game-theory based driving decision algorithm for intersection scenarios considering driver irrationality,” in Proc. IEEE 4th CAA Int. Conf. Veh. Control Intell (CVCI), Hangzhou, China, Dec. 2020, pp. 747–752, doi: 10.1109/CVCI51460.2020.9338515.
[88] D. Li, G. Liu, and B. Xiao, “Human-like driving decision at unsignalized intersections based on game theory,” Proc. Inst. Mech. Eng., Part D: J. Automob. Eng., pp. 159–173, 2022, doi: 10.1177/09544070221075423.
[89] A. Baz, P. Yi, and A. Qurashi, “Intersection control and delay optimization for autonomous vehicles flows only as well as mixed flows with ordinary vehicles,” Vehicles, vol. 2, no. 3, pp. 523–541, 2020, doi: 10.3390/vehicles2030029.
[90] H. Wei, L. Mashayekhy, and J. Papineau, “Intersection management for connected autonomous vehicles: A game theoretic framework,” in Proc. 21st Int. Conf. Intell. Transp. Syst. (ITSC), Maui, HI, USA, Nov. 2018, doi: 10.1109/ITSC.2018.8569307.
[91] M. Liu, Y. Chen, G. Lu, and Y. Wang, “Modeling crossing behavior of drivers at unsignalized intersections with consideration of risk perception,” Transp. Res. Part F, Traffic Psychol. Behav., vol. 45, pp. 14–26, 2017, doi: 10.1016/j.trf.2016.11.012.
[92] M. Lemmer, S. Schwab, and S. Hohmann, “Driver interaction at intersections: A hybrid dynamic game based model,” in Proc. IEEE Int. Conf. Syst., Man, Cybern. (SMC), Toronto, ON, Canada, Oct. 2020, doi: 10.1109/SMC42975.2020.9283079.
[93] D. Lin and S. E. Jabari, “Comparative analysis of economic instruments in intersection operation: A user-based perspective,” in Proc. 2020 IEEE 23rd Int. Conf. Int. Transp. Syst. (ITSC), Rhodes, Greece, Sep. 2020, pp. 1–6, doi: 10.1109/ITSC45102.2020.9294641.
[94] M. O. Sayin, C.-W. Lin, S. Shiraishi, and T. Başar, “Reliable intersection control in non-cooperative environments,” in Proc. Annu. Amer. Control Conf. (ACC), Milwaukee, WI, USA, Jun. 2018, pp. 609–614, doi: 10.23919/ACC.2018.8430809.
[95] D. Li, A. Liu, H. Pan, and W. Chen, “Safe, efficient and socially-compatible decision of automated vehicles: a case study of unsignalized intersection driving,” Automot. Innov., pp. 1–16, 2023, doi: 10.1007/s42154-023-00219-2.
[96] M. Liu, Y. Wan, F. L. Lewis, S. Nageshrao, and D. Filev, “A three-level game-theoretic decision-making framework for autonomous vehicles,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 11, pp. 20 298–20 308, 2022, doi: 10.1109/TITS.2022.3172926.
[97] D. Zhu, N. N. Sze, Z. Feng, and Z. Yang, “A two-stage safety evaluation model for the red light running behaviour of pedestrians using the game theory,” Safety Sci., vol. 147, no. November 2021, p. 105600, 2022, doi: 10.1016/j.ssci.2021.105600.
[98] C. Yang, J. Wang, and J. Dong, “Capacity Model of Exclusive Right-Turn Lane at Signalized Intersection considering Pedestrian-Vehicle Interaction,” J. Adv. Transp., vol. 2020, 2020, doi: 10.1155/2020/1534564.
[99] H. Wang, Q. Meng, S. Chen, and X. Zhang, “Competitive and cooperative behaviour analysis of connected and autonomous vehicles across unsignalised intersections: A game-theoretic approach,” Transp. Res. Part B, Methodol., vol. 149, pp. 322–346, 2021, doi: 10.1016/j.trb.2021.05.007.
[100] Y. Rahmati, M. K. Hosseini, and A. Talebpour, “Helping Automated Vehicles With Left-Turn Maneuvers: A Game Theory-Based Decision Framework for Conflicting Maneuvers at Intersections,” IEEE Trans. Intell. Transp. Syst., pp. 1–14, 2021, doi: 10.1109/TITS.2021.3108409.
[101] D. Arbis, V. V. Dixit, and T. H. Rashidi, “Impact of risk attitudes and perception on game theoretic driving interactions and safety,” Accid. Anal. Prev., vol. 94, pp. 135–142, 2016, doi: 10.1016/j.aap.2016.05.027.
[102] X. Lu, H. Zhao, C. Li, B. Gao, and H. Chen, “A game-theoretic approach on conflict resolution of autonomous vehicles at unsignalized intersections,” IEEE Trans. Intell. Transp. Syst., 2023, doi: 10.1109/TITS.2023.3285597.
[103] S. Jia, Y. Zhang, X. Li, X. Na, Y. Wang, B. Gao, B. Zhu, and R. Yu, “Interactive decision-making with switchable game modes for automated vehicles at intersections,” IEEE Trans. Intell. Transp. Syst., 2023, doi: 10.1109/TITS.2023.3286075.
[104] F. Molinari and J. Raisch, “Automation of Road Intersections Using Consensus-based Auction Algorithms,” in Proc. Annu. Amer. Control Conf. (ACC), vol. 2018-June, Milwaukee, WI, USA, Jun. 2018, pp. 5994–6001, doi: 10.23919/ACC.2018.8430865.
[105] Y. Xu, D. Li, and Y. Xi, “A Game-Based Adaptive Traffic Signal Control Policy Using the Vehicle to Infrastructure (V2I),” IEEE Trans. Veh. Technol., vol. 68, no. 10, pp. 9425–9437, 2019, doi: 10.1109/TVT.2019.2933317.
[106] M. Vasirani and S. Ossowski, “A Market-Inspired Approach for Intersection Management in Urban Road Traffic Networks,” J. Artif. Intell. Res., vol. 43, pp. 621–659, 2012, doi: 10.1613/jair.3560.
[107] C. Cheng, D. Yao, Y. Zhang, J. Li, and Y. Guo, “A Vehicle Passing Model in Non-signalized Intersections based on Non-cooperative Game Theory,” Proc. 2019 IEEE Intell. Transp. Syst. Conf., pp. 2286–2291, Oct. 2019, doi: 10.1109/ITSC.2019.8917396.
[108] G. Cabri, L. Gherardini, M. Montangero, and F. Muzzini, “About auction strategies for intersection management when human-driven and autonomous vehicles coexist,” Multimed. Tools. Appl., pp. 15 921–15 936, 2021, doi: 10.1007/s11042-020-10222-y.
[109] D. Carlino, S. D. Boyles, and P. Stone, “Auction-based autonomous intersection management,” in Proc. 16th Int. IEEE Conf. Intell. Transp. Syst. (ITSC), The Hague, Netherlands, Oct. 2013, pp. 529–534, doi: 10.1109/ITSC.2013.6728285.
[110] H. Schepperle and K. Böhm, “Auction-based traffic management: Towards effective concurrent utilization of road intersections,” in Proc. 2008 10th IEEE Conf. E-Commerce Tech. 5th IEEE Conf. Enterprise Comput., E-Commerce E-Services, Arlington, VA, USA, 2008, pp. 105–112, doi: 10.1109/CECandEEE.2008.88.
[111] P. Hang, C. Huang, Z. Hu, and C. Lv, “Decision making for connected automated vehicles at urban intersections considering social and individual benefits,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 11, pp. 22 549–22 562, 2022, doi: 10.1109/TITS.2022.3209607.
[112] N. Suriyarachchi, R. Chandra, J. S. Baras, and D. Manocha, “Gameopt: Optimal real-time multi-agent planning and control for dynamic intersections,” in Proc. IEEE 25th Int. Conf. Intell. Transp. Syst., Macau, China, Nov. 2022, pp. 2599–2606, doi: 10.1109/ITSC55140.2022.9921968.
[113] R. Chandra and D. Manocha, “GamePlan: Game-Theoretic Multi-Agent Planning with Human Drivers at Intersections, Roundabouts, and Merging,” IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 2676–2683, 2022, doi: 10.1109/LRA.2022.3144516.
[114] F. Molinari, A. M. Dethof, and J. Raisch, “Traffic automation in urban road networks using consensus-based auction algorithms for road intersections,” Proc. 18th Eur. Control Conf., pp. 3008–3015, Jun. 2019, doi: 10.23919/ECC.2019.8796170.
[115] R. Tian, N. Li, I. Kolmanovsky, Y. Yildiz, and A. R. Girard, “Game-Theoretic Modeling of Traffic in Unsignalized Intersection Network for Autonomous Vehicle Control Verification and Validation,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 3, pp. 2211–2226, 2022, doi: 10.1109/TITS.2020.3035363.
[116] M. W. Levin and S. D. Boyles, “Intersection auctions and reservation-based control in dynamic traffic assignment,” Transp. Res. Rec., vol. 2497, pp. 35–44, 2015, doi: 10.3141/2497-04.
[117] M. Liu, S. Zhang, and L. Teng, “Method of Controlling Vehicles at Intersections Based on the Auction Algorithm,” J. Phys. Conf. Ser., vol. 1453, no. 1, 2020, doi: 10.1088/1742-6596/1453/1/012050.
[118] H. Wang, Y. Li, and H. V. Zhao, “Performance analysis of road intersections based on game theory and dynamic level-k model,” Proc. IEEE Int. Conf Parallel Distrib. Process. With Appl., Big Data Cloud Comput., Sustain. Comput. Commun., Social Comput. Netw. (ISPA/BDCloud/SocialCom/SustainCom), pp. 1112–1119, Dec. 2020, doi: 10.1109/ISPA-BDCloud-SocialCom-SustainCom51426.2020.00166.
[119] C. Philippe, L. Adouane, A. Tsourdos, H.-S. Shin, and B. Thuilot, “Probability collectives algorithm applied to decentralized intersection coordination for connected autonomous vehicles,” in Proc. IEEE Intell. Vehicles Symp. (IV), Paris, France, Jun. 2019, pp. 1928–1934, doi: 10.1109/IVS.2019.8813827.
[120] W. Schwarting, A. Pierson, J. Alonso-Mora, S. Karaman, and D. Rus, “Social behavior for autonomous vehicles,” Proc. Natl. Acad. Sci., vol. 116, no. 50, pp. 2492–24 978, 2019, doi: 10.1073/pnas.1820676116.
[121] A. Sarkar and K. Czarnecki, “Solution Concepts in Hierarchical Games Under Bounded Rationality With Applications to Autonomous Driving,” in Proc. 35th Conf. Artif. Intell. (AAAI), vol. 6B, no. Samuelson 1995, 2021, pp. 5698–5708, doi: 10.1609/aaai.v35i6.16715.
[122] M. Yuan, J. Shan, and K. Mi, “Deep reinforcement learning based game-theoretic decision-making for autonomous vehicles,” IEEE Robot. Autom. Lett., vol. 7, no. 2, pp. 818–825, 2021, doi: 10.1109/LRA.2021.3134249.
[123] D. Rey, M. W. Levin, and V. V. Dixit, “Online incentive-compatible mechanisms for traffic intersection auctions,” Eur. J. Oper. Res., vol. 293, no. 1, pp. 229–247, 2021, doi: 10.1016/j.ejor.2020.12.030.
[124] L. Tan, X. Zhao, D. Hu, Y. Shang, and W. Ren, “A study of single intersection traffic signal control based on two-player cooperation game model,” in Proc. Int. Conf. Inf. Eng, vol. 2, Beidai, China, Aug. 2010, pp. 322–327, doi: 10.1109/ICIE.2010.172.
[125] L. Tan, Y. Wang, Q. Wang, Y. Sun, and Z. Li, “Multi-phase signal control method for single intersection based on multi-person cooperative game theory,” in Proc. 2017 2nd Jt. Int. Inf. Technol. Mech. Electron. Eng. Conf. (JIMEC 2017), Oct. 2017, pp. 335–339, doi: 10.2991/jimec-17.2017.74.
[126] K. H. Nam Bui and J. J. Jung, “Cooperative game-theoretic approach to traffic flow optimization for multiple intersections,” Comput. Electr. Eng., vol. 71, pp. 1012–1024, 2018, doi: 10.1016/j.compeleceng.2017.10.016.
[127] H. Dong and Z. Dai, “A multi intersections signal coordinate control method based on game theory,” Proc. Int. Conf. Electron., Commun. Control (ICECC), no. 3, pp. 1232–1235, 2011, doi: 10.1109/ICECC.2011.6066604.
[128] J. Raphael, E. I. Sklar, and S. Maskell, “An intersection-centric auction-based traffic signal control framework,” in Agent-Based Modeling of Sustainable Behaviors. Cham, Switzerland: Springer International Publishing, 2017, pp. 121–142, doi: 10.1007/978-3-319-46331-5_6.
[129] X.-h. Xia, “Adaptive traffic signal coordinated timing decision for adjacent intersections with chicken game,” in Intelligent Transport Systems – From Research and Development to the Market Uptake, T. Kováčiková, Ľ. Buzna, G. Pourhashem, G. Lugano, Y. Cornet, and N. Lugano, Eds. Cham: Springer International Publishing, 2018, pp. 239–251, doi: 10.1007/978-3-319-93710-6_25.
[130] M. Abdoos, “A cooperative multiagent system for traffic signal control using game theory and reinforcement learning,” IEEE Intell. Transp. Syst. Mag., vol. 13, no. 4, pp. 6–16, 2021, doi: 10.1109/MITS.2020.2990189.
[131] J. B. Clempner and A. S. Poznyak, “Modeling the multi-traffic signal-control synchronization: A Markov chains game theory approach,” Eng. Appl. Artif. Intell., vol. 43, pp. 147–156, 2015, doi: 10.1016/j.engappai.2015.04.009.
[132] R. Castillo González, J. B. Clempner, and A. S. Poznyak, “Solving traffic queues at controlled-signalized intersections in continuous-time Markov games,” Math. Comput. Simul., vol. 166, pp. 283–297, 2019, doi: 10.1016/j.matcom.2019.06.002.
[133] L. H. Xu, X. H. Xia, and Q. Luo, “The study of reinforcement learning for traffic self-adaptive control under multiagent Markov game environment,” Math. Probl. Eng., vol. 2013, 2013, doi: 10.1155/2013/962869.
[134] Y. Zhu, Z. He, and G. Li, “A bi-hierarchical game-theoretic approach for network-wide traffic signal control using trip-based data,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 9, pp. 15 408–15 419, 2022, doi: 10.1109/TITS.2022.3140511.
[135] M. Stryszowski, S. Longo, D. D’Alessandro, E. Velenis, G. Forostovsky, and S. Manfredi, “A Framework for Self-Enforced Optimal Interaction between Connected Vehicles,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 10, pp. 6152–6161, 2021, doi: 10.1109/TITS.2020.2988150.
[136] Z. Dai, H. Dong, and Q. Wang, “A multi-intersection coordinated control algorithm based on game theory and maximal flow,” in Proc. IECON 39th Annu. Conf. IEEE Ind. Electron. Soc., Vienna, Nov. 2013, pp. 3258–3263, doi: 10.1109/IECON.2013.6699650.
[137] R. P. Adkins, D. M. Mount, and A. A. Zhang, “A game-theoretic approach for minimizing delays in autonomous intersections,” in Traffic and Granular Flow ’17, S. H. Hamdar, Ed. Cham: Springer International Publishing, 2019, pp. 131–139.
[138] A. J. Malo Tamayo, A. Poznyak, and I. Á. Villalobos, “Optimization and game at an urban intersection,” IFAC Proc. Vol., no. 1983, pp. 526–530, 2009, doi: 10.3182/20090902-3-US-2007.0030.
[139] J. Guo and I. Harmati, “Evaluating semi-cooperative Nash/Stackelberg Q-learning for traffic routes plan in a single intersection,” Control Eng. Pract., vol. 102, no. June, p. 104525, 2020, doi: 10.1016/j.conengprac.2020.104525.
[140] M. Mashayekhi and G. F. List, “A multiagent auction-based approach for modeling of signalized intersections,” in Proc. IJCAI Workshops Synergies Between Multiagent Syst., Mach. Learn. Complex Syst., 2015.
[141] J. Guo and I. Harmati, “Optimization of traffic signal control with different game theoretical strategies,” in Proc. 2019 23rd Int. Conf. Syst. Theory, Control Comput., Sinaia, Romania, Oct. 2019, pp. 750–755, doi: 10.1109/ICSTCC.2019.8885458.
[142] S. Moya and J. Escobar, “Stackelberg-Nash equilibrium in a traffic control problem at an intersection on a priority road,” IMA J. Math. Control Inf., vol. 32, no. 1, pp. 161–194, 2015, doi: 10.1093/imamci/dnt036.
[143] J. Guo and I. Harmati, “Comparison of game theoretical strategy and reinforcement learning in traffic light control,” Periodica Polytechnica Transp. Eng., vol. 48, no. 4, pp. 313–319, 2020, doi: 10.3311/PPTR.15923.
[144] Y. Narahari, Game theory and mechanism design. World Scientific, 2014, vol. 4.
[145] J. C. Hayward, “Near-miss determination through use of a scale of danger,” Highway Res. Rec., no. 384, 1972.
[146] Y. Zhang, D. Yao, T. Z. Qiu, L. Peng, and Y. Zhang, “Pedestrian safety analysis in mixed traffic conditions using video data,” IEEE Trans. Intell. Transp. Syst., vol. 13, no. 4, pp. 1832–1844, 2012, doi: 10.1109/TITS.2012.2210881.
[147] A. Varhelyi, “Drivers’ speed behaviour at a zebra crossing: a case study,” Accid. Anal. Prev., vol. 30, no. 6, pp. 731–743, 1998, doi: 10.1016/S0001-4575(98)00026-8.
[148] H. M. Abdelghaffar and H. A. Rakha, “A novel decentralized game-theoretic adaptive traffic signal controller: Large-scale testing,” Sensors, vol. 19, no. 10, p. 2282, 2019, doi: 10.3390/s19102282.
[149] P. Hao, X. J. Ban, D. Guo, and Q. Ji, “Cycle-by-cycle intersection queue length distribution estimation using sample travel times,” Transp. Res. Part B, Methodol., vol. 68, pp. 185–204, 2014, doi: 10.1016/j.trb.2014.06.004.
[150] X. J. Ban, P. Hao, and Z. Sun, “Real time queue length estimation for signalized intersections using travel times from mobile sensors,” Transp. Res. Part C, Emerg. Technol., vol. 19, no. 6, pp. 1133–1156, 2011, doi: 10.1016/j.trc.2011.01.002.
[151] M. Barth, F. An, J. Norbeck, and M. Ross, “Modal emissions modeling: A physical approach,” Transp. Res. Rec., vol. 1520, no. 1, pp. 81–88, 1996, doi: 10.1177/0361198196152000110.
[152] H. Rakha, K. Ahn, and A. Trani, “Development of VT-Micro model for estimating hot stabilized light duty vehicle and truck emissions,” Transp. Res. Part D, Transp. Environ., vol. 9, no. 1, pp. 49–74, 2004, doi: 10.1016/S1361-9209(03)00054-3.
[153] Z. Wang, G. Wu, and G. Scora, “Movestar: An open-source vehicle fuel and emission model based on usepa moves,” arXiv preprint, 2020, doi: 10.48550/arXiv.2008.04986.
[154] A. Barkley, The economics of food and agricultural markets. New Prairie Press, 2019.
[155] M. Quincampoix, Differential Games. New York, NY: Springer New York, 2012, pp. 854–861, doi: 10.1007/978-1-4614-1800-9_55.
[156] M. E. Mkiramweni, C. Yang, J. Li, and W. Zhang, “A survey of game theory in unmanned aerial vehicles communications,” IEEE Commun. Surveys Tuts., vol. 21, no. 4, pp. 3386–3416, 2019, doi: 10.1109/COMST.2019.2919613.
[157] T. S. Driessen, Cooperative games, solutions and applications. Springer Science & Business Media, 2013, vol. 3.
[158] J. Wilson, “Cooperative games with transferable utility,” in Wiley Encyclopedia of Operations Research and Management Science. John Wiley & Sons, Ltd, 2011, doi: 10.1002/9780470400531.eorms0199.
[159] M. Maschler, S. Zamir, and E. Solan, Game theory. Cambridge University Press, 2020.
[160] J. F. Nash Jr, “The bargaining problem,” Econometrica: J. Econometric Soc., pp. 155–162, 1950, doi: 10.2307/1907266.
[161] P. Milgrom and P. R. Milgrom, Putting auction theory to work. Cambridge University Press, 2004.
[162] J. Nash Jr, “Non-cooperative games,” in Essays on Game Theory. Edward Elgar Publishing, 1996, pp. 22–33.
[163] C. Luo, X. Zhou, and B. Lev, “Core, shapley value, nucleolus and nash bargaining solution: A survey of recent developments and applications in operations management,” Omega, p. 102638, 2022, doi: 10.1016/j.omega.2022.102638.
[164] M. Bowling and M. Veloso, “Multiagent learning using a variable learning rate,” Artif. Intell., vol. 136, no. 2, pp. 215–250, 2002, doi: 10.1016/S0004-3702(02)00121-2.
[165] P. G. Balaji and D. Srinivasan, “An introduction to multi-agent systems,” in Innovations in multi-agent systems and applications-1. Springer, 2010, pp. 1–27.
[166] A. Nowé, P. Vrancx, and Y.-M. D. Hauwere, “Game theory and multi-agent reinforcement learning,” in Reinforcement Learning. Springer, 2012, pp. 441–470.
[167] S. El-Tantawy, B. Abdulhai, and H. Abdelgawad, “Multiagent reinforcement learning for integrated network of adaptive traffic signal controllers (marlin-atsc): methodology and large-scale application on downtown toronto,” IEEE Trans. Intell. Transp. Syst., vol. 14, no. 3, pp. 1140–1150, 2013, doi: 10.1109/TITS.2013.2255286.
[168] Y. Yang and J. Wang, “An overview of multi-agent reinforcement learning from game theoretical perspective,” arXiv preprint, 2020, doi: 10.48550/arXiv.2011.00583.
[169] L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent reinforcement learning,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 38, no. 2, pp. 156–172, 2008, doi: 10.1109/TSMCC.2007.913919.
[170] O. Vinyals, T. Ewalds, S. Bartunov, P. Georgiev, A. S. Vezhnevets, M. Yeo, A. Makhzani, H. Küttler, J. Agapiou, J. Schrittwieser et al., “Starcraft ii: A new challenge for reinforcement learning,” arXiv preprint, 2017, doi: 10.48550/arXiv.1708.04782.
[171] J. Terry, B. Black, N. Grammel, M. Jayakumar, A. Hari, R. Sullivan, L. Santos, R. Perez, C. Horsch, C. Dieffendahl et al., “Pettingzoo: Gym for multi-agent reinforcement learning,” arXiv e-prints, 2020, doi: 10.48550/arXiv.2009.14471.
[172] K. Zhang, Z. Yang, and T. Başar, “Multi-agent reinforcement learning: A selective overview of theories and algorithms,” in Handbook of Reinforcement Learning and Control. Springer, 2021, pp. 321–384.
[173] A. L. Bazzan, “Opportunities for multiagent systems and multiagent reinforcement learning in traffic control,” Auton. Agents Multi-Agent Syst., vol. 18, no. 3, pp. 342–375, 2009, doi: 10.1007/s10458-008-9062-9.
[174] L. S. Shapley, “Stochastic games,” Proc. Nat. Acad. Sci., vol. 39, no. 10, pp. 1095–1100, 1953, doi: 10.1073/pnas.39.10.1095.
[175] K. Dresner and P. Stone, “Multiagent traffic management: Opportunities for multiagent learning,” in Proc. Learn. Adaption Multiagent Syst. Springer, 2006, pp. 129–138, doi: 10.1007/11691839.
[176] P. Koonce and L. Rodegerdts, “Traffic signal timing manual.” United States. Federal Highway Administration, Tech. Rep., 2008.
[177] P. Varaiya, “The max-pressure controller for arbitrary networks of signalized intersections,” in Advances in Dynamic Network Modeling in Complex Transportation Systems. Springer, 2013, pp. 27–66.
[178] R. P. Roess, E. S. Prassas, and W. R. McShane, Traffic engineering. Prentice Hall, 2011.
[179] Y. Wu, H. Chen, and F. Zhu, “Dcl-aim: Decentralized coordination learning of autonomous intersection management for connected and automated vehicles,” Transp. Res. Part C, Emerg. Technol., vol. 103, pp. 246–260, 2019, doi: 10.1016/j.trc.2019.04.012.
[180] D. I. Robertson and R. D. Bretherton, “Optimizing networks of traffic signals in real time-the scoot method,” IEEE Trans. Veh. Technol., vol. 40, no. 1, pp. 11–15, 1991, doi: 10.1109/25.69966.
[181] S.-B. Cools, C. Gershenson, and B. D’Hooghe, Self-Organizing Traffic Lights: A Realistic Simulation. London: Springer London, 2013, pp. 45–55, doi: 10.1007/978-1-4471-5113-5_3.
[182] H. Wei, C. Chen, G. Zheng, K. Wu, V. Gayah, K. Xu, and Z. Li, “Presslight: Learning max pressure control to coordinate traffic signals in arterial network,” in Proc. 25th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2019, pp. 1290–1298, doi: 10.1145/3292500.3330949.
[183] H. Wei, N. Xu, H. Zhang, G. Zheng, X. Zang, C. Chen, W. Zhang, Y. Zhu, K. Xu, and Z. Li, “Colight: Learning network-level cooperation for traffic signal control,” in Proc. 28th ACM Int. Conf. Inf. Knowl. Manage., 2019, pp. 1913–1922, doi: 10.1145/3357384.3357902.
[184] T. Chu, J. Wang, L. Codecà, and Z. Li, “Multi-agent deep reinforcement learning for large-scale traffic signal control,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 3, pp. 1086–1095, 2019, doi: 10.1109/TITS.2019.2901791.
[185] X. Zang, H. Yao, G. Zheng, N. Xu, K. Xu, and Z. Li, “Metalight: Value-based meta-reinforcement learning for traffic signal control,” in Proc. AAAI Conf. Artif. Intell., vol. 34, no. 01, 2020, pp. 1153–1160, doi: 10.1609/aaai.v34i01.5467.
[186] F.-X. Devailly, D. Larocque, and L. Charlin, “Ig-rl: Inductive graph reinforcement learning for massive-scale traffic signal control,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 7, pp. 7496–7507, 2021, doi: 10.1109/TITS.2021.3070835.
[187] C. Chen, H. Wei, N. Xu, G. Zheng, M. Yang, Y. Xiong, K. Xu, and Z. Li, “Toward a thousand lights: Decentralized deep reinforcement learning for large-scale traffic signal control,” in Proc. AAAI Conf. Artif. Intell., vol. 34, no. 04, 2020, pp. 3414–3421, doi: 10.1609/aaai.v34i04.5744.
[188] A. Oroojlooy, M. Nazari, D. Hajinezhad, and J. Silva, “Attendlight: Universal attention-based reinforcement learning model for traffic signal control,” in Proc. Adv. Neural Inf. Process. Syst., H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 4079–4090. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2020/file/29e48b79ae6fc68e9b6480b677453586-Paper.pdf
[189] P. A. Lopez, M. Behrisch, L. Bieker-Walz, J. Erdmann, Y.-P. Flötteröd, R. Hilbrich, L. Lücken, J. Rummel, P. Wagner, and E. Wießner, “Microscopic traffic simulation using sumo,” in Proc. 21st Int. Conf. Intell. Transp. Syst. (ITSC), Maui, HI, USA, Nov. 2018, pp. 2575–2582, doi: 10.1109/ITSC.2018.8569938.
[190] G. D. Cameron and G. I. Duncan, “Paramics—parallel microscopic simulation of road traffic,” J. Supercomput., vol. 10, pp. 25–53, 1996, doi: 10.1007/BF00128098.
[191] J. Casas, J. L. Ferrer, D. Garcia, J. Perarnau, and A. Torday, Traffic Simulation with Aimsun. New York, NY: Springer New York, 2010, pp. 173–232, doi: 10.1007/978-1-4419-6142-6_5.
[192] M. Fellendorf and P. Vortisch, Microscopic Traffic Flow Simulator VISSIM. New York, NY: Springer New York, 2010, pp. 63–93, doi: 10.1007/978-1-4419-6142-6_2.
[193] H. Zhang, S. Feng, C. Liu, Y. Ding, Y. Zhu, Z. Zhou, W. Zhang, Y. Yu, H. Jin, and Z. Li, “Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario,” in Proc. World Wide Web Conf., 2019, pp. 3620–3624, doi: 10.1145/3308558.3314139.
[194] C. Wu, A. R. Kreidieh, K. Parvate, E. Vinitsky, and A. M. Bayen, “Flow: A modular learning framework for mixed autonomy traffic,” IEEE Trans. Robot., vol. 38, no. 2, pp. 1270–1286, 2021, doi: 10.1109/TRO.2021.3087314.
[195] D. Garg, M. Chli, and G. Vogiatzis, “Traffic3d: A new traffic simulation paradigm,” in Proc. Int. Jt. Conf. Auton. Agents Multiagent Syst., 2019, pp. 2354–2356, doi: 10.5555/3306127.3332110.
[196] M. Zhou, J. Luo, J. Villella, Y. Yang, D. Rusu, J. Miao, W. Zhang, M. Alban, I. Fadakar, Z. Chen et al., “Smarts: Scalable multi-agent reinforcement learning training school for autonomous driving,” arXiv preprint, 2020, doi: 10.48550/arXiv.2010.09776 .
[197] E. Leurent, “An environment for autonomous driving decision-making,” https://github.com/eleurent/highway-env, 2018.
[198] M. Treiber and A. Kesting, “An open-source microscopic traffic simulator,” IEEE Intell. Transp. Syst. Mag., vol. 2, no. 3, pp. 6–13, 2010, doi: 10.1109/MITS.2010.939208.
[199] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “Carla: An open urban driving simulator,” in Proc. Conf. Robot Learn. PMLR, 2017, pp. 1–16. [Online]. Available: https://proceedings.mlr.press/v78/dosovitskiy17a.html
[200] M. Bowling and M. Veloso, “An analysis of stochastic game theory for multiagent reinforcement learning,” Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep. No. CMU-CS00-165, 2000.
[201] F. J. Martinez, C.-K. Toh, J.-C. Cano, C. T. Calafate, and P. Manzoni, “Emergency services in future intelligent transportation systems based on vehicular communication networks,” IEEE Intell. Transp. Syst. Mag., vol. 2, no. 2, pp. 6–20, 2010, doi: 10.1109/MITS.2010.938166.

VI Biography Section