TrafficGPT: Viewing, Processing and Interacting with Traffic Foundation Models

With the growing recognition of Large Language Models (LLMs) in diverse domains, And vertical applications that more accurately meet industry-specific needs have been birthed in areas such as education[2, 3], finance[4, 5], and vision[6, 7]. Scholars in the field of traffic management and control also have made noteworthy contributions, exploring novel approaches and applications. To test the LLM’s knowledge on traffic control, Villarreal et al.[8] investigates the use of ChatGPT, a large language model, to assist novices in solving complex mixed traffic control problems in Intelligent Transportation Systems (ITS). The results of the study show that using ChatGPT can increase the number of successful policies in mixed traffic control tasks. Further, Da et al.[9] proposes a PromptGAT framework for traffic signal control, integrating prompt-based dynamics modeling using language models to enhance the learning of forward and inverse models. The framework leverages domain knowledge and real-time traffic states to predict system dynamics and optimize the policy through reinforcement learning. As for information processing, Zheng et al.[10] explores the potential applications of large language models (LLMs), specifically ChatGPT, in intelligent transportation systems. By leveraging LLMs with cross-modal encoders, they present a smartphone-based crash report auto-generation and analysis framework based on news text, images and audio inputs. The paper discusses the benefits and challenges of using LLMs in transportation, including data privacy, data quality, and model bias. Overall, LLMs hold promise for more efficient, intelligent, and sustainable transportation systems.

Nonetheless, to the author’s best understanding, the incorporation of Large Language Models (LLMs) in complex traffic-related tasks, encompassing analysis, processing, and interaction with expansive traffic-related datasets, remains sparsely explored within the current research milieu. This situation is likely attributed to the inherent challenges associated with the reliability of Large Language Models (LLMs) when handling numerical digits within numeric data[11, 12]. This dearth of integration underscores a significant gap in the application of LLMs in the field of tackling complex traffic-related issues and providing supported decision making.

Complex traffic-related tasks inevitably involve sub-tasks that requires LLM to deal with numerical traffic datasets. The processing, analysis, and visualization of traffic data are essential to support decision-making among traffic managers. Many widely recognized and used Traffic Foundation Models(TFMs) for traffic-related tasks have emerged from research and practice in the field of transportation. Traffic system analysis models can be categorized into network topology analysis[13, 14], network volume analysis[15, 16], and network performance analysis[17, 18, 19]. Apart from pure data analysis using sophisticated models, Chen et al.[20] emphasizes the importance of data visualization in understanding the behavior of traffic participants and discovering traffic patterns. Existing data visualization models could present location-based, activity-based, and device-based traffic data [21, 22] with a variety of focuses including time[23], spatial[24, 25, 26] and other properties needed [27]. Furthermore, building upon the insights gleaned from traffic data analysis, extensive research and validation efforts have been dedicated to the development of traffic management and control measures. These encompass optimizations in signal schemes[28, 29], route guidance systems[30], and more.

Although these Traffic Foundation Models (TFMs) are often confined to specific tasks with single-round inputs and outputs, the abundance of well-established TFMs, coupled with the prospect of amalgamating multiple TFMs, forms a sturdy basis for deploying Large Language Models (LLMs) to tackle complex traffic-related issues and support decision-making.

To deal with complex tasks, it is necessary to uncover the reasoning and planning capabilities of LLMs. The concept of Chain-of-Thought (CoT) [31] is introduced to harness the multi-step reasoning capabilities inherent in Large Language Models (LLMs). Further, the ReAct [32] logic investigate the application of LLMs in generating both reasoning traces and task-specific actions in an intertwined fashion, fostering enhanced synergy between the two components. Reasoning traces serve the purpose of enabling the model to deduce, monitor, and revise action plans, as well as manage exceptions, whereas actions empower it to interface with external resources. However, the application of CoT and ReAct mainly focus on decision making tasks based on text information, i.e., HotpotQA Example [32]. Our work extends the potentiality of LLM to complex traffic-related tasks regarding expansive numerical datasets, including but not limited to data processing, data visualization, traffic management and control.

TrafficGPT constitutes a system that synergizes various Traffic Foundation Models (TFMs) to facilitate the comprehension of traffic-related tasks and the subsequent generation of relevant responses based on multi-modal data sources. This integration necessitates the customization of specific system principles, subsequently transformed into prompts comprehensible to Large Language Models (LLMs). These prompts play a multifaceted role, encompassing:

• •

  Role of TrafficGPT: In general, TrafficGPT is designed as an AI to assist human with traffic-related tasks. To further refine the distinction, TrafficGPT can be used for traffic big data analysis and display, supported decision-making of traffic management, and even traffic simulation modeling and control.

• •

  TFMs Accessibility: TrafficGPT is equipped with an array of Traffic Foundation Models (TFMs) to address diverse traffic-related tasks. The selection of the particular foundation model to employ is autonomously determined by the Large Language Model (LLM), thereby facilitating seamless integration of new TFMs and the extension of support to novel traffic-related tasks.

• •

  Task Deconstruction and Planning: Depicted in “Fig. 2”, addressing what may initially appear as a straightforward command often entails the involvement of multiple Traffic Foundation Models (TFMs). For instance, consider the query ”Show me the current network heatmap,” which necessitates the utilization of both the ”Get Current Time” and ”Plot Heatmap” TFMs. To effectively handle more intricate queries by deconstructing them into subproblems, the introduction of the Thought-Action-Observation mechanism within TrafficGPT becomes essential. This mechanism aids in the determination, utilization, and distribution of multiple TFMs to successfully address the challenges posed by diverse queries.

TrafficGPT has the capacity to incorporate a diverse array of Traffic Foundation Models (TFMs), which encompass an expansive spectrum of functions, encompassing but not limited to database access, traffic flow counting, trajectory extraction, traffic performance evaluation, traffic resilience quantification, data visualization, and traffic signal optimization. These functions are tailored to address the practical requirements of traffic-related tasks.

However, it is noteworthy that distinct TFMs may share similar functionalities. For instance, for signal optimization tasks, TrafficGPT has TFMs based on various methods such as the Webster algorithm, linear programming methods, reinforcement learning techniques, and more. These tools may cater to varied scenarios and possess disparate input requisites.

Consequently, the assurance of the Large Language Model’s (LLM) capability to distinctly discern the TFMs and proficiently apply them becomes a crucial factor in facilitating the proficient execution of traffic-related tasks. To attain this objective, we craft prompts and input them into the prompt management system for each TFM. This is accomplished by outlining the subsequent attributes:

• •

  Name: The name usually gives an overview of the functionality implemented by each TFM. Its core function is to provide the LLM with an entry point to the TFM. The target TFM can only be called and run if the LLM provides an identical tool name.

• •

  Usage: The usage mainly introduces the actual application scenarios and the methodologies employed of each TFM, so that LLM can distinguish between similar TFMs.

• •

  Input: The definition of the TFM input content is key to ensuring that TrafficGPT is able to successfully invoke the tool. The input typically describes the content and format of the input parameters required to invoke each TFM. Often, textual descriptions are followed by examples to ensure that the LLM is accurately understood. For instance, the TFM used for plotting heatmap, the Input part of the TFM’s Prompt is as follows: The input must be the target point in time in the ”YYYY-MM-DD HH:MM:SS” format. For example: 2019-08-13 09:00:00.

• •

  Output: The output describes the content and format of the output of each TFM.

• •

  Priority(Optional): In order to improve the robustness of user demand response, for similar TFMs, in addition to providing information to help differentiate between them, another possible approach is to set a higher priority for certain TFMs for similar tasks. For example, when a user proposes a signal scheme optimization task but does not explicitly indicate a specific algorithm, the signal optimization TFM based on Webster’s algorithm is preferred.

Maintaining the reliability of TrafficGPT is of paramount importance, given its role in handling complex traffic-related tasks, especially for traffic data analysis. To ensure a dependable output, a meticulous strategy has been devised, incorporating specific prompts and principles.

• •

  Data authenticity: Using prompt to instruct TrafficGPT to abstain from fabricating TFM names, inputs and output data. This minimizes the potential for misleading outputs and maintains the system’s credibility.

• •

  Minimized Redundancy: Using prompt to mandate the avoidance of repetitive tool usage, fostering coherent problem-solving while mitigating the risk of errors. This approach enhances system reliability and accuracy.

• •

  Human Intervention Protocol: Using prompt to underscore the importance of human intervention in cases where existing information and TFMs fail to complete a task. It instructs TrafficGPT to conclude its actions and request additional information from humans as the final answer, ensuring that the system’s output remains accurate and reliable.

• •

  Task Precision and Response Timeliness: Given that TrafficGPT harnesses a multitude of Traffic Foundation Models (TFMs) and can interlink them to attain intricate functionality, it becomes essential to furnish Prompts. These Prompts serve the purpose of maintaining the primary task at the forefront of TrafficGPT’s cognition, thus averting the introduction of extraneous requisites. This approach is vital to ensure both task precision and prompt response times.

• •

  Robust Response: Given the stringent requirements for input parameters in Traffic Foundation Models (TFMs) and the potential insufficiency of user-provided information, Prompts play a pivotal role in guiding TrafficGPT. They dictate when to employ default input parameters and when to activate the Human Intervention Protocol to ask the user for additional information, ensuring a seamless and robust interaction.

As an AI traffic assistant, TrafficGPT’s fundamental capability lies in its ability to engage in multi-round conversations and perform basic tasks based on the instructions provided by human users.

“Fig. 3” illustrates the basic capability of TrafficGPT to execute traffic-related tasks based on natural language comprehension. The left column is a 2-round dialogue example from Data Processing Bot, which is designed to handling extensive numerical traffic data. Rather than throwing massive amounts of numerical data directly to LLM in text or markdown format, TrafficGPT provides LLM with a set of TFMs that are designed to process and analyse expansive numerical data accurately and efficiently. Empowered by TFMs, TrafficGPT is able to perform a complex statistical analysis of OD traffic flow calculation and visualization according to city-wide trip data. The right column demonstrate TrafficGPT’s ability to interact with traffic simulation software. By providing the Simulation Control Bot with tools to access the simulation model, TrafficGPT is able to provide users with traffic status data and even write a new signal control scheme into the simulation configuration file.

In addition to task execution ability, the above example also demonstrates its contextual memory and logical inference ability through multiple rounds of dialogue. For instance, in the second user command illustrated in “Fig. 3” (a), the user employs the phrase ”the same time period” without specifying a particular time. In this context, TrafficGPT references the user’s initial command and deduces the correct time period accordingly.

In practical applications of TrafficGPT, users may not possess detailed knowledge of the available tools within the system or their specific input parameter requirements. Consequently, users are prone to making fuzzy requests or inquiries. Two according dialogue cases are shown in “Fig. 4”.

The left column is an example from Data Processing Bot. The user asked for a network heatmap but did not provide a specific time point. According to the Robust Response principle demonstrated in III-C, TrafficGPT is able to perform ambiguous tasks using default parameters and correct the initial response in the follow-up conversation as instructed by users, as illustrated in “Fig. 3”(a). Similarly, in the right column, the user instruct the Simulation Control Bot to optimize the signal scheme but did not provide any information of the target intersection. According to the Human Intervention Protocol demonstrated in III-C, TrafficGPT request specific information from users and then proceeded the task.

The showcased capabilities in the preceding examples highlight TrafficGPT’s adaptability to a wide array of questions and commands expressed in natural language by human users. This further underscores its potential to replace human involvement in traffic-related tasks.

However, in order to provide more comprehensive assistance to human users, TrafficGPT must showcase its ability to aid in human decision-making, conduct holistic analyses, offer insightful suggestions, and more. It is imperative for TrafficGPT to demonstrate a profound understanding of traffic-related issues, surpassing mere adherence to mechanical execution of human instructions.

“Fig. 5” illustrates a 6-round dialogue between TrafficGPT and a human user. The user poses a broad and open-ended problem about the simulation road network, and TrafficGPT needs to deconstruct the task and plan a reasonable flow of actions based on the general knowledge of traffic management and the tools at its disposal.

The left column shows the reasoning flow of their first round of dialogue. Just as a traffic expert, TrafficGPT tried to get the overview of the traffic status since it has no specific information of the traffic network performance at the beginning. Then, TrafficGPT identified three intersections with poor traffic conditions based on the performance data in combination with the general knowledge of traffic management, and attempted to optimize their signal control scheme. In order to provide the user with more information, TrafficGPT has also thoughtfully marked the exact locations of these three intersections on the map using a visualization tool. Finally, TraffiGPT believes that it already has enough information to answer the user’s question and therefore generates detailed optimization recommendations based on the thought process. This dialogue session exemplifies TrafficGPT’s exceptional logical reasoning and execution capabilities when confronted with ambiguous and intricate tasks.

In the following conversational rounds, the user requested optimization results from TrafficGPT. Upon analyzing the data, TrafficGPT identified that out of the initial three intersections recommended for optimization, only two had undergone optimization. Due to some ambiguity in the user’s and TrafficGPT’s assessments of the optimization results, TrafficGPT meticulously elucidated its reasoning to the user. Subsequently, TrafficGPT rectified its initial recommendations based on the simulation results and presented an implementable optimization plan.