Newell’s theory based feature transformations for spatio-temporal traffic prediction

Traffic state evolution on a roadway using the basic principle of LWR continuum model requires identification of ‘shockwaves’ and their interactions on the time-space diagram. A simplified approach - Newell’s solution (Newell 1993) - can be used to estimate traffic dynamics at a particular location in terms of cumulative vehicle counts, i.e., the cumulative number of vehicles observed at that location. Using vehicle counts at a detector location, this method aims to estimate cumulative vehicle counts at some other location along the homogeneous freeway assumed to exhibit a triangular fundamental diagram. Cumulative vehicle counts at detector stations can be appropriately translated using fundamental traffic parameters (defined in Table 1) to give an estimate of cumulative counts at the target site consistent with traffic flow theory. The relevant components of Newell’s solution are summarized below. The basic idea is to determine the change in vehicle counts along characteristics of free-flow from upstream and congested flow from downstream. Therefore, this approach assumes upstream is in free-flow state, whereas the downstream is in congestion. Newell’s simplified theory states that the real state at the target site would be determined as the minimum of what would be expected along the free-flow characteristic and the congested characteristic.

In this study, we employ a combination of convolutional neural network (CNN) (Krizhevsky, Sutskever, and Hinton 2012) and long short-term memory (LSTM) (Hochreiter and Schmidhuber 1997), referred to as CNN-LSTM, as the baseline model for jointly capturing the spatio-temporal aspects of traffic states.

CNN and LSTM extract information from the input data from two different perspectives - learning the time-invariant spatial characteristics in CNN; and short- and long-term temporal patterns in LSTM. Specifically, the convolutional layer in the model helps learn an internal representation of a two (or higher)-dimensional input through a feature learning process using kernel functions or filters. Notably, the convolution allows the model to learn features that are invariant across the time dimension. These features are then passed through an activation function to introduce non-linearities into the mapping function. Other layers, such as pooling layer are also used to reduce the number of parameters, while dropout layer prevents the model from overfitting. As a result, it generates more accurate feature representations, allowing the LSTM layers to learn temporal patterns with greater accuracy. LSTM uses feedback mechanism and several gating mechanisms where output from a previous time step is fed as input to the current step such that selective information from the past can propagate into the future states – allowing them to persist, which makes it suitable for capturing the temporal evolution of traffic states.

A schematic of the model architecture considered in the study is shown in Figure 1. The architecture comprises a convolutional layer that processes the spatio-temporal input to extract relevant features using filters. The input to the model is structured as an $N\times n$ array that is constructed using data from $N$ fixed-location stations, with each row representing flows from a particular station over a period of time, $n$. Notably, the pooling layers were not utilized since the spatial dimension of traffic data is limited. After the convolutional layer, a flattening layer is employed to reshape the outputs appropriately. The resulting feature outputs from the convolutional operation are then fed into an LSTM, which generates outputs through a sequence of densely connected layers.

DL models learn the spatial and temporal characteristics specific to a detector configuration for which it has been trained using the traffic flows from neighboring locations. In other words, it is trained to map temporal patterns from the given spatial configuration to its target location. However, this mapping is specific to the exact location of detectors from which data is used (e.g., the distance between input and target detectors). Therefore, these models are difficult to adapt to varied configurations and hence cannot perform suitably, i.e. they are not transferable. To improve the transferability of the models, we aim to utilize knowledge from traffic flow theory and propose a physics-based feature transformation to the model inputs. Notably, we explicitly assume that the data available at the transfer location is limited and hence, cannot be used to train a new model.

In our proposed approach, instead of using the flow data from the input detectors, the inputs are modified using Newell’s transformations. The specific modification depends on whether the target location is upstream or downstream of the location from which input data is utilized. Hence, it is expected to learn generalized features independent of the distance between the detectors and hence be suitable for transfer learning.

The physics-based modified version in this study includes three approaches: ‘Physics FC’, ‘Physics FF’ and ‘Hybrid’. In all three approaches, it is assumed that free-flow conditions prevail at an upstream station (e.g., location $A$) and the cumulative counts for the target or transfer station ($X$) are estimated using Equation 2 based on the cumulative counts at station $A$. The three approaches differ in how they treat input from a downstream station $B$. The ‘Physics FC’ approach assumes that congestion prevails at a station downstream of the target or transfer stations, and hence uses Equation 5 on the cumulative counts at station $B$ to determine the cumulative counts for the target or transfer station, $X$. To the contrary, the ‘Physics FF’ approach assumes that free-flow prevails at the downstream station, and hence uses Equation 3 on the cumulative counts at station $B$ to determine the cumulative counts for the target or transfer station, $X$. The ‘Hybrid’ approach, on the other hand, does not make explicit assumptions regarding the prevailing conditions (free-flow or congestion) at the station downstream of the target or transfer locations. Instead, it considers two inputs for the target or transfer station, $X$ – one calculated using Equation 3 based on the cumulative counts at station $B$, and the other calculated using Equation 5 based on the cumulative counts at station $B$. This approach utilizes both pieces of information to estimate the cumulative counts for the target or transfer station, thereby combining aspects of both free-flow and congestion conditions. Once the cumulative counts are computed for each case, the corresponding flows are then calculated by taking the differences between consecutive observations of the cumulative counts.

It is important to note that accurate estimation of fundamental traffic parameters such as free-flow speed ($v_{f}$), congestion wave speed ($w$), and jam density ($k_{j}$) is crucial for the proper functioning of these transformations. The estimation of these parameters will be discussed in the following section.

In our methodology, we demonstrate the estimation of traffic parameters using data from multiple loop detectors. For a stretch of freeway, we combine data from multiple detectors to obtain mean estimates of these parameters. This aggregation of detector data assumes homogeneity within the freeway section, implying that the traffic parameters do not significantly vary within the section. This assumption aligns with the assumptions made in Newell’s solutions. However, it is important to note that for a larger number of detectors with wider spatial coverage, this assumption may not hold true.

The data provided includes measurements of flow ($q$), occupancy ($o$), and speed ($v$) recorded at 5-minute intervals from each station. Assuming a triangular fundamental diagram, we use the fundamental equation of traffic, $q=k\times v$, to calculate the corresponding densities ($k$). To determine the capacity ($q_{c}$) and free-flow speed ($v_{f}$), we consider the $95^{th}$ percentile values of flow and speed, respectively. This is done because the maximum flow and speed values do not indicate a stable state and persist for only a short period of time. The critical density ($k_{c}$) is then evaluated using the fundamental equation of traffic as $k_{c}=q_{c}/v_{f}$. These parameters provide essential insights into the traffic conditions and are fundamental for our subsequent analysis. Figure 2 illustrates the flow versus density relationship for four stations along the SR04 California highway.

The freeway operates in free-flow conditions for most of the day with congested states appearing only during peak hours. Therefore, congested states ($k>k_{c}$) are not easily observable in the flow-density curve, which do not allow us to directly estimate the full congested branch of the fundamental diagram, including congestion wave speed ($w$) and jam density ($k_{j}$). Hence, we assume a fixed value of $w=14$ for our analysis.

To evaluate and compare the performance of the models, different scenarios are considered based on the relative positions of the target and transfer stations. This relative positioning determines which Newell’s transformation is applied to the target and transfer locations. Table 2 presents these scenarios, labeled A1, A2, B1, B2, C1, C2, D1, and D2, where the model’s performance is assessed at both the target (where it has been trained) and the transfer locations without further retraining. It is assumed that up-to-date data is available at the two source locations, while no data is available at the target or transfer locations for prediction purposes. For each scenario, a regular model and three physics-modified models are considered.

Different scenarios are considered to better understand when the proposed models can improve predictions. For instance, let us consider a scenario where the model is trained using traffic information from an upstream and downstream station to predict flows at an intermediate target location (Scenario A1 and A2). In this case, the physics-modified feature inputs consist of Newell’s uncongested flow estimate (from the upstream station) and either an uncongested or congested flow estimate (from the downstream station). If the same model is applied to another intermediate location within the bounds of the upstream and downstream stations, the model will still receive similar flow estimates as inputs. However, in other scenarios (e.g., D1 or D2), the models are trained to predict flow at an upstream (or downstream) location relative to both stations but are transferred to predict flow at a downstream (or upstream) location. Since the input features in the target and transfer domains differ significantly, it is important to investigate the model’s performance under these circumstances as well.

It is worth noting that Case B always utilizes free-flow features both in the target and transfer locations, since both stations are upstream of target and transfer. Therefore, we have performance recorded only for the ‘Physics FF’ method in this case. Additionally, a hybrid model could not be employed for Case D since the dimensions of the input features in the target and transfer locations would be different, making it challenging to maintain consistency in the model architecture.

We evaluate the model performances for both single- and multi-step prediction horizons, by comparing the prediction mean with the corresponding true flow values using three metrics: root mean squared error (RMSE), mean absolute percentage error (MAPE) and $R^{2}$ as defined below.

where $y_{i}$ represents the true value of the observation $i$, $\hat{y_{i}}$ is the predicted value of $y_{i}$ for $i=1,2,\dots T$. Both RMSE and mean absolute error (MAE) measures the error of the predictions, however, RMSE is often preferred due to its higher penalization to outliers compared to MAE, where all errors are weighed equally. On the other hand, $\mathrm{R^{2}}$ measures the performance of the model in terms of proportion of the variance in data that could be explained by the regression model.

In this study, two different model architectures are used due to variations in the volume of the two datasets. For Dataset 1, the model architecture comprises a 2-dimensional convolutional layer with 12 filters and a kernel size of (3, 2), activated by the rectified linear unit (ReLU) activation function. The outputs from the convolutional layer are then flattened and reshaped appropriately to be fed into two LSTM layers with 10 and 6 units respectively. Finally, a single-unit dense layer is utilized to predict flows. On the other hand, for Dataset 2, the model architecture consists of a 2-dimensional convolutional layer with 16 filters and a kernel size of (3, 2), activated by the rectified linear unit (ReLU) activation function. This is followed by two LSTM layers with 10 and 6 units, respectively. The outputs from the LSTM layers are then passed through a dense layer with 6 units, activated by the ReLU activation function. Finally, the prediction layer is added to generate the traffic flow predictions for the specified future time point.

To ensure generalizability and prevent overfitting, the models are trained, validated, and tested using three distinct sets. The dataset is partitioned into three parts: 60% for model training, 15% for validation, and 25% for testing. The model parameters are tuned throughout the training process based on their performance on the validation set. The mean squared error (MSE) loss function is minimized during the training process, and the model with the lowest validation error is selected as the final model. For the models trained on Dataset 1, a batch size of 10 and a temporal lag of 10 are used. In contrast, the models trained on Dataset 2 uses a batch size of 20 and a temporal lag of 20. The Adadelta (Zeiler 2012) optimizer is employed with a learning rate of 0.10, a rho value of 0.95, and an epsilon value of 1e-7 to train the models.

Prediction performances of the models trained on Dataset 1 are provided in Tables 3, 4 and 5 corresponding to different performance metrics – RMSE, $R^{2}$ and MAPE.

The performance results for Cases A, C, and D demonstrate that Newell’s assumption of congested states prevailing downstream does not hold true, or the interference of ramp flows are too large for Newell’s conservation assumptions to hold. As a result, the ‘Physics FC’ method, which relies on this assumption, does not outperform the regular model either at the target or transfer locations in the cases where the source station is downstream of either the target or transfer location, or both. In contrast, we find that the assumption of free-flow states performs better in the majority of these scenarios.

The hybrid approach demonstrates superior performance in both Cases A and C compared to other approaches. In Cases A1 and A2, the feature inputs consist of three dimensions: one from the upstream station and two from the downstream stations. This configuration allows the model to leverage the free-flow characteristics from both upstream and downstream stations, as well as the congested information from downstream, thereby enhancing its prediction capabilities. Similarly, in Cases C1 and C2, the input feature size is expanded to four dimensions, with two dimensions from both the upstream and downstream stations. This enables the model to incorporate the combined information from both directions, taking into account the variations in traffic conditions along the freeway section. By considering the inputs from both upstream and downstream stations, the model can better capture the complex interactions and correlations between different segments of the freeway, leading to improved prediction accuracy.

Here, we present the modeling results on Dataset 2 in Table 6, which displays the RMSE values. Other evaluation measures exhibit similar trends and are not included in the table for brevity. In this larger dataset, we also evaluate the accuracy of the model predictions separately during free flow and congestion to better understand the benefits of the proposed approaches. To distinguish between free-flow and congestion states, we divide the daily traffic into two categories based on a speed threshold: if the speed drops below 50 mph it is assumed that that location is in congestion (as shown in blue dashed lines), See Figure 6.

The results for Cases A1 and A2, where both the target and transfer locations are in between the source detector stations, again indicate that the use of Newell’s modification for congestion conditions does not improve predictions, likely due to lack of congestion or interference of traffic dynamics due to ramp flows. This finding aligns with the observations from Dataset 1. Interestingly, assuming free-flow conditions leads to only minor improvements in some cases. However, when we employ the hybrid approach that combines information from both upstream (free-flow) and downstream (congested and uncongested) stations, we observe significant improvements in model performance. This hybrid approach outperforms both the regular model and other versions of the physics-based approaches. It is important to note that during the model training, the optimization is still based on the combined mean squared error (MSE) loss, meaning that the model does not individually optimize for losses during free-flow and congestion. The superior performance of the hybrid approach highlights the effectiveness of incorporating both free-flow and congestion information to enhance the model’s predictive capabilities.

In Case C1, we observe that Newell’s assumption of congestion prevailing downstream (Physics FC) works well at the target location (S2), where only congestion characteristics from downstream locations are used during model training. However, when this assumption is transferred to S1, it does not perform well. In contrast, the assumption of free-flow conditions (Physics FF) shows poor performance at the target location but demonstrates better performance at the transfer location in Case C1. Surprisingly, the hybrid approach also shows poor overall performance in both the target and transfer locations. But it still provides better performance during congestion at the target location.

The discrepancy in performance can be attributed to location-specific differences, such as variations in traffic patterns, congestion levels, or spatial configurations of the freeway section. In this case, the target station for Case C1 (S2) is located 0.6 mi and 1.5 mi away from the source stations, while the transfer location for Case C1 (S1) is located 2.2 mi and 3.1 mi away. These differences in spatial distances can significantly impact the traffic dynamics and patterns observed at each location. To further support this observation, we examine the flow vs. speed trends for both S2 (target in Case C1) and S1 (transfer in Case C1) as depicted in Figure 7. When comparing these trends, we find that S1 operates predominantly in free-flow conditions with only short-lived congested states. This aligns with the better transferability of the FF model, which leverages the free-flow characteristics, as opposed to the hybrid model that combines both free-flow and congested information.

In Case C2, we observe a different pattern compared to Case C1. The hybrid approach performs better than both the regular model and the ‘Physics FF’ approach in both the target and transfer locations. The ‘Physics FF’ model also demonstrates good performance, consistent with the assumption that the target station mostly operates in free-flow conditions. However, the hybrid approach excels by accurately accounting for the dependencies between the source and target station.

In Cases B1 and B2, we observe that while the regular model performs better at the target location, its transferability is compromised, indicating that it does not generalize well to different locations. On the other hand, the physics-based FF model demonstrates better transferability in these cases. This suggests that the ‘Physics FF’ model captures the underlying dynamics of traffic more effectively, allowing it to perform well even at locations it has not been explicitly trained on.

In Cases D1 and D2, we consistently observe that the ‘Physics FF’ model outperforms the ‘Physics FC’ model, indicating that free-flow characteristics play a significant role in these scenarios. However, it is important to note that there are some instances where the regular model shows minor advantages.

These inconsistencies in performance could be attributed to several factors. Firstly, the spatial distances between stations in this dataset are larger, which may violate the assumption of homogeneous traffic parameters over the entire section. Secondly, the presence of major interchanges and ramps between the mainline detectors can significantly impact traffic patterns. When ramps experience high volumes of vehicles trying to enter or exit the highway, they can disrupt the smooth flow of traffic and lead to localized congestion or bottlenecks on the mainline. For example, congestion on exit ramps can lead to queue spillbacks, causing disruptions in traffic flow on the mainline. These disruptions can result in variations in traffic conditions that may not be accurately captured by the models.

Overall, while the ‘Physics FF’ and hybrid models (where applicable) generally outperform the regular model and ‘Physics FC’ in terms of transferability, the inconsistencies in trends compared to Dataset 1 highlight the influence of spatial distances, non-homogeneous traffic parameters, and the presence of interchanges and ramps on the accuracy of the models. These factors should be carefully considered when applying and interpreting the modeling results in such complex freeway environments.

Performing a sensitivity analysis along the time dimension is crucial to assess the robustness of the model when predicting traffic flows further into the future. In this case, both the regular model and the physics-based hybrid model were trained on Dataset 1 for Case A. The models were evaluated to predict 25 minutes into the future using a temporal history of 50 minutes, which consists of 10 time steps. The analysis reveals that the prediction errors for both models increase as the prediction horizon becomes longer. This is expected, as longer prediction horizons introduce more uncertainties and make it more challenging to accurately forecast traffic flows. However, it is observed that the physics-based hybrid model consistently outperforms the regular model at all prediction horizons. The performance gap between the two models tends to widen as the prediction horizon increases, indicating the superiority of the physics-based approach in capturing the dynamics of traffic and improving the accuracy of predictions. Furthermore, the physics-based hybrid model demonstrates relatively higher benefits at the transfer location compared to the target location. This suggests that incorporating physics-based principles and combining information from both upstream and downstream stations allows for a more comprehensive representation of the traffic conditions and enhances the model’s predictive capabilities.