STC-IDS: Spatial-Temporal Correlation Feature Analyzing based Intrusion Detection System for Intelligent Connected Vehicles

The specification-based IDS focuses on defining system specifications, such as protocols and frame formats. When packets mismatch the system specification, an exception alarm is raised. In 2016, Dagan et al. [27] introduced an anti-spoofing system that detects malicious messages using each ECU, i.e., by detecting CAN message ID that was not sent by the ECU itself. Thereafter, the ECU informs the IDS, and then an interrupt pulse is sent to the CAN bus to overwrite the spoofed message. However, each ECU undertakes the IDS role, which increases a certain burden on communication.

In 2018, Studnia et al. [26] presented a signature-based IDS which utilize a list of signature derived from CAN dataset. However, this method is subject to the limitation that the length of the CAN bus words may not be known a priori. Recently, Olufowobi et al. [20] proposed a real-time IDS based on specification. The algorithm extracted the timing model to detect anomalies through observing CAN traffic rather than depending on predefined specifications, yet exhibited relatively poor performance in the real attack dataset.

The fingerprint-based approaches are mainly based on profiles defined by ECU characteristics to implement anomaly detection. In 2016, Cho and Shin [28] proposed clock-based IDS to analyze ECUs periodic frequency. The method established the ECUs clock baseline through the recursive least squares algorithm (RLS) to detect intrusion. But it workable only to periodic messages excluding non-periodic messages.

Interestingly, Cho et al. [29] found a method to establish the electrical signal characteristics of each ECU using the physical layer data of CAN communication, and harness these signal characteristics as the fingerprint for each ECU. Regrettably, the electrical characteristics may change as the vehicle ages, and thus the IDS needs to keep updating.

Unlike previous methods, the statistics-based approach implements anomaly detection by means of statistical information obtained from CAN traffic at the network level. Song et al. [30] introduced a lightweight IDS in 2016 that detected anomalies by monitoring the abnormally shortened intervals between messages. Although the proposed algorithm could have highly sensitive to common injection attacks and low computing cost, it cannot detect irregular incoming messages.

Furthermore, Young et al. [31] comprehensively analyzed the frequency characteristics of CAN messages in various driving modes, such as reverse, acceleration and hold speed, and then proposed a frequency-based intrusion detection system. In spite of the high detection accuracy, there is a high false alarm rate. Manifestly, an IDS based on conditional statistical relationship analysis to learn the normal behavior of the system can detect manipulations and incorrect payload values [37], but still does not satisfy the high detection rate and low latency required by present-day IVNs for anomalous traffic [38].

Machine learning (ML) and Deep learning (DL) based intrusion detection systems are an excellent option for extracting and learning normal or abnormal behavior, which provide models with detect and predict ability [39, 40]. Kang et al. [32] constructed a deep confidence network under unsupervised learning to detect if anomalies deviate from normal. However, inefficient and only validation of simulation data is not sufficient.

In 2019, Pawelec et al. [35] proposed a 3-layer LSTM neural network to predict the data payload for each CAN ID, which avoided reverse engineering proprietary encoding. Similarly, Qin et al. [19] also implemented anomaly detection for CAN bus based on timing features by LSTM and re-considered two data formats of CAN frames. Although these methods are implemented at the CAN bits level, they only consider timing characteristics and have relatively poor detection performance. Recently, convolutional neural network (CNN) have been implemented for traffic detection and praised for their high detection efficiency [41].

Song et al. [36] presented a reduced inception residual network to construct an IDS capable of detecting spoofing and denial of service (DoS) attacks in a continuous pattern of vehicular traffic. Since the assistance of spatial-temporal relation is not taken into account, there is still room for improvement in the false positive rate. In other studies, Tariq et al. [34] introduced a convolutional LSTM-based intrusion detection method. Although it displayed excellent detection performance in unknown attack than transfer learning, known attacks performance relatively worse that may be caused by the relevance of the CAN message features being discarded. Moreover, a multi-tiered hybrid IDS that incorporates a signature-based IDS and an anomaly-based IDS is proposed to detect both known and unknown attacks on vehicular networks by Yang et al. [33] in 2021. Their model has been proven that is effective for attacks on both in-vehicle and external networks. However, modeling with spatial-temporal features might take performance a step further.

However, these ML&DL-based methods have different in selecting the detection domain, typically the detection arbitration domain, the detection data domain, and the similar to our work that the spatial-temporal feature extraction. In a nutshell, traditional methods based on specification, fingerprint, and statistical, have limitations in terms of reliance on anomaly feature libraries, message frequency, message time domain, and fingerprint information. Instead, it is imperative in the ML&DL area to improve automotive IDS detection performance and reduce false positives by complementing spatial-temporal features in a limited message communication mode. Furthermore, the model trained with limited spatial-temporal features will not effectively improve the detection performance.

Considering the nature of attention mechanisms to capture essential features, the generation of spatial-temporal attention features is one of the potential ways to address the above problem [42]. Hence, modeling the normal behavior of CAN packets in combination with spatial-temporal attention features and then discovering the difference between anomalies and target traffic is still one of the ways to improve in-vehicle IDS.

Intelligent Connected Vehicles (ICVs), integrating modern computing and communication technologies, is designed to improve user experience and driving safety. As the most significant communication medium, the CAN is the most prevalent bus topology network employed in contemporary vehicles owing to its low cost and complexity, high reliability, and fault-tolerance characteristics [43, 44]. All ECUs, connected to the CAN bus, are capable of exchanging messages in the form of data frames [45, 46]. Fig 1 presents the structure of a CAN message, consisting of seven fields [47]: 1) start of frame (1 bit); 2) arbitration field (12 bits for standard frames, 29 bits for extended frames); 3) control field (6 bits); 4) data field (Maximally 8 bytes); 5) cyclic redundancy code (CRC) field; 6) acknowledge (ACK) field; and 7) end of frame.

In the entire CAN frame, the most important is the arbitration field and the data field, as the arbitration field determines the priority of the message [48], shown in Fig 2(a); the data field contains the actual transmitted data that defines the node actions. Moreover, if an error is detected by CRC field, the receiving node will discard the received error message, while the sending node will only assume a transient fault on the bus and enter arbitration to resend the message frame [49, 50], shown in Fig 2(b). To guarantee the system consistency, the ECU broadcasts messages at regular intervals in spite of the data values have not changed. However, security problems were poor-needy thought out at the beginning of the CAN bus design [51], including its broadcast transmission strategy, lack of authentication and encryption, and unsecured priority scheme. Hence, many network traffic injection attacks are possible.

This directly motivates the adversary to attack in-vehicle networks in a variety of ways, as shown in Fig 3. Clearly, the adversary can not only through the OBD-II port for physical attacks but also implement a remote attack easily (e.g., Wi-Fi or Bluetooth) [10]. Types of such attacks include flooding the bus with messages designed to circumvent legitimate messages or using spoofed bus identifiers with invalid information [36]. Furthermore, there are more sophisticated and stealthy attacks [52]. These attacks appear to be legitimate traffic sequences that are tough to distinguish from normal messages [1].

Once the attacker has successfully compromised, it has the opportunity to forge ECU nodes and take control of incumbent nodes to inject nefarious messages [53, 54]. In the CAN protocol, the connected nodes are synchronized with the current vehicle state by accepting the data field bits of the frame [36]. Consequently, in order to successfully deceive the ECU, the adversary must insert tampering messages in a high frequency and priority manner by following the target CAN ID message immediately after the message [55]. If the attacker injects a high-priority CAN frame, where the data field is populated with a status command to turn off the “wiper”, the driver loses judgment and even serious traffic accidents in rainy conditions while driving at high speed [56].

In this paper, we utilized car-hacking dataset which is published by Song et al. [52]. This data set is injected with four attacks, respectively DoS attack, fuzzy attack, spoofing attack including RPM and GEAR, as illustrated in Fig 4. The detailed injection rules are as follows.

1. 1.

   DoS attack: DoS attacks in the dataset are to inject a high priority message with a ‘0x000’ CAN ID every 0.3 milliseconds, with the data field populated with 0.

2. 2.

   Fuzzy attack: Fuzzy attacks in dataset are injected every 0.5 milliseconds with CAN messages where the CAN ID and DATA values are forged randomly.

3. 3.

   Spoofing attack: Spoofing attacks in dataset are injected messages every 1 millisecond with a specific CAN ID, e.g., related to RPM/Gear.

Table II indicates the number of normal and injected messages in each attack dataset. In order to summarize the spatial-temporal details of normal CAN bus traffic during the operation of a real vehicle, we first investigated the communication patterns of different CAN IDs. Since the dataset is not publicly available as to the receiving sender and receiver of the data, the paper analyzes the CAN protocol table of a brand from our laboratory. We find that an ECU has a fixed set of CAN IDs (e.g. EMS, containing 0x101, 0x278, 0x281) and that the recipients of the different CAN IDs are also fixed (e.g. 0x101, containing TCU, ESP, EPB, T-BOX). This is the initial purpose to design a single-frame detection model capable of tracking unauthorized ECUs and protecting non-attacked ECUs.

Additionally, the frequency of the different IDs is fixed by the vehicle manufacturer. It is worth noting that important automotive components have a higher priority. Hence, time-series features are reflected in the ID of the CAN messages. Despite the fact that there are some event-triggered messages with variable frequency, the set of commands is also fixed. The details are shown in Table III.

Spatial feature expression, especially the data field of the CAN frame, is most significant. Most new messages are generated at a steady rate over the period of data acquisition in the dataset. In this study, the spatial signature analysis was based on the data segment, and a single message often had a regular change in timing. We analyzed this and aggregated the messages of different periodic variation rules. Table IV displays some CAN message of different CAN IDs from the dataset, where the omissions represent same transferred bytes. We can observe that they have certain fixed byte constants (e.g., the first 7 bytes of ID=0x260), as well as fields that change more frequently (e.g., the third bytes of ID=0x316), but only in a certain range. There are distinct spatial characteristics of the data field, useful for modeling the normal behavior of the CAN packet, and also inspire the requirement to design attention convolutional blocks.

To more visually observe the byte change patterns, the data fields for each message are displayed in a heat map, as shown in Fig 5. In order to apparently visualize the differences in variation of the each byte, this paper presents 100 consecutive communication messages based on three important CAN IDs such as gear and speed. There is a certain period of color shade variation at ID = 0x260 and ID = 0x43F, while the messages sent by the RPM-specific IDs are clearly irregular, corresponding to the summary in Table IV.

In conclusion, spatial-temporal details are useful for modeling the normal behavior of CAN packets, and attention also focuses on bytes that change frequently and time-series important relationships, thus helping the model to quickly detect violations and determine the targeted traffic.

It is impractical to train an IDS based on the neural network on the original CAN dataset, so data pre-processing is a necessary part before model training. The payload field is 8 bytes as shown in Table IV, where each byte is represented by two numbers in hexadecimal format. In fact, the public source dataset was progressively judged and found to contain a large number of data frames that were inferior to 8 bytes or irregular. To ensure uniformity of model input, we filled in the missing data frames with ”00” in two scenarios: 1) where the data frame is less than 8 bytes; 2) where only the single digit ”0” is used to represent a byte. Afterward, we split and transformed the arbitration bits and data domain in the dataset into a trainable dataset containing 19 features, respectively 16 features in 8 bytes of the data domain. In particular, the CAN ID is partitioned into 3 bits in order to harmonize the operation with the split data field. On the one hands, the combination of hexadecimal features of the original multi-bit will not be too far removed from the data field features due to the introduction of temporal-frequency features; on the other hand, all messages provided by the public dataset have only 3 valid bits in hexadecimal.

In addition, the values of all features are converted to decimal from hexadecimal. After implementing missing data padding and decimal conversion to meet the basic inputs for the model, several additional data pre-processing steps still need to be completed. First, the CAN frame type is encoded using a label encoder, which is used to convert categorical features into numerical features owing to many ML&DL-based algorithms cannot directly support string features [33]. Thereafter, the network dataset is normalized by the Min-Max algorithm, as the features collected in network traffic data often have a wide range of differences that impose model deviations and emphasize only large-scale features. Furthermore, the ML&DL-based model is proven to perform more convergent easily on normalized dataset [57, 58]. Hence, the data normalization by the Min-Max method is calculated as:

The method implements equal scaling of the original data, where $X_{\text{norm }}$ is the normalized data, $X$ is the original data, and $X_{\max}$ and $X_{\min}$ are the maximum and minimum values of the original dateset respectively. Table V presents the CAN data that is available as model input, where the former three columns are the CAN ID feature fields, the next eight columns in each message present 16 data domain fields, and the last column represents the label in digital form for each message. In addition, the first two rows represent the distribution for normal CAN data features, while the distribution for injection attacks is the third row. For multi-frame detection, encoder requires an additional image conversion step that splits the collected data set into $64\times 19$ 2-D images. After the normalization completing, in order to prevent the same attacks from appearing in both the test and the segmented attack data, dataset division is for 10-fold cross-validation via StratifiedKFold function in sklearn library.

The STC-IDS consists of two steps: encoding and detection, respectively. The encoder is an analysis of the spatial-temporal characteristics of the CAN messages, and capturing the important relationships based on attention to it; the detector achieves anomaly classification to the valuable spatial-temporal features.

In this paper, statistical metrics TP (true positive) and TN (true negative) are introduced to indicate the number of frames correctly classified as attack and normal, while metrics FP (false positive) and FN (false negative) are introduced to indicate the number of data frames that are misclassified as attack and normal. The model accuracy formula is as follows.

Precision (P) and recall (R) are considered as evaluation metrics to assess classification performance. Precision refers to the rate at which the actual data frame labels are detected correctly, while recall represents the proportion of all attack frame samples that end up in the attack frame class, which are calculated as follows.

and

The F1 score evaluation is also presented, which is a harmonic average based on the detection precision and completeness. Also, the F1 score is often used to measure classification performance when the data are unevenly distributed, which is calculated as follows.

Additionally, the false negative rate (FNR) and the error rate (ER) are one way of assessing classification performance. The FNR is the proportion of frames that are not detected as belonging to the attack frame and the ER is the proportion of frames that are incorrectly classified, calculated as follows.

and

The two models designed in this paper were trained offline based on the dataset, while the testing phase was based on real vehicles, injected with malicious frames of the same rules, to check the performance and efficiency of the models. The following is the experimental training and testing environment.

1. 1.

   Intel(R) Core (TM) i7-9500U CPU@3.6GHz

2. 2.

   RAM:64.0 GB

3. 3.

   GPU RTX 2080 Ti (Training environment)

4. 4.

   CAN Test, NVIDIA Jetson AGX Xavier (16GB) (Testing environment)

The selection of hyperparameters is a crucial step in network performance and inference efficiency. Currently, automated hyperparametric optimization (HPO) services and tool-kits address the constant trial-and-error steps of deep learning developers [60]. In this paper, bayesian optimization (BO) automatic parametric tuning is used in order to quickly determine hyperparameters. It is a typical method applied to global optimization problems. Compared to grid search and stochastic search, BO is more computationally efficient and requires fewer attempts to find the optimal set of hyperparameters [57].

Based on the BO optimization library provided by Keras-tuner and on a 10-fold cross-validation dataset, we selected important hyperparameters such as learning rate (1e-2, 1e-3, 1e-4, 1e-5, 1e-6), optimizer (e.g., Adam, SGD, RMSprop), dropout rate, and filters. Afterward, we set the optimization goal to validate the accuracy (Val-Acc), a maximum number of trials of 10, and train the model 3 times per trial. Finally, the optimization results will present the set of top 3 hyperparameters for performance.

As shown in Table VI, the average accuracy on the cross-validation set reaches 99.98% in the single-frame model when setting the learning rate is 1e-6, the dropout rate is 0.4, the filters are 16, 32, 128, the dense layer is 64, and the optimizer is Adam. Similarly, the accuracy is achieved up to 99.96% to the multi-frame model under the optimal hyperparameters.

Interestingly, the inference speed of the model was significantly accelerated aided by the optimal hyperparameters. Despite the higher detection performance was obtained by the single-frame model, the inference speed is slower. Relative to multi-frame, it took 50 iterations to converge, whereas the multi-frame model only took 30 generations. Fig 10 shows the iterative training losses for each dataset. It can be observed that since the DoS attack disrupts the frequency of injection, both models learn the patterns and converge quickly. However, as the complexity of attack data increases, the fuzzy and spoofing attacks injected with random data converge slower in the single-frame model, while the training loss convergence fluctuates significantly in the multi-frame model.

After training the best two models according to the set hyperparameters, Fig 11 shows the detection performance of the models in terms of ER and FNR for the four-attack testing sets in 30 repeated experiments. The two proposed models, both in terms of FNR, and ER, presents stable performance to detect DoS attacks with mean values of 0.0047%, 0.0204%, 0.0159% and 0.0094%, respectively. On the contrary, the fuzzy attacks shows greater fluctuations in detection performance. The complexity of fuzzy attack data seems to be significantly higher than the other injected data, necessitating more training iterations to construct a stable model. Hence, the ultimate mean values of FNR are still 0.0328% and 0.0413%, as well as ER get 0.0435% and 0.0864%.

Although FNR and ER on spoofing attacks are higher than DoS attacks, they achieve stable and better results compared to fuzzy attacks. The single-frame detector averages 0.0251% and 0.0353% for the Gear attack on both metrics, while the multi-frame detector obtains 0.0294% and 0.0704%. Similarly, the performance of both models is similar to Gear attacks when detecting RPM attacks. Notably, the single-frame detection model requires mining the characteristics of different CAN packets, thus exhibiting fluctuations in FNR and ER values that are greater than the multi-frame model. Despite the multi-frame model with better detection efficiency, it has a higher false alarm rate than the former.

To reflect the advantages of our model, Table VII lists the detection performance of STC-IDS when compared to those of the other machine-learning techniques, where the highest performance values are highlighted in bold, and ”-” only means that the scheme has not been tested on this metric. The results indicate clearly that the STC-IDS model outperforms the previous methods on all datasets. The false negative rate and error rate are significantly reduced. It can be seen that the model captures spatial-temporal details of network traffic remarkably well and enhances the anomaly detection ability satisfactorily.

Evidently, we can observe that the STC-IDS for single-frame model achieves a stable precision (99.97%), a higher recall (99.96%), and an outstanding F1-score (99.96%) in average as compared to a threshold (OTIDS), classification (DCNN), prediction (3-LSTM), and clustering (DAE) based models. In contrast, the performance solely decreases by an average of 0.04% precision, 0.01% recall, and 0.03% in the F1-score while maintaining efficiency in the multi-frame model.

The 3-LSTM scheme shows high accuracy and unstable recall because of the unconsidered spatial correlation, resulting in a low F1 score and a large gap between FNR and ER with our scheme and DCNN. DCNN is currently one of the best models for in-vehicle intrusion detection, but the stacking of convolutions is ineffective in capturing the temporal relationships of the inputs. As a result, the single-frame model reduces the FNR and ER by 90% and 33% over the DCNN scheme for DoS attacks, while the multi-frame model reduces the FNR by 80% and the ER by 66%. More notably, for complex fuzzy attacks, only 0.04% FNR and 0.05% ER are achieved on the single-frame model. In addition, the FNR is slightly improved on the Gear and RPM dataset. Meanwhile, the ER gets approximate performance compared with the DCNN model.

Compared to the DAE model with a lack of labeling constraints, the proposed model has larger improvements in detection performance, while DAE only reaches 95.36% in F1-score on DoS attacks, and obtains lower performance on other attacks. Similarly, the OTIDS exhibited a high accuracy rate and extremely low recall, resulting in an FNR of over 25%, reflecting a relatively low F1-score. Thus, the robust spatial-temporal correlation feature demonstrates sufficient benefits in terms of improving detection performance and reducing the false alarm rate.

This work first implemented single frame detection in terms of security considerations. Although the performance is improved over previous algorithms, the efficiency is not guaranteed. Besides, time and resources are the major limitations to applying deep learning models to real-world vehicle IDS. Therefore, STC-IDS based on multi-frames also was proposed. Obviously, significant improvements were made in terms of model convergence time as shown in subsection V-B. To test the efficiency of proposed model a resource-constrained in-vehicle network, the model was tested on an in-vehicle class device the NVIDIA Jetson AGX Xavier. Note that only 4GB of video memory was allocated for testing. The CAN Test software injected the attack traffic by connecting to the OBD-II port, as shown in Fig 12.

In Fig 13, we present that the proposed model has lower time (in milliseconds) cost on anomaly detection in case of different batches compared to previous algorithms. The average detection time for single-frame model remains under 0.7 ms. Although small-batch requires the highest time cost, the detection time is decreased based on the increasingly batch. For example, 256 batch shows 0.54 ms time cost, which is superior to DCNN [36] and MTH-IDS [33]. Moreover, this truly indicates that the model is constantly learning as it reasoned, thus making it easier to catch malicious features later on. However, it can be estimated that the model can detect 1851 messages in 1s at the fastest detection time cost, while CAN frames transmit approximately 2000 messages in 1s. One limitation of this method is that they do not usually satisfy real-time detection.

To overcome this limitation, the multi-frame model presents prominent advantages that just need the average time cost of 0.09 ms, 0.09 ms, and 0.074 ms for three batches, respectively. Compared testing time of the previous models as listed in Fig 13, the proposed model not only present outstanding performance, but also improves about 5 times in terms of efficiency. This means that the model can infer about five times the number of CAN transmission messages in 1s. Hence, the proposed model has the feasibility for real-time detection. Unlike the DCNN [36] model, a crucial phenomenon is that the proposed model does not depends on batch size. The same result reflects on 3-LSTM model [35] and MTH-IDS [33]. But even so, we also suggest a suitable batch size needs to be determined, as large batches may delay anomaly alert.

The study presents two enhanced spatial-temporal features analyzing IDS for detecting single CAN messages and consecutive CAN frames based on open injection attack datasets. Experiments indicates that the single-frame model exhibits good detection performance. Similarly, the STC-IDS for multi-frame improves efficiency while ensuring performance.

Although the detection efficiency of the single-frame model is right limited, we believe that the model will be suitable for real-time detection when using higher-performance computing devices. In fact, it is necessary to track unauthorized ECUs in conjunction with CAN ID indexing in further in-vehicle security development based on single-frame detection. It is worth noting that the time-cost detection of proposed methods is based on in-vehicle edge computing and autonomous driving platform from our research group. Although it has satisfied the requirement for real-time intrusion detection when in a real environment, it has some impact when all tasks are performed simultaneously. Hence, more research still needs to develop in terms of practical implementation.

Clearly, the spatial-temporal feature modeling in this study is based on observing the CAN ID domain, data field, and communication protocol of the specific brand vehicles. The generality consequence of the proposed model is demonstrated due to the fixed time-interval, sender and receiver, despite different vehicle companies making distinct communication protocols. In other words, model transfer only needs to be re-trained on the new brand vehicle, which also can extract valuable spatial-temporal features, and obtains excellent detection performance.

Moreover, the division of CAN ID in the scheme is limited to public datasets. In order to accommodate more realistic in-vehicle messages, the division should be completed in practice by considering both standard frames (11 bits) and extended frames (29 bits). However, it is straightforward only to modify the dimensionality of the input for the proposed model. Most importantly, the model has a fundamental limitation in terms of detecting unlearned types of attacks as it is based on supervised learning. To address this challenge, more research is needed on unknown attack detection using models with generative functions such as adversarial training, or auto-encoder.