Spatially Focused Attack against Spatiotemporal Graph Neural Networks

In the domain of computer vision, adversarial attacks aim at fooling a machine learning-based classifier to misclassify objects with undetectable modifications. When it comes to spatiotemporal forecasting, adversarial attacks is to add unnoticeable perturbations into historical time series such that the forecasting models begin to generate bad predictions that are far away from the ground truth. We formulate this goal as the the following optimization problem:

$$~{}\begin{split}\mathop{\max}_{\bm{\rho}}&\left\|F\left(\bm{\mathcal{G}}+\bm{\rho}\right)-{\mathcal{G}}_{t+1}\right\|_{2}\\ {\rm s.t.}\ &\rho_{i}^{2}\leq\xi,\end{split}$$

(3)

where $\bm{\mathcal{G}}=\left\{\mathcal{G}_{t},...,\mathcal{G}_{t-N+1}\right\}$ denotes the input graph sequence, $\mathcal{G}_{t+1}$ denotes the future traffic state that is the ground truth of the forecasting model, $\bm{\rho}=\{\rho_{t},...,\rho_{t-N+1}\}$ denotes a collection of perturbations, and $\xi$ denotes the pre-specified constant to constrain the perturbation scale in order to make the perturbations unnoticeable.

However, it is difficult to directly solve the optimization model in  (3). As an alternative, we design a proxy optimization problem by integrating the constraint into the objective function:

$$~{}\mathop{\max}_{\bm{\rho}}\left\|F\left(\bm{\mathcal{G}}+\bm{\rho}\right)-{\mathcal{G}}_{t+1}\right\|_{2}-\alpha\sum_{i=t-N+1}^{t}\max\left(0,\rho_{i}^{2}-\xi\right),$$

(4)

where $\alpha$ denotes the penalty factor. Even though the regularization term in Eq. (4) is a soft constraint compared to the original constraint $\left\|\rho\right\|_{2}\leq\xi$, it can still strictly force the perturbation’s scale less than the upper bound, $\xi$, by setting the penalty factor to be a large value to make the scale penalty term much larger than the first term in Eq. (4). In real-world traffic applications, $\xi$ is used to balance the attack performance and degree of observability.

However, the fact is that the ground truth of a forecasting model, $\mathcal{G}_{t+1}$ in Eq. (4), is unavailable at time $t$. This issue is referred to as the “temporal gap” in Figure 1. Due to the inevitable usage of the future information, fooling spatiotemporal GNNs as Eq. (4) with the simple optimization model becomes unrealistic. A simple solution to address this issue is to directly use the most recent observations as the forecasting. This is equivalent to approximating the complex GNNs with a “most recent” forecaster:

$$~{}\mathcal{G}_{t+1}\leftarrow\mathcal{G}_{t}.$$

(5)

However, perturbations computed by the direct estimation may not be effective enough to cause significant performance drop. The reason is the direct estimation involves great errors in the perturbation computation, which is discussed in Section 5.1. In addition, the maximization optimization problem is still difficult to solve given the large search space.

To address the said issue, we propose an Inverse Estimation scheme to offer a simple but clear direction to the optimization model. We introduce the concept of “opposite state” and transform the maximization problem in (4) into a minimization problem, whose goal is to fool spatiotemporal GNNs to generate predictions opposite to the ground truth:

$$~{}\mathop{\min}_{\bm{\rho}}\left\|F\left(\bm{\mathcal{G}}+\bm{\rho}\right)-\tilde{\mathcal{G}}_{t+1}\right\|_{2}+\alpha\sum_{i=t-N+1}^{t}\max\left(0,\rho_{i}^{2}-\xi\right),$$

(6)

where $\tilde{\mathcal{G}}_{t+1}$ denotes the “opposite state” of $\mathcal{G}_{t+1}$. Perturbations can be generated more effectively by solving Eq. (6). The above idea is similar to targeted attacks (Akhtar and Mian 2018). However, classical targeted attacks still utilize the ground truth in perturbation computations.

To avoid using future information, the opposite of future state, $\tilde{\mathcal{G}}_{t+1}$, is estimated by computing the opposite of the most recent state:

$$~{}\tilde{\mathcal{G}}_{t+1}\leftarrow\tilde{\mathcal{G}}_{t}=\left\{\tilde{\mathcal{V}}_{t},\mathcal{E},W\right\},$$

(7)

where $\tilde{\mathcal{V}}_{t}=\left\{\tilde{v}_{1,t},...,\tilde{v}_{n,t}\right\}$ denotes a collection of state values opposite to these collected from sensors. Taking the traffic speed as an example, when the condition is “congested/low speed”, its opposite state should be “free/high speed”. In this case, we compute $\tilde{v}_{i,t}$—the the opposite of $v_{i,t}$—using two distinct values:

$$~{}\tilde{v}_{i,t}=\left\{\begin{aligned} \max\left(\mathcal{V}\right),&\quad v_{i,t}<{\rm mid}\left(\mathcal{V}\right),\\ \min\left(\mathcal{V}\right),&\quad v_{i,t}\geq{\rm mid}\left(\mathcal{V}\right),\end{aligned}\right.$$

(8)

where ${\rm mid}\left(\mathcal{V}\right)$, $\max\left(\mathcal{V}\right)$, and $\min\left(\mathcal{V}\right)$ represent the mean, maximum, and minimum value of the spatiotemporal dataset, respectively. We examine the effectiveness of this approach in the experiment section with real-world datasets.

In this section, we introduce a solution to avoid manipulating (introducing perturbations on) all sensors in the road network. The key idea is to identify the most vulnerable sensor, which produces the largest accuracy drop to the whole sensor network when being attacked. To achieve this goal, we first solve a universal attack problem by making the perturbation static. Thus, the universal attack implies that the perturbation is consistent and independent from the input. The universal perturbation is computed by solving the following optimization problem

$$~{}\mathop{\min}_{\rho_{u}}\left\|F\left(\bm{\mathcal{G}}+\{\rho_{u}\}\right)-\tilde{\mathcal{G}}_{t+1}\right\|_{2}+\alpha\cdot\max\left(0,\rho_{u}^{2}-\xi\right),\\$$

(9)

where $\rho_{u}$ denotes the universal perturbation. After the universal perturbation is generated, there is no need to update it when new data come. It should be noted that this universal perturbation can only break forecasting models by poisoning all sensors. Thus it is unrealistic to apply it to real-world forecasting applications. Nevertheless, this universal perturbation will be used to define and quantify sensor weakness.

We define the “weakness” score of the $j$th sensor as the number of influenced/affected sensors when sensor $j$th is attacked by the proposed universal perturbation. Specifically, the weakness is computed as

$$weak_{j,t}=\left\|K_{\theta}\left\{F\left(\bm{\mathcal{G}}+M_{j}\cdot\rho_{u}\right)-\mathcal{G}_{t+1}\right\}\right\|_{0},\\$$

(10)

where $M_{j}\cdot\rho_{u}$ denotes that all elements except the one corresponding to the $j$th sensor, and $K_{\theta}\left\{{\cdot}\right\}$ denotes an element-wise filter to set elements whose absolute value is smaller than $\theta$ to 0. As Eq. (10) shows, the weakness is actually time-dependent. By collecting the weakness in a time frame, a weakness vector can be formed. The $l$2-norm of the vector can be regarded as the time-invariant weakness for a vertex. Thus, for a sensor $j$, a greater weakness value suggests that more sensors will be influenced if it is attacked. We will attack the vertex with the largest “weakness” value. For traffic forecasting applications, we consider the vertex where the prediction error is greater than 5 km/h as the influenced vertex, and $\theta$ is set to 5 consequently.

A possible method to locate the weakest vertex is the complete traversal algorithm. However, this method is time consuming. To reduce the time cost, we utilize the genetic algorithm to locate the weakest vertex, which is shown as follows. The genetic process is presented schematically in Figure 2.

• •

  First, the initial candidate set is generated with $s$ sensors with the most edges.

• •

  Second, the updated set consists of $s$ new candidates and they are computed as

  $$c_{i}(g+1)=c_{r1}(g)+p(c_{r2}(g)-c_{r3}(g)),$$

  (11)

  where $c_{i}$ denotes the position (longitude and latitude) of the $i$th vertex, $g$ denotes the $g$th iteration, $r1$, $r2$, and $r3$ are random numbers with different values, and $p$ is set to 0.5 empirically.

• •

  Third, compare the weakness of updated candidates with the previous candidate set, then keep only $s$ candidates with the largest weakness value.

• •

  Fourth, repeat the second and third step until the candidate set is consistent or $g$ is sufficient. Select the weakest vertex to attack. It should be noted that the bound of $g$ controls the trade-off of the effectiveness and efficiency of the solution. The larger bound represents the proposed solution is much closer to the complete traversal algorithm.

Once the weakest vertex is located, spatially focused attack is proposed to fool the spatiotemporal forecasting model by poisoning only the selected vertex. The perturbation is computed by solving the following optimization problem:

$$~{}\begin{split}\mathop{\min}_{M_{J}\cdot\bm{\rho}}\left\|F\left(\bm{\mathcal{G}}+M_{J}\cdot\bm{\rho}\right)-\tilde{\mathcal{G}}_{t}\right\|_{2}+\alpha\cdot\mathcal{R},\end{split}$$

(12)

where $M_{J}\cdot\bm{\rho}$ is the generated one vertex perturbation, $J$ denotes the index of the weakest vertex, and $\mathcal{R}=\max\left(0,(M_{J}\cdot\bm{\rho})^{2}-\xi\right)$ denotes the regularization term to control the scale of the generated one vertex perturbation.

Different from adversarial attack methods as in Eq. (6) and Eq. (9) that manipulates all sensors, the proposed SFA poisons only one sensor in the road network in order to achieve the largest accuracy drop. Therefore, SFA provides a more realistic and reasonable framework for implementing real-world attacks. In reality, perturbation on a selected sensor can be introduced by making a hacker vehicle drive by or by controling the network communication to generate fake sensor readings. Overall, we consider SFA an essential and meaningful attack strategy and thus it can be used to evaluate the robustness and vulnerability of different GNNs-based traffic forecasting models.

We compare the effectiveness of perturbations generated from the inverse estimation (Eqs. (6) and  (7)) with the direct estimation (Eqs. (4) and  (5)). It should be noted that in this experiment we apply perturbations on all sensors instead of attacking only one sensor. We conduct the analysis on 15 min-head traffic prediction on the PeMS data with STGCN as the base model.

Figure 3 shows the performance comparison between the proposed inverse estimation and the baseline, direct estimation. As can be seen, perturbations generated from inverse estimation can cause the forecasting forecasting model’s accuracy drop greater compared with perturbations generated from direct estimation. Inverse Estimation outperforms directly estimating the future ground truth because the proposed IE can feed less errors into the perturbation computation process. Both strategies cannot achieve perfect estimation and their estimation errors can impact the effectiveness of perturbations. The proposed inverse estimation is a binary estimation and it is much easier than estimating a continuous value. Thus the proposed IE involves less errors in perturbation computations, which in turn, as a result, also lead to more effective adversarial examples. Take PeMS for instance, errors fed by IE is small (MAPE: 0.56%), while errors fed by the direct estimation is large (MAPE: 3.3%).

We next examine the effect of hyperparameters on locating the weakest sensor. The setting is as same as that in Section 5.1. We set the number of candidates $s$ to 5 and 10, respectively, and record the number of the iterations. We use NMAPEI and computation time to measure and compare the performance of different hyperparameter configurations.

From Figure 4, the computation time generally grows with the increase of the iteration $g$. When $s$ is set to 5, locating the weakest vertex is much harder compared with the case of setting $s$ to 10. A possible reason is that, with a few initial candidates, the proposed strategy tends to converge to a local optimum. Note that when we set $s$ to 10, the proposed strategy breaks the iteration loop. For the following experiments, we set both $s$ and $g$ to 10.

In this section, we design experiments to demonstrate the effectiveness and efficiency of different strategies in locating the weakest sensor. The setting in Section 5.1 is applied to this experiment. We compare the proposed approach with three simple baselines, including (1) locating the vertex with the highest degree (DEG), i.e., the number of connected sensors, (2) locating the vertex with the highest weighted degree centrality (CEN), i.e., row-sum of the weighted adjacency matrix, and (3) locating the weakest vertex by the complete traversal algorithm (CT). After locating the weakest vertex by different strategies, perturbations are computed based on Eq. (12) and then fed into STGCN. We evaluate different approaches using NMAPEI, 30%-IV, and computation time.

Table 1 shows the comparison results. As we can see, the proposed solution achieves the same optimal as CT—it identifies the same weakest sensor as the full enumeration. On the other hand, simply poisoning the vertices with highest degree and the highest centrality cannot ensure an effective and efficient attack. A possible reason is that the robustness of sensors is improved by their neighbors because of the local/spatial aggregation mechanism in GNNs. Besides, the proposed strategy can reduce 40% computation cost compared with the CT.

In this section, we evaluate examine the effect of perturbation scale on the attack performance. We can consider perturbation scale an indicator for attack observability. A larger perturbation is more likely to be noticed by the user of GNNs. We first propose an experiment to assess how the parameter $\xi$ in Eq. (12) influences the effectiveness of the proposed one vertex attack method. In this subsection, 15 min traffic speed forecasting is undertaken by STGCN, DCRNN, and Graph Wavenet that work as the targeted models and the experiment is conducted on META-LA(S). These models are attacked by the proposed SFA with different $\xi$. Note that only one sensor/vertex is poisoned in this experiment.

Table 2 shows the number of impacted vertices (IV) with different $\xi$. When setting $\sqrt{\xi}$ to 20 km/h, we find that around 10% sensors will have an NMAPEI greater than 40% and $\sim$90% sensors will show at least 5% increase in NMAPEI. This suggests that, when we have a large $\sqrt{\xi}$, the whole network could be severely impacted even attacking only one sensor. With a small $\sqrt{\xi}$, there are about 50% sensors are influenced for at least 5% NMAPE. Based on Table 2, we can conclude that perturbations will be effectively diffused from one vertex to most of the graph when we apply GNNs-based spatiotemporal forecasting models. The greater the perturbation is, the larger the number of sensors in the graph will be influenced. Our analysis also suggests that setting $\xi$ to an appropriate range is important. An extremely large $\xi$, which represents abnormal driving behaviors in traffic domains, can be detected easily by the user of GNNs. By analyzing PeMS and META-LA(S), speed variation within 15km/h occurs frequently, and consequently, we regard the accessible boundary of speed variation is 15 km/h, namely $\sqrt{\xi}=15$ km/h.

We set parameters and hyperparameters in SFA based on the aforementioned experimental results. Finally, we conduct experiments on PeMS to show the overall the effectiveness of the proposed method. We perform 15 min-ahead traffic speed forecasting using the three mentioned GNNs and compare the proposed SFA with four attack baselines.

• •

  GWN: Generate Gaussian White Noise (GWN) with the scale being consistent with $\sqrt{\xi}$ as in SFA, and attack the same weakest sensor. This baseline is used to evaluate the effectiveness of perturbation design as in Eq. (12).

• •

  DEG: We simply consider the sensor with the largest number of neighbors as the “weakest”, and attack it using the same optimization algorithm as in SFA. This baseline is designed to examine whether the proposed locating strategy can effectively identify the weakest sensor (or a suboptimal that also works well).

• •

  MFGSM: Attack all sensors with the modified Fast Gradient Sign Method (FGSM) (Szegedy et al. 2015; Moosavi-Dezfooli, Fawzi, and Frossard 2016). We replace the ground truth in the original FGSM with the proposed inverse estimation as Eq. (7). The modified FGSM perturbation is computed as

  $$\bm{\rho}=\epsilon\ \operatorname{sign}(\bigtriangledown\mathcal{J}(\Phi,\bm{\mathcal{G}},\tilde{\mathcal{G}}_{t})),$$

  (13)

  where $\bigtriangledown\mathcal{J}$ computes the gradient of the cost function around the prediction of the forecasting model parameterized by $\Phi$ w.r.t the input sequence $\bm{\mathcal{G}}$, $sign$ denotes the sign function, $\tilde{\mathcal{G}}_{t}$ denotes the inverse estimation of the ground truth, and $\epsilon$ control the perturbation’s scale. This baseline is chosen to compare the targeted attack in SFA with manipulating all sensors. We set the scale parameter, $\epsilon$, to 2 and 3, respectively.

In these experiments, we set $\sqrt{\xi}$ to 15 km/h for methods that attack only one sensor (i.e., SFA, GWN, and DEG). Table 3 shows the experiment results, which confirm the superior performance of the proposed SFA framework. We can see that SFA outperforms attacking the vertex with the most edges (DEG), showing that SFA can effectively identify a weak sensor. SFA also outperforms GWN that attacks the same weakest sensor, confirming that the proposed method can generate the optimal perturbations for manipulating only one vertex to fool the spatiotemporal forecasting model in the entire graph.

Fig. 5 shows an example of attack results on PeMS using STGCN as the forecasting model. We apply SFA to attack sensor A with optimized/designed perturbations (see Fig. 5(b)). In Figs. 5(c) and 5(d), we the values/results on sensors (B) and (C), including the ground truth traffic speed, the default forecasting of STGCN without attacks, and the forecasting results when implementing the perturbations on senosr A. As can be seen, the attack on sensor A can cause substantial accuracy drop on sensor B, while sensor C is less affected by the attack. A potential reason is that A and B are connected on the highway, thus having strong dependencies, while the forecasting of sensor C might be mainly determined by its local neighbors in GNNs. Overall, for traffic forecasting in PeMS, the proposed SFA framework can cause more than 15% accuracy drop for all three Spatiotemporal GNN models, and about 10% sensors are severely impacted (the NMAPEI of these sensors are greater than 30%) when setting $\sqrt{\xi}$ to 15 km/h.

Nevertheless, it should be noted that attacking all sensors will always be more effective than attacking only one vertex; however, the results in Table. 3 show that the one-vertex-based SFA can offer comparable performance as MFSDM that attack all sensors when setting $\epsilon=2\text{ km/h }\approx 0.13\sqrt{\xi}$ (both $\sqrt{\xi}$ and $\epsilon$ control the level of perturbations). Overall, the above experiments confirms the effectiveness of SFA. The perturbation can be diffused into the entire graph and even predictions on sensors that are far from the attacked one can be severely influenced.