Deep learning based black spot identification on Greek road networks

Various methods have been used for black spot prediction, including traditional statistical analysis, machine learning, and deep learning methods. These methods vary in their level of complexity, data requirements, and accuracy, and can be used to predict road accidents based on a variety of factors, such as road design and infrastructure, driver behavior, and weather conditions. The goal of this paper is to provide an overview of the different black spot prediction methods, their strengths and limitations, and to discuss their potential applications in the field of road safety. The contribution of this work is threefold as it offers an in depth literature review, it publishes a novel dataset on black spot identification (BSNG), and it establishes an identification benchmark of these problematic areas on Greek road networks.

Black spot identification is a topic of ongoing research and has been the focus of numerous studies in recent years. This is due to the increasing number of road accidents and the need to improve road safety. The objective of black spot identification is to identify locations on the road network where accidents are occurring more frequently, with the aim of reducing the frequency and severity of accidents in these areas.

A ”black spot” in road networks refers to a specific location or stretch of road that has a high frequency of accidents or incidents, that often result in fatalities or serious injuries. These areas can be identified by analyzing data from police reports, traffic accidents, and other sources. The goal is to address the underlying issues that contribute to these accidents and to make these areas safer for all road users.

The term itself was first used in the early 1760s [3] and the 1770s [4, 5]. It was popularized by road safety researchers and practitioners in Australia [6, 7] and New Zealand [8, 9], who used it to describe areas where a disproportionate number of accidents occurred. The term was later adopted by road safety authorities and organizations worldwide and has since become a widely used term in the field of road safety.

The choice of the words ”black” and ”spot” highlights the importance of identifying these areas and has helped bring attention to the need for improvements to road design, infrastructure, and driver behavior to reduce the frequency of accidents in these locations.

Different countries may have different definitions for what constitutes a black spot in their road networks, as shown in Table 1. The exact criteria used to identify black spots can vary depending on the country, type of road network, and available data [10]. For example, some countries may define a black spot as a location where a specific number of accidents occur over a certain period of time, whereas others may define it based on the severity of accidents or the number of fatalities. Some countries may also consider other factors such as traffic volume, road geometry, or driver behavior when identifying black spots [11].

Additionally, the methods for collecting and analyzing data on road accidents can differ, which can affect the accuracy and reliability of black spot identification. Some countries may use police reports [12, 13], whereas others may use more comprehensive data sources, such as road safety monitoring systems or traffic simulation models.

Given these differences, it is important to understand the specific definition of black spots used in a given country to accurately identify and address black spots in the road network. It is also worth noting that the number of black spots in a country’s road network can change over time as authorities implement safety improvements and road conditions change. Consequently, it is important to regularly monitor and analyze road safety data to identify and address black spots on an ongoing basis.

Finally, in terms of proportional comparison, it is difficult to determine which country suffers the most black spots in its road network, as this depends solely on the criteria used to define a black spot and the availability and accuracy of the data [14]. It is widely acknowledged that some countries, particularly those with a high traffic volume, are more likely to have a higher number of black spots in their road networks. For example, countries with large populations, high levels of urbanization, or extensive road networks may be more likely to have a higher number of black spots because of the increased risk of accidents in these areas.

In recent years, there has been growing interest in using machine learning, especially deep learning, in transportation research. However, the accuracy of such models depends on various factors, including the type and size of data, the location where the data were collected, and the timing of the predictions. Despite its potential, the application of machine learning for identifying black spots remains limited.

Theofilatos et al. utilized two advanced forms of machine learning and deep neural networks (DNNs) to predict road accidents in real-time. Their study used a dataset that included past accident data and current traffic and weather information from Attica Tollway in Greece [37]. The results showed an accuracy of 68.95%, a precision of 52.1%, a recall of 77%, and an AUC of 64.1%.

Fan et al. applied a classic machine learning method, such as SVM, along with a deep learning algorithm to analyze a large dataset of traffic accident records and various other relevant factors, such as road conditions and weather. They proposed an online FC deep-learning model to predict accident black spots. The reported accuracies showed that the deep learning approach outperformed the traditional statistical methods in terms of accuracy, demonstrating the potential for using deep learning in urban traffic accident prediction. The authors stressed the importance of the weather as a predictor [38].

Finally, which method would have been the most effective depends on the specific road network and the data available, and often involves a combination of them to provide a comprehensive understanding of the factors contributing to road accidents in black spot areas.

Many government agencies, such as transportation departments or road safety agencies, collect and maintain data on accidents and road safety. In the present study many sources were accessed and a plethora of resources were compiled to make compose the Black Spot Dataset of North Greece (BSNG). For instance the police, construction agencies, and experts from the academia were interviewed to frame what makes a blackspot in Greece and what they suspect as the primary cause leading to accidents based on their experience. On top of the interviews data were collected through open data portals which are publicly available to users or by requesting relevant agencies for clarifications and any additional information. Note that in Greece, a black spot is considered a stretch of road where two or more accidents have occurred¹¹1An example of data provided by ELSTAT (2014): https://www.statistics.gr/el/statistics/-/publication/SDT04/2014,²²2Hellenic legislation on National guidelines regarding road safety checks: $\Phi$EK B 1674, 13/6/2016. Thus, further analysis was conducted to detect the black spots.

Much of the data comprising the BSNG came from the web portal of Hellenic Statistical Authority (ELSTAT)³³3ELSTAT is responsible for generating new statistical data driven by its commitment to fulfill obligations towards Eurostat, as well as various European and international organizations. These obligations encompass adhering to newly established European Regulations, ensuring compliance with prevailing Regulations, fulfilling questionnaire requirements, updating databases, and more.. To acquire complementing information from the ELSTAT, official request forms were filed and anonymised data were obtained. In Greece this agency operates as an independent organization and has full administrative and financial freedom. Its mission is to safeguard and publish the country’s statistics. Collecting data regarding traffic accidents is one amongst the many topics that ELSTAT surveys. ELSTAT acquires data directly from Greek Police. When Greek Police personnel performs an autopsy at the scene of a traffic accident it fills out a report, known as Road Traffic Collision Reports (RTC). These documents (not in electronic form) are sent to ELSTAT, where they are coded and inserted into a database. RTCs can be valuable sources of information because they include information on the type of accidents that occur in a specific location, such as collisions between vehicles, pedestrian accidents, and bicycle accidents. In addition to information on the type of accidents, police reports typically provide data on the time of day and the weather conditions under which the accidents occurred. Information on the age, gender, and driving experience of drivers involved in accidents is also provided, which can help identify demographic groups that may be at a higher risk of accidents in specific areas. Hence, data collected in these reports can help to identify areas of the road network where accidents occur frequently and can provide insight into the underlying causes of these accidents.

An eight-step procedure was followed during the process of creating the BSNG, including (i) objective definition, (ii) data organization, (iii) data cleaning, (iv) data preprocessing, (v) data anonymisation, (vi) data review , (vii) analytical documentation, and (viii) data partitioning. For this work, data on road traffic accidents resulting in injury or death were collected from the national and rural network of the Regions of Macedonia and Thrace (17 Regional Units) between 2014 and 2018. Thus, the objective of the BSNG is stated as: ”To create a data collection for the identification of blackspots and the assessment of factors present in these locations regarding the traffic collisions situated in North Greece between 2014 and 2018”.

Information drawn from interviews had to be quantified regarding the degrees of injury, the visibility of vertical and horizontal traffic signs, the road surface specification, procedures to measure the road gradient and, most importantly, the identification of collision sites given that in many cases the reference was bidirectional and not concise. To organise the data into a structured format the use of spreadsheets were employed, with rows representing records and columns representing attributes or features. The following features were extracted, grouped and organised for each accident:

• •

  Accident location.

• •

  Incident and road environment details (month, week of year, number of deaths, serious injuries, minor injuries, total number of injuries, number of vehicles involved, road surface type, atmospheric conditions, road surface conditions, road marking, lane marking, road width, road narrowness, turn sequence, road gradient, straightness, right turn, left turn, boundary line marking left and right, accident severity, type of first collision).

• •

  Driver information (gender and age).

• •

  Vehicle information (type, age, and mechanical inspection status).

Duplicate values were trivially identified and removed. A very small percentage of the data records exhibited missing values. Interpolation between said records and their closest neighbor was applied to maintain the integrity of the dataset. Fifty four cases were found that had too many missing features; they were discarded promptly. Data cleaning was followed by the data prepossessing step which involved scaling numerical values and the encoding non-numerical features. Qualitative features were assigned categorical labels, whilst quantitative features were normalised. All transformations were documented analytically.

Finally, it is important to mention that special care was given in anonymizing the data records, such that any personal information is concealed and tracing the connection between an individual participated in a collision and a data record in BSNG is effectively eliminated. To achieve that personally identifiable information (PII) was removed, individual data points were aggregated into groups to hide individual-level information, like location specifics, and specific values were replaced with broader categories, for example, the exact ages were replaced with age ranges, namely 1-25, 26-64 and ¿65.

All thirty five (35) variables and their respective mean, minimum and maximum values are shown at Table 2. The number of black spots was determined by applying the guidelines indicated officially by the Law. From the total of 1811 accidents, only 142 situated at black spots (or 1/5^th of the 735 unique accident locations), as shown in Table 3.

The k-Nearest Neighbors (k-NN) algorithm is a non-parametric, lazy learning method used for both classification and regression tasks in machine learning. It’s based on the idea that similar data points in a feature space should have similar labels or outputs.

The k-NN algorithm operates in a feature space, where each initial data point is represented as a point in a multidimensional space. Each dimension of the space corresponds to a feature of the data. To determine the similarity between data points, a distance metric is used. Common distance metrics include Euclidean distance, Manhattan distance, and cosine similarity. The choice of distance metric depends on the nature of the data and the problem. The $k$ in k-NN refers to the number of nearest neighbors considered in the algorithm. A higher value of $k$ results in a more complex decision boundary and increased smoothness, while a lower value of $k$ results in a more flexible and potentially overfitting decision boundary.

A Multilayer Perceptron (MLP) is a type of artificial neural network used for classification and regression tasks in machine learning. It consists of multiple layers of nodes (neurons) organized in a feedforward structure, with each layer fully connected to the next one. MLPs are capable of learning complex, non-linear relationships between input features and output values by adjusting the weights of the connections through a process called backpropagation.

An MLP typically consists of an input layer, one or more hidden layers, and an output layer. Each layer consists of multiple nodes, where the input layer corresponds to the input features, the hidden layers perform non-linear transformations, and the output layer produces the final predictions.

The nodes in the hidden layers use an activation function to introduce non-linearity into the model. Common activation functions include the sigmoid, hyperbolic tangent (tanh), and Rectified Linear Unit (ReLU). Given an input vector $x$, the MLP computes the output through a process called forward propagation. For each layer $l$, the pre-activation value (a) and the activation value $(z)$ are computed as follows:

where$W^{(l)}$is the weight matrix for layer $l$, $b^{(l)}$ is the bias vector for layer$l$, $f$ is the activation function, and $z^{(0)}=x$.

The output layer uses an output function to produce the final predictions. For classification tasks, the softmax function is commonly used to compute class probabilities, while for regression tasks, a linear output function is typically used. To train the MLP, a loss function is used to measure the discrepancy between the predicted values and the true output values. Common loss functions include the cross-entropy loss for classification and the mean squared error for regression. The weights and biases in the MLP are adjusted using an optimization algorithm, usually gradient descent or a variant thereof, in combination with the backpropagation algorithm. Backpropagation computes the gradients of the loss function with respect to the weights and biases by applying the chain rule of calculus. The gradients are then used to update the weights and biases to minimize the loss function.

The proposed approach is divided into the following four steps. At the beginning, each variable is transformed into labeled and one-hot encodings, to ensure the applicability of machine learning methods. Then a self-supervised deep learning architecture is used to reduce the dimensions of the input features into a compact latent vector. An augmentation method, like mixup, is used to linearly combine the feature vectors of the samples of the blackspot class to even the number of samples of the two classes of the BSNG dataset. The latent vectors are used as input in a classifier to approximate a binary class problem, e.g., whether the sample is a black spot or not. A graphical overview of the proposed method is shown in Figure 1.

One hot encoding is a widely used technique for encoding categorical variables in machine learning and data analysis. Categorical variables are variables that can take on a limited set of values, such as colors, types of animals, or car brands. Since many machine learning algorithms cannot process categorical variables directly, we need to convert them to numerical values before using them in models.

One hot encoding is a way of representing categorical variables as numerical values in a binary format. This technique creates a binary vector where each column corresponds to a possible category, and only one of the columns is ”hot” or ”on” at a time, indicating the presence of that category in the input data.

In practical terms, suppose a weather dataset contains a column named ”Weather” that has four possible values: ”sunny”, ”cloudy”, ”rainy”, and ”snowy”. Using one hot encoding, the ”Weather” column would be transformed into four binary columns, with each row containing only a single ”1” value in the column that corresponds to the weather condition of that row (see, Table 4).

One hot encoding has several advantages over other encoding techniques. For one, it preserves the information in the original categorical variable and does not introduce any artificial ordering or ranking of the categories. Additionally, it allows for easy comparison of different categories since they are represented by separate columns, and it works well with many machine learning algorithms.

An autoencoder is a type of unsupervised artificial neural network used for dimensionality reduction, feature learning, and data compression. It consists of two main components: an encoder that maps the input data to a lower-dimensional latent representation, and a decoder that reconstructs the input data from the latent representation. The autoencoder learns to compress and reconstruct the input data by minimizing the difference between the input and the reconstructed output, often using a loss function like the mean squared error. Both the encoder and the decoder can consist of multiple layers.

Given an input vector $x$, the encoder computes the latent representation $z$ through a series of transformations:

where $f_{i}$ is the activation function for layer $i$, $W_{i}$ and $b_{i}$ are the weight matrix and bias vector for layer $i$, and $L$ is the number of layers in the encoder.

Given the latent representation $z$, the decoder computes the reconstructed input data $x^{\prime}$ through a series of transformations:

where $f_{i}$ is the activation function for layer $i$, $W_{i}$ and $b_{i}$ are the weight matrix and bias vector for layer $i$, and $M$ is the number of layers in the decoder.

The autoencoder learns to compress and reconstruct the input data by minimizing the difference between the input $x$ and the reconstructed output $x^{\prime}$. A common loss function for this purpose is the mean squared error (MSE):

where $N$ is the number of input samples.

The autoencoder is trained using an optimization algorithm, usually gradient descent or a variant thereof, in combination with backpropagation to compute the gradients of the loss function with respect to the weights and biases. The weights and biases are then updated to minimize the loss function.

MixUp is a data augmentation technique proposed for training deep learning models, particularly in the context of image classification tasks. MixUp generates new training samples by taking convex combinations of pairs of input samples and their corresponding labels. This technique encourages the model to learn more robust features and improves its generalization performance.

Given a dataset with input samples $X$ and labels $Y$, two samples $x_{1}$ and $x_{2}$ are randomly selected with corresponding labels $y_{1}$ and $y_{2}$. A mixing parameter $\lambda$ is sampled as well from a predefined distribution. Typically the Beta distribution with parameters $\alpha>0$ and $\beta>0$ (e.g., $\alpha=\beta=0.4$).

A new mixed sample $x^{\prime}$ and its corresponding label $y^{\prime}$ are created using the convex combination of $x_{1}$, $x_{2}$, $y_{1}$, and $y_{2}$ with the mixing parameter $\lambda$:

The new mixed sample $x^{\prime}$ and its corresponding label $y^{\prime}$ are appended to the training dataset.

The MixUp technique encourages the model to behave linearly between the training samples, leading to smoother decision boundaries and improved generalization performance. It can be used in combination with other data augmentation techniques, such as random cropping, flipping, or rotations, to further enhance the model’s robustness.

In this work, the BSNG dataset is introduced, a comprehensive collection of collision data that occurred on the national road network of North Greece between 2014 and 2018. This dataset comprises 1,811 samples, including 142 black spots, as classified by the ELSTAT. To analyze these black spots, ten classic machine learning methods are employed, as well as a novel approach is proposed. The machine learning methods utilized include Poisson regression, naive Bayes, Gaussian processes, k-NN, linear and non-linear SVM, decision tree, ensemble methods such as random forest, extra randomised trees, adaptive boosting, and a neural network. All models and experiments were conducted on the same platform, using Python and Keras as the deep learning framework. This paper aims to provide valuable insights into the analysis of black spots using machine learning techniques, which can potentially contribute to reducing traffic accidents on the national road network in North Greece.

All methods applied to the BSNG dataset required hyperparameter optimization. In all experiments, hyperparameters were optimized using 5-fold cross-validation, with the aim of maximizing the F1 score. For K-Nearest Neighbors, the value of $k$ was set to $4$, which is twice the threshold applied by Greek authorities. Poisson regression was tuned using a regularization strength of $\alpha=0.9$ and the Low memory Broyden-Fletcher-Goldfarb-Shanno(LBFGS) solver with a tolerance of $10^{-5}$. Gaussian process modeling required an appropriate kernel to fit the data, and a radial basis kernel (RBF) with a small length scale of $I=0.125$ was used. The non-linear case of SVM also used an RBF kernel, with $\gamma=\frac{1}{2\cdot I^{2}}$. The dimensionality of the data were reduced to five components with PCA, in order for the SVMs to work properly. For tree-based methods, feature bagging was used without pruning. Ensemble methods used 30 estimators. The multilayered perceptron used five fully connected layers with $(512,7,64,32,4)$ nodes, ReLU activation, a learning rate of $10^{-4}$, the Adam solver, and 100 training epochs. The number of nodes in these layers were discovered by a grid searching approach.

In the following experiments, the original BSNG dataset underwent transformations in an effort to improve the performance of the methods. First, a one-hot encoding was applied to each variable, resulting in a dataset with $687$ dimensions, up from the original $35$. In the last experiment, the MixUp technique was used to augment the dataset. This involved sampling $6000$ random pairs with replacement and creating $11$ additional samples for each selected pair of initial samples, using a beta distribution with $\beta(0.2,0.2)$. This resulted in a total of $67448$ training samples.

The proposed method is unique compared to others because it takes a one-hot encoded BSNG as input and employs an autoencoder to generate latent representations. Data augmentation is performed using MixUp. The autoencoder’s architecture has a bottleneck layout with the encoder consisting of three layers with ReLU activations and nodes of sizes $(256,64,32)$. The decoder is the reverse layer order. The optimizer used is Adam with a learning rate of $10^{-4}$. MixUp is applied by sampling $6000$ random pairs with replacement and creating $11$ additional samples for each selected pair of initial samples using a beta distribution with $\beta(0.2,0.2)$. This resulted in a total of $67448$ training samples. The classifier is a MLP with three fully connected layers with $(32,24,6)$ nodes, ReLU activation, a learning rate of $10^{-4}$, the Adam solver, and 100 training epochs. The number of nodes in these layers were discovered by a grid searching approach as before.

The BSNG dataset poses a significant challenge to classification algorithms, as indicated by the findings presented in Table 5. Although high accuracy rates are reported, they are misleading and should not be relied upon. The imbalanced nature of the dataset makes it inappropriate to judge performance based on the fraction of correct predictions. For example, a classifier that assigns all samples to the ”not black spot” class would achieve an accuracy of $87.5\%$, but would be useless in practice.

To provide a more complete picture of performance, we present four additional metrics: precision, recall, f₁-score, and area under the curve (AUC). Using the dataset as-is results in poor classification performance for all methods, due to the imbalance between the two classes and the difficulty of identifying the features that distinguish black spots from non-black spots. Encoding the feature space, which contains both continuous and categorical variables, improves performance slightly, but the overall performance remains subpar due to the limited number of samples in the dataset.

To address this issue, we augment the dataset with virtual samples created using the MixUp technique. This approach improves the classification performance of all methods, particularly when the feature space is small. However, statistical modeling appears inadequate for handling larger feature spaces and low sample counts. In contrast, tree-based ensemble methods outperform other approaches in terms of f₁-score and AUC.

The proposed method, which utilizes a shallow MLP and the low dimensionality of the latent features obtained through MixUp, outperforms all other methods on all metrics. Although the black spot detection scores are low by modern standards, they are comparable to the results reported in [39], which achieved an accuracy of 68.95%, a precision of 52.1%, a recall of 77%, an AUC of 64.1%, and an f₁-score of 62.14%. Overall, the findings of this work underscore the importance of carefully considering dataset characteristics and choosing appropriate metrics when evaluating classification algorithms.

In the field of statistical modeling, many approaches rely on strict assumptions, such as pre-determined error distributions, and often struggle to handle issues like multicollinearity and noisy or missing data [42]. To address these challenges, ensemble methods like Random Forest and AdaBoost have gained popularity. Random Forest is an ensemble of decision trees, each trained on a random subset of the data and features, and the final prediction is made by aggregating the individual tree predictions. In contrast, AdaBoost iteratively trains weak classifiers, typically shallow decision trees, by adjusting weights of misclassified samples. The final prediction is made by taking a weighted majority vote of the weak classifiers.

In the context of the BSNG dataset, Random Forest and Xtra Trees performed better than AdaBoost in identifying accidents that occurred in blackspots. Increasing the sample population through techniques like MixUp also helped improve the performance of most algorithms. The proposed method in this study uses an alternative encoding technique and augmented the dataset with virtual samples created through MixUp. In the experimental design, the performance of the proposed method was compared to other setups that included using a textbook encoding technique like one-hot encoding, and augmenting the dataset with MixUp. The results show that performance gains were achieved when increasing the sample population and using the proposed method’s encoding technique. The proposed method outperformed all other setups in terms of accuracy, precision, recall, f₁-score and AUC. This suggests that the proposed method can be a promising solution for identifying accidents that occurred in blackspots, despite the challenges posed by the imbalanced and complex BSNG dataset.

Previous studies have shown mixed effects of traffic flow on accident rates, with some suggesting a non-linear correlation and others indicating a linear relationship [40, 43], the impact of traffic flow and congestion on blackspot determination still remains uncertain. Similarly, weather patterns seemed to vary and low visibility consistently influenced road safety, as expected.

It is noteworthy to mention that some findings of this research diverge from the prevalent understanding of the main causes of accidents involving young or novice drivers, as highlighted in an earlier study [44]. Contrary to the reference which examined the period from 1995 to 2001 in Greece, the results of the current study indicate that non-compliant behavior and lack of driving experience may not be the primary contributing factors to accidents in the region under investigation.

According to the data collected, it was found that a mere 3.51% of the vehicles involved in blackspot accidents during the specified period did not possess a valid technical control certificate. This finding suggests that the issue of non-compliance with vehicle technical regulations may not have a substantial impact on accident occurrence. Furthermore, the analysis revealed that the age group most frequently involved in accidents at blackspots was not composed of young or novice drivers, but rather individuals driving cars that were more than 10 years old. In fact, these older vehicles accounted for approximately 70% of all accidents that transpired in blackspots within the study area.

These unexpected findings challenge the previously established notions regarding the main causes of accidents in the region, as well as the assumptions underlying the recent changes in traffic enforcement policies. The aforementioned research study [44] conducted in Greece highlighted the necessity for stricter police enforcement to address non-compliant behavior among drivers. While our findings do not necessarily contradict this recommendation, they shed light on the fact that other factors may also play a significant role in accidents. Rather, it is plausible to consider that advancements in car manufacturing technology and improved safety standards have contributed to the relatively lower frequency of modern cars being involved in such accidents.

It is important to acknowledge that the focus of this study was not to directly assess the impact of evolving technology and safety standards on accident rates. However, the observed pattern of older vehicles constituting a significant proportion of accidents in blackspots implies that the enhanced safety features and design elements present in newer cars may have played a role in mitigating the occurrence of accidents in these specific areas. Further investigations into the specific mechanisms through which modern cars exhibit improved safety performance, particularly in blackspots, would be valuable in confirming this hypothesis and informing future policy decisions in the field of traffic safety.

Despite the valuable insights provided by the current research regarding the accidents transpiring in blackspots, it is crucial to acknowledge certain limitations that may impact the generalizability and comprehensiveness of the findings. Recognizing these limitations can result in a more nuanced understanding of the research outcomes and the implications they hold.

It is essential to note that the study focused exclusively on the region of northern Greece and the specific timeframe of 2014 to 2018. The findings, therefore, may not be representative of the broader traffic patterns and characteristics of other regions within Greece or different time periods. Variations in road infrastructure, and enforcement policies across regions and timeframes could influence the results and limit their applicability beyond the study area.

Moreover the investigation primarily relied on quantitative data analysis. A method was designed to deal with the identification of blackspots as a supervised learning problem in the context of machine learning. While this approach provided valuable statistical insights, it may not fully capture the complex interplay of various factors contributing to accidents, such as driver behavior, environmental distractions or other temporal phenomena. Augmenting the dataset with in-depth interviews of drivers and stakeholders or perhaps gathering real-time accident data from the collided vehicles via Internet of Things (IoT) sensors, could have offered a more comprehensive understanding of the underlying causes of accidents in blackspots.