Abnormal Occupancy Grid Map Recognition using Attention Network

The research on automatic map construction using robots has always been a key point for researchers [19, 20]. Elfes et al. [21] introduced a grid-based algorithm for 2D environment modeling. The continuous spaces of the environment are discretized by these evenly-spaced grids. According to the probabilistic formulation, each grid cell of constructed map represents the probability of the corresponding region that may be occupied, free or even not yet explored. With the help of occupancy grid map, the robot can identify the presence or absence of an obstacle in the space of the environment.

However, the occupancy grid map suffers from corrupted obstacle silhouettes, occlusions, and false distance estimates in static environments. To address these problems, a highly engineered method [11] was proposed to extract information for the environment modeling. A data-driven method for map recognition is proposed in [13], where the radar sensor measurement model was trained and this method outperforms the manually designed model. Piewak et al. [14] trained a neural network to reduce false distance estimation in a dynamic environment. Their approach referred to a pixel-wise classification task to determine whether a cell in the actual environment is occupied or free.

In the last years, many attempts have been made to improve the original CNN architecture to achieve better accuracy. In image classification and object detection, recent approaches like AlexNet [22], VGG-16 [23], InceptionNet [24], or MobileNet [25] are based on the plain CNN architecture. However, CNN-based networks have problems in gradient propagation and convergence with the amount of data increases [26]. To address this issue, ResNet [27] proposes a simple identity skip-connection to ease the optimization problem of deep networks. Our attention network is a backbone architecture based on ResNet, which can enhance the input features but also well benefit gradient propagation on occupancy grid map data training process.

Recently, various attention mechanisms had been proposed for image recognition tasks. Hu et al. [17] had proposed a compact SE block to extract the inter-channel information. Residual Attention Network (RAN) [28] modified ResNet by stacking identical soft attention modules which is beneficial to refine the feature maps. Spatial attention [29] had been made to recalibrate the channel dependency as an effective extraction module. Then, Woo et al. [18] introduced a CBAM module that sequentially recalibrates channel and spatial attention to refine intermediate feature maps. However, these methods miss the global spatial information, which is an important factor for feature fusion and accurate attention map generation. Inspired by the global context block[30], our csRSE attention network, which contains a residual block for producing hierarchical features, followed by both cSE block and sSE block for the sufficient spatial information extraction and the resulted hierarchical features enhancement.

The constructed OGMD covers diverse daily-life indoor environment scenarios, such as residential houses, super-markets, office buildings, hotels, and schools. These occupancy grid maps are created with an initial size of 50m×50m. To further increase the number of training examples, we applied random rotation and offset to cropped areas of 34m×34m used as training examples. These occupancy grid maps are labeled as two categories, including 3210 normal occupancy grid maps and 3706 abnormal occupancy grid maps respectively. In the following experiments, 6916 occupancy grid maps are randomly divided into 4150 training examples, 2079 validation examples, and 690 test examples, and the ratio is roughly 6:3:1.

In OGMD, the occupancy grid maps are generated by the scan data of the robot laser sensor. For detail, each cell of occupancy grid map is obtained by the scan measurement data. In real collection scene, the occupancy grid maps are created by using either one scan or an accumulation of multiple sensor scans. The occupancy grid maps created by mobile robot are depicted in Fig. 2.

Through the scans of the laser sensor, all obstacles are displayed as a line segment in the occupancy grid map. The aggregation of the scans represents the feature parameters of obstacles in the indoor environment. As shown in Fig. 2 (a) and (b), the bright white region of the occupancy grid map indicates a flat and open space. In contrast, the dark line segment represents the obstacles in the indoor environment.

Each grid cell of occupancy grid map contains the distributed probability of obstacles and free-space in the actual indoor environment. Fig. 2 (a)–(d) show the created occupancy grid maps in different indoor scenarios. The criteria for evaluating the abnormal occupancy grid maps are as follows:

• •

  The region in occupancy grid map is inconsistent with the actual environment, such as the repeated mapping and the overlap region, like Fig. 2 (c).

• •

  The edges of obstacles on the map are unclear, like Fig. 2 (d).

• •

  The meaningful information is missing in the scanning region of mobile robot, like Fig. 2 (c) and (d).

In the cSE block, we recalibrate the inter-channel relationship for the feature response, which involves two steps, spatial squeezation and channel excitation. In the first step, for the feature map $I_{1}\in\mathbb{R}^{C\times H\times W}$, we use the Global Average Pooling (GAP) to squeeze the global information. Then a unique channel vector $V_{c}\in\mathbb{R}^{C\times 1\times 1}$of each channel is produced by the mean of GAP.

In the second step, to estimate attention from the $c$-th element of statistic channel vector, we use the Multi-layer Perceptron (MLP) which contains two Fully Connected (FC) layers, the Rectified Linear Units (ReLU) function, and the sigmoid function to capture channel-wise dependencies. The purpose of this MLP is to emphasize the channels with the meaningful information. In short, the output of channel attention is computed as:

where $W_{1}\in\mathbb{R}^{\bar{C}\times C},W_{2}\in\mathbb{R}^{C\times\bar{C}}$ are the weights of the $\mathrm{FC}$ layers, $\bar{C}=\frac{C}{r},r$ is the reduction ratio ($r$ is set to 16). $\delta$ refers to the Rectified Linear Units (ReLU) function, $\sigma$ refers to the sigmoid function.

In the sSE block, we use the convolutional layers to implement channel squeeze and spatial excitation. Here, it is assumed an alternative representation of the input tensor as $I_{2}$. Four standard convolution layers $W$ are concatenated to produce the spatial attention map $F_{s}(I_{2})=W*I_{2}$. The sigmoid function $\sigma\left(F_{s}(I_{2})\right)$ determines the importance of the specific location across the feature map. Like the previous block, this recalibration process indicates which locations are more meaningful during the training procedure. As a result, the output of spatial attention block can be expressed as

where $\sigma$ refers to the sigmoid function, $*$ denotes the convolution operation. The $f^{1\times 1}$ and $f^{3\times 3}$ represents a convolution operation with the filter size of $1\times 1$ and $3\times 3$, respectively.

For the occupancy grid map recognition task, the goal is to improve representation ability of network architecture by using attention mechanism, which can focus on meaningful features and suppress unnecessary ones. As illustrated in Fig. 3 csRSE module, the residual block is designed for aggregating hierarchical features at multiple levels and accelerating convergence. To contain sufficient spatial information of the intermediate layers, the sSE block are stacked after the cSE block for spatial feature extracting. The sSE block extracts the hierarchical features and improves the representation of interests via four convolution operation. All in all, cSE block and sSE block can be regarded as the extracting core, which can aggregate specific global context features of grid maps. These global features are weighted averaged from all areas via a specific attention map to each specific map areas. As attention maps are computed for each areas, the detail heat map of the OGMD dataset are displayed for the specific areas (as shown in Fig. 5).

Secondly, to extract multi-scale features, the convolution layers and csRSE modules are employed as the core of feature extraction in the proposed network architecture. Through the convolution operations, it can extract multi-scale features by different kernel size of convolution layer. After the convolution layer, we adopt a csRSE module to emphasize meaningful grid map features along two principal dimensions (channel and spatial axes). To achieve the abnormality in occupancy grid maps, we sequentially apply cSE block and sSE block, so that the csRSE modules can learn ‘what’ and ‘where’ to attend in the channel and spatial axes, respectively. As a result, the convolution layers and csRSE modules efficiently helps the information flow within the network by learning which map feature information to emphasize or suppress.

As illustrated in Fig. 3, there are four convolutional layers, four csRSE modules, one GAP, and one FC layer in the proposed csRSE-integrated network (ResNet32 + csRSE) architecture. At the beginning of proposed network, the input 3-channel occupancy grid map image is transformed into a 64-channel feature map. The ResNet in the proposed csRSE-integrated network is similar to [27] but use dilated convolutions [31]. The stacked block is defined as one consecutive convolution layer, one residual block, one cSE block, and one sSE block. The outputs of two attention blocks are fused to generate high-level contextual features, which will be used to guide the low-level contextual features extracted by the residual block. The residual block and two attention blocks are stacked for blending cross-channel information and spatial hierarchical features together. After each stacked block, the max-pooling is performed and each the filter size gets doubled, which is designed to double the spatial size of feature maps and halve channels. The bottom block is stacked with GAP, FC and softmax layer to compute probabilities of predicted category.

We implement occupancy grid map recognition experiments on our OGMD dataset using the PyTorch framework [32]. For training, the image resolution is converted to 224 × 224 and normalized with zero-mean normalization [33]. The entire network is trained end-to-end for 400 epochs by Stochastic Gradient Descent (SGD) with the momentum of 0.9 and a weight decay of 1e-4. The mini-batch size is set to 16. It takes about 8 hours for network to converge on an NVIDIA GTX 2080Ti GPU. We initialize the weights according to the method in [34] and use binary cross-entropy (BCE) loss function. The formulation of BCE loss is as follows:

where ${y}_{i}$ and $\hat{y_{i}}$ denote the actual label category and the probability of prediction, respectively.

We conduct experiments to validate the effectiveness of csRSE module by comparing with two popular attention modules (SE, CBAM). In experiments, we evaluate these methods on strong backbones, by replacing two different baseline network (ResNet20, ResNet32) and adding three attention modules (SE, CBAM, and csRSE). From Tab. I, all baseline network methods have achieved measurable competitive results through few calculation parameters. The networks with attention modules get the better accuracy than those without any attention methods.

Compared with the csRSE-integrated network (ResNet32 + csRSE) and CBAM-integrated network (ResNet32 + CBAM), we can find that the networks with two attention modules get the better performance than the methods with only one channel-wise attention block (ResNet32 + SE). Through the result of csRSE-integrated networks (ResNet32 + csRSE), it is seen that the attention network shows a clear advantage against the other as it gets the better accuracy (96.23%) on the test dataset. By comparison results on SE block and CBAM, the proposed csRSE attention modules capture the global context information as well as preserve more structural information.

Fig. 4 (a) shows the training and validation loss curve of the csRSE-integrated network. We can see that both the training loss and the validation loss are high at the beginning epochs. As the training epoch increases, both the losses remain quite stable. Fig. 4 (b) shows the curve of the training accuracy and validation accuracy based on the OGMD dataset. At the beginning, both the training accuracy and validation accuracy are getting stable as the learning rate gradually shrinks. Finally, we can get the best accuracy as the losses decrease.

Fig. 4 (c) shows the confusion matrix for our csRSE-integrated network on OGMD test dataset. The y-axis is the true labels and the x-axis is the predicted labels. Overall, the model has achieved a good prediction accuracy of 96.23%. Fig. 4 (d) shows the precision-recall curve of our OGMD test dataset. The y-axis shows the precision value and the x-axis shows the recall value. The experiment results show the actual prediction situation in the occupancy grid map recognition.

To intuitively verify the effectiveness of the csRSE module, we visualize the attention maps (heat maps) by the Grad-CAM [35] on OGMD test dataset. In recent year, Grad-CAM is a visualization method which shows the attended regions of spatial locations in convolutional layers. The Grad-CAM results show attended regions of image clearly. By observing the attended regions, the networks can extract the important feature of image for predicting a class.

In Fig. 5, we randomly test six abnormal images with different attention networks, including the baseline network (ResNet32), SE-integrated network (ResNet32 + SE), CBAM-integrated network (ResNet32 + CBAM) and our csRSE-integrated network. It can clearly see that the Grad-CAM masks cover attended regions that marked in the red frame represent the abnormal regions. It can be seen that our method is capable of accurately detecting both duplicate maps (e.g., the abnormal1, abnormal2, and abnormal6 input images) and overlapping maps (e.g., the abnormal3, abnormal4, and abnormal5 input images). This is mainly because the global contextual features extracted by the csRSE module can help the network better locate abnormal regions and detect abnormal occupancy grid map. The P denotes final softmax score of networks for the ground-truth class. The higher P has the better classification result, meaning that how this network is making good use of features.

As can be observed, our method can successfully eliminate such occupancy grid maps and detect the abnormal regions. This is mainly contributed by the proposed csRSE attention network, which exploits contextual information in target abnormal regions and provides more texture details for abnormal region localization. Based on the visualization, we conjecture that the proposed csRSE attention network can more effectively detect the abnormal areas from occupancy grid maps and store more global texture details for the occupancy grid map recognition tasks.