DT-LSD: Deformable Transformer-based Line Segment Detection

The Hough Transform (HT) [7] remains an important method for line detection. First, the Hough Transform applies Canny edge detection [1] to obtain line segment candidates. Candidate lines are represented in polar form. Here, we note that candidate lines are evaluated based on the overlapping number of pixels between the lines and the detected edges. Variations include the use of the Radon Transform and the Revoting Hough Transform.

In regions dominated by a large density of edges, the HT can generate a large number of false positives. Grompone von Gioi et al. proposed a linear-time Line Segment Detector (LSD) [19] to address this problem. LSD uses line-supported regions and line segment validation. The approach also reduced time complexity through the use of a pseudo-sorting algorithm based on gradient magnitudes. A fundamental advantage of traditional methods is that they do not require training for specific datasets.

Learning-based line segment detectors have shown significant improvements compared to traditional approaches. The methods include different approaches that focus on line junctions, attraction field maps (AFM), transformers, and combining traditional approaches with deep learning techniques.

The Holistically-Attracted Wireframe Parser (HAWP) [23] proposed a 4-dimensional attraction field map, and later HAWPv2 [24], a hybrid model of HAWP and self-supervision, was introduced. MLNet [25] and SACWP [27] incorporated cross-scale feature interaction on HAWP model. In [22, 28], the authors developed a method for detecting line junctions which were used to provide candidate line segments. Then, a classifier validated the candidates and produced the final set of predicted line segments. LSDNet [18] used a CNN model to generate an angle field and line mask that were used to detect line segments using the LSD method. HT-HAWP and HT-LCNN [14] added global geometric line priors through the Hough Transform in deep learning networks to address the lack of labeled data. However, the above methods requires of post-processing steps to produce the final output. In contrast, Line segment transformers (LETR) [21] remove post-processing steps by using an end-to-end transformer-based model that relies on a coarse-to-fine strategy with two encoder-decoder transformer blocks.

One crucial factor of LETR’s slow convergence is the global attention mechanism, which only does intra-scale feature processing. Plus, the global attention leads to very high computational complexity.

To understand the complexity requirements of LETR, we revisit the global attention mechanism. We define global attention using [18]:

where $Q,K,V,\text{ and }d$ represent the queries, keys, values and the hidden dimensions, respectively. For object and line segment detection, we define $K=V\in\mathbb{R}^{HW\times d}$ as the flattened form of the feature map $f\in\mathbb{R}^{H\times W\times d}$ where $H$ and $W$ are the height and width, respectively. In the encoder, we have $Q=K=V$ resulting in a complexity time of $O(H^{2}W^{2}d)$. Similarly, for the decoder, we have $Q\in\mathbb{R}^{N\times d}$ where $N$ is the number of queries producing a complexity time of $O(HWC^{2}+NHWC)$.

Based on our previous discussion, it is clear that the bottleneck of transformer-based models is the encoder, whose complexity time quadratically increases with respect to the spatial size of the feature map. To address this issue, Zhu et al. [29] proposed the deformable attention mechanism, inspired by deformable convolution [3]. Unlike global attention, the deformable attention module only attends to a fixed number $k$ of keys for each query (see Fig. 2 from [29] for a visual representation).

Given an input feature map ${f}\in\mathbb{R}^{H\times W\times d}$, let $q$ be the index of a query element with content feature $z_{q}$ and a 2d-reference point $p_{q}$, the deformable attention for one attention head²²2The multi-head deformable attention equation is in page 5 section 4.1 in [29] is mathematically defined as

where $i$ indexes the sampling keys and $k$ is the total sampling keys number ($k\ll HW$). The $i^{th}$ sampling key is computed as $p_{q}+\Delta p_{qi}$ where $\Delta p_{qi}$ is the sampling offset. $A_{qi}$ is the $i^{th}$ row of the attention weight $A_{q}\in\mathbb{R}^{k\times d}$. The weights of $A_{qi}$ satisfy $\sum_{i=1}^{k}A_{qi}=1$.

Comparing Eq. 1 to Eq. 2, $\mathrm{softmax}\big{(}QK^{T}/\sqrt{d}\big{)}$ is replaced by $A_{q}$, and $V$ by $x(p_{q}+\Delta p_{qi})$. So, the complexity time for the deformable encoder is $O(HWd^{2})$, which has a linear complexity. For the deformable decoder, the complexity time is $O(kNd^{2})$ where $N$ is the total number of queries, and the spatial dimensions of $x$ are irrelevant.

Apart from reducing the memory and time complexities, deformable attention has a variation called multi-scale deformable attention ($\mathrm{MSDeformAttn}(z_{q},\hat{p}_{q},\{f_{l}\}_{l=1}^{L})$) that allows cross-scale feature interaction and is defined as

where $\{f_{l}\}_{l=1}^{L}$ is a set of $L$ multi-scale feature maps, and $f_{l}\in\mathbb{R}^{H_{l}\times W_{l}\times d}$. The normalized 2d coordinates $\hat{p}_{q}$ has its values lying in the range of $[0,1]$, and is re-scaled to dimensions of the $l^{\text{th}}$-level feature map by the function $\phi_{l}$. Like Eq. 2, the attention weight $A_{lqi}$ satisfies $\sum_{l=1}^{L}\sum_{i=1}^{k}A_{lqi}=1$.

We use the SwinL [16] as a backbone for our deformable encoder-decoder transformer model. For the deformable transformer, we followed the recommendation of DINO [26]. We used 4 sampling offsets for the encoder and decoder, 900 queries to predict line segments, 6 stacked-encoder layers, and 6 stacked-decoder layers. We summarize the DT-LSD architecture parameters and training parameters in Table 1. We train DT-LSD on a single Nvidia RTX A5500 GPU with a batch size of 2.

Our comparisons are based on the number of epochs and the sAP metric as summarized in Table 3. For a fair comparison, we train all the DINO⁵⁵5We use DINO-ResNet50-4scale from [26]. variations using ResNet50 [8] as the backbone.

Compared to the 500 epochs required for the vanilla DETR, DINO converges at just 36 epochs. Furthermore, plain DINO results in a maximum accuracy drop of 0.6, while our additions boost performance significantly (e.g., compare 66.3 and 68.8 versus 53.8 and 57.2).

The performance improvement against vanilla DETR is because DINO uses the MultiScale Deformable Attention mechanism described in Eq. 3, which promotes the cross- and intra-scale interaction. However, DINO has a lower performance than LETR. Fig. 4 shows that the contrastive denoising (CDN) technique from DINO does not work for line segments because applying CDN to line segments results in scaling and translating both positive and negative queries. Therefore, these two operations lead to the model accepting non-line-segment candidates as potential candidates. We test our idea by removing the box scaling. We noticed an improvement of around 7 points for sAP¹⁰. We gain 3 extra points for sAP¹⁰ by adding the line scaling technique, reaching a score of $63.4$. By combining the line scaling and the line rotation (our line contrastive denoising technique), we obtain the maximum of $66.3$ in sAP¹⁰.

Based on our experiments, we conclude that our enhancing method and training technique are effective since DINO with LCDN outperforms LETR using a lower-parameter backbone and fewer training epochs. We want to highlight that the input size of the image for all DINO variations was $512\times 512$ for both training and testing. On the other hand, vanilla DETR and LETR resize the images following the procedure described in Sec. 4.1 for training, and they resize the image with the shortest side of at least 1100 pixels for testing.

An important element of our model is the feature maps generated by the backbone. Here, we evaluate the effectiveness of different feature maps and backbones. We report the results in Table 4. All models are trained using 24 epochs. Adding the S1 feature map produces more precise line segments while slowing down our inference time (measured in the number of frames per second (FPS)). Using SwinL as the backbone gives the best results, but slows down the inference speed. For example, the configuration SwinL with 5 feature maps achieves a score of $69.3$ in the sAP¹⁰ at 10.5 FPS.

Most transformer-based algorithms use upsampling techniques to improve their performance. To evaluate the effects of upsampling an image, we train and test our model at different resolutions for 24 epochs. As documented in Table 5, upsampling benefits both CNN- and transformer-based models. First, we train DT-LSD following popular CNN-based methods by resizing the original image to 512$\times$512. For DT-LSD 512, we obtain the fastest inference time at 13.6 FPS among all DT-LSD variations.

Motivated by DETR-based models [2, 29, 15, 10, 21, 26, 20], we also apply scale augmentation consisting of resizing the input image so that the shortest side is a minimum of 480 pixels and a maximum of 800 pixels, while the longest side is a maximum of 1333 pixels. Here, we choose five different testing sizes, 1) 480, the minimum size used for training, 2), 512, the size used for CNN-based line segment detectors, 3) 640, the size use for YOLO detectors, 4) 800, the maximum size used for training, and 5) 1100, LETR’s [21] testing size.

We note that this scaling technique improved our results over 512$\times$512. For example, DT-LSD 480^† produces better results than DT-LSD 512. As the testing size increases, the sAP metric improves while reducing inference speed (as measured by FPS). As a balance between speed and precision, we choose DT-LSD 640^† because it increases sAP¹⁰ and sAP¹⁵ by around 2 points, while its FPS is only by 2.9 fps less than DT-LSD 520^†. We did not choose DT-LSD 800^† because the 0.4 improvement in sAP does not justify a drop of 2.5 fps in inference performance.

We report the results for training DT-LSD with 12, 24, and 36 epochs in Table 6. DT-LSD gets to competitive performance after training with just 12 epochs. At 12 epochs, DT-LSD achieves a sAP¹⁰ of 68.4. There is very little difference between 24 and 36 epochs. At 36 epochs, we get a minor increase of 0.3, 0.2, and 0.1 in the sAP⁵, sAP¹⁰, and sAP¹⁵, respectively.

We report results based on pre-training on different datasets in Table 7. For our experiments, we use 24 epochs. In our first example, the backbone was pre-trained with the ImageNet-22k dataset [5]. In our second example, the DINO was pre-trained using the COCO object detection dataset [13]. From the results, it is clear that it is essential to pre-train the entire network and not just the backbone.