Weakly Supervised Segmentation Framework for Thyroid Nodule Based on High-confidence Labels and High-rationality Losses

To learn the location information as described in (1), the alignment loss consists of two components: 1) alignment of projected segmentation results with bounding box labels, and 2) topological continuity loss in high-confidence foreground regions.

For a predicted result $m_{i}$ and bounding box label $G_{box}$ derived from the points annotations, we define the projection of each box label mask as:

where $p_{x}$ and $p_{y}$ denote projections derived from the box label $G_{box}$ onto the x-axis and y-axis, respectively, $\tilde{p_{x}}$ and $\tilde{p_{y}}$ are the projections of predicted result $m_{i}$ onto the x and y axixes.

The first part $L_{ali-proj}$ of alignment loss between the predicted mask and the box label is then computed as follows:

This projection loss provides region localization constraints with high feasibility and efficiency.

To prevent extreme predictions on coordinate projections, we introduce topological continuity loss as the second component of the alignment loss. Given the high-confidence foreground labels $G_{f}$ and the prediction result ${m_{i}}$, target regions in the high-confidence foreground are calculated as:

where $m_{i}^{\prime}$ represents the predicted areas in high-confidence foreground areas. The topological continuity loss $L_{ali-topo}$ is defined as:

This loss ensures topological continuity within predicted foreground regions covered by the high-confidence foreground. Despite variations in the shapes of thyroid nodules, pixels within these high-confidence regions achieve classification accuracy of over $99.66\%$. This specific loss function is exclusively utilized to identify foreground regions to prevent local optima in coordinate axis-based optimization.

By combining loss $L_{ali-proj}$ and loss $L_{ali-topo}$, we obtain the final alignment loss $L_{align}$ as follows:

This loss allows weak supervision labels to constrain segmentation localization effectively while avoiding over-fitting to a single shape.

Due to the absence of ground truth masks in weak supervision, as outlined in 3.2, we propose to learn the segmentation shape by refining the feature representation of foreground and background regions to achieve (2).

The proposed contrastive loss function aims to learn region-level discriminative feature representations from high-confidence foreground areas $G_{f}$ and background areas $G_{b}$, the generic contrastive loss with a sampling scale size $k\times k$, denoted as $L_{con}^{k}$, which is formally defined as:

where $Q_{p}$ and $Q_{n}$ represent the sets of positive and negative feature embedding queues, respectively. $q_{k}^{+}$ denotes a foreground feature sample of size $k\times k$, while $q_{k}^{-}$ represents corresponding background feature samples. The anchor feature queue $q$ is selected from high-confidence foreground regions $G_{f}$. The temperature parameter $\tau>0$ controls the slope of the loss function and its smoothness. Empirically, we evaluate the contrastive loss at scale sizes $k=3$ and $k=1$, with $\tau=0.07$.

The contrastive loss is designed to minimize the distance between intra-class features (i.e., either both foregrounds or both backgrounds) in the embedding space and maximize the distance between inter-class features (i.e., foreground and background). Through this learning mechanism, the obtained feature representations effectively sharpen the classification boundaries between foreground and background regions. This enhancement directly improves the network’s ability to distinguish between these regions, ultimately refining its predicted prototypes for both categories to align closely with reference standards.

To learn more precise segmentation shapes, building on the segmentation objective defined as (3), we extend the shape constraint to edge-level by introducing a prototype correlation loss, as illustrated in Fig. 2. This loss comprises two components: 1) complementary consistency between feature correlations refer to high-confidence foreground prototype and background prototype, and 2) consistency between network segmentation results and fused correlation segmentation results.

Given the high-confidence foreground and background labels $G_{f}$ and $G_{b}$, we first extract the corresponding region features $P_{f}$ and $P_{b}$. Global pooling is then applied to reduce the dimensionality of these features to $C\times 1\times 1$ where C means channels to obtain the prototype features. Using metric learning, we evaluate the correlation response of each position in the feature map $F$ with respect to the foreground prototypes and background prototypes, obtaining the foreground correlation response $C_{f}$ and the background correlation response $C_{b}$ as follows:

Since foreground and background are mutually exclusive in segmentation tasks, the foreground correlation response $\hat{C_{f}}$ derived from the background prototype $C_{b}$ as follows:

The correlation map represents the similarity between image features and each prototype. The first component $L_{cor-fe}$ of the prototype correlation loss reflects the complementary consistency between the feature correlations referring to high-confidence foreground and background prototypes, which is defined as follows:

The fused predictions $C_{i}$ that should exhibit the same region distribution as that of the segmentation branch are obtained by balancing the foreground correlation maps $C_{f}$ based on the foreground and those $\hat{C_{f}}$ on the background prototypes. Therefore, the second component $L_{cor-seg}$ of the correlation loss measures the consistency between the fused correlation map and the segmentation prediction is defined as:

The total prototype correlation loss $L_{corr}$ is then calculated as follows:

By considering the complementary consistency of foreground and background prototype correlation, the algorithm obtained segmentation edges with low uncertainty. By directly propagating the fused segmentation edges to the predicted results, we achieved explicit shape learning, effectively addressing the challenge of refining the edges of the nodule region.

By combining the losses of learning location information and shape information mentioned above, the overall loss function during the training process can be expressed as:

where $L_{all}$ indicates the overall loss, $L_{align}$ represents alignment loss. is $L_{con}$ represents Contrastive Loss and $L_{corr}$ is prototype correlation loss, $\lambda$ and $\beta$ are weighted parameter.

To evaluate the effectiveness of the proposed segmentation framework, we conduct ablation and comparative experiments on two publicly available thyroid ultrasound datasets: TN3K[29] and DDTI[30]. The TN3K dataset consists of 3,494 high-resolution thyroid nodule images, following a standardized clinical split protocol with 2,879 images designated for training and 614 for testing. For the DDTI dataset, which contains 637 thyroid ultrasound scans, the data is randomly split into training and testing sets at a 4:1 ratio. Additionally, 10-fold cross-validation was applied to both datasets to address the challenge of limited data while ensuring statistical reliability.

Notably, in the TN3K and DDTI datasets, the ground truth annotations of thyroid nodules in ultrasound images for metric calculation were performed by experienced radiologists, while point annotations were carried out by less-experienced senior medical students to simulate a real-world weakly supervised learning scenario.

We performed a quantitative comparison using four common segmentation evaluation metrics: Mean Intersection over Union (mIoU) [53], Dice Similarity Coefficient (DSC) [53, 54, 55], Hausdorff Distance (HD) [55], and Prediction Precision (Pr) [53, 54], which evaluate the segmentation’s overall accuracy, precision of segmented regions, boundary matching, and the reliability of predicted positives, respectively.

For training the network, we set a learning rate of 0.0001, a batch size of 16, and trained for 100 epochs with images resized to $256\times 256$. The network was optimized using the Adam optimizer. For comparative experiments, we adhered to the parameter configurations outlined in their respective papers. All experiments were carried out using PyTorch on an Nvidia 3090 GPU equipped with 24GB of memory.

The segmentation performance of our proposed method was compared against the state-of-art weakly supervised methods SCRF[14], UNCRF[15], BoxInst[16], WSDAC[20], S2ME[22], IDMPS[21], and fully supervised methods U-Net[8], Dense-UNet[9] and Cenet[12]. The quantitative results of TN3K and DDTI are shown in Table. 2. The qualitative results of TN3K and DDTI are shown in Fig. 3 and Fig.4, respectively.

As shown in Fig. 3, SCRF and UNCRF tended to produce box-like segmentation with significant over-segmentation. WSDAC and S2ME frequently under-segmented thyroid nodules in images with multiple nodules or irregular nodule shapes. While BoxInst and IDMPS outperformed other existing methods in high-contrast scenarios, such as the example in the third row, they still struggled with under-segmentation of incomplete nodules located at image boundaries. In contrast, our proposed method achieved more consistent segmentation regions and delicate segmentation edges, while exhibiting even less over- and under-segmentation than fully supervised networks using the same backbone. Quantitative results in Table 2 further support that our framework using the U-Net backbone outperformed state-of-the-art weakly-supervised methods. Specifically, it achieved an average mIOU of 69.30%, a Hausdorff distance (HD) of 5.01 mm, a DSC of 79.10%, and a precision (Pr) of 80.64%. These results were even better than the fully supervised U-Net, which achieved 64.76% mIOU, 5.83 mm HD, 75.13% DSC, and 77.66% Pr.

As illustrated in Fig. 4, fully supervised networks exhibited substantial over-segmentation in images where the foreground and background tissues are similar. Weakly supervised algorithms, such as SCRF, UNCRF, and S2ME, struggled with severe over-segmentation of small targets, as well as in images with similar foreground and background tissues. Both BoxInst and WSDAC encountered significant challenges with over- and under-segmentation when processing such images. Notably, BoxInst delivered superior segmentation results with minimal over-segmentation in simpler background scenarios, while WSDAC tended to exhibit more under-segmentation in these cases. IDMPS outperformed other weakly supervised algorithms by reducing over-segmentation in small nodules, however, it still faced substantial issues with under-segmentation and excessive over-segmentation in more complex cases. In contrast, our algorithm not only reduced over-segmentation but also illustrated enhanced shape adaptation, surpassing fully supervised methods. Additionally, by incorporating a more efficient feature extraction backbone such as CENet, we achieved better segmentation precision, with accurate shape delineation and fine edge fitting. The quantitative comparison presented in Table 2 further substantiates these qualitative observations.

The effectiveness of the alignment loss for spatial learning is demonstrated through quantitative segmentation results on two datasets, summarized in Table 4. Compared to Model A, Model B showed nearly 1.5% improvements in mIoU and DSC across both datasets, indicating that decoupling weakly supervised tasks and incorporating positional constraints improves segmentation accuracy and region consistency. While precision also increased significantly, the Hausdorff Distance (HD) slightly increased by 0.18 for TN3K and 0.16 for DDTI, because the alignment loss focused primarily on region localization and required complementary shape loss for shape refinement.

Fig. 5 illustrated Model B reduced under-segmentation and over-segmentation compared to Model A, achieving more accurate location capturing and shape fitting. These observations further illustrate the network’s improved ability to accurately capture target location without misleading the segmentation shapes with the inclusion of alignment loss.

Model C combines alignment losses with contrastive loss, where the weight of the prototype correlation loss is controlled by parameter $\lambda$. Experimental results demonstrate that Model C achieved its best comprehensive performance when $\lambda$ was set to 0.8, as shown in Fig. 6.

Table 4 shows significant improvements in Model C over Model B across various metrics. Specifically, Model C achieved improvements in mIoU and DSC of more than 5.5% on TN3K and over 5% on DDTI. Similarly, there is a notable increasement in Precision for both datasets. Additionally, the Hausdorff distance decreased by 0.56 mm on TN3K and 0.76 mm on DDTI. These improvements indicate that contrastive loss contributes to more accurate and refined segmentation. The segmentation feature heatmaps shown in Fig. 5 visually support the quantitative findings.

Furthermore, by integrating contrastive loss with other losses, Model E also showed more precise segmentation regions than Model D without contrastive loss, reducing over-segmentation and under-segmentation. The inclusion of contrastive loss leads to predicted regions that closely align with the ground truth, demonstrating its effectiveness in enhancing feature learning of segmentation shapes.

Model D combines alignment losses with prototype correlation loss, where the weight of the prototype correlation loss is controlled by parameter $\beta$. Experimental results illustrate that Model D achieved its best comprehensive performance when $\beta$ was set to 1.0 on the TN3K dataset and 0.5 on the DDTI dataset, as shown in Fig. 6.

To validate the benefits of adding prototype correlation loss as a shape constraint, we compared Model D with Model B, which only using alignment loss. The experimental results show that Model D achieved nearly 7% improvement in mIoU on the TN3K dataset and 6% improvement on the DDTI dataset compared to Model B, and the Hausdorff Distance was significantly reduced, with a decrease of 0.71 on the TN3K dataset and 0.67 on the DDTI dataset. These results are supported by qualitative results shown in Figure 5, Model D generated more accurate segmentation of shapes, with lower uncertainty boundaries in feature heatmaps.

When comparing Model E and Model C, the integration of prototype correlation loss provided superior segmentation regions and edges. This resulted in clear boundary delineation and segmentation outcomes closely aligning with ground truth annotations as shown in Fig. 6. Although the effects of combined losses were not linear, this approach leaded to 0.98% to 1.69% improvement in mIoU, positioning Model E as the best-performing model among all variants. These observations confirm that prototype correlation loss effectively constrains shape information, enhancing segmentation accuracy.