Impact of ultrasound image reconstruction method on breast lesion classification with neural transfer learning

Ultrasound (US) imaging is widely used for breast lesion detection and differentiation. In comparison to other imaging modalities, such as magnetic resonance imaging, mammography or thomosynthesis, US is safe, accurate low cost and highly universalize. These features make the US imaging suitable for breast cancer screening, lesion monitoring and preoperative staging. However, US imaging is operator dependent. Data acquisition needs to be carried out by radiologists or physicians who know how to efficiently operate an ultrasound scanner. The operator has to locate the lesion within the examined breast and properly record the US images. Moreover, interpretation of the US images is not straightforward, but requires deep knowledge of characteristic image features connected to specific lesion types.

So far, various computer-aided diagnosis (CADx) systems have been proposed to support the radiologists and to improve differentiation of malignant and benign breast lesions [1, 2]. These systems usually use hand-crafted features for classification. Although good classification performance was reported [3], with the current rise of deep learning (DL) methods CADx systems with automatic feature extraction are gaining momentum for breast lesion differentiation [4, 5, 6, 7]. DL methods commonly apply convolutional neural networks to process US images in order to provide the probability that the examined breast lesion is malignant. However, deep neural networks require large datasets for efficient training. This requirement is rarely met in the case of US imaging. Due to this issue, researchers commonly rely on transfer learning methods for CADx system development [8]. The aim of the transfer learning is to employ a DL model pre-trained on an existing, large dataset from a different domain to address the problem of interest. This gives the advantage that instead of having to handcraft features, the pre-trained DL model is used to extract the features for classification. Performance of the transfer learning relies on the similarity of the medical images to the images that were used to develop the pre-trained DL model.

In this paper we investigate the impact of US image reconstruction algorithm on transfer learning based classification of breast lesions. Our study is motivated by several papers that report unstable properties of DL models. Notably because even a small modification of image intensities may cause a well performing DL model to drastically change the output [9, 10]. The visibility of tissue in US images is related to applied image reconstruction techniques. Modification of the reconstruction algorithm properties results in change of US image intensities, which may have a negative impact on DL based classification. We investigate this issue using raw US data acquired from breast lesions. First, raw data are used to reconstruct US images. Next, a classifier is developed using transfer learning. The classification performance is evaluated on two test sets. The first one contains images reconstructed in the same way as in the training set. The second set contains images reconstructed differently. Several experiments are performed. Results and a discussion are presented at the end of the paper.

Tables I shows the classification performance of the classifiers developed using InceptionV3 model. The classifiers were trained and evaluated using differently reconstructed images. Table II shows similar results obtained for the VGG19 model. Tables depict that the approach utilizing InceptionV3 model achieves overall better classification performance. Moreover, the higher is the difference in the threshold levels between the training and the test set, the lower is the classification performance. In the worst case, the AUC value decreased by around 0.1 which corresponded to a drop in accuracy by about 10%. The Bland-Altman plots in Fig. 4 illustrate the agreement between the classifiers developed using images reconstructed at compression level of 40 dB and 60 dB.

As we can see in Tables I and II, using $\text{Train}_{\text{ALL}}$ for training leads to a more robust classification. In both cases, the proposed augmentation technique improved the results, the classification performance across all test sets increased and was less sensitive to modification of the compression threshold level.

Table III depicts the classification performance of the classifiers developed using $\text{Train}_{\text{ALL}}$ and evaluated on $\text{Test}_{\text{ALL}}$. In the case of the InceptionV3 model, we obtained the best classification performance with the AUC value equal to 0.857. The AUC value for the approach utilizing the VGG19 model was equal to 0.822. Fig. 5 shows the ROC curves obtained for these best performing classifiers.

According to Tables I and II, the classification performance decreases if the test set contains images reconstructed differently than in the training set. Modification of US image intensities affects feature extraction and has negative impact on the breast lesion classification. Both neural models could not provide features that would be invariant to small modifications of input images’ intensities. Accurately recognized images were mislabeled after the modification. Moreover, the Bland-Altman plots in Fig. 4 illustrate the disagreement between the classifiers developed using 40dB and 60. The difference in the threshold levels was the highest in this case and caused the highest disagreement, see Table I and II. The classifiers developed using $\text{Train}_{40\text{dB}}$ achieved better classification performance. This might be explained by the fact that high image compression is related to edge enhancement, see Fig. 2. As reported in several studies, more irregular lesion boundary is expected in the case of malignant lesions [20, 21]. Moreover, morphological features are considered to be the most effective handcrafted features for breast lesion classification [3].

Tables I and II depicts that using $\text{Train}_{\text{ALL}}$ leads to a more robust classification. The proposed technique of data augmentation that employs images reconstructed using different threshold levels leads to better classification performance. In some sense, the proposed method is similar to Gaussian blurring and other techniques that modify image intensities. However, it is specific for the US imaging.

Our study shows the usefulness of the transfer learning for breast lesion classification. According to Table III, the best overall classification performance was obtained for the classifier developed using $\text{Train}_{\text{ALL}}$ and the InceptionV3 model, with AUC of 0.857. In the case of the VGG19 neural network, the AUC value was equal to 0.822. In [4] the VGG19 neural network and the same transfer learning method as ours was used to classify breast lesions in ultrasound. The authors reported a little bit higher AUC value of 0.85. In [5] a specific approach to transfer learning was applied, which included modification of the InceptionV3 architecture and the ImageNet dataset. The authors used an ensemble of DL models for classification and reported high AUC value of 0.9601. In our case, we used the InceptionV3 model for transfer learning in a more standard following the approach proposed in [4] what makes direct comparison of the results difficult.

Our results depict several issues related to neural transfer learning. First of all, the image reconstruction procedures implemented in medical scanners should be taken into account during CADx system development. It is important to know how the medical images were acquired and reconstructed. In our study we used a unique dataset including raw ultrasound signals acquired with a research US scanner. However, usually little is known about the image reconstruction algorithms implemented in US scanners. Usually the researchers involved in CADx systems development agree that a particular system might not test well on data acquired in another medical center using different scanners and protocols. Our study clearly shows that this issue might be related also to a CADx system developed using data recorded in the same medical center. Another problem is connected to US physics. Due to wave attenuation, US echoes coming from larger depths have lower intensities and therefore are differently mapped to image intensity levels. The same object may look slightly differently depending on its depth. Moreover, in this study we examined only the impact of compression threshold level on classification, but other factors exist that can decrease the performance. For example, the texture of the image might depend on applied beamforming technique [22] or imaging frequency [23]. The proposed method of data augmentation method is general and can be applied to all kinds of US images to improve the CADx system performance with deep learning. However, the proposed method requires raw ultrasound data what limits its usefulness. Nevertheless, our study suggests how to improve the classification performance and the robustness. First, several B-mode images of the same tissue should be acquired with different scanner settings. Second possibility is to always use the same image reconstruction algorithms and scanner settings. This requirement, however, might be difficult to fulfill in reality. One more possibility is to use quantitative ultrasound techniques. These method can be used to create scanner independent parametric maps that illustrate various physical properties of tissue [12, 24].

In this paper we investigated the impact of US image reconstruction algorithm on breast lesion classification with transfer learning. Modification of the reconstruction algorithm leads to decrease of classification performance. To minimize this effect, we proposed a method of data augmentation. We presented that by using differently reconstructed US images for training a better classification performance can be obtained. We believe that our approach is general and can be used to improve the performance of any CADx system that utilizes US images and convolutional neural networks.

This work was partially supported by the National Science Center (Poland), Grant Number UMO-2014/13/B/ST7/01271.

The authors do not have any conflict of interest.