Edge-competing Pathological Liver Vessel Segmentation with Limited Labels

Zunlei Feng^1,5#, Zhonghua Wang^1#, Xinchao Wang², Xiuming Zhang¹, Lechao Cheng³, Jie Lei⁴, Yuexuan Wang¹, Mingli Song^1,5
Corresponding author. ^#Both the authors have equal contribution to this work.

Abstract

The microvascular invasion (MVI) is a major prognostic factor in hepatocellular carcinoma, which is one of the malignant tumors with the highest mortality rate. The diagnosis of MVI needs discovering the vessels that contain hepatocellular carcinoma cells and counting their number in each vessel, which depends heavily on experiences of the doctor, is largely subjective and time-consuming. However, there is no algorithm as yet tailored for the MVI detection from pathological images. This paper collects the first pathological liver image dataset containing $522$ whole slide images with labels of vessels, MVI, and hepatocellular carcinoma grades. The first and essential step for the automatic diagnosis of MVI is the accurate segmentation of vessels. The unique characteristics of pathological liver images, such as super-large size, multi-scale vessel, and blurred vessel edges, make the accurate vessel segmentation challenging. Based on the collected dataset, we propose an Edge-competing Vessel Segmentation Network (EVS-Net), which contains a segmentation network and two edge segmentation discriminators. The segmentation network, combined with an edge-aware self-supervision mechanism, is devised to conduct vessel segmentation with limited labeled patches. Meanwhile, two discriminators are introduced to distinguish whether the segmented vessel and background contain residual features in an adversarial manner. In the training stage, two discriminators are devised to compete for the predicted position of edges. Exhaustive experiments demonstrate that, with only limited labeled patches, EVS-Net achieves a close performance of fully supervised methods, which provides a convenient tool for the pathological liver vessel segmentation. Code is publicly available at https://github.com/zju-vipa/EVS-Net.

Introduction

Liver cancer is one of the malignant tumors with the highest mortality rate, which witnesses an increasing number of cases in recent years (D, Suna, and Boyacioglu 2017). Hepatocellular carcinoma represents the most frequent primary liver tumor, taking up about $90\%$ of all liver cancer cases (Perumpail et al. 2017). The microvascular invasion (MVI) is a major prognostic factor in hepatocellular carcinoma, which significantly affects the choice of treatment and the overall survival of patients (Sumie et al. 2008; Rodriguezperalvarez et al. 2013). However, the current MVI detection heavily depends on the experiences of doctors, which is a largely subjective and time-consuming to diagnose. Existing machine learning algorithms for predicting MVI mainly focused on computed tomography, magnetic resonance imaging, and positron emission tomography (Cuccurullo et al. 2018), yet ignore pathological images. Even to data, there is no learning-based method yet for automatically detecting MVI in pathological liver images.

Refer to caption — Figure 1: The overview of the edge-competing mechanism. The Over-Segmentation Discriminator (OS-D) and Under-Segmentation Discriminator (US-D) are devised for distinguishing whether the segmented background and segmented vessel contain residual features. The over-segmented and under-segmented patches are generated with different erosion and dilation operations following binarization processing. Only when the predicted mask depicts the precise segmentation will the true criterion of the two discriminators be satisfied simultaneously for an input patch.

As deep learning approaches have achieved promising results in the medical image analysis area (Ker et al. 2018; Altaf et al. 2019), we expect they have the potential to be deployed in the pathological liver vessel segmentation. There are many deep learning based segmentation methods for general natural images (Minaee et al. 2020) and medical images (Ronneberger, Fischer, and Brox 2015; Xue et al. 2018; Dai et al. 2019). Due to the specific attributes of different images, however, existing methods cannot be directly applied to the pathological liver vessel segmentation. Moreover, the success of deep learning are highly based on a large number of annotated samples; yet for pathological liver vessel segmentation, there was not a dedicated dataset available, as it is extremely tedious, time-consuming, and expensive to annotate those vessels accurately.

In this paper, we first collect a pathological liver image dataset containing $522$ whole slide images with labels of vessels, MVI, and hepatocellular carcinoma grades. It can readily serve as a benchmark for the research on the analysis of pathological liver images. Fig. 2 shows some cut patches and the whole slide pathological image, which has a size of $200,000*100,000$ . For the patches, the coarse edges of most vessels can be obtained by binarization processing. For the vessels containing hepatocellular carcinoma cells, binarization processing will fail to segment those patches. However, combining with the erosion and dilation operation, binarization processing may easily produce some over-segmented or under-segmented vessels.

Based on such findings, we devise an edge-competing mechanism to assist the vessel segmentation, for which the overview is shown in Fig. 1. With the easily obtained over-segmented patches as the positive training samples, the over-segmentation discriminator is devised for distinguishing whether the segmented background contains residual features of vessels. Similarly, the under-segmentation discriminator is devised for distinguishing whether the segmented vessels contain residual features of the background. For an input patch, only when the predicted mask depicts the perfect segmentation will the true criterion of the two discriminators be satisfied simultaneously. It will effectively relieve the dependence of annotation by incorporating the edge-competing mechanism into the vessel segmentation framework.

Besides, from the whole slide image in Fig. 2, we can see that the pathological liver images have some unique attributes, such as super-large sample size, multi-scale vessel, blurred vessel edge, which is challenging for the accurate vessel segmentation. Given such specific characteristics of pathological liver images, we propose the Edge-competing Vessel Segmentation Network (EVS-Net), which is shown in Fig. 2. For the massive unlabeled patches, the doctor only needs to label limited patches as guidance. For the rest of the unlabeled patches, the binarization processing combined with erosion and dilation will generate many over-segmented and under-segmented patches.

As shown in Fig. 2, the proposed EVS-Net is composed of a Pathological Vessel Segmentation Network (PVSN), an Over-Segmentation Discriminator (OS-D), and an Under-Segmentation Discriminator (US-D). In the training stage, the limited labeled patches will supervise the PVSN to gain initial segmentation ability for the pathological vessel patches. The edge-competing mechanism is then incorporated into EVS-Net for assisting the training of the segmentation network in an adversarial manner.

To further strengthen consistency of the segmentation on massive unlabeled patches, an edge-aware self-supervision module is introduced to generate the self-supervision information for training PVSN. The core idea of edge-aware self-supervision is that, for the same image and segmentation network, the predicted mask of the transformed image should be equal to the transformed mask predicted by the network with the original image as input.

Our contribution can be summarized as follows. Firstly, we collect a pathological liver image dataset that contains $522$ whole slide images with labels of vessels, MVI, and hepatocellular carcinoma grades. It can readily serve as a benchmark for the analysis of hepatocellular carcinoma. Next, the proposed segmentation network combines an edge-aware self-supervision module and two segmentation discriminators to efficiently solve the challenges in the pathological vessel segmentation. Furthermore, the edge-competing mechanism between two discriminators is firstly proposed for learning accurate segmentation with over-segmented and under-segmented patches, which effectively relieves the dependence on a large number of annotations. Finally, with limited labels, the EVS-Net achieves results on par with fully supervised ones, providing a handy tool for the pathological liver vessel segmentation.

Related Works

To our knowledge, there are no vessel segmentation methods for pathological images until now. Existing methods, which are related to pathological images, mainly focus on the pathological grading of various carcinomas (Yang et al. 2019; Jiao et al. 2020; Liao et al. 2020). Here, we give the related segmentation works for general natural images and medical images from two perspectives.

Edge-aware Segmentation. For edge-aware semantic segmentation, the commonly adopted framework is a two-branch network that simultaneously predicts segmentation maps and edges, where different constraints (Bertasius, Shi, and Torresani 2016; Chen et al. 2016; Cheng et al. 2017; Yu et al. 2018; Yin et al. 2018; Ye et al. 2019; Takikawa et al. 2019) are devised for strengthening the segmentation results with the predicted edges. Unlike predicting the edge directly, Hayder, He, and Salzmann (2017) proposed predicting pixels’ distance to the edge of object and post-processed the distance map into the final segmented results. Khoreva et al. (2017) proposed edge-aware filtering to improve object delineation. Zhang et al. (2017) proposed a local edge refinement network to learn the position-adaptive propagation coefficients so that local contextual information from neighbors can be optimally captured for refining object edges. Qin et al. (2019) adopted the patch-level SSIM loss (Wang, Simoncelli, and Bovik 2003) to assign higher weights to the edges. Peng et al. (2017) proposed an edge refinement block to improve the localization performance near the object edges. Unlike the above methods, two edge-competing discriminators are devised for distinguishing whether the segmented vessels and background contain residual features. The over-segmented and under-segmented patches are adopted to train the two discriminators, focusing on the outer and inner area of edges.

GAN based Segmentation contains two categories: composition fidelity based methods (Ostyakov et al. 2018; Remez, Huang, and Brown 2018; Chen, Artieres, and Denoyer 2019; Qiu et al. 2020) and mask distribution-based methods (Luc et al. 2016; Han and Yin 2017; Arbelle and Raviv 2018; Hung et al. 2018; Ye et al. 2020). The composition fidelity based methods (Ostyakov et al. 2018; Remez, Huang, and Brown 2018; Chen, Artieres, and Denoyer 2019) adopted the discriminator to distinguish the fidelity of natural images and the composited images, which is composition of the segmented objects and some background images. For the mask distribution-based methods, Luc et al. (Luc et al. 2016) adopted the adversarial optimization between segmented results and GT mask to train the segmentation network, which is the first GAN based semantic segmentation network. Next, some researchers (Han and Yin 2017; Arbelle and Raviv 2018; Xue et al. 2018) applied the same adversarial framework on the medical image segmentation task. Furthermore, Souly et al. (Souly, Spampinato, and Shah 2017) and Hung et al. (Hung et al. 2018) adopted the adversarial strategy to train the discriminator output class confidence maps. Unlike the above existing GAN-based methods, we devised two-discriminator architecture, where the two discriminators will compete the vessel edge’s position of the unlabeled patch in the adversarial training stage. In addition, the true inputs of the two discriminators are over-segmented and under-segmented patches.

Proposed Method

In the real diagnostic scenario, the assisting diagnosis tool of MVI should be objective, accurate, and convenient. The less time and labor cost, the better. Based on the real requirement of diagnosis, we introduce a pathological liver vessel segmentation method that can achieve applicable performance with limited labels. The proposed method contains two parts: the pathological patches pre-processing and pathological vessel segmentation. In the former part, the doctor only needs to label limited patches. Some over/under-segmented patches are generated with different erosion and dilation operations following binarization processing. In the latter part, the EVS-Net is devised for segmenting the pathological liver vessels based on the above limited labeled patches, over-segmented and under-segmented patches.

Pathological Patch Preprocessing

From Fig. 2, we can see that the pathological liver image has a super-large size and multi-scale vessel. Meanwhile, the areas of vessels and background are unbalanced. To get the useful training samples, we first locate the vessels’ general positions according to the white color. Then, three sizes are adopted to cut patches from the whole slide image around the located positions.

For the massive cut patch set $\textbf{\em S}=\{\textbf{\em x}_{1},\textbf{\em x}_{2},\textbf{\em x}_{3},...,\textbf{\em x}_{N}\}$ , the doctor only needs to label limited patches $\mathbb{S}=\{(\textbf{\em x}_{1},\textbf{\em m}_{1}),(\textbf{\em x}_{2},\textbf{\em m}_{2}),(\textbf{\em x}_{3},\textbf{\em m}_{3}),...,(\textbf{\em x}_{K},\textbf{\em m}_{K})\}$ . Next, for a patch $\textbf{\em x}\in\textbf{\em S}$ , the under-segmented mask $\overline{\textbf{\em m}}$ of patch x can be calculated as follows:

\overline{\textbf{\em m}}=[\mathbf{1}]-T_{ostu}(\textbf{\em x})\ominus\mathfrak{S}_{r1}\oplus\mathfrak{S}_{r2},r1>r2,

(1)

where, $[\mathbf{1}]$ denotes the unit matrix that has the same size of the mask, $\mathcal{T}_{ostu}$ denotes the OTSU binarization algorithm (Otsu 1979), $\mathfrak{S}$ denotes the disk structuring element. $\ominus$ denotes the erosion operation by structure $\mathfrak{S}_{r1}$ with radius $r1$ , $\oplus$ denotes the dilation operation by structure $\mathfrak{S}_{r2}$ with radius $r2$ . The $r1$ and $r2$ are integers. The obtained $\overline{\textbf{\em m}}$ is the mask of background. The over-segmented patches constitute the over-segmented image set $\mathbb{O}=\{(\textbf{\em x}_{1},\overline{\textbf{\em m}}_{1}),(\textbf{\em x}_{2},\overline{\textbf{\em m}}_{2}),(\textbf{\em x}_{3},\overline{\textbf{\em m}}_{3}),...,(\textbf{\em x}_{N},\overline{\textbf{\em m}}_{N})\}$ .

Similarly, the under-segmented image set $\mathbb{U}=\{(\textbf{\em x}_{1},\underline{\textbf{\em m}}_{1}),(\textbf{\em x}_{2},\underline{\textbf{\em m}}_{2}),(\textbf{\em x}_{3},\underline{\textbf{\em m}}_{3}),...,(\textbf{\em x}_{N},\underline{\textbf{\em m}}_{N})\}$ is calculated as follows:

\underline{\textbf{\em m}}=T_{ostu}(\textbf{\em x})\ominus\mathfrak{S}_{r1}\oplus\mathfrak{S}_{r2},r1<r2.

(2)

Pathological Vessel Segmentation

As shown in Fig. 2, the proposed EVS-Net is composed of a segmentation network and two discriminators. The segmentation network is trained with supervision loss on the labeled limited patches and an edge-aware self-supervision loss on unlabeled patches. The two discriminators are trained on over-segmented and under-segmented patches for distinguishing whether the segmented background and vessel contain residual features, respectively. In the adversarial training stage, the edge competition between two discriminators will facilitate the vessels’ accurate segmentation.

Segmentation Network.

In the EVS-Net, the segmentation network $\mathcal{F}_{\theta}$ is designed to be an encoder-decoder architecture. With a labeled patch $\textbf{\em x}\in\mathbb{S}$ , the segmented result $\ddot{\textbf{\em m}}=\mathcal{F}_{\theta}(\textbf{\em x})$ is expected to approximate the GT mask m, which can be achieved by minimizing the pixel-wise two-class Dice loss $\mathbf{\mathcal{L}_{dic}}$ :

\mathbf{\mathcal{L}_{dic}}=1-\frac{2|\ddot{\textbf{\em m}}\cap\textbf{\em m}|+\zeta}{|\ddot{\textbf{\em m}}|+|\textbf{\em m}|+\zeta},

(3)

where, $\zeta$ is the Laplace smoothing parameter for preventing zero error and reducing overfitting. For the labeled samples, we augment samples’ diversity through compositing background. The composited background is generated by adding other pure background and normalizing the new values into the normal range.

Edge-aware Self-supervision. To reduce the dependence on massive labeled samples, inspired by Wang et al. (2019), we introduce an edge-aware self-supervision strategy, which can strengthen the edge consistency of vessels. The core idea is that the predicted mask of a transformed image should be equal to the transformed mask predicted by the network with the original image as input. Formally, for the robust segmentation network, given an affine transformation matrix M, segmented result $\mathcal{F}_{\theta}(\textbf{\em M}\textbf{\em x})$ of the affine transformed image should be consistent with the affine transformed mask $\textbf{\em M}\mathcal{F}_{\theta}(\textbf{\em x})$ in the following way: $\mathcal{F}_{\theta}(\textbf{\em M}\textbf{\em x})=\textbf{\em M}\mathcal{F}_{\theta}(\textbf{\em x})$ . Furthermore, we obtain the edge neighborhood weight map w as follows:

\textbf{\em w}=\ddot{\textbf{\em m}}\oplus\mathfrak{S}_{r3}-\ddot{\textbf{\em m}}\ominus\mathfrak{S}_{r3},

(4)

where, $r3$ denotes the radius of the structure $\mathfrak{S}$ . With the weight map w, the edge-aware self-supervision loss $\mathbf{\mathcal{L}_{sel}}$ is defined as follows:

\mathbf{\mathcal{L}_{sel}}=||\textbf{\em w}^{\prime}\mathcal{F}_{\theta}(\textbf{\em M}\textbf{\em x})-\textbf{\em M}\{\textbf{\em w}\mathcal{F}_{\theta}(\textbf{\em x})\}||^{2}_{2},\textbf{\em x}\in\textbf{\em S},

(5)

where, $\textbf{\em w}^{\prime}$ and w are the weight maps of the predicted masks $\mathcal{F}_{\theta}(\textbf{\em M}\textbf{\em x})$ and $\mathcal{F}_{\theta}(\textbf{\em x})$ , respectively. The edge-aware self-supervision mechanism not only strengthens the edge consistency of the segmented vessels but also can eliminate the unreasonable holes in the predict masks. The root reason is that unreasonable holes will be assigned high weights.

Edge-competing Discriminator.

Based on the obtained under-segmented set $\mathbb{O}$ and under-segmented set $\mathbb{U}$ , we devised two discriminators for distinguishing whether the segmented patch is a pure vessel or a pure background.

Over-segmentation Discriminator. For the over-segmented patch pairs $(\textbf{\em x},\overline{\textbf{\em m}})\in\mathbb{O}$ , the background $\textbf{\em x}_{b}$ is computed using the following equation: $\textbf{\em x}_{b}=\overline{\textbf{\em m}}*\textbf{\em x}$ , where ‘ $*$ ’ denotes pixel-wise multiplication. Then the concatenated triplet $\textbf{\em I}_{b}=[\textbf{\em x},\overline{\textbf{\em m}},\textbf{\em x}_{b}]$ is set as the positive sample of OS-D. The true condition of OS-D is that the segmented background doesn’t contain any residual vessels’ features. For the unlabeled patch $\textbf{\em x}\in\textbf{\em S}$ , the segmentation network predicts the mask of vessel $\ddot{\textbf{\em m}}=\mathcal{F}_{\theta}(\textbf{\em x})$ . The reversed mask is calculated as follows: $\ddot{\textbf{\em m}}^{\prime}=[\mathbf{1}]-\ddot{\textbf{\em m}}$ , where the $[\mathbf{1}]$ denotes the unit matrix that has the same size of the mask. Next, the segmented background $\ddot{\textbf{\em x}}_{b}$ is computed using the following equation: $\ddot{\textbf{\em x}}_{b}=\ddot{\textbf{\em m}}^{\prime}*\textbf{\em x}$ , Then the concatenated triplet $\ddot{\textbf{\em I}}_{b}=[\textbf{\em x},\ddot{\textbf{\em m}}^{\prime},\ddot{\textbf{\em x}}_{b}]$ is fed to the OS-D $\mathcal{D}^{b}_{\phi}$ , which discriminates whether the segmented background $\ddot{\textbf{\em x}}_{b}$ contains the vessels’ residual features. The $\ddot{\textbf{\em I}}_{b}$ is regarded as a false triplet. The adversarial optimization between the segmentation network and OS-D will constrain the segmented background predicted by the segmentation network don’t contain vessels’ features. The over-segmentation adversarial loss $\mathbf{\mathcal{L}^{ovr}_{adv}}$ is given as follows:

\begin{split}\mathbf{\mathcal{L}^{ovr}_{adv}}&=\mathop{\mathbb{E}}\limits_{\ddot{\textbf{\em I}}_{b}\sim\ddot{\mathbb{P}}_{b}}[\mathcal{D}^{b}_{\phi}(\ddot{\textbf{\em I}}_{b})]-\mathop{\mathbb{E}}\limits_{\textbf{\em I}_{b}\sim\mathbb{P}_{b}}[\mathcal{D}^{b}_{\phi}(\textbf{\em I}_{b})]\\ &+\lambda\mathop{\mathbb{E}}\limits_{\hat{\textbf{\em I}}_{b}\sim\mathbb{P}_{\hat{\textbf{\em I}}_{b}}}[(\|\nabla_{\hat{\textbf{\em I}}_{b}}\mathcal{D}^{b}_{\phi}(\hat{\textbf{\em I}}_{b})\|_{2}-1)^{2}],\end{split}

(6)

where, the $\ddot{\mathbb{P}}_{b}$ , $\mathbb{P}_{b}$ are the generated background triplet distribution and under-segmented triplet distribution, respectively. The $\hat{\textbf{\em I}}_{b}$ is sampled uniformly along straight lines between pairs of points sampled from the generated background triplet distribution $\ddot{\mathbb{P}}_{b}$ and the under-segmented triplet distribution $\mathbb{P}_{b}$ . The $\hat{\textbf{\em I}}_{b}=\varepsilon\textbf{\em I}_{b}+(1-\varepsilon)\ddot{\textbf{\em I}}_{b}$ , where the $\varepsilon$ is a random number between $0$ and $1$ . The second term (gradient penalty) is firstly proposed in WGAN-GP (Gulrajani et al. 2017). The $\lambda$ is the gradient penalty coefficient.

Under-segmentation Discriminator. For the under-segmented patch pairs $(\textbf{\em x},\underline{\textbf{\em m}})\in\mathbb{U}$ , the segmented vessel $\textbf{\em x}_{v}$ is calculated with the following equation: $\textbf{\em x}_{v}=\underline{\textbf{\em m}}*\textbf{\em x}$ . Then the concatenated triplet $\textbf{\em I}_{v}=[\textbf{\em x},\underline{\textbf{\em m}},\textbf{\em x}_{v}]$ satisfies the true condition of the US-D that the segmented vessel doesn’t contain any residual background features. For the unlabeled patch $\textbf{\em x}\in\textbf{\em S}$ , the concatenated triplet $\ddot{\textbf{\em I}}_{v}=[\textbf{\em x},\ddot{\textbf{\em m}},\ddot{\textbf{\em x}}_{v}]$ is fed to the US-D $\mathcal{D}^{v}_{\varphi}$ , which distinguishes whether the segmented vessel $\ddot{\textbf{\em x}}_{v}$ contains the background residual features. The $\ddot{\textbf{\em I}}_{v}$ is regarded as the false triplet for US-D. The under-segmentation adversarial loss $\mathbf{\mathcal{L}^{udr}_{adv}}$ is given as follows:

\begin{split}\mathbf{\mathcal{L}^{udr}_{adv}}&=\mathop{\mathbb{E}}\limits_{\ddot{\textbf{\em I}}_{v}\sim\ddot{\mathbb{P}}_{v}}[\mathcal{D}^{v}_{\varphi}(\ddot{\textbf{\em I}}_{v})]-\mathop{\mathbb{E}}\limits_{\textbf{\em I}_{v}\sim\mathbb{P}_{v}}[\mathcal{D}^{v}_{\varphi}(\textbf{\em I}_{v})]\\ &+\lambda\mathop{\mathbb{E}}\limits_{\hat{\textbf{\em I}}_{v}\sim\mathbb{P}_{\hat{\textbf{\em I}}_{v}}}[(\|\nabla_{\hat{\textbf{\em I}}_{v}}\mathcal{D}^{v}_{\varphi}(\hat{\textbf{\em I}}_{v})\|_{2}-1)^{2}],\end{split}

(7)

where, the $\ddot{\mathbb{P}}_{v}$ , $\mathbb{P}_{v}$ are the generated vessel triplet distribution and under-segmented triplet distribution, respectively. The $\hat{\textbf{\em I}}_{v}$ is sampled uniformly along straight lines between pairs of points sampled from the generated vessel triplet distribution $\ddot{\mathbb{P}}_{v}$ and the under-segmented triplet distribution $\mathbb{P}_{v}$ . The $\hat{\textbf{\em I}}_{v}=\varepsilon\textbf{\em I}_{v}+(1-\varepsilon)\ddot{\textbf{\em I}}_{v}$ , where the $\varepsilon$ is a random number between $0$ and $1$ . The optimization on $\mathbf{\mathcal{L}^{udr}_{adv}}$ will constrain the segmented vessel predicted by the segmentation network don’t contain background features.

Complete Algorithm

To sum up, the supervised Dice loss $\mathbf{\mathcal{L}_{dic}}$ is adopted to train the pathological vessel segmentation network with the limited labeled patches. Then, edge-aware self-supervision is introduced to strengthen the edge consistency of the segmented mask with massive unlabeled patches. For the over-segmented patches and under-segmented patches, the edge adversarial losses $\mathbf{\mathcal{L}^{ovr}_{adv}}$ and $\mathbf{\mathcal{L}^{udr}_{adv}}$ are adopted to train the OS-D and US-D. In the training stage, the two discriminators compete for the edge’s position of the predicted mask, which will refine the segmented mask. During training, we alternatively optimize the segmentation network $\mathcal{F}_{\theta}$ and two edge discriminators $\mathcal{D}^{b}_{\phi},\mathcal{D}^{v}_{\varphi}$ using the randomly sampled samples from unlabeled patch set S, labeled patch set $\mathbb{S}$ , under-segmented set $\mathbb{O}$ , and under-segmented set $\mathbb{U}$ . The complete algorithm is summarized in Algorithm 1.

Algorithm 1 The Training Algorithm for EVS-Net

1: The gradient penalty coefficient

\lambda

, interval iteration number

n_{critic}

, the batch size

T

, the Laplace smoothing parameter

\zeta

, Adam hyperparameters

\alpha,\beta_{1},\beta_{2}

, the balance parameters

\tau,\eta

for

\mathcal{L}_{dic}

and

\mathbf{\mathcal{L}_{sel}}

2: Initial critic parameters

\varphi,\phi

, initial segmentation networt parameters

\theta

3:while

\theta

has not converged do

4: for

i=1,...,n_{critic}

5: for

t=1,...,T

6: Sample

\textbf{\em x}\in\textbf{\em S}

(\textbf{\em x},\overline{\textbf{\em m}})\in\textbf{\em O}

(\textbf{\em x},\underline{\textbf{\em m}})\in\textbf{\em U}

7: a random number

\varepsilon\sim[0,1]

8: Obtain real triplet

\textbf{\em I}_{b}

and

\textbf{\em I}_{v}

9: Obtain false triplet

\ddot{\textbf{\em I}}_{b}

and

\ddot{\textbf{\em I}}_{v}

10:

\mathbf{\mathcal{L}^{ovr}_{adv}}^{(t)}\!\leftarrow\!\mathcal{D}^{b}_{\phi}(\ddot{\textbf{\em I}}_{b})\!-\!\mathcal{D}^{b}_{\phi}(\textbf{\em I}_{b})

11:

+\lambda(\|\nabla_{\hat{\textbf{\em I}}_{b}}\mathcal{D}^{b}_{\phi}(\hat{\textbf{\em I}}_{b})\|_{2}-1)^{2}

12:

\mathbf{\mathcal{L}^{udr}_{adv}}^{(t)}\leftarrow\mathcal{D}^{v}_{\varphi}(\ddot{\textbf{\em I}}_{v})-\mathcal{D}^{v}_{\varphi}(\textbf{\em I}_{v})

13:

+\lambda(\|\nabla_{\hat{\textbf{\em I}}_{v}}\mathcal{D}^{v}_{\varphi}(\hat{\textbf{\em I}}_{v})\|_{2}-1)^{2}

14:

\phi\leftarrow Adam(\nabla_{\phi}\frac{1}{T}\sum^{T}_{t=1}\mathbf{\mathcal{L}^{ovr}_{adv}}^{(t)},\phi,\alpha,\beta_{1},\beta_{2})

15:

\varphi\leftarrow Adam(\nabla_{\varphi}\frac{1}{T}\sum^{T}_{t=1}\mathbf{\mathcal{L}^{udr}_{adv}}^{(t)},\varphi,\alpha,\beta_{1},\beta_{2})

16: Sample unlabeled batch

\{\textbf{\em x}_{t}\}^{T}_{t=1}

from S and

17: labeled batch

\{(\textbf{\em x}_{t},\textbf{\em m}_{t})\}^{T}_{t=1}

from

\mathbb{S}

18:

\mathbf{\mathcal{L}_{dic}}\leftarrow 1-\frac{2|\mathcal{F}_{\theta}(\textbf{\em x})\cap\textbf{\em m}|+\zeta}{|\mathcal{F}_{\theta}(\textbf{\em x})|+|\textbf{\em m}|+\zeta}

19:

\mathbf{\mathcal{L}_{sel}}\leftarrow||\textbf{\em w}^{\prime}\mathcal{F}_{\theta}(\textbf{\em M}\textbf{\em x})-\textbf{\em M}\{\textbf{\em w}\mathcal{F}_{\theta}(\textbf{\em x})\}||^{2}_{2}

20:

\theta\leftarrow Adam(\nabla_{\theta}\frac{1}{T}\sum^{T}_{t=1}\{\mathbf{\tau\mathcal{L}_{dic}}+\eta\mathbf{\mathcal{L}_{sel}}

21:

-\mathcal{D}^{b}_{\phi}(\mathcal{F}_{\theta}(\textbf{\em x}))-\mathcal{D}^{v}_{\varphi}(\mathcal{F}_{\theta}(\textbf{\em x}))\}),\theta,\alpha,\beta_{1},\beta_{2})

22:return Segmentation networt parameters

\theta

, critic parameters

\varphi,~{}\phi

Experiments

Dataset. The collected pathological liver image dataset contains $522$ whole slide images. Each image size has about $200,000*100,000$ . For each image, pathologists labeled the vessels’ masks, MVI, and hepatocellular carcinoma grades. There are total $98,398$ vessels where $2,890$ vessels contain hepatocellular carcinoma cells. The sample number for levels one, two, three and four tumor are $52$ , $245$ , $189$ , and $36$ , respectively. It can be a benchmark for the research on the analysis of pathological liver images. For all images, we finally collect $180,000$ patches, which have three scales ( $128*128$ , $512*512$ and $1024*1024$ ). All patches are resized into $128*128$ . The train, validation, and test sample number are $144,000$ , $18,000$ , and $18,000$ , respectively.

Network architecture and parameters. The segmentation network we adopted is the DeeplabV3+ (backbone: resnet50 ) (Chen et al. 2017). In the experiment, the parameters are set as follows: $\zeta=1,\tau=1,\eta=2,\lambda=10$ , $n_{critic}=5$ , the batch size $T=64$ , Adam hyperparameters for two discriminators $\alpha=0.0001,\beta_{1}=0,\beta_{2}=0.9$ . The radius $r1$ and $r2$ are random values between $5$ and $30$ . The radius $r3$ equals to $15$ . The learning rate for the segmentation network and two discriminators are all set to be $1e^{-4}$ .

Metric. The metrics we adopted include Pixel Accuracy (PA), Mean Pixel Accuracy (MPA), Mean Intersection over Union (MIoU), Frequency Weighted Intersection over Union (FWIoU), and Dice.

	Type	Unsupervised		Weakly Supervised		Edge-aware		Fully Supervised					Ours
	Method	ReDO	CAC	ALSSS/ 500	USSS/ 500	G-SCNN	BFP	Unet	FPN	PSPNet	PAN	Deeplab	E(100)
All patch	MPA	87.89	84.32	62.92/ 91.84	89.88/ 94.93	97.29	96.83	96.45	98.56	98.29	98.44	98.82	96.59
	MIOU	82.20	78.35	55.41/ 89.40	84.57/ 92.35	97.28	96.04	92.76	97.40	96.86	97.11	97.79	94.38
	FWIoU	89.34	80.57	74.45/ 93.77	90.75/ 95.46	95.89	96.32	95.64	98.46	98.13	98.29	98.69	96.67
	DICE	96.56	88.32	91.99/ 98.08	97.02/ 98.58	98.76	98.21	98.62	99.52	99.42	99.47	99.60	98.96
\cdashline1-14[0.8pt/2pt] MVI patch	MPA	81.85	78.32	56.09/ 77.71	80.61/ 89.34	95.87	95.43	89.14	97.55	96.61	97.17	97.97	90.85
	MIOU	74.06	69.87	48.21/ 72.43	73.82/ 84.83	92.85	94.72	81.77	95.60	94.45	94.82	95.84	86.19
	FWIoU	84.75	79.10	71.55/ 84.44	84.81/ 91.24	95.90	94.53	89.16	97.48	96.80	97.00	97.60	91.98
	DICE	94.95	85.32	91.28/ 95.19	95.09/ 97.25	94.83	97.53	96.42	99.23	99.01	99.07	99.29	97.46

Table 1: The quantitative results of different methods. Underline indicates the best performance among all methods. Bold indicates the best performance among all non-fully supervised methods. ‘

E

(K)’ denotes EVS-Net with ‘K’ labeled patches. The ‘ALSSS/

500

’ and ‘USSS/

500

’ denote methods with

10

500

labeled patches (All scores are the average of three runs).

Quantitative Evaluation

To verify the effectiveness of EVS-Net, we compare the proposed method with the SOTA methods, including unsupervised methods (ReDO (Chen, Artieres, and Denoyer 2019), CAC (Hsu, Lin, and Chuang 2018)), weakly-/semi-supervised methods (ALSSS (Hung et al. 2018), USSS (Kalluri et al. 2019)) and fully supervised methods (edge-aware methods:{ G-SCNN (Takikawa et al. 2019), BFP (Ding et al. 2019)},Unet (Ronneberger, Fischer, and Brox 2015), FPN (Lin et al. 2017), PSPNet (Zhao et al. 2017), PAN (Li et al. 2018) and DeeplabV3+ (Chen et al. 2017)). For the weakly-/semi-supervised methods (USSS and ALSSS), we provide two groups of labeled patches ( $10$ patches and $500$ patches). The same data augmentation strategy is applied on the above labeled patches. Table 1 shows the quantitative results, where all the scores are average of three runs. For each trained model, we test it on all patches (‘All patch’) and patches (’MVI patch’) that contain hepatocellular carcinoma cells. It is obvious that DeeplabV3+ achieves the best performance among all the methods. The $\mathbf{E}(100)$ achieves SOTA results on par with existing non-fully supervised methods. Even with $10$ labeled samples, EVS-Net still achieves higher scores than non-fully supervised methods trained with hundreds of labeled patches on the ‘All patch’. In addition, even without labeled samples, $\mathbf{E}(0)$ (given in Table 2) still achieves better results than weakly/semi-supervised methods trained with $10$ labeled patches, which validates the effectiveness of the edge-competing mechanism. For ‘All patch’, $\mathbf{E}(10)$ has about $2\%$ increase on the scores of $\mathbf{E}(0)$ . Meanwhile, for ‘MVI patch’, $\mathbf{E}(10)$ has about $4\%$ performance increase compared with $\mathbf{E}(0)$ , which verifies the necessity of the limited labeled samples. More results of EVS-Net with different labeled patches are given in Table 2 and Fig. 4.

	Index $\backslash$ Ablation	$\mathbf{E}_{self}^{-}$	$\mathbf{E}_{over}^{-}$	$\mathbf{E}_{under}^{-}$	$\mathbf{E}_{disc}^{-}$	$\mathbf{E}(0)$	$\mathbf{E}(5)$	$\mathbf{E}(10)$	$\mathbf{E}(20)$	$\mathbf{E}(50)$	$\mathbf{E}(100)$	$\mathbf{E}(fully)$
All patch	MPA	95.73	94.28	93.59	90.90	94.77	95.47	96.24	96.38	96.72	96.59	98.48
	MIoU	92.38	90.69	90.02	78.83	91.93	92.40	93.34	93.55	93.68	94.38	97.25
	FWIoU	95.44	94.02	93.63	86.38	95.20	95.46	96.02	96.21	96.22	96.67	98.37
	DICE	98.56	97.42	97.30	95.08	98.50	98.58	98.75	98.76	98.81	98.96	99.50
\cdashline1-2[0.8pt/2pt] \cdashline3-6[0.8pt/2pt] \cdashline7-13[0.8pt/2pt] MVI patch	MPA	87.80	88.42	87.26	80.47	85.12	87.89	89.81	89.63	89.25	90.85	97.16
	MIoU	78.97	78.73	78.05	62.22	77.57	81.32	82.03	82.91	83.14	86.19	95.05
	FWIoU	87.33	86.64	86.32	74.16	86.74	89.01	89.25	89.83	90.10	91.98	97.14
	DICE	95.71	94.75	94.67	89.19	94.66	96.43	96.42	96.35	96.80	97.46	99.12

Table 2: The ablation study result on EVS-Net.

E_{self}^{-}

E_{cond}^{-}

E_{over}^{-}

E_{under}^{-}

and

E_{disc}^{-}

denote the EVS-Net without edge-aware self-supervision, over-segmentation discriminator, under-segmentation discriminator, and two discriminators.

E(K)

denotes the EVS-Net with

K

labeled patches. Italic indicates the best performance among methods trained with

10

labeled patches. Bold and underline indicate the best and second-best performance among all methods (All scores in

\%

Qualitative Evaluation

The visual results of different methods are shown in Fig. 3, where we can see that most of the fully supervised methods achieve similar results. Deep learning-based methods indeed have the potential to be applied to the pathological liver vessel segmentation. For the unsupervised methods (ReDO and CAC), the segmented background and vessels contain many small holes, which indicates the instability of the unsupervised methods. From the column $5$ , we can see that ALSSS with $10$ labeled patches fails to segment most cases, demonstrating that ten labeled patches cannot support enough supervision information for the semi-supervised method. Even with $500$ labeled patches, USSS still fails to segment the vessel with a large hepatocellular carcinoma area (row 4, column 8). For the blurry edge vessels (row 1 and row 5), EVS-Net with larger than $10$ labeled patches can achieve more accurate edge segmentation than non-fully supervised methods. For MVI patches, $\mathbf{E}(0)$ achieves inaccurate segmentation on those hepatocellular carcinoma areas, which indicates that limited labeled patches are necessity for guiding the segmentation of hepatocellular carcinoma areas. In summary, with limited labeled patches, EVS-Net can achieve close results on par with fully supervised methods.

Ablation Study

To verify the effectiveness of each component, we conduct an ablation study on the edge-aware self-supervision, over-segmentation discriminator, under-segmentation discriminator, two discriminators, and the different numbers of labeled patches. The $\mathbf{E}_{self}^{-}$ , $\mathbf{E}_{over}^{-}$ , $\mathbf{E}_{under}^{-}$ and $\mathbf{E}_{disc}^{-}$ denote the EVS-Net without edge-aware self-supervision, over-segmentation discriminator, under-segmentation discriminator, and two discriminators, which are trained in the same setting of $10$ labeled patches. Table 2 shows the quantitative results of the above methods. The $\mathbf{E}(K)$ denotes the EVS-Net with $K$ labeled patches. We can see that $\mathbf{E}(10)$ achieves the best performance among $\mathbf{E}_{self}^{-}$ , $\mathbf{E}_{over}^{-}$ , $\mathbf{E}_{under}^{-}$ and $\mathbf{E}_{disc}^{-}$ , which demonstrates that all components of EVS-Net are helpful for the segmentation performance. The performance of $\mathbf{E}_{disc}^{-}$ that doesn’t contain two discriminators degrades about $10\%$ compared with the $\mathbf{E}(10)$ . Methods with only one discriminator ( $\mathbf{E}_{over}^{-}$ and $\mathbf{E}_{under}^{-}$ ) reduce about $2\%$ compared with the $\mathbf{E}(10)$ . From Fig. 4, we can see that predicted masks of $\mathbf{E}(10)$ have accurate edges than $\mathbf{E}_{over}^{-}$ and $\mathbf{E}_{under}^{-}$ , which verifies the effectiveness and necessity of edge-competing. In addition, Fig. 4 shows that the result of $\mathbf{E}_{self}^{-}$ has some holes. By contrast, the result of $\mathbf{E}(10)$ is accurate, which indicates that the edge-aware self-supervision is also beneficial for eliminating the incorrect holes. For the different numbers of labeled patches, we find that the critical cut-off point is $10$ -labeled-samples, which can supply relatively sufficient guidance. For the patches that have MVI, vessels’ edges are usually blurry, and vessels’ inner usually contain the large area of the hepatocellular carcinoma cells. Without any guidance of labeled samples, the pathological vessel segmentation network will fail to obtain the vessels’ desired edges. Therefore, the $\mathbf{E}(0)$ achieves the worst performance among all $\mathbf{E}(K)$ methods, which also verifies the necessity of the limited labeled patches. In addition, the comparisons between the segmented vessels of $\mathbf{E}_{over}^{-}$ , $\mathbf{E}_{under}^{-}$ , $\mathbf{E}(10)$ are shown in the last three columns on the right of Fig. 4, where we can see that segmented vessels’ contours of $\mathbf{E}_{over}^{-}$ and $\mathbf{E}_{under}^{-}$ contract inward and expand outward, respectively. With the edge-competing between OS-D and US-D, $\mathbf{E}(10)$ have accurate edges’ contours, which effectively clarifies the effectiveness of the edge-competing mechanism.

Conclusion

In this paper, we propose the first computer-aided diagnosis method for the pathological liver vessel segmentation, which is an essential and critical step for the diagnosis of MVI. For massive unlabeled pathological patches, the doctor only needs to label limited patches. Next, binarization processing combining with erosion and dilation is adopted to generate some under-segmented and under-segmented masks. Then, the EVS-Net is proposed for segmenting the pathological liver vessel with limited labels. The EVS-Net contains a pathological vessel segmentation network and two discriminators. With the limited labeled patches, the segmentation network is initialized with the supervision training. Meanwhile, an edge-aware self-supervision module is proposed for enhancing the edge consistency of massive unlabeled patches. With the over-segmented and under-segmented patches, two discriminators are devised for distinguishing whether the segmented vessels and background contain residual features. The segmentation network and two discriminators are trained in an adversarial manner. The edge-competing mechanism is verified to be very effective for facilitating the segmentation network segmenting more accurately. Exhaustive experiments demonstrate that the proposed EVS-Net achieves a close performance of fully supervised methods with limited labeled patches. It provides a convenient tool for the pathological liver vessel segmentation. In future work, we will focus on detecting and counting the hepatocellular carcinoma cells in the segmented vessels, which will provide a complete assisting diagnosis method for the diagnosis of MVI.

Acknowledgments

This work was supported by National Natural Science Foundation of China (No.62002318), Zhejiang Provincial Natural Science Foundation of China (LQ21F020003), Zhejiang Provincial Science and Technology Project for Public Welfare (LGF21F020020), Programs Supported by Ningbo Natural Science Foundation (202003N4318), and Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies.

References

Altaf et al. (2019) Altaf, F.; Islam, S.; Akhtar, N.; and Janjua, N. K. 2019. Going Deep in Medical Image Analysis: Concepts, Methods, Challenges, and Future Directions. IEEE Access 7: 99540–99572.
Arbelle and Raviv (2018) Arbelle, A.; and Raviv, T. R. 2018. Microscopy Cell Segmentation via Adversarial Neural Networks. IEEE International Symposium on Biomedical Imaging 645–648.
Bertasius, Shi, and Torresani (2016) Bertasius, G.; Shi, J.; and Torresani, L. 2016. Semantic Segmentation with Boundary Neural Fields. IEEE Conference on Computer Vision and Pattern Recognition 3602–3610.
Chen et al. (2016) Chen, L.; Barron, J. T.; Papandreou, G.; Murphy, K.; and Yuille, A. L. 2016. Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform. IEEE Conference on Computer Vision and Pattern Recognition 4545–4554.
Chen et al. (2017) Chen, L.; Papandreou, G.; Schroff, F.; and Adam, H. 2017. Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv: Computer Vision and Pattern Recognition .
Chen, Artieres, and Denoyer (2019) Chen, M.; Artieres, T.; and Denoyer, L. 2019. Unsupervised Object Segmentation by Redrawing. Neural Information Processing Systems 12726–12737.
Cheng et al. (2017) Cheng, D.; Meng, G.; Xiang, S.; and Pan, C. 2017. FusionNet: Edge Aware Deep Convolutional Networks for Semantic Segmentation of Remote Sensing Harbor Images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 10(12): 5769–5783.
Cuccurullo et al. (2018) Cuccurullo, V.; Stasio, G. D. D.; Mazzarella, G.; and Cascini, G. L. 2018. Microvascular Invasion in HCC: The Molecular Imaging Perspective. Contrast Media & Molecular Imaging 2018: 9487938.
D, Suna, and Boyacioglu (2017) D, O. E.; Suna, N.; and Boyacioglu, A. S. 2017. Management of Hepatocellular Carcinoma: Prevention, Surveillance, Diagnosis, and Staging. Experimental and Clinical Transplantation 15: 31.
Dai et al. (2019) Dai, C.; Mo, Y.; Angelini, E.; Guo, Y.; and Bai, W. 2019. Transfer Learning from Partial Annotations for Whole Brain Segmentation. Medical Image Computing and Computer Assisted Intervention Society Workshop 199–206.
Ding et al. (2019) Ding, H.; Jiang, X.; Liu, A. Q.; Thalmann, N. M.; and Wang, G. 2019. Boundary-Aware Feature Propagation for Scene Segmentation. IEEE International Conference on Computer Vision 6819–6829.
Gulrajani et al. (2017) Gulrajani, I.; Ahmed, F.; Arjovsky, M.; Dumoulin, V.; and Courville, A. 2017. Improved training of wasserstein GANs. Neural Information Processing Systems 5769–5779.
Han and Yin (2017) Han, L.; and Yin, Z. 2017. Transferring Microscopy Image Modalities with Conditional Generative Adversarial Networks. IEEE Conference on Computer Vision and Pattern Recognition 2017: 851–859.
Hayder, He, and Salzmann (2017) Hayder, Z.; He, X.; and Salzmann, M. 2017. Boundary-Aware Instance Segmentation. IEEE Conference on Computer Vision and Pattern Recognition 587–595.
Hsu, Lin, and Chuang (2018) Hsu, K.; Lin, Y.; and Chuang, Y. 2018. Co-attention CNNs for Unsupervised Object Co-segmentation. International Joint Conference on Artificial Intelligence 748–756.
Hung et al. (2018) Hung, W.; Tsai, Y.; Liou, Y.; Lin, Y.; and Yang, M. 2018. Adversarial Learning for Semi-Supervised Semantic Segmentation. British Machine Vision Conference 65.
Jiao et al. (2020) Jiao, W.; Atwal, G.; Polak, P.; Karlic, R.; Cuppen, E.; Danyi, A.; De Ridder, J.; Van Herpen, C. M. L.; Lolkema, M. P.; Steeghs, N.; et al. 2020. A deep learning system accurately classifies primary and metastatic cancers using passenger mutation patterns. Nature Communications 11(1): 728.
Kalluri et al. (2019) Kalluri, T.; Varma, G.; Chandraker, M.; and Jawahar, C. V. 2019. Universal Semi-Supervised Semantic Segmentation. IEEE International Conference on Computer Vision 5259–5270.
Ker et al. (2018) Ker, J.; Wang, L.; Rao, J.; and Lim, T. 2018. Deep Learning Applications in Medical Image Analysis. IEEE Access 6: 9375–9389.
Khoreva et al. (2017) Khoreva, A.; Benenson, R.; Hosang, J.; Hein, M.; and Schiele, B. 2017. Simple Does It: Weakly Supervised Instance and Semantic Segmentation. IEEE Conference on Computer Vision and Pattern Recognition 1665–1674.
Li et al. (2018) Li, H.; Xiong, P.; An, J.; and Wang, L. 2018. Pyramid Attention Network for Semantic Segmentation. British Machine Vision Conference 285.
Liao et al. (2020) Liao, H.; Long, Y.; Han, R.; Wang, W.; Xu, L.; Liao, M.; Zhang, Z.; Wu, Z.; Shang, X.; Li, X.; Peng, J.; Yuan, K.; and Zeng, Y. 2020. Deep learning-based classification and mutation prediction from histopathological images of hepatocellular carcinoma. Clinical and Translational Medicine 10(2): e102.
Lin et al. (2017) Lin, T.; Dollar, P.; Girshick, R.; He, K.; Hariharan, B.; and Belongie, S. 2017. Feature Pyramid Networks for Object Detection. IEEE Conference on Computer Vision and Pattern Recognition 936–944.
Luc et al. (2016) Luc, P.; Couprie, C.; Chintala, S.; and Verbeek, J. 2016. Semantic Segmentation using Adversarial Networks. Neural Information Processing Systems .
Minaee et al. (2020) Minaee, S.; Boykov, Y.; Porikli, F.; Plaza, A.; Kehtarnavaz, N.; and Terzopoulos, D. 2020. Image Segmentation Using Deep Learning: A Survey. CoRR abs/2001.05566. URL https://arxiv.org/abs/2001.05566.
Ostyakov et al. (2018) Ostyakov, P.; Suvorov, R.; Logacheva, E.; Khomenko, O.; and Nikolenko, S. I. 2018. SEIGAN: Towards Compositional Image Generation by Simultaneously Learning to Segment, Enhance, and Inpaint. arXiv: Computer Vision and Pattern Recognition .
Otsu (1979) Otsu, N. 1979. A Threshold Selection Method from Gray-Level Histograms. IEEE Transactions on Systems, Man, and Cybernetics 9(1): 62–66.
Peng et al. (2017) Peng, C.; Zhang, X.; Yu, G.; Luo, G.; and Sun, J. 2017. Large Kernel Matters ¡ª Improve Semantic Segmentation by Global Convolutional Network. IEEE Conference on Computer Vision and Pattern Recognition 1743–1751.
Perumpail et al. (2017) Perumpail, B. J.; Khan, M. A.; Yoo, E. R.; Cholankeril, G.; Kim, D.; and Ahmed, A. 2017. Clinical epidemiology and disease burden of nonalcoholic fatty liver disease. World Journal of Gastroenterology 23(47): 8263–8276.
Qin et al. (2019) Qin, X.; Zhang, Z.; Huang, C.; Gao, C.; Dehghan, M.; and Jagersand, M. 2019. BASNet: Boundary-Aware Salient Object Detection. IEEE Conference on Computer Vision and Pattern Recognition 7479–7489.
Qiu et al. (2020) Qiu, J.; Yang, Y.; Wang, X.; and Tao, D. 2020. Hallucinating Visual Instances in Total Absentia. In European Conference on Computer Vision.
Remez, Huang, and Brown (2018) Remez, T.; Huang, J.; and Brown, M. 2018. Learning to Segment via Cut-and-Paste. European Conference on Computer Vision 39–54.
Rodriguezperalvarez et al. (2013) Rodriguezperalvarez, M.; Luong, T. V.; Andreana, L.; Meyer, T.; Dhillon, A. P.; and Burroughs, A. K. 2013. A systematic review of microvascular invasion in hepatocellular carcinoma: diagnostic and prognostic variability. Annals of Surgical Oncology 20(1): 325–339.
Ronneberger, Fischer, and Brox (2015) Ronneberger, O.; Fischer, P.; and Brox, T. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer Assisted Intervention Society 234–241.
Souly, Spampinato, and Shah (2017) Souly, N.; Spampinato, C.; and Shah, M. 2017. Semi Supervised Semantic Segmentation Using Generative Adversarial Network. IEEE International Conference on Computer Vision 5689–5697.
Sumie et al. (2008) Sumie, S.; Kuromatsu, R.; Okuda, K.; Ando, E.; Takata, A.; Fukushima, N.; Watanabe, Y.; Kojiro, M.; and Sata, M. 2008. Microvascular Invasion in Patients with Hepatocellular Carcinoma and Its Predictable Clinicopathological Factors. Annals of Surgical Oncology 15(5): 1375–1382.
Takikawa et al. (2019) Takikawa, T.; Acuna, D.; Jampani, V.; and Fidler, S. 2019. Gated-SCNN: Gated Shape CNNs for Semantic Segmentation. arXiv: Computer Vision and Pattern Recognition .
Wang et al. (2019) Wang, Y.; Zhang, J.; Kan, M.; Shan, S.; and Chen, X. 2019. Self-supervised Scale Equivariant Network for Weakly Supervised Semantic Segmentation. arXiv: Computer Vision and Pattern Recognition .
Wang, Simoncelli, and Bovik (2003) Wang, Z.; Simoncelli, E. P.; and Bovik, A. C. 2003. Multiscale Structural Similarity for Image Quality Assessment. Asilomar Conference on Signals, Systems and Computers 2: 1398–1402.
Xue et al. (2018) Xue, Y.; Xu, T.; Zhang, H.; Long, L. R.; and Huang, X. 2018. SegAN: Adversarial Network with Multi-scale L 1 Loss for Medical Image Segmentation. Neuroinformatics 16(3): 383–392.
Yang et al. (2019) Yang, D.; Jia, X.; Xiao, Y.; Wang, X.; Wang, Z.; and Yang, Z. 2019. Noninvasive Evaluation of the Pathologic Grade of Hepatocellular Carcinoma Using MCF-3DCNN: A Pilot Study. BioMed Research International 2019: 9783106.
Ye et al. (2020) Ye, J.; Ji, Y.; Wang, X.; Gao, X.; and Song, M. 2020. Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN. In IEEE Conference on Computer Vision and Pattern Recognition.
Ye et al. (2019) Ye, J.; Ji, Y.; Wang, X.; Ou, K.; Tao, D.; and Song, M. 2019. Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More. In IEEE Conference on Computer Vision and Pattern Recognition.
Yin et al. (2018) Yin, X.; Wang, X.; Yu, J.; Zhang, M.; Fua, P.; and Tao, D. 2018. FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification. In European Conference on Computer Vision.
Yu et al. (2018) Yu, C.; Wang, J.; Peng, C.; Gao, C.; Yu, G.; and Sang, N. 2018. Learning a Discriminative Feature Network for Semantic Segmentation. IEEE Conference on Computer Vision and Pattern Recognition 1857–1866.
Zhang et al. (2017) Zhang, R.; Tang, S.; Lin, M.; Li, J.; and Yan, S. 2017. Global-residual and local-boundary refinement networks for rectifying scene parsing predictions. International Joint Conference on Artificial Intelligence 3427–3433.
Zhao et al. (2017) Zhao, H.; Shi, J.; Qi, X.; Wang, X.; and Jia, J. 2017. Pyramid Scene Parsing Network. IEEE Conference on Computer Vision and Pattern Recognition 6230–6239.