Simulation of Arbitrary Level Contrast Dose in MRI Using an Iterative Global Transformer Model

Gadolinium-Based Contrast Agents (GBCAs) are widely used in MRI scans owing to their capability of improving the border delineation and internal morphology of different pathologies and have extensive clinical applications[1]. However, GBCAs have several disadvantages like contraindications in patients with reduced renal function[2], patient inconvenience, high operation costs and environmental side effects[3]. Therefore, there is an increased emphasis on the paradigm of "as low as reasonably achievable" (ALARA)[4]. To tackle these concerns of GBCAs, several dose reduction[5, 6] and elimination approaches[7] have been proposed. However, these deep learning(DL)-based dose reduction approaches require high quality low-dose contrast-enhanced (CE) images paired with pre-contrast and full-dose CE images. Acquiring such a dataset requires modification of the standard imaging protocol and involves additional training of the MR technicians. Therefore, it is important to simulate the process of T1w low-dose image acquisition, using images from the standard protocol. Moreover, it is crucial for these dose reduction approaches to establish the minimum dose level required for different pathologies as these are dependent on the scanning protocol and the GBCA compound injected. Therefore the simulation tool should also have the ability to synthesize images with multiple contrast enhancement levels, that correspond to multiple arbitrary dose levels.

Currently MRI dose simulation is done using physics-based models[8]. However, these physics-based methods are dependent on the protocol parameters and the type of GBCA and their relaxation parameters. Deep learning (DL) models have been widely used in medical imaging application due to their high capacity, generazibility, and transferability[9, 10]. The performance of these DL models heavily depend on the availability of high quality data. There is a dearth of data-driven approaches to MRI dose-simulation given the lack of diverse ground truth data of the different dose levels. To this effect, we introduce a vision transformer based DL model^bbbA part of this work was presented as a poster in the conference of International Society for Magnetic Resonance in Medicine (ISMRM) 2023, held in Toronto. that can synthesize brain^cccRefer Supplementary Material Figure 1(b) for examples on Spine MRI. MRI images that correspond to arbitrary dose levels, by training on a highly imbalanced dataset with only T1w pre-contrast, T1w 10% low-dose, and T1w CE standard dose images. The model backbone consists of a novel Global transformer (Gformer) with subsampling attention that can learn long-range dependencies of contrast uptake features. The proposed method also consists of a rotational shift operation that can further capture the shape irregularity of the contrast uptake regions. We performed extensive quantitative evaluation in comparison to other state-of-the art methods. Additionally, we show the clinical utility of the simulated T1w low-dose images using downstream tasks. To the best of our knowledge, this is the first DL based MRI dose simulation approach.

Iterative learning design: DL based models tend to perform poorly when the training data is highly imbalanced [11]. Furthermore, the problem of arbitrary dose simulation requires the interpolation of intermediate dose-levels using a minimum number of data points. Iterative models [12, 13] are suitable for such applications as they work on the terminal images to generate step-wise intermediate solutions. We first utilize this design paradigm for the dose simulation task and train an end-to-end model on a highly imbalanced dataset where only T1w pre-contrast, T1w low-dose, and T1w post-contrast are available.

As shown in Fig. 1(a), the proposed iterative model $\boldsymbol{\mathcal{G}}=\boldsymbol{\mathcal{F}}\circ\boldsymbol{\mathcal{F}}\circ\cdots\circ\boldsymbol{\mathcal{F}}$ learns a transformation from the post-contrast to the low-dose image in $k$ iterations, where $\boldsymbol{\mathcal{F}}$ represents the base model. At each iteration $i$, the higher enhancement image from the previous step and the pre-contrast images are fed into $\boldsymbol{\mathcal{F}}$ to predict the image with lower enhancement. The iterative model can be formulated as follows,

where $\mathbf{P_{pre}}$, $\mathbf{P_{post}}$, and $\widehat{{\mathbf{P_{low}}}}$ denote the pre-contrast, post-contrast, and predicted low-dose images, respectively and $\widehat{\mathbf{P_{i-1}}}$ denotes the image with a higher enhancement than $\widehat{\mathbf{P_{i}}}$. This way, the intermediate outputs $\{\widehat{\mathbf{P_{i}}}\}^{k}_{i=1}$ having different enhancement levels, correspond to images with different contrast dose level with a uniform interval. This iterative model essentially learns a gradual dose reduction process, in which each iteration step removes a certain amount of contrast enhancement from the full-dose image.

Loss functions and model convergence: The proposed iterative model aims to learn a mapping from the post-contrast & pre-contrast images to the synthesized low-dose images $\widehat{{\mathbf{P_{low}}}}$ and is trained with the true 10% low-dose image ${\mathbf{P_{low}}}$ as the ground truth. We used the $\mathbf{L_{1}}$ and structural similarity index measure (SSIM) losses. To tackle the problem of gradient explosion or vanishing, "soft labels" are generated using linear scaling. These "soft labels" serve as a reference to the intermediate outputs during the iterative training process and also aid model convergence, without which the model has to directly learn from post-contrast to low-dose. Given $k$ iterations, the soft label $\{\boldsymbol{\mathcal{S}_{i}}\}^{k-1}_{i=1}$ for iteration $i$ is calculated as follows:

where $\widetilde{\mathbf{P_{post}}}$ and $\widetilde{\mathbf{P_{pre}}}$ denote the skull-stripped post-contrast and pre-contrast images. $\gamma=0.1$ represents the dose level of the final prediction, and $\tau=0.1$ denotes the threshold to extract the estimated contrast uptake $\boldsymbol{\mathcal{U}}=\mathrm{ReLU}(\widetilde{\mathbf{P_{post}}}-\widetilde{\mathbf{P_{pre}}}-\tau)$. Finally, the total losses are calculated as

Where $\boldsymbol{\mathcal{L}_{\mathrm{e}}}=\boldsymbol{\mathcal{L}_{\mathrm{L1}}}+\boldsymbol{\mathcal{L}_{\mathrm{SSIM}}}$ and $\alpha=0.1$ and $\beta=1$. The "soft labels" are assigned a small loss weight so that they do not overshadow the contribution of the real low-dose image. Additionally, in order to recover the high frequency texture information and to improve the overall perceptual quality, adversarial [14] and perceptual losses [15] are applied on $(\widehat{{\mathbf{P_{low}}}},{\mathbf{P_{low}}})$ with a weight of $0.1$.

Global transformer (Gformer): Transformer models have risen to prominence in a wide range of computer vision applications [10, 16]. Traditional Swin transformers compute attention on non-overlapping local window patches. To further exploit the global contrast information, we propose a hybrid global transformer (Gformer) as a backbone for the dose simulation task. As illustrated in Fig. 1(b), the proposed model design includes six sequential Gformer blocks as the backbone module with shortcuts. As shown in Fig. 1(c), the Gformer block contains a convolution block, a rotational shift module, a sub-sampling process, and a typical transformer module. The convolution layer extracts granular local information of the contrast uptake while the self-attention emphasizes more on the coarse global context, thereby paying attention to the overall contrast uptake structure.

Subsampling attention: The sub-sampling is a key element in the Gformer block which generates a number of sub-images from the whole image as attention windows as shown in Fig 2. Gformer performs self-attention on the sub-sampled images, which encompasses global contextual information with minimal self-attention overhead on small feature maps. Given the entire feature map $\mathbf{M_{e}}\in\mathbb{R}^{b\times c\times h\times w}$, where $b,c,h$, and $w$ are the batch size, channel dimension, height, and width, respectively, the subsampling process aggregates the strided positions to the sub-feature maps as follows,

where $d$ denotes sampling a position every $d$ pixels, and $\mathbf{M_{s}}\in\mathbb{R}^{b\times c\times\frac{h}{d}\times\frac{h}{d}}$ is the subsampled feature map. We set $h,d=0$ to avoid any information loss during subsampling. These $d^{2}$ sub-feature maps are stacked onto the batch dimension as the attention windows for the transformer block shown in Fig 1(c).

Dataset: With IRB approval and informed consent, we retrospectively used 126 clinical cases (113 training, 13 testing) from a internal private dataset^dddThe dataset used in this study was obtained from Site A and B and are not available under a data sharing license using Gadoterate meglumine contrast agent (Site A). For downstream task assessment we used 159 patient studies from another site (Site B) using Gadobenate dimeglumine. The detailed cohort description is given in Table 1. The clinical indications for both sites included suspected tumor, post-op tumor follow-up and routine brain. For each patient, 3D T1w MPRAGE scans were acquired for the pre-contrast, low-dose, and post-contrast images. These paired images were then mean normalized and affine co-registered (pre-contrast as the fixed image) using SimpleElastix [17]. The images were also skull-stripped, to account for differences in fat suppression, using the HD-BET brain extraction tool [18] for generating the "soft labels".

Implementation details: All experiments were conducted with a single Tesla V100-SXM2 32GB GPU on a Intel(R) Xeon(R) CPU E5-2698 v4. The Subsampling stride for the six levels of Gformer block were {4,8,16,16,8,4}. The Rotational shift angles were {0,10,20,20,10,0} across all blocks. The model was optimized using the Adam optimizer with an initial learning rate of 1e-5 and a batch size of 4.

1) Low-dose to full-dose synthesis Using the DL-based algorithm to predict full-dose image from pre-contrast and low-dose images described in [5], we synthesized T1CE volumes using true low-dose (T1CE-real-ldose) and Gformer (rot) synthesized low-dose (T1CE-synth-ldose). We computed the PSNR and SSIM metrics of T1CE vs T1CE-synth/T1CE vs T1CE-synth-sim which are $29.82\pm 3.90$ dB/$28.10\pm 3.20$ dB and $0.908\pm 0.031$/$0.892\pm 0.026$ respectively. This shows that the synthesized low-dose images perform similar^eeep ¡ 0.0001 (Wilcoxon signed rank test) to that of the low-dose image in the dose reduction task. For this analysis we used the data from Site B.

2) Tumor segmentation Using the T1CE volumes synthesized in the above step, we perform tumor segmentation using the winning solution of BraTS 2018 challenge [26]. Let $\mathbf{M}_{true}$, $\mathbf{M}_{ldose}$ and $\mathbf{M}_{ldose-sim}$ be the whole tumor (WT) masks generated using T1CE, T1CE-real-ldose and T1CE-synth-ldose (+ T1, T2 and FLAIR images) respectively. The mean Dice scores $Dice(\mathbf{M}_{true},\mathbf{M}_{ldose})$ and $Dice(\mathbf{M}_{true},\mathbf{M}_{ldose-sim})$ on the test set were $0.889\pm 0.099$ and $0.876\pm 0.092$ respectively. Fig. 6 shows visual examples of tumor segmentation performance. This shows that the clinical utility provided by the synthesized low-dose is similar^fffp ¡ 0.0001 (Wilcoxon signed rank test) to that of the actual low-dose image.

We have proposed a Gformer-based iterative model to simulate low-dose CE images. Extensive experiments and downstream task performance have verified the efficacy and clinical performance of the proposed model compared to other state-of-the art methods. In the future, further reader studies are required to assess the diagnostic equivalence of the simulated low-dose images. The model can be guided using physics-based models [27] that estimate contrast enhancement level using signal intensity. This simulation technique can easily be extended to other anatomies and contrast agents.