A Unified Low-level Foundation Model for Enhancing Pathology Image Quality

Super resolution refers to computational techniques that reconstruct high-resolution images from low-resolution inputs by recovering lost high-frequency details and fine structures. In digital pathology, this process enhances the visibility of diagnostically critical features that may be obscured due to limitations in scanning resolution or image acquisition conditions. The ability to faithfully reconstruct these microscopic details is particularly important because pathologists routinely examine tissue specimens at multiple magnification levels, where fine cellular and subcellular features directly inform diagnostic decisions. Therefore, it is important to evaluate the super-resolution abilities of different models. In this section, we conducted experiments on a total of 18 tasks, including 9 internal tests and 9 external tests. The detailed experimental results are presented in Extended Data (Tab. 1-6). More generated samples are shown in Extended Data (Fig. 14).

To simulate realistic image degradation for super-resolution evaluation, we generated low-resolution counterparts by downscaling original high-resolution pathology images by factors of 2$\times$, 4$\times$, and 8$\times$ using the comprehensive degradation model detailed in Sec. 4.1.2. For internal validation, we employed three benchmark datasets (CAMELYON16[38], PANDA[39], and PAIP2020[40]), which were rigorously partitioned into training (70%), validation (10%), and test (20%) sets with no data overlap to ensure unbiased evaluation. All model comparisons were performed exclusively on the held-out test sets under standardized conditions. To further validate generalization capability, we incorporated external test datasets (TIGER2021, MIDOG2022 and OCELOT[41]) representing diverse tissue types, staining protocols, and scanner variations, providing robust assessment across different clinical scenarios and imaging conditions.

To comprehensively evaluate the performance of different methods across multiple super-resolution tasks, we conducted a thorough ranking analysis where each of the 18 super-resolution tasks was ranked according to all three metrics (Fig. 2a). Our proposed method demonstrated superior and consistent performance across the comprehensive evaluation. In the 18 super-resolution tasks assessed by three metrics, our approach achieved an outstanding average ranking of 1.33 across all evaluation criteria (Fig. 2a). More impressively, in 15 out of these 18 tasks, our method simultaneously secured either first or second place rankings in all three metrics (PSNR, SSIM, and LPIPS). Specifically, the LPFM attained remarkable quantitative results with average values of 30.27 dB (PSNR), 0.85 (SSIM), and 0.1647 (LPIPS) across all tasks, surpassing the second-best methods by significant margins of 4.14 dB and 0.12 in PSNR and SSIM, respectively (Fig. 2b-d).

Furthermore, we showed some generated visual samples of various methods in internal and external datasets (Fig. 2g-h). The mean absolute error (MAE) of the generated images and GT images was computed to evaluate the visual quality of the generated images. LPFM achieved the lowest MAE for the samples in internal and external samples. Additionally, we analyzed the intensity profiles of the generated samples in internal and external datasets produced by the top four best-performance models alongside the GT images (Fig. 2e-f). To better validate the fidelity of reconstructed details, we conducted pixel-wise Pearson Correlation analysis [42] between the generated images and GT images. The results demonstrated that LPFM achieved the highest correlation coefficient. As shown in Fig. 2e-f, the intensity curve of LPFM-generated images exhibited nearly perfect alignment with the GT profile, particularly in preserving critical high-frequency components that correspond to cellular boundaries and nuclear details. The quantitative correlation analysis, combined with our previous metric evaluations, provided comprehensive evidence that LPFM delivered both perceptually convincing and accurate super-resolution results for pathology images.

In computational pathology, effective deblurring enhances critical diagnostic features by restoring sharp boundaries and fine cellular details that may be lost due to optical limitations or focus variations during slide scanning. This capability directly impacts diagnostic accuracy in applications such as tumor margin assessment and mitotic figure detection.

Our evaluation framework comprised 18 deblurring tasks (9 internal and 9 external tests), with complete results available in Extended Data (Tab. 13-18). We simulated clinically relevant blur conditions using Gaussian kernels with varying parameters (kernel sizes: 7-15, $\sigma_{1},\sigma_{2}$: 1.5-3.5) as detailed in Sec. 4.1.2. The same rigorous dataset partitioning (CAMELYON16, PANDA, PAIP2020) and external validation protocol (OCELOT, MIDOG2022, TIGER2021) described in Sec. 2.1 were maintained. More generated samples are shown in Extended Data (Fig. 15).

The statistical analysis showed that LPFM achieved the best average ranking scores across PSNR, SSIM and LPIPS metrics for the 18 deblurring tasks (Fig. 3a). Specifically, LPFM ranked among the top two methods in all three metrics for 16 out of 18 tasks (84.2% of cases), demonstrating remarkable consistency across different evaluation criteria. Notably, LPFM achieved the highest PSNR values in all 18 tasks, with an average score of 27.36 dB that significantly outperformed the second-best method (24.17 dB, +3.19 dB improvement). For structural similarity assessment, LPFM maintained superior performance with SSIM scores consistently leading all comparison methods across various blur conditions (Fig. 3c). The average SSIM of 0.770 substantially exceeded competing approaches, with particularly notable advantages in challenging cases involving large kernel sizes where LPFM achieved up to 0.948 SSIM versus 0.824-0.845 for other methods. Perceptual quality evaluation through LPIPS further confirmed LPFM’s advantages (Fig. 3d), with an average score of 0.220 representing 31.5% reduction in perceptual distance compared to traditional methods (CycleGAN: 0.293) and 17.3% improvement over recent diffusion-based approaches (HistoDiff: 0.266).

Visual comparisons presented in Fig. 3g-h showed LPFM consistently producing sharper cellular boundaries and better-preserved nuclear details compared to other methods. Quantitative analysis revealed LPFM achieved the lowest MAE across internal and external datasets, with internal samples showing 7.02 (versus 19.45 for the second-best method) and external datasets demonstrating 14.48 (versus 15.41 for the second-best method). The evaluation of structural consistency through the pixel-level PCC analysis further confirmed LPFM’s exceptional performance, achieving 0.987 on internal datasets and maintaining strong generalization with 0.953 on external datasets (Fig. 3e-f). These results collectively validated LPFM’s capability to faithfully restore diagnostically critical features while maintaining structural consistency.

Noise corruption in pathology images presents a significant challenge for both clinical diagnosis and computational analysis. Multiple factors introduce noise, including electronic sensor limitations during slide scanning, uneven staining artifacts, tissue preparation inconsistencies, and optical imperfections in imaging systems. These noise patterns obscure critical cellular and subcellular features essential for accurate diagnosis, including nuclear membrane integrity, chromatin distribution patterns, and subtle morphological characteristics. Therefore, it is necessary to remove possible noise inside pathology images and restore high-quality images for downstream tasks. To evaluate the noise restoration performance of various methods, we conducted 18 denoising experiments, including 9 internal and 9 external tests. The detailed experimental results are presented in Extended Data (Tab. 7-12). More generated samples are shown in Extended Data (Fig. 16).

To evaluate denoising performance, we generated synthetic datasets by corrupting high-quality pathology images with Gaussian noise at varying intensities ($\sigma$ = 21, 31, 41), following the degradation model in Sec. 4.1.2. We used the same internal datasets (CAMELYON16, PANDA, PAIP2020) with a 7:1:2 train/val/test split and further validated generalization on external datasets (OCELOT, MIDOG2022, TIGER2021).

The ranking analysis of 18 denoising tasks revealed the superior performance of LPFM (Fig. 4a), which achieved an outstanding average ranking score of 1.48, significantly outperforming the second-best method SwinIR (average ranking 2.65) by 1.17. LPFM ranked among the top two methods in all three evaluation metrics (PSNR, SSIM, and LPIPS) for 14 out of the 18 tasks, showcasing its remarkable robustness in balancing different aspects of image quality. The substantial performance gap between LPFM and competing methods was particularly evident in high-noise scenarios ($\sigma$=41) where LPFM maintained superior detail preservation while effectively suppressing noise artifacts.

The comprehensive evaluation across all 18 denoising tasks revealed an interesting performance landscape among the compared methods (Fig. 4b-d). While LPFM did not achieve the highest scores in PSNR (SwinIR led with 27.02 dB average) or LPIPS (HistoDiff led with 0.172 average), it demonstrated exceptional balance across all three metrics (PSNR, SSIM, and LPIPS). Crucially, LPFM’s superior SSIM (average 0.837), which measures structural similarity, indicated that it best maintained critical tissue and cellular details essential for pathological diagnosis, even if its PSNR or LPIPS was marginally lower than the best performing methods. This balance is vital in medical imaging where over-smoothing (high PSNR but loss of detail) or perceptual artifacts (good LPIPS but unnatural textures) can compromise diagnostic accuracy.

Visual and quantitative analysis of denoising performance was provided in Fig. 4g-h. In terms of MAE metric, LPFM showed differentiated performance across internal and external datasets. On internal test sets, LPFM achieved an MAE of 9.56, slightly higher than SwinIR’s 7.76. However, in external validation datasets, LPFM showed superior generalization with the lowest MAE of 8.14, outperforming SwinIR’s 8.52. Structural consistency analysis through the pixel-level PCC (Pearson Correlation Coefficient) showed similar trends. On internal data, LPFM achieved a high PCC of 0.906 (second to SwinIR’s 0.922), while on external data it reached 0.953 - nearly identical to SwinIR’s 0.954. These results indicated that both methods preserved local structures well, with SwinIR having a marginal advantage on familiar data distributions. While SwinIR achieved strong quantitative scores, its outputs frequently exhibited over-smoothing artifacts that erased diagnostically important cellular details and tissue textures. In contrast, LPFM maintained superior perceptual quality, preserving nuclear boundaries and chromatin patterns, even if some pixel-level metrics showed slight disadvantages.

Virtual staining, enabled by AI models, offers a transformative approach in pathology by digitally replicating the appearance of chemically stained tissue samples without the need for physical dyes. This technology significantly accelerates diagnostic workflows, generating high-quality stained images in minutes rather than the hours or days required for traditional chemical staining methods, while also reducing costs associated with reagents and laboratory labor.

For the virtual staining tasks, we employed multiple paired staining datasets, including AF2HE [43], HE2PAS, and HEMIT [44] datasets, to rigorously validate the performance of LPFM and various compared methods. Each dataset served a distinct purpose: AF2HE evaluated the model’s ability to transform autofluorescence (AF) images into H&E stains, crucial for rapid preliminary diagnostics; HE2PAS assessed the conversion between H&E and Periodic Acid-Schiff-Alcian Blue (PAS-AB) stains, important for detecting glycoproteins and mucins in conditions like kidney and liver diseases; and HEMIT tested the model’s capability to predict multiplex immunohistochemistry (mIHC) staining from H&E, enabling advanced biomarker analysis without repeated physical staining. These datasets were selected to cover diverse staining modalities and clinical scenarios, ensuring robust validation across different tissue structures and diagnostic needs. The detailed experimental results are presented in Extended Data (Tab. 19-21). More generated samples are shown in Extended Data (Fig. 18-20).

To comprehensively evaluate our approach, we compared LPFM against several widely used methods for virtual staining tasks, each with distinct architectures and advantages. CycleGAN, an unsupervised generative adversarial network (GAN), excelled in unpaired image-to-image translation through its cyclic consistency loss, making it suitable when strictly paired training data was unavailable. Pix2Pix, a conditional GAN, leveraged paired data for precise pixel-to-pixel translation, offering superior performance in scenarios where exact input-output alignments were critical. HER2 specialized in histopathology image translation by incorporating hierarchical feature extraction, enhancing structural preservation in complex tissue architectures. RegGAN introduced a registration-based loss to improve spatial alignment between input and output images, particularly beneficial for maintaining morphological accuracy in virtual staining tasks. Lastly, Latent Diffusion Models (LDM) employed a denoising diffusion process in latent space, combining the generative power of diffusion models with computational efficiency.

Our proposed LPFM demonstrated superior performance across all three virtual staining datasets (AF2HE, HE2PAS, and HEMIT) when compared to existing methods, as evidenced by both quantitative metrics and qualitative assessments. Quantitatively, LPFM achieved the highest average PSNR values (Fig. 5a-c). Specifically, in the AF2HE task, LPFM achieved a PSNR of 27.81 dB, representing a 5.3% improvement over the second-best method RegGAN (26.42 dB). The superior performance was further confirmed in the HE2PAS and HEMIT tasks where LPFM attained PSNR values of 19.29 dB and 26.51 dB respectively, corresponding to 4.1% and 3.0% improvements over RegGAN.

The architectural innovations in LPFM yielded substantial benefits in both structural preservation and perceptual quality. For structural similarity, LPFM achieved SSIM scores of 0.763, 0.563, and 0.820 across the three tasks, outperforming the second-best methods by 4.5%, 33.8%, and 10.7% respectively. The perceptual quality metrics (LPIPS) showed consistent advantages, with LPFM demonstrating 20.1%, 7.0%, and 20.0% reductions in perceptual error compared to the leading alternatives for each task.

Qualitatively, LPFM-generated virtual stains (Fig. 5d-f) exhibited remarkable fidelity to chemical staining results, with significantly reduced artifacts and better preservation of critical diagnostic features compared to other methods. The MAE heatmaps revealed that LPFM produced the smallest errors in challenging regions such as cell nuclei boundaries (H&E), glomerular basement membranes (PAS-AB), and biomarker expression patterns (mIHC). The intensity profile analyses (Fig. 5g-i) further confirmed this advantage, with LPFM showing the highest Pearson correlation coefficients (PCC) with ground truth stained images, particularly in capturing fine-grained histological features.

Real-world pathology images often suffer from multiple coexisting degradations, including blur, noise, and low resolution. While existing methods perform well on single degradation types, their effectiveness significantly decreases when handling such composite cases. This performance drop primarily occurs because optimizing for one degradation type may interfere with addressing others, and models trained on isolated degradations fail to capture the complex interactions present in actual clinical images.

To rigorously evaluate model robustness under such challenging conditions, we constructed a comprehensive degradation framework by applying multiple sequential distortions to high-quality pathology images from our datasets. These distortions included randomized combinations of Gaussian blur, Poisson noise, and low resolution with parameters carefully sampled to reflect real clinical imaging conditions. The degradation process preserved the biological relevance of the images while introducing realistic artifacts.

We evaluated our method using a rigorous two-tier validation strategy. First, internal testing was conducted on held-out datasets from the training distribution, including CAMELYON16, PAIP2020 and PANDA. Second, external evaluation was performed on completely independent datasets, including MIDOG2022, TIGER2021 and OCELOT, to assess generalization capability. The detailed experimental results are presented in Extended Data (Tab. 22-23). More generated samples are shown in Extended Data (Fig. 17).

As shown in Fig. 6a-c, the quantitative results demonstrated LPFM’s consistent advantages over competing approaches. In terms of PSNR, LPFM achieved a mean score of 26.15 dB, outperforming the second-best method SwinIR (24.05 dB) by a significant margin of 2.10 dB. This improvement was particularly notable in the OCELOT dataset where LPFM reached 28.20 dB, suggesting exceptional generalization capability to diverse tissue types. The SSIM results further confirmed this trend, with LPFM (0.720) substantially exceeding Pix2Pix (0.642) and other methods, indicating better structural preservation. LPFM maintained this performance advantage across all six datasets, demonstrating remarkable robustness to different degradation patterns and tissue characteristics.

LPIPS metrics, which assess perceptual quality, revealed additional insights. While HistoDiff achieved competitive LPIPS scores (0.138), LPFM demonstrated more balanced performance across all quality metrics. This suggested that while some methods might optimize for specific aspects of image quality, LPFM successfully maintained excellence in both pixel-level accuracy (PSNR) and perceptual similarity (LPIPS). The detailed experimental results are presented in Extended Data (Tab. 22-23).

Visual analysis of the results (Fig. 6d-e) corroborated these quantitative findings. Compared to other methods, LPFM better preserved nuclear details and tissue architecture while more effectively suppressing artifacts. This was especially evident in the error maps where LPFM showed minimal deviation from ground truth. The intensity profile (Fig. 6f-g) comparisons further demonstrated LPFM’s accuracy in maintaining original image characteristics, with Pearson correlation coefficients consistently exceeding 0.9 across all test cases.

Clinical histopathology workflows frequently encounter degraded tissue specimens due to suboptimal staining, sectioning artifacts, or imaging imperfections. As illustrated in Fig. 7a-d, these degradations significantly impact virtual staining quality, motivating our rigorous evaluation of method robustness. Our experiments simulated realistic degradation scenarios by applying compound artifacts to H&E images from both HE2PAS and HEMIT datasets, then processing them through pretrained models without architectural modifications.

Our experimental design simulated this clinical challenge through a two-stage process (Fig. 6a-d). First, we applied coupled degradations (blur: $\sigma_{1},\sigma_{2}$=2.5, noise: $\sigma$=31, and 4$\times$ downsampling) to high-quality H&E images from HE2PAS and HEMIT datasets. These degraded inputs then underwent virtual staining to PAS-AB and mIHC respectively, without any intermediate restoration steps. This direct transformation tested the model’s ability to simultaneously address staining conversion and artifact suppression.

The quantitative results in Figure 7e-h demonstrated LPFM’s exceptional stability across degradation conditions. On HE2PAS (Figure 7e), LPFM maintained a narrow PSNR distribution (17.94-18.24 dB) for degraded inputs, outperforming RegGAN’s wider range (12.68-13.09 dB) while preserving superior mean PSNR (18.09 dB vs 12.89 dB). This robustness became more pronounced in Fig. 7f where LPFM showed merely 6.2

The HEMIT dataset results (Figure 7g) revealed an even more striking advantage - LPFM actually achieved higher PSNR on degraded inputs (26.99 dB) than on high-quality ones (26.49 dB), suggesting its unique capability to compensate for certain artifacts. As shown in Figure 7h, this inverse relationship contrasted sharply with other methods’ expected performance degradation, particularly HER2’s 3.1% drop. Visual comparisons in Figure 7c-d confirmed that LPFM successfully suppressed noise while preserving critical histological features in mIHC staining, whereas CycleGAN introduced false positive signals and Pix2Pix lost structural detail.

To ensure the robustness and generalizability of our model on image restoration and virtual staining tasks, we collected a comprehensive, multi-source dataset encompassing diverse tissue types and staining protocols. Our dataset includes H&E (Hematoxylin and Eosin), IHC (Immunohistochemistry), mIHC (multiplex immunohistochemistry) images, and PAS (Periodic Acid-Schiff) stained slides from multiple organs (e.g., liver, kidney, breast, and lung) (Extended Data Tab. 27).

The objective of this work is to develop a unified low-level pathology foundation model for image restoration and virtual staining. To ensure robust interactivity and enable precise control across multiple tasks, our model employs a prompt-based conditioning mechanism, which dynamically guides the generation process toward the desired output modality (e.g., restoration, virtual staining, or coupled degradation reversal).

Our framework employs a two-stage training approach to achieve high-fidelity pathology image generation through progressive refinement. In the first stage, a contrastive learning-based autoencoder learns to extract consistent low-level features and produce coarse restorations guided by task-specific prompts. Building upon this foundation, the second stage trains a denoising diffusion model that takes both the coarse reconstruction and prompt embedding as inputs to generate detailed, high-quality outputs. The diffusion model progressively refines the image through an iterative denoising process, while maintaining anatomical consistency from the coarse input and adhering to task-specific requirements through prompt conditioning. This hierarchical approach allows our unified model to first capture global structural information and then synthesize precise local details, enabling flexible performance across diverse tasks, including image restoration and virtual staining, without requiring architectural modifications. The diffusion model’s conditioning on both the initial reconstruction and textual prompts ensures that the final output not only preserves the input’s structural integrity but also accurately reflects the desired transformation specified by the prompt, whether it involves stain conversion, artifact removal, or resolution enhancement.

During the inference stage, our unified low-level pathology foundation model takes a user-defined prompt and noisy image as input to generate high-quality image through a controlled reverse diffusion process. The pretrained encoder first processes the textual or embedding-based prompt to extract task-specific conditioning signals, while the diffusion model progressively denoises the initial random noise across multiple timesteps. This iterative refinement enables the synthesis of both structurally accurate and task-aligned results - whether restoring low-quality H&E pathology images (prompt: "Obtain high-quality H&E pathology image") or generating specific stains (prompt: "Synthesize H&E staining") - while maintaining tissue-level consistency.

In the pre-training phase, we aim to pretrain a low-level autoencoder that can extract consistent low-level features and produce coarse restorations guided by task-specific prompts. Following LDM[28] and CLIP[45], we pretrain the KL-Autoencoder as the low-level pathology image autoencoder to generate the coarse restorations and directly use the CLIP[45] text encoder to obtain the textual features for guidance.

Given the WSIs, we firstly pre-process the WSIs into 256$\times$256 pathology patches as early mentioned in sec. 4.1.1. As shown in Extended Data Fig. 8a, we adopt a unified training paradigm for low-level pathology tasks by leveraging contrastive learning to capture shared feature representations across different staining protocols and image quality levels. This approach enables the model to learn robust latent embeddings that are invariant to degradation types and staining variations in histopathology images. The contrastive loss operates in the latent space, pulling together features from different views (e.g., degraded/restored or differently stained versions) of the same tissue while pushing apart features from different tissue samples:

where $x$ and $x^{+}$ are positive pairs (different views of same tissue), $x^{-}$ are negative samples (different tissues), $\mathcal{E}(\cdot)$ is the feature encoder, and $\tau$ is a temperature parameter.

For high-quality reference images in restoration tasks and target-stain images in virtual staining tasks, the KL-autoencoder is optimized to reconstruct the original input through the objective:

where $\mathcal{E}$ and $\mathcal{D}$ represent the encoder and decoder respectively. Simultaneously, when processing degraded inputs for restoration or source-stain images for virtual staining, the model is trained to generate enhanced outputs that approximate the target distribution using a composite loss function:

where $x_{p}$ represent the paired degraded or source-stained pathology images. The reconstruction loss ensures pixel-level accuracy by minimizing absolute differences between generated and target images. However, pixel-wise losses alone often produce blurry results, so we incorporate a perceptual loss (LPIPS) [37] that operates in the feature space of a pretrained VGG network [46]:

where $\psi_{l}(\cdot)$ denotes features from layer $l$ of a pretrained VGG network, and $H_{l}/W_{l}$ are spatial dimensions of the feature maps.

To achieve more realistic quality for pathology images, we employ an adversarial loss that aligns the generated image distribution with original pathology images through a discriminator.

where discriminator $D$ is trained to distinguish between real ($y$) and generated ($D(E(x))$) images. Therefore, the final pretraining loss can be denoted as:

The textual prompts encoded by the CLIP text encoder are injected into the encoder through cross-attention layers, providing explicit guidance to steer the reconstruction toward the desired output. This integrated approach ensures the autoencoder learns semantically meaningful representations while maintaining flexibility across diverse pathology image processing tasks.

Building upon the pretrained autoencoder’s coarse restorations, we implement a conditional diffusion model to recover fine pathological details through noise-to-image synthesis. As shown in Extended Data Fig. 8b, we aim to remove the additive noises in the pathology images with the coarse image generation and textual prompts.

Our framework builds upon diffusion probabilistic models, which learn a target data distribution $p(x)$ through iterative denoising of normally distributed variables. These models formulate the generation process as the reverse of a fixed Markov chain spanning $T$ timesteps, effectively implementing a sequence of denoising autoencoders $\epsilon_{\theta}(x_{t},t);t=1\dots T$. Following the standard controllable diffusion framework[47], we train a U-Net to iteratively denoise corrupted versions of high-quality pathology images. For a target image $x_{0}$ progressively noised to $x_{t}$ at timestep $t$, the model is trained to predict and remove noise from corrupted versions $x_{t}$ of input images $x$, following the reweighted variational lower bound objective:

with t uniformly sampled from ${1,...,T}$.

In our diffusion model development, we adopt a two-phase training strategy to ensure robust noise modeling and precise conditional control. First, we pretrain a standard denoising diffusion model without any conditional inputs, optimizing the baseline objective from Eq. 11. This initial phase establishes fundamental denoising capabilities for pathology images. After convergence, we freeze these parameters and introduce a trainable controllable module that shares the U-Net[48] encoder with the fixed diffusion model. The complete architecture then processes both the coarse reconstruction $z$ and textual prompt embedding $c$ through and minimize the joint optimization:

where $\epsilon_{\theta}(x_{t},t,z,c)$ represents the predicted noised inside the image from the diffusion model. It should be noted that all the denoised process could be directly applied in the latent space of the pretrained autoencoder from the first stage. This latent diffusion approach can reduce computational costs as the dimensionality of the latent space is much lower than that of the original image space.

In the inference stage, our model synthesizes high-quality pathology images through an iterative denoising process that combines coarse image reconstructions and textual prompts in Fig. 9. The generation begins with pure Gaussian noise $x_{T}\sim\mathcal{N}(0,\mathbf{I})$ and progressively refines it over T timesteps using the trained diffusion model conditioned on both the encoded coarse image $z$ and the prompt embedding $c$. At each timestep t, the model predicts the noise component using:

and updates the image estimate through:

where $\alpha_{t}$ defines the noise schedule and $z\sim\mathcal{N}(0,\mathbf{I})$ for $t>1$. This process maintains anatomical fidelity through the z-conditioning while allowing precise control over stain characteristics and artifact correction via the prompt c. The complete inference typically converges in 50-100 steps using the DDIM scheduler, producing outputs that balance clinical utility with prompt-specific transformations. Our experiments demonstrate that this approach successfully generates diagnostically valid images while adhering to diverse transformation goals specified through textual prompts.

To quantitatively evaluate the performance of different methods in image restoration and virtual staining tasks, we employ three popular metrics: Peak Signal-to-Noise Ratio (PSNR) [35], Structural Similarity Index Measure (SSIM) [36], and Learned Perceptual Image Patch Similarity (LPIPS) [37]. These metrics assess different aspects of image quality, including pixel-level fidelity, structural similarity, and perceptual consistency.

To make it clear, a good generated image should have high PSNR, high SSIM, and low LPIPS. High PSNR suggests strong pixel-wise accuracy but does not guarantee visually pleasing results. High SSIM indicates better structural preservation, such as edges and textures. Low LPIPS reflects superior perceptual quality, meaning the generated image is more realistic to human observers. By combining these metrics, we comprehensively assess the performance of different methods in terms of pixel fidelity, structural consistency, and perceptual quality.

To thoroughly evaluate the performance of our proposed LPFM framework, we conduct extensive comparisons with eight state-of-the-art methods that represent the current spectrum of approaches in computational pathology image enhancement. These baseline methods encompass three major architectural paradigms: (1) generative adversarial networks (CycleGAN, Pix2Pix, BSRGAN, RegGAN), (2) transformer-based models (SwinIR), and (3) diffusion models (LDM, HistoDiff), along with (4) specialized hierarchical networks (HER2). This diverse selection enables us to assess LPFM’s advantages across different architectural designs and application scenarios, from general image-to-image translation to pathology-specific enhancement tasks. Each comparator was carefully selected based on its demonstrated effectiveness in either medical image analysis or general computer vision tasks with potential pathology applications.

The code will be available on Github(https://github.com/ziniBRC/LPFM).

This project has been reviewed and approved by the Human and Artefacts Research Ethics Committee (HAREC) of Hong Kong University of Science and Technology. The protocol number is HREP-2024-0429.

Z.L. and H.C. conceived and designed the work. Z.L., Z.X., J.M. and W.L. curated the data included in the paper. Z.X. and Z.L. contributed to the technical implementation of the LPFM framework and performed the experimental evaluations. Z.L, J.H., F.H., and X.W. wrote the manuscript with inputs from all authors. R.C.K.C. supplied data and offered medical guidance. T.T.W.W. provided autofluorescence data. All authors reviewed and approved the final paper. H.C. supervised the research.

This work was supported by the National Natural Science Foundation of China (No. 62202403), Innovation and Technology Commission (Project No. MHP/002/22 and ITCPD/17-9), Research Grants Council of the Hong Kong Special Administrative Region, China (Project No: T45-401/22-N) and National Key R&D Program of China (Project No. 2023YFE0204000).