Zero-shot denoising via neural compression:
Theoretical and algorithmic framework

Denoising is a fundamental problem in classical signal processing and has recently gained renewed attention from the machine learning community. Let ${\bm{x}}=(x_{1},\ldots,x_{n})\in\operatorname{\mathbb{R}}_{+}^{n}$ denote a non-negative signal of length $n$, where signal ${\bm{x}}$ is not observable in many systems. Instead, we observe a noisy version ${\bm{y}}=(y_{1},\ldots,y_{n})$, where the observations are conditionally independent given ${\bm{x}}$, and each entry is distributed according to a common conditional distribution:

We assume that the noise mechanism is memoryless (independent across coordinates) and homogeneous (identical across entries). The goal of a denoising algorithm is to estimate ${\bm{x}}$ from the noisy observations ${\bm{y}}$. Given its prevalence in imaging and data acquisition systems, denoising has been a central topic in signal processing for decades. Classical denoising methods rely on explicit structural assumptions about the underlying signal ${\bm{x}}$, often hand-crafted by domain experts [1, 2, 3, 4, 5, 6, 7, 8]. In contrast, recent advances in machine learning have enabled a new class of data-driven denoising algorithms. These methods learn the optimal denoising function from data, leveraging statistical patterns directly from signal and noise distributions.

While learning-based approaches achieve state-of-the-art performance and often outperform classical methods in controlled settings, they face significant challenges in practice:

1. 1.

   Supervision requirement: Most learning-based methods require training set of paired samples $\{({\bm{y}}_{i},{\bm{x}}_{i})\}_{i=1}^{m}$, where ${\bm{x}}_{i}$ is clean signal and ${\bm{y}}_{i}$ is its noisy counterpart. In practical scenarios such as medical imaging, $\{{\bm{x}}_{i}\}_{i=1}^{m}$ are unavailable or prohibitively expensive to obtain.

2. 2.

   Data efficiency: These methods usually need lots of training data. Acquiring sufficient samples is difficult or costly, particularly in domains with strict data acquisition constraints.

To mitigate the reliance on paired clean and noisy samples, several self-supervised denoising methods have been developed that learn directly from noisy observations, without access to clean ground truth signals [9, 10, 11, 12, 13]. While these approaches alleviate the supervision requirement, they typically depend on access to large collections of noisy data and often yield suboptimal performance compared to methods trained with clean targets. Moreover, the absence of clean supervision necessitates the use of complex neural architectures and training schemes, which can make these methods computationally demanding and difficult to optimize in practice.

These challenges have sparked growing interest in zero-shot denoisers, which aim to recover clean signals from noisy observations without access to paired data or extensive noisy training data. Such methods are particularly appealing in domains where acquiring clean data is infeasible, and they offer the potential for deployable denoisers that adapt to individual inputs with general purpose.

Denoising algorithms—ranging from classical signal processing techniques to deep learning methods—fundamentally rely on the assumption that real-world signals are highly structured. Compression-based denoising leverages this same principle, but rather than directly solving the inverse problem, it instead performs lossy compression on the noisy observation ${\bm{y}}$, under the hypothesis that the clean signal ${\bm{x}}$ lies in a lower-complexity subspace and is therefore more compressible.

In lossy compression, the goal is to represent signals from a target class using discrete encodings with minimal distortion. When applied to noisy data, the intuition is that a lossy compressor—operating at a distortion level matched to the noise—will favor reconstructions close to the original clean signal. While this approach has a strong theoretical foundation [14, 15], classical compression-based denoisers have shown limited empirical success, particularly for natural image denoising.

In this work, we revisit this idea in light of recent progress in neural compression, where learned encoders and decoders have demonstrated strong rate-distortion performance across a variety of image domains [16, 17]. Building on this foundation, we propose a zero-shot denoising method that we call the Zero-Shot Neural Compression Denoiser (ZS-NCD). Unlike traditional neural compression models that are trained on large corpora of clean high-resolution images, ZS-NCD learns directly from a single noisy input image. Specifically, we extract overlapping patches from the noisy image, and train a neural compression network on those patches alone—without any clean supervision or prior dataset. Once trained, the denoiser is applied to all patches from the same image, and the final output is obtained by averaging the predictions in overlapping regions. This approach is illustrated in Figure 1.

Despite relying solely on the noisy input and operating without supervision, ZS-NCD achieves state-of-the-art performance among zero-shot denoising methods across diverse noise models, and remains robust even on inputs that lie outside the natural image distribution. We compare it against the baselines in Figure 2, ZS-NCD shows superior performance in denoising and training stability.

This paper introduces a zero-shot image denoising framework based on neural compression. Our main contributions are:

• $\bullet$

  A zero-shot denoising algorithm using neural compression. We propose a fully unsupervised method that trains a neural compression network on image patches of the noisy input. It does not rely on clean images, paired datasets, or prior training on the target distribution. It is architecture-agnostic and leverages only the structure present in the observed noisy image.

• $\bullet$

  Theoretical results connecting denoising and compression performance. We establish finite-sample upper bounds on the reconstruction error of the proposed compression-based maximum likelihood denoisers, for both Gaussian and Poisson noise models.

• $\bullet$

  Extensive empirical validation. We demonstrate that our method achieves state-of-the-art performance among zero-shot denoising techniques across a range of noise models and datasets.

Supervised learning-based denoisers such as DnCNN [25] and Restormer [19] achieve state-of-the-art performance across various noise models, but require large datasets of paired clean and noisy images—often impractical in real-world settings. To avoid clean images, self-supervised methods have been proposed, including Noise2Noise [10], Noise2Self [12], Noise2Void [11], Noise2Same[26] and Noise2Score [13], which only use noisy images for training. However, their reliance on large noisy datasets remains a limitation. Zero-shot denoisers address this by training on a single noisy image. These include (i) untrained networks like DIP [27] and Deep Decoder [22], and (ii) single-image adaptations of self-supervised methods, e.g., ZS-N2N [24], ZS-N2S [12] and its augmented variant with ensembling, S2S [23]. DIP-based models avoid masking and leverage full-image context, but require early stopping or under-parameterization to avoid overfitting. Self-supervised variants suffer from masking-induced information loss. Hybrid approaches, such as masked pretraining-based method [28], uses external datasets for training and perform zero-shot inference, thus falling outside the zero-shot setting studied here.

Learning-based lossy compression, often referred to as neural compression, uses an autoencoder architecture combined with an entropy model to estimate and constrain the bitrate at the bottleneck [29, 30]. These methods have significantly outperformed traditional codecs, particularly in image [29, 31, 32, 33, 34] and video [35, 36, 37] compression. In addition to these standard settings, several works have also explored applying neural compression to noisy data, either by adapting neural compression models for more efficient encoding of noisy images [38, 39, 40, 41].

Compression-based denoising leverages the insight that structured signals are inherently more compressible than their noisy counterparts. This connection was formalized by Donoho [14], who introduced the minimum Kolmogorov complexity estimator, and further refined by Weissman et al. [15], showing that, under certain conditions on both the signal and the noise, optimal lossy compression of a noisy signal-followed by suitable post-processing-can asymptotically achieve optimal denoising performance. Prior to these theoretical developments, early empirical methods such as wavelet-based schemes [42, 43, 44] and MDL-inspired heuristics [45] had explored this principle. Nevertheless, traditional compression-based denoisers have generally underperformed in high-dimensional settings such as image denoising.

Learning-based joint compression and denoising using neural compression has been explored in recent works [46, 47], where the goal is to achieve lower rate in compression. The empirical application of training neural compression for AWGN denoising was also proposed in [48]. Training the neural compression models in these works requires a dataset of images. In contrast, our proposed ZS-NCD is a two-step denoiser based on neural compression, trained on a single noisy image. It achieves state-of-the-art performance across both AWGN and Poisson noise models. Moreover, we contribute new theoretical results that advance the foundations of compression-based denoising.

Let $\mathcal{Q}\subset\operatorname{\mathbb{R}}^{n}$ denote the signal class of interest, such as vectorized natural images of a fixed size. A lossy compression code for $\mathcal{Q}$ is defined by an encoder-decoder pair $(f,g)$, $f:\mathcal{Q}\to\{1,\ldots,2^{R}\}$, and $g:\{1,\ldots,2^{R}\}\to\operatorname{\mathbb{R}}^{n}$. The performance of a lossy code is characterized by: i) Rate $R$, indicating the number of distinct codewords; ii) Distortion $\delta$, defined as the worst-case per-symbol mean squared error (MSE) over the signal class:

The set of reconstructions produced by the decoder forms the codebook:

We propose compression-based denoising as a structured maximum likelihood (ML) estimation. Given a noisy observation ${\bm{y}}\sim\prod_{i=1}^{n}p(y_{i}\mid x_{i})$ and a a lossy compression code $(f,g)$ for $\mathcal{Q}$, the compression-based ML denoiser solves

This formulation leverages the fact that clean signals, by virtue of their structure, are more compressible than their noisy counterparts. Therefore, the most likely codeword under the noise model, when selected from a codebook designed to represent clean signals, serves as a natural denoising estimate. This ML-based view unifies denoising across noise models and provides a principled way to select reconstructions from a discrete, structure-aware prior.

In the case of AWGN: ${\bm{y}}={\bm{x}}+{\bm{z}}$, where ${\bm{z}}\sim\mathcal{N}(\mathbf{0},\sigma_{z}^{2}I_{n})$, the described denoiser simplifies to:

That is, denoising corresponds to projecting the noisy observation onto the nearest codeword.

Poisson noise commonly arises in low-light and photon-limited imaging scenarios. In this setting, each $y_{i}$ is modeled as a Poisson random variable with mean $\alpha x_{i}$: $y_{i}\sim\mathrm{Poisson}(\alpha x_{i})$. Under this model, the compression-based ML denoiser simplifies to

While the loss function in (2) is statistically well-motivated, it is more sensitive to optimization issues than its Gaussian counterpart due to the curvature and nonlinearity of the log term. To improve robustness and simplify optimization, we also consider an alternative loss based on a normalized squared error between ${\bf c}$ and the rescaled observations:

We begin by analyzing the performance of compression-based ML denoising under AWGN. The following result provides a non-asymptotic upper bound on the reconstruction error in terms of the compression rate and distortion. All proofs can be found in Appendix 6.

This bound decomposes the denoising error into two terms: a distortion term $\sqrt{\delta}$, which reflects the approximation quality of the compression code, and a rate-dependent term that scales with the square root of the code rate $R$. The latter captures the likelihood concentration around the clean signal in high-probability regions of the noise distribution. Notably, the result holds non-asymptotically and does not assume the code is optimal, only that it provides a distortion-$\delta$ covering of $\mathcal{Q}$. This highlights that even non-ideal compression codes can enable effective denoising, provided the rate-distortion tradeoff is well-calibrated.

To better understand the implications of Theorem 1, in the following corollary, we focus on the special case of $k$-sparse signals.

To contextualize this result, consider the asymptotic setting where $n\to\infty$ and $\sigma_{z}^{2}\to 0$. For an i.i.d. Bernoulli–Gaussian source with $\mathbb{P}\left\lparen x_{i}\neq 0\right\rparen=p$, the minimum mean squared error (MMSE), normalized by the noise power $\sigma_{z}^{2}$, converges to $p$ [49]. In contrast, our result is non-asymptotic and worst-case: it shows that the MSE of compression-based denoising is bounded by $C\cdot\frac{k\log n}{n}$. In the probabilistic setting where $k/n\approx p$, the two results exhibit consistent scaling with sparsity.

We next extend our analysis to signal-dependent noise model. Poisson noise is particularly relevant in imaging applications such as microscopy and astronomy, where photon counts vary with signal intensity. Unlike Gaussian noise, Poisson observations induce a non-linear likelihood surface, making analysis more delicate. Theorem 2 and 3 establish performance guarantees for compression-based Poisson denoising, using both exact ML formulation and a practical squared-error surrogate.

Let $\mathcal{P}_{(i,j)}:\operatorname{\mathbb{R}}^{h\times w}\to\operatorname{\mathbb{R}}^{k\times k}$ denote the patch extraction operator, which returns a $k\times k$ patch whose top-left corner is at pixel $(i,j)$. Let $f_{\bm{\theta}_{1}}$ and $g_{\bm{\theta}_{2}}$ denote the encoder and decoder networks, parameterized by weights $\bm{\theta}_{1}$ and $\bm{\theta}_{2}$, respectively. Define $\mathcal{I}$ as the set of all coordinates $(i,j)\in\{1,\ldots,h-k+1\}\times\{1,\ldots,w-k+1\}$ from which a valid $k\times k$ patch can be extracted.

Given a single noisy image ${\bm{y}}$, the ZS-NCD is trained to minimize the following patchwise objective:

where $\mathbb{P}\left\lparen f_{\bm{\theta}_{1}}(\mathcal{P}_{(i,j)}({\bm{y}}))\right\rparen$ denotes the likelihood (or entropy model) of the latent code produced by the encoder, and $\lambda>0$ is a hyperparameter controlling the trade-off between fidelity and compressibility. In (7), $K=k^{2}$, and the function $\mathcal{L}_{K}:\operatorname{\mathbb{R}}^{K}\times\operatorname{\mathbb{R}}^{K}\to\operatorname{\mathbb{R}}_{+}$ is a distortion loss determined by the noise model, as defined in Section 3. For example, in the AWGN case, $\mathcal{L}_{K}$ corresponds to the squared $\ell_{2}$ norm of the distance between a noisy patch and its neural compression reconstruction. Note that $f_{\bm{\theta}_{1}}$ maps the input into a discrete latent space, which is non-differentiable and thus incompatible with standard gradient-based optimization. To address this, we follow the neural compression framework of [29], using a continuous relaxation during training (e.g., uniform noise injection) and applying actual discretization only at test time. The entropy term $\mathbb{P}$ is modeled using a factorized, non-parametric density [31].

After training, the denoised image is obtained by applying the encoder and decoder to each patch and averaging the overlapping outputs. For each pixel $(i,j)$, let $\mathcal{I}_{(i,j)}\subset\mathcal{I}$ denote the set of patch locations such that $\mathcal{P}_{(i^{\prime},j^{\prime})}$ includes the pixel $(i,j)$. The final estimate at location $(i,j)$ is given by

where $|\mathcal{I}_{(i,j)}|$ denotes the number of patches covering pixel $(i,j)$, and $\left.\cdot\right|_{(a,b)}$ denotes the $(a,b)$-th pixel of the patch output. For interior pixels away from the boundary, $|\mathcal{I}_{(i,j)}|=K$. As shown later in Section 5, this aggregating of reconstructed patches significantly enhances denoising performance.

The ZS-NCD objective in (7) includes a hyperparameter, $\lambda$, which balances reconstruction fidelity and compressibility. Interpreted through the lens of lossy compression, varying $\lambda$ allows the model to explore different rate-distortion trade-offs. However, in the context of denoising, our goal is not compression but accurate signal recovery from the noisy observation ${\bm{y}}$. This raises the central question: how should $\lambda$ be selected to optimize denoising performance?

Let $\hat{\bm{x}}$ denote the output of the ZS-NCD denoiser, and consider the AWGN model ${\bm{y}}={\bm{x}}+{\bm{z}}$ with ${\bm{z}}\sim\mathcal{N}(0,\sigma_{z}^{2}I_{n})$. Then,

the first term is the noise variance, and the second is the true denoising error. While ${\bm{z}}$ and $\hat{\bm{x}}$ are not fully independent, they are intuitively weakly correlated in successful denoising regimes, where the estimate $\hat{\bm{x}}$ depends only indirectly on the noise. Thus, the cross term is expected to be small: $\frac{1}{n}\mathbb{E}\left[{\bm{z}}^{\top}({\bm{x}}-\hat{\bm{x}})\right]\approx 0.$ This approximation suggests that $\frac{1}{n}\|{\bm{y}}-\hat{\bm{x}}\|_{2}^{2}$ should be close to $\sigma_{z}^{2}$ when $\hat{\bm{x}}$ is a high-quality estimate. Based on this insight, we propose a simple and effective heuristic for choosing $\lambda$: select the value that makes $\frac{1}{n}\|{\bm{y}}-\hat{\bm{x}}\|_{2}^{2}$ closest to the known noise variance $\sigma_{z}^{2}$. This procedure can be implemented efficiently via a tree-based search strategy, as described in Algorithm 1. To apply Algorithm 1, one needs an estimate of the noise power $\sigma_{z}^{2}$. This is a well-studied problem and there exist robust algorithms for estimating the power of noise [2, 50]. For example, in [2], it is shown that the noise power can be estimated from the median of the absolute differences of wavelet coefficients. Consequently, the performance of ZS-NCD is relatively robust to the choice of $\lambda$. For instance, on the Nuclei dataset (Figure 3), ZS-NCD outperforms the state-of-the-art zero-shot learning-based denoiser, ZS-Noise2Noise, across a wide range of $\lambda$ values, for both $\sigma_{z}=10$ and $\sigma_{z}=20$. A similar approach can be used in the case of Poisson noise as well. (See Section A.2)

We consider two synthetic noise models, AWGN $\mathcal{N}(0,\sigma^{2}_{z})$, where $\sigma_{z}$ is the standard deviation of the Gaussian distribution, and Poisson noise defined as $\mathrm{Poisson}(\alpha x)$, where $\alpha$ is the scale factor. Note that Poisson noise is signal-dependent noise with $\mathbb{E}\left[{\bm{y}}\right]=\alpha{\bm{x}}$. To re-scale the noisy image to the range of clean image, we followed the literature by assuming that the the scale $\alpha$ is known, and normalize the noisy image as ${\bm{y}}/\alpha$ in the experiments of this section. We evaluate on grayscale Set11 [25], RGB Set13 [51] (center-cropped to $192\times 192$) and Kodak24 [52] datasets. Table 1 presents the denoising performance of various methods. BM3D achieves the strongest results on grayscale images, though it relies on accurate knowledge of the noise power parameter. Existing learning-based zero-shot denoisers, in contrast, often exhibit inconsistent performance across noise levels and image resolutions. For example, ZS-N2S and Self2Self degrade on high-resolution images, likely due to the limitations of training with masked pixels. ZS-N2N performs well on high-resolution images from Kodak24 but suffers on lower-resolution images in Set13 ($192\times 192$), as it is trained to map between two downscaled versions of the same noisy image. In comparison, ZS-NCD maintains robust performance across different noise levels and image sizes. The more realistic case of not having access to the noise parameter $\alpha$ is discussed in Appendix A.2. In both noise regimes, we use MSE as the loss function. However, for Poisson noise, minimizing the negative log-likelihood is also a natural choice. We defer the results using this loss to Appendix A.2.

To evaluate performance in low-data and domain-shift settings, we test ZS-NCD on Mouse Nuclei fluorescence microscopy images [53], which differ significantly from natural images in structure and texture. We also assess real-world denoising using the PolyU dataset [54], which contains high-resolution images captured by Canon, Nikon, and Sony cameras. Ground-truth images are obtained by averaging multiple captures, while the noisy inputs are single-shot acquisitions. Results are shown in Table 2. ZS-NCD consistently outperforms other learning-based zero-shot denoisers, demonstrating robustness to unknown noise models and non-natural image distributions.

Most learning-based zero-shot methods are prone to overfitting due to the lack of clean targets and the use of overparameterized networks.

In contrast, ZS-NCD, grounded in compression-based denoising theory, overcomes this issue given the entropy constraint. To further highlight this key aspect of ZS-NCD, we replace the convolutional encoder-decoder ($\approx 0.4$M params) with a fully connected MLP ($\approx 2.3$M params) and observe that, instead of degradation, the performance improves using the same $\lambda$ (see Table 3).

As described in Section 4 and illustrated in Fig. 1, ZS-NCD denoises each pixel by aggregating outputs from overlapping patches, where each patch is first compressed and then decompressed using a learned neural compression model. Intuitively, one might expect the most accurate reconstruction for a given pixel to come from the patch in which it lies at the center, as this location benefits from the largest available spatial context, which has been observed in [55].

This observation leads to the question: Does averaging over overlapping reconstructions improve denoising quality, or would it suffice to use only the patch where pixel appears at a fixed position (e.g., the center)? From a computational perspective, both strategies are equivalent, since in both methods every patch is processed, but in averaging scheme, each patch contributes to all the pixels it covers.

To investigate this, we conducted an ablation in which, instead of averaging, each pixel $(i,j)$ is reconstructed solely from one of the $k\times k$ patches in which it appears, using a fixed location in the patch (e.g., top-left, center, etc.). The results are shown in Fig. 4, where each heatmap entry reports the PSNR

obtained by using only that specific location in the patch for reconstruction. As expected, performance is best when the pixel is centrally located, and degrades as it moves toward the patch boundaries.

However, the key observation is that averaging across all overlapping reconstructions yields a substantial performance gain. For instance, in denoising Parrot (from Set11 dataset), the best single-location reconstruction achieves 25.90 dB (center), while averaging achieves 28.14 dB, a gain of over 2 dB. This highlights the denoising benefit of combining multiple noisy views of each pixel, consistent with principles from ensembling and variance reduction.