Our proposed inverse flow framework is built upon continuous-time generative models such as diffusion models, conditional flow matching, and consistency models. Here we present a unified view of these methods that will help connect inverse flow with this entire family of models (Section 3).

These generative modeling methods are connected by their equivalence to continuous normalizing flow or neural ODE (Chen et al., 2019). They can all be considered as explicitly or implicitly learning the ODE that transforms between the prior distribution $p(\mathbf{x}_{1})$ and the data distribution $p(\mathbf{x}_{0})$

in which $\mathbf{u}_{t}(\mathbf{x})$ represents the vector field of the ODE. We use the convention that $t=0$ corresponds to the data distribution and $t=1$ corresponds to the prior distribution. Generation is realized by reversing this ODE, which makes this family of methods a natural candidate for extension toward denoising problems.

Continuous-time generative models typically involve defining a conditional ODE or SDE that determines the $p(\mathbf{x}_{t}|\mathbf{x}_{0})$ that transforms the data distribution to the prior distribution. Training these models involves learning the unconditional ODE (Eq. 1) based on $\mathbf{x}_{0}$ and the conditional ODE or SDE (Lipman et al., 2023; Tong et al., 2024; Song et al., 2021) (Figure 1). The unconditional ODE can be used for generation from noise to data.

Denoising without access to ground truth data requires assumptions about the noise or the signal. Most contemporary approaches are based on assumptions about the noise, as the noise distribution is generally much simpler and better understood. Because prior methods have been comprehensively reviewed (Kim & Ye, 2021; Batson & Royer, 2019; Lehtinen et al., 2018; Xie et al., 2020; Soltanayev & Chun, 2018; Metzler et al., 2020), and our approach is not directly built upon these approaches, we only present a brief overview and refer the readers to Appendix LABEL:appendix:denoise referenced literature for more detailed discussion. None of these approaches are generally applicable to any noise types.

To solve the inverse flow problem, we first consider learning a mapping from $\mathbf{x}_{1}$ to $\mathbf{x}_{0}$ through an ODE with vector field $\mathbf{v}^{\theta}_{t}(\mathbf{x})$. We propose to learn $\mathbf{v}^{\theta}_{t}(\mathbf{x})$ with the inverse flow matching (IFM) objective

where the expectation is taken over $t$, $p(\mathbf{x}_{1})$, and $p(\mathbf{x}_{t}\mid\mathbf{x}_{0}=\textrm{ODE}^{\mathbf{v}^{\theta}}_{1\rightarrow 0}(\mathbf{x}_{1}))$. This objective differs from conditional flow matching (Eq. 2) in two key aspects: using only transformed data $\mathbf{x}_{1}$ rather than unobserved data $\mathbf{x}_{0}$, and choosing the conditional ODE based on the conditional distribution $p(\mathbf{x}_{1}|\mathbf{x}_{0})$. Specifically,

1. 1.

   Sampling from the data distribution $p(\mathbf{x}_{0})$ is replaced with sampling from $p(\mathbf{x}_{1})$ and simulating the unconditional ODE backward in time based on the vector field $\mathbf{v}$, denoted as $\textrm{ODE}^{\mathbf{v}^{\theta}}_{t\rightarrow 0}(\mathbf{x}_{1})$. We refer to this distribution as the recovered data distribution $q(\mathbf{x}_{0})$.

2. 2.

   The conditional ODE vector field $\mathbf{u}_{t}\left(\mathbf{x}\mid\mathbf{x}_{0}\right)$ is chosen to match the given conditional distribution $p(\mathbf{x}_{1}|\mathbf{x}_{0})$ at time $1$.

For easier and more flexible application of IFM, similar to conditional flow matching (Eq. 3), an alternative form of the conditional ODE $\mathbf{u}_{t}\left(\mathbf{x}\mid\mathbf{x}_{0},\mathbf{x}_{1}^{\prime}\right)$ can be used instead of $\mathbf{u}_{t}\left(\mathbf{x}\mid\mathbf{x}_{0}\right)$. Since $\mathbf{x}_{1}^{\prime}$ is sampled from the noise distribution $p(\mathbf{x}_{1}|\mathbf{x}_{0})$, the above condition is automatically satisfied. The conditional ODE vector field can be easily chosen as any smooth interpolation between $\mathbf{x}_{0}$ and $\mathbf{x}_{1}^{\prime}$, such as $\mathbf{u}_{t}\left(\mathbf{x}\mid\mathbf{x}_{0},\mathbf{x}_{1}^{\prime}\right)=\mathbf{x}_{1}^{\prime}-\mathbf{x}_{0}$. We detailed the inverse flow matching training in Algorithm 1 with the alternative form in Appendix A.1.

Next, we discuss the theoretical justifications of the IFM objective and the interpretation of the learned model. We show below that when the loss converges, the recovered data distribution $q\left(\mathbf{x}_{0}\right)$ matches the ground truth distribution $p(\mathbf{x}_{0})$. The proof is provided in Appendix LABEL:appendix:IFM.

Moreover, we show that with IFM the learned ODE trajectory from $\mathbf{x}_{1}$ to $\mathbf{x}_{0}$ can be intuitively interpreted as always pointing toward the direction of the estimated $\mathbf{x}_{0}$. More formally, the learned unconditional ODE vector field can be interpreted as an expectation of the conditional ODE vector field.

The proof is provided in Appendix LABEL:appendix:IFM. Specifically, with the choice of $\mathbf{u}_{t}\left(\mathbf{x}\mid\mathbf{x}_{0},\mathbf{x}_{1}\right)=\mathbf{x}_{1}-\mathbf{x}_{0}$, Eq. 8 has an intuitively interpretable form

which means that the unconditional ODE vector field at any time $t$ points straight toward the expected ground truth $\mathbf{x}_{0}$.

IFM can be computationally expensive during training and inference because it requires solving ODE in each update. We address this limitation by introducing inverse consistency model (ICM), which learns a consistency function to directly solve the inverse flow without involving an ODE solver.

However, the original consistency training formulation is specific to one type of diffusion process (Karras et al., 2022), which is only applicable to independent Gaussian noise distribution for inverse generation application. Thus, to derive inverse consistency model that is applicable to any transformation, we first generalize consistency training so that it can be applied to arbitrary transformations and thus flexible to model almost any noise distribution.

To test the capability of inverse flow in inverting complex transformations, we first attempted the deterministic inverse generation problem of inverting the transformation by Navier-Stokes fluid dynamics simulation³³3Inverse flow algorithms can be applied to deterministic transformations from $\mathbf{x}_{0}$ to $\mathbf{x}_{1}$ by using a matching conditional ODE, even though the general forms consider stochastic transforms described by $p(\mathbf{x}_{1}\mid\mathbf{x}_{0})$.. We aim to recover the earlier state of the system without providing them for training (Figure 2). Navier-Stokes equations describe the motion of fluids by modeling the relationship between fluid velocity, pressure, viscosity, and external forces. These equations are fundamental in fluid dynamics and remain mathematically challenging, particularly in understanding turbulent flows. The details of the simulation are described in Appendix LABEL:appendix:syn_datasets.

To test inverse flow algorithms on a denoising inverse generation problem, we generated a synthetic 8-gaussians dataset (Appendix LABEL:appendix:syn_datasets for details). Both IFM and ICM are capable of noise removal (Figure 2). ICM achieved a similar denoising performance as IFM, even though it is much more computationally efficient due to the iterative evaluation of ODE (NFE=10) by IFM.

We evaluated the proposed method on images in the benchmark dataset BSDS500 (Arbeláez et al., 2011), Kodak, and Set12 (Zhang et al., 2017). To test the model’s capability to deal with various types of conditional noise distribution, we generated synthetic noisy images for three different types of noise, including correlated noise and adding noise through a diffusion process without a closed-form transition density function (Appendix LABEL:appendix:realdata for details). All models were trained using the BSDS500 training set and evaluated on the BSDS500 test set, Kodak, and Set12. We show additional qualitative results in Appendix LABEL:appendix:qualitative.

1. 1.

   Gaussian noise: we added independent Gaussian noise with fixed variance.

2. 2.

   Correlated noise: we employed convolution kernels to generate correlated Gaussian noise following the method in (Mäkinen et al., 2020)

   $\displaystyle\eta=\nu\circledast g$

   (12)

   where $\nu\sim\mathcal{N}(0,\sigma^{2}I)$ and $g$ is a convolution kernel.

3. 3.

   Jacobi process: we transformed the data with Jacobi process (Wright-Fisher diffusion), as an example of SDE-based transform without closed-form conditional distribution

   $$\mathrm{d}\mathbf{x}=\frac{s}{2}[a(1-\mathbf{x})-b\mathbf{x}]\mathrm{d}t+\sqrt{s\mathbf{x}(1-\mathbf{x})}\mathrm{d}\mathbf{w}.$$

   (13)

   We generated corresponding noise data by simulating the Jacobi process with $s=1$ and $a=b=1$. Notably, the conditional noise distribution generated by the Jacobi process does not generally has an expectation that equals the ground truth (i.e. non-centered noise), which violates the assumptions of Noise2X methods.

Our approach outperformed alternative unsupervised methods in all three noise types, especially in correlated noise and Jacobi process (Appendix LABEL:appendix:qualitative, Table 1). This can be attributed to the fact that both Noise2X methods assumes independence of noise across different feature dimensions as well as centered-noise which were violated in corrleated noise and Jacobi process respectively.

Moreover, Our approach outperformed the supervised method on both Gaussian noise and correlated noise. Further analysis revealed that the supervised method encountered overfitting during the training process, which led to suboptimal performance. In contrast, our method did not exhibit such issues, highlighting the superiority of our approach.

In addition, in Appendix LABEL:appendix:addition, we conducted a series of experiments that demonstrate the reliability of our method under different intensities and types of noise. Furthermore, our method yielded satisfactory results even when there is a bias in the estimation of noise intensity. It also achieved excellent performance on RGB images and small sample-size datasets.