Learning with Constraint Learning:
New Perspective, Solution Strategy and Various Applications

Formally, the learning process of GAN can be conceptualized as the minimization of a distance metric, denoted as $``\mathcal{\mathtt{Distance}}"$, between the generated distribution $P_{G}$ and the data distribution $P_{data}$, represented as $\min``\mathcal{\mathtt{Distance}}"(P_{G},P_{data}).$ Under the standard definition, vanilla GAN dynamics advocate the incorporation of an auxiliary discriminator $D$ to facilitate the training of the generator $G$ through an alternating learning strategy, seeking to minimize the divergence in the objective $\min_{G}\max_{D}\mathcal{V}(G,D)$. In essence, as the most representative instance of LwCL, it can be formulated as a dynamically coupled game process, expressed as $\mathcal{V}\big{[}G(\bm{\theta}),D(\bm{\omega})\big{]}=\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}(\bm{\theta})=-\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}(\bm{\omega}).$

By employing an alternating direction iteration strategy, the original learning strategy establishes two gradient flows using gradient descent: $\bm{\omega}_{t+1}\leftarrow\bm{\omega}_{t}-\nabla_{\bm{\omega}}\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}\left(\bm{\omega}_{t}\right)$ and $\bm{\theta}_{t+1}\leftarrow\bm{\theta}_{t}-\nabla_{\bm{\theta}}\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}\left(\bm{\theta}_{t}\right).$ This leads to two independent optimization branches for $\bm{\omega}$ and $\bm{\theta}$ that proceed in parallel. However, the optimization of the generator depends on the discriminator’s parameters from the previous step, rather than the current step. This inaccurate approximation fails to capture the coupled best response gradient term depicted in Eq. (3). In contrast, our LwCL framework accurately formulates and characterizes the potential dependency of $\bm{\theta}$ on the current $\bm{\omega}$. Consequently, the optimization of the discriminator can be described by an exact estimated dynamic best response, which is then dynamically back-propagated to the optimization process for the generator dynamics. In light of these considerations, our proposed framework fundamentally circumvents the occurrence of training instabilities and mitigates mode collapse issues. To validate its effectiveness in addressing these challenges, we conduct a comprehensive set of experiments in Sec. 5, which showcase the results of these experiments and provide compelling support for the effectiveness of our framework.

Image generation aims to generate intricate and diverse images from compact seed inputs. Existing research focuses on developing diverse generative models, but they often encounter challenges during model training and require manual tuning techniques to mitigate mode collapse issues. With the versatility of our proposed framework, we apply our learning strategy to state-of-the-art generative model architectures. Specifically, we introduce different constrained objectives, referred to as CL energy functions $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}$, such as binary cross-entropy loss, least squares loss and 1-Lipschitz limit-loss with spectral norm. These objectives correspond to discriminators with different network structures. Remarkably, our LwCL framework seamlessly integrates into various advanced GAN variants without necessitating architectural modifications or loss selection changes, thereby consistently enhancing performance. In Sec. 5, we present comprehensive experimental results to demonstrate the effectiveness and efficiency of our LwCL framework. These results showcase more stable training performance and improved generalization capabilities, substantiating the practical benefits of our approach.

Style transfer aims to impose style constraints by optimizing the adversarial loss between two distinct datasets, while ensuring content preservation through reverse transformations. Drawing inspiration from the circular generative adversarial architecture proposed in Zhu et al. [10], we establish a bidirectional adversarial learning framework to guide the style transfer task. Specifically, by creating a cyclic mapping between two domains, denoted as $X$ and $Y$, we introduce two generators, namely $G_{1}$ and $G_{2}$, along with two discriminators, denoted as $D_{X}$ and $D_{Y}$. To capture the complexity of unsupervised learning, we design two components for our loss functions: the least squares loss $\mathcal{L}_{GAN}$ and cycle consistency loss $\mathcal{L}_{cyc}$, which account for the original input and the output after inverse transformation. Within our LwCL framework, we define the objective learner $\mathcal{N}_{\bm{\theta}}^{O}$ and the constraint learner $\mathcal{N}_{\bm{\omega}}^{C}$ as the two generators with parameters $\bm{\theta}$ and the two discriminative classifiers with parameters $\bm{\omega}$, respectively. The OL and CL objectives can then be expressed as $\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}=-\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}=\mathcal{L}(G_{1},G_{2},D_{X},D_{Y})=\mathcal{L}_{GAN}(G_{1},D_{Y},X,Y)+\mathcal{L}_{GAN}(G_{2},D_{X},Y,X)+\mathcal{L}_{cyc}(G_{1},G_{2})$. More details on the setup of the forward and backward cyclic consistency functions can be found in the experimental section.

Imitation learning endeavors to achieve optimal decision-making by interacting with the environment and acquiring knowledge from experiences. The objective of imitation learning is to simultaneously learn a state-action value function, denoted as $Q^{\pi}$, which predicts the expected discounted cumulative reward, and an optimal policy that aligns with the value function. Formally, we have: $Q^{\pi}(s,a)=\mathbb{E}_{s_{i+j}\sim\mathcal{P},r_{i+j}\sim\mathcal{R},a_{i+j}\sim\pi}(\sum_{k=0}^{\infty}\gamma^{j}r_{i+j}|s_{i}=s,a_{i}=a),$ where $\mathcal{P}$ and $\mathcal{R}$ denote dynamics of the environment and reward function, respectively. Here, $s$ and $a$ are the state and action, $i$ and $j$ refer to the i-th and j-th steps in the learning process. Within our LwCL paradigm, the actor and critic correspond to the objective learner $\mathcal{N}_{\bm{\theta}}^{O}$ and constraint learner $\mathcal{N}_{\bm{\omega}}^{C}$, respectively. Let $\bm{\theta}$ denote the parameters of the state-action value-function and $\bm{\omega}$ denote the parameters of the policy $\pi$. The expressions for $\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}$ and $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}$ are given by: $\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}:=\mathbb{E}{s_{i},a_{i}\sim\pi}[\mathtt{div}(\mathbb{E}_{s_{i+1},a_{i+1},r_{i+1}}\left(D_{Q}\right)\|Q(s_{i},a_{i}))],$ and $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}:=-\mathbb{E}_{s_{0}\sim p_{0},a_{0}\sim\pi}Q^{\pi}(s_{0},a_{0}),$ where $\mathtt{div}(\cdot\|\cdot)$ represents any divergence and $D_{Q}=r_{i+1}+\gamma Q(s_{i+1},a_{i+1})$. For specific settings of the state-action value function, please refer to the experimental section.

Medical image analysis involves the extraction of anatomical structures or lesions from medical images. Drawing inspiration from the concept that learning registration can provide additional pseudo-labeled training data to assist segmentation [26], we leverage our LwCL framework to dynamically address inter-task dependencies. In our framework, the registration process serves as the objective learner $\mathcal{N}_{\bm{\theta}}^{O}$, while the segmentation process acts as the constraint learner $\mathcal{N}_{\bm{\omega}}^{C}$. Building upon a base model [7], we incorporate a semantic consistency constraint into the segmentation task. Under our LwCL framework, the OL procedures for the registration task can be formulated as $\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}:=\min_{\theta}\mathcal{L}_{reg}(\mathcal{N}_{R}(I_{mov},I_{fix},\theta))$, where $\mathcal{N}_{R}$ represents the registration network with learnable parameters $\bm{\theta}$, and $I_{mov}$ and $I_{fix}$ denote the moving image and fixed image, respectively. Subsequently, by obtaining the warped image $I_{war}$, the CL procedures for the segmentation task can be formulated as $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}:=\min_{\theta}\mathcal{L}_{seg}(\mathcal{N}_{S}(\mathcal{N}_{R}(I_{war},I_{fix},\theta));\omega)$, where $\mathcal{N}_{S}$ represents the segmentation network with learnable parameters $\bm{\omega}$. Please refer to the experimental section in Sec. 6 for specific details on the settings of the loss functions.

Low-light image enhancement aims to reveal hidden information in dark areas to improve overall image quality. Drawing on the principles of the Retinex theory, we delve into the impact of downstream perceptual tasks, such as object detection, on the performance of upstream enhancement tasks. Guided by this concept, we construct a low-light enhancement network, inspired by recent advancements [39], as our objective learner $\mathcal{N}_{\bm{\theta}}^{O}$. Furthermore, we introduce a face detector proposed by [40] as an auxiliary constraint learner $\mathcal{N}_{\bm{\omega}}^{C}$. Within our LwCL framework, we employ the unsupervised illumination learning loss $\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}:=\mathcal{L}_{p}+\mathcal{L}_{s}$ as the OL function. Here, $\mathcal{L}_{p}$ and $\mathcal{L}_{s}$ represent the pixel fidelity term and smoothness term, respectively, which evaluate the performance of the upstream enhancement task. For the energy function $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}$ of the constraint learner, we introduce the anchor-based multi-task loss and progressive anchor loss to assess the detection performance, defined as: $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}:=\mathcal{L}_{SSL}(a)+\mathcal{L}_{SSL}(sa)$. Detailed configurations of the training loss and hyper-parameters can be found in the experimental Sec. 6.

Hyper-parameter learning aims to determine the optimal configuration of hyper-parameters for a given machine learning task. Hyper-parameters are parameters that remain fixed during the training process of a machine learning model. In essence, the goal of hyper-parameter learning is to find a set of hyper-parameters that minimizes the loss or maximizes the accuracy of the objective learning task. Mathematically, it can be expressed as $\bm{\theta}^{*}=\arg\min E_{(\mathcal{D}_{\mathtt{tr}},\mathcal{D}_{\mathtt{val}})\sim\mathcal{D}}L(g_{\bm{\omega}}(\cdot),\bm{\theta},\mathcal{D}_{\mathtt{tr}},\mathcal{D}_{\mathtt{val}})$, where $L$ represents the objective function, $g_{\bm{\omega}}(\cdot)$ denotes the learning algorithm applied to the hyper-parameters $\bm{\theta}$, and the model is trained on the training dataset $\mathcal{D}_{\mathtt{tr}}$ and validated on the validation dataset $\mathcal{D}_{\mathtt{val}}$. Within our LwCL framework, the objective learner $\mathcal{N}_{\bm{\theta}}^{O}$ aims to minimize the loss on the validation set $\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}(\bm{\theta},\bm{\omega};\mathcal{D}_{\mathtt{val}})$ with respect to the hyper-parameters $\bm{\theta}$, which include parameters such as learning rate, batch size, optimizer, and loss weights. On the other hand, the constraint learner $\mathcal{N}_{\bm{\omega}}^{C}$ is responsible for generating a learning algorithm by minimizing the training loss $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}(\bm{\theta},\bm{\omega};\mathcal{D}_{\mathtt{tr}})$ with respect to the model parameters $\bm{\omega}$, which encompass weights and biases.

Image deblurring aims to recover a latent clear image u from a blurred counterpart b. The physical model governing this process is represented by $\textbf{b}=\textbf{K}\ast\textbf{u}+\textbf{n}$, where K, n, and $\ast$ denote the blur kernel, additional noise, and the two-dimensional convolution operator, respectively. Typically, image deblurring entails two main tasks: deconvolution, involving the estimation of sharp images from blurred observations, and denoising. Drawing inspiration from the plug-and-play framework, which leverages semi-quadratic splitting, the deblurring problem is decomposed into alternating iterations of two sub-problems concerning u and an auxiliary variable z. Within our LwCL framework, we define the constraint learner $\mathcal{N}_{\bm{\omega}}^{C}$ as the fidelity learning sub-problem, addressing u and z. This can be formulated as $\{\textbf{u},\textbf{z}\}=\arg\min_{\textbf{u},\textbf{z}}\{\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}(\textbf{u},\textbf{z}):=\|\textbf{K}\ast\textbf{u}-\textbf{b}\|_{2}^{2}+\mu\|\textbf{u}-\textbf{z}\|^{2}+k\|\textbf{Wu}\|_{1}\},$ where $\mu$ represents a penalty parameter, and W denotes the wavelet transform matrix. The objective learner $\mathcal{N}_{\bm{\theta}}^{O}$ can be viewed as the prior learning process, governed by a denoiser $\mathtt{Net}_{\bm{\theta}}(\textbf{u})$, with regard to the variable $\bm{\theta}$. Mathematically, it can be expressed as $\bm{\theta}=\arg\min_{\textbf{z}}\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}(\mathtt{Net}{\bm{\theta}}(\textbf{u}),\bm{\theta})$. For further details on the parameter configurations, please refer to Sec. 6.

Multi-task meta-learning represents a formidable challenge that revolves around swiftly adapting to novel tasks with limited examples. Among these tasks, meta-feature learning stands out as a prominent representative of multi-task and meta-learning, with the objective of acquiring a shared meta feature representation that encompasses all tasks. This is achieved by bifurcating the network architecture into two distinct components: the meta feature extraction part, responsible for generating the cross-task intermediate representation layers (parameterized by $\bm{\theta}$), and the task-specific part, characterized by the multinomial logistic regression layer (parameterized by $\bm{\omega}^{j}$). This framework allows for building accurate machine learning models utilizing a smaller training dataset, especially in the context of few-shot classification tasks, which are widely recognized in the field. Within the LwCL framework we propose, the intermediate representation layers that produce the meta-features can be viewed as the objective learner $\mathcal{N}_{\bm{\theta}}^{O}$ for multiple task-specific assignments. Consequently, we optimize the performance of these meta-features using the validation set through the defined loss function $\mathcal{F}^{\bm{\omega}^{j}}_{\mathtt{OL}}(\bm{\theta};\mathcal{D}_{\mathtt{val}}^{j})$. Additionally, the forward propagation of the classification layers at the network’s end serves as the constraint learner $\mathcal{N}_{\bm{\omega}}^{C}$, wherein the training set loss $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}(\bm{\omega}^{j};\mathcal{D}_{\mathtt{tr}}^{j})$ is utilized to guide the learning process.

Initially, we conduct extensive experiments on synthesized datasets following a Mixed of Gaussian (MOG) distribution. These experiments aim to provide a quantitative and qualitative evaluation of our algorithm, considering aspects such as mode generation, computational efficiency, and training stability. To establish a performance comparison, we benchmark our approach against several state-of-the-art GAN architectures, including VGAN [1], WGAN [9], ProxGAN [18], and LCGAN [20]. For the synthetic data, we generate four distinct types of MOG distributions: 2D Ring (consisting of 5 or 8 2D Gaussians arranged in a ring), 2D Random (comprising 10 2D Gaussians with random magnitudes and positions), 2D Grid (comprising 25 2D Gaussians arranged in a grid), and 3D Cube (comprising 27 3D Gaussians within a cube). Each Gaussian distribution has a fixed variance of 0.02. During the training phase, we construct training batches with 512 samples from each mixture of Gaussian models, consisting of both real and generated data. Additionally, we sample 512 generated images for testing purposes.

To optimize the two networks, we uniformly employ the Adam optimizer, with a learning rate of $10^{-4}$ for the discriminator and $10^{-3}$ for the generator. Both the generator and discriminator adopt a 3-layer linear network with a width of 256. The activation function utilized is a leaky ReLU with a threshold of 0.2. To provide a comprehensive comparison, we employ three well-established metrics: Frechet Inception Distance (FID) [47], Jensen-Shannon divergence (JS) [1], number of Modes (Mode). These metrics serve as a basis for evaluating and contrasting the performance of different approaches.

Fig. 4 presents a comprehensive comparison of the number of generated samples among various advanced GAN methods, both with and without our LwCL methodology. When VGAN and LCGAN are combined with the NAL strategy, they exhibit a mapping of diverse inputs to the same output, resulting in limited capturing of different distributions. It is evident that the original GAN models struggle to capture a significant number of distributions, leading to severe mode collapse and unsatisfactory performance. However, when integrated into our framework, these methods are able to capture a relatively larger number of distributions with the assistance of our approach, ultimately generating a more diverse range of realistic distributions. In the case of the 3D Cube distribution, ProxGAN and LCGAN, when combined with our methodology, demonstrate the ability to fit almost all Gaussian distributions accurately, preserving intricate details. Tab. II further demonstrates the effectiveness of our methodology in alleviating the mode collapse issue. Specifically, WGAN combined with LwCL achieves the lowest FID score in the 2D Ring distribution, and obtains the lowest JS score in the 3D Cube dataset. It is noteworthy that our flexible LwCL methodology uniformly improves a wide range of existing GANs, enhancing their overall performance.

In our experimental evaluation, we examine the performance of several well-established generative models, namely DCGAN [19], LSGAN [42], and SNGAN [48]. To assess their capabilities, we employ widely used benchmark datasets, including CIFAR10 [43], CIFAR100 [44] and CelebA-HQ [45]. For evaluating the generative models, we employ two widely recognized metrics: Inception Score (IS) for evaluating generation quality and diversity, and Fréchet Inception Distance (FID) for capturing the issue of mode collapse. Under our LwCL framework, we consider DCGAN, which incorporates standard binary cross-entropy loss for unsupervised training and employs a coupled game process of OL and CL. The objective functions $\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}$ and $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}$ for DCGAN are defined as follows: $\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}:=E_{\mathbf{v}\sim\mathcal{N}_{(0,1)}}\left[\log(1-D(G(\mathbf{v})))\right]$, and $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}:=E_{\mathbf{u}\sim P_{data}}\left[\log(D(\mathbf{u}))\right]+E_{\mathbf{v}\sim\mathcal{N}_{(0,1)}}\left[\log(1-D(G(\mathbf{v})))\right]$. Similarly, for LSGAN, which employs a least squares loss, the objective functions $\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}$ and $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}$ are constructed as follows: $\mathcal{F}^{\bm{\omega}}_{\mathtt{OL}}:=E_{\mathbf{v}\sim\mathcal{N}_{(0,1)}}\left[D(G(\mathbf{v}))-c\right]^{2}$, and $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}:=E_{\mathbf{u}\sim P_{data}}\left[D(\mathbf{u})-b\right]^{2}+E_{\mathbf{v}\sim\mathcal{N}_{(0,1)}}\left[D(G(\mathbf{v}))-a\right]^{2}$. As for SNGAN, it incorporates spectral normalization to ensure the 1-Lipschitz continuity constraint. The objective function $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}$ for SNGAN is defined as follows: $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}:=\sup_{||D||\leq 1}E_{\mathbf{u}\sim P_{data}}\left[D(\mathbf{u})\right]-E_{\mathbf{v}\sim\mathcal{N}_{(0,1)}}\left[D(G(\mathbf{v}))\right]$. Furthermore, in our face generation experiment on the high-resolution CelebA-HQ dataset [45], we employ StyleGAN as the backbone architecture.

Tab. III further highlights the consistent performance improvements achieved by state-of-the-art GAN architectures when incorporated into our LwCL framework. Moreover, Fig. 5 visually demonstrates the efficacy of our LwCL methodology in conjunction with StyleGAN. It showcases the superior style-content trade-off achieved, validating the versatility and effectiveness of our flexible solution strategy. Notably, our approach excels in generating realistic facial structures while effectively mitigating twist distortion.

We select the CycleGAN model [10] as the foundation for our architecture and conduct experiments on the FFHQ dataset [46] to explore unsupervised style transfer. Our approach leverages two Generator networks and two Discriminator networks, forming a bidirectional LwCL framework. This framework incorporates cycle-consistency loss in two loops to assess the ability to reconstruct an image from its transformed counterpart. The cycle consistency loss, denoted as $\mathcal{L}_{cyc}(G_{1},G_{2})$, captures the discrepancy between the original image and its reconstructed version in both the forward and backward mapping processes. Mathematically, it is defined as the summation of the $\ell_{1}$ norm of the difference between the transformed and reconstructed images, computed as $\mathcal{L}_{cyc}(G_{1},G_{2})=E_{\mathbf{x}\sim P_{data}(\mathbf{x})}\left[\|G_{2}(G_{1}(\mathbf{x}))\|_{1}\right]+E_{\mathbf{y}\sim P_{data}(\mathbf{y})}\left[\|G_{1}(G_{2}(\mathbf{y}))\|_{1}\right].$ In the bidirectional mapping process, the optimization of variables in the two objective functions can be mutually exchanged and modeled. We ensure a fair comparison by following the experimental setups and model architecture detailed in [10].

Fig. 6 visually demonstrates the remarkable advantages of our proposed framework in generating samples of higher quality and richer detailed textures. In comparison to the sparse and unrealistic textures produced by the standard CycleGAN, it is evident that the network integrated with our LwCL framework can generate zebra stripes that are more abundant, natural, and realistic. In Fig. 7, we observe significant fluctuations in the visual perceptual quality of the generated images by the standard CycleGAN throughout the training process. With the standard CycleGAN, high-quality zebra images can be generated at 40 epochs. However, at 60 epochs, a small amount of background stripes starts to appear, and by 120 epochs, the stripes that should be generated on the horse are completely transferred to the background. In contrast, when our LwCL strategy is incorporated, the generation of zebra patterns becomes gradually stable, and the authenticity of the texture is significantly enhanced. This improvement effectively mitigates the occurrence of mode collapse, ensuring the preservation of desirable characteristics in the generated images.

In the formulated Markov decision process, denoted as $\mathcal{M}=(\mathcal{S},\mathcal{A},\rho,P,\mathcal{R},\gamma)$, we define the action space $\mathcal{A}$, state space $\mathcal{S}$, and reward function $\mathcal{R}:\mathcal{S}\times\mathcal{A}\mapsto\mathbb{R}$. The initial state $s_{0}$ is drawn from the distribution $\rho(\cdot)$, and the discount factor $\gamma\in(0,1)$ is applied. In this context, the actor $\pi$ interacts with the environment to learn the state-action value function $Q^{\pi}(s,a)$, followed by the update of the actor $\pi$ based on $Q^{\pi}(s,a)$. The objective of the policy is to maximize the expected discounted cumulative reward, given by: $\pi^{*}=\arg\max_{\pi}\mathbb{E}_{s_{0}\sim p_{0},a_{0}\sim\pi}\left(Q^{\pi}(s_{0},a_{0})\right),$ where $s_{0}$, $a_{0}$, and $p_{0}$ represent the initial state, initial action, and initial state distribution, respectively. For our experiment, we employ the PyBullet physics simulator and publicly available datasets tailored for data-driven deep reinforcement learning. Following the definitions and experimental settings of recent studies [22, 23], we design an agent with an actor-critic structure to predict actions that deceive the discriminator. The discriminator, on the other hand, is trained to distinguish between samples generated by the policy $\pi^{*}$ and an expert policy $\hat{\pi}$. To compute the reward function $\mathcal{R}$, we adopt the form $h(s,a)=\log(D(s,a))-\log(1-D(s,a)).$ Additionally, both Generative Adversarial Imitation learning (GAIL) and our method incorporate the $R_{1}$ gradient penalty regularizers.

Fig. 8 presents the average policy return and standard error for two simulated environments, namely “Walker2D” and “Hopper”. The average policy return and its standard error are plotted to illustrate the performance of GAIL. It is evident that the GAIL curve exhibits significant instability and lacks a convergent trend. In contrast, our proposed solution technique ensures a more stable convergence during training with reduced deviation. Additionally, we provide the final return achieved throughout the training episodes, which serves as an indicator of the performance improvement achieved by GAIL when employing our novel solution techniques.

We evaluate the segmentation performance on a hybrid dataset by using 426 mixed medical scans, which is sampled from three standard datasets, ABIDE [49], ADNI [50] and PPMI [51]. During the training phase, the scanned images are divided into 346, 40, and 40 volumes for training, validation, and testing, respectively. To capture both global and local differences in appearance, we define the energy function $\mathcal{L}_{reg}=\mathcal{L}_{sim}+\lambda_{scc}*\mathcal{L}_{scc}$, where $\mathcal{L}_{sim}$ is the multi-scale local cross-correlation in appearance, and $\mathcal{L}_{scc}$ is the semantic content consistency loss, i.e., $\mathcal{L}_{scc}=\frac{1}{2}\big{(}\mathtt{KL}(p_{t}||\frac{p_{w}+p_{t}}{2})+\mathtt{KL}(p_{w}||\frac{p_{w}+p_{t}}{2})\big{)}$. Here $\mathtt{KL}$ denotes the Kullback-Leibler divergence, and $p_{w}$ and $p_{t}$ are the warped prediction and target prediction, respectively. For segmentation, the energy function is defined as a hybrid loss $\mathcal{L}_{seg}=\mathcal{L}_{dice}+\lambda_{mce}*\mathcal{L}_{mce}$ composed of multi-class cross entropy loss $\mathcal{L}_{mce}$ and Dice coefficient loss $\mathcal{L}_{dice}$. In our training, we empirically set the balancing parameters $\lambda_{scc}=10$ and $\lambda_{mce}=1$, and use ADAM optimizer with learning rate of $4*10^{4}$. To evaluate the performance of registration and segmentation, we employed well-established metrics such as the Dice score, Hausdorff distance (HD95), and average surface distance (ASD). We compare our method to several state-of-the-art methods, including a) Deep learning registration-based segmentation methods, VxM[52], LKU-Net[53], and TransMorph[54]. b) Deep learning segmentation methods, UNet[7], MASSL[55], and CPS[56]. d) joint registration and segmentation methods, SST[28], DeepAtlas[26], DataAug[29], and BRBS[27].

Tab. IV presents quantitative comparisons for joint registration and segmentation tasks, demonstrating that our method achieved the highest Dice score and the lowest HD95 and ASD metrics in registration and segmentation, respectively. Fig. 9 illustrates the two-dimensional visualization results of the registration method compared to other approaches. The framework solely relying on registration and segmentation tends to exhibit more misalignment errors along anatomical boundaries. In contrast, our method demonstrates the smallest mislabeling regions on the lateral ventricle (LV) and brainstem (BS), as indicated by the yellow and pink areas. Additionally, we provide qualitative results in Fig. 10, illustrating the robust segmentation performance of our method at complex termination sites in the structural white matter of the brain and the finer segmentation quality achieved on the cerebellar tissue and the 3/4 ventricles.

We conduct low-light enhancement experiments on DarkFace [57] dataset, and adopt the well-known no reference metrics (i.e., DE and LOE). To benchmark our method, we compare it against several state-of-the-art approaches, including MBLLEN [58], RetinexNet [59], KinD [60], ZeroDCE [61], DeepUPE [62], FIDE [63], DRBN [64] and SCI [39]. As mentioned in [39], we introduce the pixel fidelity term $\mathcal{L}_{p}$ and a smoothness term $\mathcal{L}_{s}$ for $\mathcal{N}_{\bm{\theta}}^{O}$, which are formulated as $\mathcal{L}_{p}=\sum_{t=1}^{T}\|\mathbf{x}^{t}-\hat{\mathbf{y}}_{t-1}\|$, $\mathcal{L}_{s}=\sum_{i=1}^{N}\sum_{j\in\mathcal{N}(i)}w_{i,j}\|\mathbf{x}_{i}^{t}-\mathbf{x}_{j}^{t}\|$, where $\hat{\mathbf{y}}_{t-1}$ denotes the self-calibrated variable, $N$ is the total number of pixels, $w_{i,j}$ represents the weight function. As for $\mathcal{N}_{\bm{\omega}}^{C}$, we also introduce the anchor-based multi-task loss and progressive anchor loss [40], defined as: $\mathcal{F}^{\bm{\theta}}_{\mathtt{CL}}:=\mathcal{L}_{SSL}(a)+\lambda\mathcal{L}_{SSL}(sa)$. Here, $\mathcal{L}_{SSL}(p_{i},p_{i}^{*},t_{i},g_{i},a_{i})=\frac{1}{N_{conf}}((\sum_{i}L_{conf}(p_{i},p_{i}^{*}))+\frac{\beta}{N_{loc}}\sum_{i}p_{i}^{*}L_{loc}(t_{i},g_{i},a_{i})),$ where $N_{conf}$ and $N_{loc}$ indicate the number of positive and negative anchors, and the number of positive anchors, respectively. $L_{loc}$ is the smooth loss between the predicted box $t_{i}$ and ground-truth box $g_{i}$ using the anchor $a_{i}$, and $L_{conf}$ is the softmax loss in terms of two classes.

In Fig. 11, we compare the visualization results. It can be seen that although some methods can successfully enhance the brightness of the image, none of them can restore the clear image texture. The DE score is reported below, and a higher DE value indicates better visual quality. In comparison, our method produces the most visually pleasing results, can not only learns to enhance the dark area while restoring more visible details but also avoids over-exposure artifacts. We report the quantitative results in Tab. V. It can be seen that our method numerically outperforms existing methods by large margins and ranks first across all metrics. This further endorses the superiority of our method over current state-of-the-art methods in generating high-quality visual results.