Curriculum Guided Domain Adaptation in the Dark

Domain gap or domain shift occurs when the data distribution of the training data (source domain) is considerably different from that of the testing data (target domain) [3]. Long et al. [13], and Tzeng et al. [4] proposed to mitigate this distribution shift by minimizing the maximum mean discrepancy (MMD) between the two distributions, while Zellinger et al. [5] proposed to match the higher order central moments of source and target probability distributions, and thus minimize central moment discrepancy (CMD) for UDA. Sun and Saenko [14] devised Deep CORAL to minimize second-order distribution statistics to mitigate domain shift. Ganin et al. [1] utilized a domain discriminator module, and introduced gradient reversal layer (GRL) to adversarially align the two distributions. Many methods followed since then that have utilized adversarial alignment on the latent feature space. [15, 16]. While [1] uses a common encoder for the source and target data, Tzeng et al. [17] proposed to decouple the encoders by first training an encoder and a classifier on the labelled source data, followed by training a separate target data encoder using a domain discriminator, and finally deploying the same source classifier as the target classifier. Hoffman et al. [18] produced source-like images using generative image-to-image translation [19] and adversarially aligned source and target data distributions at the low-level or pixel-level. Global domain-wise adversarial alignment however may cause loss of intrinsic target class discrimination in the embedding space, and lead to suboptimal performance. To preserve class-wise feature discrimination, Li et al. [20] simultaneously aligned the domain-wise and class-wise distributions across the source and target data by solving two complementary domain-specific and class-specific minimax problems. In a non-adversarial approach, Pan et al. [21] proposed to calculate the source class prototypes for the labelled source data, and target class prototypes from the pseudo-labelled target data, and then enforce consistency on the prototypes in the embedding space. Tang et al. [22] similarly bases structural domain similarity to enforce structural source regularization and conducts discriminative clustering of target data without any domain alignment. Chen et al. [23] introduced graph matching to formulate cross-domain adaptation, and minimized Wasserstein distance for entity-matching, and Gromov-Wasserstein distance for edge matching. In order to reduce negative transfer introduced by the target samples that are either near the source-data generated decision boundaries, or are far away from their corresponding class centers, Xu et al. [24] proposed a weighted optimal transport strategy to achieve a reliable precise-pair-wise optimal transport procedure for domain adaptation.

Although domain divergence minimization [3, 4, 5], adversarial adaptation [1, 2], and optimal transport [23, 24] are widely used techniques for UDA, they require access to both the source and target data during adaptation. Addressing situations where source data is unavailable, several source-free DA (SFDA) methods have been proposed recently. Chidlovskii et al. [25] proposed to use a few source prototypes or representatives in place of the entire source data for semi-supervised domain adaptation. Liang et al. [26] proposed to conduct target adaptation using source-free distant supervision to iteratively find taret pseudo-labels, a domain invariant subspace where the source and target data centroids are only moderately shifted, and finally target centroids/prototypes by implementing an alternating minimization strategy. Liang et al. [6] introduced SHOT as an SFDA framework which transfers the source hypothesis or classifier to the target model, and adapts via self-training with information maximization [27, 28, 29] and class centroid-based pseudolabel refinement. Yang et al. [7] proposed G-SFDA which refines the pseudolabels further via consistency regularization among neighboring target samples. Ding et al. [30] introduced SFDA-DE which samples from an estimated source data distribution, and conducts contrastive alignment between the estimated source and target distributions.

Extending the premise of SFDA further, Liang et al. [8] introduced a newer paradigm of black box DA where, in addition to the source data, the source model parameters are also unavailable during adaptation. This new challenging scenario is important to protect intellectual property (source model parameters) from the end users. Liang et al. proposed DINE which distills knowledge from the black-box source model to the target model in the first stage, followed by finetuning with target pseudolabels in the second stage. Yang et al. [9] proposed BETA as a method that separates easy- and hard-to-learn pseudolabels using a conventional noisy label learning technique [11], and applies a twin-network co-training strategy similar to [10], and adversarial alignment during adaptation.

In this paper, we identify Jensen-Shannon distance (JSD) as a more appropriate criterion for clean-noisy sample separation for the unbounded noise rate in UDA, compared to traditional low CE loss for the bounded loss in NLL. We formulate a curriculum learning strategy to train the target model end-to-end with cleaner samples first, and progressively with noisy samples later.

The predictions $\hat{y_{t}}$ generated by the black box source model $f_{s}$ are noisy and unreliable due to domain shift between $\mathcal{D}_{s}$ and $\mathcal{D}_{t}$. Research on learning with noisy labels shows that deep learning models tend to fit on the clean samples first, and on the noisy samples later during training. [32, 10] We follow this insight and separate the target domain data into a clean sample set $\mathcal{X}_{tc}$ with reliable predictions, and a noisy sample set $\mathcal{X}_{tn}$ with unreliable predictions. In traditional noisy label settings, the noisy labels are either caused by wrong annotations from humans or from image search engines. The noise rate is, therefore, bounded. However, as the noisy labels in UDA are generated by the source model, the noise rate in this case is unbounded and can approach unity [33]. We propose Jensen-Shannon distance (JSD) [34] between the source predicted hard labels $\hat{y}_{t}^{i}$ and the target features as the criterion for clean-noisy sample separation under unbounded noise rate. JSD is calculated as,

where, $KL(a,b)$ is the Kullback-Leibler divergence between $a$ and $b$, and $p_{t}^{i}$ is the target model output probability for target sample $x_{t}^{i}$. Compared to cross-entropy loss, JSD is symmetric by design, and ranges between 0 and 1, thus becoming less susceptible to noise. When applied to the network response, JSD produces a bimodal distribution, which is modelled by a 2-component Multivariate Gaussian Mixture Model (GMM) with equal priors. In DA, the target model may confidently categorize an image as the wrong class with very high prediction probability. Therefore, this is a poor criterion for identifying whether a sample is clean or noisy. For the potentially unbounded pseudolabel noise rate in DABP, we take the probability of belonging to the JSD Gaussian distribution with the lower mean value as the confidence metric of being a clean sample in our clean-noisy sample separation stage. Empirically, we apply a threshold $\delta_{t}$ on our confidence score of belonging to the lower-mean GMM distribution to select our clean sample set $\mathcal{X}_{tc}$, at the beginning of each epoch for adaptation. The remaining target samples are included in the noisy label set $\mathcal{X}_{tn}$.

In order to produce robust target model pseudolabels $y_{t}^{i}$, we apply a series of augmentations on the target samples and produce an ensemble of output prediction probabilities from our two target models. We give equal weights to each output prediction and take the mean of the outputs as the soft pseudolabel as follows.

where $M$ is the number of augmentations for the $i$-th target sample. The predictions are further sharpened with a temperature factor $T(0<T<1)$ and then normalized as follows.

where $y_{t}^{iC}$ is the $C$-th dimensional value of the pseudolabel vector $y_{t}^{i}$.

In order to mitigate early training time memorization [32] induced from noisy labels during the adaptation of deep models, we introduce a curriculum guided learning to train the target model on the clean samples first, and on the noisy samples later. As the adaptation/training progresses, more noisy samples are reclassified as clean samples.

We employ separate training losses for the clean and noisy sample set. The clean set is trained with standard cross-entropy (CE) loss as follows.

where $\sigma_{k}(a)=\frac{exp(a_{k})}{\sum_{i}exp(a_{i})}$ is the softmax function and $C$ is the number of classes. For the noisy set, we minimize a combination of active-passive losses [35] constructed of normalized cross-entropy loss $\mathcal{L}_{tn_{NCE}}$ and reverse cross-entropy loss $\mathcal{L}_{tn_{RCE}}$. [35] showed that such normalization makes a model robust to noisy data. Reverse cross-entropy loss is applied to avoid any underfitting on the noisy set. Due to the unbounded nature of noise rate in UDA and conservative clean-noisy separation criteria in CABB, we employ this particular combination of active-passive losses as our noisy set loss $\mathcal{L}_{tn}$ to make target training/adaptation robust and comprehensive on the noisy sample set. The loss function is expressed as follows.

where $\beta$ is a hyperparameter.

To promote learning of clean samples first and to mitigate noisy label memorization, target training is done under curriculum guidance [36]. Based on the success of the clean-noisy sample separation, the pseudolabels in the clean sample set $\mathcal{X}_{tc}$ are more likely to be correct, while those in the noisy sample set $\mathcal{X}_{tn}$ have a much higher noise rate. Therefore, a deep network tends to easily learn from the unambiguous $\mathcal{X}_{tc}$ set. We set a curriculum factor $\gamma_{n}$ according to the following equation.

where, $\alpha$ is a hyperparameter and $n$ is the iteration number. $\gamma_{n-1}$ is the curriculum factor for the previous iteration. The ratio $L_{x_{n}}/L_{x_{n-1}}$ determines how much the curriculum factor decreases from iteration $n-1$ to $n$. If the CE loss on the clean set increases, $\gamma$ decreases by a small value to allow for further training on the clean set in the subsequent iterations. But if the CE loss decreases by a large margin, $\gamma$ decreases accordingly to accommodate learning from the noisy sample set in the coming iterations. Our curriculum guidance balances the supervised and unsupervised losses on the respective clean and noisy sets as follows.

We adopt the formulation of information maximization (IM) loss [27, 28, 6] from [12] to help our model produce precise predictions, while maintaining a global diversity across all classes in the output predictions. The IM loss is a combination of the following entropy loss $\mathcal{L}_{ent}$ and equal diversity loss $\mathcal{L}_{eqdiv}$.

Minimization of entropy loss $\mathcal{L}_{ent}$ is gradually activated as the model sufficiently adapts to the clean sample. Such curriculum guidance ensures that the potentially erroneous predictions produced in the early stages of self-training are not accumulated. The $\mathcal{L}_{eqdiv}$ loss enforces diversity in the output predictions throughtout the training process. The overall objective function is,

A brief demonstration of the CABB pipeline can be found in Algorithm 1.

We evaluate CABB on three popular domain adaptation datasets viz. Office-31 [43], Office-Home [44], and VisDA-C [45]. Office-31 is a small-scale DA dataset consisting of images of 31 classes of common objects found in an office across 3 domains viz. Amazon (A), Webcam (W), and DSLR (D). Office-Home is a medium-sized DA dataset consisting of 4 domains viz. Art (A), Clipart (C), Product (P), and Real-World (R). The dataset contains images of 65 classes of items found in office and home environments. VisDA-C is a large-scale dataset consisting of 12 classes of objects across 2 domains: Synthetic (S) and Real (R). The 152K synthetic images are generated by 3D rendering and taken as the source domain. The 55K real samples are taken from MS COCO dataset [46] and taken as the target domain.

We follow the same protocol in [8, 9] for source training to ensure fairness for comparison. Our target models are initialized with ImageNet pretrained weights, since source model parameters are inaccessible. For Office-31 and Office-Home, we use ResNet50, and for VisDA-C we use ResNet101 as the backbone [47], on top of which we attach an MLP-based classifier, similar to [8, 9]. The target models are trained with SGD optimizer with $0.9$ momentum and weight decay $1e^{-3}$. The learning rate for the backbone is set to $1e^{-3}$, while that of the classifier is set to $1e^{-2}$. $\alpha$ in the curriculum factor is set to $2e^{-3}$ for Office-31, and $2e^{-4}$ for Office-Home and VisDA-C, depending on the size of the dataset. The model is adapted for 50 epochs for Office-31 and Office-Home datasets, and for 5 epochs for the VisDA-C dataset. Temperature sharpening factor $T$ is set to $0.5$. We implement our method using the PyTorch library on an NVIDIA-A100 GPU.

Liang et al. [8] pioneered this area and formulated the problem statement. They also presented a number of baselines for comparison. Among them NLL-KD and NLL-OT are inspired by noisy label learning and utilize KL divergence and optimal transport respectively for refining pseudolabels. HD-SHOT and SD-SHOT are based on the SHOT [6] model and treat the source model predictions as hard labels and soft labels, respectively. In addition to these baselines, we compare CABB against state-of-the-art black-box DA models DINE [8] and BETA [9]. We further compare against a number of standard DA methods, such as DANN [1], ALDA [37], GVB-GD [38], SRDC [22], SHOT [6], A²-Net [39], SFDA-DE [30] etc.

In Figure 4, we present the accuracy of the clean sample set after clean-noisy sample separation for the first epoch after distillation from the source teacher model to the target student model. We can see that our choice of low JSD separation criterion in CABB consistently outperforms the low CE loss criterion used in BETA by 1%-7% across all 12 source-target domain pairs for Office-Home dataset.

The classification accuracies after adaptation across the 6 domain pairs for Office-31 dataset are shown in Table 1. CABB outperforms BETA and DINE on average by 0.5% and 2.3%, respectively. While CABB beats DINE across all the domain pairs, it only underperforms BETA for Webcam-Amazon adaptation by 0.5%. Overall, CABB is on-par with white-box source-free model SHOT and non-source-free model ALDA.

The results for Office-Home dataset are presented in Table 2. CABB outperforms BETA and DINE by 0.7% and 3.1%, respectively. Moreover, CABB outperforms several standard non-source-free DA methods such as SRDC and FixBi, and is either better than, or on par with existing state-of-the-art white-box source-free DA models like HCL, $A^{2}$Net, and SFDA-DE.

A comparative evaluation of CABB against other state-of-the-art DA methods and DABP baselines on the VisDA-C dataset is shown in Table 3. CABB surpasses both DINE and BETA by $9.3\%$ and $2.3\%$, respectively in terms of mean-per-class accuracy. CABB beats BETA in the most challenging catergory truck by 10.4%. CABB also outperforms white-box source-free models SHOT and $A^{2}$Net comfortably.

A detailed ablation study on the efficacy of our curriculum adaptation method is given in Table 4. The impact of curriculum on the noisy set loss $\mathcal{L}_{tn}$ and entropy loss $\mathcal{L}_{ent}$ is shown, as curriculum is applied to these two components. In this table, in the absence of curriculum adaptation, $\gamma_{n}$ is set to $0.5$. In row 2, $\mathcal{L}_{tn}$ is set to $0$.

The results clearly indicate the benefit of a guided adaptation framework that progressively learns from the clean samples first and the noisy samples later. We see in the first three rows in Table 4 that without curriculum guidance, adaptation performance suffers significantly. In the absence of curriculum guidance, we see that leaving out learning from the noisy samples during the adaptation process is better than adapting to the noisy samples with $\mathcal{L}_{tn}$ loss, and further enforcing the wrong predictions with $\mathcal{L}_{ent}$ loss. The drawback of blindly adapting to noisy samples becomes evident in the second and third rows, particularly in the most challenging truck class. By adapting to unrefined noisy samples from the beginning, the model performance drastically deteriorates and accuracy on truck can fall to as low as $~{}1\%$.

The results in the 4th through 6th rows in Table 4 show the necessity for curriculum guidance during adaptation. In the presence of curriculum learning, CABB outperforms existing state-of-the-art DABP methods. Curriculum guidance progressively refines the noisy sample pseudolabels. While enforcing the refined predictions by minimizing the $\mathcal{L}_{ent}$ loss produces improved results, learning from the noisy pseudolabels by minimizing the $\mathcal{L}_{tn}$ loss significantly boosts the model performance. Minimizing losses $\mathcal{L}_{tn}$ and $\mathcal{L}_{ent}$ on the refined pseudolabels together produce the strongest results.