ST-CoNAL: Consistency-Based Acquisition Criterion Using Temporal Self-Ensemble for Active Learning

The acquisition methods for pool-based AL can be roughly categorized into three classes: 1) uncertainty-based, 2) representation-based and 3) consistency-based methods. The uncertainty-based methods [2, 9, 10, 19, 26, 42] estimate the prediction uncertainty for the given data instances. Max-entropy [26, 36], least confidence [26] and variation ratio [18] are the widely used criteria. Recently, the representation-based method [38, 47, 20, 34] significantly improved the acquisition performance. VAAL [38] performed adversarial learning to find informative unlabeled data in the task-agnostic latent representation of the data. TA-VAAL [20] is the latest state-of-the-art method, which incorporated a task-aware uncertainty-based approach to the VAAL baseline. The consistency-based methods measure the disagreement among the predictions of the ensemble models [9, 10, 15, 11]. TOD-Semi [16], Query-by-committee [35], and DBAL [10] are the well known criteria used in these methods.

One of the most successful approaches to SSL is consistency regularization. The $\Pi$ model [22, 33] enforces consistency for model $f_{\theta}(x)$ by minimizing $d(f_{\theta}(x),f_{\theta}(\hat{x}))$, where $\hat{x}$ denotes the data perturbed by noise and augmentation, and $d(a,b)$ measures the distance between $a$ and $b$. While $\Pi$ model uses the predictions as a target, the consistency loss for regularizing the consistent behavior between an ensemble prediction and the model prediction was shown to be effective. Mean teacher method [40] generates the weights of the teacher model with EMA of the student weights over SGD iterations. ICT [43] and MixMatch [4] enforce consistency between linearly interpolated inputs and model predictions.

Several studies have attempted to use temporal self-ensemble obtained through model optimization to improve the inference performance [22, 6, 25, 17]. Izmailov et al. [17] proposed SWA method, which determines an improved inference model using the equally weighted average of the intermediate model weights traversed via SGD iterates. Using cyclical learning rate scheduling [27], SGD can yield the weights in the flat regions in the loss surface. By averaging the weights found at SGD trajectory, SWA can find the weight solution that can generalize well. In [1], consistency regularization and modified cyclical learning rate scheduling assisted in finding the improved weights through SWA.

The combination of SSL and AL was discussed in several works [8, 10, 11, 32, 41, 48]. However, most works considered applying SSL and AL independently. Only a few works considered the joint design of SSL and AL. Zhu et al. [48] used a Gaussian random field on a weighted graph to combine SSL and AL. Additionally, the authors of [11] proposed an enhanced acquisition function by adopting the variance over the data augmentation of the input image.

The proposed algorithm follows the setup of a pool-based AL for the $K$-image class classification task. Suppose that the labeling budget is fixed to $b$ data samples. In the $j$th sample acquisition step, we are given the dataset $\mathcal{D}^{j}$, which consists of a set of labeled data $\mathcal{D}_{L}^{j}$ and a set of unlabeled data $\mathcal{D}_{U}^{j}$. $f_{\mathcal{D}^{j}}$ is the model trained with dataset $\mathcal{D}^{j}$. We consider two cases; 1) $f_{\mathcal{D}^{j}}$ is trained with only $\mathcal{D}_{L}^{j}$ via supervised learning and 2) $f_{\mathcal{D}^{j}}$ is trained with $\mathcal{D}_{L}^{j}\cup\mathcal{D}_{U}^{j}$ via SSL. The AL algorithm aims to acquire the set of $b$ data samples $\{x_{1}^{*},...,x_{b}^{*}\}$ from the pool of unlabeled data $\mathcal{D}_{U}^{j}$. To select the data samples to be labeled, the acquisition function $a_{\rm ST-CoNAL}\left(x_{i},f_{\mathcal{D}^{j}}\right)$ is used to score the given data instance $x_{i}$ with the currently trained model $f_{\mathcal{D}^{j}}$. The $b$ samples that yield the highest score are selected as

The selected samples are then labeled, i.e., $\ \mathcal{D}^{j+1}_{L}\leftarrow\mathcal{D}^{j}_{L}\cup\{x_{1}^{*},...,x_{b}^{*}\}$ and $\mathcal{D}^{j+1}_{U}\leftarrow\mathcal{D}^{j}_{U}\setminus\{x_{1}^{*},...,x_{b}^{*}\}$, and the model is retrained using the newly labeled dataset $\mathcal{D}^{j+1}=\{\mathcal{D}^{j+1}_{L},\mathcal{D}^{j+1}_{U}\}$. This procedure is repeated until a labeled dataset is obtained as large as required within the limited budget.

Suppose that we have $Q$ student models $f_{\mathcal{D}^{j}}^{(s,1)},...,f_{\mathcal{D}^{j}}^{(s,Q)}$ and one teacher model $f_{\mathcal{D}^{j}}^{(t)}$ trained with the dataset $\mathcal{D}^{j}$. Each model produces the $K$ dimensional output through the softmax output layer. The method for obtaining these models will be discussed in the next section. We first apply sharpening [4, 3, 44] to the output of the teacher model as

where $f(k)$ denotes the $k$th element of the output produced by the function $f(\cdot)$ and $T$ denotes the temperature hyper-parameter. Then, the acquisition function of the ST-CoNAL is obtained by accumulating the KL divergence between the predictions produced by each student model and the teacher model, i.e.,

where $KL(p||q)=\sum_{i}p_{i}\log(p_{i}/q_{i})$. The KL divergence in this acquisition function measures the inconsistency between the student and teacher models. As the sharpening makes the prediction of teacher model to have low-entropy, the high KL divergence value reflects the uncertainty in the prediction of the student models.

The ST-CoNAL computes the student and teacher models using the temporal self-ensemble networks. The parameters of these self-ensemble networks are obtained by capturing the network weights at the intermediate check points of the weight trajectory. The study in [1] revealed that the average of the weights obtained through the SGD iterates can yield the weights corresponding to the flat minima of the loss surface, which are known to generalize well [29, 45]. In ST-CoNAL, the average of these weights forms a teacher model. To obtain the diverse student models, we adopt the learning rate (LR) schedule used for SWA [17, 29, 45]. The learning rate $l$ used for our method is given by

where $l_{0}$ denotes the initial learning rate, $\gamma<1$ denotes the parameter for learning rate decay, $t$ is the current training epoch, and $T_{0}$ denotes the training epoch at which the learning rate will be switched to a smaller value. After running $T_{0}$ epochs (i.e., $t\geq T_{0}$), the $Q$ intermediate network weights are stored every $c$ epoch, which constitute the weights of the student networks. We obtain $Q$ self-ensemble networks $f_{\mathcal{D}^{j}}^{(s,1)},...,f_{\mathcal{D}^{j}}^{(s,Q)}$ and these models are treated as the student models. The teacher model $f_{\mathcal{D}^{j}}^{(t)}$ is then obtained by taking the equally weighted average of the parameters of $f_{\mathcal{D}^{j}}^{(s,1)},...,f_{\mathcal{D}^{j}}^{(s,Q)}$. Note that this learning rate schedule does not require any additional training cost for obtaining the temporal self-ensemble.

Algorithm 1 presents the summary of the ST-CoNAL algorithm.

Datasets. We evaluated ST-CoNAL on four benchmarks: CIFAR-10 [21], CIFAR-100 [21], Caltech-256 [13] and Tiny ImageNet [23]. CIFAR-10 contains $50k$ training examples with 10 categories and CIFAR-100 contains $50k$ training examples with 100 categories. Caltech-256 has $24,660$ training examples with 256 categories and Tiny ImageNet has $100k$ training examples with 200 categories.

To see how ST-CoNAL performed on class-imbalanced scenarios, we additionally used the synthetically imbalanced CIFAR-10 datasets; the step imbalanced CIFAR-10 [20, 7] and the long-tailed CIFAR-10 [5, 7]. The step imbalance CIFAR-10 has 50 samples for the first five smallest classes and 5,000 samples for the last five largest classes. Long-tailed CIFAR-10 followed the configuration used in [5], where 12,406 training images were generated with 5,000 samples from the largest class and 50 samples from the smallest class. The imbalance ratio of 100 is the widely used setting adopted in the literature [20, 5].

In all experiments, the temperature parameter $T$ was set to $0.7$, which was determined through empirical optimization. For CIFAR-10, we set $\gamma=1.0,c=10,b=1k$ and $|\mathcal{S}|=10k$, and for CIFAR-100, we set $\gamma=0.5,c=10,b=2k$ and $|\mathcal{S}|=10k$. For Tiny ImageNet, we set $\gamma=0.5,c=10,b=2k$ and $|\mathcal{S}|=20k$. For Caltech-256, we set $\gamma=0.3,c=10,b=1k$ and $|\mathcal{S}|=10k$. These configurations followed the convention adopted in the existing methods. The parameter $T_{0}$ used for the learning rate scheduling was set to 160 epochs for all datasets except Tiny ImageNet. We used $T_{0}$ of 60 epochs for Tiny ImageNet.

CIFAR-10 and CIFAR-100: Fig. 2 (a) and (b) show the performance of ST-CoNAL with respect to the number of labeled samples evaluated on CIFAR-10 and CIFAR-100 datasets, respectively. As mentioned earlier, we use the performance difference obtained compared to the classification accuracy of random sampling as a performance indicator. We observe that the proposed ST-CoNAL outperforms other AL methods on both CIFAR-10 and CIFAR-100. After the last acquisition step, our method achieves a performance improvement of 4.68% and 5.47% compared to the random sampling baseline on CIFAR-10 and CIFAR-100 datasets, respectively. It should be noted that ST-CoNAL achieves performance comparable to TA-VAAL, a state-of-the-art method that requires the addition of subnetworks and complex training optimization. Entropy shows a good performance at the beginning acquisition steps but the performance improves slowly as compared to other methods. After the last acquisition, ST-CoNAL achieves 1.37% and 4.15% higher accuracy than Entropy on CIFAR-10 and CIFAR-100, respectively. Our method outperforms CSSAL, another consistency-based AL method that only considers data uncertainty, not model uncertainty.

Caltech-256 and Tiny ImageNet: Fig. 3 (a) and (b) show the performance of several AL methods on Caltech-256 and Tiny ImageNet, respectively. The ST-CoNAL also achieves significant performance gain on these datasets. After the last acquisition step, ST-CoNAL achieves 3.92% higher accuracy than the random sampling baseline on Caltech-256 while it achieves 1.92% gain on Tiny-Imagenet dataset. Note that the proposed method far outperforms the CSSAL and performs comparable to more sophisticated representation-based methods, Coreset and TA-VAAL.

Class-Imbalanced Datasets. We also evaluate ST-CoNAL on class imbalanced datasets. Fig. 4 (a) and (b) present the performance of ST-CoNAL on (a) the step imbalanced CIFAR-10 [20] and (b) the long-tailed CIFAR-10 [5]. We see that ST-CoNAL achieves remarkable performance improvements compared to random sampling on the class imbalanced datasets. Specifically, after the last acquisition step, ST-CoNAL achieves a 9.2% performance improvement on the step-imbalanced CIFAR-10 and a 15.41% improvement on the long-tailed CIFAR-10. Furthermore, ST-CoNAL achieves larger performance gain over the current state-of-the-art, TA-VAAL. It achieves 3.58% and 4.86% performance gains over TA-VAAL on the step imbalanced CIFAR-10 and the long-tailed CIFAR-10, respectively. Note that the consistency-based method CSSAL does not perform well under this class-imbalanced setup.

SSL Setups. Fig. 5 (a) and (b) show the performance evaluated on CIFAR-10 and CIFAR-100 datasets when ST-CoNAL is applied to SSL. ST-CoNAL maintains performance gains even in SSL settings and outperforms other AL methods by a larger margin than in supervised learning settings. After the last acquisition step, ST-CoNAL achieves the 4.87% and 6.66% performance gains over the random sampling baseline on CIFAR-10 and CIFAR-100, respectively. In particular, Fig. 5 (b) shows that ST-CoNAL achieves substantial performance gains over the existing methods in CIFAR-100.

In this subsection, we investigate the benefit of using self-ensemble for an acquisition criterion through some experiments.