Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning

By introducing an intermediate class, the transition matrix $T$ can be factorized in the following way:

where $Y^{\prime}$ represent the random variable for the introduced intermediate class, ${T}^{\spadesuit}_{lj}(Y=i)=P(\bar{Y}=j|{Y}^{\prime}=l,Y=i)$, and ${T}^{\clubsuit}_{il}=P({Y}^{\prime}=l|Y=i)$. Note that ${T}^{\spadesuit}$ and ${T}^{\clubsuit}$ are two transition matrices representing the transition from the clean and intermediate class labels to the noisy class labels and transition from the clean labels to the intermediate class labels, respectively.

By looking at Eq. (2), it seems we have changed an easy problem into a hard one. However, this is totally not true. Actually, we break down a problem into simple sub-problems. Combining the solutions to the sub-problems gives a solution to the original problem. Thus, in philosophy, our idea belongs to the divide and conquer paradigm. In the rest of this subsection, we will explain why it is easy to estimate the transition matrices ${T}^{\spadesuit}$ and ${T}^{\clubsuit}$. Moreover, in the next subsection, we will theoretically compare the estimation error of the dual-$T$ estimator with that of the $T$ estimator.

It can be found that ${T}^{\clubsuit}_{ij}=P(Y^{\prime}=j|Y=i)$ has a similar form to ${T}_{ij}=P(\bar{Y}=j|Y=i)$. We can employ the same method that is developed for $T$, i.e., the $T$ estimator, to estimate ${T}^{\clubsuit}$. However, there seems to have two challenges: (1) it looks as if difficult to access $P(\bm{Y}^{\prime}|\bm{x})$; (2) we may also have an error for estimating $P(\bm{Y}^{\prime}|\bm{x})$. Fortunately, these two challenges can be well addressed by properly introducing the intermediate class. Specifically, we design the intermediate class $Y^{\prime}$ in such a way that $P(\bm{Y}^{\prime}|\bm{x})\triangleq\hat{P}(\bar{\bm{Y}}|\bm{x})$, where $\hat{P}(\bar{\bm{Y}}|\bm{x})$ represents an estimated noisy class posterior. Note that $\hat{P}(\bar{\bm{Y}}|\bm{x})$ can be obtained by exploiting the noisy data at hand. As we have discussed, due to the randomness of label noise, estimating $T$ directly will have a large estimation error especially when the noisy training sample size is limited. However, as we have access to $P(\bm{Y}^{\prime}|\bm{x})$ directly, according to Eq. (1), the estimation error for ${T}^{\clubsuit}$ is zero if anchor points are given²²2If the anchor points are to learn, the estimation error remains unchanged for the $T$ estimator and dual-$T$ estimator by employing $\bm{x}^{i}=\arg\max_{\bm{x}}\hat{P}(\bar{Y}=i|\bm{x})$..

Although the transition matrix $T^{\spadesuit}$ contains three variables, i.e., the clean class, intermediate class, and noisy class, we have class labels available for two of them, i.e., the intermediate class and noisy class. Note that the intermediate class labels can be assigned by using $P(\bm{Y}^{\prime}|\bm{x})$. Usually, the clean class labels are not available. This motivates us to find a way to eliminate the dependence on clean class for $T^{\spadesuit}$. From an information-theoretic point of view [5], if the clean class $Y$ is less informative for the noisy class $\bar{Y}$ than the intermediate class $Y^{\prime}$, in other words, given $Y^{\prime}$, $Y$ contains no more information for predicting $\bar{Y}$, then $Y$ is independent of $\bar{Y}$ conditioned on $Y^{\prime}$, i.e.,

A sufficient condition for holding the above equalities is to let the intermediate class labels be identical to noisy labels. Note that it is hard to find an intermediate class whose labels are identical to noisy labels. The mismatch will be the main factor that contributes to the estimation error for $T^{\spadesuit}$. Note also that since we have labels for the noisy class and intermediate class, $P(\bar{Y}=j|{Y}^{\prime}=l)$ in Eq. (3) is easy to estimate by just counting the discrete labels, and it will have a small estimation error which converges to zero exponentially fast [3].

Based on the above discussion, by factorizing the transition matrix $T$ into $T^{\spadesuit}$ and $T^{\clubsuit}$, we can change the problem of estimating the noisy class posterior into the problem of fitting the noisy labels. Note that the noisy class posterior is in the range of $[0,1]$ while the noisy class labels are in the set $\{1,\ldots,C\}$. Intuitively, learning the class labels are much easier than learning the class posteriors. In Section 4, our empirical experiments on synthetic and real-world datasets further justify this by showing a significant error gap between the estimation error of the $T$ estimator and dual-$T$ estimator.

Implementation of the dual-$T$ estimator. The dual-$T$ estimator is described in Algorithm 1. Specifically, the transition matrix $T^{\clubsuit}$ can be easily estimated by letting $P({Y}^{\prime}=i|\bm{x})\triangleq\hat{P}(\bar{Y}=i|\bm{x})$ and then employing the $T$ estimator (see Section 2). By generating intermediate class labels, e.g., letting $\operatorname*{arg\,max}_{i\in\{1,\dots,C\}}P({Y}^{\prime}=i|\bm{x})$ be the label for the instance $\bm{x}$, the transition matrix $T^{\spadesuit}$ can be estimating via counting, i.e.,

where $\mathbb{1}_{\{A\}}$ is an indicator function which equals one when $A$ holds true and zero otherwise, $(\bm{x}_{i},\bar{y}_{i})$ are examples from the training sample $S_{\text{tr}}$, and $\land$ represents the AND operation.

Many statistically consistent algorithms [8, 27, 46, 38] consist of a two-step training procedure. The first step estimates the transition matrix and the second step builds statistically consistent algorithms, for example, via modifying loss functions. Our proposed dual-$T$ estimator can be seamlessly embedded into their frameworks. More details can be found in Section 4.

In this subsection, we will justify that the estimation error could be greatly reduced if we estimate ${T}^{\spadesuit}$ and ${T}^{\clubsuit}$ rather than estimating $T$ directly.

As we have discussed before, the estimation error of the $T$ estimator is caused by estimating the noisy class posterior; the estimation error of the dual-$T$ estimator comes from the estimation error of $T^{\spadesuit}$, i.e., fitting the noisy class labels and estimating $P(\bar{Y}|Y^{\prime})$ by counting discrete labels. Note that to eliminate the dependence on the clean label for $T^{\spadesuit}$, we need to achieve $P(Y^{\prime}=\bar{Y}|\bm{x})=1$. Let the estimation error for the noisy class posterior be $\Delta_{1}$, i.e., $\left|P(\bar{Y}=j|\bm{x})-\hat{P}(\bar{Y}=j|\bm{x})\right|=\Delta_{1}$. Let the estimation error for $P(\bar{Y}=j|Y^{\prime}=l)$ by counting discrete labels is $\Delta_{2}$, i.e., $|P(\bar{Y}=j|Y^{\prime}=l)-\hat{P}(\bar{Y}=j|Y^{\prime}=l)|=\Delta_{2}$. Let the estimation error for fitting the noisy class labels is $\Delta_{3}$, i.e., $|P(\bar{Y}=j|Y^{\prime}=l,Y=i,\bm{x})-P(\bar{Y}=j|Y^{\prime}=l,\bm{x})|=\Delta_{3}$. We will show that under the following assumption, the estimation error of the dual-$T$ estimator is smaller than the estimation error the $T$ estimator.

Assumption 1 is easy to hold. Theoretically, the error $\Delta_{2}$ involves no predefined hypothesis space, and the probability that $\Delta_{2}$ is larger than any positive number will converge to zero exponentially fast [3]. Thus, $\Delta_{2}$ is usually much smaller than $\Delta_{1}$ and $\Delta_{3}$. We therefore focus on comparing $\Delta_{1}$ with $\Delta_{3}$ by ignoring $\Delta_{2}$. Note that when $\hat{p}(\bar{Y}|\bm{x})=p(\bar{Y}|\bm{x})$, we have $\Delta_{1}=0$; when $Y^{\prime}=\bar{Y}$, we have $\Delta_{3}=0$. Intuitively, the error $\Delta_{3}$ is smaller than $\Delta_{1}$ because it is easy to obtain a small estimation error for fitting noisy class labels than that for estimating noisy class posteriors. We note that the noisy class posterior is in the continuous range of $[0,1]$ while the noisy class labels are in the discrete set $\{1,\ldots,C\}$. For example, suppose we have an instance $(\bm{x},\bar{y})$, then, as long as the empirical posterior probability $\hat{P}(\bar{Y}=\bar{y}|\bm{x})$ is greater than $1/C$, the noisy label will be accurately learned. However, the estimated error of the noisy class posterior probability can be up to $1-1/C$. We also empirically verify the relation among these errors in Appendix 2.

We compare the estimation error between our estimator and the $T$ estimator on both synthetic and real-world datasets with different sample size and different noise types. The synthetic dataset is created by sampling from 2 different 10-dimensional Gaussian distributions. One of the distribution has unit variance and zero mean among all dimension. Another one has unit variance and mean of two among all dimensions. The real-world image datasets used to evaluate transition matrices estimation error are MNIST [17], F-MINIST [40], CIFAR10, and CIFAR100 [16].

We conduct experiments on the commonly used noise types [11, 38]. Specifically, two representative structures of the transition matrix $T$ will be investigated: Symmetry flipping (Sym-$\epsilon$) [27]; (2) Pair flipping (Pair-$\epsilon$) [11]. To generate noisy datasets, we corrupt the training and validation set of each dataset according to the transition matrix $T$.

Neural network classifiers are used to estimate transition matrices. For fair comparison, the same network structure is used for both estimators. Specifically, on the synthetic dataset, a two-hidden-layer network is used, and the hidden unit size is $25$; on the real-world datasets, we follow the network structures used by the state-of-the-art method [27], i.e., using a LeNet network with dropout rate $0.5$ for MNIST, a ResNet-$18$ network for F-MINIST and CIFAR10, a ResNet-$34$ network for CIFAR100, and a ResNet-$50$ pre-trained on ImageNet for Clothing1M. The network is trained for 100 epochs, and stochastic gradient descent (SGD) optimizer is used. The initial learning rate is $0.01$, and it is decayed by a factor $10$ after $50$-th epoch. The estimation error is calculated by measuring the $\ell_{1}$-distance between the estimated transition matrix and the ground truth $T$. The average estimation error and the standard deviation over $5$ repeated experiments for the both estimators is illustrated in Fig. 2 and 3.

It is worth mention that our methodology of anchor points estimation are different from the original paper of $T$ estimator [27]. Specifically, the original paper [27] includes a hyper-parameter to estimate anchor points for different datsets, e.g., selecting the points with $97\%$ and $100\%$ largest estimated noisy class posteriors to be anchor points on CIFAR-10 and CIFAR-100, respectively. However, clean data or prior information is needed to estimate the value of the hyper-parameter. By contrast, we assume that we only have noisy data, then it remains unclear to select the value of the hyper-parameter. In this case, from a theoretical point of view, we discard the hyper-parameter and select the points with the largest estimated intermediate class posteriors to be the anchor points for all datasets. This estimation method has been proven to be a consistent estimation of anchor points in binary case, and it also holds for multi-class classification when the noise rate is upper bounded by a constant [4]. Note that $T$ estimator [27] also selects the points with the largest estimated class posteriors to be anchor points on some datasets, which is the same as ours. Comparison on those datasets could directly justify the superiority of the proposed method. In addition, to estimate the anchor points, we need to estimate the noisy class posterior. However, deep networks are likely to overfit label noise [47, 1]. To prevent overfitting, we use $20\%$ training examples for validation, and the model with the best validation accuracy is selected for estimating anchor points. The anchor points are estimated on training sets.

Fig. 2 illustrates the estimation error of the $T$ estimator and the dual-$T$ estimator on the synthetic dataset. For two different noise types and sample sizes, the estimation error of the both estimation methods tend to decrease with the increasing of the training sample size. However, the estimation error of the dual-$T$ estimator is continuously smaller than that of the $T$ estimator. Moreover, the estimation error of the dual-$T$ estimator is less sensitive to different noise types compared to the $T$ estimator. Specifically, even the $T$ estimator is trained with all the training examples, its estimation error on Pair-$45\%$ noise is approximately doubled than that on Sym-$20$ noise, which is observed by looking at the right-hand side of the estimation error curves. In contrast, when training the dual $T$ estimator

with all the training examples, its estimation error on the different noise types does not significantly different, which all less than $0.1$. Similar to the results on the synthetic dataset, the experiments on the real-world image datasets illustrated in Fig. 3 also shows that the estimation error of the dual-$T$ estimator is continuously smaller than that of the $T$ estimator except CIFAR100, which illustrates the effectiveness of the proposed dual-$T$ estimator. On CIFAR100, both estimators have a larger estimation error compared to the results on MNIST, F-MINIST, and CIFAR10. The dual-$T$ estimator outperforms the $T$ estimator with the large sample size. However, when the training sample size is small, the estimation error of the dual-$T$ estimator can be larger than that of the $T$ estimator. Because the numbers of images per class are too small to estimate the transition matrix $T^{\spadesuit}$, leading to a large estimation error.

We investigate how the estimation of the $T$ estimator and the dual-$T$ estimator will affect the classification accuracy in label-noise learning. The experiments are conducted on MNIST, F-MINIST, CIFAR10, CIFAR100, and Clothing1M. The classification accuracy are reported in Table 1 and Table 2. Eight popular baselines are selected for comparison, i.e., Coteaching [11], and MentorNet [14] which use diagonal entries of the transition matrix to help selecting reliable examples used for training; Forward [27], and Revision [38], which use the transition matrix to correct hypotheses; Reweighting [20], which uses the transition matrix to build risk-consistent algorithms. There are three baselines without requiring any knowledge of the transition matrix, i.e., CE, which trains a network on the noisy sample directly by using cross entropy loss; Decoupling [24], which trains two networks and updates the parameters only using the examples which have different prediction from two classifiers; Mixup [48] which reduces the memorization of corrupt labels by using linear interpolation to feature-target pairs. The estimation of the $T$ estimator and the dual-$T$ estimator are both applied to the baselines which rely on the transition matrix. The baselines using the estimation of $T$ estimator are called $T$ Coteaching, $T$ MentorNet, $T$ Forward, $T$ Revision, and $T$ Reweighting. The baselines using estimation of dual-$T$ estimator are called $DT$ Coteaching, $DT$ MentorNet, $DT$ Forward, $DT$ Revision, and $DT$ Reweighting.

The settings of our experiments may be different from the original paper, thus the reported accuracy can be different. For instance, in the original paper of Coteaching [11], the noise rate is given, and all data are used for training. In contrast, we assume the noise rate is unknown and needed to be estimated. We only use $80\%$ data for training, since $20\%$ data are leaved out as the validation set for transition matrix estimation. In the original paper of $T$ revision [38], the experiments on Clothing1M use clean data for validation. In contrast, we only use noisy data for validation.

In Table 1 and Table 2, we bold the better classification accuracy produced by the baseline methods integrated with the $T$ estimator or the dual-$T$ estimator. The best classification accuracy among all the methods in each column is highlighted with $*$. The tables show the classification accuracy of all the methods by using our estimation is better than using that of the $T$ estimator for most of the experiments. It is because that the dual-$T$ estimator leads to a smaller estimation error than the $T$ estimator when training with large sample size, which can be observed at the right-hand side of the estimation error curves in Fig. 3. The baselines with the most significant improvement by using our estimation are Coteaching and MentorNet. $DT$ Coteaching outperforms all the other methods under Sym-$20\%$ noise. On Clothing1M dataset, $DT$ revision has the best classification accuracy. The experiments on the real-world datasets not only show the effectiveness of the dual-$T$ estimator for improving the classification accuracy of the current noisy learning algorithms, but also reflect the importance of the transition matrix estimation in label-noise learning.

We empirically verify the relations among the three different errors in Assumption 1 on different type of classifiers. Note that $\Delta_{1}$ is the estimation error for the noisy class posterior, i.e., $\Delta_{1}=\left|P(\bar{Y}=j|\bm{x})-\hat{P}(\bar{Y}=j|\bm{x})\right|$; $\Delta_{2}$ is the estimation error for counting discrete labels, i.e., $|P(\bar{Y}=j|Y^{\prime}=l)-\hat{P}(\bar{Y}=j|Y^{\prime}=l)|=\Delta_{2}$; $\Delta_{3}$ is the estimation error for fitting the noisy class labels, i.e., $|P(\bar{Y}=j|Y^{\prime}=l,Y=i,\bm{x})-P(\bar{Y}=j|Y^{\prime}=l,\bm{x})|=\Delta_{3}$.

The experiments are conducted on the synthetic dataset, and setting is same as those of the synthetic experiments in Section $4$. The three errors are calculated on the training set, since both the $T$ estimator and the dual-$T$ estimator estimates the transition matrix on the training set. We employ a $16$-layer neural network with short cut connection [13] for feature extraction. Then the extracted features are passed to a linear classifier by employing cross entropy loss and a logistic regression classifier by employing logistics loss, respectively.

Both Figure 4 and Figure 5 show that the error $\Delta_{2}$ is very small and can be ignored. $\Delta_{3}$ is continuously smaller than $\Delta_{1}$ when the sample size is small. The recent work [6] shows that the sample complex of the network is linear in the number of parameters, which means that, usually, we may not have enough training examples to learn the noisy class posterior well (e.g., CIFAR10, CIFAR100, and Fashion-MNIST), and Assumption 1 can be easily satisfied. It is also worth to mention that, even Assumption 1 does not hold, the estimation error of the dual-$T$ estimator may also be smaller than the $T$ estimator. Specifically, the error $\epsilon_{DT}$ of the proposed estimator is upper bounded by $C^{2}(\Delta_{2}+\Delta_{3})$. Generally, the increasing of the upper bound $C^{2}(\Delta_{2}+\Delta_{3})$ does not imply the increasing of the error $\epsilon_{DT}$. Figure 4 and Figure 5 also show that different types of the classifiers have different capabilities to fit the noisy labels. Specifically, the linear classifier has smaller error $\Delta_{3}$ on pair flipping $45\%$ compared to that on symmetry flipping $20\%$, by contrast, the logistic regression classifier has larger error $\Delta_{3}$ on pair flipping $45\%$ compared to that on symmetry flipping $20\%$.