Unsupervised Learning of Debiased Representations with Pseudo-Attributes

Real-world datasets are inevitably biased due to their insufficiently controlled collection process, and consequently, deep neural networks often capture unintended correlations between the true labels and the spuriously correlated ones. Measuring and mitigating the potential risks posed by dataset or algorithmic bias has been extensively investigated in various computer vision tasks [6, 31, 21, 16, 33, 3, 36]. For example, VQA models frequently exploits statistical regularities between answer occurrences and patterns of questions while ignoring visual information [3, 7]. Semantic segmentation models typically take advantage of scene context for pixel-level predictions of semantic labels [6]. To prevent using undesirable correlation in biased datasets, existing approaches often rely on human supervision for bias annotations and present several technical directions such as data augmentation [13, 39], model ensembles [3, 7], and statistical regularizations [1]. Such supervised debiasing techniques have been applied to various computer vision tasks by exploiting known application-specific bias information, including uni-modality of the dataset in visual question answering [3], stereotype textures in image recognition [13], temporal invariance in action recognition [20], and demographic information in facial image recognition [26, 39, 35].

Distribution shift has recently emerged as a critical challenge in machine learning, where the goal of the optimization is to learn a robust model in a test environment with a different distribution. Distributionally robust optimization (DRO) [2, 11, 9, 17] has been proposed to improve the worst-case generalization performance over a family of target distributions, and has provided theoretical background of Group DRO [26] and its variation [30]. However, the objective of DRO often leads to an overly conservative model and results in performance degeneration on unseen environments [10, 15]. To relax the constraints for the uncertainty set of test distributions, some approaches pose additional assumptions. For instance, the dataset consists of multiple groups with the shared properties and the uncertainty set is represented by a mixture of these groups. This assumption is also used in robust federated learning [24, 19], algorithmic fairness [37, 5, 8], and domain generalization [18, 4, 29]. Our framework also takes advantage of this assumption but does not rely on the supervision of group information.

There exist several generic debiasing techniques via sample reweighting based on observed task-specific losses under the supervised environment [26] or the unsupervised setting [25, 30]. Group DRO [26] exploits the group information specified by the bias attributes and aims to improve the worst-group generalization performance. On the other hand, Nam et al. [25] employ the difference between the generalized and standard cross-entropy loss to capture the bias-alignment for sample reweighting while Sohoni et al. [30] estimate subclass labels via clustering and utilize the information for distributionally robust optimization to mitigate hidden stratification. Although the unsupervised approaches work well in small and artificial datasets such as MNIST, their performance improvement becomes marginal in real-world datasets including CelebA. Our framework also belongs to unsupervised methods that do not rely on the bias information to learn debiased representations.

Let an example $\mathbf{x}$ be associated with a set of $m$ attributes $\mathcal{A}:=\{a_{1},...,a_{m}\}$. The goal of our model is to predict a target attribute $a_{t}\in\mathcal{A}$ by estimating the intended causation $p(a_{t}|\mathbf{x})$, which does not involve any undesirable correlation to other latent attributes, i.e., $p(a_{t}|\mathbf{x})=p(a_{t}|\mathbf{x},a_{i})$, $\forall a_{i}\in\mathcal{A}-\{a_{t}\}$. On the other hand, spurious correlation indicates strong coincidence between two attributes $a_{i},a_{j}\in\mathcal{A}$; the conditional entropy $H(a_{i}|a_{j})$ is close to zero and there exists no causal relationship between them. A machine learning algorithm is considered biased if a certain attribute $a_{b}\in\mathcal{A}$ has a spurious correlation with the target attribute $a_{t}$ and affects the prediction, i.e., $p(a_{t}|\mathbf{x})\neq p(a_{t}|\mathbf{x},a_{b})$. Our approach performs debiasing by estimating groups in the dataset without supervision, where the group is defined by a pair of target and bias attributes, e.g., $g=(a_{t},a_{b})$.

If a bias attribute is highly correlated to a target attribute while being easy to learn, the model may ignore few conflicting examples and learn its decision rule based on the bias attributes with spurious correlations to maximize accuracy [25, 27]. To prevent this undesirable situation, a simple group upweighting or resampling strategies [27] are known to be effective while they work poorly in realistic scenarios, where the bias information is unknown during training.

To overcome this challenge, from our intuition, we analyze the feature semantics over the target and bias attributes. We first naïvely train a base model on the CelebA dataset to classify hair color, and visualize the representation of the examples after convergence with a sufficient number of epochs ($T=100$). We select gender as a bias attribute, but do not utilize any information of the bias attribute during training. It turns out that, even without using the bias information during training, the examples drawn from certain groups, which are given by a combination of hair color and gender attribute values in this case, e.g., (male, non-blonde) and (female, blonde), are located closely in the feature space. This observation implies that it is possible to identify bias pseudo-attributes by taking advantage of the embedding results even without attribute supervisions. Our unsupervised debiasing framework is based on the capability to identify the bias pseudo-attributes via clustering.

Suppose that training examples, $(\mathbf{x},y)$, are drawn from a certain distribution $\tilde{P}$. Given a loss function $\ell(\cdot)$ and model parameters $\theta$, the objective of the empirical risk minimization (ERM) is to optimize the following expected loss:

where ${P}$ is the empirical distribution over training data that approximates the true distribution $\tilde{P}$. Although ERM generally works well, it tends to ignore examples in minority groups that conflict with bias attributes and implicitly assumes the consistency of the underlying distributions for training and testing data. Consequently, the approach often leads to high unbiased and worst-group test error [9, 26].

Several distributionally robust optimization (DRO) techniques [2, 9, 17] can be employed to tackle the dataset bias and distribution shift problems and maximize unbiased generalization accuracy. They consider a particular uncertainty set $\mathcal{Q}_{P}$, which is close to the training distribution $P$, e.g., $\mathcal{Q}_{P}=\{Q:D_{f}[Q||P]\leq\delta\}$, where $D_{f}[\cdot||\cdot]$ indicates an $f$-divergence function²²2Let $P$ and $Q$ be probability distributions over a space $\Omega$, then f-divergence is $D_{f}(P||Q)=\int_{\Omega}f\big{(}\frac{dP}{dQ}\big{)}dQ$.. To minimize the worst-case loss over the uncertainty distribution set $\mathcal{Q}_{P}$, DRO optimizes

However, this objective is overly pessimistic and makes the model consider the implausible worst cases [10, 15].

The group distributionally robust optimization, referred to as group DRO [26] creates more realistic sets of possible test distributions by leveraging the prior knowledge of group information. They assumes the training distribution $P$ is a mixture of $G$ groups, $P_{g}$, which is given by

where $\mathcal{G}=\{1,...,G\}$ and $\Delta_{G}$ is a $(G-1)$-dimensional simplex. Then the uncertainty set $\mathcal{Q}_{P}$ is defined by a set of all possible mixtures of these groups, i.e., $\mathcal{Q}_{P_{\mathcal{G}}}=\{\sum_{g\in\mathcal{G}}c_{g}P_{g}:c\in\Delta_{G}\}$. Because $\mathcal{Q}_{P_{\mathcal{G}}}$ is a simplex, its optimum is achieved at a vertex, thus minimizing the worst-case risk of $\mathcal{Q}_{P_{\mathcal{G}}}$ is equivalent to

Different from the group DRO setting, we do not know the group assignment for each training example. Instead, we use the bias pseudo-attribute information, obtained by any clustering algorithm in the feature embedding space, to define groups. Note that the clustering is performed with the representations given by the base model trained without debiasing, which is parameterized by $\tilde{\theta}$. Our goal is to alleviate dataset bias and maximize unbiased accuracy, and we need to consider all groups fairly for optimization. To this end, we assign a proper importance weight, $\omega_{k}$, to the $k^{\text{th}}$ cluster, where $k\in\mathcal{K}=\{1,...,K\}$, and the final objective of our framework is given by minimizing a weighted empirical risk as follows:

where $h((\mathbf{x},y);\tilde{\theta})$ denotes the cluster membership of an example $(\mathbf{x},y)$. The details of the weight assignment method will be discussed next.

Based on our observation described in Section 3.2, we first cluster training examples in each class on the feature embedding space defined by the base model optimized sufficiently, e.g., for 100 epochs using the standard classification loss. We suppose that each cluster corresponds to a bias pseudo-attribute. Among all clusters, we focus on the examples in the minority clusters, especially when they have high average losses. A common failure case in the presence of dataset bias is incurred by ignoring specific subpopulation groups for minimizing the overall training loss, and minority clusters are prone to be ignored due to their sizes. The problematic cases among the clusters are the ones that contain many bias-conflicting examples, having high losses, and thus result in poor worst-case errors. If the minority clusters consist of mostly bias-aligned samples, they will apparently achieve high classification accuracy.

Therefore, to handle dataset bias issue, we should consider both scale and average difficulty (loss) of each cluster, unlike group DRO [26] and George [30], which focus only on the average loss. We calculate the importance weight of each cluster by our reweighting scheme to train the target model, which is given by

where $\theta$ and $\tilde{\theta}$ indicates the parameters of the final and base models, respectively, $h(\cdot,\cdot)$ is a cluster membership function, and $\mathds{1}(\cdot)$ is an indicator function. Note that ${P_{k}}$ denotes the sample distribution of the $k^{\text{th}}$ cluster and $N_{k}$ is the number of samples in the $k^{\text{th}}$ cluster, where $k\in\mathcal{K}=\{1,...,K\}$.

Algorithm 1 presents the optimization procedure of the proposed framework. We first näively train a baseline network (line 4-7), parameterized by $\tilde{\theta}$. Then, we cluster all training examples based on the features extracted from the network to obtain the membership distribution $P_{k}$ and the size of cluster $N_{k}$ (line 8-11). Based on the cluster assignments, we calculate the importance weight of each cluster $\omega_{k}$ using the target model, parameterized by $\theta$, where the weight is updated by exponential moving average at each iteration (line 15). We finally use the normalized importance weight of each sample $\overline{\alpha}_{i}$ over a mini-batch to train the target model (line 18-19).

CelebA [22] is a large-scale face dataset for face image recognition, containing 40 attributes for each image. Following the previous works [26, 25], we set hair color and heavy makeup as the target attribute. Note that gender attribute is spuriously correlated to the two attributes and employed as the bias attribute for worst-group accuracy evaluation in our experiment. For more comprehensive results, we also consider the other 32 attributes as the target attributes.

Waterbirds [26] is a synthesized dataset with 4,795 training examples, created by combining birds photographs from the Caltech-UCSD Birds-200-2011 (CUB) dataset [32] and the background images in the Places dataset [40]. There exist two attributes in the dataset; one is the type of a bird, {waterbird, landbird}, which is the target attribute, and the other is the background place, {water, land}.

The Colored-MNIST dataset [21, 25, 1] is an extension of MNIST with the color attributes, where each digit is highly correlated to a certain color. There are 60K training examples and 10K test images, where the ratio of bias-aligned samples³³3It denotes the samples that can be correctly classified by using the bias attribute (color). is 95%. We follow the protocol employed in [25] for the experiment.

For CelebA and Waterbirds, we use a ResNet-18 as our backbone network, which is pretrained on ImageNet. We train both the base and target models using the Adam optimizer with a learning rate of $1\times 10^{-4}$, a batch size of 256, and a weight decay rate of 0.01. For the Colored MNIST dataset, we adopt a multi-layer perceptron with three hidden layers, each of which has 100 hidden units. We also employ the same Adam optimizer with a learning rate of $1\times 10^{-3}$. We train the models for 100 epochs for all experiments, and decay the learning rate using the cosine annealing [23].

For clustering, we extract features from a separately trained base network with the standard classification loss and perform $k$-means clustering with $K=8$ in all experiments. The cluster weight of the $k^{\text{th}}$ cluster, $\omega_{k}$, is updated by exponential moving average at each iteration with a momentum $m$ of 0.3. All the results reported in our paper are obtained from the average of three runs.

To evaluate the unbiased accuracy with an imbalanced evaluation set, we measure the accuracy of each group $g=({a}_{t}$, ${a}_{b})$, defined by a pair of target and bias attribute values. We finally report the average accuracy of each group and worst-group accuracy among all groups as in [25].

We present our main results on the standard benchmarks including CelebA, WaterBirds, and Colored-MNIST. In the rest of this section, our method is denoted by BPA, which stands for bias pseudo-attribute.