Accuracy and Fairness Trade-offs in Machine Learning:
A Stochastic Multi-Objective Approach

Fairness in machine learning basically requires that prediction outcomes do not disproportionally benefit people from majority and minority or historically advantageous and disadvantageous groups. In the literature of fair machine learning, several prevailing criteria for fairness include disparate impact (Barocas & Selbst, 2016) (also called demographic parity (Calders et al., 2009)), equalized odds (Hardt et al., 2016), and its special case of equal opportunity (Hardt et al., 2016), corresponding to different aspects of fairness requirements.

In this paper, we will focus on binary classification to present the formula for fairness criteria and the proposed accuracy and fairness trade-off framework, although they can all be easily generalized to other ML problems (such as regression or clustering). We point out that many real decision-making problems such as college admission, bank loan application, hiring decisions, etc. can be formulated into binary classification models.

Let $Z\in\mathbb{R}^{n},A\in\{0,1\},Y\in\{-1,+1\}$ denote feature vector, binary-valued sensitive attribute (for simplicity we focus on the case of a single binary sensitive attribute), and target label respectively. Consider a general predictor $\hat{Y}\in\{-1,+1\}$ which could be a function of both $Z$ and $A$ or only $Z$. The predictor is free of disparate impact (Barocas & Selbst, 2016) if the prediction outcome is statistically independent of the sensitive attribute, i.e., for $\hat{y}\in\{-1,+1\}$,

However, disparate impact could be unrealistic when one group is more likely to be classified as a positive class than others, an example being that women are more dominating in education and healthcare services than men (Kelly, 2020). As a result, disparate impact may never be aligned with a perfect predictor $\hat{Y}=Y$.

In terms of equalized odds (Hardt et al., 2016), the predictor is defined to be fair if it is independent of the sensitive attribute but conditioning on the true outcome $Y$, namely for $y,\hat{y}\in\{-1,+1\}$,

Under this definition, a perfectly accurate predictor can be possibly defined as a fair one, as the probabilities in (2) will always coincide when $\hat{Y}=Y$. Equal opportunity (Hardt et al., 2016), a relaxed version of equalized odds, requires that condition (2) holds for only positive outcome instances ($Y=+1$), for example, students admitted to a college and candidates hired by a company.

From the perspective of multi-objective optimization (MOO), most of the in-training methods in the literature (Barocas & Selbst, 2016; Calders et al., 2009; Kamishima et al., 2011; Woodworth et al., 2017; Zafar et al., 2017a, b) are based on the so-called a priori methodology, where the decision-making preference regarding an objective (the level of fairness) must be specified before optimizing the other (the accuracy). For instance, the constrained optimization problems proposed in (Zafar et al., 2017a, b) are to some extent nothing else than the $\epsilon$–constraint method (Haimes, 1971) in MOO. Such procedures highly rely on the decision-maker’s advanced knowledge of the magnitude of fairness, which may vary from criterion to criterion and from dataset to dataset.

In order to better frame our discussion of accuracy vs fairness, let us introduce the general form of a multi-objective optimization problem

with $m$ objectives, and where $F:\mathbb{R}^{n}\rightarrow\mathbb{R}^{m}$. Usually, there is no single point optimizing all the objectives simultaneously. The notion of dominance is used to define optimality in MOO. A point $y$ (weakly) dominates $x$ if $F(y)\leq F(x)$ holds element-wise and additionally $f_{i}(y)<f_{i}(x)$ holds for at least one objective $i=1,\ldots,m$. A point $x$ is said to be (strictly) nondominated if it is not (weakly) dominated by any other point $y$. An unambiguous way of considering the trade-offs among multiple objectives is given by the so-called Pareto front, which lies in the criteria space $\mathbb{R}^{m}$ and is defined as the set of points of the form $F(x)$ for all nondominated points $x$. The concept of dominance is carried out from the decision space to the image one, and one can also say that $F(y)$ (strictly) dominates $F(x)$ when $F(y)\leq F(x)$ holds element-wise and $f_{i}(y)<f_{i}(y)$ holds for at least one $i=1,\ldots,m$. Finally, the terminology trade-off refers to compromising and balancing among the set of equally good nondominated solutions.

In this paper, instead of looking for a single predictor that satisfies certain fairness constraints, our goal is to directly construct a complete Pareto front between prediction accuracy and fairness, and thus to identify a set of predictors associated with different levels of fairness. We propose a stochastic multi-objective optimization framework, and aim at obtaining good approximations of true Pareto fronts. We summarize below the three main advantages of the proposed framework.

• •

  By applying an algorithm for stochastic multi-objective optimization (such as the Pareto front stochastic multi-gradient (PF-SMG) algorithm developed in (Liu & Vicente, 2021)), we are able to obtain well-spread and accurate Pareto fronts in a flexible and efficient way. The approach works for a variety of scenarios, including binary and categorical multi-valued sensitive attributes. It also handles multiple objectives simultaneously, such as multiple sensitive attributes and multiple fairness measures. Compared to the constrained optimization approaches, e.g., (Zafar et al., 2017a, b), our framework is proved to be computational efficient in constructing the whole Pareto fronts.

• •

  The proposed framework is quite general in the sense that it has no restriction on the type of predictors and works for any convex or nonconvex smooth objective functions. In fact, it can not only handle the fairness criteria mentioned in Section 1.1 based on covariance approximation, but also tackle other formula proposed in the literature, e.g., mutual information (Kamishima et al., 2012) and fairness as a risk measure (Williamson & Menon, 2019).

• •

  The PF-SMG algorithm falls into a Stochastic Approximation (SA) algorithmic approach, and thus it enables us to deal with the case where the training data is arriving on a streaming mode. By using such an SA framework, there is no need to reconstruct the Pareto front from scratch each time new data arrives. Instead, a Pareto front constructed based on consecutive arriving samples will eventually converge to the one corresponding to the overall true population.

By combining existing optimization methods and fairness concepts, this paper offers solutions to open ML fairness problems (non-convex measures, streaming data, multiple attributes, and multiple fairness measures). Our work provides a systematic and thorough trade-off approach from the viewpoint of stochastic multi-objective optimization for ML fairness.

The remainder of this paper is organized as follows. Our stochastic bi-objective formulation using disparate impact is suggested in Section 2. The PF-SMG algorithm, used to solve the multi-objective problems, is briefly introduced in Section 3 (more details in Appendix B). A number of numerical results for both synthetic and real data are presented in Section 4 to support our claims. Further exploring our line of thought, we introduce another stochastic bi-objective formulation, this time for trading-off accuracy vs equal opportunity (see Section 5), also reporting numerical results. In Section 6, we show how to handle multiple sensitive attributes and multiple fairness measures. For the purpose of getting more insight on the various trade-offs, two tri-objective problems are formulated and solved. Finally, a preliminary numerical experiment described in Section 7 will illustrate the applicability of our approach to streaming data. The paper is ended with some conclusions and prospects of future work in Section 8.

To numerically illustrate our approach based on the bi-objective formulation (9), we have used both the Adult Income dataset (Kohavi, 1996), which is available in the UCI Machine Learning Repository (Dua & Graff, 2017) and synthetic data (see Appendix D for the data generation).

There are several parameters to be tuned in PF-SMG for a better performance: (1) $p_{1}$: number of times SMG is applied at each point in the current list; (2) $p_{2}$: number of SMG iterations each time SMG is called; (3) $\{\alpha_{k}\}_{1}^{T}$: step size sequence; (4) $\{b_{1,k}\}_{1}^{T},\{b_{2,k}\}_{1}^{T}$: batch size sequences used in computing stochastic gradients for the two objectives. To control the rate of generated nondominated points, we remove nondominated points from regions where such points tend to grow too densely¹¹1Our implementation code is available at https://github.com/sul217/MOO_Fairness.. In our experiments, the parameters are selected using grid search. Specifically, for each combination of parameters, we run the PF-SMG using training data to get a set of nondominated solutions. The best parameter combination is chosen as the one with the most accurate predictor on the Pareto fronts which are evaluated using a set of validation samples.

Our approach is compared to the $\epsilon$-constrained optimization model proposed in Zafar et al. (2017b, Equation (4)). From now on, we note their $\epsilon$-constrained method as EPS-fair. It basically minimizes prediction loss subject to disparate impact being bounded above by a constant $\epsilon$, i.e.,

Since the bi-objective problem (9) under investigation is convex, EPS-fair is able to compute a set of nondominated points by varying the value of $\epsilon$. The implementation details of EPS-fair method can be found in (Zafar et al., 2017b). First, by solely minimizing prediction loss, a reasonable upper bound is obtained for disparate impact. Then, to obtain the Pareto front, a sequence of thresholds $\epsilon$ is evenly chosen from $0$ to such an upper bound, leading to a set of convex constrained optimization problems. The Sequential Least SQuares Programming (SLSQP) solver (Kraft, 1988) based on Quasi-Newton methods is then used for solving those problems. We found that 70-80% of the final points produced by this process were actually dominated ones, and we removed them for the purpose of analyzing results.

The cleaned up version of Adult Income dataset contains 45,222 samples. Each instance is characterized by 12 nonsensitive attributes (including age, education, marital status, and occupation), a binary sensitive attribute (gender), and a multi-valued sensitive attribute (race). The prediction target is to determine whether a person makes over 50K per year. Tables 2 and 2 in Appendix D show the detailed demographic composition of the dataset with respect to gender and race.

In the following experiments, we have randomly sampled 60% of the whole dataset for training, using 10% for validating and the remaining 30% instances as the testing dataset. The PF-SMG algorithm is applied using the training dataset, but all the Pareto fronts and the corresponding trade-off information will be presented using the testing dataset.

Performance in terms of trade-offs. Considering gender as the sensitive attribute, the obtained Pareto front by PF-SMG is plotted in Figure 1 (a), reconfirming the conflicting nature of the two objectives. Given a nondominated solution $x=(c,b)$ from Figure 1 (a), the probability of getting positive prediction for each sensitive group is approximated by the percentage of positive outcomes for the data samples, i.e., $\mathbb{P}\{\hat{Y}(Z;x)=1|A=a\}\simeq\frac{N(\hat{Y}=1,A=a)}{N(A=a)}$, where $N(\hat{Y}=1,A=a)$ denotes the number of instances predicted as positive in group $a$ and $N(A=a)$ is the number of instances in group $a$. For conciseness, we will only compute the proportion of positive outcomes for analysis.

From Figure 1 (a), predictors of high accuracy are those of high disparate impact. It is then observed from Figure 1 (b) that as $f_{2}^{\text{DI}}$ increases, the proportion of high income adults in females decreases, which means that the predictors of high accuracy are actually unfair for females. Figure 1 (c) we can conclude that the value of $f_{2}^{\text{DI}}$ has positive correlation with CV score for this dataset. Figure 1 (d) implies that almost zero disparate impact can be achieved by reducing 1.5% of accuracy (the range of the x-axis is nearly 1.5%). To eliminate the impact of the fact that female is a minority in the dataset, we ran the algorithms for several sets of training samples with $50\%$ females and $50\%$ males. It turns out that the conflict is not alleviated at all. For comparison, the trade-off for the same training-testing split obtained by EPS-fair is also shown in Figures 1 (a), (c), and (d). More comparisons using different training-testing splits are given in Appendix E.1. It is clear that PF-SMG is more robust in capturing more complete and well-spread Pareto fronts.

Dealing with multi-valued sensitive attribute race is more complicated. In general, if a multi-valued sensitive attribute has $K$ categorical values, we convert it to $K$ binary attributes denoted by $A^{1},\ldots,A^{K}\in\{0,1\}$. Note that the binary attribute $A^{i}$ indicates whether the original sensitive attribute has $i$-th categorical value or not. The second objective is then modified as follows

which is still a convex function. We have observed that the non-smoothness introduced by the max operator in (10) led to more discontinuity in the true trade-off curves, and besides stochastic gradient type methods are designed for smooth objective functions. We have thus approximated the max operator in (10) using $S_{\beta}(\max(x^{1},\ldots,x^{\ell}))=\sum_{i=1}^{\ell}x^{i}e^{\beta x^{i}}/\sum_{i=1}^{\ell}e^{\beta x^{i}}$. In our experiments, we set $\beta=8$. Figure 2 (a) plots the obtained Pareto front of the bi-objective problem of $\min(f_{1}(c,b),f_{3}^{\text{DI}}(c,b))$. Figure 2 (b) implies that solely optimizing over prediction accuracy might result in unfair predictors for American-Indian, Black, and Other. Even though the individual percentage is not monotonically decreasing, the maximum gap of percentage is increasing while $f_{3}^{\text{DI}}$ increases (Asian group excluded). Regardless of the noise, it is observed that the value of $f_{3}^{\text{DI}}$ is increasing with CV score (Figure 2 (c)) and that the prediction accuracy and CV score have positive correlation (Figure 2 (d)). Note that CV score in this case was computed as the absolute difference between maximum and minimum proportions of positive outcomes among $K$ groups. The dominance effect of the Pareto fronts produced by PF-SMG over the ones produced by EPS-fair, seen in Figures 1 (a), (c), and (d), is now mixed in the ones of Figures 2 (a), (c), and (d). However, the spread and overall coverage of the fronts is better for PF-SMG in all the three plots. More results can be found in Appendix E.2.

Performance in terms of efficiency and robustness. We systematically evaluated the performance of the two approaches by comparing CPU time per nondominated solution, number of gradient evaluations per nondominated solution, and the quality of Pareto fronts. Such a quality is measured by a formula called purity (which tries to evaluate how the fronts under analysis dominate each other) and two formulas for the spread of the fronts ($\Gamma$ and $\Delta$, measuring how well the nondominated points on a Pareto front are distributed). Higher purity corresponds to higher accuracy, while smaller $\Gamma$ and $\Delta$ indicate better spread. In addition, we also used hypervolume (Zitzler & Thiele, 1999), a performance measure combining purity and spread. The detailed formulas of the four measures are given in Appendix C.

We formed a set of 40 datasets with different random seeds: $20$ from synthetic data and $20$ from Adult income data ($10$ binary and $10$ multi-valued problems). For EPS-fair, 1,500 and 100 different values of thresholds $\epsilon$ are evenly selected to form Pareto fronts for synthetic and real datasets respectively. To guarantee a fair comparison in terms of purity, spread, and hypervolume, the Pareto front generated by PF-SMG is down-sampled to matches the size of the corresponding Pareto front obtained by EPS-fair. The five performance profiles (see (Dolan & Moré, 2002)) are shown in Figure 3. The purity (see Figure 3 (a)) of the Pareto fronts produced by the EPS-fair method is slightly better than the one of those determined by PF-SMG. However, notice that PF-SMG produced better spread fronts than EPS-fair without compromising accuracy too much (see Figures 3 (b)–(c)). In fact, one can observe from Appendix E that the resulting Pareto fronts from PF-SMG are quite close to that from EPS-fair in the cases where the latter ones dominate the former ones. Further, Figure 3 (d) shows that PF-SMG wins against EPS-fair in terms of the combination of individual solution quality and objective space coverage. In addition, PF-SMG outperforms EPS-fair in terms of computational cost quantified by CPU time and gradient evaluations (see Figures 3 (e)–(f)).

Let us see first how we can handle more than one sensitive attribute. One can consider a binary sensitive attribute (e.g. gender) and a multi-valued sensitive attribute (e.g. race), and formulate the following tri-objective problem

In our experiments, we use the Adult Income dataset and the splitting of training and testing samples of Subsection 4.2. A 3D Pareto front is plotted in Figure 5 (a) resulting from the application of PF-SMG to (13), with gender ($f_{2}^{\text{DI}}$) and race ($f_{3}^{\text{DI}}$) as the two sensitive attributes.

Figure 5 (b) depicts all the nondominated points projected onto the $f_{2}$–$f_{3}$ objective space, where the points are colored according to the range of prediction loss. It is observed that there is no conflict between $f_{2}^{\text{DI}}$ and $f_{3}^{\text{DI}}$. Although it could happen for other datasets, eliminating disparate impact with respect to gender does not hinder that with respect to race for this dataset. Intuitively, one could come up with a predictor where the proportions of positive predictions for female and male are equalized and the proportions of positive predictions for different races are equalized within the female and male groups separately, which would lead to zero disparate impact in terms of gender and race simultaneously.

Now we see how to handle more than one fairness measure. As an example, we consider handling two fairness measures (disparate impact and equal opportunity) in the case of a binary sensitive attribute, and formulate the following tri-objective problem

In our experiments, we use the whole ProPublica COMPAS two-years-violent dataset (see Section 5) for both training and testing. Figure 5 (c) shows an approximated 3D Pareto front (resulting from the application of PF-SMG to (14)). By projecting all the obtained nondominated points onto the 2D $f_{2}$–$f_{4}$ objective space, we have subplot (d), where the different colors indicate the different levels of prediction loss. From Figure 5 (d), one can easily find that an unique minimizer (in the gray area with lower prediction accuracy) exists for both $f_{2}^{\text{DI}}$ and $f_{4}^{\text{FNR}}$, and thus conclude that there is indeed no conflict between disparate impact and equal opportunity. In fact, by definition, the CV score (11) generalized to equal opportunity is a component of the CV score (4) measuring disparate impact. Therefore, in the darkgreen area where the accuracy is high enough, the values of the two fairness measures are aligned and increasing as the prediction accuracy increases. Interestingly, we have discovered a little Pareto front between $f_{2}$ and $f_{4}$ when the accuracy is limited in a certain medium level in between.

The proposed multi-objective approach works well in handling more than one sensitive attribute or multiple fairness measures. We point out that looking at Pareto fronts for three objectives helps us identifying the existence of conflicts among any subset of two objectives (compared to looking at Pareto fronts obtained just by solving the corresponding bi-objective problems). In the above experiments, by including $f_{1}$, we were able to obtain additional helpful information in terms of decision-making reasoning.