Fair Adaptive Experiments¹¹1This manuscript is accepted for publication at NeurIPS 2023.

Randomized experiments are considered gold standards for evaluating the effectiveness of public policies, medical treatments, or advertising strategies [33, 35, 36]. They involve randomly assigning participants to different treatment groups, allowing for rigorous causal conclusions and robust evidence for decision-making and policy implementation. However, classical randomized experiments, which maintain fixed treatment assignment probabilities, often do not optimize data utilization. This limitation is problematic due to the high costs associated with conducting such experiments. Consequently, maximizing information gain is crucial, but classical randomized experiments do not prioritize this objective [26, 47].

Compared to classical randomized experiments, adaptive experiments provide enhanced information gain, leading to improved statistical efficiency. As a result, adaptive experiments have gained popularity in diverse domains such as field experiments, online A/B testing, and clinical trials [27, 57, 64, 65]. The information gain of adaptive experiments stems from their ability to iteratively adjust treatment allocations based on refined knowledge obtained from accumulated data during the experiment. This iterative process often favors the treatment arm that offers more informative or beneficial outcomes, maximizing the information gained from each participant and optimizing the overall statistical efficiency of the experiment [46]. Moreover, adaptive experiments, thanks to their adept utilization of data resources, often exhibit greater statistical testing power when practitioners employ the collected data to assess the null hypothesis of zero treatment effect upon experiment completion [15].

Despite their appealing benefits in improving data use efficiency and boosting statistical power, adaptive experiments potentially bring fairness concerns in applications. This issue is neither sufficiently explored nor fully addressed in the existing literature. Below, we shall concretely discuss the fairness concerns under a scenario where the study population can be divided into distinct groups based on demographic information or biomarkers — a scenario frequently encountered in field experiments or clinical trials. The first fairness concern in adaptive experiments arises when there are significant disparities in treatment allocations among different participant groups [15]. This is because when the treatment is potentially beneficial, it is crucial to ensure a fair chance for each group to receive beneficial treatment. Similarly, it is important to avoid disproportionately burdening any specific group with unfavorable treatment. However, conventional adaptive experiments prioritizing efficiency gains may inadvertently result in unfair treatment allocations. For example, if the outcome of a particular group of participants exhibits a higher variance in response to the treatment, more participants in the group will be allocated to the treatment arm. Consequently, this group would have a significantly higher treatment assignment probability than the others, regardless of the sign and magnitude of the treatment effect. This may lead to an unfair allocation of treatments among participants. Completely randomized experiments with fixed one-half treatment assignments would avoid this challenge, but they suffer from information loss. The second fairness concern arises when the adaptive treatment allocation does not adequately account for the overall welfare of experimental participants. This is crucial as a fair experiment is expected to not only assign a large proportion of participants to a beneficial treatment arm but also assign a small proportion of participants to a harmful treatment to avoid adverse effects.

There are evident challenges in addressing fairness concerns while optimizing information gain in adaptive experiments due to the potential trade off among fairness concerns, welfare improvement, and information gain. For example, if most of the participants are assigned to the beneficial treatment to maximize welfare, then there would be insufficient sample size in the control arm, resulting in imprecisely estimated treatment effect and hence reduced statistical efficiency for conducting inference. To overcome these challenges, we propose a fair adaptive experimental design strategy that balances competing objectives: improving fairness, enhancing overall welfare, and gaining efficiency. By rigorously demonstrating the effectiveness of our approach and providing statistical guarantees, we offer a practical solution that is both grounded in theory and reconciles fairness concerns with the requirement for robust information gain in adaptive experiments. Our contributions can be summarized as follows:

First, in comparison to existing adaptive experiments, our proposed strategy integrates fairness and welfare considerations while optimizing information gain. As a result, the treatment allocation probability generated by our method avoids extreme values and exhibits minimal necessary variations across different groups. These desirable characteristics are supported by our simulation studies and empirical illustration using synthetic data. It is important to note that due to the additional constraints of welfare and fairness, the optimal treatment allocation probability does not have a closed-form expression, which brings additional technical challenges to studying the theoretical properties of our design. Despite this challenge, we demonstrate that the constructed treatment allocation rule for each group of our design converges to its oracle counterpart (Theorem 2). This implies that our proposed designs in Section 2.3, despite not relying on prior knowledge about the underlying data distribution before the start of the experiment, can allocate treatments in a similar manner to a scenario where perfect knowledge of the data distribution is available.

Second, we do not impose any specific parametric modeling assumptions on the outcomes beyond mild moment conditions. We instead estimate the mean and variance of potential outcomes at the group level, which are further incorporated into our algorithm. The nonparametric nature of our procedure delivers an efficient and accurate estimation of the average treatment effect. As an important theoretical contribution, we prove that those group-level estimates are asymptotically consistent (Theorem 1).

Third, our theoretical framework addresses the challenges and complexities associated with adaptive experiment design, where data are sequentially accumulated, and treatment allocation decisions are adaptively revised, resulting in non-independently and non-identically distributed data. By leveraging the martingale methods, we demonstrate that the estimate of the average treatment effect is consistent and asymptotically normally distributed (Theorems 1 and 3). An important methodological and practical innovation of our framework is that it does not require the number of participants enrolled in the first stage to be proportional to the overall sample size. This flexibility allows researchers to allocate more participants in later stages of the experiment, enabling a truly adaptive approach to experiment design and implementation. This innovation has significant implications for the methodology and practical application of adaptive experiments.

Our proposed fair adaptive experiment strategy has a natural connection with the response adaptive randomization (RAR) design literature. The early work develops the randomized play-the-winner rule in clinical trial settings based on urn models [47, 49, 61]. Theoretical properties of urn models are investigated in [7] and [30]. Another conventional response adaptive design is the doubly adaptive biased coin (DBCD) design [16, 27, 28, 54]. However, to our best knowledge, many existing works on response adaptive designs do not take fair treatment allocations into account [25, 48]. An insightful work in [34] proposes an efficient RAR design to minimize the variance of the average treatment effects and discusses some directions for fair experimental design. Compared with [34], our method does not require estimating outcome models, which can be challenging in the presence of correlated data in RAR designs. In addition, our design centers around “group” fairness, aiming to enhance participants’ well-being while avoiding extra fairness complications among distinct individuals. Furthermore, RAR designs that further incorporate covariate information are known as covariate-adjusted response adaptive (CARA) designs [8, 10, 38, 50, 58, 72, 73]. Some early work proposes to balance covariates based on the biased coin design [45, 70]. Later work considers CARA designs that account for both efficiency and ethics [29] and extends the CARA design framework to incorporate nonparametric estimates of the conditional response function [2]. It is worth mentioning that another strand of literature focuses on ethical designs using Bayesian frameworks. Some recent work proposes to use the Gittins index to improve participants’ welfare [56, 59]. A later work develops a Bayesian ethical design to further improve statistical power [63]. Some other ethical designs are discussed in [19, 53, 69].

Our manuscript also relates to the literature on semiparametric efficiency and treatment effect estimation [21, 43, 55]. Our algorithm adaptively allocates participants to treatment and control arms, with the aim of not only minimizing the variance of the estimated average treatment effect but also incorporating constraints on fairness and welfare. There is also a large literature on efficient estimation of treatment effects and, more broadly, on estimation and statistical inference in semiparametric models. See, for example, [9, 11, 12, 13, 17, 23, 39, 40, 60, 62] and references therein. Our algorithm takes the group structure as given. Another strand of literature studies stratified randomization. Some recent contributions in this area include [6, 52].

Lastly, our proposed design is connected to the multi-armed bandit (MAB) literature. [51] studies the trade-off between regret and statistical estimation efficiency by formulating a minimax multi-objective optimization problem and proposing an effective Pareto optimal MAB experiment. They provide insightful theoretical results on the sufficient and necessary conditions for the Pareto optimal solutions. Our procedure attains the minimax lower bound for fair experiment design problems. Our work has a different focus on uncovering the underlying causal effect by providing an adaptive procedure for efficient treatment effect estimation while incorporating fairness and welfare considerations. Furthermore, in the theoretical investigations, we focus on the asymptotic normality of the proposed estimator and its variance estimator, which enables valid statistical inference. Our work broadly connects with fair contextual bandit literature [14, 18, 20, 32, 41, 44]. [66] and [67] propose algorithms under subpopulation fairness and equity requirements for the tasks of best arm identification and ranking and selection. The work in [31] characterizes fairness under the contextual bandit setting by bridging the fair contextual bandit problems with “Knows What It Knowns” learning. While [31] defines fairness metric on the individual level, we focus on group-level fairness and further incorporate a welfare constraint.

In this section, we formalize our adaptive experiment framework and introduce necessary notations. In adaptive experiments, participants are sequentially enrolled across $T$ stages. We denote the total number of enrolled participants as $N=\sum_{t=1}^{T}n_{t}$, where $n_{t}$ is the number of participants in Stage $t$, $t=1,\ldots,T$. In line with the existing literature [25, 26, 29], we assume $T\to\infty$, and $n_{t}$ is small relative to the overall sample size $N$, meaning that we have many opportunities to revise the treatment allocation rule during the experiment (see Assumption 3 below). At Stage $t$, we denote participant $i$’s treatment assignment status as $D_{it}\in\{0,1\}$, $i=1,\ldots,n_{t}$, with $D_{it}=1$ being the treatment arm and $D_{it}=0$ being the control arm. Denote participant $i$’s covariate information as $X_{it}\in\mathbb{R}^{p}$ and the observed outcome as $Y_{it}\in\mathbb{R}$.

Next, we quantify causal effects under the Neyman-Rubin potential outcomes framework. Define $Y_{it}(d)$ as the potential outcome we would have observed if participant $i$ receives treatment $d$ at Stage $t$, $d\in\{0,1\}$. The observed outcome can be written as

In accordance with classical adaptive experiments literature, we assume that the outcomes are observed without delay, and their underlying distributions do not shift over time [26]. The average treatment effect (ATE) is the mean difference between the two potential outcomes:

In our proposed fair adaptive experiment strategy, to protect the participant’s welfare (more discussions in Section 2.2), we also consider the group-level treatment effects. We assume the study population can be partitioned based on demographics or biomarkers, which is frequently seen in clinical settings or social science studies [4, 37, 68]. More concretely, by dividing the sample space $\mathcal{X}$ of the covariate $X_{it}$ into $m$ non-overlapping regions, denoted as $\{\mathcal{S}_{j}\}_{j=1}^{m}$, we define the treatment effect in each group as

We further denote the total number of participants enrolled in the group $j$ as $N_{j}=\sum_{t=1}^{T}n_{tj}$.

In adaptive experiments, as we aim to adaptively revise the treatment assignment probabilities based on the evidence accrued during the experiment to meet our fairness and efficiency goals, we define treatment assignment probability (or propensity scores) for participants in groups $j$ at stage $T$ as

The goal of our experiment is to dynamically revise $e_{tj}$ for efficiency improvement, fairness guarantee, and welfare enhancement.

Classical adaptive experiments, aimed at reducing variance (or, equivalently, efficiency improvement), often assign treatment using Neyman allocation for participants in each group, that is,

where $\sigma_{j}^{2}(d)=\mathbb{V}[Y_{it}(d)|X_{it}\in\mathcal{S}_{j}],\ d\in\{0,1\}$, denotes the variance of the potential outcome under treatment arm $d$ in group $j$. Although Neyman allocation improves the estimation efficiency of the ATE, it brings two critical fairness concerns. First, different groups of participants may have substantially different probabilities of receiving treatments. The form of Eq (5) implies that the treatment assignment probabilities solely rely on group-level variances under different arms. More specifically, the group of participants with a larger variance under the treatment arm will have a higher probability of being treated, which may lead to disproportionate treatment allocations across different groups. Second, some participants’ welfare could be harmed under the adaptive experiment strategy in Eq (5). To see this, assume a group of participants exhibits a large variance yet rather negative responses under the treatment arm. However, more participants in this group will be assigned to the treatment arm to improve the estimation efficiency of ATE despite the impairment of those participants’ welfare.

To address fairness concerns and facilitate the introduction of our experimental goals, we begin with an infeasible “oracle” setting, where we possess knowledge of the true underlying data distribution before the experiment begins. Since adaptive experiments naturally allow for sequential learning of unknown parameters and adjustment of treatment allocations during the experiment, we will present our adaptive experimental design strategy in the following section (Section 2.3), which attains the same theoretical guarantee for estimating the ATE as in the oracle setting (see Theorems 2 and 3 for justification).

In the oracle setting, given we have perfect knowledge of the underlying data distribution (thus $\tau_{j}$ and $\sigma_{j}^{2}(d)$ are known to us), our goal is to find optimal treatment allocations $\bm{e}^{*}=(e^{*}_{1},\ldots,e_{m}^{*})^{\intercal}$ that solve the following optimization problem:

where $c_{1}\in(0,1)$ and $c_{2}\in(0,1/2)$. Here, the objective function captures the goal of improving information gain from study participants, which is formalized as minimizing the asymptotic variance of the inverse probability weighting estimator of the ATE (c.f. Theorem 3). The “feasibility” constraint restricts that the treatment assignment probability in each group is bounded away from $0$ and $1$ by a positive constant $c_{1}$.

To ensure fair treatment assignment and mitigate significant disparities in treatment allocations among participant groups, we introduce the “envy-freeness” constraint. This constraint limits the disparity in treatment assignment probabilities across different groups in an acceptable pre-specified range. The concept of “envy-freeness” originates from game theory literature and ensures that agents are content with their allocated resources without envying their peers [3, 5, 24, 42]. By incorporating this envy-freeness constraint, we address the first fairness concern and promote equitable treatment allocation.

To enhance the overall welfare of experiment participants, we introduce the “welfare” constraint. This constraint ensures that a group of participants is more likely to receive the treatment if their treatment effects are positive and less likely to receive the treatment otherwise. Specifically, when the group-level treatment effect $\tau_{j}\geq 0$, indicating that group $j$ benefits from the treatment, we want the treatment assignment probability $e_{j}$ to be larger than $\frac{1}{2}$. The welfare constraint achieves this by ensuring that the sign of $\log(\frac{e_{j}}{1-e_{j}})$ aligns with the sign of $\tau_{j}$. When incorporating the welfare constraint, we effectively address the second fairness concern by providing more treatment to beneficial groups.

In this section, we present our fair adaptive experimental design strategy for realistic scenarios where we lack prior knowledge about the underlying data distribution, and our approach achieves the same desirable properties as in the oracle setting (refer to Section 3 for justification).

We present our proposed fair adaptive experiment strategy in Algorithm 1.

Concretely, in Stage $1$ (line 2–3), because we have no prior knowledge about the unknown parameters, we obtain initial estimates of the group-level treatment effect $\hat{\tau}_{1j}$ and the associated variances $\hat{\sigma}_{1j}^{2}(d)$. Then, in Stage $t$ (line 6–8), our design solves the following sample analog of Problem $\bm{A}$ at each experimental stage:

Here, we define

One important feature of Problem B is that we introduce a relaxation of the welfare constraint through $\delta(N_{t-1})$. From a theoretical perspective, the function $\delta(\cdot)$ should be strictly positive, and satisfies $\lim_{x\to\infty}\delta(x)=0$ and $\lim_{x\to\infty}\sqrt{x}\delta(x)=\infty$. For implementation, we recommend using $\delta(N_{t-1})=\sqrt{\log(N_{t-1})/N_{t-1}}$. It is also possible to incorporate the standard error of the estimated subgroup treatment effects into the welfare constraint, which motives the more sophisticated version:

where $\hat{v}_{t-1,j}$ is the adaptively estimated standard deviation defined in the Supplementary Materials. Scaling the welfare constraint by $\hat{v}_{t-1,j}$, a measure of randomness in $\hat{\tau}_{t-1,j}$, delivers a more clear interpretation: the above now corresponds to a t-test for the subgroup treatment effect $\tau_{j}$ with a diverging threshold, and the specific choice stems from Schwarz’s minimum BIC rule.

After the final Stage $T$, we have the group-level treatment effect estimates $\hat{\tau}_{j}:=\hat{\tau}_{Tj}$, the variance estimates $\hat{\sigma}^{2}_{j}(d):=\hat{\sigma}^{2}_{Tj}(d)$, and the group proportions $\hat{p}_{j}:=\hat{p}_{Tj}$ (that is, we omit the time index $T$ for estimates obtained after the completion of the experiment). Together with valid standard errors, one can conduct statistical inference at a some pre-specified level $\alpha$. To be precise, the estimated ATE is $\hat{\tau}=\sum_{j=1}^{m}\hat{p}_{j}\hat{\tau}_{j}$, and we define the following

Lastly, we can construct the two-sided confidence intervals for $\hat{\tau}_{j}$ and $\hat{\tau}$ as