Combining Biomarkers by Maximizing the True Positive Rate for a Fixed False Positive Rate

The ROC curve provides a means to evaluate the ability of a biomarker or, equivalently, biomarker combination $Z$ to identify individuals who have or will experience a binary outcome $D$. For example, in a diagnostic setting, $D$ denotes the presence or absence of disease and $Z$ may be used to identify individuals with the disease. The ROC curve provides information about how well the biomarker discriminates between individuals who have or will experience the outcome, that is, the cases, and individuals who do not have or will not experience the outcome, that is, the controls (Pepe, 2003). Mathematically, if larger values of $Z$ are more indicative of having or experiencing the outcome, for each threshold $\delta$ we can define the TPR as $P(Z>\delta\mid D=1)$ and the FPR as $P(Z>\delta\mid D=0)$ (Pepe, 2003). For a given $\delta$, the TPR is also referred to as the sensitivity, and one minus the specificity equals the FPR (Pepe, 2003). The ROC curve is a plot of the TPR versus the FPR as $\delta$ ranges over all possible values; as such, it is non-decreasing and takes values in the unit square (Pepe, 2003). A perfect biomarker has an ROC curve that reaches the upper left corner of the unit square, and a useless biomarker has an ROC curve on the 45-degree line (Pepe, 2003).

The most common summary of the ROC curve is the AUC, the area under the ROC curve. The AUC ranges between 0.5 for a useless biomarker and 1 for a perfect biomarker (Pepe, 2003). The AUC has a probabilistic interpretation: it is the probability that the biomarker value for a randomly chosen case is larger than that for a randomly chosen control, assuming that higher biomarker values are more indicative of having or experiencing the outcome (Pepe, 2003). Both the ROC curve and the AUC are invariant to monotone increasing transformations of the biomarker $Z$ (Pepe, 2003).

The AUC summarizes the entire ROC curve, but in many situations it is more appropriate to only consider certain FPR values. For example, screening tests require a very low FPR, while diagnostic tests for fatal diseases may allow for a slightly higher FPR if the corresponding TPR is very high (Hsu and Hsueh, 2013). Such considerations led to the development of the partial AUC, the area under the ROC curve over some range $(t_{0},t_{1})$ of FPR values (Pepe, 2003). Rather than considering a range of FPR values, there may be interest in fixing the FPR at a single value, determining the corresponding threshold $\delta$, and evaluating the TPR for that threshold. As opposed to the AUC and the partial AUC, this method returns a single classifier, or decision rule, which may appeal to researchers seeking a tool for clinical decision-making.

Many methods to combine biomarkers have been proposed, and they can be divided into two categories. The first includes indirect methods that seek to optimize a measure other than the performance measure of interest, while the second category includes direct methods that optimize the target performance measure. We focus on the latter.

Targeting the entire ROC curve (that is, constructing a combination that produces an ROC curve that dominates the ROC curve for all other linear combinations at all points) is very challenging and is only possible under special circumstances. Su and Liu (1993) demonstrated that, when the vector X of biomarkers has a multivariate normal distribution conditional on $D$ with proportional covariance matrices, it is possible to identify the linear combination that maximizes the TPR uniformly over the entire range of FPRs; this linear combination is Fisher’s linear discriminant function.

McIntosh and Pepe (2002) used the Neyman-Pearson lemma to demonstrate optimality (in terms of the ROC curve) of the likelihood ratio function and, consequently, of the risk score $P(D=1|\textbf{X}=\textbf{x})$ and monotone transformations of $P(D=1|\textbf{X}=\textbf{x})$. Thus, if the biomarkers are conditionally multivariate normal and the $D$-specific covariance matrices are equal, the optimal linear combination dominates not just every other linear combination, but also every nonlinear combination. This results from the fact that in this case, the linear logistic model $\mbox{logit}\{P(D=1|\textbf{X}=\textbf{x})\}={\boldsymbol{\theta}}^{\top}\textbf{x}$ holds for some $p$-dimensional ${\boldsymbol{\theta}}$, where $p$ is the dimension of X. If the covariance matrices are proportional but not equal, the likelihood ratio is a nonlinear function of the biomarkers, as shown in the Appendix A for $p=2$, and the optimal biomarker combination with respect to the ROC curve is nonlinear.

In general, there is no linear combination that dominates all others in terms of the TPR over the entire range of FPR values (Su and Liu, 1993; Anderson and Bahadur, 1962). Thus, methods to optimize the AUC have been proposed. When the biomarkers are conditionally multivariate normal with nonproportional covariance matrices, Su and Liu (1993) gave an explicit form for the best linear combination with respect to the AUC. Others have targeted the AUC without any assumption on the distribution of the biomarkers; many of these methods rely on smooth approximations to the empirical AUC, which involves indicator functions (Ma and Huang, 2007; Fong, Yin, and Huang, 2016; Lin et al., 2011).

Acknowledging that often only a range of FPR values is of interest clinically, methods have been proposed to target the partial AUC for some FPR range $(t_{0},t_{1})$. Some methods make parametric assumptions about the joint distribution of the biomarkers (Yu and Park, 2015; Hsu and Hsueh, 2013; Yan et al., 2018) while others do not (Wang and Chang, 2011; Komori and Eguchi, 2010; Yan et al., 2018). The latter group of methods generally uses a smooth approximation to the partial AUC, similar to some of the methods that aim to maximize the AUC (Wang and Chang, 2011; Komori and Eguchi, 2010; Yan et al., 2018). One challenge faced in partial AUC maximization is that for narrow intervals, that is, when $t_{0}$ is close to $t_{1}$, the partial AUC is often very close to 0, which can make optimization difficult (Hsu and Hsueh, 2013).

Some work in constructing biomarker combinations by maximizing the TPR has been done for conditionally multivariate normal biomarkers. In this setting, procedures for constructing a linear combination that maximizes the TPR for a fixed FPR (Anderson and Bahadur, 1962; Gao et al., 2008) as well as methods for constructing a linear combination by maximizing the TPR for a range of FPR values (Liu, Schisterman, and Zhu, 2005) have been proposed. Importantly, in the method proposed by Liu, Schisterman, and Zhu (2005), the range of FPR values over which the fitted combination is optimal may depend on the combination itself; that is, the range of FPR values may be determined by the combination and so may not be fixed in advance. Thus, this method does not optimize the TPR for a prespecified FPR. Baker (2000) proposed a flexible nonparametric method for combining biomarkers by optimizing the ROC curve over a narrow target region of FPR values. However, this method is not well-suited to situations in which more than a few biomarkers are to be combined.

An important benefit of constructing linear biomarker combinations by targeting the performance measure of interest is that the performance of the combination will be at least as good as the performance of the individual biomarkers (Pepe, Cai, and Longton, 2006). Indeed, several authors have recommended matching the objective function to the performance measure, i.e., constructing biomarker combinations by optimizing the relevant measure of performance (Hwang et al., 2013; Liu, Schisterman, and Zhu, 2005; Wang and Chang, 2011; Ricamato and Tortorella, 2011). To that end, we propose a distribution-free method to construct biomarker combinations by maximizing the TPR for a given FPR.

Figure 1 illustrates the importance of targeting the measure of interest in constructing biomarker combinations. In this example, combinations of three biomarkers are constructed by (i) maximizing the logistic likelihood, (ii) maximizing the AUC via the optAUC package in R (i.e., the method of Huang, Qin, and Fang (2011)), and (iii) maximizing the TPR for an FPR of 20% using the proposed method. The ROC curves for the three combinations differ markedly near the prespecified FPR of 20%. In particular, the TPRs at an FPR of 20% for the three combinations are 18.0%, 24.0%, and 34.0% for maximum likelihood, AUC optimization, and maximization of the TPR for a given FPR, respectively. This example highlights the utility of methods that target the TPR for a specific FPR as opposed to methods that target other measures.

Cases are denoted by the subscript $1$ and controls are denoted by the subscript $0$. Let $\textbf{X}_{1i}$ denote the vector of biomarkers for the $i^{th}$ case, and let $\textbf{X}_{0j}$ denote the vector of biomarkers for the $j^{th}$ control.

We propose constructing a linear biomarker combination of the form ${\boldsymbol{\theta}}^{\top}\textbf{X}$ for a $p$-dimensional X by maximizing the TPR when the FPR is below some prespecified, clinically acceptable value $t$. We define the true and false positive rates for a given X as a function of ${\boldsymbol{\theta}}$ and $\delta$:

Since the true and false positive rates for a given combination ${\boldsymbol{\theta}}$ and threshold $\delta$ are invariant to scaling of the parameters $({\boldsymbol{\theta}},\delta)$, we must restrict $({\boldsymbol{\theta}},\delta)$ to ensure identifiability. Specifically, we constrain $||\boldsymbol{\theta}||=1$ as in Fong, Yin, and Huang (2016). For any fixed $t\in(0,1)$, we can consider

where $\Omega_{t}=\{{\boldsymbol{\theta}}\in\mathbb{R}^{p},\delta\in\mathbb{R}:||{\boldsymbol{\theta}}||=1,\mbox{FPR}({\boldsymbol{\theta}},\delta)\leq t\}.$ This provides the optimal combination $\boldsymbol{\theta}_{t}$ and threshold $\delta_{t}$. We define $(\boldsymbol{\theta}_{t},\delta_{t})$ to be an element of $\operatorname*{arg\,max}_{({\boldsymbol{\theta}},\delta)\in\Omega_{t}}\mbox{TPR}({\boldsymbol{\theta}},\delta)$, where $\operatorname*{arg\,max}_{({\boldsymbol{\theta}},\delta)\in\Omega_{t}}\mbox{TPR}({\boldsymbol{\theta}},\delta)$ may be a set.

Of course, in practice, the true and false positive rates are unknown, so $\boldsymbol{\theta}_{t}$ and $\delta_{t}$ cannot be computed. We can replace these unknowns by their empirical estimates,

where $n_{1}$ is the number of cases and $n_{0}$ is the number of controls, giving the total sample size $n=n_{1}+n_{0}$. We can then define

where $\hat{\Omega}_{t,n_{0}}=\{{\boldsymbol{\theta}}\in\mathbb{R}^{p},\delta\in\mathbb{R}:||{\boldsymbol{\theta}}||=1,\hat{\mbox{FPR}}_{n_{0}}({\boldsymbol{\theta}},\delta)\leq t\}.$ It is possible to conduct a grid search over $({\boldsymbol{\theta}},\delta)$ to perform this constrained optimization, though this becomes computationally demanding when combining more than two or three biomarkers.

Since the objective function involves indicator functions, it is not a smooth function of the parameters $({\boldsymbol{\theta}},\delta)$ and thus not amenable to derivative-based methods. However, smooth approximations to indicator functions have been used for AUC maximization (Ma and Huang, 2007; Fong, Yin, and Huang, 2016; Lin et al., 2011). One such smooth approximation is $1(w>0)\approx\Phi(w/h)$, where $\Phi$ is the standard normal distribution function, and $h$ is a tuning parameter representing the trade-off between approximation accuracy and estimation feasibility such that $h$ tends to zero as the sample size grows (Lin et al., 2011). We can use this smooth approximation to implement the method described above, writing the smooth approximations to the empirical true and false positive rates as

Thus, we propose to compute

where $\tilde{\Omega}_{t,n_{0}}=\{{\boldsymbol{\theta}}\in\mathbb{R}^{p},\delta\in\mathbb{R}:||{\boldsymbol{\theta}}||=1,\tilde{\mbox{FPR}}_{n_{0}}({\boldsymbol{\theta}},\delta)\leq t\}.$ Since both $\tilde{\mbox{TPR}}_{n_{1}}$ and $\tilde{\mbox{FPR}}_{n_{0}}$ are smooth functions, we can use gradient-based methods that incorporate the necessary constraints, e.g., Lagrange multipliers. In particular, $(\tilde{\boldsymbol{\theta}}_{t},\tilde{\delta}_{t})$ can be obtained using existing software for constrained optimization of smooth functions, such as the Rsolnp package in R. An R package including code for our method based on Rsolnp, maxTPR, is available on CRAN. Other details related to implementation, including the choice of tuning parameter $h$, are discussed below.

We present a theorem establishing that, under certain conditions, the combination obtained by optimizing the smooth approximation to the empirical TPR while constraining the smooth approximation to the empirical FPR has desirable operating characteristics. In particular, its FPR is bounded almost surely by the acceptable level $t$ in large samples. In addition, its TPR converges almost surely to the supremum of the TPR over the set where the FPR is constrained. We focus on the operating characteristics of $(\tilde{\boldsymbol{\theta}}_{t},\tilde{\delta}_{t})$ since these are of primary interest to clinicians.

Rather than enforcing $(\tilde{\boldsymbol{\theta}}_{t},\tilde{\delta}_{t})$ to be a strict maximizer, in the theoretical study below we allow it to be a near-maximizer of $\tilde{\mbox{TPR}}_{n_{1}}(\boldsymbol{\theta},\delta)$ within $\tilde{\Omega}_{t,n_{0}}$ in the sense that

where $a_{n}$ is a decreasing sequence of positive real numbers tending to zero. This provides some flexibility to accommodate situations in which a strict maximizer either does not exist or is numerically difficult to identify.

Before stating our key theorem, we give the following conditions.

1. (1)

   Observations are randomly sampled conditional on disease status $D$, and the group sizes tend to infinity proportionally, in the sense that $n=n_{1}+n_{0}\rightarrow\infty$ and $n_{1}/n_{0}\rightarrow\rho\in(0,1)$.

2. (2)

   For each $d\in\{0,1\}$, observations $\textbf{X}_{di}$, $i=1,2,\ldots,n_{d}$, are independent and identically distributed $p$-dimensional random vectors with distribution function $F_{d}$.

3. (3)

   For each $d\in\{0,1\}$, no proper linear subspace $S\subset\mathbb{R}^{p}$ is such that $P(\textbf{X}\in S\mid D=d)=1.$

4. (4)

   For each $d\in\{0,1\}$, the distribution and quantile functions of ${\boldsymbol{\theta}}^{\top}\textbf{X}$ given $D=d$ are globally Lipschitz continuous uniformly over ${\boldsymbol{\theta}}\in\mathbb{R}^{p}$ such that $\|{\boldsymbol{\theta}}\|=1$.

5. (5)

   The map $(\boldsymbol{\theta},\delta)\mapsto\mbox{TPR}({\boldsymbol{\theta}},\delta)$ is globally Lipschitz continuous over $\Omega=\{{\boldsymbol{\theta}}\in\mathbb{R}^{p},\delta\in\mathbb{R}:||{\boldsymbol{\theta}}||=1\}$.

The proof of Theorem 1 is given in Appendix B. The proof relies on two lemmas, also in Appendix B. Lemma 1 demonstrates almost sure convergence to zero of the difference between the supremum of a function over a fixed set and the supremum of the function over a stochastic set that converges to the fixed set in an appropriate sense. Lemma 2 establishes the almost sure uniform convergence to zero of the difference between the FPR and the smooth approximation to the empirical FPR and the difference between the TPR and the smooth approximation to the empirical TPR. The proof of Theorem 1 demonstrates that Lemma 1 holds for the relevant function and sets, relying in part on the conclusions of Lemma 2. The conclusions of Lemmas 1 and 2 then demonstrate the claims of Theorem 1.

Certain considerations must be addressed to implement the proposed method, including the choice of tuning parameter $h$ and starting values $(\tilde{{\boldsymbol{\theta}}},\tilde{\delta})$ for the optimization routine. In using similar methods to maximize the AUC, Lin et al. (2011) proposed using $h=\tilde{\sigma}n^{-1/3}$, where $\tilde{\sigma}$ is the sample standard error of $\tilde{{\boldsymbol{\theta}}}^{\top}\textbf{X}$. In simulations, we considered both $h=\tilde{\sigma}n^{-1/3}$ and $h=\tilde{\sigma}n^{-1/2}$ and found similar behavior for the convergence of the optimization routine. Thus, we use $h=\tilde{\sigma}n^{-1/2}$. We must also identify initial values $(\tilde{{\boldsymbol{\theta}}},\tilde{\delta})$ for our procedure. As done in Fong, Yin, and Huang (2016), we use normalized estimates from robust logistic regression, which is described in greater detail below. Based on this initial value $\tilde{{\boldsymbol{\theta}}}$, we choose $\tilde{\delta}$ such that $\tilde{\mbox{FPR}}_{n_{0}}(\tilde{{\boldsymbol{\theta}}},\tilde{\delta})=t$. In addition, we have also found that when $\tilde{\mbox{FPR}}_{n_{0}}$ is bounded by $t$, the performance of the optimization routine can be poor. Thus, we introduce another tuning parameter, $\alpha$, which allows for a small amount of relaxation in the constraint on the smooth approximation to the empirical FPR, imposing instead $\tilde{\mbox{FPR}}_{n_{0}}({\boldsymbol{\theta}},\delta)\leq t+\alpha.$ Since the effective sample size for the smooth approximation to the empirical FPR is $n_{0}$, we chose to scale $\alpha$ with $n_{0}$, and have found $\alpha=1/(2n_{0})$ to work well.

Our method involves computing the gradient of the smooth approximations to the true and false positive rates defined above, which is fast regardless of the number of biomarkers involved. This is in contrast with methods that rely on brute force (e.g., grid search), which typically become computationally infeasible for combinations of more than two or three biomarkers. However, we note that for any method, the risk of overfitting is expected to grow as the number of biomarkers increases relative to the sample size. We emphasize that our method does not impose constraints on the distribution of the biomarkers that can be included, except for weak conditions that allow us to establish its large-sample properties.

First, we considered bivariate normal biomarkers with contamination, similar to a scenario described by Croux and Haesbroeck (2003). In particular, we considered a setting where two biomarkers $(X_{1},X_{2})$ were independently normally distributed with mean zero and variance one. $D$ was then defined as $D=1(2X_{1}+2X_{2}+\zeta>0),$ where $\zeta$ was distributed as a logistic random variable with location parameter zero and scale parameter one. Next, the sample was contaminated by a set of points with $D=0,X_{1}=6,$ and $X_{2}=6$. We consider simulations where the training set consisted of 800 or 1600 “typical” observations and 50 or 100, respectively, contaminating observations (this yielded a disease prevalence of approximately 47%). The test set consisted of $10^{6}$ “typical” observations and 62,500 contaminating observations. The maximum acceptable FPR, $t$, was 0.2 or 0.3. We performed 1000 simulations.

We considered five approaches: (1) logistic regression, (2) the robust logistic regression method proposed by Bianco and Yohai (1996), (3) grid search, (4) the method proposed by Su and Liu (1993) and (5) the proposed method. As discussed above, the method proposed by Su and Liu (1993) yields a combination with maximum AUC when the biomarkers have a conditionally multivariate normal distribution. We did not consider the optimal AUC method proposed by Huang, Qin, and Fang (2011) as the implementation provided in R is too slow for use in simulations (and, as illustrated in Figure 1, may not yield a combination with optimal TPR). While the methods recently proposed by Yan et al. (2018) to optimize the partial AUC are compelling and may yield a combination with high TPR at the specified FPR value, implementation of their method, particularly the nonparametric kernel-based method, is non-trivial, and so is not included here. Finally, the method of Liu, Schisterman, and Zhu (2005), discussed above, may also yield a combination with high TPR at a particular FPR. However, given the shortcomings of this method described above (namely, that the range of FPRs over which the combination is optimal cannot be fixed in advanced and the biomarkers are assumed to have a conditionally multivariate normal distribution), we do not include this as a comparison method. Above all, none of these methods specifically target the TPR for a specified FPR which, as indicated by Figure 1, may lead to combinations with reduced TPR at the specified FPR.

We focused on evaluating the operating characteristics of the fitted combination rather than the biomarker coefficients as the former is typically of primary interest. In particular, we evaluated the TPR in the test data for FPR = $t$ in the test data. In other words, for each combination, the threshold used to calculate the TPR in the test data was chosen such that the FPR in the test data was equal to $t$. Evaluating the TPR in this way puts the combinations on equal footing in terms of the FPR, and so allows a fair comparison of the TPR. We evaluated the FPR of the fitted combinations in the test data using the thresholds estimated in the training data, i.e., the $(1-t)$th quantile of the fitted biomarker combination among controls in the training data. While we could have used the estimate of $\delta_{t}$ provided by our method in the evaluation, we found improved performance (that is, better control of the FPR) when re-estimating the threshold based on the fitted combination in the training data.

Table 1 summarizes the results. For both sample sizes and FPR thresholds, all methods adequately controlled the FPR, while for the TPR, the proposed method outperformed logistic regression, robust logistic regression, and the method of Su and Liu (1993). Furthermore, the results from the proposed method were comparable to those from the grid search, which may be regarded as a performance reference but is infeasible for more than two or three biomarkers.

We also considered simulations with conditionally multivariate lognormal biomarkers (Mishra, 2019). In particular, we considered three biomarkers $(X_{1},X_{2},X_{3})$. Among controls, $\mbox{log}(X_{1})$ and $\mbox{log}(X_{2})$ had a bivariate normal distribution with $E(\mbox{log}(X_{1}))=1.1,E(\mbox{log}(X_{2}))=1.1,Var(\mbox{log}(X_{1}))=0.04,Var(\mbox{log}(X_{2}))=0.5$ and $Cov(\mbox{log}(X_{1}),\mbox{log}(X_{2}))=0.09$. Among cases, $\mbox{log}(X_{1})$ and $\mbox{log}(X_{2})$ had a bivariate normal distribution with $E(\mbox{log}(X_{1}))=1,E(\mbox{log}(X_{2}))=1,Var(\mbox{log}(X_{1}))=0.05,Var(\mbox{log}(X_{2}))=0.05$ and $Cov(\mbox{log}(X_{1}),\mbox{log}(X_{2}))=0.015$. The third biomarker, $X_{3}$ was simulated from an independent lognormal distribution with $E(\mbox{log}(X_{3}))=1.65$ and $Var(\mbox{log}(X_{3}))=4.66$ among both cases and controls. Given the performance of the method of Su and Liu (1993) and the performance of the proposed method relative to grid search observed above (and the computational challenges of implementing grid search for three biomarkers), we considered three methods here: (1) logistic regression, (2) robust logistic regression, and (3) the proposed method. Although neither logistic regression nor robust logistic regression performed particularly well in the simulations in Section 4.1, these methods represent the most commonly used approach for constructing biomarker combinations and the method used to provide starting values for the proposed method, respectively. Thus, it was important to include them here.

The maximum acceptable FPR, $t$, was 0.2 and 1000 simulations were performed. The training data consisted of either 400 cases and 400 controls, or 800 cases and 800 controls. The test data consisted of $10^{6}$ observations. The TPR and FPR were evaluated as described above. We present the results in Table 2. All three methods did well in controlling the FPR at the specified value. Furthermore, the proposed method substantially outperformed logistic regression and robust logistic regression: the mean TPR based on the proposed method was at least 20% larger than the mean TPRs from logistic regression and robust logistic regression.

The above simulations demonstrate superiority of our approach relative to alternative methods in particular scenarios. We conducted further simulations to demonstrate the feasibility of our approach in other settings (for instance, small sample size, small and large prevalence, and low FPR cutoffs) relative to logistic regression and robust logistic regression.

We considered simulations with and without outliers in the data-generating distribution, and simulated data under a model similar to that used by Fong, Yin, and Huang (2016). We considered two biomarkers $X_{1}$ and $X_{2}$ constructed as

where $\Delta$ was a Bernoulli random variable with success probability $\pi=0.05$ when outliers were simulated and $\pi=0$ otherwise, and $Z_{0}$ and $Z_{1}$ were independent bivariate normal random variables with mean zero and respective covariance matrices

$D$ was then simulated as a Bernoulli random variable with success probability $f\left\{\beta_{0}+4X_{1}-3X_{2}\right.$ $\left.-0.8(X_{1}-X_{2})^{3}\right\}$. We considered two $f$ functions: $f_{1}(v)=\mbox{expit}(v)=e^{v}/(1+e^{v})$ and a piecewise logistic function,

We varied $\beta_{0}$ to reflect varying prevalences, with a prevalence of approximately 50–60% for $\beta_{0}=$ 0, 16–18% for $\beta_{0}<$ 0, and 77–82% for $\beta_{0}>$ 0. We considered $t=$ 0.05, 0.1, and 0.2. A plot illustrating the data-generating distribution with $f=f_{1}$ and $\beta_{0}=$ 0, with and without outliers, is given in Appendix D.

The training data consisted of 200, 400, or 800 observations while the test set included $10^{6}$ observations. The TPR and FPR were evaluated as described above. The results are presented in Appendix C. When no outliers were present, the proposed method was comparable to logistic regression and robust logistic regression in terms of both the TPR and FPR. In the presence of outliers, robust logistic regression tended to provide combinations with higher TPRs than did logistic regression, and the TPRs of the combinations provided by the proposed method tended to be comparable to or somewhat better than the results from robust logistic regression. In all scenarios, all three methods controlled the FPR, particularly as sample size increased. In addition to demonstrating feasibility of our approach, these simulations highlight the fact that logistic regression is relatively robust to violations of the linear-logistic model (e.g., nonlinear biomarker combinations and deviations from the logit link).

In most simulation settings, convergence of the proposed method was achieved in more than $96\%$ of simulations. For some of the more extreme outlier scenarios considered in Section 4.3, convergence failed in up to $7.3\%$ of simulations.