Our paper “How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions” [2] looks at racial disparities in the Allegheny Family Screening Tool [6], an algorithm used to aid child welfare workers deciding which families the county government should investigate. The paper uses a mixed methods approach, reporting both quantitative and qualitative findings. The quantitative findings are based on real, anonymized data on children who were reported to the Allegheny County Office of Children, Youth, and Families (CYF), the AFST scores that these children received, whether the children were screened in for investigation or not, and whether the children were placed in foster care.

We retrieved the data within a secure file location in Carnegie Mellon University used to store sensitive data received from Allegheny County CYF, which is maintained by colleagues of ours. The primary data we used in our paper was stored in files named PAN_Retro_Run_Referrals_for_2014-2016_provided_2018-08-29.csv and PAN_Retro_Run_08012016_07132018.csv. At the time of writing our paper, we believed this data to be the same as those used in an CYF-commissioned Impact Evaluation of the AFST done by Stanford researchers [4]. As such, we preprocessed the data in the same way that Goldhaber-Fiebert and Prince [4] describe in their paper: First, we excluded all referrals not marked General Protective Services (GPS) by deleting all data entries where the variable REFER_TYPE_GPS_NULL was not 1. Second, we excluded referrals connected to active and closed cases by deleting entries where the variable CALL_SCRN_OUTCOME was either ‘Accept: Actively working with this family’ or ‘Screen Out: **Assessment Completed on Active Family**’ in a secondary dataset named LIM_REFERRAL_CLIENTS_08262021.csv. We exclude referrals which the AFST would not have had an impact on: non-GPS referrals and referrals that are connected to active cases are automatically screened in by workers, so the AFST would have had no impact (because workers had no choice).

On April 19, 2022, we received notice from employees at Allegheny County CYF that they have been using a different dataset and pre-processing steps for their internal analysis. They recommended that we not use the PAN_Retro files (referred to above) to define the population of referrals, the CALL_SCRN_OUTCOME variable to exclude active cases, nor the MCI_ID variable we used to identify each individual child. Particularly, they said the MCI_ID variable contained some errors, where one child could be assigned multiple children. Instead, they recommended that we use a new variable called MCI_UNIQ_ID, which is similar to MCI_ID but removes many of these erroneous redundancies, to identify individual children; and that we use a new variable called ACTIVE_FAMILY_IND to identify active cases. Neither MCI_UNIQ_ID nor ACTIVE_FAMILY_IND were available to us at the time we wrote our paper. On April 21, 2022 we had an online video meeting with a member from the CYF Child Welfare Analytics team and received confirmation on the new pre-processing steps we would take to re-run our analysis. On April 27, 2022, a CYF employee sent us a new dataset LIM_REFERRAL_CLIENTS_UNIQapp_04212022_v3.csv which includes information on all referrals from January 1, 2015 through July 13, 2018, including the ACTIVE_FAMILY_IND and MCI_UNIQ_ID variables. They also sent us a file called RETRO_FILES_COMBINED_04212022_v3.csv which contains retroactively-run AFST scores (which corrected for a glitch in AFST V1 scores [3]) for children reported during this time period. Following CYF’s recommendations to use this new data they sent us, as well as which pre-processing steps to use, we re-ran all of the quantitative analyses in our paper. For clarity, we will refer to the data, pre-processing, and quantitative analysis in our original paper as Analysis 1 and this second analysis based on new data and pre-processing as Analysis 2.¹¹1All of our code is publicly available at https://github.com/logan-stapleton/AFST_racial_disparity.

Although all of the specific numbers we reported in Analysis 1 changed in Analysis 2, our primary conclusions remain the same. Our primary quantitative results are almost identical under both Analyses 1 and 2, even with changes in the data and pre-processing. This lends support to the robustness of our findings that: 1) The AFST on its own was more racially disparate than workers, both in terms of screen-in rate and accuracy; and 2) that workers were able to reduce this disparity in the algorithm. This second result is interesting and surprising, given that child welfare workers are known to make racially disparate decisions without algorithms [5]. See Figure 1 for a comparison of screen-in rate disparities across Analyses 1 and 2. See Table 1 for a full comparison of disparities by risk level.

The two Analyses on disparity in accuracy show similar patterns: AFST-only decisions had higher racial disparity in accuracy than worker-AFST decisions (see Figure 2(a) and 2(b)). However, one aspect of the accuracy comparison that we explicitly mentioned in the paper changed between Analyses: In Analysis 1, we found that overall AFST-only decisions were more accurate than worker-AFST decisions (by 4.5%). In Analysis 2, we found that overall AFST-only decisions were less accurate than worker-AFST decisions (by 2%). See Figure 2 on page 2. This changes our interpretation of these findings from the original paper. We no longer believe that we can draw clear conclusions based upon the current analyses, about whether the AFST is more accurate than workers or vice versa, even when evaluating performance on the predictive targets used by the AFST itself. It is notable that the results of this comparison changed between Analysis 1 and 2: this highlights the sensitivity of claims about overall accuracy in the AFST to changes in data and pre-processing. However, we reiterate the arguments presented in our paper that there are additional reasons to be cautious when interpreting and reporting quantitative results about accuracy and comparisons of accuracy because of (1) the inherent challenges of accuracy measurement in risk assessments for which the predictions affect the predictive targets (see Section 4.2 of our paper [2]), and (2) our qualitative finding that workers and the AFST do not agree on prediction outcomes and accuracy measures, meaning that evaluating human workers’ performance on the AFST’s benchmarks alone is akin to evaluating a player’s performance at a game they are not actually playing (see Section 6.2 of our paper [2]).

Another quantitative result (that we did not highlight in the paper due to concerns around confounding factors) was that the actual screen-in rate increased from before to after the deployment of the AFST. In Analysis 1, we found that, pre-AFST, 52.5% of Black children and 41.2% of white children in discretionary referrals were screened in; and post-AFST, 61.8% of Black children and 52.8% of white children were screened in. In Analysis 2, we found that, pre-AFST, 50.5% of Black children and 42.7% of white children in discretionary referrals were screened in; and post-AFST, 50.2% of Black children and 43.1% of white children were screened in. Thus, following Analysis 1, we calculated that the screen-in rate increased about 10% from pre- to post-AFST, whereas in Analysis 2 there was no noticeable change in the screen-in rate. This indicates that claims about the overall screen-in rate may be more sensitive to changes in data and pre-processing (the design of exclusion criteria).

Summary
Our main findings were robust to changes in data and pre-processing. Some results changed. However, as these results were less novel or important, we believe they do not significantly alter the primary contributions of our original paper. See the appendix below for a full comparison of Analyses 1 and 2.