Deepfake Detection by Human Crowds, Machines, and Machine-informed Crowds

As stated in our pre-analysis plan¹¹1Pre-analysis plan for non-recruited participants in Experiment 1: https://aspredicted.org/blind.php?x=wg84ic for Experiment 1, we examined the accuracy of all participants who saw at least 10 pairs of videos, for a total of 882 participants. 82% of participants outperform the leading model, which achieves 65% accuracy on the holdout dataset [36]. Half of the stimuli set (28 of 56 pairs of videos) was identified correctly by over 83% of participants, 16 pairs of videos were identified correctly by between 65% and 83% of participants, and 12 pairs of videos were identified correctly by less than 65% of participants. Out of these 12 pairs of videos, 3 pairs of videos were identified correctly by less than 50% of participants. Figure 2a presents the distribution of participants’ performance in Experiment 1 (in blue in the second column) next to the model’s overall performance (in black in the first column).

We do not find any evidence that participants improve in their ability to detect these videos within the first ten videos seen ($p=0.112$) (all p-values reported in this paper are generated by linear regression with robust standard errors clustered on participants unless otherwise stated). On average, participants took 42 seconds to respond to each pair of videos, and we find that for every additional ten seconds participants take to respond, participants’ accuracy decreased by 1.1 percentage points ($p<0.001$). We embedded three randomized experiments in Experiment 1 to evaluate the roles of specialized processing of faces, time for reflection, and emotion elicitation. We find participants are 5.6 percentage points less accurate ($p=0.004$) at detecting pairs of inverted videos than pairs of upright videos. In contrast, we do not find statistically significant effects of the additional time for reflection intervention or this particular emotion elicitation intervention. The custom emotion elicitation intervention in this first experiment did not have a statistically significant influence on participants’ self-reported emotions, which suggests the custom emotion elicitation experiment did not work here. We provide additional details on the interventions in Experiment 1 in the Supplementary Information section.

For participants who pass the attention check, recruited participants accurately identified deepfakes from the randomly sampled holdout videos in 66% of attempts while the non-recruited participants accurately identified videos in 69% of trials (or 72% of attempts when limiting the analysis to non-recruited participants who saw at least 10 videos). In comparison, the leading model accurately identified deepfakes on 80% of the sampled videos, which is significantly better than the 65% accuracy rate this model achieves on the full holdout dataset of 4,000 videos [36].

In a direct comparison of performance, 13% of recruited participants, 27% of non-recruited participants who saw at least 10 videos, and 37% of non-recruited participants who saw fewer than 10 videos outperform the model. Figure 2a presents the distribution of participants’ accuracy on the sampled holdout videos (in teal for recruited participants and gold for non-recruited participants). Relative to the leading model, participants are less accurate at identifying deepfakes than they are at identifying real videos. Recruited participants accurately identify deepfakes as deepfakes in 57% of attempts compared to the leading model identifying deepfakes as deepfakes in 84% of videos while both recruited participants and the leading model identify real videos as real videos at nearly same rate (75% of participants’ observations and 76% of videos). Recruited participants predicted the sampled holdout videos were real (57% of observation) considerably more often than fake (38% of observations) while the computer vision model predicted videos were real (44% of observations) barely more frequently than fake (42% of observations). In 5% of recruited participant observations and 14% of computer vision model observations, the prediction was a 50-50 split between real and fake. We report confusion matrices for each treatment condition in Tables 3-7 in the Supplemental Information.

On the additional set of videos of political leaders, participants outperform the leading model. Specifically, 60% of recruited participants and 68% of non-recruited participants who saw at least ten videos outperform the model on these videos. For the deepfake videos of Kim Jong-un, Vladimir Putin, and the attention check, the state-of-the-art computer vision model outputs a 2%, 8%, and 1% probability score that each respective video is a deepfake, which is both confident and inaccurate.

We do not find any evidence that participants’ overall accuracy changes as participants view more videos ($p=0.433$). However, we find that for every additional video seen by recruited participants, they are 0.9% ($p<0.001$) more likely to report any video as a deepfake. This corresponds to performing about 18% better at detecting deepfakes and 18% worse at identifying real videos by the last video.

Recruited participants spent a median duration of 22 seconds before submitting their initial guess and a median duration of 3 seconds adjusting (or not adjusting) their initial guess when prompted with the model’s predicted likelihood. Non-recruited participants spend a similar amount of time. For both sets of participants, we find that for every ten additional seconds of participant response time, participants’ accuracy decreases by 1 percentage point ($p<0.001$).

The crowd mean, participants’ responses averaged per video, is on par with the leading model performance on the sampled holdout videos. For recruited participants, the crowd mean accurately identifies 76% of videos. For non-recruited participants, the crowd mean accurately identifies 80% of videos. For the 1,879 non-recruited participants who saw at least 10 videos, the crowd mean is 86% accurate. In comparison, the leading model accurately identifies 80% of videos.

In Figure 2b, we compare statistics on participants’ accuracy (the mean and interquartile range) with the model’s predictions for each video. In Table 2 in the Appendix, we present the mean accuracy of recruited and non-recruited participants and the computer vision model for all videos. There are 2 videos (both deepfakes) on which both the crowd mean and the leading model are at or below the 50% threshold. One of these videos (video 7837) is extremely blurry, while the other video (video 4555) is filmed from a low angle and the actress’s glasses show significant glare.

There are 8 videos on which the crowd mean is accurate but the model is at or below the 50% threshold and another 5-12 videos on which the model is accurate but the crowd mean (depending on how the population selected) is below the 50% threshold.

In addition to comparing individual and collective performance to the leading model’s performance, we examined how an AI model could complement human performance. After participants’ submitted their initial response for how confident they are that a video is or is not a deepfake in Experiment 2, we revealed the likelihood that the video is a deepfake – as predicted by the leading model – and gave participants a chance to update their response. After taking into account the model’s prediction, participants updated their confidence in 24% of trials (and crossing the 50% threshold for accurate identification in 12% of trials). By updating their responses, recruited participant’s accurate identification increased from 66% to 73% of observations ($p<0.001$ based on a Student’s t-test). Figure 2c presents the distribution of changes in overall participant accuracy for the 50 videos sampled from the DFDC. For the 40 videos upon which the model accurately identifies the video as a deepfake or not, participants updated their responses to be on average 10.4% more accurate at identification than before seeing the model’s prediction. For the remaining 10 videos on which the model made an incorrect or equivocal prediction, participants updated their responses to be on average 2.7% less accurate at identification than before seeing the model’s predictions. In the most extreme example of incorrect updating, the model predicted a 28% likelihood the video was a deepfake when it was indeed a deepfake and participants updated their responses to be on average 18% less accurate at identifying the deepfake. This particular video (video 7837) is quite blurry, and perhaps, participants changed their responses because it’s very difficult to discern manipulations in low quality video.

For the additional deepfake videos of Kim Jung-un and Vladimir Putin that are not included in the overall analysis, the model predicted a 2% and 8% likelihood the video was a deepfake, respectively. This prediction is not only incorrect but confidently so, which led participants to update their responses such that participant’s accurate identification dropped from 56% to 34% on the Kim Jung-un deepfake and 70% to 55% on the Vladimir Putin deepfake.

In Figure 2d, the receiver operating characteristic (ROC) curve of the leading model is plotted alongside the ROC curves of the crowd mean and crowd-mean responses where participants have access to the model’s prediction for each video. While the model has a slightly higher AUC score of 0.957 relative to the crowd means’s AUC score of 0.936, either the model or the crowd mean could be considered to perform better depending on the acceptable false positive rate. However, the crowd mean response after seeing the model’s predictions strictly outperforms both the performance of the crowd mean and leading model. When we condition the ROC analysis on confidence following methods for estimating the reliability of eyewitness identifications [66], we find that medium and high confidence responses outperform low confidence responses by a large degree. We define low confidence as responses between 33.5 and 66.5, medium confidence as responses between 17 and 33.5 or 66.5 and 83, and high confidence as responses between 0 and 17 or 83 and 100. Figure S2 in the Supporting Information ROC curves presents a visual comparison of model performance to low, medium, and high confidence responses from participants, which reveals medium and high (but not low) confidence responses can outperform the model’s predictions depending on the acceptable false positive rate.

Given the heterogeneity in both participants’ and the leading model’s performance on videos, we extend the analysis of performance across seven video-level features: graininess, blurriness, darkness, presence of a flickering face, presence of two people, presence of a floating distraction, and the presence of an individual with dark skin. These video-level characteristics were hand-labeled by the research team. On the 14 videos that are either grainy, blurry, or dark, the crowd wisdom of recruited participants is correct on 8 videos while the crowd wisdom of non-recruited participants and the model is correct on 10 videos. When we examine the 36 videos that are neither grainy, blurry, nor extremely dark, the crowd wisdom of recruited participants is correct on 29 out of 36 videos, the crowd wisdom of non-recruited participants is correct on 32 out of 36 videos, and the model is correct on 30 of 36 videos. The presence of a flickering face is associated with an increase in recruited participants’ accuracy rates by 24.2 percentage points ($p<0.001$) and an increase in the model’s accuracy rates by 16.9 percentage points ($p=0.007$) in detecting a deepfake. The presence of two people in a video instead of a single person is associated with an overall increase in recruited participants’ accuracy rates by 7.6% ($p<0.001$) and a 21.8% decrease ($p=0.024$) by the model in identifying real videos. The presence of a floating distraction is associated with a decrease in recruited participants’ accuracy rates on real videos of 3.5% ($p=0.034$) and an increase in recruited participants’ accuracy rates on fake videos of 11.3% ($p<0.001$). In 12 of 50 videos, at least one person in the video has dark skin (precisely defined as skin classified as type 5 or 6 on the Fitzpatrick scoring system, which is a classification system developed for dermatology that computer vision researchers have used to examine the context of skin color) [67]. We find that the presence of an individual with dark skin in the video is associated with a decrease in recruited participants’ accuracy by 8.8% ($p<0.001$) and a decrease in the model’s accuracy by 11.8% ($p=0.187$). In order to control for these seven comparisons conducted simultaneously, we can apply a Bonferroni correction of 1/7 to the standard statistical significance thresholds (e.g., a p-value threshold of 0.01 becomes 0.0014). Based on this correction, the influence of a flickering face, two people in the same video, floating distractions, and the presence of an individual with dark skin continue to be statistically significant for participants if the original p-value threshold is chosen as 0.01.

Within Experiment 2, we embedded two randomized experiments to examine potential cognitive mechanisms underpinning how humans discern between real and fake videos. Specifically, we examine an affective intervention designed to elicit anger based on a well-established intervention [68] (see Figure S3 in the Supporting Information) and a perceptual intervention designed to obstruct specialized processing of faces via inversion (videos presented upside down), misalignment (videos presented with actors’ faces horizontally split), and occlusion (videos presented with a black bar over the actors’ eyes).

We present results of the anger elicitation intervention in columns 1-3 in Table 1. We do not find statistically significant effects ($p=0.280$) of the anger elicitation intervention on overall accuracy. However, in our pre-registered follow-up analysis limiting the dataset to real videos, we find that participants who were assigned to the anger elicitation treatment underperformed control participants by 5.2 percentage points ($p=0.032$). In other words, participants in the anger elicitation treatment are more 5.2 percentage points more likely than participants in the control group to make a false positive identification that a real video is a deepfake. Notice here that the floor is not 0% accuracy but rather 50% accuracy (i.e., chance responding); the maximum effect of the anger elicitation treatment is 21.6 percentage points (71.6 from the constant term in column 2 of Table 1 minus 50), so a 5.2 percentage point reduction represents an effect that is 24.1% of the maximum possible effects under these conditions.

Figure S3 in the Supporting Information presents accuracy and confidence scores by treatment assignment to visually reveal the heterogeneous effect of anger elicitation on how participants discern between real and fake videos. When we examine the relationship between assignment to the anger elicitation group and how confident participants guess, we do not find a statistically significant effect ($p=0.347$). When we examine real videos and the relationship between anger elicitation and updating predictions after seeing what the model would predict, we find that participants assigned to the anger elicitation group are 3.7% ($p=0.035$) more likely to change their guess to a correct answer than participants assigned to the control group. As a result, we do not find statistically significant effects of anger elicitation on accuracy after participants update their response ($p=0.246$).

We present results of the perceptual obstruction intervention in columns 1-6 in Table 1. We find statistically significant effects of all three specialized processing obstructions on participants’ ability to accurately identify deepfakes from authentic videos. The overall effects – reported in column 1 of Table 1 – are all statistically significant and range from a decrease of 4.3 percentage points in accuracy for the inversion treatment ($p=0.002$) to a decrease of 4.4 percentage points in accuracy for the eye occlusion treatment ($p=0.004$) to a decrease of 6.3 percentage points for the misalignment treatment ($p<0.001$) on a base rate of 65.5% accuracy when controlling for video fixed effects. In addition, we find that inverting the videos decreases participants’ reported confidence scores (absolute distance in guesses from the 50-50 selection) by 2 percentage points ($p=0.002$), but we do not find similar decreases in reported confidence on videos with misalignment or occlusion transformations.

In the sample of recruited participants, the specialized face processing obstructions have different effects depending on whether the videos are manipulated or not. When we limit the analysis to the algorithmically manipulated deepfakes (column 3 of Table 1), we do not find statistically significant effects on the inversion treatment ($p=0.638$) but we do find that the misalignment and eye occlusion treatments show a decrease in 7.7 ($p=0.002$) and 6.3 ($p=0.008$) percentage points, respectively, relative to the control videos. In contrast, when we limit the analysis to the other half of videos that have not been manipulated (column 2 of Table 1), we do not find statistically significant effects for misalignment ($p=0.075$) or eye occlusion interventions ($p=0.263$), but we find participants’ accuracy on inverted authentic videos is 9.1 percentage points lower than when the videos are upright ($p<0.001$).

The experimental results on non-recruited participants provide a replication and robustness check for the results on the recruited participants. The results from the non-recruited participants were not pre-registered because we weren’t expecting many people to continue visiting our website organically. In fact, 9,188 visitors participated in the single video design between November 2020 and January 2021. In columns 4 through 6 in Table 1, we present the linear regressions results of the specialized face processing obstruction interventions on non-recruited participants’ accuracy. Similar to the results for the recruited sample, we find statistically significant effects ($p<0.001$) of all three obstruction interventions on ability to accurately discern deepfakes from authentic videos. The number of observations in the non-recruited sample is over 16 times larger than the number of observations in the recruited sample. Likewise, the number of participants is 33 times larger. These numbers differ because the number of videos seen by participants in the non-recruited sample varied depending on participants’ interest. With a larger sample size, we see statistically significant and negative effects of obstructions on all videos ranging from 4% percentage point drop in accuracy from the eye occlusion intervention ($p<0.001$) to a 7% drop on accuracy from the misalignment intervention ($p<0.001$). We also find all three treatments decrease participants’ confidence scores by one half to one percentage point ($p<0.001$).

In the non-recruited sample, each of the 50 videos were viewed by between 945 to 1168 participants. We run separate linear regressions for each video and find statistically significant and negative effects at the 1% significance level for inversion in 22 videos, misalignment in 14 videos, and occlusion in 22 videos. Furthermore, we find at least one of these specialized processing obstructions is negative and statistically significant at the 1% significance for 29 of the 50 videos.

In the sample of videos of political leaders, the specialized face processing obstructions had a significant effect on participants’ ability to accurately identify the Vladimir Putin deepfake as a deepfake. The misalignment obstruction leads to a drop in accuracy of 20.7 percentage points ($p=0.001$). Likewise, the occlusion obstruction leads to a drop of 10.3 percentage points ($p=0.002$) and the inversion obstruction leads to a drop of 5.2 percentage points ($p=0.072$).

In columns 7 through 9 in Table 1, we present the linear regression results of the specialized face processing obstruction interventions on the model’s predictions and find the computer vision model is affected by one specialized face processing obstruction but not the other two. We find the computer vision model’s predictive accuracy drops by 12.1 percentage points on the inverted videos ($p=0.005$). We do not find a statistically significant difference in accuracy between either the control and misalignment sets of videos ($p=0.800$) or the control and occlusion sets of videos ($p=0.944$).

In the pilot experiment, we selected a sample of 56 pairs of videos from the competition training dataset of 19,154 authentic videos and 100,000 associated deepfakes. We selected a sample based on two criteria: First, we developed a neural network model to predict which videos are deepfakes and selected the 3% of deepfake videos on which our model was highly confident it made the right decision but in fact was wrong. The goal of this selection process was to approximately identify the hardest videos for machines. Second, based on this subset of 3,000 deepfakes and 1,809 associated authentic videos, we randomly selected 56 unique deepfakes and their associated real video excluding obvious manipulations with fast flickering faces.

On the Detect Fakes website, two videos – one authentic and one deepfake manipulation of that authentic video – were displayed side-by-side. Videos were randomly assigned to be on the left or right side of the screen. Participants were asked to guess which video contains a deepfake manipulation. After each selection, participants were told which video was a deepfake and which was the original.

5,524 individuals participated in this experiment. The sample size was pre-specified in the pre-analysis plan as the number of participants 3 months after the \nth1000 participant engages in the experiment. Participants visited from all over the world and the top 5 most represented countries were United States (34%), Germany (6%), United Kingdom (5%), Saudi Arabia (3%), and Canada (3%).

When participants visit the experiment, we store a cookie to keep track of each anonymous individual. We randomly assign participants to see one of 56 pairs of videos and allow participants to watch the videos and submit their guess as to which video is a deepfake. After participants submit their guess, we randomly assign another pair of videos without replacement.

We cross-randomize participants to treatments within three interventions: (1) inversion, (2) emotion elicitation, and (3) reflection. In the inversion intervention, we randomly assign participants to see an inverted pair of videos on the \nth4 or \nth5 pair of videos that they see and show all participants a pair of inverted videos on the \nth9 pair of videos shown. In the emotion elicitation intervention, we randomly assign participants to four control arms asking how happy, angry, anxious, or sad a participant feels and three treatment arms where we attempt to elicit incidental happiness, anger, and anxiety. In each control and treatment arm, we asked participants how happy, angry, anxious, or sad they feel on a scale of 1 to 10. The treatment arms include pop-up messages in between participants’ guesses. The emotion elicitation intervention included a message, "There is scientific evidence that emotions help us make decisions. Think about a time you were really [angry/happy/anxious]. Before you continue, think about why you were really [angry/happy/anxious]." After each guess a similar message would pop up before participants would see the next videos. In the reflection intervention, we randomly assigned participants to a control group with normal load times and a treatment group where the website loads with a 600 millisecond delay.

We examine the causal effects of the three treatment assignments, $T_{x}$ on the accuracy score, $y_{i,j}$ of participant $i$ on manipulated video $j$. We include video fixed effects, $\mu_{j}$, and the following fixed-effects linear regression model reflects the pre-analysis plan with $\epsilon_{i,j}$ representing the error term:

We cluster errors at the individual level with Eicker-Huber-White robust standard errors because the emotion elicitation assignment and reflection assignment is conducted at the participant level [90]. As hypothesized in the pre-analysis plan, the inversion treatment has a negative effect ($p=0.004$). The emotion elicitation treatments did not appear to effectively elicit emotions, and as a result we do not find an effect ($p=0.209$ for anger, $p=0.867$ for happy, $p=0.560$ for anxiety). In fact, we do not find evidence that the self-reported emotional intensities of the participants assigned to the emotion elicitation treatment were different than the intensities reported by the control group ($p=0.190$ for happy, $p=0.558$ for anxiety, $p=0.091$ for anger). We interpret these results to indicate that our emotion elicitation intervention in Experiment 1 did not reliably elicit emotional responses. We do not find an effect of the reflection treatment on outcomes ($p=0.592$).

In Experiment 2, we randomly sampled 50 videos from the 4,000 videos in the DeepFakes Detection Competition holdout set [31]. 25 videos are deepfakes and 25 videos are authentic. Unlike videos from the competition training set sampled in the pilot experiment, some the these videos included distractions such as flickering shapes and faces that move around the video intended to throw off the computer vision model. In order to ensure participants did not confuse these distractions for deepfake manipulations, we explicitly described these distractions as not necessarily deepfakes in the instructions. In addition, we show several examples to recruited participants before the experiment begins. We do not show these examples to non-recruited participants because we wanted to increase engagement for people visiting the website on their own volition. We also included one attention check deepfake video and four videos of Vladimir Putin and Kim Jong-un (two real videos and two deepfakes each). All videos were 10 seconds long.

After completing the attention check, participants see a sequence of single videos that are randomly chosen from the remaining 54 videos and are randomly assigned to include a holistic obstruction manipulation.

We generate predictions for the computer vision model by running the competition winning computer vision model on the 54 sampled videos and on the 162 inversion, misalignment, and occlusion transformations of these videos [40].

We recruited 304 individuals from Prolific to participate in the pre-registered experiment. Participants were recruited from Prolific based on a nationally representative sample from the United States across age, ethnicity, and sex [91]. In addition, 9,188 non-recruited individuals participated. The sample size for the non-recruited sample was not pre-specified in the pre-analysis plan because we did not expect so many people to continue visiting the website. As such, we analyze the non-recruited participants in Experiment 2 in the spirit of the other two pre-analysis plans. Non-recruited participants visited from all over the world and the top 5 most represented countries were United States (31%), Brazil (9%), United Kingdom (7%), Canada (4%), and Germany (4%).

We present the design flow for Experiment 2 in Figure 3. Recruited participants begin at the start while non-recruited participants begin at the attention check video. All participants see an instruction modal when they first visit the website.

First, participants are randomly assigned to either an incidental anger elicitation intervention or a control exercise. These exercises are based on asking participants to respond to two questions displayed in Appendix Table 2, which have been shown to elicit incidental emotions in research experiments [68, 92].

Next, participants are shown five example videos to learn the difference between deepfakes and other distractions including both the flickering faces and the treatment interventions (inversion, misalignment, and occlusion).

Next, all participants in Experiment 2 see an attention check video where a man’s face clearly morphs from one configuration to another while he says, "This is a PSA. You can’t believe anything you see these days. These glasses aren’t even real. Neither is my face…"

After participants view the attention check video, they are shown a single video randomly drawn from the sample of 54 videos. Videos are never shown twice and participants see just as many deepfakes as real videos. 50% of videos are shown without a holistic processing obstruction. The other half are are shown upside down (1/6 of total observations), with the top and bottom half of the actor’s face misaligned (1/6 of total observations), and one sixth are shown with the eyes occluded by a thin black strip (1/6 of total observations). Recruited participants are asked to view 20 videos while non-recruited participants can view up to 45 videos.

We examine the causal effects of the four treatment assignments, $T_{x}$ on the accuracy score (normalized for correctness), $y_{i,j}$ of participant $i$ on manipulated video $j$. We include video fixed effects, $\mu_{j}$, and the following fixed-effects linear regression model reflects the pre-analysis plan with $\epsilon_{i,j}$ representing the error term:

We include video fixed effects to control for the differential accuracy across videos. We cluster errors at the individual level with Eicker-Huber-White robust standard errors because the emotion elicitation assignment is conducted at the participant level [90]. As specified in the pre-analysis plan, all analyses exclude participants who fail this attention check. Nevertheless, the results in these experiments remain qualitatively the same when including these participants.