MEWS: Real-time Social Media Manipulation Detection and Analysis

One of the most challenging aspects of online disinformation is the overwhelming volume of content published on social media platforms. Typical approaches to combating this problem rely on tracking headlines, news stories, hashtags, and accounts Ferrara (2017); Glenski et al. (2018). However, disinformation campaigns are becoming increasingly visual Theisen et al. (2020), but organizing and analyzing this volume of content in the hope of detecting disinformation campaigns in near real-time is impossible for humans without the assistance of automated tools. This problem is especially pertinent in young and struggling democracies whose traditional media organizations lack the ability to keep pace with the explosion of deep-fake, manipulated, altered, or plainly-fake online media Yankoski et al. (2021). To provide such capacity, we have developed a real-time social media manipulation detection and analysis system called MEWS (Misinformation Early Warning System) Yankoski et al. (2020). Figure 1 shows MEWS’ similarity graph with top images present (on the left) and a single image’s detail page (on the right).

This system combines work in digital forensics, computer vision, graph analysis, and media studies to accomplish three specific tasks:

1. 1.

   MEWS ingests enormous amounts of images and video from various social media platforms (e.g., Facebook, Instagram, Twitter, Telegram) using keyword targets provided by partner media organizations with context-specific knowledge domains from across the world;

2. 2.

   MEWS employs state-of-the-art AI systems to detect and extract faces Albiero et al. (2021); Guo et al. (2021), objects Carion et al. (2020), text (including meme-text) Lee and Smith (2012), image features Bay et al. (2006), and any potential manipulations Bianchi et al. (2011) from the visual content; and

3. 3.

   MEWS constructs a media-graph which pairs similar sub-images, objects, and manipulations for display in an interactive, easily-navigable, and searchable user interface.

For NeurIPS 2021 we demonstrated MEWS’ organizational and analytic capabilities using tens of millions of images (and other media) collected from several social media platforms (Facebook, Instagram, and Twitter) from within the Indonesian social-media context. In particular, this demonstration highlighted MEWS’ ability to:

1. 1.

   Interactively reveal emergent trends in social media images in (near) real-time.

2. 2.

   Identify media manipulations and alterations that recur across media items and platforms.

3. 3.

   Represent the relationships between social media posts on a variety of axes, including: meme-text, ancillary post-text (e.g., hashtags), detected objects, faces, and their identities, etc.

4. 4.

   Provide a searchable interface that users (i.e., newsrooms, civil society, government agencies, and others) can use to understand the way in which disinformation is spreading within online social media.

5. 5.

   Provide authenticated end-users the ability to upload media and visualize their relationships to other media contained in the MEWS dataset.

For example, MEWS extracted the manipulated/inserted finger-and-thumb motif from the image in \figurereffig:img2 (left) as well as the overlayed text and several other image features. In this particular instance, we find that the finger-and-thumb motif was frequently inserted to show support for a political candidate in Indonesia inauthentically. \figurereffig:img2 (right) shows another example of a cluster of altered images of an industrial accident that MEWS detected Theisen et al. (2020).

MEWS also identified an image manipulation aimed at political influence: \figurereffig:img3. A 2019 Presidential Candidate is pictured standing with another person whose shirt has been manipulated to display a hammer and sickle. The flag at the left in the image has also been manipulated with similar symbols. The heat map on the right is the algorithmic detection of the image’s manipulated portion(s) as identified by the MEWS system.

MEWS provided NeurIPS attendees an opportunity to witness several state-of-the-art technologies applied to a pertinent social problem. During the virtual conference, users were able to browse the existing collection of images through a Google Maps-like interface. They explored the faces, objects, alterations, text extractions, and information MEWS collected from each image.

In addition to demonstrating the combined power of several AI technologies, MEWS is one of the first image-based social listening and detection services, filling a wide gap in the study of social media, communications, and international security.

Disinformation campaigns will likely continue to influence social, political, and economic processes for the foreseeable future. We believe that these disinformation campaigns will becoming increasingly visual – taking the style of low-effort memes and out-of-context or cropped photography rather than sophisticated Deep-fakes.

MEWS provides an early-stage example of the application of AI technologies in the service of helping human users better navigate their social media networks. Despite its performance capabilities, MEWS is not intended to be a standalone solution. Rather, we envision MEWS as a tool for use by a robust partner network of fact-checkers, journalists, human rights watchers, and potentially government representatives who will use the information MEWS surfaces to identify and respond to disinformation threats more efficiently as they emerge on social media.

This work was supported by the US Agency for International Development (USAID) Cooperative Agreement number 7200AA18CA00059 and by the Defense Advanced Research Projects Agency and the Air Force Research Laboratory under agreement number FA8750-16-2-0173.