Surfacing contextual hate speech words within social media

Firstly, we must define our assumptions about hate speech and the role that context takes in our approach. Our goal is to obtain data from online hate speech communities, data which can be used to build models that create word representations of relatedness and similarity. We present our rationale for collecting data from online hate speech communities and explain the various types of context used throughout our methodology.

While there exists words or phrases that are known to be associated with hate speech²²2We used lists scraped from http://www.hatebase.org/ as used by Nobata et al. (Nobata et al., 2016)., it can often be expressed without any of these keywords. Additionally, it is difficult for human annotators to identify hate speech if they are not familiar with the meaning of words or any context that may surround the text as outlined in (Waseem and Hovy, 2016). These issues make it difficult to identify hate speech with Natural Language Processing [NLP] approaches. Further compounding the issue of hate speech detection, the members of these online communities have adopted strategies for bypassing the automatic detection systems that social networks employ. One such strategy being used is word substitution, where explicit hate speech words are replaced with benign words which have hidden meanings. Ultimately, the issue with code words is one of word polysemy and it is particularly difficult to address because these alternate meanings do not exist in the public lexicon.

To deal with the problem of code word substitution, we use word similarity and word relatedness features to train contextual representations of words that our model can use to identify possible hate speech usage. To do this, it is necessary to use models that align words into vector space in order to get the neighbours of a word under different uses. These models are referred to as Neural Embeddings and while most in the same fundamental way, the distinction comes from the input (hereafter referred to as context) that they make use of. We introduce topical context and functional context as key concepts that will influence our features.

Neural Embeddings refer to the various NLP techniques used for mapping words or phrases to dense vector representations that allow for efficient computation of semantic similarities of words. The idea is based on Distributional Hypothesis by Harris (Harris, 1981) which states that “words that appear in the same contexts share semantic meaning”, meaning that a word shares characteristics other words that are typically its neighbours in a sentence. Cosine similarity is the measure used for vector similarity, it will hereafter appear as $sim$. Neural Embedding models represent words in vector space. Given a target word $w$, an embedding model $\mathcal{E}$, it and a specified $topn$ value, it is possible to retrieve the $topn$ most $sim$ words in $\mathcal{E}$, $simByWord$ will be used to reference this function hereafter.

Topical Context is the context used by word embedding approaches like word2vec (Mikolov et al., 2013), that utilize a bag-of-words in an effort to rank words by their domain similarity. Context here is considered as the window for each word in a sentence, the task being to extract target words and their surrounding words (given a window size) to predict each context from its target word. In doing so it models word relatedness. However, functional context describes and ranks words by the syntactic relationships that a word participates in. Levy and Goldberg (Omer and Yoav, 2014) proposed a method of adapting word2vec to capture the Syntactic Dependencies in a sentence with dependency2vec. Intuitively, Syntactic Dependencies refers to the word relationships in a sentence. Such a model might tell us words close to Florida, words might be New York, Texas, California; words that reflect that Florida is a state in the United States. We simplify this with the term similarity, to indicate that words that share similar functional contexts are similar to each other.

Functional context is modelled by dependency2vec, a modification of word2vec proposed by Levy and Goldberg (Omer and Yoav, 2014) who build the intuition behind Syntactic Dependency Context. The goal of the model is to create learned vector representations which reveal words that are functionally similar and behave like each other, i.e., the model captures word similarity. dependency2vec operates in the same way as word2vec with the only difference being the representation of context. The advantage of this approach is that the model is then able to capture word relations that are outside of a linear window and can thus reduce the weighting of words that appear often in the same window as a target word but might not actually be related.

Topical context reflects words that associate with each other (relatedness) while functional context reflects words that behave like each other (similarity). In our work we wish answer the following: how do we capture the meaning of code words that we do not know the functional context of? To provide an intuitive understanding and motivation for the use of both topical and functional context we provide an example. Consider the following real document drawn from our data:

With the example we generate Table 1 which displays the top 4 words closest to the target word skypes, across two different datasets and word contexts. We assume the existence of embedding models trained on relatively clean text and another trained on text filled with hate speech references. For words under the relatedness columns, we see that they refer to internet companies. In this case, while we know that skypes is a hate speech code word it still appears alongside the internet company words because of the word substitution problem. We see the same effect for similarity under Clean Texts. However, when looking at similarity under the Hate Texts we can infer that the author is not using Skype in its usual form. The similarity columns gives us words that are functionally similar. We do not yet know what the results mean but anecdotally we see that the model returns groups of people and it is this type of result we wish to exploit in order to detect code words within our datasets. It is for this reason that we desire Neural Embedding models that can learn both word similarity and word relatedness. We propose that this can be used as an additional measure to identify unknown hate speech code words that are used in similar functional contexts to words that already have defined hate speech meanings.

The entire process consists of four main steps:

• •

  Identifying online hate speech communities

• •

  Creating Neural Embedding models that capture word relatedness and word similarity.

• •

  Using graph expansion and PageRank scores to bootstrap our initial HS seed words.

• •

  Enriching our bootstrapped words to learn out-of-dictionary terms that bare some hate speech relation and behave like code words.

The approach will demonstrate the effectiveness of our hate speech community detection process. Additionally, we will leverage existing research that confirmed the utility of using hate speech blacklists, syntactic features, and various neural embedding approaches. We provide a an overview of our community detection methodology, as well as the different types of word context, and how they can be utilised to identify possible code words.

A key part of our method concerns the data and the way in which it was collected and partitioned, as such it is important to first outline our method and rationale. There exists words that can take on a vastly different meanings depending on the way in which they are used, that is, they act as codewords under different circumstances. Collecting data from extremist communities which produce hate speech content is necessary to build this representation. There are communities of users on Twitter and elsewhere that share a high proportion of hate speech content amongst themselves and it is reasonable to expect that they would want to share writing or other content that they produced with like minded individuals. We are of the belief that new hate speech codewords are created by these communities and that if there was any place to build a dataset that reflects a “hate speech community” it would be at the source. We began the search by referencing the Extremist Files maintained by the Southern Poverty Law Center[SPLC]³³3https://www.splcenter.org/, a US non-profit legal advocacy organization that focuses on civil rights issues and litigation.

The SPLC keeps track of prominent extremist groups and individuals within the US, including several websites that are known to produce extremist and hate content, most prominent of these being DailyStormer⁴⁴4https://www.dailystormer.com/ and American Renaissance⁵⁵5https://www.amren.com/. The articles on these websites are of a White Supremacist nature and are filled with references that degrade and threaten non white groups, as such, it serves as an ideal starting point for our hate speech data collection. The two websites mentioned were selected as our seed and we crawled their articles, storing the author name, the article body, and its title. The list of authors was then used for a manual lookup in order to tie the article author to their Twitter account. We were not able to identify the profile of each author as some of the accounts in our list self identified as being pseudo-names. For each of these Twitter accounts we extracted their followers and friends, building an oriented graph where each vertex represents a user and edges represent a directional user-follower relationship. In order to discover authors that were missed during the initial pass, we use the centrality betweenness of different vertices to get prominent users. Due to preprocessing constraints we opted to compute an approximate betweenness centrality.

The computed betweenness centrality for every element in $V^{\prime}$ is then used to extrapolate the value of other nodes, as described in Brandes et.al (Brandes and Pich, 2007). From there, an extended seed of a specified size will be selected based on the approximated centrality of the nodes and the original author. With this extended seed, it becomes possible to collect any user-follower relationships that were initially missed. After the initial graph processing, over 3 million unique users IDs were obtained. A random subset of vertices was then taken to reduce the size of the graph for computational considerations. This random subset forms a graph $\mathcal{G}$ containing $V_{f}$ vertices, $|V_{f}|\approx 20000$. Each vertex of $\mathcal{G}$ represents a user while directed edges represent relationships. Consider $s,t\in V$, if $s$ is following $t$ then a directed edge $(s,t)$ will exists. Historical tweet data was collected from these vertices, representing over 36 millions tweets. We hereafter refer to graph $\mathcal{G}$ as HateComm, our dataset which consists of the article content and titles previously mentioned in addition to the historical tweets of users within the network of author followers.

The issue with code words is that they are by definition secret or at best, not well known. Continuing with the examples of Skype and Google we previously introduced, if we were to attempt to get related or even similar words from a Neural Embedding model trained on generalized data, it is unlikely that we would observe any other words that share some relation to hate speech. However, it is not enough to train models on data that is dense with hate speech. The results might highlight a relation to hate speech but would provide no information on the frequency of use in different situations, in short, we need to have some measure of the use of a word in the general English vocabulary in order to support the claim that these words can also act as hate speech code words. It is for this reason that we propose a model that includes word similarity, relatedness and frequency of use, drawn from the differing datasets. We therefore introduce two additional datasets that we collect from Twitter, the first using hate speech keywords and the second collected from the Twitter stream without any search terms. Twitter offers a free 1% sample of the total tweets sent on the platform and so we consider tweets collected in this manner to be a best effort representation of the average.

Hate Speech Keywords is is defined as a set of words $\mathcal{H}$ = $\left\{\text{$h_{1},..,h_{n}$}\right\}$ typically associated with hate speech in the English language. We made use of the same word source as (Nobata et al., 2016). TwitterHate refers to our dataset of tweets collected using $\mathcal{H}$ as seed words. While TwitterClean refers to our dataset collected without tracking any specific terms or users, only collecting what Twitter returned, free from the bias of collecting data based on keywords. We filter and remove any tweet that contains a word $w\in$ $\mathcal{H}$.

For our work we dynamically generate contextual word representations which we use for determining if a word acts as a hate speech code word or not. To create contextual word representations we use the Neural Embedding models proposed by dependency2vec(Omer and Yoav, 2014) and fasttext (Bojanowski et al., 2016). As we wish to identify out-of-dictionary words that can be linked to hate speech under a given context, as part of our preprocessing we we then define a graph based approach to reduce the word search space. Finally, our method for highlighting candidate code words is presented. We report our code words as well as the strength of the relationship that they may have to hate speech.

In order to partition our data and train our Neural Embeddings we first collected data from Twitter. Both TwitterClean and TwitterHate are composites of data collected over several time frames, including the two week window leading up to the 2016 US Presidential Elections, the 2017 US Presidential Inauguration, and at other points during early 2017, consisting of around 10M tweets each. In order to create HateComm we crawled the websites obtained from the SPLC as mentioned in Section 4.2 and obtained a list of authors and attempted to link them to their Twitter profiles. This process yielded 18 unique profiles from which we collected their followers and built a graph of user:followers. We then randomly selected 20,000 vertices and collected their historical tweets, yielding around 400K tweets. HateComm thus consists of tweets and the article contents that were collected during the scraping stage.

We normalize user mentions as user_mention, preserve hashtags and emoji, and lowercase text. The tokenizer built for Twitter in the Tweet NLP⁶⁶6http://www.cs.cmu.edu/ ark/TweetNLP/ package was used. It should be noted that the Neural Embeddings required a separate preprocessing stage, for that we used the NLP package Spacy⁷⁷7https://spacy.io/ to extract syntactic dependency features.

As mentioned previously, we utilized fasttext and dependency2vec to train our Neural Embeddings. For our Dependency Embeddings we used 200 dimension vectors and for fasttext we utilized 300.

To initialize our list of seed words for our approach we built a contextual graph with the following settings.

1. (1)

   $\mathcal{D}_{H}$was used to build a $\mathcal{CG}$ based on $\mathcal{H}$ similarity

2. (2)

   to generate boost we set $topn=20$

3. (3)

   We consider singular and plural variations of each $w\in$ $\mathcal{H}$

4. (4)

   Vertices were added with $topn=3$ and $depth=2$

This process for expanding our $\mathcal{H}$ seed returned $994$ words after trimming with the frequency rationale. For the contextual code word search we used the following:

1. (1)

   $depth=2$, $topn=5$, $th=0.2$

We set $th=0.2$ after experiments showed that most words did not return more than 1 known hate speech keyword when checking its 5 closest words. This process return 55 primary and 262 secondary bucket words. It should be noted that we filtered for known $\mathcal{H}$ including any singular or plural variations. An initial manual examination of this list gave the impression that while the words were not directly linked to hate speech, the intent could be inferred under certain circumstances. It was not enough the do a manual evaluation as we needed a way to verify if the words we had surfaced could be recognized as being linked to hate speech under the right context. We saw fit to design an experiment to test our results.

The major difficulty of our work has been choosing a method to evaluate our results as their are few direct analogues. As our baseline benchmark we calculate the tf-idf word scores for HateComm and compare with frequencies for our surfaced code words. Using the tf-idf scores is a common approach for discovering the ideas present in a corpus. Where higher tf-idf scores indicate a higher weight, due to low frequency use of our code words lower scores represent a higher weight. For the code word weights we use inverse document frequency. Figures 4 and 5 show the difference between the TF-IDF baseline and our contextual code word search. The TF-IDF output appears to be of a topical nature, particularly politics while the code word output features multiple derogatory references throughout.

The contextual code word approach is not without drawbacks, as ultimately these words are a suggestion of possible hate speech code words. However, it represents an improvement over attempting to find these hate speech code words manually as the model can learn new hate speech code words as they are introduced.

We have claimed throughout or work that context is important and we designed an experiment to reflect that. Our aim was to determine if a selection of annotators would be able to identify when a given word was being used in a hate speech context without the presence of known hate speech keyword and without known the meaning of the code words. The experiment featured manually selected code words including 1 positive and 1 negative control word. It is important to have some measure of control as many different works including  (Waseem, 2016) have highlighted the difficulty of annotating hate speech. The positive and negative samples were designed to test if annotators could identify documents that featured explicit hate speech (positive) and documents that were benign (negative).

We built three distinct experiments where:

1. (1)

   Documents refer to tweets and article titles.

2. (2)

   10 code words were manually selected and participants were asked to rate a document on a scale of very unlikely (no references to hate speech) to very likely (hate speech) [0 to 4].

3. (3)

   HateCommunity, TwitterClean, and TwitterHate were utilized as the sample pool, randomly drawing 5 documents for each code word (10 word X 5 documents for each experiment).

4. (4)

   Control documents were the same across all three experiments and did not feature known HS words apart from the positive control.

5. (5)

   Direct links were only provided for the experiments drawn from HateCommunity and TwitterClean. After completing these experiments, participants were given the option to move on to the TwitterHate experiment.

The experiment was designed to draw for our distinct datasets which would reflect the use of the same word across differing situations and contexts. We obtained 52, 53, and 45 responses for HateCommunity, TwitterClean, and TwitterHate respectively. The full list can be seen in Table 3. Table 4 provides a view of a few of the documents annotators were asked to rate. None of the examples features known $\mathcal{H}$ but it is possible to infer the intent of the original author. The experiment also featured control questions designed to test if participants understood the experiment, we provided 5 samples that featured the use of the word nigger as positive for hate speech and water as negative for hate speech. An overwhelming majority of the were able to correctly rate both control questions, as can be seen in Figs. 6.

To get an understanding of the quality of the data and to facilitate further experiments, we created a ground truth result and aggregated the annotators based on their majority ratings. We first calculated inter annotator agreement with Krippendorff’s Alpha which is a statistical measure of agreement that can account for ordinal data. With the majority rankings, we recorded $K=0.871$, $K=0.676$ and $K=0.807$ for HateComm , TwitterClean and TwitterHate . The experiment demonstrated the importance of data sources as the sentences sampled from each data set were at times in stark contrast to each other, reflecting the advantage of the extremist community data over the traditional keyword collection method. We achieved the highest agreement scores for hate speech when using the extremist community dataset.

We then moved to calculate precision and recall scores. As we used a likert scale for our ratings, we took ratings that were above the neutral point (2) to as hate speech and ratings below as not hate speech. Interestingly, when taking the majority none of the questions featured the Neutral label as the majority. Our aim for this experiment was to determine if the ratings of the annotator group would reflect hate speech classification when aggregated. The results below show the classification across all 10 $\times$ 5 documents for each experiment (the matrix sums to 10). The Precision, Recall, and F1 scores can be seen in Table 5 which shows the F1 scores of 0.93 and 0.86 for HateComm and TwitterClean respectively. This result indicates that the annotators were able to correctly classify the usage of the same word under different contexts, from data that is dense in hate speech and data that reflects the general Twitter sample. The scores show that when taking the annotators as a single group that they were in line with the ground truth. This gives supports to our claim that it is possible in some cases to infer hate speech intent without the presence or absence of specific words.

One of the ideas that we wanted to verify in the experiment was whether the rankings of the annotators would align with the ground truth. We include the ranking distribution for the HateComm experiment results in Table 6. The results compare the majority ranking for each word as well as the percentage against the ground truth.