The Challenge of Online Abuse Detection; Breaking the Back of the Beast

Context
Thilini Wijesiriwardene

 How many conversations did you have this week? How many of them were face to face conversations with someone across from you or someone on telephone/VoIP? And most importantly how many of you were caught up in the reply comment loop of social media? With the dawn of the internet and social media, we have acquired the ability to maintain conversations with vast audiences or put forth opinions for the whole world to see. Twitter is a social networking site which was founded back in 2006 by Jack Dorsey. From the first tweet tweeted by him in 2006, Twitter has come a long way. Today Twitter has become many peoples' go-to platform for online scanning of news and gossip (isn't this what most of us do nowadays online; we prefer to scan few sentences rather than read a lengthier article)

Apart from this, Twitter has given all of us direct access to lives of many figures deemed influential and famous in a myriad of fields ranging from entertainment to religion to politics. You can know what your president thinks about a pressing issue and how other vital figures respond to his/her outlook. This may seem like your personalized grapevine. Our opinions can be less censored and more relaxed. We could, if we wish, address thousands of followers instantly. For famous figures, it is an appealing route to bypass the traditional media and reach their fans. As delightful as it seems; this path of direct access to one's perspectives can sometimes result in disaster than in benefit. Am I not entitled to rant on Twitter? Can't I express my honest opinion on "my" twitter account? Aren't we allowed to have disagreements/ different views? All of these questions point out to the genuinely peculiar and dangerous nature of abuse and harassment on Twitter.

Be a celebrity or ordinary citizen, harassment and abuse on Twitter have reached and affected the lives of many users. Why is Twitter still struggling to bring this issue under control? The simple answer is that we are incredibly nuanced in our expressions; in this time of technology and connectivity, we can create and understand new cultural contexts more so than ever before. Filtering Twitter streams by offensive or profane word lists to identify hate and abuse seems less effective and counterproductive because of this ability of ours. Censoring tweets based on the use of words that are classified as offensive or harassing does not necessarily cut it, and it could anger the users who use such words in their regular conversations.

One of the most recent examples that depict the difficulty of online abuse detection sprout from the controversial tweet from Roseanne Barr, a renowned American comedian. She has posted a racially charged tweet about Valerie Jarrett, the former senior advisor to the president Barack Obama, who is an African-American woman. Barr's tweet did not include any offensive or profane words per se, but instantaneously got picked up by followers and interested parties as racist and eventually got her sitcom canceled by ABC. Following is Barr's tweet:

“if the muslim brotherhood & planet of the apes had a baby=vj”

As humans, we connect millions of concepts with each other in our minds, and these are colored with a multitude of cultural contexts. We are extremely quick to understand the connotations of discourse. That is how this tweet got picked up as racist by humans. Can the same filtering be achieved by algorithms? As of today, no algorithm comes close to the human brain in understanding the overtones of human communication, and no algorithms possess the context including socio-cultural, linguistics and other dimensions, that humans are equipped with. But it gives us hope to know that we would be able to make the algorithms more "aware" of the context. What if there were knowledge graphs that connect the concept of the planet of the apes with apes and apes with the concept of racial stereotyping of African Americans? It would make the algorithms less inept in "understanding" the context. If Valerie Jarrett is an entity in the same graph with links pointing to the political nature of the position held by her, it would also be helpful for an algorithm to put things in perspective.

As hopeful as it seems, it would be naive to believe that encoding world knowledge in knowledge graphs and similar structures is the only answer to the question of providing context to algorithms. There are rich and, often overlooked content that is embedded in the metadata of online conversations. What are these content that an algorithm might be able to glean context from? When a message is posted on Twitter, the audience has many ways to react to the tweet. They could comment on it or/and they could retweet it (as it is or with minor changes introduced by themselves). This reaction oriented behavior of the audience can be leveraged to identify "controversial" content. In this case, the tweet of Roseanne Barr might be picked up by an algorithm if it were to be "taught" to monitor "controversial" content. From the number of replies started to pour in within short time intervals and from the amount of retweets with minor changes (mostly disapproving the original racially charged tweet) an algorithm would be able to "learn" that the original tweet is controversial and might need to be flagged for more thorough inspection - may be by a human.

Above mentioned approach also emphasizes another critical fact; tweets are rarely by themselves. Therefore if you try to evaluate the abusive/ harassing nature of tweets by only looking at single tweets, and not looking at the entire conversation (if available), it would not be as effective. Looking at the conversation could reveal the type of the relationship between the parties engaged in the conversation; friends could be using certain flagged words in a playful manner without the intention to harass, but strangers who use similar words might actually have a harassing intention. Nature of the conversation; whether it is a heated argument or a casual conversation also can provide context. Therefore it is fruitful to look at the entire conversations, not at individual tweets taken out of their context.

As social creatures, in every social situation both online and offline, we tend to release specific markers that can be picked up and analyzed. It is essential to identify and look at the markers that are released in online interactions because today most of our online decisions, experiences, and secrets tend to increasingly influence our offline lives; Online advertisements are identified to change our votes offline, online harassment tends to lead to dire results offline, etc. Believing that an algorithm one day would be able to "understand" the undertones of communication as humans and also to be equipped with context as humans seem far-fetched. But as researchers, it would be worthwhile and exciting to try and leverage these markers and context to make algorithms more capable.

Implicit Information Extraction

Implicit entity

People communicate their ideas, opinions, and facts using natural language. It is one of the powerful tools that we as humans collectively developed over hundreds of thousands of years and will continue to develop in years to come. While the debate on the evolution of the languages has not reached a consensus, the theories have estimated that it first evolved around 150,000 - 350,000 years ago, which is roughly the time frame accepted for the evolution of modern Homo sapiens.1 English is one of the most used descendants of this evolution and it has evolved over 1,400 years on its own.2 The evolution of the language accounts for the social, cultural, and economic changes which have taken place in society. For example, the industrial revolution took place in the 18th and 19th century and had a great impact on these three dimensions. It added new words to the language such as condenser, vacuum, reservoir, taxonomy, sodium, and platinum [1].

The evolution of the language has enriched it with many features. One such feature is its ability to express facts, ideas, and opinions in an implicit manner. As humans, we seamlessly use implicit constructs in our daily conversations and rarely find it difficult to decode the content of the messages. Consider the following tweet:
This tweet contains an implicit mention of the movie 'Gravity'. A human with up-to-date knowledge on movies would instantly understand that the tweet talks about the movie 'Gravity'. However, the whole field of information extraction, which has the objective of automatically extracting structured information from unstructured and/or semi-structured data, almost exclusively focused on extracting explicit information from the text. Consider the following text snippet extracted from a clinical narrative.

"Bob Smith is a 61-year-old man referred by Dr. Davis for outpatient cardiac catheterization because of a positive exercise tolerance test. Recently, he started to have left shoulder twinges and tingling in his hands.  A stress test done on 2013-06-02 revealed that the patient exercised for 6 1/2 minutes, stopped due to fatigue. However, Mr. Smith is comfortably breathing in room air. He also showed accumulation of fluid in his extremities. He does not have any chest pain."

The state-of-the-art information extraction algorithms would extract information like 'Bob Smith' and 'Dr. Davis' are entities of type person, 'cardiac catheterization' and 'chest pain' are the same as entities identified with concept unique identifiers (CUI) C0018795 and C0008031 in unified medical language system (UMLS), and there is a cause-effect relationship between 'fatigue' and exercising. The algorithms developed for named entity recognition, entity linking, and relationship extraction would help to extract this structured information from the above text snippet. However, the two sentences "Mr. Smith is comfortably breathing in room air" and "He also showed accumulation of fluid in his extremities" implicitly indicate that the patient does not have the clinical condition named 'shortness of breath', but he has 'edema.' This is very important information for assessing the health status of the patient and a medical professional reading this snippet would easily decode the mentions of these two clinical conditions in the text. An automatic entity linking technique should identify these mentions as same as the entities identified with CUIs C0013404 ('shortness of breath') and C0013604 ('edema') in UMLS. Commercially, it has applications such as Computer Assisted Coding and Computerized Document Improvement. Unfortunately, current information extraction algorithms would not be able to extract this implicit information.

Implicit constructs are not a rare occurrence. Our studies found that 21% of the movie mentions and 40% of the book mentions are implicit in tweets, and about 35% and 40% of 'edema' and 'shortness of breath' mentions are implicit in clinical narratives. There are genuine reasons why people tend to use implicit mentions in daily conversations. Here are few reasons that we have observed:
  1. To express sentiment and sarcasm : The following tweet has an element of sarcasm and a negative sentiment towards the movie 'Transformers: Age of Extinction.' These feelings were expressed implicitly in this tweet. It is  noted that people heavily use implicit constructs to express sarcasm [2].
  2. To provide descriptive information : For example, it is a common practice to describe the features of an entity rather than simply list down its name in clinical narratives. Consider the sentence "small fluid adjacent to the gallbladder with gallstones which may represent inflammation." This sentence contains an implicit mention of the clinical condition 'cholecystitis' and provides important information about the patient's health status that would be missing if the author chose to list down only the name of clinical condition. The condition 'cholecystitis' means "inflammation in gallbladder" with multiple causes and the sentence provides a detailed explanation of 'cholecystitis' along with the possible cause. This descriptive information is critical in understanding the patient's health status and treating the patient.
  3. To emphasize the features of an entity : Sometimes we replace the name of the entity with its special characteristics in order to give importance to those characteristics. For example, the text snippet "Mason Evans 12 year long shoot won big in golden globe" has an implicit mention of the movie 'Boyhood.' There is a difference between this text snippet and its alternative form "Boyhood won big in golden globe." The speaker is interested in emphasizing the distinct feature of the movie, which would have been ignored if he had used the name of the movie as in the second phrase.
  4. To communicate shared understanding : We do not bother spelling out everything when we know that the other person has enough background knowledge to understand the message conveyed. A good example is the fact that clinical narratives rarely mention the relationships between entities explicitly (e.g., relationships between symptoms and disorders, relationships between medications and disorders), rather it is understood that the other professionals reading the document have the expertise to understand such implicit relationships in the document.

 The above examples show the value added by the implicit constructs to daily communications. Another important observation is the role of world knowledge in interpreting implicit constructs. A human reading the text with implicit information would only be able to decode implicit information if he/she has relevant knowledge on the domain. A reader who does not know about Michael Bay's movie release would have no clue about the movie mentioned in the tweet with sarcasm; a reader who does not know the characteristics of the clinical conditions 'shortness of breath' and 'edema' would not be able to decode their mentions in the clinical text snippet shown above; a reader who is not a medical expert would not be able to connect the diseases and symptoms mentioned in a clinical narrative.

The implicit information extraction task demands comprehensive and up-to-date world knowledge. Individuals resort to a diverse set of entity characteristics to make implicit references (also see [3]). For example, the implicit references to the movie 'Boyhood' use phrases like "Richard Linklater movie", "Ellar Coltrane on his 12-year movie role", "12-year long movie shoot", "latest movie shot in my city Houston", and "Mason Evan's childhood movie." Hence, it is important to have comprehensive knowledge about the entities to decode their implicit mentions. Another complexity is the temporal relevancy of the knowledge. The same phrase can be used to implicitly refer to different entities at different time intervals. For instance, the phrase "space movie" could refer to the movie 'Gravity' in fall 2013 while the same phrase in fall 2015 would likely refer to the movie 'The Martian.' On the flip side, the most salient characteristics of the movies may change over time, and so will the phrases used to refer to them. The movie 'Furious 7' was frequently referred to with the phrase "Paul Walker's last movie" in November 2014. This was due to the actor's death around that time. However, after the movie release in April 2015 the same entity was often mentioned through the phrase "fastest film to reach the $1 billion."


At Kno.e.sis, we have developed a knowledge-driven solution to perform implicit information extraction. This solution acquires relevant domain knowledge from a diverse set of structured and unstructured knowledge sources, processes acquired knowledge to represent it in a machine readable manner, and contains information extraction techniques that uses these knowledge sources to decode the implicit information in the text. We have successfully applied this solution to extract implicit entities and relationships in clinical narratives [4] [6] and implicit entities in tweets [5].


References:
[1] Bragg, Melvyn. The adventure of English: The biography of a language. Arcade Publishing, 2006.
[2] Davidov, Dmitry, Oren Tsur, and Ari Rappoport. "Semi-supervised recognition of sarcastic sentences in twitter and amazon." Proceedings of the fourteenth conference on computational natural language learning. Association for Computational Linguistics, 2010.
[3] "Help For HealthCare: Mapping Unstructured Clinical Notes To ICD-10 Coding Schemes." Http://www.dataversity.net/. N.p., 26 Nov. 2013. Web. 19 Aug. 2016.
[4] Sujan Perera, Pablo N. Mendes, Amit Sheth, Krishnaprasad Thirunarayan, Adarsh Alex, Christopher Heid, and Greg Mott. "Implicit entity recognition in clinical documents." In Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics (*SEM), pp. 228-238. 2015.
[5] Sujan Perera, Pablo N. Mendes, Adarsh Alex, Amit P. Sheth, and Krishnaprasad Thirunarayan. "Implicit Entity Linking in Tweets." In Extended Semantic Web Conference, pp. 118-132. Springer International Publishing, 2016.
[6] Sujan Perera, Cory Henson, Krishnaprasad Thirunarayan, Amit Sheth, and Suhas Nair. "Semantics driven approach for knowledge acquisition from EMRs." IEEE journal of biomedical and health informatics 18, no. 2 (2014): 515-524.