Bots in the Election

4:15:00 PM
by Unknown

In the Kno.e.sis Center at Wright State University, we continue to refine our Twitris technology (Licensed by Cognovi Labs LLC) for collective social intelligence to analyze social media (esp. Twitter) in real time. Kno.e.sis and Cognovi Labs teamed up with the Applied Policy Research Institute (APRI) early in the year and created some tools to monitor the debates. See press coverage on TechCrunch. From the time that we first began following the nominees on Twitter, one thing became clear: Donald Trump was considerably more popular than his competition during the primaries as well as the general election. To be honest, I had never considered the possibility that social bots may have been playing a role in this popularity.


After the conclusion of the first debate all parties who had watched our "Debate Dashboard" were shocked not just by the volume of tweets but by the sentiment and emotion that appeared more positive for Trump than Clinton. When we investigated news from major media outlets, we became more and more concerned that our tool had some serious flaws. Due to articles we had seen discussing the large support Trump had on Twitter, we decided to focus on sentiment. Up until a few days before the election, we continued to update and improve our sentiment analysis algorithm.   


Notwithstanding the improvements in precision to our sentiment classifier, we continued to see Trump as the clear leader. As the debates came and went, our data remained consistent. We began an urgent quest for reason. We added gender analysis because media outlets were telling us that women were down on Trump and would be a major force in the election. Our analysis did not show this, despite having 96% precision in determining female and male users. We developed a proprietary process to separate users into left-leaning and right-leaning users. We could even say whether a user was strongly or loosely associated with a particular political party. Unfortunately, analyzing the data based on political association didn't help either. Surprisingly many strongly left leaning users were anti-Hillary, just a bit behind the right-leaning users.


After the second debate, we began to see many articles pop up about social bots. Once we began to look more into the issue, we found many articles from early in the year talking about Trump's "Bot Army" (Trumps' Biggest Lie? The Size of His Twitter Following). We had our aha moment. In that article, there is a reference to The Atlantic's use of a tool called BotOrNot. We decided to attempt to use this or some similar tool during the last debate to remove users with bot accounts and analyze the remaining data.


BotOrNot is a tool developed at the University of Indiana, Bloomington with collaboration from the University of Southern California, Marina Del Rey. Their tool creates over one thousand metrics by looking at the user account and analyzing retweets, hashtags, metadata, etc. The tool performed extremely well in the DARPA Twitter Bot Challenge, having correctly identifying all of the known bots (though it did incorrectly mark some additional users as bots). We were excited to learn that they had made their tool available through an API endpoint and decided to run our tweets through the system to test the speed at which we could process users. Twitris at this point was processing nearly 35 tweets per second for the election analysis alone, and it was clear very quickly that their service would not be able to handle the volume of data we would be consuming.


Though the final presidential debate was only several days away, we still had hope that we would find an answer. We saw Prof. Philip Howard, from the University of Oxford, mention in The Washington Post that they considered any user that tweets more than 50 times in one day a bot. It would be relatively simple to create an index of users and simply increment the count per tweet and then check the index quickly as the tweets roll through. We may have done this if our team had been free at the time. Some were working on bug fixes, others on improving sentiment, and still others were working to fix some infrastructure issues that we were experiencing at the time. Our corpus of tweets for the election campaign was on its way to exceeding 60 million tweets. A robust implementation would have required more time than anyone had to offer.


I suppose that now is a better time than never to define what a "bot" is. Phil Howard says, in that Washington Post article mentioned above, that "A Twitter bot is nothing more than a program that automatically posts messages to Twitter and/or auto-retweets the messages of others". I personally like this definition, but it leaves a little wiggle room where tools like BotOrNot, which focus primarily on the user accounts, are considered. A Twitter user can be a real, living, breathing, human being and still exhibit bot-like tweeting habits. I think that this is the reason that Howard's group settled on the 50-tweets-per-day stat instead of relying on a classifier. A user can tweet for themselves part of the time, but still have some automation process that tweets certain things on their behalf. There are many reasons that someone would do this, to increase their Klout score (imagine a LuLaRoe seller, YouTuber, or Blogger), for example. There are many companies that you can pay to automate this kind of activity for you. Some, like Linkis' "Convey" (more on this later) work by finding influential tweets and tweeting them on your behalf. These tweets are fairly easy to spot because they attach "via @c0nvey" to the end of the original tweet.


In the end, what we did was to develop a system that is able to quickly and accurately weed out tweets that were not authored by humans, even if the user account is owned by an actual human. Let's see how well our system stacks up to the BotOrNot service. We collected all of the bots found over one fifteen minute period and ran them through BotOrNot. 67.16% of the accounts from the tweets we labeled were determined by BotOrNot to be “bot” owned accounts. I was a little disappointed by this, so I took a look at the users that BotOrNot dismissed as human. In the first pass, I decided to apply Howard's 50-tweets-per-day rule. Doing this increased the percentage of accurately labeled bot tweets to 73.88%. One of the screen names found among this group actually contained the word "bot" in it. The most important thing for us at this point was to make sure that we weren’t getting a lot of false positives. We looked at each account one by one; we looked at each tweet in our system, both those classified as "likely bot" and "likely human".


 
Figure: Real-time labeling of bot and human tweets in Twitris




Many of the users with a low tweet-per-day usage had a combination of the two, and looking at the tweets that were labeled "likely bot", nearly all of them contained "via @c0nvey". Digging a bit further, I found that the Convey service has been accused of tweeting on behalf of unwitting users in the past. Here's a thread from one Twitter user talking about his experience:
   c0nvey.jpg


So, our system is able to accurately detect automated (bot) tweets despite the fact that the user is not a bot. Our average detection rate also looks reasonable compared to other services. We experience bot traffic at a rate of about a 5% per day in our election campaign, though there are days that get to nearly 8%.


Moving Forward (oh, and Fake News)


Elections come and go. Once they are gone, we seek to apply our findings to future work. Being able to detect bots is great; but, what else can we do with that? Well, post-election we have learned the term “fake news” (with content that was entirely made up, not grounded in truth or reality), something that not too many people were concerned about before: see the Google Trend.
fakenews.jpg
Where does all of this fake news come from? There have been troves of news reports blaming Facebook and Twitter for altering the outcome of the election. Obviously, Facebook and Twitter themselves weren’t creating pro-Trump news (Facebook in particular was accused of killing pro-Trump trends); however, some blame these companies for not alerting people that the news that they were “allowing” to spread was fake. I don’t think that that is fair. These companies are reliant on the fact that they don’t publish news (for more, read this article on Facebook, the “News Feed”, and the Communications Decency Act 230) to avoid lawsuits.


During the evaluation of our bot detection system, we noticed something interesting. A large majority of the bot-labeled tweets contained links to dubious looking news stories. Because of our ability to identify these bot tweets, we can exclude them from analysis when considering a brand-centered campaign (like Samsung during the Note 7 battery “situation”). I think that it is important to note that, despite the elimination of “fake news” and “bot tweets” from our analysis, Trump was always winning. We saw the same thing with the Brexit referendum earlier this year, where Twitris helped us to correctly predict the Brexit outcome before the polls closed. There is clear evidence to support the fact that high tweet volume translates to success (except Bernie Sanders, but there may be, uh-hum, other reasons for that). It seems to me that for bots and “fake news” to have swayed the election, they would have needed to be ready to go as soon as Trump announced that he was running, but what we have seen is that he was always ahead.


We will continue to find new ways to leverage everything that we learned from the 2016 Election. If you want to stay up to date on our analysis, please sign up to receive CognoviLabs’ newsletter at www.CognoviLabs.com or join Kno.e.sis on FB, and while you are there check the other post-election analysis.

You Might Also Like

22 comments

  1. M88 Casino Review 2021 | Up to C$100 Welcome Bonus - thakasino
    M88 casino welcome bonus | 100% first deposit bonus up to C$100 | 100% first deposit bonus up to C$100 | m88 ทางเข้า 100% 우리카지노 first 188bet deposit bonus up to C$100 | 100% first deposit bonus up

    ReplyDelete
  2. Hello, nice to read your all blogs and learned a ot of new knowledge. Thank you! 토토사이트

    ReplyDelete
  3. You’re doing a great job, keep it up doing a great work like this one. 파워볼사이트닷컴

    ReplyDelete
  4. I am always searching online for articles that can help me. I think you made some good points. Keep working, great job ! 바카라

    ReplyDelete
  5. I'm happy to see your website for valuable information.Why didn't this accident happen earlier? I was surprised!For me, it's bookmarked.

    온라인카지노
    메이저토토사이트
    토토사이트

    ReplyDelete
  6. I just want to say that your article is amazing. The clarity of your pretense is truly wonderful, and I can guess you are an expert on this subject. If you don't mind, I'll get an RSS feed and update you on future posts. Thank you very much.Please continue to enjoy your work.

    스포츠토토
    토토
    바카라사이트

    ReplyDelete
  7. Excellent information on your blog, thank you for taking the time to share with us. 슬롯머신

    ReplyDelete
  8. I blog often ɑnd I sеriously apⲣreciate yօur c᧐ntent.
    Yߋur article һaѕ truly peaked my іnterest. I am ցoing to book mark
    your site and keep checking ffor neѡ details аbout once a wеek.
    I subscribed to ʏоur RSS feed tοo.

    My webpage :: https://www.casinoholic.info 바카라사이트

    ReplyDelete
  9. Everything is very open with a very clear explanation. sportstoto It was definitely informative. Your site is very useful.
    Thank you for sharing!

    스포츠토토
    파워볼게임
    안전놀이터

    ReplyDelete
  10. All together, they make up for 21% of casino table game action. Poker is one of the|is among 카지노사이트 the|is doubtless one of the} most regularly played card video games, and it is probably the most famous of all of them, from a mediatic viewpoint, and in addition typically speaking. There are dozens of various variations of this game, probably the most famous of which are Texas hold 'em and the classic five-card draw poker.

    ReplyDelete
  11. I LOVE THIS POST THAT YOU CREATED.
    THANK YOU!****

    ReplyDelete
  12. ~~~THIS IS A GREAT INFORMATION, THANKS FOR THIS BLOG!

    ReplyDelete
  13. ****THIS IS GREAT BLOG FOR TODAY THAT I READ, THANKS FOR THIS!

    ReplyDelete
  14. THANKS FOR THIS EXCELLENT INFORMATION THAT YOU CREATED.----

    ReplyDelete
  15. Thank you for sharing blog comments on this website.We offer a variety of research and educational programs that are designed to help students and professionals develop the skills and knowledge they need to succeed in the digital age. Our research centers are focused on addressing some of the most pressing challenges facing organizations today.keep updates more.
    contract dispute lawyers
    contract dispute attorney

    ReplyDelete
  16. This comment has been removed by the author.

    ReplyDelete
  17. Bots, short for "robots" or automated software programs, have become a topic of concern and discussion in the context of elections and political campaigns. Bots can be used for various purposes during an election, and their impact can vary depending on how they are employed. Here are some key points to consider regarding bots in elections: DUI Lawyer Lunenburg VA

    ReplyDelete
  18. Bots, an acronym for "robots" or automated software programs, have gained attention and generated debate in relation to political campaigns and elections. During an election, bots can be used for a variety of things, and the results will depend on how they are used. The following are important things to think about when it comes to election bots:protective orders in virginia

    ReplyDelete
  19. This comment has been removed by the author.

    ReplyDelete
  20. Thanks for this wonderful post

    The collaborative divorce expert's expertise lies in their ability to view divorce as a transformational process rather than solely a legal procedure. How to Divorce in New York

    ReplyDelete