Detecting fake social media accounts a matter of national importance

It can be nearly impossible to tell the difference between computer-generated and human-generated content online due to significant development in the ability of computers to write text that mimics human language. The advancement has made it more difficult to identify bots, or fake accounts, on social media and poses a problem as individuals, governments and corporations can use bots to influence public opinion on everything from voting to shopping habits.

In order to more accurately detect bots, Toronto Metropolitan University (TMU) math professor Pawel Pralat has designed an embedding algorithm that explores the networks around fake accounts, looking at which accounts they follow and interact with, instead of simply analyzing the content bots produce.

The embedding research is part of a collaboration with Patagona Technologies, a company founded by two of professor Pralat’s former students, TMU alumni Andrei Betlen and David Miller. They asked professor Pralat to partner with them on a cybersecurity project for the Department of National Defence. The project aims to detect hostile influencers who spread disinformation on social networks.

“The common belief now is that if we cannot distinguish real accounts from fake accounts through the texts they write, maybe we can look at the network around them,” said professor Pralat. “If we combine the classical machine learning tools looking at the text, but also look at the network around bots and around legitimate users, the differences between networks around bots and networks around real users can be used to identify them.”

Professor Pralat’s research team analyzed two Twitter datasets. One dataset explores follower/following relationships and the other looks at interactions between users.

“We want to explore the online neighborhood of each user and what kind of accounts they interact with,” said professor Pralat. He explains the idea is that there will be an identifiable difference in how bots and humans interact with others on social networks like Twitter.

These interactions are used to embed users in a high-dimensional space with the hope that similar types of users would be placed in similar positions in that space, making it possible to identify the bots.

The embedding algorithm the team developed is very promising, says professor Pralat, who adds it is still a work in progress but that they can already detect bots with much higher accuracy than existing methods for bot detection.

The research team included TMU PhD student Ash Dehghan, Warsaw School of Economics professor Bogumil Kaminski and three visiting PhD and graduate students from Poland and India. One of the four research papers based on this research project was presented at DATA 2022 11th International Conference on Data Science, Technology and Applications, winning the Best Paper Award.

While this specific project is set to conclude in 2022, there is interest from Patagona Technologies and the Department of National Defence in continuing the work. 

Professor Pralat will also continue exploring fake accounts and misinformation during a fellowship at the Simons Institute for the Theory of Computing at University of California, Berkeley this fall and in a separate, two-year project with the Government of Canada’s Communications Security Establishment, which is a national cybersecurity and foreign intelligence agency.

The common belief now is that if we cannot distinguish real accounts from fake accounts through the texts they write, maybe we can look at the network around them.

The project, Detecting and Responding to Hostile Information Activities: Unsupervised Methods for Measuring the Quality of Graph Embeddings, was funded by Patagona Technologies (external link, opens in new window) , which received funds under a Department of National Defence innovation program (external link, opens in new window) .