In today’s fragmented digital environment, it has become more challenging than ever to spot malicious entities like trolls and purveyors of false information.
Frequently, attempts to identify malicious accounts concentrate on examining their content. Nevertheless, our recent study implies that we ought to pay greater heed to their activities instead.
do
—and how they accomplish it.
We've created a method for spotting potential problematic individuals online purely through their behavioral patterns—their interactions with others—rather than what they actually post.
presented our results
at the recent
ACM Web Conference
, and received the Best Paper award.
Past merely examining people’s statements
Conventional strategies for identifying troublesome online conduct generally depend on two techniques. The first involves examining
content
(The feedback from others). The alternative is to conduct an analysis.
network connections
(who follows whom).
These methods have limitations.
Users can circumvent
Content analysis shows they might delicately word their language or spread misinformation without employing clear trigger terms.
Network analysis proves inadequate on platforms like
Reddit
Here, user connections aren't explicitly defined. Communities are structured based on interests rather than personal interactions.
We aimed to develop a method for spotting harmful individuals that wouldn’t be simple to manipulate. It dawned on us that concentrating on behavior—how people engage with each other instead of what they claim—could achieve this.
Instructing artificial intelligence to grasp human behavior on the internet
Our method employs a strategy known as
inverse reinforcement learning
This approach is commonly employed to grasp how humans make decisions within areas like self-driving cars or game theory.
We modified this technology to examine user behavior on various social media platforms.
The system operates by monitoring a user's activities, including generating new posts, submitting comments, and responding to others. It then deduces the fundamental approach or "strategy" behind their conduct from these actions.
In our Reddit case study, we examined 5.9 million interactions spanning six years. We recognized five different user archetypes, with one standout category being dubbed "contrarians."
Meet the ‘disagreers’
One of our most notable findings was identifying a distinct group of Reddit users who primarily aim to contradict others. These individuals actively look for chances to share opposing viewpoints, particularly when they encounter dissent, before quickly moving on without awaiting responses.
The individuals who disagreed were most frequently seen in politically-oriented subreddits (discussion forums centered around specific subjects) like
r/news
,
r/worldnews
, and
r/politics
Interestingly, they were far less prevalent in the formerly prohibited pro-Trump online community.
r/The_Donald
despite its political focus.
This pattern highlights how behavioural analysis can reveal aspects that content analysis might overlook. Within r/The_Donald, users generally agreed among themselves but showed hostility towards external groups. This behaviour could be why conventional content moderation approaches have often failed.
struggled
to tackle issues within these communities.
Soccer fans and gamers
Our investigation uncovered unforeseen links as well. Individuals engaging with entirely distinct subjects often exhibited strikingly comparable behavioral trends.
We discovered notable parallels among users talking about soccer (on
r/soccer
) and e-sports (on
r/leagueoflegends
).
This resemblance stems from the intrinsic characteristics shared by both groups. Football and video game enthusiasts exhibit similar behaviours: they ardently back particular squads, closely monitor games with great enthusiasm, join fervent debates regarding tactics and individual playstyles, rejoice in triumphs, and scrutinise losses.
Each community nurtures robust tribal identities. Members champion their preferred teams whilst criticizing opponents.
Regardless of whether they're discussing strategies for the Premier League or choosing champions in League of Legends, the fundamental interaction dynamics—the rhythm, order, and emotional tenor of replies—stay similar within these distinctly different groups.
These findings contradict traditional views on online polarization. Although echo chambers are frequently cited as a cause of growing divisiveness, our study indicates that behavioral tendencies might extend beyond specific topics. It appears that users' divisions stem more from their interaction methods rather than the subjects they engage with.
Beyond troll detection
The impact of this study reaches far outside academia. Moderators working on these platforms might utilize behavioral trends to spot possibly troublesome users prior to them posting significant amounts of detrimental material.
Content moderation relies on comprehending language, whereas behavioural analysis doesn’t. This makes it harder to avoid because altering your behavior takes more effort than tweaking what you say or write.
This method might also assist in crafting better approaches to combat misinformation. Instead of concentrating only on the material itself, we can develop frameworks that promote healthier interaction habits.
For social media users, this research offers a reminder that how we engage online – not just what we say – shapes our digital identity and influences others.
With ongoing challenges like manipulation, harassment, and polarization persisting in digital environments, strategies that integrate behavioral trends along with content evaluation might provide superior methods for cultivating safer online societies.
Marian-Andrej Rizoiu has received financial support from entities including the Advanced Strategic Capabilities Accelerator, the Australian Department of Home Affairs, the Defence Innovation Network, as well as the National Science Centre in Poland.
Lanqin Yuan and Philipp Schneider do not hold positions with, provide consultancy services for, possess stocks in, or receive financial support from any entity that could gain from this article. They have declared no additional associations outside of their academic appointments.
Comments
Post a Comment