Authors: Adrian Bermudez-Villalva, Maryam Mehrnezhad, and Ehsan Toreini
Accepted to be published by IEEE Transactions on Technology and Society
Introduction
The internet has revolutionised communication, enabling connection and collaboration across the globe. However, it has also fostered a darker side, online hate speech. While mainstream platforms have taken steps to moderate harmful content, less regulated spaces such as 4chan remain breeding grounds for extreme ideologies. The AGENCY Project is dedicated to understanding and mitigating online harms, and our latest research delves into the troubling prevalence of hate speech on 4chan’s politically incorrect board (/pol/), using cutting-edge deep learning techniques.
Exploring Online Hate: The 4chan /pol/ Study
4chan is infamous for its anonymous nature and minimal moderation, making it a unique case study for analysing hate speech. Our research employed state-of-the-art Natural Language Processing (NLP) models, including RoBERTa and Detoxify, to measure the extent and types of hate speech found on /pol/. With a dataset of 500,000 posts collected over time, we aimed to quantify the scale of harmful discourse and uncover its hidden patterns.
Our key objectives were:
- To measure the prevalence of different forms of hate speech on /pol/.
- To assess the toxicity of discussions and their impact on digital safety.
- To identify recurring topics within hate speech to better understand its context.
Disturbing Findings: A Breakdown of Online Hate
Our analysis revealed that 11.2% of posts contained hate speech, targeting various communities based on race, religion, gender, and sexual orientation. Among the different categories:
- Racism was the most prevalent, making up 35.9% of hateful posts.
- Religious hate followed closely, accounting for 23.3%.
- Sexual orientation hate comprised 16.5%.
- Sexism was found in 12% of hateful discussions.
Beyond hate speech classification, our research also evaluated toxic content, uncovering high levels of obscenity, identity attacks, and threats. Discussions involving sexual orientation and racism were among the most toxic, with nearly 99% of flagged posts classified as highly offensive.
Beyond Numbers: The Language of Hate
Using topic modelling techniques, we identified recurring themes within these hateful discussions. Racial hate speech often incorporated conspiracy theories and dehumanising rhetoric. Religious hate was largely directed at Jewish and Muslim communities, filled with stereotypes and aggressive language. Meanwhile, sexism on /pol/ displayed strong misogynistic tendencies, reducing women to derogatory labels and objectification.
The findings highlight the complexity of online hate, where discussions are not limited to overt slurs but extend into coded language, political rhetoric, and ideological indoctrination.
Implications for Moderation and Policy
This study has profound implications for policymakers, platform developers, and researchers:
- For policymakers, understanding the nature and prevalence of online hate can help craft more effective regulations, such as the UK Online Safety Act and EU content moderation policies.
- For tech developers, our findings stress the importance of advanced AI-driven moderation tools that can detect not only explicit hate speech but also nuanced and coded language.
- For society, these results underscore the need for digital literacy initiatives to educate users on identifying and countering online hate speech.
Conclusion: Addressing Online Hate in Unmoderated Spaces
The findings from this study expose the dangers of unmoderated digital spaces and their role in spreading hate speech. While freedom of expression is a fundamental right, it must be balanced against the risks of online harm. As part of the AGENCY Project’s ongoing mission, we aim to develop actionable solutions that empower individuals to navigate online spaces safely while holding platforms accountable for fostering inclusive digital environments.