In September 2024, after a two-year hiatus, X (formerly Twitter) released its latest transparency report for the first half of the year, sparking intense debate over the role of AI in content moderation. While such reports from major platforms like Facebook, Google, and X aim to shed light on content management and enforcement policies, the latest findings raise more questions than answers. The report reveals alarming shifts in how X handles issues such as child exploitation content and hate speech, especially in the context of its growing reliance on artificial intelligence (AI) for moderation.
Alarming Content, Limited Action
X’s 2024 report displays a staggering surge in reported accounts and tweets, with over 224 million reports in just six months. This represents a massive increase of nearly 1,830% compared to the 11.6 million accounts reported in late 2021. Despite this exponential rise, account suspensions only saw a modest increase—from 1.3 million in 2021 to 5.3 million in 2024, representing roughly 300% growth.
Most concerning, however, is X’s handling of child exploitation content. Out of over 8.9 million reports related to child safety, the platform took action on only 14,571 instances. This discrepancy highlights a critical issue in X’s ability to effectively manage harmful behavior, especially when comparing the lack of significant action to the massive volume of reports. The report also showcases a sharp decline in actions taken against hate speech. In 2021, X suspended over 104,000 accounts for hateful conduct. By contrast, in 2024, only 2,361 accounts faced suspension for similar offenses.
One of the contributing factors is X’s shift in defining hate speech and misinformation. Under previous management, hate speech and misinformation policies were more stringently defined. However, current policies, such as no longer classifying misgendering or deadnaming as hate speech, have blurred enforcement metrics, making it difficult to assess the effectiveness of moderation measures.
The Role of AI in Content Moderation
Central to the discussion is X’s increasing reliance on AI to manage content. The report outlines how moderation is now carried out through “a combination of machine learning and human review.” This brings up a fundamental question: Can machines make moral judgments in content moderation, or are they exacerbating the very issues they are meant to resolve?
Research and real-world examples suggest that AI is not foolproof in this domain. Automated content reviewers have been known to misclassify benign content as harmful while failing to catch more nuanced harmful behavior. For instance, in 2020, Facebook’s algorithms mistakenly blocked ads from struggling businesses, and earlier this year, its system mistakenly flagged posts from the Auschwitz Museum as violating community standards.
Moreover, AI moderation systems are often trained on data primarily derived from the Global North, leaving them less effective in handling diverse languages and cultural contexts. A report from the Centre for Democracy and Technology notes that a lack of diversity in natural language processing (NLP) teams can impair the accuracy of moderation in dialects like Maghrebi Arabic. Similarly, Mozilla Foundation has raised concerns about AI reinforcing existing societal inequalities, emphasizing the importance of addressing these risks responsibly .
Complexities of Detecting Hate Speech
One of the biggest challenges in AI-driven moderation is detecting hate speech. A 2021 study by Oxford University and the Alan Turing Institute found that hate speech detection models often perform poorly across different categories of hate speech. The study tested several models, including Google’s Jigsaw Perspective API and Two Hat’s SiftNinja, revealing significant shortcomings. While Perspective tended to over-flag non-hateful content, SiftNinja consistently under-detected hate speech.
This inconsistency may explain why X’s report shows a decline in actions taken against hateful conduct. AI systems are often unable to interpret sarcasm, coded language, or culturally-specific hate speech, leading to both under-enforcement and false positives. This also raises concerns over the potential suppression of free speech, especially among marginalized communities that often use coded language as a form of expression.
A Wider Industry Problem
The issues revealed in X’s transparency report are not unique to the platform. Meta, which owns Facebook, Instagram, and Threads, has also faced criticism for the effectiveness of its AI-driven content moderation systems. In many instances, Meta’s algorithms have failed to identify disinformation or hate speech, leading to false positives and missed harmful content .
In 2023, Elon Musk’s decision to discontinue free access to X’s API further restricted the ability of researchers and watchdogs to monitor trends in content moderation and disinformation. Similarly, Meta’s shutdown of CrowdTangle, a tool used to track misinformation, has made it more challenging to assess social media dynamics. As platforms increasingly shape public discourse, the lack of transparency raises concerns over the future of content moderation, especially in the lead-up to key elections.
The Need for Regulatory Action
The troubling trends identified in X’s report are likely to fuel calls for regulatory intervention. With AI now embedded into various aspects of social media moderation, platforms need to be held accountable for the ethical challenges posed by automated systems. Experts, including those from the AI Now Institute, advocate for more transparency and ethical standards in AI moderation. Lawmakers may need to mandate a combination of AI and human moderators to strike a balance between accuracy and fairness .
As AI continues to evolve, social media platforms will need to address the broader ethical implications of automating content moderation. The future of social media may well depend on whether we can trust machines to make moral judgments — or if a more human-centered approach will be required to ensure fairness and accountability.
Learn more about AI and its impact on technology trends at CodeSin Blog.
Read the full report on Forbes
