Some of the largest social media platforms are facing criticism for their failure to detect and remove harmful suicide and self-harm content, as highlighted by a recent study conducted by the Molly Rose Foundation. The study revealed that out of over 12 million content moderation decisions made by six major platforms, only Pinterest and TikTok were able to detect and remove over 95% of harmful content related to suicide and self-harm. The remaining platforms, including Facebook, Instagram, Snapchat, and X, formerly known as Twitter, were found to have inconsistent and inadequate responses to such content.
The Molly Rose Foundation, a charity established by Ian Russell and his family in memory of their daughter, Molly, who tragically took her own life at the age of 14 after being exposed to harmful content on social media, expressed concerns over the lack of effective content moderation by most platforms. According to the foundation, Meta’s Instagram and Facebook were each responsible for only 1% of the suicide and self-harm content detected by the major sites studied, while X accounted for just 700 content decisions. This alarming trend has raised questions about the effectiveness of the current regulatory framework, particularly in light of the Online Safety Act.
Ian Russell, chairman of the Molly Rose Foundation, called for stronger regulation and urged the government to prioritize the implementation of a new Online Safety Bill to address the systemic failures in content moderation by social media companies. He emphasized the need for more proactive measures to protect vulnerable users and prevent further tragedies like the loss of Molly. Russell criticized tech companies for their inaction and called for decisive action to safeguard young lives.
In response to the foundation’s report, a Meta spokesperson defended the company’s efforts to remove harmful content related to suicide and self-injury, stating that they had removed over 50.6 million pieces of such content globally on Facebook and Instagram in the past year. However, the spokesperson acknowledged limitations in deploying all measures in the EU, which may have impacted the detection and removal of harmful content. Similarly, Snapchat emphasized its commitment to safety and well-being, highlighting the platform’s proactive approach to content moderation and support for users in distress.
The study also highlighted specific shortcomings in the content moderation practices of social media sites, such as the underreporting of harmful content on video-based features like Instagram’s Reels. Despite the growing popularity of short-form videos on social media, the study found that only a small fraction of suicide and self-harm posts detected by Instagram were videos, raising concerns about the platforms’ ability to effectively monitor high-risk content. Additionally, the study criticized TikTok for suspending only two accounts out of nearly three million items of harmful content detected, suggesting a lack of enforcement of their own rules.
Looking ahead, the Department for Science, Innovation and Technology underscored the importance of social media companies taking responsibility for the safety of their users and implementing effective processes to address harmful content. Under the Online Safety Act, platforms are required to proactively remove illegal content promoting serious self-harm and prevent children from accessing material that encourages self-harm or suicide. The government emphasized the need for swift action to protect users and urged companies to prioritize user safety without waiting for legislative mandates.
Despite the mounting pressure on social media platforms to enhance their content moderation practices, Pinterest and X have yet to respond to the criticism raised by the Molly Rose Foundation’s report. The ongoing debate surrounding the regulation of harmful content on social media underscores the urgent need for collaborative efforts between tech companies, regulators, and advocacy groups to ensure a safer online environment for all users.