Talking the talk, but not walking the walk?
Online hate speech can lead to real-world harms, as seen in the use of Facebook to spur ethnic violence in Myanmar, or the radicalisation of the Christchurch shooter on YouTube. Major tech platforms and many national laws prohibit hate speech and incitement to violence, yet the research of Global Witness and others have shown social media corporations have a terrible track record when it comes to detecting this banned content.
In response to the exposure of harmful content on their platforms, these corporations have pointed to the tool they give users to report such material, allowing it to be reviewed and removed if it violates the company’s policies. But does the reporting process actually work? Our investigation into two platforms reveals startling oversights.
Testing the platforms’ reporting tool
We set out to test the reporting process on two social media platforms, the video sharing site YouTube and the microblogging site Koo, in both India and the US. We decided to conduct our research in these two different geographic contexts because they are both large global democracies with national elections in 2024, where online hate speech and misinformation have already led to offline violence. Assessing the performances of a well-established and popular video platform headquartered in the US and a newer Twitter-like platform based in India gives a useful indication of their preparedness for dealing with prohibited content around the elections.
Evidence suggests the impact of online harassment is heavier for women than men (notably for journalists and politicians), and that online attacks against women are more often based on gender. Given this, we chose to focus on hate speech against women on the basis of sex/gender.
We began by identifying real examples of gendered hate speech content in English and in Hindi that were viewable on the platforms but clearly violated the companies’ hate speech policies. In November 2023 we reported the content and details of the violation using each platform’s reporting tool, totalling 79 videos on YouTube and 23 posts on Koo.
This story was originally published in globalwitness.org. Read the full story here.