shaping algorithms to proactively promote certain types of harmful content in the pursuit of profit.187 Her testimony was informed by a huge cache of internal Meta documents which Haugen shared with members of the U.S. Congress. Amnesty International has reviewed many of these documents in detail – known as “the Facebook Papers” – and conducted an analysis of their human rights implications in respect of Meta’s duty to provide a remedy to the Rohingya. This analysis – informed by interviews with subject matter experts and other sources – is contained throughout this chapter. 6.1 “PLAYING WHACK-A-MOLE”: THE FALLACY OF CONTENT-BASED SOLUTIONS TO ALGORITHMIC HARMS “We are never going to remove everything harmful from a communications medium used by so many, but we can at least do the best we can to stop magnifying harmful content by giving it unnatural distribution.” Extract from an internal Meta memo, December 2019 188 As demonstrated in Section 5.4, above, Meta consistently failed to enforce its own content policies in respect of “hate speech” against the Rohingya in Myanmar. However, even if the company had conducted more effective content moderation, these measures may not have sufficed to mitigate the human rights risks and harms caused by the company’s business practices. This section assesses the limitations of content-based solutions (such as content moderation) to harms resulting from content-shaping algorithms. In 2018, alongside the publication of its human rights impact assessment for Myanmar, Meta partially acknowledged its role in the mass violence perpetrated against the Rohingya. The company stated: “prior to this year, we weren’t doing enough to help prevent our platform from being used to foment division and incite offline violence. We agree that we can and should do more.”189 This statement reflects Meta’s position that its primary failing was its inadequate moderation of content posted by other actors on the platform. This position effectively covers up Meta’s proactive role in the amplification of anti-Rohingya content, and it seeks to cast content moderation as the main solution to the spread of harmful content on the Facebook platform. But there are several problems with this position, most notably the fact that Meta’s own content-shaping algorithms promote and fuel the spread of harmful content, including content that advocates hatred (as detailed in Section 6.2, below). These content-shaping algorithms – which are optimized to maximize user engagement and therefore profit – are central to Meta’s overall business model.190 As a result, content moderation fails to address the root cause of Meta’s “hate speech” problem. Michael, the international aid worker who was subject of a viral Facebook post including threats to his life, noted the near impossibility of taking action against every single post among a tide of viral content: “My friends in Myanmar and in the US were flagging and reporting these posts. Many of them were being taken down, as far as I know. But the one with my passport photo had been shared over 1000 times by then… My friends had been reporting hundreds of different posts, it was like a game of whack-a-mole.”191 Recognizing these dynamics, Meta whistle-blower Frances Haugen stated during her testimony to the U.S. See: Karen Hao, “The Facebook whistleblower says its algorithms are dangerous. Here’s why.”, MIT Technology Review, 5 October 2021, technologyreview.com/2021/10/05/1036519/facebook-whistleblower-frances-haugen-algorithms 187 188 The Facebook Papers, “We are Responsible for Viral Content”, 11 December 2019, p.17, emphasis added. Meta, “An Independent Assessment of the Human Rights Impact of Facebook in Myanmar”, 5 November 2018, about.fb.com/news/2018/11/myanmar-hria 189 For a thorough analysis of the inadequacy of content moderation as a solution for business model-related harms, including the algorithmic amplification of problematic content, see: Ranking Digital Rights, ‘It's Not Just the Content, It's the Business Model: Democracy’s Online Speech Challenge’, 17 Mach 2020, available at newamerica.org/oti/reports/its-not-just-content-its-business-model; and Ranking Digital Rights, Getting to the Source of Infodemics: It’s the Business Model’, 27 May 2020, available at newamerica.org/oti/reports/getting-to-the-source-of-infodemics-its-the-business-model 190 191 Amnesty International interview by video call with Michael (pseudonym), 24 March, 2022. THE SOCIAL ATROCITY META AND THE RIGHT TO REMEDY FOR THE ROHINGYA Amnesty International 39