7.2 META’S KNOWLEDGE OF THE RISKS INHERENT IN ITS ALGORITHMS There is a substantial body of evidence which suggests that Meta was aware, or at least should have been aware, that its content-shaping algorithms risked contributing to adverse human rights impacts in Myanmar since as early as 2011. In a best-selling 2011 book, author and activist Eli Pariser outlined the risks inherent in the algorithms used on social media platforms, including Facebook: “Personalization filters serve a kind of invisible autopropaganda, indoctrinating us with our own ideas, amplifying our desire for things that are familiar and leaving us oblivious to the dangers lurking in the dark territory of the unknown.”285 Studies conducted by Meta researchers have repeatedly shown how the platform’s content-shaping algorithms can negatively impact users’ thoughts and behaviours. In 2012, Meta researchers undertook an experiment on 689,003 Facebook users which sought to assess the impact of curated News Feed content on users’ behaviour and mood. The research found that by algorithmically tweaking the content seen by Facebook users, “emotional states can be transferred to others via emotional contagion, leading people to experience the same emotions without their awareness”.286 Meta researchers again studied the impact of Meta’s algorithms on political polarization in 2015.287 The study found that “Facebook’s newsfeed algorithm decreases ideologically diverse, cross-cutting content people see from their social networks on Facebook by a measurable amount”.288 As noted in Section 6.2.4, internal Meta research from 2016 found “extremist content thriving in more than one-third of large German political groups on the platform”, and, significantly, it also recognized that Meta’s content-shaping algorithms were responsible for the growth of these groups.289 The report noted: The 2016 presentation stated that “64% of all extremist group joins are due to our recommendation tools” and that most of the activity came from the platform’s “Groups You Should Join” and “Discover” algorithms: “Our recommendation systems grow the problem [emphasis added].”290 These studies should have made Meta aware that its content-shaping algorithms posed a significant risk to human rights, especially in sensitive and conflict-affected contexts – long before the 2017 atrocities in Northern Rakhine State began. Though they mostly originate after 2017, the documents contained in the Facebook Papers reveal in greater detail than ever before the extent of Meta’s detailed knowledge of the harms associated with its business model and engagement-centric algorithms. They also highlight how the company has continued to pursue engagement at all costs even in the face of such detailed evidence. In one document contained in the Facebook Papers, entitled ‘What is Collateral Damage?’, dated 12 August 2019, an employee comments: “We have evidence from a variety of sources that hate speech, divisive political speech, and misinformation on Facebook and the family of apps are affecting societies around the world. We also have compelling evidence that our core product mechanics, such as virality, recommendations, and optimizing for engagement, are a significant part of why these types of speech flourish on the platform. 285 Eli Pariser, “The Filter Bubble: What The Internet Is Hiding From You”, 2011, p.13. Adam D.I. Kramer et al, “Experimental evidence of massive-scale emotional contagion through social networks”, Psychological and Cognitive Sciences, 2 June 2014, pnas.org/doi/full/10.1073/pnas.1320040111. See also: Robinson Meyer, “Everything We Know About Facebook’s Secret Mood-Manipulation Experiment”, The Atlantic, 28 June 2014, theatlantic.com/technology/archive/2014/06/everythingwe-know-about-facebooks-secret-mood-manipulation-experiment/373648 286 Zeynep Tufekci, “How Facebook’s Algorithm Suppresses Content Diversity (Modestly) and How the Newsfeed Rules Your Clicks”, 7 May 2015, Medium, medium.com/message/how-facebook-s-algorithm-suppresses-content-diversity-modestly-how-the-newsfeed-rules-theclicks-b5f8a4bb7bab 287 Zeynep Tufekci, “How Facebook’s Algorithm Suppresses Content Diversity (Modestly) and How the Newsfeed Rules Your Clicks”, 7 May 2015, Medium, medium.com/message/how-facebook-s-algorithm-suppresses-content-diversity-modestly-how-the-newsfeed-rules-theclicks-b5f8a4bb7bab 288 The Wall Street Journal, ”Facebook Executives Shut Down Efforts to Make the Site Less Divisive”, 26 May 2020, wsj.com/articles/facebook-knows-it-encourages-division-top-executives-nixed-solutions-11590507499 289 The Wall Street Journal, ”Facebook Executives Shut Down Efforts to Make the Site Less Divisive”, 26 May 2020, wsj.com/articles/facebook-knows-it-encourages-division-top-executives-nixed-solutions-11590507499 290 THE SOCIAL ATROCITY META AND THE RIGHT TO REMEDY FOR THE ROHINGYA Amnesty International 54