The post is written by an unknown employee and dated 11 December 2019. It explains Meta’s responsibility for viral content in clearly understandable terms: “Unlike communication with close friends and family, virality is something new we have introduced to many ecosystems… and it occurs because we intentionally encourage it for business reasons”203 The post acknowledges that: “[R]anking content about higher stakes topics like health or politics based on engagement leads to perverse incentives and integrity issues.”204 In a clear articulation of how Meta’s algorithmic ranking systems operate in practice, combined with a recognition of Meta’s responsibility for the harms it produces, the author states: “Our ranking systems have specific separate predictions for not just what you would engage with, but what we think you may pass along so that others may engage with. Unfortunately, research has shown how outrage and misinformation are more likely to be viral, and recent experiments that deprecate these models indicate that removing these models does positively impact metrics for misinformation and hate”.205 The post further reveals that the viral content shown to Facebook users is not solely based on users’ preferences, but rather targeted in a manner which would maximize engagement by stoking emotive responses. It also gives an insight into how ‘engagement’ is measured and rewarded on Facebook: “We prioritize content based on engagement, but engagement doesn’t necessarily mean that a user actually wants to see more of something. One of our biggest signals we use to provide more similar content is comments and… a comment that you hate a thing can be seen as a positive signal leading content to get outsized distribution. People game this in various ways, posting ever more outrageous things to get comments and reactions that our algorithms interpret as signs we should allow things to go viral…”.206 The post clearly acknowledges how Meta’s optimization of its platform for maximizing engagement in the pursuit of maximum profit carries significant systemic risks in the societies in which Meta operates: “…[A]s long as we continue to optimize for overall engagement and not solely what we believe individual users will value, we have an obligation to consider what the effect of optimizing for business outcomes has on the societies we engage in.” 207 With specific reference to high-risk environments, such as Myanmar, the author highlights the risks which this model poses to ethnic groups at risk of discrimination or violence: “Recent research has shown how misinformation and divisive messages travel faster on social media… What happens when you introduce virality into places where people will believe anything or in places where people are pre-disposed to believe the worst about vulnerable ethnic groups? The honest answer is we don’t know.”208 The author of the document offers some suggestions to address the risks and harms associated with engagement-centric virality: “Since we have introduced virality to the world, our integrity efforts should focus on making virality a force for good – or at least less bad. We are never going to remove everything harmful from a communications medium used by so many, but we can at least do the best we can to stop magnifying harmful content by giving it unnatural distribution [emphasis added]. We should also take seriously the charge that we are affecting media ecosystems by creating perverse incentives.” 209 Although these warnings and recommendations were authored in 2019, Meta appears to have done little to change the fundamental design and orientation of its content-shaping algorithms since then. Meta’s failure to act on warnings such as these led whistle-blower Frances Haugen to testify before the U.S. Senate in 203 The Facebook Papers, “We are Responsible for Viral Content”, 11 December 2019, emphasis added. 204 The Facebook Papers, “We are Responsible for Viral Content”, 11 December 2019, p.26. 205 The Facebook Papers, “We are Responsible for Viral Content”, 11 December 2019, emphasis added. 206 The Facebook Papers, “We are Responsible for Viral Content”, 11 December 2019, p.8. 207 The Facebook Papers, “We are Responsible for Viral Content”, 11 December 2019, p.7. 208 The Facebook Papers, “We are Responsible for Viral Content”, 11 December 2019, p.17. 209 The Facebook Papers, “We are Responsible for Viral Content”, 11 December 2019, p.17, emphasis added. THE SOCIAL ATROCITY META AND THE RIGHT TO REMEDY FOR THE ROHINGYA Amnesty International 42