The risk that Meta could contribute to mass violence against Muslims in Myanmar should have been crystal clear to the company long before the 2017 atrocities perpetrated against the Rohingya. In July 2014, a viral Facebook post triggered an outbreak of lethal violence between Buddhist and Muslim groups in the city of Mandalay. The post falsely claimed that two Muslim men were to blame for the rape of a Buddhist girl in the city. The ensuing riots led the Myanmar authorities to temporarily block Facebook in recognition of the key role played by the platform in fuelling the “instigation” of this violence. Yet Meta’s minimal efforts to respond to this stark warning fell dramatically short. In fact, they may have even made matters worse. In one such example, Meta supported a civil society-led initiative known as ‘Panzagar’ or ‘flower speech’ by releasing a flower speech ‘sticker pack’. Users in Myanmar could post ‘Panzagar’ stickers on posts containing hate speech as a means of promoting peace and countering hatred. However, civil society activists noticed that the stickers were having serious unintended consequences. According to one activist who was involved in promoting ‘Panzagar’, Facebook’s algorithms counted the stickers as one more way people were enjoying a post. Instead of diminishing the number of people who saw a piece of “hate speech”, the stickers had the opposite effect of making the posts even more visible and more popular. After the IIFFMM highlighted the “significant” role of the Facebook platform in the atrocities perpetrated against the Rohingya, and as public criticism of the company’s failures grew increasingly severe, Meta eventually admitted in 2018 that “we weren’t doing enough to help prevent our platform from being used to foment division and incite offline violence”. In the intervening years, Meta has touted certain improvements in its community engagement and content moderation practices in Myanmar. Yet this report finds that these measures have proven wholly inadequate. This is largely because they focused primarily on improved content moderation (i.e., the detection, demotion and removal of content which violates platform rules). This approach rests on the premise that Meta is a neutral arbiter of content. It therefore fails to tackle the significant role played by the company’s own algorithms in proactively amplifying anti-Rohingya content – systems which are central to the company’s destructive business model. Amnesty International’s analysis of newly available evidence gleaned from internal Meta documents leaked by whistle-blower Frances Haugen – the “Facebook Papers” – enables a shocking new understanding of the true nature and extent of Meta’s contribution to harms suffered by the Rohingya. This evidence shows that the core content-shaping algorithms which power the Facebook platform – including its news feed, ranking, and recommendation features – all actively amplify and distribute content which incites violence and discrimination, and deliver this content directly to the people most likely to act upon such incitement. As a result, content moderation alone is inherently inadequate as a solution to algorithmically-amplified harms. Internal Meta documents recognize these limitations, with one document from July 2019 stating, “we only take action against approximately 2% of the hate speech on the platform”. Another document reveals that some Meta staff, at least, recognize the limitations of content moderation. As one internal memo dated December 2019 reads: “We are never going to remove everything harmful from a communications medium used by so many, but we can at least do the best we can to stop magnifying harmful content by giving it unnatural distribution.” This report further reveals that Meta has long been aware of the risks associated with its algorithms, yet failed to act appropriately in response. Internal studies stretching back to as early as 2012 have consistently indicated that Meta’s content-shaping algorithms could result in serious real-world harms. In 2016, before the 2017 atrocities in Northern Rakhine State, internal Meta research clearly recognized that “[o]ur recommendation systems grow the problem” of extremism. These internal studies could and should have triggered Meta to implement effective measures to mitigate the human rights risks associated with its algorithms, but the company repeatedly failed to act. Rather than addressing these risks appropriately, the Facebook Papers reveal in damning detail how Meta continues to ignore such risks to this day in the relentless pursuit of profit. As one former Meta employee outlined in an internal document dated August 2019: “We have evidence from a variety of sources that hate speech, divisive political speech, and misinformation on Facebook and the family of apps are affecting societies around the world. We also have compelling evidence that our core product mechanics, such as virality, recommendations, and optimizing for engagement, are a significant part of why these types of speech flourish on the platform.” THE SOCIAL ATROCITY META AND THE RIGHT TO REMEDY FOR THE ROHINGYA Amnesty International 8