The Platform Question: Power, Accountability and Global South
Andean region and the Southern Cone were left adrift. There
was no longer anyone human to refer cases involving their
platforms to. Eva Cruells, founder and coordinator of Fembloc,
a feminist initiative dedicated to combating digital genderbased violence, told the Spanish media outlet Público:
“The thing is, the reporting mechanisms are very opaque
and often unresponsive. We have our own data: of the total
number of cases we have reported, only 4% have received any
affirmative response from the platform. Because often they
are bots or automated emails. Meta, for example, has greatly
reduced its content moderation staff.”
We revealed a similar situation as NGO Amaranta in 2020, with
the study Chile and gender-based violence on the internet,[4]
where we revealed that 11% of victims of doxing, digital
harassment, cyberflashing, defamation, and loss of access to
their account or unauthorized access to it, tried to report it
to the platforms without success. When victims were asked
what actions they took with the platforms, the vast majority
said that the reporting sites were in English, did not respond to
them, or had bugs that made them return to the same page to
fill in their details over and over again.[5]
Who takes care of what we don’t want to see on
the internet?
The history
of digital
moderation
reveals not only
the technical
complexity of
the process,
but also its
profound human
inequalities.
34
The history of digital moderation reveals not only the
technical complexity of the process, but also its profound
human inequalities. Meta began filtering content in 2007,
after New York Attorney General Andrew Cuomo demanded
that sexual images proliferating on Facebook be removed
within 24 hours. Since then, the company has outsourced
the task to subcontractors, some of which, such as oDesk,
paid moderators in Morocco as little as $1 an hour, a situation
that is repeated in several countries. Behind the algorithms
that protect the audience, there are people exposed daily to
traumatic content without psychological support or decent
working conditions. A 2019 article in The Verge reveals how
workers at a Cognizant subsidiary in Tampa faced precarious
working conditions, nine-minute breaks, the death of a coworker from a heart attack at his desk, and even how they had
to explicitly approve certain violent content simply because it
did not comply with Facebook’s moderation policies.[6] Many
A Big Tech Accountability in Content Moderation with Feminist Perspective
The Platform Question: Power, Accountability and Global South
of these workers joined the platform under the promise of job
stability and believing that they were performing a protective
role for social media users. However, one animal rights activist
worker recounts in the article that he had to watch a video
showing a case of extreme animal torture by teenagers
repeatedly. The platform’s policies did not allow him to take
it down, and his superiors consoled him by pointing out that
keeping it posted could prompt law enforcement authorities
to launch an investigation and find the culprits. That never
happened. The case highlights how moderation policies,
rather than protecting, end up reproducing other forms of
violence and trauma. For example, it is not possible to remove
a sexually aggressive video without a valid argument as to why
it cannot be shown. Spanish lawyer Francesc Feliu argues that
this precision is intended to train AI. “It cannot be trained if
the content it feeds on is violent, pedophilic, or murderous,” he
adds. Feliu legally represents more than forty workers dedicated
to moderating META content in Spain, who have jointly sought
compensation for the physical and mental damage to which
they are exposed in this work. The lawyer points out that most
of his clients suffer from anxiety disorders, depression, and
post-traumatic stress, and that several are even admitted to
psychiatric centers. All of them live with the pressure of not
being able to talk openly about the problem, out of shame or
fear of reprisals from the company, due to the confidentiality
clauses they signed.[7]
This reality is repeated on other platforms such as TikTok. In
2022, it was revealed that some of the staff responsible for
moderating content must follow accelerated review protocols
(sometimes even seconds per video) to keep up with the
massive volumes of posts they are asked to analyze every
day[8] in categories such as “explicit violence,” “hate speech,”
and “sensitive content.”[9] Automated language models are no
exception to this reality. The work of content moderators for
ChatGPT follows this same pattern of bad practices and job
insecurity. In Kenya, workers have revealed that they review
up to 700 passages of text per day, many of them laden with
explicit sexual violence, suicide, murder, child abuse, and
bestiality. They report having lost their families in the process
and not having any psychological support to cope with the
situation. While large companies advertise technological
advances that promise to free society from arduous tasks,
that same development depends on people who bear the
emotional burden of working with the dark side of the internet.
While large
companies
advertise
technological
advances that
promise to free
society from
arduous tasks, that
same development
depends on people
who bear the
emotional burden
of working with
the dark side of the
internet.
A Big Tech Accountability in Content Moderation with Feminist Perspective
35