The Platform Question: Power, Accountability and Global South
The Platform Question: Power, Accountability and Global South
experiences, the backend repeats existing, often reductive,
classifications. The consequences of this narrow perspective
in classification have implications across the board, like the
ability of a young woman on social media being able to report
an image of herself being posted online without her consent.
When annotators label what is considered “sexual content,”
they may focus solely on nudity. But in contexts like India,
intimacy or what is sexual does not need to be in the form of
nudity, but can be of two people holding hands in public.
These nuances are lost when the annotation logic is framed
and designed through a Western lens and then becomes
embedded into the functioning of AI systems. This is even more
concerning when “some AI scientists have stated their desire
to capture the world and to supersede other forms of knowing”
(Crawford 2021). When selective stories are compressed into
data, and that data is shaped by certain biases which produce
AI outputs of “beautiful bodies” that users come to accept as
reality, alternative stories and representations are effectively
overwritten.
Annotators
are provided
parameters
and guidelines
for labeling by
developers. Studies
have shown that
annotators’ of
socio-demographic
backgrounds and
lived experiences
impact how they
label.
126
The data is then labeled by low-wage workers from the Global
South who determine what is “beautiful, and which “bodies”
count. While the labour is global, the power to determine or
influence is not, because those who monopolise resources
monopolise imagination.” (Benjamin, 2019). Annotators are
provided parameters and guidelines for labeling by developers.
Studies have shown that annotators’ of socio-demographic
backgrounds and lived experiences impact how they label.
For example, in a study of large language models, researchers
found that female annotators were more likely to label texts
related to gender, religion, or cultural insensitivity as hateful
than male annotators (Das et al, 2024).
However, when speaking with AI/ML practitioners, annotators
were generally treated as “an apparatus for achieving a
representation of the world” (Kapania et al, 2023). The
subjective nature of the annotator would result in noting
disagreement and labeling ambiguous content on the basis
of their interpretations, informed by their own assumptions
and biases. In a model, any disagreements and inconsistencies
by an annotator are averaged across others and against a
The Missing Blocks of AI
“golden data set”.4 The annotator is treated as an objective
tool who would simply need to follow the parameters provided
to them. Yet, like the example of ImageNet, where bias was
attempted to be neutralised by crowdsourcing annotations
from Amazon Turk workers’ (Deng et al, 2009), the error rate
was 6% (Northcutt et al, 2021), demonstrating that subjectivity
matters even if notations were averaged out.
The Sociotechnical Machine
If we begin with the understanding that neither data nor bodies
are neutral, then we must also accept that AI is not an abstract
force - it is built, shaped, and sustained by human choices.
However, if social bias continues to be treated as a bug and not
a feature, as argued by several feminist authors, the redressal
mechanisms will often be piecemeal and limited to one model
or narrow benchmarks or ‘fairness’ or ‘transparency’. Take, for
example, Responsible AI. Rather than a universal framework,
it is an overarching term that encompasses the intention
of differing companies. Implementation varies as they are
principles rather than accountability mechanisms. More
importantly, existing within the same structures, assumptions,
and biases of the architects of major AI systems, it cannot be
transformative in changing the way that AI can be developed.
Bias and unequal power structures are built into the system, so
“how can one be “fair” in an unjust society?” (Hampton 2021).
Therefore, there is a need to interrogate and go beyond
frameworks like Responsible AI. To include the lived worlds
of the structurally marginalised people, questions around
consent and ownership of data, performance and exploitation
of labour, and co-opting social justice or feminist ideologies
have to be addressed. Feminist AI principles remind us that the
design of technology is never separate from systems of power
(Costanza-Chock, 2023). They ask us to center care, context,
and co-creation; to ask not just what data is collected, but
who gets to define it, label it, and benefit from it. This is where
imagination becomes essential. As Ruha Benjamin (2024)
urges, imagination is a tool of resistance, especially in the
face of systems designed to exclude, flatten, or erase. And
so reclaiming the right to imagine, especially for those whose
stories have long been left out of technological futures, is a
deeply political act.
The Missing Blocks of AI
To include the
lived worlds of
the structurally
marginalised
people, questions
around consent
and ownership of
data, performance
and exploitation
of labour, and
co-opting social
justice or feminist
ideologies have to
be addressed.
127
Select target paragraph3
Connect to a paragraph
Connect to an entity
Disable highlights
Add to table of contents