Systemic Oppression and Machine Learning

Many words have been written about the biases in “AI” training datasets. I last blogged publicly on this topic about 6 years ago, and it feels like the right time to write about it again.

Bias and privilege have long played a significant role in what parts of our world get remembered. And, for decades now, they have continued to play a crucial role in what of those memories get digitized.

Colonial narratives and indigenous erasure, patriarchy and women’s history erasure, hetero-normative and cis-normative bias and LGBTQ+ erasure, the examples are all in the world around us.

Recognizing oppression

One of my favourite books on the topic of systemic oppression is Ibram X. Kendi’s “How to Be an Antiracist.” For me, this book brought a critical perspective to my emergent understanding of how systemic oppression works, and the lengths to which we will be required to go to dismantle it, through the lens of racism.

To correct for it, we need to go further than our own personal feelings on these issues. We are empathetic humans, and bigotry is less often tolerated openly. However we live with that bigotry codified in the way we’ve structured the world around us. “I’m not racist” doesn’t matter, for example, when our financial systems favour a certain socio-economic strata dominated by a particular skin colour. When housing policies separate those strata into different neighbourhoods. When we gather data in only the places from which it’s easy to gather.

Modern “AI”

There’s no question that “AI” has again captured the imagination of the public today. The ideas aren’t new, but the power of the computers we can build has tipped a threshold.

Large Language Models (LLMs) and Large Multimodal Models (LMMs) are a sub-category of machine learning that have become possible as computing technology has become more powerful, and as the collective works of our biased world have become digitized.

The remix

“Generative AI” is, by definition, incapable of new ideas. All it can do is remix.

In Professor Lawrence Lessig’s notion of Remix Culture, he introduces the idea that the remix can be its own legitimate art form. We’ve fallen into it because so much of our existing realms of creativity have already been explored.

Several years ago, this prompted Damien Riehl and Noah Rubin to create all 68 billion possible 8-note melodies, copyright them, and release them to the public domain in an attempt to end frivolous copyright lawsuit claims in the music industry.

Issues of who owns someone voice and their visual likeness plague the TV and film industry. Modern copyright law and “who owns the remix” is a complex legal issue unto itself that distinguished individuals like Lawrence Lessig, Cory Doctorow, and Canada’s own Michael Geist have been working to advance.

The legal entanglement challenge in the remix is reflective of the level of entanglement of oppression in that same remix.

Removing systemic oppression from the remix

Some good work is being done from pre-screened training data-sets to reduce bias in training data, to “logic-aware models” that attempt to mitigate an LLM’s tendency to regurgitate the bias with which it has been trained.

Will “logic-aware models” be anti-racist enough to remove racist responses from these models’ statistical responses? Time will tell.

Meanwhile, damage has been done, and we still haven’t recovered from it. And it’s still being done.

Unravelling systemic oppression is an unending endeavour. Because of human cognitive biases, the content we produce will always reflect our blind spots.

Continued bias - “AI” is for the rich

Leaked data about OpenAI’s GPT-4 indicated that it took 25,000 Nvidia A100 GPUs 90-200 days to train the model. That would have been about 3,125 Nvidia HGX servers at $118K each - a $370M server farm, drawing about 19 megawatts of power. GPT-4 is estimated to be a 1.76 trillion parameter model. We can barely run 7 or 13 billion parameter models on current consumer hardware.

Needless to say, while OpenAI, Microsoft, and Nvidia are showing us the leading edge of what’s possible with technology, there are few organizations today that can afford to do this kind of work.

Our responsibility

As we are those with the privilege, thus we also hold the responsibility - how will we incorporate anti-oppression in our products and services.

It’s not just good for humanity, it’s good for business.

Efforts to democratize machine learning technologies are blossoming, the 78,000 training datasets and 400,000 machine learning models on the Hugging Face hosting platform are a direct indicator of that.

But today, it takes still tremendous resources to leverage these technologies. It behooves us to use those same resources to get past our historically naive approaches to equality and risk mitigation in this new technological climate.

Reputational risk is high because mistakes we make will be well publicized.

We have an opportunity to counter that with mitigation strategies that can withstand the test of scrutiny.

“Benigning” AI

“AI” is simply mathematical formulae.

Those formulae are complex, can surprise and delight us, but they do not empathize nor do they feel nor can they.

Maybe there are folks that believe that our bio-mechanical bodies can be reduced to complex formulae, but I’m reminded of so many prominent people in the field of science who saw so much in the nature of the universe around us that saw the edges we couldn’t explain and couldn’t help believing that there was more than we could explain.

Can “AI” think and feel and behave like a human being? Will it always be limited to mimicry?

Nobody knows today.

But today, in the explosion of its application, it’s up to us to render this technology benign.

Brain on, folx, brain on

For decades in the software field I’ve held the stance that we cannot fall into autopilot.

We are humans writing software to help other humans.

The computer in the middle constantly changes, but we are still the humans on either side.

While I believe today we live in a “post-agile” world, where it has undergone sufficient semantic diffusion as to render the word “agile” useless to communicate intent, the foundation of all of this work has always been about bringing humanity into what was perceived as a mechanical endeavour - writing software.

I’ve been telling people for many years that writing code may be the least useful thing developers could be doing.

Writing code was never the problem.

Sure, we can almost always be doing it better, and using generative AI as a tool in my own software development endeavours over the past two years has without question propelled my work forward.

When my brain was on.

When my brain wasn’t on, generative AI remixed volumes of noise at a staggering rate that slowed me down until I realized what I was falling into.

Every line of code we adopt into our systems is a liability. It slows us down. It’s a bug waiting to announce itself. It’s something someone else has to understand. It’s clouded with our biases.

As our own hand emitted the words or code we wrote, we could hold accountability for its biases and quality. We could create a form that asked for a first name and last name, and realize, “hey, not all cultures have a first and last name, that’s a very colonial construct.”

Now, our own hand is enhanced with the words and code of others remixed, with all of the inherent oppressive bias. We may miss that opportunity to look at that form and think.

So, be intentional in your thinking. In your empathy for people on the other side of the words and code you’re writing.

Don’t let the computer in the middle take away your humanity in the eyes of your users and stakeholders.