AI Isn’t Neutral — And That Should Worry Us

The widely held view on AI is that it is unbiased, objective, impartial. Pause for a moment to think about it and it’s obvious that nothing could be further from the truth. Everything AI “says” is based on the data we train it with and, increasingly, reflect the economic interests driving its rapid expansion.

Let’s start with the most the most obvious: the instinctive or native biases that come from the data. AI used for medical diagnosis learns from histories, images, hospital records. If these data come mostly from men, or from people in middle age, or from large cities, the machine will not see much “beyond its scope of expertise”. It will fail more with women, with the elderly, with those who live in areas not covered by the healthcare system. In this case, AI is not making a conscious decision to exclude certain demographics, it is simply mirroring the biases of our society.

There are even biases we introduce to ourselves: if we try to use artificial intelligence as a psychologist, a completely absurd idea but one that seems to be gaining popularity, our own questions generate a convergence of answers that tend to support us and always agree with us, except in very specific matters. Very satisfying, perhaps, but also very much lacking in rigor, and definitely not what we want from a psychologist. In extreme cases, it even ends up helping us write our suicide note. A charming psychologist, really…

Other examples are facial recognition systems that work better with light-skinned people than with dark-skinned people or language processing algorithms that make more mistakes with non-standard accents or dialects that don’t appear in their training corpora. In the field of marketing, language models already adjust their message according to age, gender or educational level, generating different slogans for different groups, which shows that even text-generating AI subtly discriminate.

As if that weren’t bad enough, the rapid growth of AI has now created a more dangerous source of bias: advertising, sponsorship and the business model that underlies each AI platform. Perplexity is going to insert advertising in the form of sponsored follow-up questions or paid content next to answers, supposedly ensuring that “the content of the answers will not be influenced by advertisers.” That promise, however, is extremely fragile, because it is enough to modify the relative weight of certain topics, prioritize certain sources or make certain ads appear first, to tilt the result. That a search AI vacillates between offering the best answer and serving the highest bidder is a real dilemma.

Meanwhile, OpenAI has begun recruiting advertising specialists from Google and other competitors with expertise in monetization, suggesting that the commercial side of the AI business is already being planned. OpenAI’s most recent DevDay focused less on the sheer power of models and more on how they fit into workflows, apps and ecosystems where there may be implicit business incentives.

The confluence of native biases and sponsored biases leads us to an uncomfortable conclusion: artificial intelligence is not “omniscient,” it is not “infallible,” and it is not above ethics. When we say “artificial intelligence said so,” we’re assuming it can’t be wrong. But it can: it omits realities, silences voices, privileges interests. There is no oracle: there are statistical models with hidden priorities.

In a recent study titled “Generative AI search engines as arbiters of public knowledge”, researchers found that AI models show geographic and commercial biases in the sources they cite and in the tone with which they address topics. That “authority” may seem impartial, but it often reproduces existing power imbalances.

So what to do? Using more AI isn’t going to help; what’s needed is “critical AI”: independent ethical audits, transparency in ranking criteria, mitigation of bias in data, and public oversight over how those systems are monetized. That we know not only what artificial intelligence says, but why it says it.

It’s up to us as users to put pressure on AI: we can’t leave key decisions related to all kinds of areas, from health, justice and credit to reputation and much more to unsupervised black boxes.

Remember: this is not a problem of the future, it is already happening. And recognizing and understanding AI’s problems and limitations is the first step to avoiding them.

(En español, aquí)

—

This post was previously published on MEDIUM.COM.

—