Generative Algorithms and the Sleep of Reason

The growing use of generative algorithms raises the question as to who is responsible when they hallucinate (let’s stop using this term), particularly if this leads to legal issues.

Max Schrems, an Austrian lawyer and activist who has been already making life difficult for some US companies over data exchange agreements between Europe and the United States by pointing out that the National Security Agency there has constantly failed to guarantee privacy rights, is now suing OpenAI for violating its users’ privacy by producing false information about people, a problem that OpenAI admits it cannot correct.

In short, the question we now face is whether our privacy is being violated if someone makes up something about us? Can our privacy be violated by someone who has no information stored about us? It could, obviously, be a case of defamation, as when a professor found that ChatGPT said that he had been accused of sexual harassment, but was that an infringement of his privacy?

The answer requires some thought. ChatGPT, for example, has married me to the wife of the founder of my company, while Perplexity certainly perplexed me when it said I met my wife through a Twitter message, as well as that I have three children, when I have just one daughter. For me, these are mistakes of little transcendence, and that I use as funny examples in my classes and lectures, but what happens when you are a lawyer who is sanctioned because the algorithm invented several cases on which you based your defense, for example?

We all know that generative algorithms save us creation time, but increase the supervision work load, and that therefore, not carrying out that supervision correctly is irresponsible. But in the end, we are talking about incorrect information, the result of mistaken correlations, which OpenAI simply shrugs off with a generic warning that says “ChatGPT can make mistakes. Consider checking important information.”

Is that enough? If a tool is liable to make up potentially damaging stuff about anyone and to write it in a credible way, surely that involves admitting some liability? When asked “What responsibility does a generative algorithm have for the data it invents about a person?”, ChatGPT replies:

A generative algorithm, such as those used in artificial intelligence models to create content, has no “liability” in the human or legal sense, because it is simply a technological tool operated by humans. However, the developers and users of these algorithms have an ethical and, in some cases, legal responsibility to ensure that the content generated does not infringe on rights such as privacy and reputation. This involves implementing security and control measures to prevent the generation of false or defamatory information about individuals. In addition, it is crucial that there are clear usage policies and mechanisms to rectify any errors that algorithms may generate.

OpenAI admits it has no such policies and mechanisms, which looks highly irresponsible, and the company says they don’t have them because, as of today, it’s technically unfeasible. That said, would we be willing to give up the potential productivity of generative algorithms, because they can occasionally provide false information about people?

How common is it for generative algorithms to make things up? Quite common, actually. Chatbots “hallucinate” much more than some people think: anywhere between 3% and 27%; figures that suggest some kind of liability for those who market this tool.

The sleep of reason produces monsters. Algorithms do not dream, but they find correlations which they use to construct supposedly impeccable reasoning that, in reality, can be dangerous, often involving real people and that can create all kinds of problems. Should we demand that the companies that develop them accept their share of responsibility?

(En español, aquí)

—

This post was previously published on MEDIUM.COM.

***

You may also like these posts on The Good Men Project:

White Fragility: Talking to White People About Racism

Escape the “Act Like a Man” Box

The Lack of Gentle Platonic Touch in Men’s Lives is a Killer

What We Talk About When We Talk About Men

—

Photo credit: iStock.com