Here’s the Dark Side to ChatGPT: Billions of Fake Social Network Accounts in the Hands of Bad Actors

The BBC has raised a question that has been a concern for some time: mass language models (LLMs) like ChatGPT are being used by bot factories to create misinformation cheaply and easily on social networks and spread through fake accounts.

I’ve already experienced this: every time I update my Twitter account, a couple of bots reply to it with some scam. Every. Damn. Time. In my case, that means I have to waste time reporting those accounts, explaining that they’re automated spam bots, and blocking them… which serves no purpose. Only by tracking the hundreds of accounts I’ve been blocking for months could an algorithm be trained to learn to recognize them and block them at the root. But as before with Twitter, nothing is being done about it.

It will come as no surprise to learn that there is apparently no dedicated team at Twitter to deal with this. Despite claiming to delete more than one million spam accounts a day and having created a specific way to report bots, Twitter seems to be doing nothing to stop the mass production of fake accounts that spews thousands of messages promoting scams on the internet every day. The fact that Elon Musk himself claims that 90% of comments on Twitter are made by bots is hardly a ringing endorsement that the root and branch reform he promised is underway.

Creating a bot that generates content on social networks is easy, and we have already seen cases in which it has been shown that it was possible to create not just an account, but a fake influencer using algorithms. In the algorithmic arms race between social media managers and spambot factories, the bad guys are way ahead. Creating algorithms capable of reliably recognizing a fake account and shutting it down is simple, but isn’t being done. As far as we can see, there are other priorities. Soon, algorithmically created spambots will be sold to the highest bidder, be it a spammer, a criminal, a scammer or a political party; and they will look more realistic and be able to do more harm. The prospect of an election where networks of millions of bots sow dissent and spread dangerous propaganda, is more likely than ever.

Technology is a tool, and as such, it can be used for good and ill. Given the enormous amount of data that a social network generates, training an algorithm that is capable of discriminating which accounts respond to patterns that correspond to spambot activity should be very simple, and has been done before. The key is to understand that this has to be done 24/7, because spammers will always try to circumvent the criteria of those algorithms and continue to pass off their activity as genuine, and that if a social network drops its guard for a moment, those spammers will flood in and not only tarnish its reputation, but also ruin the experience of many users. If we also, through sheer inaction, allow these spammers to pass themselves off as real, and take advantage of technologies that are already widely available to “improve” their activity, we will soon have a major crisis on our hands.

—

This post was previously published on Enrique Dans’ blog.

***

You Might Also Like These From The Good Men Project

You Said ‘Race’, but Are You Actually Talking About Race?

Understanding the Nonbinary: Are You Confusing Gender With Sex?

The Difference Between Compassion for Those With Disabilities & Ableism?

‘Masculinity’ Is Having an Identity Crisis