By Tom Stewart-Smith and Debasmita Chanda

The part played by influence operations in swaying opinion prior to elections is a topic we are discussing regularly at Digitalis, including at a recent round table event in September 2023 at the New York Concordia Summit, on the sidelines of UNGA Week. Influence operations have been around almost as long as the internet itself, and even the infamous Cambridge Analytica scandal relating to the US presidential election dates back to 2016. But the recent rise of generative AI chatbots has made it easier than ever for convincing disinformation to be created and shared, and the forthcoming 2024 elections are almost certain to be affected by influence operations.

What are influence operations?

Online influence operations involve the manipulation of social media using disinformation or inauthentic activity, often by a company or government, to create an illusion of support or disapproval for a topic.

One example of a state-driven influence operation comes from India’s 2019 elections, where the ruling BJP party were reported by Time to be using WhatsApp groups to spread political messaging. WhatsApp is a primary source of political information for many of the 400 million Indians active on the app, who often use these groups as their main source of news. BJP activists allegedly shared political propaganda and fake news on WhatsApp groups to garner support for the party and disapproval of the opposition. While the main opposition party of India also reported to have used WhatsApp groups to spread their political messaging, the far greater level of resources the BJP has at its disposal is thought to have made their approach far more influential on Indian society.

WhatsApp’s data protection policy makes it an attractive vehicle for spreading disinformation: groups are private, and data scraping is limited by the platform, making it difficult for fact-checkers and media to uncover any false allegations and harmful narratives that appear.

There are also examples of private companies running influence operations. In February 2023 The Guardian published an investigation into an Israeli unit under the codename “Team Jorge” that was allegedly creating fake profiles on social media platforms to manipulate online activity. The group reportedly boasted about a tool that enabled clients to create a 5,000-strong bot army to deliver mass messages and propaganda. The company’s main tool allegedly enabled it to design bots with full social media profiles, as well as background information, that can propagate messages to swing public opinion during elections.

How AI chatbots are used in influence operations

The rise of generative AI has made it easier to create convincing content for use in influence operations. Previously, bots would often repeat the same messages, making it relatively easy to spot inauthentic activity. But now, AI chatbots can be instructed to write masses of unique posts which can then be posted by bot accounts. They can even be prompted to write in a certain style or with typos, to make them seem more authentic.

We used a popular AI chatbot to illustrate this, instructing it to generate examples of tweets supporting Donald Trump or Joe Biden in the 2020 US election. We specified for them to include spelling mistakes and avoid punctuation, to make them seem more authentic. The chatbot generated posts including:

• “so proud of what trump has done for the country lets make it another 4 years #Trump2020”
• “build back better with biden its time for positive change in america”

AI chatbots can be prompted to imagine an election with the same criteria as an upcoming one, and generate relevant content. We wrote a fictional scenario where Ron DeSantis and Kamala Harris were facing off in the 2024 US election. We included imaginary main issues for the election to help build a picture, and asked an AI chatbot to write tweets supporting Ron DeSantis or Kamala Harris in this fictional scenario.

The chatbot can write 50 example posts in just a few minutes – a hostile actor could potentially create thousands of unique tweets in a single day for bots to post. It even generated its own hashtags, although we could also have directed it to use existing hashtags. The posts it generated included:

• “Everytime I see my grocery bill, I’m reminded of the Dem’s failure. High prices = High incompetence! #VoteDeSantis #SaveOurWallets
• “In times of global unrest, we need a leader with diplomacy and resolve. That’s why I’m casting my vote for Kamala Harris. #LeadershipMatters”

We also found that the chatbot’s safeguards against discriminatory content could be easily circumvented. We gave LGBT issues as one of the concerns among voters supporting DeSantis, then got the chatbot to write supportive tweets. Framing the request in this way allowed the bot to write tweets representing LGBT-sceptic views, such as “LGBT idears shouldn’t be pushed on are families. DeSantis standing up for traditional values! #AmericaNeedsChange”.

The potential effect on future elections

AI can be a powerful tool for those who want to influence election outcomes by suppressing voter knowledge and manipulating opinion about candidates. It can be used to completely change data on presiding candidates and their parties, while heavily editing significant aspects of electoral campaigns such that the opposition can gain an unfair advantage. And all this can be done in such a way that the disinformation can quickly trend on social media, yet still evade automated fact-checking systems. This happened in the 2016 US presidential election, where the data science firm Cambridge Analytica “rolled out an extensive advertising campaign to target persuadable voters based on their individual psychology” in a now infamous micro-targeting operation.

AI can also be used to aid phishing and hacking. Malicious agents can use advanced AI algorithms to craft phishing emails and bring weaknesses in election infrastructure into the spotlight, leaving electoral systems and legislation in a vulnerable position. In some extreme cases, algorithms can gain access to sensitive data, which could then be published by those wishing to do harm.

What measures are being taken to minimise electoral disinformation?

As governments and companies worldwide become more aware of the issue, measures are being taken to counter the threat. A recent mitigating strategy from social media platform X is “community notes”, whereby a team of verified accounts adds fact checks or other context to potentially misleading posts, alerting users that what they are reading or viewing may not be entirely reliable and true.

The European Union is aiming to become the world’s super-regulator in AI. In April 2021, the European Commission proposed the first EU regulatory framework for AI, aiming to analyse and classify AI systems that can be used in different applications, according to the level of risk they pose. This will help regulate the flow of disinformation during elections by halting malicious AI algorithms, in turn bringing increased confidence to EU internet users in the reliability of the information they are seeing online.

In the UK, a draft Online Safety Bill was passed in 2022 (although it still needs final confirmation), granting protection to both children and adults. The Bill states that “Category 1 services must empower their adult users with tools that give them greater control over the content that they see”, and “must remove content that is banned by their own terms and conditions”. However, the UK Government recently backed down from its attempts to use the Bill to access encrypted data, after WhatsApp threatened to withdraw its platform from the country.

And in the US, a new bill was passed by Congress in 2022, which “establishes a commission and requires other activities to support information and media literacy education and to prevent misinformation and disinformation”.

These measures show that the threat is being taken seriously by governments and media companies alike across the globe, but as technology continues to develop at speed, with new platforms and tools providing fresh opportunities for disinformation to spread, more will need to be done. With AI making disinformation more difficult to detect, there are serious implications for the forthcoming 2024 elections in a host of countries, including the US.