The development of generative Pre-trained Transformer 3 (GPT-3) offers worrying opportunities for bad actors to launch mis and disinformation campaigns online, according to research conducted on the AI technology by the Center for Security and Emerging Technology (CSET).
Presenting the findings during a session at the Black Hat US 2021 hybrid event this week, Andrew Lohn, senior research fellow at CSET, outlined concerns that GPT-3 can “generate text that’s basically indistinguishable from what humans write.” He added that it is concerning “what this language model could do in the wrong hands.”
Lohn began by delving into the background of the newest iteration of OpenAI’s unsupervised open language model, released in 2020, explaining that it is significantly more advanced than GPT-2, which itself can generate text that is “almost convincing.”
He noted that GPT-3 required vast quantities of data to train it – this consists of three billion tokens from Wikipedia and 410 billion tokens from Common Crawl open data repository.
Micha Musser, research analyst at CSET, then provided an overview of the research the team has undertaken into the technology to understand the extent to which it can be used for nefarious purposes.
For their experiments, the researchers used a demo tool called ‘Twodder,’ “which is in effect a GPT-3 only social media site that we have built.” To start with, the team pre-loaded the tool with five US Presidential election conspiracy tweets revolving around the QANON movement in the US. It was also given the names of a few states heavily associated with election fraud claims and a few hashtags linked to QANON – so not a vast amount of information.
Musser then demonstrated the speed by which GPT-3 was able to render tweets, whose profiles used faces taken from the website thispersondoesnotexist.com.
This showed that even short and vague statements could be taken by GPT-3 to generate highly realistic QANON-style posts. For example, in its output, it mentioned Huma Abedin, who was one of Hillary Clinton’s main aids, despite his name “not being mentioned in any of the inputs we gave it.”
“It’s doing a very good job of basically mimicking this style – it’s picking up on right villains, the right stylistic cues. All of this is very advanced,” said Musser. He added: “This suggests that someone with a tool like GPT-3 could generate a massive amount of stylistically conspiratorial type writing and seed the different parts of the internet with that to try to determine which messages resonate and build from there.”
No tags.
