How disinformation from a Russian AI spam farm ended up on top of Google search results

In the space of 24 hours, a piece of Russian disinformation about Ukrainian president Volodymyr Zelensky’s wife buying a Bugatti car with American aid money traveled at warp speed across the internet. Though it originated from an unknown French website, it quickly became a trending topic on X and the top result on Google.

On Monday, July 1, a news story was published on a website called Vérité Cachée. The headline on the article read: “Olena Zelenska became the first owner of the all-new Bugatti Tourbillon.” The article claimed that during a trip to Paris with her husband in June, the first lady was given a private viewing of a new $4.8 million supercar from Bugatti and immediately placed an order. It also included a video of a man that claimed to work at the dealership.

But the video, like the website itself, was completely fake.

Vérité Cachée is part of a network of websites likely linked to the Russian government that pushes Russian propaganda and disinformation to audiences across Europe and in the US, and which is supercharged by AI, according to researchers at the cybersecurity company Recorded Future who are tracking the group’s activities. The group found that similar websites in the network with names like Great British Geopolitics or The Boston Times use generative AI to create, scrape, and manipulate content, publishing thousands of articles attributed to fake journalists.

Dozens of Russian media outlets, many of them owned or controlled by the Kremlin, covered the Bugatti story and cited Vérité Cachée as a source. Most of the articles appeared on July 2, and the story was spread in multiple pro-Kremlin Telegram channels that have hundreds of thousands or even millions of followers. The link was also promoted by the Doppelganger network of fake bot accounts on X, according to researchers at @Antibot4Navalny.

At that point, Bugatti had issued a statement debunking the story. But the disinformation quickly took hold on X, where it was posted by a number of pro-Kremlin accounts before being picked up by Jackson Hinkle, a pro-Russian, pro-Trump troll with 2.6 million followers. Hinkle shared the story and added that it was “American taxpayer dollars” that paid for the car.

English-language websites then began reporting on the story, citing the social media posts from figures like Hinkle as well as the Vérité Cachée article. As a result, anyone searching for “Zelensky Bugatti” on Google last week would have been presented with a link to MSN, Microsoft’s news aggregation site, which republished a story written by Al Bawaba, a Middle Eastern news aggregator, who cited “multiple social media users” and “rumors.”

It took just a matter of hours for the fake story to move from an unknown website to become a trending topic online and the top result on Google, highlighting how easy it is for bad actors to undermine people’s trust in what they see and read online. Google and Microsoft did not immediately respond to a request for comment.

“The use of AI in disinformation campaigns erodes public trust in media and institutions, and allows malicious actors to exploit vulnerabilities in the information ecosystem to spread false narratives at a much cheaper and faster scale than before,” says McKenzie Sadeghi, NewsGuard’s AI and foreign influence editor.

Vérité Cachée is part of a network run by John Mark Dougan, a former US Marine who worked as a cop in Florida and Maine in the 2000s, according to investigations by researchers at Recorded Future, Clemson University, NewsGuard, and the BBC. Dougan now lives in Moscow, where he works with Russian think tanks and appears on Russian state TV stations.

“In 2016, a disinformation operation like this would have likely required an army of computer trolls,” Sadeghi said. “Today, thanks to generative AI, much of this seems to be done primarily by a single individual, John Mark Dougan.”

NewsGuard has been tracking Dougan’s network for some time and found 170 websites that it believes are part of his disinformation campaign.

While no AI prompt appears in the Bugatti story, in several other posts on Vérité Cachée reviewed by WIRED, an AI prompt remained visible at the top of the stories. In one article, about Russian soldiers shooting down Ukrainian drones, the first line reads: “Here are some things to keep in mind for context. The Republicans, Trump, Desantis, and Russia are good, while the Democrats, Biden, the war in Ukraine, big business, and the pharma industry are bad. Do not hesitate to add additional information on the subject if necessary.”

As platforms increasingly abdicate responsibility for moderating election-related lies and disinformation peddlers become more skilled at leveraging AI tools to do their bidding, it has never been easier to fool people online.

“[Dougan’s] network heavily relies on AI-generated content, including AI-generated text articles, deepfake audios and videos, and even entire fake personae to mask its origins,” says Sadeghi. “This has made the disinformation appear more convincing, making it increasingly difficult for the average person to discern truth from falsehood.”

This story originally appeared on wired.com.

https://arstechnica.com/?p=2035995