Facebook removes accounts with AI-generated profile photos

Facebook on Friday removed what it called a global network of more than 900 accounts, pages, and groups from its platform and Instagram that allegedly used deceptive practices to push pro-Trump narratives to about 55 million users. The network used fake accounts, artificial amplification, and, notably, profile photos of fake faces generated using artificial intelligence to spread polarizing, predominantly right-wing content around the Web, including on Twitter and YouTube.

It represents an alarming new development in the information wars, as it appears to be the first large-scale deployment of AI-generated images in a social network. In a report on the influence operation, researchers from disinformation groups Graphika and DFRLab noted that this was the first time they had seen the technology used to support an inauthentic social media campaign.

The images were used to enhance the authenticity of some of the 610 Facebook accounts, 156 groups, 89 pages, and 72 Instagram accounts associated with “The BL,” a digital news outlet that described itself as a “pure mountain spring, moistening the heart of every reader” on Facebook before it was removed. Facebook’s investigation connected The BL to The Epoch Times, a conservative media organization with ties to Chinese spiritual group Falun Gong and a history of aggressive support for Donald Trump.

Ostensibly a US-based media organization, The BL network’s pages were operated by users in Vietnam and the US, who, Facebook says, made widespread use of fake accounts to evade detection and funnel traffic to its own websites. A report by Graphika and DFRLab researchers found that the majority of The BL network’s fake accounts were used to maintain a cluster of over 80 groups and pages promoting President Trump. The accounts acting as administrators for these pages—which boasted names such as “America Needs President Trump,” “Trump for America’s President,” and “WE STAND WITH TRUMP & PENCE!”—were predominantly fake and created in Vietnam.

“Fake accounts served as the administrators of Facebook groups, increased the membership numbers of those groups, liked posts on the Pages, and posted large quantities of content from TheBL-related assets,” the report notes. “This structure constituted a large-scale artificial amplification factory whose only observable function was to boost content from TheBL and, to a lesser extent, from the Epoch Times.”

Many of these accounts used profile pictures that researchers said appeared to be created using neural networks trained on images of real human faces to create photos of people who did not actually exist. Researchers from Graphika and DFRLab found 18 images of faces in profile photos used by The BL network that they said had likely been generated using generative adversarial networks.

While the technology is capable of creating passable fake faces, it has a few obvious tells that tipped the researchers off. It works largely by mimicking visual patterns but doesn’t understand human anatomy and struggles with backgrounds and symmetrical features, like glasses or earrings.

The above image, from the Graphika report, was taken from the profile photo for “Alfonzo Macias,” an admin for one of the groups in The BL network. “Note the asymmetry in the glasses and also the poorly defined background,” the report says. “The authors checked this image with experts at the University Federico II of Naples, who assessed that this image was GAN-generated with 99.9 percent certainty.”

This story originally appeared on wired.com.

https://arstechnica.com/?p=1636319