What ‘The Social Dilemma’ misunderstands about social networks

I. A dilemma

On Sunday night, after being encouraged to by friends and family, I hit play on a new documentary about our digital lives. Directed by Jeff Orlowski, The Social Dilemma explores the effect of smartphones and social networks on human behavior. Blending talking-head interviews with some well known Silicon Valley apostates and fictional, after-school special-style dramatizations of what happens when Johnny and Janey scroll through feeds all day, the film presents itself as an urgent warning about our modern condition.

I’m more than a little sympathetic to these concerns. I started writing this newsletter in 2017 after coming to the belated realization that social networks really did have an outsized impact on modern life, and deserved to be taken as seriously. My thinking has benefited tremendously from speaking over the years with some of the interview subjects in the film, including Tristan Harris, Renee DiResta, Tim Kendall, Jeff Seibert, and Justin Rosenstein. In particular, Harris’ work on screen time triggered a powerful sea change in the industry, and DiResta’s explorations of misinformation have been essential to helping social networks understand themselves.

And yet despite all that … the film is ridiculous? The dramatized segments include a fictional trio of sociopaths working inside an unnamed social network to design bespoke push notifications to distract their users. They show an anguished family struggling to get the children to put their phones away during dinner. And the ominous piano score that pervades every scene, rather than ratcheting up the tension, gives it all the feeling of camp. If someone asked me to reimagine this newsletter as a drag show, I would start where The Social Dilemma leaves off.

And as Adi Robertson points out at The Verge, the idea that algorithmic recommendation engines are at the heart of our troubles leaves out vast swathes of the internet that are arguably just as important as the big social networks, and perhaps in some cases even more so. She writes:

Propaganda, bullying, and misinformation are actually far bigger and more complicated. The film briefly mentions, for instance, that Facebook-owned WhatsApp has spread misinformation that inspired grotesque lynchings in India. The film doesn’t mention, however, that WhatsApp works almost nothing like Facebook. It’s a highly private, encrypted messaging service with no algorithmic interference, and it’s still fertile ground for false narratives. As Alexis Madrigal notes, condemning the platforms together comes “uncomfortably close to admitting that mobile communications pose fundamental challenges to societies across the world.” There’s a fair case for that, he argues — but a case with much more alarming implications.

Radicalization doesn’t just happen on Facebook and YouTube either. Many of the deadliest far-right killers were apparently incubated on small forums: Christchurch mosque killer Brenton Tarrant on 8chan; Oregon mass shooter Chris Harper-Mercer on 4chan; Tree of Life Synagogue killer Robert Bowers on Gab; and Norwegian terrorist Anders Breivik on white supremacist sites including Stormfront, a 23-year-old hate site credited with inspiring scores of murders.

These sites aren’t primarily driven by algorithms or profit motives. Instead, they twist and exploit the open internet’s positive ability to connect like-minded people. When harmful content surfaces on them, it raises complex moderation questions for domain hosts and web infrastructure providers — a separate set of powerful companies that have completely different business models from Facebook.

This isn’t to let social networks off the hook. Nor is it an effort to make the problem feel so complicated that everyone just throws their hands up and walks away from it. But I’m shocked at how appealing so many people find the idea that social networks are uniquely responsible for all of society’s ills. (The Social Dilemma has been among the 10 most watched programs on Netflix all week.)

This cartoon super villain view of the world strikes me as a kind of mirror image of the right-wing conspiracy theories which hold that a cabal of elites are manipulating every world event in secret. It is more than a little ironic that a film that warns incessantly about platforms using misinformation to stoke fear and outrage seems to exist only to stoke fear and outrage — while promoting a distorted view of how those platforms work along the way.

Some folks who worked on the film told me that this kind of approach is necessary to “communicate in a way that appeals to a broad audience.” But I say that’s a cop out. If you’re going to argue that social platforms are uniquely responsible for the fraying of society, you have to show your work.

II. A memo

On the other hand, meet Sophie Zhang. She was a data scientist who was fired in August and left this month in the fashion increasingly popular among departing Facebook employees — which is to say, quite dramatically.

Craig Silverman, Ryan Mac, and Pranav Dixit scooped her 6,600-word farewell memo in BuzzFeed. Zhang wrote:

“In the three years I’ve spent at Facebook, I’ve found multiple blatant attempts by foreign national governments to abuse our platform on vast scales to mislead their own citizenry, and caused international news on multiple occasions,” wrote Zhang, who declined to talk to BuzzFeed News. Her LinkedIn profile said she “worked as the data scientist for the Facebook Site Integrity fake engagement team” and dealt with “bots influencing elections and the like.”

“I have personally made decisions that affected national presidents without oversight, and taken action to enforce against so many prominent politicians globally that I’ve lost count,” she wrote.

She added: “I know that I have blood on my hands by now.”

Unlike the Social Dilemma filmmakers, Zhang showed her work — first to her bosses, and then, inadvertently, to the world. She describes governments in Azerbaijan and Honduras using Facebook against their own citizens, employing large numbers of fake accounts to promote their own interests and attack critics. And she found what she described as coordinated influence campaigns in countries including India, Ukraine and Bolivia.

Zhang’s official job was to police Facebook for “fake engagement” — people buying inauthentic likes, comments, and shares. From this perch she continually wandered into an adjacent realm that Facebook calls “civic integrity,” to the apparent frustration of her bosses. It’s a higher-stakes realm that works on some of the most pressing issues a social platform will face, including foreign influence operations and election integrity. It’s also famed for its difficulty — academics tell me that unearthing these operations and properly attributing them requires significant domain expertise. Many of the people who do this at Facebook and other networks previously worked for US intelligence agencies.

Zhang, by contrast, was a relatively junior employee who was essentially moonlighting on civic integrity issues. That may have been one reason why she struggled to get her colleagues’ attention, I’m told. Everyone I’ve spoken to at Facebook over the past day says Zhang was bright and dedicated to her work. But navigating large organizations can be a challenge even for the most senior employees, and it seems like Facebook’s sheer size often prevented Zhang’s findings from getting prompt attention.

After BuzzFeed’s story ran, some people who work on the company’s integrity team — and there are more than 200 of them — were frustrated at the implication that they are sitting on their hands all day, or otherwise bad at their jobs. (I don’t think Zhang meant to imply this, but that was certainly the tenor of the discussion about BuzzFeed’s story on Twitter.) Many of them had worked with Zhang on the takedown efforts she described, and felt undermined by her memo, I’m told. Sometimes team leaders set priorities differently than their own employees would, and Facebook’s efforts — which focus on the largest and most active networks, particularly during elections — sometimes set aside other legitimate threats, like the ones Zhang had found.

Ultimately, that’s the aspect of Zhang’s memo that sticks. Facebook mostly doesn’t deny that her findings were accurate, significant, and sometimes received delayed responses. The company says only that the issues she found, however significant, were less pressing than the many other issues the civic integrity team was policing at the time, in other countries all over the world.

Mastering the geopolitics of each country and rooting out every influence operation that pops up while also policing hate speech and misinformation while promoting free speech and interpersonal connections is a mind-bendingly enormous task. But it’s also the task that Facebook, by virtue of its huge investment in growth and fighting off competitors over the years, has signed up for.

I can’t take seriously a film like The Social Dilemma, which seemingly wants to hold one company accountable for every change society has undergone since it was founded. But when someone takes her employer to task for the things she found on its service — and she leaves with a feeling of blood on her hands — that’s something different.

Not every issue raised by an employee will get immediate attention. But Zhang’s memo raises questions about Facebook’s size, power, and accountability to its users — particularly its non-Western users — that outsiders have been poking at for years. Increasingly, as we have learned from a summer rife with Facebook leaks, those calls are now coming from inside the house. And they deserve better answers than Sophie Zhang has gotten to date.

The Ratio

Today in news that could affect public perception of the big tech platforms.

Trending up: More than 400,000 people have registered to vote on Snapchat. The app has rolled out a series of voting tools, including a feature that lets users register to vote in the app. (Makena Kelly / The Verge)

Trending up: Facebook is investing $5 million in local news organizations that serve historically marginalized communities. It’s specifically partnering with newsrooms led by people of color. (Facebook)

Trending sideways: Facebook pledged to reach “net zero” carbon emissions, offsetting its global power consumption by investing in renewable energy projects that capture and store carbon dioxide. That puts Facebook in the middle of the pack when it comes to climate commitments from Big Tech. (Justine Calma / The Verge)

Governing

⭐ TikTok is set to become a standalone company as part of the Oracle deal. ByteDance will be the majority shareholder of the new entity, if the deal goes through. James Fontanella-Khan and Miles Kruppa at the Financial Times have the details:

The creation of a new entity shows how ByteDance is attempting to put distance between the company’s Chinese ownership and TikTok’s operations, even as it seeks to avoid the full sale of the app that Mr Trump had desired. […]

The new TikTok entity will have independent oversight and will be managed at arms-length from ByteDance, said people briefed on the discussions. The Chinese company will continue to collaborate with TikTok globally and will retain control of the powerful algorithm that keeps users engaged by predicting what sort of videos they will enjoy.

How did Oracle position itself to become TikTok’s trusted tech partner? The negotiations bore little resemblance to regular deal talks, this piece says. They were driven by US-China tensions, commercial rivalries, and President Trump’s personal interventions. (Georgia Wells, Aaron Tilley and John D. McKinnon / The Wall Street Journal)

ByteDance CEO Yiming Zhang decided to not sell TikTok to Microsoft over concerns from major investors about the financial hit of selling the app for less than it is worth. Zhang opted for a sale of only a stake to Oracle, rather than an outright divestment. (Echo Wang, Keith Zhai and Joshua Franklin / Reuters)

A TikTok ban won’t prevent employees from getting paid, the Trump administration says. The news comes in response to a lawsuit from a TikTok worker over Trump’s executive order banning the app. (Steven Musil / CNET)

The Trump campaign released a “support our troops” ad on September 11th that used a stock photo of Russian-made fighter jets and Russian models dressed as soldiers. Come on! (Daniel Lippman and Bryan Bender / Politico)

The Health and Human Services official who made false accusations about the Centers for Disease Control harboring a “resistance unit” to undermine President Trump indicated he might be stepping down from his role. He apologized to his staff for remarks he made on Facebook. (Adam Cancryn, Dan Diamond and Sarah Owermohle / Politico)

Industry

⭐Kim Kardashian West is temporarily leaving Facebook and Instagram to support the Stop Hate for Profit campaign. Kardashian West is the 7th most followed account on Instagram with 188 million followers, and she’s taking Wednesday off! OK, Kim. Axios reports on what she said:

“I love that I can connect directly with you through Instagram and Facebook, but I can’t sit by and stay silent while these platforms continue to allow the spreading of hate, propaganda and misinformation – created by groups to sow division and split America apart – only to take steps after people are killed,” Kardashian West wrote.

“Misinformation shared on social media has a serious impact on our elections and undermines our democracy. Please join me tomorrow when I will be “freezing” my Instagram and FB account to tell Facebook to #StopHateForProfit.”

Facebook launched automatic closed captions for Facebook Live and Workplace Live in six languages, to help people who are deaf or hard of hearing. The company said the move is particularly important during quarantine, when people rely on the platform to tune in to broadcasts and government briefings. (Facebook)

Douyin, the Chinese-market version of TikTok, has more than 600 million daily active users. It had 400 million at the start of the year. (Zheping Huang / Bloomberg)

TikTok’s most popular creator, Charli D’Amelio, joined rival app Triller in a non-exclusive deal. She and her family are trying out the app, but will continue to post content on TikTok. (Julia Alexander / The Verge)

“M to the B” is the song of the summer thanks to a TikTok revival. Also, here’s a TikTok meme power ranking. (Rebecca Jennings / Vox)

Gen Zers are using Discord to collaborate on projects they hope will shape the future of Silicon Valley. They’re using the Discord server Gen Z Mafia to upend a system they say is exclusive and elitist. (Taylor Lorenz / The New York Times)

Amazon launched Luxury Stores on its mobile app, with Oscar de la Renta as its first brand partner. More upscale brands are expected to join the new platform in the coming weeks. (Nicole Phelps / Vogue)

Zoom is working on a major upgrade to its messaging capabilities, in a move that would challenge Slack. The video conferencing tool currently has rudimentary text chatting options. (Kevin McLaughlin / The Information)

Twitch is experimenting with running ads automatically in the middle of streams. The ads will run across affiliate and partnered channels and creators will get paid for every ad that runs. (Bijan Stephen / The Verge)

Casino-style smartphone games have drained people of millions of dollars. Unlike the gambling industry, this market is almost entirely unregulated. (Cyrus Farivar / NBC)

And finally…

Talk to us

Send us tips, comments, questions, and Social Dilemma reviews: casey@theverge.com and zoe@theverge.com.

https://www.theverge.com/interface/2020/9/16/21437942/social-dilemma-netflix-review-orlowski-sarah-zhang-memo-facebook-buzzfeed