Snapchat’s parent company, Snap, yesterday suspended two apps that allowed users to send anonymous messages to other users on the platform. The move came in response to a lawsuit filed Monday against Snap and the two messaging apps.
The lawsuit seeks class-action status to represent all 92 million Snapchat users, and it demands that Snap ban both Yolo and LMK from its app store. The developers of both apps, the suit alleges, did not implement adequate safeguards against harassing and bullying behavior.
The suit was brought by Kristin Bride, the mother of Carson Bride, a 16-year-old who suffered from cyberbullying on the Yolo and LMK apps. Over half the messages he received on Yolo were “meant to humiliate him, often involving sexually explicit and disturbing content,” according to the lawsuit. After a particularly personal string of insults, 16-year-old Carson searched in vain for how to reveal the identity of his bullies. Just over two weeks later, he took his own life. His last search was “reveal Yolo username online.”
Both Yolo and LMK rely on Snap Kit, a platform that allows developers to tap into Snapchat-specific functionality like logins, Bitmoji avatars, stickers, and various app-specific content features. Of the two, Yolo, a reference to the phrase “you only live once,” has garnered the most attention. First released in May 2019, the app shot to the top of the charts in the week after its launch. In February 2020, the company raised $8 million. “What we are trying to build is a new kind of network where there’s a fluidity to identity,” Yolo co-founder Greg Henrion told Tech Crunch at the time. “We weren’t sure if Yolo was here to stay, but we’re still ranking well and there seems to be a real opportunity in anonymity starting with Snapchat Q&A.”
Yolo users could post questions to their Snap accounts for other users to answer anonymously, and Yolo later added a group chat feature allowing users to invite friends or friends of friends. In group chat mode, users were represented by their Bitmoji avatars rather than usernames, and once per hour, users could use a “Superpower” function to send completely anonymous messages to the entire group.
Other anonymous chat apps like YikYak folded after becoming littered with hateful speech. But years after those apps shut down, Yolo’s developers persisted, apparently thinking they had the problem solved. As part of the sign-up process, Yolo included a pop-up warning users, “YOLO has no tolerance for objectionable content or abusive users. You’ll be banned for any inappropriate usage.” The company initially outsourced moderation to Two Hat, though its unclear if that arrangement remains in place.
“We’re strict on moderation. When looking at the reviews about bullying, it’s like nothing compared to any other anonymous app. I think we solved 90% of the problem,” Henrion said in the Tech Crunch interview.
Either Henrion overestimated his company’s mastery of the problem or the remaining 10 percent was skewed toward bullying and other harmful messages. Archived app store reviews for the iOS version show that users were complaining of bullying behavior as recently as last Friday. “I keep getting bullied on here and every time I try blocking the person who is sending nasty messages it doesn’t work!!” one review read. We’ve reached out to Yolo and Snap for comment and will update this article if we hear from them.
The lawsuit alleges that the Yolo and LMK apps were “dangerous and defective,” a tack that attempts to work around Section 230 to hold platforms and app developers accountable. It’s an approach that the 9th Circuit Court of Appeals seemed to endorse in a decision issued last week. That case was brought against Snap by the parents of two deceased teenagers who died in a high-speed crash while using the Snapchat Speed Filter in an attempt to unlock a “reward” the teens thought existed for exceeding 100 mph.
“In short, Snap, Inc. was sued for the predictable consequences of designing Snapchat in such a way that it allegedly encouraged dangerous behavior. Accordingly, the panel concluded that Snap, Inc. did not enjoy immunity from this suit under § 230(c)(1) of the CDA,” the appellate court said in its opinion, which reversed a lower court decision dismissing the case.
The new suit against Snap, Yolo, and LMK also argues that the app-makers did not fulfill their own terms of service, a strategy that relies on recent decisions relating to consumer protection law. Though no court has ruled in favor of this argument yet, one recent ruling in a case against the dating app Grindr seemed to open the door.
Yolo may be particularly at risk here. The company made claims that it had “no tolerance” for abusive users, but when Bride’s parents attempted to contact the company months after their son’s death to try to unmask his bullies, the email Yolo provided for law enforcement cases bounced. Bride’s parents tried again, this time using the customer support email. The company has not responded.
Update: A Snap spokesperson sent the following statement to Ars after publication: “In light of the serious allegations raised by the lawsuit, and out of an abundance of caution for the safety of the Snapchat community, we are suspending both YOLO and LMK’s Snap Kit integrations while we investigate these claims.”
Correction: Yolo uses Hive for image moderation, not text moderation, which it outsourced to Two Hat. This post has been updated to reflect that.
If you or someone you know is feeling suicidal or in distress, please call the Suicide Prevention Lifeline number: 1-800-273-TALK (8255), which will put you in touch with a local crisis center.
https://arstechnica.com/?p=1764515