In January, my coworker received a peculiar email. The message, which she forwarded to me, was from a handful of corporate Walmart employees calling themselves the “Concerned Home Office Associates.” (Walmart’s headquarters in Bentonville, Arkansas, is often referred to as the Home Office.) While it’s not unusual for journalists to receive anonymous tips, they don’t usually come with their own slickly produced videos.

The employees said they were “past their breaking point” with Everseen, a small artificial intelligence firm based in Cork, Ireland, whose technology Walmart began using in 2017. Walmart uses Everseen in thousands of stores to prevent shoplifting at registers and self-checkout kiosks. But the workers claimed it misidentified innocuous behavior as theft and often failed to stop actual instances of stealing.

They told WIRED they were dismayed that their employer—one of the largest retailers in the world—was relying on AI they believed was flawed. One worker said that the technology was sometimes even referred to internally as “NeverSeen” because of its frequent mistakes. WIRED granted the employees anonymity because they are not authorized to speak to the press.

The workers said they had been upset about Walmart’s use of Everseen for years and claimed colleagues had raised concerns about the technology to managers but were rebuked. They decided to speak to the press, they said, after a June 2019 Business Insider article reported Walmart’s partnership with Everseen publicly for the first time. The story described how Everseen uses AI to analyze footage from surveillance cameras installed in the ceiling and can detect issues in real time, such as when a customer places an item in their bag without scanning it. When the system spots something, it automatically alerts store associates.

“Everseen overcomes human limitations. By using state-of-the-art artificial intelligence, computer vision systems, and big data, we can detect abnormal activity and other threats,” a promotional video referenced in the story explains. “Our digital eye has perfect vision, and it never needs a day off.”

In an effort to refute the claims made in the Business Insider piece, the Concerned Home Office Associates created a video, which purports to show Everseen’s technology failing to flag items not being scanned in three different Walmart stores. Set to cheery elevator music, it begins with a person using self-checkout to buy two jumbo packages of Reese’s White Peanut Butter Cups. Because the packages are stacked on top of each other, only one is scanned, but both are successfully placed in the bagging area without issue.

The same person then grabs two gallons of milk by their handles and moves them across the scanner with one hand. Only one is rung up, but both are put in the bagging area. They then put their own cell phone on top of the machine, and an alert pops up saying they need to wait for assistance—a false positive. “Everseen finally alerts! But does so mistakenly. Oops again,” a caption reads. The filmmaker repeats the same process at two more stores, where they fail to scan a heart-shaped Valentine’s Day chocolate box with a puppy on the front and a Philips Sonicare electric toothbrush. At the end, a caption explains that Everseen failed to stop more than $100 of would-be theft.

False positives

The video isn’t definitive proof that Everseen’s technology doesn’t work as well as advertised, but its existence speaks to the level of frustration felt by the group of anonymous Walmart employees, and the lengths they went to prove their objections had merit.

In interviews, the workers, whose jobs include knowledge of Walmart’s loss-prevention programs, said their top concern with Everseen was false positives at self-checkout. The employees believe that the tech frequently misinterprets innocent behavior as potential shoplifting, which frustrates customers and store associates, and leads to longer lines. “It’s like a noisy tech, a fake AI that just pretends to safeguard,” said one worker.

The coronavirus pandemic has given their concerns more urgency. One Concerned Home Office Associate said they worry false positives could be causing Walmart workers to break social-distancing guidelines unnecessarily. When Everseen flags an issue, a store associate needs to intervene and determine whether shoplifting or another problem is taking place. In an internal communication from April obtained by WIRED, a corporate Walmart manager expressed strong concern that workers were being put at risk by the additional contact necessitated by false positives and asked whether the Everseen system should be turned off to protect customers and workers.

Before COVID-19, “it wasn’t ideal, it was a poor customer experience,” the worker said. “AI is now creating a public health risk.” (HuffPost reported last week that corporate Walmart employees were concerned about Everseen’s technology putting store associates at risk amid the pandemic.)

Good for sales

When COVID-19 reached the United States, Americans rushed to stock up on food and household essentials at Walmart, and sales soared. Workers soon began falling sick; at least 20 Walmart associates have now died after contracting the coronavirus, according to United for Respect, a nonprofit that advocates for retail workers and that is crowdsourcing COVID-19 infection rates and working conditions at Walmart stores across the country. Last month, United for Respect said hundreds of Walmart employees participated in a national strike demanding safer working conditions and better benefits.

A spokesperson for Walmart said the company has been working diligently to protect customers and its workforce and believes the rate at which associates have contracted COVID-19 is lower than that of the general US population. They denied that false positives caused by Everseen were a widespread issue and said the company had not considered turning the system off due to concerns about COVID-19.

“We assess our technology regularly, and as evident with the large scale implementation of Everseen across the chain, we have confidence it is currently meeting our standards,” the spokesperson said in an email. Just prior to the start of the pandemic, Walmart said it made significant improvements to its Everseen system, which resulted in fewer alerts overall. The spokesperson declined to answer questions about what the updates may have entailed.

The spokesperson also noted that there are a number of different reasons an associate might intervene during a self-checkout transaction, like when a customer has problems with their credit card. The company said it has taken a number of steps to ensure people are protected during these interactions, including regularly cleaning self-checkout kiosks and providing employees with protective equipment. In addition, workers are given handheld devices that allow them to handle most interventions from a distance, the company said.

Questions

Everseen declined to answer questions about its technology. In a statement, a spokesperson said the company “accurately and effectively identifies potential theft [sic] is why retailers have successfully deployed it at thousands of locations to date, with many more installations planned.” They added that Everseen typically accounts only for less than 10 percent of total interventions at self-service checkouts. In a separate statement, the spokesperson said “Everseen is committed to helping its customers deliver the best possible experience for shoppers and store associates, especially during the COVID-19 pandemic. Self-checkout offers the benefits of a generally contactless shopping experience, allowing for proper social distancing and avoiding manned-lanes in busy stores with limited staff available.”

But the Concerned Home Office Associates said their worries about Everseen long predate the pandemic. Emails obtained by WIRED show that other corporate employees raised issues about the technology failing to prevent theft in both 2017 and 2018. The employees said they were particularly vexed by Walmart’s continued investment in Everseen because NCR Corporation, which makes the majority of Walmart’s registers, had acquired an Everseen competitor called StopLift. They considered the acquisition an endorsement and were confused as to why StopLift’s technology wasn’t being further explored.

What’s more, the workers said an internal Walmart research and development group, the Intelligent Retail Lab (IRL), created its own anti-theft software they believed was more accurate than Everseen’s, according to information they were given internally. One Walmart employee said the technology, the existence of which was previously reported by The Wall Street Journal, is now being tested in roughly 50 stores.

Walmart declined to answer questions about its internal anti-theft software but did not dispute WIRED’s reporting. “At an enterprise level, there are a number of tests happening at any given time across our footprint of nearly 5,000 stores,” a spokesperson said in a statement. “The goal of IRL is to build AI capabilities that can be transferred to additional stores. We regularly test capabilities built internally in a small number of stores.”

Rising shrink

Everseen’s technology was designed in part to help solve a persistent problem with self-checkout. While allowing customers to scan and pay for their own items cuts down on labor costs for retailers, it has also led to more inventory loss, or “shrinkage,” due to shoplifting, employee theft, and other problems. “Theft through self-checkout lanes is exponentially higher than through traditional checkout lanes,” says Christopher Andrews, a sociology professor at Drew University and the author of The Overworked Consumer: Self-Checkouts, Supermarkets, and the Do-It-Yourself Economy.

In the past, Walmart and other retailers relied on weight sensors to prevent shoplifting through self-checkout, but those were prone to error and frustrated customers. Some stores are now turning instead to firms like Everseen, which promise to reduce shrink and increase customer satisfaction by relying instead on surveillance cameras and machine vision. Everseen has said that it works with a number of major retailers. Amazon uses similar technology in its Amazon Go convenience stores, where a network of cameras automatically log the products customers take. (Amazon is now licensing its “Just Walk Out” tech to other companies.)

Value of self-checkout

During the coronavirus pandemic and its aftermath, self-checkout may become even more important for stores, as customers look for low-risk ways to shop. NCR corporation said it’s now helping retailers modify its equipment to be as touchless as possible: for example, by reconfiguring machines so that customers can insert a debit or credit card without needing to press the “credit card” payment option. “It is fascinating to see self-checkout become poised as a public health strategy, in addition to things like cashless payment,” says Alexandra Mateescu, a researcher at the nonprofit institute Data & Society, who has written about the effects of new technology on retail workers.

“Self-checkout is just one of the ways that we’ve offered customers solutions to get the items they need safely during this time, in addition to other options like delivery, pickup, touchless payment at the register and shopping online,” the Walmart spokesperson said in a statement. “Customers are using this option now, as much as ever, and we will continue to work hard to ensure the in-store experience for our customers is safe, affordable and convenient, as well as safe for our associates.”

This story originally appeared on wired.com.

https://arstechnica.com/?p=1679858