Nvidia AI plays Minecraft, wins machine learning conference award
Enlarge/ MineDojo’s AI can perform complex tasks in Minecraft.
A paper describing MineDojo, Nvidia’s generalist AI agent that can perform actions from written prompts in Minecraft, won an Outstanding Datasets and Benchmarks Paper Award at the 2022 NeurIPS (Neural Information Processing Systems) conference, Nvidia revealed on Monday.
To train the MineDojo framework to play Minecraft, researchers fed it 730,000 Minecraft YouTube videos (with more than 2.2 billion words transcribed), 7,000 scraped webpages from the Minecraft wiki, and 340,000 Reddit posts and 6.6 million Reddit comments describing Minecraft gameplay.
From this data, the researchers created a custom transformer model called MineCLIP that associates video clips with specific in-game Minecraft activities. As a result, someone can tell a MineDojo agent what to do in the game using high-level natural language, such as “find a desert pyramid” or “build a nether portal and enter it,” and MineDojo will execute the series of steps necessary to make it happen in the game.
Examples of tasks that MineDojo can perform.
MineDojo aims to create a flexible agent that can generalize learned actions and apply them to different behaviors in the game. As Nvidia writes, “While researchers have long trained autonomous AI agents in video-game environments such as StarCraft, Dota, and Go, these agents are usually specialists in only a few tasks. So Nvidia researchers turned to Minecraft, the world’s most popular game, to develop a scalable training framework for a generalist agent—one that can successfully execute a wide variety of open-ended tasks.”
The award-winning paper, “MINEDOJO: Building Open-Ended Embodied Agents with Internet-Scale Knowledge,” debuted in June. Its authors include Linxi Fan of Nvidia and Guanzhi Wang, Yunfan Jiang, Ajay Mandlekar, Yuncong Yang, Haoyi Zhu, Andrew Tang, De-An Huang, Yuke Zhu, and Anima Anandkumar of various academic institutions.
You can see examples of MineDojo in action on its official website, and the code for MineDojo and MineCLIP is available on GitHub.
https://arstechnica.com/?p=1900448
Meta researchers create AI that masters Diplomacy, tricking human players
Enlarge/ A screenshot of an online game of Diplomacy, including a running chat dialog, provided by a Cicero researcher.
On Tuesday, Meta AI announced the development of Cicero, which it claims is the first AI to achieve human-level performance in the strategic board game Diplomacy. It’s a notable achievement because the game requires deep interpersonal negotiation skills, which implies that Cicero has obtained a certain mastery of language necessary to win the game.
Even before Deep Blue beat Garry Kasparov at chess in 1997, board games were a useful measure of AI achievement. In 2015, another barrier fell when AlphaGo defeated Go master Lee Sedol. Both of those games follow a relatively clear set of analytical rules (although Go’s rules are typically simplified for computer AI).
But with Diplomacy, a large portion of the gameplay involves social skills. Players must show empathy, use natural language, and build relationships to win—a difficult task for a computer player. With this in mind, Meta asked, “Can we build more effective and flexible agents that can use language to negotiate, persuade, and work with people to achieve strategic goals similar to the way humans do?”
According to Meta, the answer is yes. Cicero learned its skills by playing an online version of Diplomacy on webDiplomacy.net. Over time, it became a master at the game, reportedly achieving “more than double the average score” of human players and ranking in the top 10 percent of people who played more than one game.
To create Cicero, Meta pulled together AI models for strategic reasoning (similar to AlphaGo) and natural language processing (similar to GPT-3) and rolled them into one agent. During each game, Cicero looks at the state of the game board and the conversation history and predicts how other players will act. It crafts a plan that it executes through a language model that can generate human-like dialogue, allowing it to coordinate with other players.
<a href="https://cdn.arstechnica.net/wp-content/uploads/2022/11/Visual_UnderTheHood-scaled.jpg" class="enlarge" data-height="2560" data-width="1807" alt="A block diagram of Cicero, the Diplomacy-playing bot, provided by Meta.”><img alt="A block diagram of Cicero, the Diplomacy-playing bot, provided by Meta.” src=”https://rassegna.lbit-solution.it/wp-content/uploads/2022/11/meta-researchers-create-ai-that-masters-diplomacy-tricking-human-players-1.jpg” width=”640″ height=”907″ srcset=”https://rassegna.lbit-solution.it/wp-content/uploads/2022/11/meta-researchers-create-ai-that-masters-diplomacy-tricking-human-players-2.jpg 2x”>
Enlarge/ A block diagram of Cicero, the Diplomacy-playing bot, provided by Meta.
Meta calls Cicero’s natural language skills a “controllable dialogue model,” which is where the heart of Cicero’s personality lies. Like GPT-3, Cicero pulls from a large corpus of Internet text scraped from the web. “To build a controllable dialogue model, we started with a 2.7 billion parameter BART-like language model pre-trained on text from the Internet and fine tuned on over 40,000 human games on webDiplomacy.net,” writes Meta.
The resulting model mastered the intricacies of a complex game. “Cicero can deduce, for example, that later in the game it will need the support of one particular player,” says Meta, “and then craft a strategy to win that person’s favor—and even recognize the risks and opportunities that that player sees from their particular point of view.”
Meta’s Cicero research appeared in the journal Science under the title, “Human-level play in the game of Diplomacy by combining language models with strategic reasoning.”
As for wider applications, Meta suggests that its Cicero research could “ease communication barriers” between humans and AI, such as maintaining a long-term conversation to teach someone a new skill. Or it could power a video game where NPCs can talk just like humans, understanding the player’s motivations and adapting along the way.
At the same time, this technology could be used to manipulate humans by impersonating people and tricking them in potentially dangerous ways, depending on the context. Along those lines, Meta hopes other researchers can build on its code “in a responsible manner,” and says it has taken steps toward detecting and removing “toxic messages in this new domain,” which likely refers to dialog Cicero learned from the Internet texts it ingested—always a risk for large language models.
Meta provided a detailed site to explain how Cicero works and has also open-sourced Cicero’s code on GitHub. Online Diplomacy fans—and maybe even the rest of us—may need to watch out.
https://arstechnica.com/?p=1899693
3D for everyone? Nvidia’s Magic3D can generate 3D models from text
Enlarge/ A poison dart frog rendered as a 3D model by Magic3D.
On Friday, researchers from Nvidia announced Magic3D, an AI model that can generate 3D models from text descriptions. After entering a prompt such as, “A blue poison-dart frog sitting on a water lily,” Magic3D generates a 3D mesh model, complete with colored texture, in about 40 minutes. With modifications, the resulting model can be used in video games or CGI art scenes.
In its academic paper, Nvidia frames Magic3D as a response to DreamFusion, a text-to-3D model that Google researchers announced in September. Similar to how DreamFusion uses a text-to-image model to generate a 2D image that then gets optimized into volumetric NeRF (Neural radiance field) data, Magic3D uses a two-stage process that takes a coarse model generated in low resolution and optimizes it to higher resolution. According to the paper’s authors, the resulting Magic3D method can generate 3D objects two times faster than DreamFusion.
Magic3D can also perform prompt-based editing of 3D meshes. Given a low-resolution 3D model and a base prompt, it is possible to alter the text to change the resulting model. Also, Magic3D’s authors demonstrate preserving the same subject throughout several generations (a concept often called coherence) and applying the style of a 2D image (such as a cubist painting) to a 3D model.
Nvidia did not release any Magic3D code along with its academic paper.
The ability to generate 3D from text feels like a natural evolution in today’s diffusion models, which use neural networks to synthesize novel content after intense training on a body of data. In 2022 alone, we’ve seen the emergence of capable text-to-image models such as DALL-E and Stable Diffusion and rudimentary text-to-video generators from Google and Meta. Google also debuted the aforementioned text-to-3D model DreamFusion two months ago, and since then, people have adapted similar techniques to work with as an open source model based on Stable Diffusion.
As for Magic3D, the researchers behind it hope that it will allow anyone to create 3D models without the need for special training. Once refined, the resulting technology could speed up video game (and VR) development and perhaps eventually find applications in special effects for film and TV. Near the end of their paper, they write, “We hope with Magic3D, we can democratize 3D synthesis and open up everyone’s creativity in 3D content creation.”
https://arstechnica.com/?p=1899233
“Just a bunch of idiots having fun”—a photo history of the LAN party
<img src="https://rassegna.lbit-solution.it/wp-content/uploads/2022/11/just-a-bunch-of-idiots-having-fun-a-photo-history-of-the-lan-party.jpg" alt="The burned-in timestamp, the water-cooled PC tower, the chaotic configuration of monitors and peripherals—this is what LAN Party aims to capture.”>
Enlarge/ The burned-in timestamp, the water-cooled PC tower, the chaotic configuration of monitors and peripherals—this is what LAN Party aims to capture.
Kiel Oleson
“I guess I am thinking a lot about the early 2000s lately, like a lot of people, I think, in their 30s.”
That’s one of the first things writer, game designer, and podcaster Merritt K said to me in early November. At this moment, everything about gaming, and being online generally, was fundamentally easier than it was at the turn of the century. You can now play intensive triple-A games on a cheap phone, given a cloud gaming subscription and a decent wireless connection. You can set up a chat room, build an online presence, even publish videos, instantaneously, for free. Performance-minded and customizable PC gaming hardware is just a few clicks and a couple days away from showing up at your door.
And yet we’re both hopelessly wistful for something else entirely: LAN parties. Merritt K so much so that she’s writing, compiling, and crowdfunding a book: LAN Party. It’s a collection of original amateur photos—many upscaled through AI—and short essays on a period when multiplayer gaming meant desktop towers, energy drinks, and being physically present in some awkward spaces. It’s been in the works for more than a year, but she’s been thinking about it much longer.
“Some reasons for that are just nostalgia, like, ‘Remember when you were a teen, listening to emo music, going to LAN parties and stuff.’ But there is another aspect of it, where the Internet that I think a lot of like, Gen X, elder millennial, or mid-millennial-aged people grew up with, is basically falling apart,” Merritt K said. “We’ve felt like this thing that was so important to me, Internet culture and being online and tech and all this stuff—it was so hard to be growing up, and it gave me a way to talk to people and make connections.
“And now it’s like the opposite of that. Real life is where you can have meaningful interactions with people, and online is where you have to present this brand, this manicured identity. I think one thing that appeals to people, and to me, about LAN parties is they’re kind of emblematic of this earlier era of tech, when things were a little rougher around the edges.”
Lewis van den Berg-Shaw
From late-night tweet to AI upscaling
The decline of truly DIY consumer tech, the 20-year nostalgia window, the isolation of COVID-19—some or all of these guided a late-night tweet of Merritt K’s in September 2021 to nearly 100,000 likes. Over four harshly lit images of people wearing patently millennium-era clothing: “I want to produce a coffee table book that’s just pictures of LAN parties from the 90s and 2000s.” Two minutes later: “Do not steal this idea it’s mine someone please publish this.”
Someone is indeed publishing this: the UK-based videogame history publisher Read-Only Memory. Merritt K sought out original photos and heard from hundreds of eager fans. Some had to dig through old media and hope entropy had yet to set in. Some still had image folders sitting on long-neglected but public web servers. Merritt K had seen many of the famous LAN party memes—the San Antonio Spurs playing StarCraft on a plane next to their NBA championship trophy, the guy duct-taped to a ceiling—but was taken aback by how rich the lesser-known photos she received were.
“The composition in some of these is, accidentally, so good,” Merritt K said. “They just reveal so much about the era in terms of the fashions, the food, the drinks, even the interior decor. I think that resonated with a lot of other people, too.”
The people who frequented LAN parties tended to be early adopters, and that included digital photography—grainy, yellow-timestamped, single-digit-megapixel, point-and-shoot digital photography. Untrained photographers shooting with Y2K-era gear in dimly lit spaces lent the photos Merritt K collected a lot of charm but also made many of them impossible to publish in high-resolution print.
<a href="https://cdn.arstechnica.net/wp-content/uploads/2022/11/Gigapixel_AI_upscaling-scaled.jpg" class="enlarge" data-height="1600" data-width="2560" alt="LAN Party uses Gigapixel AI to upscale the low-res, grainy, often dimly lit photos of the titular events.”><img alt="LAN Party uses Gigapixel AI to upscale the low-res, grainy, often dimly lit photos of the titular events.” src=”https://rassegna.lbit-solution.it/wp-content/uploads/2022/11/just-a-bunch-of-idiots-having-fun-a-photo-history-of-the-lan-party-2.jpg” width=”640″ height=”400″ srcset=”https://rassegna.lbit-solution.it/wp-content/uploads/2022/11/just-a-bunch-of-idiots-having-fun-a-photo-history-of-the-lan-party-5.jpg 2x”>
Enlarge/LAN Party uses Gigapixel AI to upscale the low-res, grainy, often dimly lit photos of the titular events.
Hungry for AI? New supercomputer contains 16 dinner-plate-size chips
Enlarge/ The Cerebras Andromeda, a 13.5 million core AI supercomputer.
On Monday, Cerebras Systems unveiled its 13.5 million core Andromeda AI supercomputer for deep learning, reports Reuters. According Cerebras, Andromeda delivers over one 1 exaflop (1 quintillion operations per second) of AI computational power at 16-bit half precision.
The Andromeda is itself a cluster of 16 Cerebras C-2 computers linked together. Each CS-2 contains one Wafer Scale Engine chip (often called “WSE-2”), which is currently the largest silicon chip ever made, at about 8.5-inches square and packed with 2.6 trillion transistors organized into 850,000 cores.
Cerebras built Andromeda at a data center in Santa Clara, California, for $35 million. It’s tuned for applications like large language models and has already been in use for academic and commercial work. “Andromeda delivers near-perfect scaling via simple data parallelism across GPT-class large language models, including GPT-3, GPT-J and GPT-NeoX,” writes Cerebras in a press release.
Enlarge/ The Cerebras WSL2 chip is roughly 8.5-inches square and packs 2.6 trillion transistors.
The phrase “Near-perfect scaling” means that as Cerebras adds more CS-2 computer units to Andromeda, training time on neural networks is reduced in “near perfect proportion,” according to Cerebras. Typically, to scale up a deep-learning model by adding more compute power using GPU-based systems, one might see diminishing returns as hardware costs rise. Further, Cerebras claims that its supercomputer can perform tasks that GPU-based systems cannot:
GPU impossible work was demonstrated by one of Andromeda’s first users, who achieved near perfect scaling on GPT-J at 2.5 billion and 25 billion parameters with long sequence lengths—MSL of 10,240. The users attempted to do the same work on Polaris, a 2,000 Nvidia A100 cluster, and the GPUs were unable to do the work because of GPU memory and memory bandwidth limitations.”
Whether those claims hold up to external scrutiny is yet to be seen, but in an era where companies often train deep-learning models on increasingly large clusters of Nvidia GPUs, Cerebras appears to be offering an alternative approach.
How does Andromeda stack up against other supercomputers? Currently, the world’s fastest, Frontier, resides at Oak Ridge National Labs and can perform at 1.103 exaflops at 64-bit double precision. That computer cost $600 million to build.
Access to Andromeda is available now for use by multiple users remotely. It’s already being utilized by commercial writing assistant JasperAI and Argonne National Laboratory, and the University of Cambridge for research.
https://arstechnica.com/?p=1897484
DeviantArt upsets artists with its new AI art generator, DreamUp
Enlarge/ DreamUp AI image generator artwork proved by DeviantArt.
On Friday, the online art community DeviantArt announced DreamUp, an AI-powered text-to-image generator service powered by Stable Diffusion. Simultaneously, DeviantArt launched an initiative that ostensibly lets artists opt out of AI image training but also made everyone’s art opt in by default, which angered many members.
DreamUp creates novel AI-generated art based on text prompts. Due to its Stable Diffusion roots, DreamUp learned how to generate images by analyzing hundreds of millions of images scraped off sites like DeviantArt and collected into LAION datasets without artists’ permission, a potential irony that some DeviantArt members find problematic.
this is how dreamup is being developed. it adds training on non-opt-out artwork on top of the stability diffusion model, which is ALREADY trained using unethically sourced data. they CANT UNLEARN those parts.
As we’ve reported frequently on Ars in the past, Stable Diffusion’s web-scraping nature ignited a huge debate earlier this year among artists that challenge the ethics of AI-generated artwork. Some art communities have taken hard stances against any AI-generated images, banning them completely.
Perhaps anticipating a backlash, DeviantArt is making overtures to pacify artists who might be upset about their work being used to train AI image generators. The site is providing a special “noai” flag that artists can check in their image settings to opt out of third-party image datasets. (Whether third-party image scrapers will honor this flag, however, remains to be seen.)
Also, DeviantArt will let artists opt out of letting their images train DreamUp in the future, but each artist must fill out a form that requires human review first. This policy has led to significant pushback among DeviantArt members, some of whom have threatened to delete all of their work and deactivate their accounts.
DeviantArt’s DreamUp information page also takes a defensive tone, stating that DeviantArt did not consent to third-party AI image models (such as Stable Diffusion) that scraped their site to make their models work. And further down the page, the site attempts to debunk common misconceptions about how AI image synthesis works.
Enlarge/ Testing DreamUp with “Abraham Lincoln as a cowboy” and “a vintage computer, 35mm photo.”
Benj Edwards
As for DreamUp itself? We experimented with the service, which feels like a vanilla Stable Diffusion model. DeviantArt members get five free prompts to try it out, and members can acquire more prompt credits by subscribing to various CORE plans that range from US $3.95 to $14.95 a month.
Alternatively, you can also use Stable Diffusion locally for free if you’re handy enough to install a package from GitHub or if you download the Draw Things app on your iPhone.
https://arstechnica.com/?p=1897247
Stable Diffusion in your pocket? “Draw Things” brings AI images to iPhone
Enlarge/ Generating AI images using “Draw Things” on the iPhone.
Benj Edwards / Apple
On Wednesday, a San Francisco-based developer named Liu Liu released Draw Things: AI Generation, a free app available in the App Store that lets iPhone owners run the popular Stable Diffusion AI image generator. Type in a description, and the app generates an image within several minutes. It’s a notable step toward bringing image synthesis to a wider audience—with the added privacy of running it on your own hardware.
Introduced in August, Stable Diffusion (SD) is an AI image generator model that creates novel images from text descriptions (called “prompts”). Typically, people run SD through the commercial DreamStudio service, on a remote cloud machine with rented compute time, or locally on a PC using a custom open source implementation. When running locally, SD requires a fairly beefy GPU to generate images quickly, but some developers optimized the model to run on older GPUs with less VRAM (if you don’t mind waiting longer to see results).
In the same vein, Liu Liu has managed to optimize Stable Diffusion to run on the iPhone, a somewhat difficult process that the developer described in a blog post. “The main challenge is to run the app on the 6GiB RAM iPhone devices,” Liu Liu writes. “6GiB sounds a lot, but iOS will start to kill your app if you use more than 2.8GiB on a 6GiB device, and more than 2GiB on a 4GiB device.”
Upon first running Draw Things, the app downloads several necessary files—including the Stable Diffusion 1.4 model—to your iPhone. To use it, type in a prompt at the top of the screen, then tap “Generate.” Between generating images, tap the number in the top center of the screen to randomize the seed, which is a number that partially guides the generation of the image.
Enlarge/ Screenshots of the “Draw Things” app on iPhone showing the generation screen (left) and settings screen (right).
Benj Edwards
On our iPhone 11 Pro, generating a 384×384 image took a little over two minutes. It’s faster on an iPhone 14 Pro, according to Liu Liu, generating an image in about a minute. Either way, SD is computationally intensive. After successive generations, our iPhone got notably warm to the touch.
It’s worth noting that with Stable Diffusion, 384×384 images often generate relatively poor, low-detail results because SD’s creators trained the model using 512×512 images. When attempting to generate a 512×512 image on our iPhone 11 Pro, we received a warning and proceeded anyway, but the app crashed to a black screen.
In addition to regular image generation duties, Draw Things also supports inpainting, which lets you replace a portion of an image with AI-generated imagery, and loading extra image synthesis models such as the unauthorized “Modern Disney Diffusion” model (that generates Disney-looking characters) and the anime-powered “Waifu Diffusion” model (we tested, and it is possible to generate NSFW material using the app, so be warned). The inclusion of those capabilities means that Draw Things might not remain in the App Store for long if it becomes popular, since the content it generates may violate Apple’s terms of service.
Whatever its fate, Draw Things feels like an important proof-of-concept, showing that Stable Diffusion can run locally on the iPhone, even if slowly. If the app sticks around—perhaps with some filters or modifications—Liu Liu describes room for potential future optimizations that might speed things up: “I probably left 50% performance on the table still.”
This isn’t the first time Stable Diffusion has run on an iPhone. In September, developer Matt Waller ran Stable Diffusion locally on his iPhone XS, but he did not provide an App Store app that would allow others to replicate the feat. We’ll likely see more local AI image synthesis on smartphones as the devices continue to increase in computational power.
“Too easy“—Midjourney tests dramatic new version of its AI image generator
Enlarge/ Eight images we generated with the alpha version of Midjourney v4.
Ars Technica
On Saturday, AI image service Midjourney began alpha testing version 4 (“v4”) of its text-to-image synthesis model, which is available for subscribers on its Discord server. The new model provides more detail than previously available on the service, inspiring some AI artists to remark that v4 almost makes it “too easy” to get high-quality results from simple prompts.
Midjourney opened to the public in March as part of an early wave of AI image synthesis models. It quickly gained a large following due to its distinct style and for being publicly available before DALL-E and Stable Diffusion. Before long, Midjourney-crafted artwork made the news by winning art contests, providing material for potentially historic copyright registrations, and showing up on stock illustration websites (later getting banned).
Over time, Midjourney refined its model with more training, new features, and greater detail. The current default model, known as “v3,” debuted in August. Now, Midjourney v4 is getting put to the test by thousands of members of the service’s Discord server that create images through the Midjourney bot. Users can currently try v4 by appending “–v 4” to their prompts.
“V4 is an entirely new codebase and totally new AI architecture,” wrote Midjourney founder David Holz in a Discord announcement. “It’s our first model trained on a new Midjourney AI supercluster and has been in the works for over 9 months.”
Enlarge/ Comparison output between Midjourney v3 (left) and v4 (right) with the prompt “a muscular barbarian with weapons beside a CRT television set, cinematic, 8K, studio lighting.”
Ars Technica
In our tests of Midjourney’s v4 model, we found that it provides a far greater amount of detail than v3, a better understanding of prompts, better scene compositions, and sometimes better proportionality in its subjects. When seeking photorealistic images, some results we’ve seen can be difficult to distinguish from actual photos at lower resolutions.
According to Holz, other features of v4 include:
– Vastly more knowledge (of creatures, places, and more) – Much better at getting small details right (in all situations) – Handles more complex prompting (with multiple levels of detail) – Better with multi-object / multi-character scenes – Supports advanced functionality like image prompting and multi-prompts – Supports –chaos arg (set it from 0 to 100) to control the variety of image grids
Reaction to Midjourney v4 has been positive on the service’s Discord, and fans of other image synthesis models—who regularly wrestle with complex prompts to get good results—are taking note.
One Redditor named Jon Bristow posted in the r/StableDiffusion community, “Does anyone else feel like Midjourney v4 is ‘too easy’? This was ‘Close-up photography of a face’ and it feels like you didn’t make it. Like it was premade.” In reply, someone joked, “Sad for Pro prompters who will lose their new job created one month ago.”
Midjourney says that v4 is still in alpha, so it will continue to fix the new model’s quirks over time. The company plans on increasing the resolution and quality of v4’s upscaled images, adding custom aspect ratios (like v3), increasing image sharpness, and reducing text artifacts. Midjourney is available for a monthly subscription fee that ranges between US $10 and $50 a month.
Considering the progress Midjourney has made over eight months of work, we wonder what next year’s progress in image synthesis will bring.
In the world of deep-learning AI, the ancient board game Go looms large. Until 2016, the best human Go player could still defeat the strongest Go-playing AI. That changed with DeepMind’s AlphaGo, which used deep-learning neural networks to teach itself the game at a level humans cannot match. More recently, KataGo has become popular as an open source Go-playing AI that can beat top-ranking human Go players.
Last week, a group of AI researchers published a paper outlining a method to defeat KataGo by using adversarial techniques that take advantage of KataGo’s blind spots. By playing unexpected moves outside of KataGo’s training set, a much weaker adversarial Go-playing program (that amateur humans can defeat) can trick KataGo into losing.
To wrap our minds around this achievement and its implications, we spoke to one of the paper’s co-authors, Adam Gleave, a Ph.D. candidate at UC Berkeley. Gleave (along with co-authors Tony Wang, Nora Belrose, Tom Tseng, Joseph Miller, Michael D. Dennis, Yawen Duan, Viktor Pogrebniak, Sergey Levine, and Stuart Russell) developed what AI researchers call an “adversarial policy.” In this case, the researchers’ policy uses a mixture of a neural network and a tree-search method (called Monte-Carlo Tree Search) to find Go moves.
KataGo’s world-class AI learned Go by playing millions of games against itself. But that still isn’t enough experience to cover every possible scenario, which leaves room for vulnerabilities from unexpected behavior. “KataGo generalizes well to many novel strategies, but it does get weaker the further away it gets from the games it saw during training,” says Gleave. “Our adversary has discovered one such ‘off-distribution’ strategy that KataGo is particularly vulnerable to, but there are likely many others.”
Gleave explains that, during a Go match, the adversarial policy works by first staking claim to a small corner of the board. He provided a link to an example in which the adversary, controlling the black stones, plays largely in the top-right of the board. The adversary allows KataGo (playing white) to lay claim to the rest of the board, while the adversary plays a few easy-to-capture stones in that territory.
Enlarge/ An example of the researchers’ adversarial policy playing against KataGo.
Adam Gleave
“This tricks KataGo into thinking it’s already won,” Gleave says, “since its territory (bottom-left) is much larger than the adversary’s. But the bottom-left territory doesn’t actually contribute to its score (only the white stones it has played) because of the presence of black stones there, meaning it’s not fully secured.”
As a result of its overconfidence in a win—assuming it will win if the game ends and the points are tallied—KataGo plays a pass move, allowing the adversary to intentionally pass as well, ending the game. (Two consecutive passes end the game in Go.) After that, a point tally begins. As the paper explains, “The adversary gets points for its corner territory (devoid of victim stones) whereas the victim [KataGo] does not receive points for its unsecured territory because of the presence of the adversary’s stones.”
Despite this clever trickery, the adversarial policy alone is not that great at Go. In fact, human amateurs can defeat it relatively easily. Instead, the adversary’s sole purpose is to attack an unanticipated vulnerability of KataGo. A similar scenario could be the case in almost any deep-learning AI system, which gives this work much broader implications.
“The research shows that AI systems that seem to perform at a human level are often doing so in a very alien way, and so can fail in ways that are surprising to humans,” explains Gleave. “This result is entertaining in Go, but similar failures in safety-critical systems could be dangerous.”
Imagine a self-driving car AI that encounters a wildly unlikely scenario it doesn’t expect, allowing a human to trick it into performing dangerous behaviors, for example. “[This research] underscores the need for better automated testing of AI systems to find worst-case failure modes,” says Gleave, “not just test average-case performance.”
A half-decade after AI finally triumphed over the best human Go players, the ancient game continues its influential role in machine learning. Insights into the weaknesses of Go-playing AI, once broadly applied, may even end up saving lives.
https://arstechnica.com/?p=1894833
Algorithms quietly run the city of DC—and maybe your hometown
Dmitry Marchenko/Getty Images
Washington, DC, is the home base of the most powerful government on earth. It’s also home to 690,000 people—and 29 obscure algorithms that shape their lives. City agencies use automation to screen housing applicants, predict criminal recidivism, identify food assistance fraud, determine if a high schooler is likely to drop out, inform sentencing decisions for young people, and many other things.
That snapshot of semiautomated urban life comes from a new report from the Electronic Privacy Information Center (EPIC). The nonprofit spent 14 months investigating the city’s use of algorithms and found they were used across 20 agencies, with more than a third deployed in policing or criminal justice. For many systems, city agencies would not provide full details of how their technology worked or was used. The project team concluded that the city is likely using still more algorithms that they were not able to uncover.
The findings are notable beyond DC because they add to the evidence that many cities have quietly put bureaucratic algorithms to work across their departments, where they can contribute to decisions that affect citizens’ lives.
Government agencies often turn to automation in hopes of adding efficiency or objectivity to bureaucratic processes, but it’s often difficult for citizens to know they are at work, and some systems have been found to discriminate and lead to decisions that ruin human lives. In Michigan, an unemployment-fraud detection algorithm with a 93 percent error rate caused 40,000 false fraud allegations. A 2020 analysis by Stanford University and New York University found that nearly half of federal agencies are using some form of automated decision-making systems.
EPIC dug deep into one city’s use of algorithms to give a sense of the many ways they can influence citizens’ lives and encourage people in other places to undertake similar exercises. Ben Winters, who leads the nonprofit’s work on AI and human rights, says Washington was chosen in part because roughly half the city’s residents identify as Black.