Rosalie Yu turns her sweet tooth into a virtual reality art form

It sometimes seems like technology is at odds with the art world — a tension between brain and heart. But plenty of artists, from Da Vinci to Cory Arcangel, have proved that’s not true and continue to prove it as technology evolves. In Technographica, we explore how contemporary artists are using technology in unusual and unexpected ways.

For two years, Rosalie Yu documented everything sweet that she ate. Before she consumed a berry macaroon or a crème brûlée muffin or her friend Shannon’s birthday cake, Yu took a series of iPhone photos of the dessert from all angles, which she would use to create a 3D scan and eventually turn more than 200 desserts into a 360-degree virtual reality experience. “Sometimes I’ve ended up in weird situations for the sake of capturing a dessert,” she told The Verge. “Once, I was at the movies and I had to bring an ice cream sandwich with me to the restroom so I could scan it.”

The project, titled A Ritual of Habits, is composed of two parts. First, what Yu calls “virtual theater,” which consisted of a VR experience in which the viewer is transplanted into a world of sweets, and an augmented reality experience in which the sweets. Second, it’s an installation of 3D-printed objects.

Yu says the project was inspired by the way she says eating sweets was habitually passed down through generations of her family. “At some point, I noticed how much sugar we ate compared to other households,” she told The Verge. “When we visited my grandfather in the hospital, my mother would open a big bag of treats and lay them on the table. An assortment of Twix, Snickers, those cloying sweet instant coffees, and a random assortment of cookies that she had collected during the week.” After her grandfather passed away, Yu noticed the same sweets were being served at his funeral, and she decided to start documenting how she interacted with sugar.

The process of creating the VR experience was lengthy. After taking 20–50 close-up photos of a pastry, Yu would transfer photos into the desktop photogrammetry app Agisoft PhotoScan to generate a 3D model of the food. Then she used the digital sculpting tool ZBrush to make repairs after a scan and to apply texture. After the 3D models were finished, Yu used ARKit and Unity to build the AR and VR experiences.

“I’ve been fascinated by the history of Dutch sugar plantations in colonial Taiwan and the decadent representation of sweets in Dutch still-life paintings,” she said. “Photogrammetry enabled me to reflect on the global history of sugar alongside my own personal history and habits.”

But photogrammetry also has its downsides, according to Yu. Photogrammy scans won’t work with objects that have reflective surfaces on them, which means treats with glazes, egg washes, and jellies were out. Even ice cream wouldn’t work. “I also started noticing that the technology was conditioning my preferences — glossy and shiny treats now translate to distorted and broken scans in my head,” she says.

I visited Yu at the experimental cultural center Pioneer Works in Red Hook, Brooklyn, where she was finishing up a residency. The VR setup is simple: a desktop computer and an HTC Vive headset. (She also has an Oculus.) In A Ritual of Habits, the viewer gets a sense of Yu’s process: different sweets are shown from all angles until they encircle the viewer in the final scene. At one point, you find yourself inside a lemon tart, as it squishes and deflates around you. Her friend and producer Matthew Dougherty designed the sound. The AR experience allows users to see the treats on the floor in front of them, using a smartphone.

“Sugar is representative of physical and moral decay, but I preserved these desserts through my ritual of photogrammetry — the mindless consumption, the ephemeral pleasure — all captured as a lasting subject of reflection,” Yu told The Verge.

Yu’s sculptures of the sweets were 3D printed with stereolithography. A raw digital file goes into the printer, which hardens liquid resin with a laser light. The resin goes through an isopropyl alcohol bath and is baked under a UV light, like gel nails at a salon. The result is a series of near-translucent versions of pies, and donuts, and macaroons with one bite taken out.

Much of Yu’s work relies on 3D models and canvases. Yu’s other work includes a live 3D mapping of cities and a series of 3D self-portraits. “My work with photogrammetry tries to represent a transitional state between 2D representation and the physical world,” she says.

Yu says her work is informed by limitations in available technology, and she sees 3D scanning as currently occupying a similar space as early photography. “The way a chemical image takes shape on photo paper is similar to the software’s process of stitching photos, connecting a point cloud, and reconstructing a mesh in 3D space,” she says. “You can’t immediately see the results of photogrammetry, just like traditional photographs that had to be developed in a dedicated space like darkroom. There is something magical and lonesome about both of these expansive processes.”

“I eat fewer desserts now.”

Photography by Amelia Holowaty Krales / The Verge

https://www.theverge.com/2018/2/17/16990752/rosalie-yu-virtual-reality-art-ritual-of-habits-sweets