A Swedish machine learning researcher named Emil Wallner has released a free web tool called Palette.fm that automatically colorizes black-and-white photos using AI. After uploading a photo, users can choose a color filter or refine the colors using a written text description.

Palette.fm uses a deep learning model to classify images, which guides its initial guesses for the colors of objects in a photo or illustration. We asked Wallner what kind of back-end technology runs the site, but he didn’t go into specifics. “I’ve made a custom AI model that uses the image and text to generate a colorization,” Wallner replied. “One model creates the text and the other takes the image and the text to generate the colorization.”

After you upload an image, the site’s sleek interface provides an estimated caption (description) of what it thinks it sees in the picture. If you don’t like any of the preset color filters, you can click the pencil icon to edit the caption yourself, which guides the colorization model using a text prompt.

To test it, we took a photo of a small pumpkin and removed the color using Photoshop. Then we uploaded the black-and-white version and experimented with selecting the pre-made filters that Palette.fm provides. Once we found a good filter, we edited the caption to refine the colors by describing the objects in the scene. For example, Palette.fm originally thought the pumpkin was a “claw” and didn’t recognize the sidewalk. But once we put those terms in the written prompt, the colors made more sense. We further refined the image later (not pictured in the example below) by specifying “green leaves” in the background.

For now, Palette.fm is available as a free service, but Wallner plans on adding a paid option. The site processes the images online, in the cloud. As far as the privacy of the uploaded photos is concerned, the Palette.fm site reads, “We don’t store your images.” But as with any cloud service, take that with a grain of salt regarding private photos. Refreshingly, Palette.fm does not require any kind of user account registration at the moment.

So far, Palette.fm has resonated with people on Hacker News who have used the currently free tool to colorize photos of beloved relatives and historical photographs.

Update:

It’s worth pointing out that the concept of colorizing photos with AI is not new to Palette.fm, including a “colorize” neural filter in Photoshop. Wallner himself has been working on AI-powered colorization for five years, he says. A variation of Wallner’s colorization technology has also been available as a bot on Twitter since late last year.

https://arstechnica.com/?p=1892071