At the Adobe MAX 2017 conference last week, the engineers at Adobe Research showed off a deep learning system called Project Scribbler that can transform a line sketch or black-and-white portrait into a fully colorized and realistic image.

The technology utilizes Adobe Sensei artificial intelligence technology to generate a realistic color image based on a partially-colored sketch created by the user.

As with all of their "Sneaks" at Adobe MAX, Adobe describes this as an early technology, teasing that it "may or may not" become a future product or feature in an existing product like Photoshop.

In a description posted on the Adobe Research website, researchers explain that existing deep convolutional networks are able to generate realistic images, but that users have little control over what the final images look like. This makes the technology "of limited utility," according to the researchers, a reality not shared by their Scribbler network.

Unlike the aforementioned deep convolutional networks, Scribbler gives users a great deal of control over the final image it generates. This is made possible by feeding Scribbler a sketch where the lines serve as boundaries for the final image. As well, users are able to control how Scribbler colorizes the image by applying "sparse color strokes" to the sketch. These color strokes serve as a guideline so that Scribbler knows what colors to use for each part of the sketch.

Scribbler is capable of realistically coloring sketches and black-and-white photos without the aid of user-created color strokes, though, as demonstrated by Adobe researcher Jingwan Lu in the video above.

Scribbler was also demonstrated as capable of realistically applying existing textures to a sketch. To do this, the user places a texture over the part of the sketch where the desired texture should be applied. And, of course, Scribbler can apply more than one texture to different parts of the same sketch.

A product or feature based on Project Scribbler could prove useful in a variety of ways. Photographers, for example, could use this technology to rapidly colorize a black-and-white photo. Similarly, an artist could use Scribbler to flesh out a quick sketch of a planned art piece, enabling the artist or their client to see a representation of the final artwork before time is invested in creating it.

The full paper detailing the Scribbler colorizing technology is available here, the paper about Scribbler's texture control technology is available here, and, of course, you should definitely check out the demo at the top.

2017-10-25 19:50