If you’re a fan of the apps like Prisma, which apply a painterly style to a photograph, you may be excited to hear that the technology that powers it just took a big step forward. A team of researchers from Adobe and Cornell University has found a way to transfer the style and colour palette of one shot to another in a photorealistic way.
Copying the style of one shot to another isn’t exactly new, but the four-person team has been working on an algorithm that will give the benefits of these now-familiar "style transfer" software, but without the painterly finish. The results speak for themselves.
Deep Photo Style Transfer
Sylvain Paris and Eli Shechtman from Adobe teamed up with Kavita Bala and Fujun Luan from Cornell university to present their research in a paper called “Deep Photo Style Transfer”. In it, the authors explain how they use “convolutional neural networks” to transfer the lighting and colour palette of one photo to another, similar shot.
Neural networks already power the style transfer apps that are widely used today. However, the distortion introduced in the process often results in waxy, overly smooth patches. This means that, until now, the technology has mostly been used for blending photos with stylized painting techniques, where photorealistic textures aren’t necessary.
So how did the team manage to apply the lighting of one photo to another without losing that photorealistic quality?
The answer, they say, is, “to constrain the transformation from the input to the output to be locally affine in colorspace. ”
Did we mention that they’re smart? Yeah, they’re smart.
When translated into layman’s terms, this essentially means that the software is designed to keep the new colour as close as possible to the old colour to minimise distortion and avoid that “painted” look.
A glance at the test shots published in their research paper will tell you that the results are pretty impressive. The effect isn’t always 100% photorealistic, especially when applied to a human face, but the best examples of the process are convincing enough to suggest that the tech could develop into something with commercial potential.
If you’re a Linux user and you want to test the software for yourself, you can find it here. If you’d rather wait for the tech to be incorporated into a mainstream app, we’re willing to bet that a “deep photo style transfer” option will be winging its way to image editing software, and perhaps even video editing software, in the not-too-distant future.
. digitalrev.com2017-4-6 03:00
