Nearly every smartphone manufacturer claims their newest device takes the best photos. They'll cite different qualities, such as resolution, low light performance, autofocus, stabilization, and lens zoom.
To help cut through the noise, popular YouTuber Marques Brownlee has published his third annual 'Blind Smartphone Camera Test. '
While the test is not wholly scientific, and it's not aiming to be, it does help shed light on what real-world users care about when it comes to smartphone cameras. The test includes 20 new smartphones. Brownlee and the rest of the MKBHD team seeded each phone and then assigned them a letter for the purposes of a blind bracket.
The first round test image featured Marques as the subject and the scene includes many bright, warm colors and a bright blue sky in the background. It's a strong test for skin tone rendition, dynamic range and saturation levels.
Every phone was used to capture images of the same scene. The team selected scenes that are like what most people capture with their smartphone, including human subjects, bright colors, and challenging lighting.
Using his massive social media following, Brownlee solicited votes from followers on Twitter and Instagram to select which of the two presented images was the best. After more than 10M votes were tallied, the tournament was completed, and a surprising winner was crowned.
Bracket-style competition has some shortcomings. For example, a camera that otherwise might have made it far in one seeding scenario could be eliminated in the first round by the eventual winner. Speaking of potential favorites being eliminated early, the iPhone 12 Pro Max was clobbered in the first round by the OnePlus 8T, a mid-range smartphone. The next round saw the OnePlus 8T suffer the biggest defeat in Blind Smartphone Camera Test history, gaining only 5% of the vote against the Note20 Ultra. Including the iPhone SE and 12 Pro Max this year, no iPhone has ever made it out of the first round in the three blind tests.
The second round shot, shown here as captured by the Mate 40 Pro smartphone, includes bright colors and a candle on the table. The candle was the bellwether for this round's winners. If an image retained detail in the candle better than its competitor, voters preferred it.
Crowning a winner is only one thing you can learn from the test. When putting a pair of photos of the same subject side by side, it's interesting to consider which differences make the winners stand out. Time and again, saturation matters. If an image looks paler, a bit less vibrant, it loses. Brownlee thinks this and a cooler white balance is what has undone iPhone competitors each year. More accurate or realistic color rendition doesn’t necessarily translate to the general population considering an image to be better.
White balance also proved important to voters. In the first round, with a human subject and warm colors in the scene, the iPhone 12 Pro Max captured the image with the coolest white balance. Images with cool white balance lost basically across the board.
The third round image, shown here as captured by the Xiaomi MI 10 Ultra, highlighted differences between smartphones with respect to depth of field, white balance and dynamic range.
Dynamic range was an important factor to voters as well. In many matchups, the image with more dynamic range won. That said, different factors took priority in different scenes. When a person is a subject, colors and skin tones mattered a lot, understandably. In a still life scene, such as the second round shot of the plate of food, dynamic range took precedence.
As we can see in this screenshot from Brownlee, Twitter (left) and Instagram (right) depicted the same image files in different ways. The white balance and saturation levels are not even close to identical.
There's also something to be learned about images on Instagram versus Twitter. In the final round, the warmer image won on Instagram but was crushed by its competition on Twitter. As it turns out, the images look very different on the two platforms. It's possible this difference impacted the results in each round.
To view more from Marques Brownlee, check out his YouTube channel. If you want to watch previous Blind Smartphone Camera Test videos, click here for 2019 and here for the 2018 edition.
. dpreview.com2020-12-5 21:14