With Microsoft’s focus on making the Surface Duo impossibly thin, the company’s ceiling for camera tech improvements based on raw hardware seems fairly limited. Fortunately, the mind behind Google’s industry-leading Pixel camera magic doesn’t see that as a problem in the future for something like the Duo.
A couple of weeks ago, Marc Levoy joined editor-in-chief of The Verge, Nilay Patel on the publication’s podcast The Vergecast, to discuss how computational photography raises a bar that sheer camera spec’d brute force cannot.
The conversation happened on the eve of the Duo release and neither Patel nor Levoy make any direct mentions of Microsoft’s efforts, but Levoy’s explanation of “diminishing returns due to the laws of physics” seems increasingly poignant in light of Microsoft dropping a single 11-megapixel camera into a $1,400 flagship. Microsoft has promised improvements to the camera quality of the Duo over time and that pledge echoes a Levoy sentiment, that with an adequate sensor and some creative computational camera tech, any company can overcome most of the pitfalls that come with creating a photography experience with limited physical space.
As I said, the signal to noise ratio is basically a matter of the total sensor size. If you want to put 96 megapixels and you can’t squeeze a larger sensor physically into the form factor of the phone, then you have to make the pixels smaller, and you end up close to the diffraction limit and those pixels end up worse. They are noisier. It’s just not clear how much advantage you get.
There might be a little bit more headroom there. Maybe you can do a better job of de-mosaicing — meaning computing the red, green, blue in each pixel — if you have more pixels, but there isn’t going to be that much headroom there. Maybe the spec on the box attracts some consumers. But I think, eventually, like the megapixel war on SLRs, it will tone down, and people will realize that’s not really an advantage.
With that being said, the question arises, just how important is the physical camera in a world of computational photography? Thankfully, Levoy has departed from Google on a mission to democratize his Pixel camera experience with the help of Adobe as the two work on a Universal Camera App. The goal of Levoy’s universal camera app is to “leave the creative decisions in the hands of the photographers, more so than the phone vendors have been doing in their software.” Furthermore, Levoy wants to use all the machine learning the cloud can provide to circumvent the mega-pixel crutch some device makers use to checkbox camera innovation year over year.
Adobe already offers a camera app with Photoshop Camera as well as a solution in Lightroom but with Levoy’s background and previous career, accomplishments have many believing that a more robust effort could make its way to Adobe users and even as a potentially stand-alone app to others.
Perhaps, more importantly, Levoy sees the next battleground for mobile cameras occurring with vide. As apps such as TikTok and Instagram increase their daily usage numbers by creating more interactive video experiences, Levoy may have a point. While Instagram began as mostly a still images sharing site, it has increasingly leaned toward animations, augmented reality, and short-form videos to keep pace with competitors who are zeroing in on the evolving ephemeral sharing and entertainment behaviors of users. Exacerbating the need to focus on video has been the surge in video conferencing software use that has occurred amid the global pandemic substituting in-person engagements.
In this regard, mobile devices that can keep up with and match user demands for such video engagement could be seen as more useful per use-case than more traditional point and shoot experiences, especially if stills can be extracted from videos at near-identical quality as industry-leading cameras.
How does any of this help the Surface Duo, well Microsoft’s Surface team has already begun to deploy similar Levoy tactics of cloud processing over a pure mega-pixel approach to its current camera. The latest Duo software update managed to add subtle saturation and a mild HDR level color correction that noticeably improved images captured from the 11 megapixels shooter. The camera on the Duo is a far cry from being competitive at the moment but Levoy’s proven with the Pixel that with an off-the-shelf sensor and clear focus on photography tenants, that the Surface Duo’s camera isn’t being held back by lack of space or necessarily its sensor, but lack of creativity.
