5D Mark IV “Dual Pixel Raw” – Focus Overlay

Canon’s new 5D Mark IV has been announced and includes a feature that’s got our attention. The Dual Pixel Raw format allows for new types of post capture analysis, it’s early days but so far looks very interesting!

Dual Pixel Raw

Apart from the usual incremental improvements on the 5D line of cameras, Canon have introduced something they’re calling “Dual Pixel Raw”.  I won’t explain in detail as there are plenty of other resources available on the internet which have all the pretty diagrams and explanations, but the short version is that each pixel of the sensor is split in half so it can look at light coming from a slightly different direction to the other half.  While this has been present in earlier Canon cameras, there’s never been a way to see anything other than the combined image data.  Dual Pixel Raw stores information in the raw file which allows recovery of the light data from each side of the pixels.  This doesn’t sound much, but opens up the possibilities for a few funky tricks.  As well as the Canon touted features (image refocusing, bokeh shift and ghosting reduction), the guys at RawDigger have been showing that there’s potentially an extra stop of information in the images if you process them properly.

Experimenting…

I wanted to have a play with the data in a Dual Pixel Raw file.  Taking what is becoming the current Dual Pixel Raw test file from Kamera & Bild, I’ve used a development version of FoCal to play with the contents.

As detailed in the RawDigger post, the file contains two Raw images – one is the A+B image which is the combination of both halves of the pixels, and the other is the A image which is the information from a single side.  By reading this raw data, I can apply FoCal’s processing algorithms to extract individual channel data, then using a bit of maths and some curve fitting functions  it’s possible to effectively run phase-detect autofocus at any point in the Dual Pixel Raw image – the results are both magnitude of focus offset and direction, just like “proper” phase-detect AF.

From the raw data, it’s possible to plot the focus offset like this:

The focus detection algorithm I’ve written is pretty crude, but it does highlight the sort of information contained in the files.

Green is perfect focus – not the results of any autofocus operation, but real, perfect focus.  The colour of the overlay shifts towards blue for back-focus and red for front-focus.  You can see the figures in the foreground are green, and the detail towards the very bottom of the frame is shifting towards yellow (and ultimately red) indicating front focus at these points.  The background of the image clearly shifts towards blue.

Like current in-camera phase detect autofocus, you can’t determine focus information for very low contrast areas (which is why there’s quite a bit of uncoloured image), but anywhere there is contrast in the image should be able to give a result.

What’s Next?

Dual Pixel Raw has some really interesting possibilities outside of the scope of those things Canon have implemented in DPP.  We may potentially be able to use it to speed up FoCal’s automatic autofocus calibration, or at the very least to verify the validity of the suggested results.  There’s also good possibilities for extending the profiling capabilities of certain lens characteristics.  I don’t want to say too much at the moment as there’s a strong possibilities the range and resolution of focus information extractable from Dual Pixel Raw files won’t be particularly useful in the context of FoCal, but as soon as we have the 5D Mark IV we will be investigating.

We’ll be getting our hands on a 5D Mark IV as soon as they are available in the UK (hopefully tomorrow!), with a view to adding automated support to FoCal once there is Canon support software available.