Comparison of Demosaicing Methods available in Free, Open Source Raw Processors

My previous article included a table listing the various demosaicing algorithms offered by the four raw processors considered and I wondered why we (as users) needed such a wide choice. The table is reproduced below.I decided to investigate those offered by RawTherapee by looking closely at the detail in an image of tree branches against the sky – the same part of the same raw file processed by each of the algorithms. This was based on a suggestion by Elle Stone (http://ninedegreesbelow.com) whose articles on FLOSS (Free Libre Open Source Software) raw processors are well worth reading. The idea is to compare the performance of the different algorithms with a good clean image and with a very noisy image. As we use different cameras (Pentax and Canon) we should see if this also makes a difference.

Pentax K20D plus Sigma 17-70mm f/2.8-4.5 DC Macro f/8.0 1/200s iso100 f=36.88mm. Image is 14.6 MP (4680×3116)

This is the original image. I chose to use RawTherapee as this offers nine different algorithms and a convenient zoom window to see part of the image at 400% or higher. No input profile was applied, working colour space was set to ProPhoto and monitor profile set to the system monitor profile (recently calibrated and profiled). No sharpening or other processing apart from demosaicing was applied.

In viewing the images which follow, the fact that diagonal lines are not reproduced as a simple “staircase” in black and white might be thought of as a defect of the lens (or of the photographer, or of the autofocus system) but, it can easily be shown that the sharpness seen in these samples is close to the theoretical “best” that can be expected at an aperture of f/8 and a sensor 23.5mm wide and 4680 pixels wide. (These parameters relate to the Pentax K20D used). The limiting factor is diffraction. See e.g. “The Manual of Photography” Eighth Edition (Focal Press) p64 (Formerly “The Ilford Manual of Photography”).

For a lens focused at or near infinity, the diameter, D, of the Airy disc (the image of a point source of light) is given by: D = 2.44 λN

Where λ = the wavelength of the light and N = the f number. So, putting the values in, we have: D = 2.44 x 550 x 8 nanometers (λ = 550nm) which gives approximately 0.01mm.

The sensor in the K20D (on its longest side) has 4680 pixels in 23.5mm, so each pixel is:  23.5 ÷ 4680 = 0.005mm.

Conclusion: At f/8, the Airy disc has a diameter equal to 2 pixels. i.e. a point source of light (having zero size) will be reproduced as a disc about 2 pixels in diameter. It follows that a diagonal straight line transition from white to black will have a “softness” of about 2 pixels – and this is what we see in the images which follow.

Note that this is pure physics – there is nothing that Pentax, Canon or Nikon can do about it!

Amaze

The images which follow are self-documenting in that the panel on the right shows the Demosaicing Method and the number of False Color Suppression Steps.

With the default setting of amaze (above), some false colour is evident but with just one suppression step we get the image below.

And with two steps of suppression almost no colour.

It is probably best to use the minimum number of steps necessary as we do not know what “damage” each step does to the image. In this context it is useful to think in terms of signal (wanted) and noise (unwanted) as there is a clear analogy with, for example, audio signal processing. Too much filtering to remove noise can result in a poorer quality signal.

In assessing the various demosaicing algorithms it was not practicable to consider every possible combination of false colour suppression steps, line noise filter settings etc. but I took 20 screenshots in total. Here are a few more.

igv

igv behaves rather differently, showing hardly any colour artefacts even with no suppression steps.

With one suppression step we get the following.

dcb

With no colour suppression dcb is quite unusable.

With two suppression steps and the “enhancement” box checked the result is much improved.

Objective versus subjective assessment of the screenshots

Ranking 20 screenshots by subjectively deciding which looks better is not an easy task, but whilst browsing through the files I noticed that the more colourful images had larger jpeg file sizes. This came as no surprise and I wondered to what extent file size could be used as a measure of quality.

File size comparison

I have no knowledge of the jpeg compression algorithm, but just applying the principles of Information Theory (developed by Claude E. Shannon), we can assume that the larger the file the more information it contains. In the case of our jpeg files the word “information” equates to signal plus noise (or image plus artefacts).

I decided to use a more severe crop (from the original screenshot (.png) down to the square sample of the original image) before comparing file sizes so that the results would not be affected by the settings panel on the right. I used Darktable for this and copied the history file to all screenshots after cropping the first, so that we are comparing not just the same number of pixels but exactly the same pixels in each sample. These files were saved with filenames indicationg the time (hh:mm:ss) of the original screenshot and gave no direct indication of the demosaicing method, in the hope that an impartial ranking could then be performed by visual examination.

Whilst it is reasonable to assume that the largest file belongs to the poorest quality image, is the smallest file the best? As file size is a measure of signal plus noise there is always the danger (as mentioned earlier in the audio analogy) that too ambitious noise removal could remove some signal and this could be part of the reason for the smaller file.

In practice it proved impossible to differentiate visually between several of the demosaicing methods as the only effective criterion was the degree of false colour. Perhaps the only exception being that dcb appears slightly sharpened when the “enhanced” box is checked.

The results of ranking the 20 samples by file size are shown in the table. Those on the left I would consider “acceptable”, those on the right “unacceptable”. Note that there is quite a gap in file size between the “worst of the best” and the “best of the worst” – this is largely because I jumped from 0 to 2 steps of false colour suppression in many of the methods.

Profiles of pixel values

I also looked at pixel values. In all cases examined, the contrast and brightness were about the same and there was no significant difference in sharpness, as evidenced by the slope of the transitions. An example of pixel plots produced by ImageJ is shown below. The demosaicing methods shown are igv with one false colour suppression step (above) and dcb with two suppression steps (below).

ImageJ and its Java source code are freely available and in the public domain. No license is required. It can be downloaded from http://rsbweb.nih.gov/ij/download.html.

Hot pixels

The different demosaicing methods produce quite different effects on hot pixels. Below are some samples from my Town Hall pictures (see previous article). The settings panel included shows the method.

A single hot pixel as rendered by amaze – no filtering, no false colour suppression. The ideal?

Two iterations of dcb (the default settimg) spreads the damage to 14 surrounding pixels in a pattern which has horizontal symmetry and vertical symmetry (but different along the two axes).

Fewer pixels are damaged by ahd but there is no symmetry at all.

But, does it matter? In all cases the application of the Hot/Dead pixel filter effectively removes the problem before it reaches the demosaicing stage.

Conclusion

In comparing demosaicing methods visually the only obvious defect was false colour and interestingly this required different numbers of suppression steps for the different methods; for example, igv with no false colour suppression was very similar to dcb with two steps of suppression. In general, with the particular image file studied, once a suitable setting for false colour suppression has been found (for each) there is little to choose between the various demosaicing methods.

Whilst more testing would be necessary to prove the hypothesis “…that jpeg file size is a good indicator of quality – the smaller the better – within reason – for this type of image – etc.”, I was not able to disprove it by visual examination of the images. I would appreciate any comments on this.

Examination of pixel values by plotting profiles using ImageJ revealed nothing of particular significance, the results being very similar for the few that I tested. No difference in brightness, contrast or sharpness was evident.

I was a little surprised to see how differently hot pixels are treated by the different demosaicing methods, many producing spurious effects in surrounding pixels. Amaze and fast gave similar clean results, displaying a single white pixel as one might expect. However, provided you use the Hot/Dead pixel filter this should have no effect.

One final point: The default setting for false colour suppression steps is in every case zero (in RawTherapee). A comparison of demosaicing methods with this default setting would be to the disadvantage of many of them – amaze, for instance, requires two steps to compare with igv with one step. Perhaps the designers of RawTherapee should consider choosing default values somewhere near the optimum.

Based on these findings I intend to set up a default profile as a starting point for my image processing which will include (at least) hot pixel filtering and one of the better demosaicing methods with an optimal number of false colour suppression steps.

I don’t yet have a preference and remember, I used a Pentax K20D, your camera may well be different. Different sensors may require different demosaicing methods to give their best results.

I plan to follow this with a Part 2, looking at how different demosaicing methods deal with a noisy raw image.