Raw Processors: Which Demosaicing Method? (Part1)

Comparison of Demosaicing Methods available in Free, Open Source Raw Processors

My previous article included a table listing the various demosaicing algorithms offered by the four raw processors considered and I wondered why we (as users) needed such a wide choice. The table is reproduced below.DemosaicI decided to investigate those offered by RawTherapee by looking closely at the detail in an image of tree branches against the sky – the same part of the same raw file processed by each of the algorithms. This was based on a suggestion by Elle Stone (http://ninedegreesbelow.com) whose articles on FLOSS (Free Libre Open Source Software) raw processors are well worth reading. The idea is to compare the performance of the different algorithms with a good clean image and with a very noisy image. As we use different cameras (Pentax and Canon) we should see if this also makes a difference.

The Original Image

Pentax K20D plus Sigma 17-70mm f/2.8-4.5 DC Macro f/8.0 1/200s iso100 f=36.88mm. Image is 14.6 MP (4680×3116)

This is the original image. I chose to use RawTherapee as this offers nine different algorithms and a convenient zoom window to see part of the image at 400% or higher. No input profile was applied, working colour space was set to ProPhoto and monitor profile set to the system monitor profile (recently calibrated and profiled). No sharpening or other processing apart from demosaicing was applied.

In viewing the images which follow, the fact that diagonal lines are not reproduced as a simple “staircase” in black and white might be thought of as a defect of the lens (or of the photographer, or of the autofocus system) but, it can easily be shown that the sharpness seen in these samples is close to the theoretical “best” that can be expected at an aperture of f/8 and a sensor 23.5mm wide and 4680 pixels wide. (These parameters relate to the Pentax K20D used). The limiting factor is diffraction. See e.g. “The Manual of Photography” Eighth Edition (Focal Press) p64 (Formerly “The Ilford Manual of Photography”).

For a lens focused at or near infinity, the diameter, D, of the Airy disc (the image of a point source of light) is given by: D = 2.44 λN

Where λ = the wavelength of the light and N = the f number. So, putting the values in, we have: D = 2.44 x 550 x 8 nanometers (λ = 550nm) which gives approximately 0.01mm.

The sensor in the K20D (on its longest side) has 4680 pixels in 23.5mm, so each pixel is:  23.5 ÷ 4680 = 0.005mm.

Conclusion: At f/8, the Airy disc has a diameter equal to 2 pixels. i.e. a point source of light (having zero size) will be reproduced as a disc about 2 pixels in diameter. It follows that a diagonal straight line transition from white to black will have a “softness” of about 2 pixels – and this is what we see in the images which follow.

Note that this is pure physics – there is nothing that Pentax, Canon or Nikon can do about it!

Amaze

The images which follow are self-documenting in that the panel on the right shows the Demosaicing Method and the number of False Color Suppression Steps.

amaze 0With the default setting of amaze (above), some false colour is evident but with just one suppression step we get the image below.

amaze 1And with two steps of suppression almost no colour.

amaze 2

It is probably best to use the minimum number of steps necessary as we do not know what “damage” each step does to the image. In this context it is useful to think in terms of signal (wanted) and noise (unwanted) as there is a clear analogy with, for example, audio signal processing. Too much filtering to remove noise can result in a poorer quality signal.

In assessing the various demosaicing algorithms it was not practicable to consider every possible combination of false colour suppression steps, line noise filter settings etc. but I took 20 screenshots in total. Here are a few more.

igv

igv behaves rather differently, showing hardly any colour artefacts even with no suppression steps.

igv 0With one suppression step we get the following.

igv 1

dcb

With no colour suppression dcb is quite unusable.

dcb 0With two suppression steps and the “enhancement” box checked the result is much improved.

dcb 2 enhanced

Objective versus subjective assessment of the screenshots

Ranking 20 screenshots by subjectively deciding which looks better is not an easy task, but whilst browsing through the files I noticed that the more colourful images had larger jpeg file sizes. This came as no surprise and I wondered to what extent file size could be used as a measure of quality.

File size comparison

I have no knowledge of the jpeg compression algorithm, but just applying the principles of Information Theory (developed by Claude E. Shannon), we can assume that the larger the file the more information it contains. In the case of our jpeg files the word “information” equates to signal plus noise (or image plus artefacts).

I decided to use a more severe crop (from the original screenshot (.png) down to the square sample of the original image) before comparing file sizes so that the results would not be affected by the settings panel on the right. I used Darktable for this and copied the history file to all screenshots after cropping the first, so that we are comparing not just the same number of pixels but exactly the same pixels in each sample. These files were saved with filenames indicationg the time (hh:mm:ss) of the original screenshot and gave no direct indication of the demosaicing method, in the hope that an impartial ranking could then be performed by visual examination.

Whilst it is reasonable to assume that the largest file belongs to the poorest quality image, is the smallest file the best? As file size is a measure of signal plus noise there is always the danger (as mentioned earlier in the audio analogy) that too ambitious noise removal could remove some signal and this could be part of the reason for the smaller file.

In practice it proved impossible to differentiate visually between several of the demosaicing methods as the only effective criterion was the degree of false colour. Perhaps the only exception being that dcb appears slightly sharpened when the “enhanced” box is checked.

The results of ranking the 20 samples by file size are shown in the table. Those on the left I would consider “acceptable”, those on the right “unacceptable”. Note that there is quite a gap in file size between the “worst of the best” and the “best of the worst” – this is largely because I jumped from 0 to 2 steps of false colour suppression in many of the methods.

methods by file sizeProfiles of pixel values

I also looked at pixel values. In all cases examined, the contrast and brightness were about the same and there was no significant difference in sharpness, as evidenced by the slope of the transitions. An example of pixel plots produced by ImageJ is shown below. The demosaicing methods shown are igv with one false colour suppression step (above) and dcb with two suppression steps (below).

ImageJ and its Java source code are freely available and in the public domain. No license is required. It can be downloaded from http://rsbweb.nih.gov/ij/download.html.

pixel profiles

Hot pixels

The different demosaicing methods produce quite different effects on hot pixels. Below are some samples from my Town Hall pictures (see previous article). The settings panel included shows the method.

amaze hot pixel

A single hot pixel as rendered by amaze – no filtering, no false colour suppression. The ideal?

dcb hot pixel

Two iterations of dcb (the default settimg) spreads the damage to 14 surrounding pixels in a pattern which has horizontal symmetry and vertical symmetry (but different along the two axes).

ahd hot pixel

Fewer pixels are damaged by ahd but there is no symmetry at all.

But, does it matter? In all cases the application of the Hot/Dead pixel filter effectively removes the problem before it reaches the demosaicing stage.

Conclusion

In comparing demosaicing methods visually the only obvious defect was false colour and interestingly this required different numbers of suppression steps for the different methods; for example, igv with no false colour suppression was very similar to dcb with two steps of suppression. In general, with the particular image file studied, once a suitable setting for false colour suppression has been found (for each) there is little to choose between the various demosaicing methods.

Whilst more testing would be necessary to prove the hypothesis “…that jpeg file size is a good indicator of quality – the smaller the better – within reason – for this type of image – etc.”, I was not able to disprove it by visual examination of the images. I would appreciate any comments on this.

Examination of pixel values by plotting profiles using ImageJ revealed nothing of particular significance, the results being very similar for the few that I tested. No difference in brightness, contrast or sharpness was evident.

I was a little surprised to see how differently hot pixels are treated by the different demosaicing methods, many producing spurious effects in surrounding pixels. Amaze and fast gave similar clean results, displaying a single white pixel as one might expect. However, provided you use the Hot/Dead pixel filter this should have no effect.

One final point: The default setting for false colour suppression steps is in every case zero (in RawTherapee). A comparison of demosaicing methods with this default setting would be to the disadvantage of many of them – amaze, for instance, requires two steps to compare with igv with one step. Perhaps the designers of RawTherapee should consider choosing default values somewhere near the optimum.

Based on these findings I intend to set up a default profile as a starting point for my image processing which will include (at least) hot pixel filtering and one of the better demosaicing methods with an optimal number of false colour suppression steps.

I don’t yet have a preference and remember, I used a Pentax K20D, your camera may well be different. Different sensors may require different demosaicing methods to give their best results.

I plan to follow this with a Part 2, looking at how different demosaicing methods deal with a noisy raw image.

Advertisements
This entry was posted in Computers, Photography and tagged , , , , , . Bookmark the permalink.

4 Responses to Raw Processors: Which Demosaicing Method? (Part1)

  1. spodzone says:

    I’ve just gone through the same process with an Olympus Pen-F. I share the concern about JPEG file-size as a means of comparison; IME it does do a good job of compressing signal but knowing whether larger file-sizes are due to good signal or noise is difficult to establish without a visual check.

    I also use entropy as a measure of information density.

    It so happens that both HPHD and AmaZe are fairly middling for JPEG file-size, but they’re high on entropy. So I junked the others and retained those two.

    Adding 1 step of false-colour reduction led to a massive drop in JPEG size, but again the entropies are fairly high.

    When I added further detail enhancements using an optimum mixture of wavelet, light USM and microcontrast for detail, the entropy increased by 0.4% (*very* significant in this case, far outweighing the range of entropy values in demosaicing algorithms alone).

    In my case, the best result was actually HPHD with extra detail enhancements as above; AmaZe came a close second. Profile saved, now I know I’ve got a good one 🙂

  2. Given that they are quite similar in output, I am using 3.o ish version of RT because higher versions crash (I have a weird large system, 2 different video cards, 16 gigs of ram etc) And I found AMaZE to be very slow. I am experimenting with hphd 1 step now to see if it seems faster. Thanks for the analysis.

  3. John Widdall says:

    I did not specifically comment on sharpness as there is no noticeable difference in sharpness in the published (or the unpublished) images. In all cases the theoretical limit of sharpness appears to have been achieved. I agree that a larger aperture would theoretically give a sharper image but this would only be achieved in practice if imperfections of the lens did not dominate. The reduced depth of field would be a disadvantage.

  4. Anthony New says:

    I notice you make no attempt to discern any effects on sharpness. Do you not expect any?
    And it might be interesting to repeat your tests on an image with a smaller airy disc.

Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s