My camera has a 14-bit ADC.  Can it accurately record information lower than 14 stops below full scale? Can it store sub-LSB signals in the raw data?

With a well designed sensor the answer, unsurprisingly if you’ve followed the last few posts, is yes it can.  The key to being able to capture such tiny visual information in the raw data is a well behaved imaging system with a properly dithered ADC. 

Well behaved means among other things that the system needs to keep non random noise sources like pixel response non uniformities, especially dark signal (DSNU) and fixed pattern noise (FPN), in check; properly dithered means that random noise rms at the input of the ADC needs to be greater than about 1 count (= 1 bit, LSB, DN, ADU).  To achieve that objective in all channels it appears that standard deviation of more than 2 DN is often necessary at the output of current camera ADCs, as typically measured directly off the raw data of a dark frame.

Less dithering will result in visual information loss;  more will not hurt the integrity of the signal itself but it may make for a noisier picture.

The question is whether such sub-bit signals are of good enough quality (IQ) to be useful to the artist.  Recall that engineers typically consider image information with a signal to noise ratio of less than 1 to be unacceptable, but most photographers would be more stringent, often considering minimum signals with SNRs quite a bit higher than that still unusable.  There is one class of photographers, however, that sometimes finds single captures with scene SNRs below one quite acceptable: astrophotographers.

The Well-Dithered ADC: read noise rms > 1 DN

Here is  the simulated picture of  a signal with maximum SNR of about 0.3 and minimum SNR of 0.033, well below acceptable IQ in a typical photographic  application, assuming a camera with 3 e- read noise, conversion gain of 0.5 DN/e- and no non-uniformities (you may want to click on the images to see them at 100%, hit ctrl-zero in your browser to make sure it is not zoomed):

Unusable.  In fact the image seems to be composed of just noise, so how do we know that the visual information has been properly captured and stored in its (raw) file?

Stacking to the Rescue

One way to reduce the influence of random noise familiar to astrophotographers is stacking: several images are taken of the same subject and added together.  The signal, if it is there, will add up with every new capture stacked, doubling on average every time the number of added frames is doubled.   As we know random noise standard deviation will on the other hand add in quadrature, growing at the rate of the square root of the number of images in the stack.   The result is that signal grows faster than noise and SNR improves.  Here are 10 frames captured in the same conditions as in Figure 1, stacked:

Since random noise standard deviation of 3e- was higher than 1 DN in Figure 1 above (1DN = 2e-, per system gain), information theory suggests that in this ideal case light information from the signal (the numbers) was nevertheless ‘properly’ captured into the raw data: assuming no DSNU and FPN all we need to do to pull out the information buried deep into the noise is to stack many such captures to reduce the proportion of random read noise in the final image.  Here are 100 images stacked:

Coming out nicely. If we keep at this ideal exercise long enough we should be able to get at the visual information stored in the individual captures with any arbitrary target SNR. And in fact the result of stacking very many such captures is the following image, showing the original signal numbers in all their glory

There you go, we’ve effectively recorded and brought usefully back to light a signal (the ‘1’) at 1/20th the intensity of the least significant bit, more than 4 stops below it.

Of course I am not suggesting that anybody sane should attempt to capture the number of frames required to recover a signal recorded at an SNR of 1/30 per frame, just that as long as the information is in the data in the first place one can do so if one so desires.  And the condition for that to be true is a well behaved sensor and random noise at the input of the ADC  higher than one LSB. In this case it was and so we could.  You can read how the images were generated at the bottom of the article*.

Improperly Dithered ADC = Information Loss

We have now shown that with enough frames we can pretty well recover to our satisfaction any signal information captured in the raw data as long as the imaging system is properly behaved and dithered.   What happens if the system is well behaved but the random read noise rms at the input of the ADC is not greater than about 1 LSB?

Again assuming an ideal camera with no FPN or other forms of non-uniformities, the following four strips correspond to Figures 1-4 above with the same signal intensity (the numbers) but this time a lower random read noise of 1e- and gain of 0.1 DN/e-.  This means that there are 10e- for each 1 LSB (the a7S has a similar gain at base ISO) and that random read noise rms is only 1/10th of 1 LSB: improper dithering.  Can we recover the visual information in this situation?  Here is the first single image, equivalent to Figure 1 above:

The tiny mean signal from 0.1 to 1 e-* just can’t make it to the 5e- threshold (1/2 LSB) to be counted without the help of the earlier relatively stronger dithering power, so most of the individual frames are full of zeros (0, zip).   Zero plus zero is still zero, even when stacking.  The few specs one can see are statistical oddities.  Below are stacks of 10 and 100 such frames:

The oddities are piling up, but because they are oddities they are not in the right proportions.  Therefore by the time the stack gets to the same number of frames as in Figure 4 above the resulting information is only partly there:  the signal appears nonlinear, with the distorted means and standard deviations investigated in the previous article on sub-LSB quantization.  This is what information loss looks like.

Of course not all information was lost and, hey, in quantum mechanics if you play tennis against a wall long enough the ball goes through it.  But most of this visual information bears only a tenuous relationship to the original, with the majority of the intensity values in the stacked image still equal to zero.  Here again is the properly dithered data in Figure 4 above for direct comparison

where instead signal means and standard deviations are linear and proportional to the original scene intensities, as expected from captures that were able to ‘properly’ record scene visual information into the raw data.

Why a 1 DN threshold at the Input of the ADC?

Janesick says in chapter 11 of his fine book that ‘ADC quantizing noise is made negligible by hiding it in random noise […]  For proper noise encoding [1 LSB] is made equal to the read noise level in electron units’.

Therefore, ignoring pixel response non-uniformities, the key to being able to capture visual information from the scene accurately in the raw data is a properly dithered ADC: random noise rms at its input of at least 1 DN (= 1 LSB). Less will result in visual information loss; more will not help but will produce noisier pictures.   If this condition is met one can then rest assured that the information is in one’s raw files – and possibly stack a few captures if the subject allows it in order to bring out the signal at the desired final target SNR.

The one-least-significant-bit approximate threshold for random read noise can also be gleaned from the article on quantization error.   Below are two graphs that show the theoretical average quantization error in both mean and standard deviation in the conditions of the two simulations above.   The error is  shown as a ratio of actual values measured before and after quantization.  A ratio of one means that the values before and after quantization are the same, therefore we assume it represents  no error.

First, the improperly dithered second set of frames with 0.1DN read noise of 1 e- facing gain of 0.1DN/e- shows quite large error excursions:

Then the properly dithered first set, with read noise well above the threshold at 1.5 DN (3e- with a gain of 0.5DN/e-), stuck at a ratio of one, indicating virtually no error.

For those who are wondering about the apparently tiny residual error in standard deviation as mean signal approaches zero, that’s due to the standard deviation of the rounding error inherent in uniformly distributed random noise around 1 LSB;  the asymptotic y-axis intercept with no signal should be  sqrt(1.5^2+1/12) / 1.5 = 1.018, as approximately shown.

The Output of the ADC is not the same as its Input

The following information is probably mostly of interest to astrophotographers or those who plan to stack many images.  How does one find out the random read noise level at the input of one’s camera’s ADC at various ISOs?

Of course only the manufacturers’ engineers know the value of read noise before the ADC, but we can guess at it by measuring the output of the ADC stored in the raw data of, say, a dark frame as a proxy for the amount of noise just before it, hoping the ADC is not too noisy itself.

Helpfully Bill Claff provides estimates of read noise in DNs ( the dithering ratio we are after) for most cameras by measuring noise directly off raw data dark frames at every ISO. Keep a bit of a margin, however, because the resulting figures refer to the average variance of the four channels at the output of the ADC – and supposedly include ADC noise, prnu, FPN, quantization noise and more.

The actual random noise at the input of the ADC will therefore be somewhat lower than that once the above components have been subtracted out.  And could be lower still for individual channels.  How much lower depends on how well the camera imaging system was tuned during design and manufacturing. Every generation is better than the last.

To be Safe: Standard Deviation of a couple of DNs in the Raw Data, Irrespective of ISO

Based on what I have seen read noise of a couple of output referred DNs per LSB should be enough to bring all channels up to the information-recording input threshold in most current advanced cameras, whatever the ISO.  For instance Bill Claff’s data suggests average read noise just shy of 1 DN for the D810 at base ISO.  Its green raw channels look well behaved as you can see in the smooth curve below (PTC curve obtained from data collected by Jim Kasson)

but the red and blue channels show telltale signs of  quantization error, a consequence of Nikon’s White Balance pre-conditioning that scales  their data digitally before writing it into the raw file, resulting in skipped codes.

Skipped codes mean that there are 2 LSBs between some consecutive raw values that cannot be dithered away by the 1 DN standard deviation of read noise, therefore resulting in quantization errors in the deepest shadows at base ISO as shown above.  A couple of stops up, at ISO 200, average read noise is over 2 DN and the ‘ringing’ in the weaker channels appears to be pretty well gone:

Incidentally I think these low level effects, if present, are virtually impossible to recognize visually in a normal capture.  They can be measured here with decent precision only because we are sampling about 100,000 uniformly illuminated pixels.

Bit Depth is not Always What it Seems

Also careful about the actual bit depth of some cameras when comparing random noise to the size of one LSB: some cameras advertised as producing 14-bit raw data actually run at 13 or even 12 bits depending on operating mode. For instance the a7mkII , like many advanced Sonys, appears to run the ADC at 13 bits in single shot mode at base ISO.  Bill Claff’s read noise in DN figures refer to a 14-bit scale instead, so a measured read noise of 1.1DN in the raw data at ISO100 could actually represent a lower figure in front of the camera’s 13-bit ADC.  Jim Kasson does in fact show quantization artifacts in the a7mkII a stop or so up from base ISO.

Jim also performed an interesting exercise with a D810, which as mentioned has a base ISO read noise of about 1 DN at 14-bits: with the green channel not showing symptoms of quantization at base ISO he switched the camera to 12-bit mode. This could in theory corresponds to feeding data to the 12-bit ADC with a read noise of only about 1/4LSB, insufficient for proper information capture. Sure enough the measured green channel started showing ringing, one of the symptoms of quantization. He then increased ISO a couple of stops on the 12-bit captures, bringing read noise back to about 1LSB, and re-measured: quantization effects gone.

So based on this and other tests I would suspect that for most current cameras the required threshold for ‘proper’ information capture of all channels be an average read noise of more than a couple of DN in the raw data (hopefully corresponding to the output of the ADC).  For today’s cleanest cameras that means a couple of stops up from base.  On the other hand most Canon DSLRs have larger read noise than that straight from base ISO.  Again, this information is probably mainly useful to astrophotographers looking to stack many images to bring out the weakest signals.

Limiting Factors: Other Types of Noise

So far we have assumed an ideal imaging system with no pixel response non uniformities.  If the sensor has a lot of fixed pattern noise or other non-uniformities, they will add linearly with the number of images stacked just as the signal does.  Depending on their entity they could quickly be brought to the fore and become visibly objectionable in forms such as banding, overwhelming a weak signal and limiting the ability to bring it to light by stacking.  Here is one such example by astrophotographer Sir Canon:

Clearly in the choice of equipment for astrophotography  applications, FPN and dark current noise performance become key additional considerations.

So there you have it, proper encoding of light information into the raw data in  well behaved imaging systems seems to depend principally on proper noise encoding and dithering independently of signal amplitude, pixel pitch or unity gain.

 

*Image Simulation Details

In the first set of images, Figures 1-4, read noise per frame () was Gaussian at 3e- rms with gain () of 0.5 DN/e-.  This means that  the simulated ADC was properly dithered to the tune of 1.5 LSB (=3*0.5).

The numbers are Poisson objects recorded in the original image at a linearly increasing mean intensity of 0.1e-/number; so left is no signal, number 4 corresponds to a mean signal () of 0.4 e- and 10 is 1.0 e- per frame.  Ignoring PRNU at these low signal levels and referring to the simple sensor model (uppercase LSB, lowercase e-):

   

SNR of the number 10 with = 1 e- is 1/sqrt(1+3^2) = 0.316
SNR of the number 1 with = 0.1 e- is 0.1/sqrt(0.1+3^2) = 0.033.

From gain, one LSB corresponds to 2 e-.  Therefore the number 1 in the first Figure at a signal of 0.1e- is log2(0.1/2) = 4.3 stops below ‘bit depth’, number 10 is 1 stop below.

 

In the second set of images (Figures 1a-4a) read noise per frame () was Gaussian at 1e- rms with gain () of 0.1 DN/e-.  This means that  the simulated ADC was insufficiently dithered by random read noise of only 0.1 LSB (=1*0.1).

SNR of the number 10 with = 1 e- is 1/sqrt(1+1^2) = 0.707
SNR of the number 1 with = 0.1 e- is 0.1/sqrt(0.1+1^2) = 0.095.

So despite a theoretically higher SNR, insufficient dithering in the second set of images resulted in information loss.

To make the resulting images somewhat comparable they were subjected to a levels adjustment by matlab imshow(image,[]) so that the lowest value in the files  became zero and the highest value 255.