IQ, Sensors

Sensor IQ’s Simple Model

Leave a comment

Imperfections in an imaging system’s capture process manifest themselves in the form of deviations from the expected signal.  We call these imperfections ‘noise’ because they introduce grain and artifacts in our images.   The fewer the imperfections, the lower the noise, the higher the image quality.

However, because the Human Visual System is adaptive within its working range, it’s not the absolute amount of noise that matters to perceived Image Quality (IQ) as much as the amount of noise relative to the signal – represented for instance by the Signal to Noise Ratio (SNR). That’s why to characterize the performance of a sensor in addition to signal and noise we also need to determine its sensitivity and the maximum signal it can detect.

In this series of articles I will describe how to use the Photon Transfer method and a spreadsheet to determine basic IQ performance metrics of a digital camera sensor.  It is pretty easy if we keep in mind the simple model of how light information is converted into raw data by digital cameras:

Sensor photons to DN A
Figure 1.

The Signal

Light in units of photons, typically reflected by the scene, travels through the lens and strikes pixels on the sensor proportionately to the given, uniform Exposure.  In current Bayer implementations only a sensor-specific 15-30% of incident photons get converted to photoelectrons thanks to the photoelectric effect.  We call this proportion Effective Quantum Efficiency (eQE).

The mean number of photoelectrons generated by such uniformly illuminated pixels is the Signal for the given exposure.  In the diagram above we denote the signal out of the sensor with the lowercase letter s, in units of photoelectrons (e-),

Noise: Deviation from the Expected Signal

For this simple sensor IQ model we will ignore minor sources of noise and concentrate on the main ones, which can mostly be considered to be random.  Random quantities are characterized by their distribution and standard deviation: the higher the deviation, the higher the noise.

We will aggregate noise sources into three main categories: inherent to the Signal (shot noise); related to  the electronics (read noise); and Signal related non-uniformities in the sensor (PRNU).  If you are interested in a more in-depth description of these noise sources I suggest Emil Martinec’s pages.

The photoelectric effect performs its conversion of photons to photoelectrons producing Poisson statistics; if we were to count the number of photoelectrons generated by pixels in an area of a sensor containing 200×200 of them, all with the same number of photons striking silicon*, we would find that the count would have a standard deviation equal to the square root of the mean.  This is  ‘shot’ noise inherent in the signal (s) in units of e- rms.  Therefore the standard deviation of Shot Noise is:

    \[ \sigma_{shot}=\sqrt{s} \: \:\text{ e-} \]

Read noise introduced by the electronics will have its own Gaussian standard deviation r, in e-

    \[ \sigma_{read}=r \: \:\text{ e-} \]

PRNU is modeled proportional to signal (s) in e- and has standard deviation

    \[ \sigma_{prnu}=s \cdot p \: \:\text{ e-} \]

with p the PRNU factor usually expressed as a percentage of the signal.  Ignoring Pattern and other types of noise, if the three noise sources are uncorrelated (they are) they add in quadrature so that the Total Random Noise standard deviation is

    \[ \sigma_{total}=\sqrt{\sigma_{shot}^2+\sigma_{read}^2+\sigma_{prnu}^2} \]

For convenience, in this phase of the simple model description we are assuming that the signal and all noise sources are present at the output of the sensor before amplification and expressed in the same units as the signal s, photoelectrons (labeled in red in the figure).   Therefore  written using our definitions above, this total input-referred random noise (n) is equal to

(1)   \begin{equation*} n=\sqrt{s+r^2+(s \cdot p})^2} \: \:\text{ e-} \end{equation*}

all variables in units of e- rms.

How to Measure RN and PRNU – without Gain

Moving along our model, the signal (s) and total random noise (n) output by the sensor in units of photoelectrons are then amplified according to gain g (in units of DN/e-** ) controlled by the in-camera ISO setting,  converted by the Analog to Digital Converter (ADC) to digital units – sometimes referred to as Analog to Digital Units (ADU) or simply ‘the count’ – before finally being stored in the camera’s Raw File as Data Numbers (DN).   The value for gain g in the figure is typical of some current full frame sensors at base ISO.  The mean signal written to the raw file is (uppercase letters represent units in DN, lowercase in e-)

    \[ S = s \cdot g \:\: DN \]

and total noise standard deviation as measured in the raw file is

    \[ N = n \cdot g \:\: DN \]

Therefore the Signal-to-Noise-Ratio (SNR) relevant to IQ of the information recorded in the camera’s raw file is

(2)   \begin{equation*} SNR=\frac{S}{N}=\frac{s \cdot g}{n \cdot g}= \frac{s}{n}=\frac{s}{\sqrt{s+r^2+(s \cdot p})^2}} \end{equation*}

with S and N in units of DN, g in DN/e- and all other variables in units of e-.

Equation (2) is powerful because it ties the signal (S) and total noise (N) – that we can easily measure in DN as the mean and standard deviation of a uniformly illuminated patch in the raw data – to their corresponding input-referred values internal to the sensor in physical units of photoelectrons,  that we do not have access to.

How to Measure RN and PRNU – with Gain

Another way to determine sensor metrics from statistics collected from  data in the raw file is to rewrite the total input-referred standard deviation noise equation (1) in output-referred DN by using n = N / g as follows (uppercase denotes units of DN, lowercase e-):

    \[ N=\sqrt{(s \cdot g) \cdot g + (r \cdot g)^2+((s \cdot g) \cdot p)^2} \]

With signal S = s \cdot g and read noise R = r \cdot g, the total standard deviation measured in the raw data in units of DN is

(3)   \begin{equation*} N=\sqrt{S \cdot g + R^2+(S \cdot p)^2} \end{equation*}

Contrary to Equation (2) the formula above contains an additional variable, gain (g), its significance outlined below.

The Full Simple Model of Sensor IQ

The full simplified model including random noise (but excluding pattern noise) then becomes

Sensor photons to DN B
Figure 2. A simple model for random noise in photographic imaging sensors.

Note that shot noise is easily calculated as the square root of signal s in e- in the example above:  \sqrt{25} = 5 e- (=1 DN).  Without gain g we would not be able to calculate it from mean signal S measured in DN in the raw data, though: \sqrt{5} is not equal to 1 DN.  Does this model work in practice?  Indeed it does, as you can read in this article.

In the next posts I will show how to use measured statistics from the raw data and equations (2) or (3) to obtain the sensor’s noise figures of merit r (read noise), f (saturation) and p (PRNU constant) as a function of gain g**.

 

* Just for clarity for this example.  In practice photons are emitted randomly but at a constant rate by natural light sources, therefore they would not actually arrive in equal numbers on a uniformly illuminated pixel patch.  They would instead arrive according to Poisson statistics as well, their standard deviation equal to the square root of the signal in photons referred to as ‘photon’ noise.  The generation of Photoelectrons by such arriving photons absorbed in silicon follows a Binomial process, which together happen to produce a Poisson distributed output – what we call ‘shot’ noise.  You can read more about the theory of shot noise in photography in this article.

** Some authors use sensitivity k in e-/DN instead of gain g, in DN/e-.   It makes no difference as long as you remember that g = 1/k.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.