Category Archives: Sensors

Photons, Shot Noise and Poisson Processes

Every digital photographer soon discovers that there are three main sources of visible random noise that affect pictures taken in normal conditions: Shot, pixel response non-uniformities (PRNU) and Read noise.[1]

Shot noise (sometimes referred to as Photon Shot Noise or Photon Noise) we learn is ‘inherent in light’; PRNU is per pixel gain variation proportional to light, mainly affecting the brighter portions of our pictures; Read Noise is instead independent of light, introduced by the electronics and visible in the darker shadows.  You can read in this earlier post a little more detail on how they interact.

Read Noise Shot Photon PRNU Photo Resonse Non Uniformity

However, shot noise is omnipresent and arguably the dominant source of visible noise in typical captures.  This article’s objective is to  dig deeper into the sources of Shot Noise that we see in our photographs: is it really ‘inherent in the incoming light’?  What about if the incoming light went through clouds or was reflected by some object at the scene?  And what happens to the character of the noise as light goes through the lens and is turned into photoelectrons by a pixel’s photodiode?

Fish, dear reader, fish and more fish.

Continue reading Photons, Shot Noise and Poisson Processes

Connecting Photographic Raw Data to Tristimulus Color Science

Absolute Raw Data

In the previous article we determined that the three r_{_L}g_{_L}b_{_L} values recorded in the raw data in the center of the image plane in units of Data Numbers per pixel – by a digital camera and lens as a function of absolute spectral radiance L(\lambda) at the lens – can be estimated as follows:

(1)   \begin{equation*} r_{_L}g_{_L}b_{_L} =\frac{\pi p^2 t}{4N^2} \int\limits_{380}^{780}L(\lambda) \odot SSF_{rgb}(\lambda)  d\lambda \end{equation*}

with subscript _L indicating absolute-referred units and SSF_{rgb} the three system Spectral Sensitivity Functions.   In this series of articles \odot is wavelength by wavelength multiplication (what happens to the spectrum of light as it progresses through the imaging system) and the integral just means the area under each of the three resulting curves (integration is what the pixels do during exposure).  Together they represent an inner or dot product.  All variables in front of the integral were previously described and can be considered constant for a given photographic setup. Continue reading Connecting Photographic Raw Data to Tristimulus Color Science

The Physical Units of Raw Data

In the previous article we (I) learned that the Spectral Sensitivity Functions of a given digital camera and lens are the result of the interaction of light from the scene with all of the spectrally varied components that make up the imaging system: mainly the lens, ultraviolet/infrared hot mirror, Color Filter Array and other filters, finally the photoelectric layer of the sensor, which is normally silicon in consumer kit.

Figure 1. The journey of light from source to sensor.  Cone Ω will play a starring role in the narrative that follows.

In this one we will put the process on a more formal theoretical footing, setting the stage for the next few on the role of white balance.

Continue reading The Physical Units of Raw Data

The Spectral Response of Digital Cameras

Photography works because visible light from one or more sources reaches the scene and is reflected in the direction of the camera, which then captures a signal proportional to it.  The journey of light can be described in integrated units of power all the way to the sensor, for instance so many watts per square meter. However ever since Newton we have known that such total power is in fact the result of the weighted sum of contributions by every frequency  that makes up the light, what he called its spectrum.

Our ability to see and record color depends on knowing the distribution of the power contained within a subset of these frequencies and how it interacts with the various objects in its path.  This article is about how a typical digital camera for photographers interacts with the spectrum arriving from the scene: we will dissect what is sometimes referred to as the system’s Spectral Response or Sensitivity.

Figure 1. Spectral Sensitivity Functions of an arbitrary imaging system, resulting from combining the responses of the various components described in the article.

Continue reading The Spectral Response of Digital Cameras

Pi HQ Cam Sensor Performance

Now that we know how to open 12-bit raw files captured with the new Raspberry Pi High Quality Camera, we can learn a bit more about the capabilities of its 1/2.3″ Sony IMX477 sensor from a keen photographer’s perspective.  The subject is a bit dry, so I will give you the summary upfront.  These figures were obtained with my HQ module at room temperature and the raspistill – -raw (-r) command:

Raspberry Pi
HQ Camera
raspistill
--raw -ag 1
Comments
Black Level256.3 DN256.0 - 257.3 based on gain
White Level4095Constant throughout
Analog Gain1Gain Range 1 - 16
Read Noise3 e-, gain 1
1.5 e-, gain 16
1.53 DN from black frame
11.50 DN
Clipping (FWC)8180 e-at base gain, 3400e-/um^2
Dynamic Range11.15 stops
11.3 stops
SNR = 1 to Clipping
Read Noise to Clipping
System Gain0.47 DN/e-at base analog gain
Star Eater AlgorithmPartly DefeatableAll channels - from base gain and from min shutter speed
Low Pass FilterYesAll channels - from base gain and from min shutter speed

Continue reading Pi HQ Cam Sensor Performance

Opening Raspberry Pi High Quality Camera Raw Files

The Raspberry Pi Foundation recently released an interchangeable lens camera module based on the Sony  IMX477, a 1/2.3″ back side illuminated sensor with 3040×4056 pixels of 1.55um pitch.  In this somewhat technical article we will unpack the 12-bit raw still data that it produces and render it in a convenient color space.

still life raw capture data file raspberry pi high quality hq cam f/8 1/2s base analog gain iso adobe rgb
Figure 1. 12-bit raw capture by Raspberry Pi High Quality Camera with 16 mm kit lens at f/8, 1/2 s, base ISO. The image was loaded into Matlab and rendered Half Height Nearest Neighbor in the Adobe RGB color space with a touch of local contrast and sharpening.  Click on it to see it in its own tab and view it at 100% magnification. If your browser is not color managed you may not see colors properly.

Continue reading Opening Raspberry Pi High Quality Camera Raw Files

Diffracted DOF Aperture Guides: 24-35mm

As a landscape shooter I often wonder whether old rules for DOF still apply to current small pixels and sharp lenses. I therefore roughly measured  the spatial resolution performance of my Z7 with 24-70mm/4 S in the center to see whether ‘f/8 and be there’ still made sense today.  The journey and the diffraction-simple-aberration aware model were described in the last few posts.  The results are summarized in the Landscape Aperture-Distance charts presented here for the 24, 28 and 35mm focal lengths.

I also present the data in the form of a simplified plot to aid making the right compromises when the focusing distance is flexible.  This information is valid for the Z7 and kit in the center only.  It probably just as easily applies to cameras with similarly spec’d pixels and lenses. Continue reading Diffracted DOF Aperture Guides: 24-35mm

Canon’s High-Res Optical Low Pass Filter

Canon recently introduced its EOS-1D X Mark III Digital Single-Lens Reflex [Edit: and now also possibly the R5 Mirrorless ILC] touting a new and improved Anti-Aliasing filter, which they call a High-Res Gaussian Distribution LPF, claiming that

“This not only helps to suppress moiré and color distortion,
but also improves resolution.”

Figure 1. Artist’s rendition of new High-res Low Pass Filter, courtesy of Canon USA

In this article we will try to dissect the marketing speak and understand a bit better the theoretical implications of the new AA.  For the abridged version, jump to the Conclusions at the bottom.  In a picture:

Canon High-Res Anti-Aliasing filter
Figure 16: The less psychedelic, the better.

Continue reading Canon’s High-Res Optical Low Pass Filter

The HV Spectrogram

A spectrogram, also sometimes referred to as a periodogram, is  a visual representation of the Power Spectrum of a signal.  Power Spectrum answers the question “How much power is contained in the frequency components of the signal”. In digital photography a Power Spectrum can show the relative strength of repeating patterns in captures and whether processing has been applied.

In this article I will describe how you can construct a spectrogram and how to interpret it, using dark field raw images taken with the lens cap on as an example.  This can tell us much about the performance of our imaging devices in the darkest shadows and how well tuned their sensors are there.

Pixel level noise spectrum
Figure 1. Horizontal and Vertical Spectrogram of noise captured in the raw data by a Nikon Z7 at base ISO with  the lens cap on.  The plot shows clear evidence of low-pass filtering in the blue CFA color plane and pattern noise repeating every 6 rows there and in one of the green ones.

Continue reading The HV Spectrogram

Phase One IQ3 100MP Trichromatic vs Standard Back Linear Color, Part III

Over the last two posts we’ve been exploring some of the differences introduced by tweaks to the Color Filter Array of the Phase One IQ3 100MP Trichromatic Digital Back versus its original incarnation, the Standard Back.  Refer to those for the background.  In this article we will delve into some of these differences quantitatively[1].

Let’s start with the compromise color matrices we derived from David Chew’s captures of a ColorChecher 24 in the shade of a sunny November morning in Ohio[2].   These are the matrices necessary to convert white balanced raw data to the perceptual CIE XYZ color space, where it is said there should be one-to-one correspondence with colors as perceived by humans, and therefore where most measurements are performed.  They are optimized for each back in the current conditions but they are not perfect, the reason for the word ‘compromise’ in their name:

Figure 1. Optimized Linear Compromise Color Matrices for the Phase One IQ3 100 MP Standard and Trichromatic Backs under approximately D65 light.

Continue reading Phase One IQ3 100MP Trichromatic vs Standard Back Linear Color, Part III

Phase One IQ3 100MP Trichromatic vs Standard Back Linear Color, Part II

We have seen in the last post that Phase One apparently performed a couple of main tweaks to the Color Filter Array of its Medium Format IQ3 100MP back when it introduced the Trichromatic:  it made the shapes of color filter sensitivities more symmetric by eliminating residual transmittance away from the peaks; and it boosted the peak sensitivity of the red (and possibly blue) filter.  It did this with the objective of obtaining more accurate, less noisy color out of the hardware, requiring less processing and weaker purple fringing to boot.

Both changes carry the compromises discussed in the last article so the purpose of this one and the one that follows is to attempt to measure – within the limits of my tests, procedures and understanding[1] – the effect of the CFA changes from similar raw captures by the IQ3 100MP Standard Back and Trichromatic, courtesy of David Chew.  We will concentrate on color accuracy, leaving purple fringing for another time.

Figure 1. Phase One IQ3 100MP image rendered linearly via a dedicated color matrix from raw data without any additional processing whatsoever: no color corrections, no tone curve, no sharpening, no nothing. Brightness adjusted to just avoid clipping.  Capture by David Chew.

Continue reading Phase One IQ3 100MP Trichromatic vs Standard Back Linear Color, Part II

Phase One IQ3 100MP Trichromatic vs Standard Back Linear Color, Part I

It is always interesting when innovative companies push the envelope of the state-of-the-art of a single component in their systems because a lot can be learned from before and after comparisons.   I was therefore excited when Phase One introduced a Trichromatic version of their Medium Format IQ3 100MP Digital Back last September because it could allows us to isolate the effects of tweaks to their Bayer Color Filter Array, assuming all else stays the same.

Figure 1. IQ3 100MP Trichromatic (left) vs the rest (right), from PhaseOne.com.   Units are not specified but one would assume that the vertical axis is relative spectral sensitivity and the horizontal axis represents wavelength.

Thanks to two virtually identical captures by David Chew at getDPI, and Erik Kaffehr’s intelligent questions at DPR, in the following articles I will explore the effect on linear color of the new Trichromatic CFA (TC) vs the old one on the Standard Back (SB).  In the process we will discover that – within the limits of my tests, procedures and understanding[1] – the Standard Back produces apparently more ‘accurate’ color while the Trichromatic produces better looking matrices, potentially resulting in ‘purer’ signals. Continue reading Phase One IQ3 100MP Trichromatic vs Standard Back Linear Color, Part I

Wavefront to PSF to MTF: Physical Units

In the last article we saw that the intensity Point Spread Function and the Modulation Transfer Function of a lens could be easily approximated numerically by applying Discrete Fourier Transforms to its generalized exit pupil function \mathcal{P} twice in sequence.[1]

Numerical Fourier Optics: amplitude Point Spread Function, intensity PSF and MTF

Obtaining the 2D DFTs is easy: simply feed MxN numbers representing the two dimensional complex image of the Exit Pupil function in its uv space to a Fast Fourier Transform routine and, presto, it produces MxN numbers representing the amplitude of the PSF on the xy sensing plane.  Figure 1a shows a simple case where pupil function \mathcal{P} is a uniform disk representing the circular aperture of a perfect lens with MxN = 1024×1024.  Figure 1b is the resulting intensity PSF.

Figure 1a, left: A circular array of ones appearing as a white disk on a black background, representing a circular aperture. Figure 1b, right: Array of numbers representing the PSF of image 1a in the classic shape of an Airy Pattern.
Figure 1. 1a Left: Array of numbers representing a circular aperture (zeros for black and ones for white).  1b Right: Array of numbers representing the PSF of image 1a (contrast slightly boosted).

Simple and fast.  Wonderful.  Below is a slice through the center, the 513th row, zoomed in.  Hmm….  What are the physical units on the axes of displayed data produced by the DFT? Continue reading Wavefront to PSF to MTF: Physical Units

Aberrated Wave to Image Intensity to MTF

Goodman, in his excellent Introduction to Fourier Optics[1], describes how an image is formed on a camera sensing plane starting from first principles, that is electromagnetic propagation according to Maxwell’s wave equation.  If you want the play by play account I highly recommend his math intensive book.  But for the budding photographer it is sufficient to know what happens at the Exit Pupil of the lens because after that the transformations to Point Spread and Modulation Transfer Functions are straightforward, as we will show in this article.

The following diagram exemplifies the last few millimeters of the journey that light from the scene has to travel in order to be absorbed by a camera’s sensing medium.  Light from the scene in the form of  field  U arrives at the front of the lens.  It goes through the lens being partly blocked and distorted by it as it arrives at its virtual back end, the Exit Pupil, we’ll call this blocking/distorting function P.   Other than in very simple cases, the Exit Pupil does not necessarily coincide with a specific physical element or Principal surface.[iv]  It is a convenient mathematical construct which condenses all of the light transforming properties of a lens into a single plane.

The complex light field at the Exit Pupil’s two dimensional uv plane is then  U\cdot P as shown below (not to scale, the product of the two arrays is element-by-element):

Figure 1. Simplified schematic diagram of the space between the exit pupil of a camera lens and its sensing plane. The space is assumed to be filled with air.

Continue reading Aberrated Wave to Image Intensity to MTF

The Units of Discrete Fourier Transforms

This article is about specifying the units of the Discrete Fourier Transform of an image and the various ways that they can be expressed.  This apparently simple task can be fiendishly unintuitive.

The image we will use as an example is the familiar Airy Pattern from the last few posts, at f/16 with light of mean 530nm wavelength. Zoomed in to the left in Figure 1; and as it looks in its 1024×1024 sample image to the right:

Airy Mesh and Intensity
Figure 1. Airy disc image I(x,y). Left, 1a, 3D representation, zoomed in. Right, 1b, as it would appear on the sensing plane (yes, the rings are there but you need to squint to see them).

Continue reading The Units of Discrete Fourier Transforms

A Simple Model for Sharpness in Digital Cameras – Sampling & Aliasing

Having shown that our simple two dimensional MTF model is able to predict the performance of the combination of a perfect lens and square monochrome pixel with 100% Fill Factor we now turn to the effect of the sampling interval on spatial resolution according to the guiding formula:

(1)   \begin{equation*} MTF_{Sys2D} = \left|(\widehat{ PSF_{lens} }\cdot \widehat{PIX_{ap} })\right|_{pu}\ast\ast\: \delta\widehat{\delta_{pitch}} \end{equation*}

The hats in this case mean the Fourier Transform of the relative component normalized to 1 at the origin (_{pu}), that is the individual MTFs of the perfect lens PSF, the perfect square pixel and the delta grid;  ** represents two dimensional convolution.

Sampling in the Spatial Domain

While exposed a pixel sees the scene through its aperture and accumulates energy as photons arrive.  Below left is the representation of, say, the intensity that a star projects on the sensing plane, in this case resulting in an Airy pattern since we said that the lens is perfect.  During exposure each pixel integrates (counts) the arriving photons, an operation that mathematically can be expressed as the convolution of the shown Airy pattern with a square, the size of effective pixel aperture, here assumed to have 100% Fill Factor.  It is the convolution in the continuous spatial domain of lens PSF with pixel aperture PSF shown in Equation (2) of the first article in the series.

Sampling is then the product of an infinitesimally small Dirac delta function at the center of each pixel, the red dots below left, by the result of the convolution, producing the sampled image below right.

Footprint-PSF3
Figure 1. Left, 1a: A highly zoomed (3200%) image of the lens PSF, an Airy pattern, projected onto the imaging plane where the sensor sits. Pixels shown outlined in yellow. A red dot marks the sampling coordinates. Right, 1b: The sampled image zoomed at 16000%, 5x as much, because in this example each pixel’s width is 5 linear units on the side.

Continue reading A Simple Model for Sharpness in Digital Cameras – Sampling & Aliasing

A Simple Model for Sharpness in Digital Cameras – Diffraction and Pixel Aperture

Now that we know from the introductory article that the spatial frequency response of a typical perfect digital camera and lens (its Modulation Transfer Function) can be modeled simply as the product of the Fourier Transform of the Point Spread Function of the lens and pixel aperture, convolved with a Dirac delta grid at cycles-per-pixel pitch spacing

(1)   \begin{equation*} MTF_{Sys2D} = \left|\widehat{ PSF_{lens} }\cdot \widehat{PIX_{ap} }\right|_{pu}\ast\ast\: \delta\widehat{\delta_{pitch}} \end{equation*}

we can take a closer look at each of those components (pu here indicating normalization to one at the origin).   I used Matlab to generate the examples below but you can easily do the same with a spreadsheet.   Continue reading A Simple Model for Sharpness in Digital Cameras – Diffraction and Pixel Aperture

A Simple Model for Sharpness in Digital Cameras – I

The next few posts will describe a linear spatial resolution model that can help a photographer better understand the main variables involved in evaluating the ‘sharpness’ of photographic equipment and related captures.   I will show numerically that the combined spectral frequency response (MTF) of a perfect AAless monochrome digital camera and lens in two dimensions can be described as the magnitude of the normalized product of the Fourier Transform (FT) of the lens Point Spread Function by the FT of the pixel footprint (aperture), convolved with the FT of a rectangular grid of Dirac delta functions centered at each  pixel:

    \[ MTF_{2D} = \left|\widehat{ PSF_{lens} }\cdot \widehat{PIX_{ap} }\right|_{pu}\ast\ast\: \delta\widehat{\delta_{pitch}} \]

With a few simplifying assumptions we will see that the effect of the lens and sensor on the spatial resolution of the continuous image on the sensing plane can be broken down into these simple components.  The overall ‘sharpness’ of the captured digital image can then be estimated by combining the ‘sharpness’ of each of them.

The stage will be set in this first installment with a little background and perfect components.  Later additional detail will be provided to take into account pixel aperture and Anti-Aliasing filters.  Then we will look at simple aberrations.  Next we will learn how to measure MTF curves for our equipment, and look at numerical methods to model PSFs and MTFs from the wavefront at the aperture. Continue reading A Simple Model for Sharpness in Digital Cameras – I

Sub Bit Signal

My camera has a 14-bit ADC.  Can it accurately record information lower than 14 stops below full scale? Can it store sub-LSB signals in the raw data?

With a well designed sensor the answer, unsurprisingly if you’ve followed the last few posts, is yes it can.  The key to being able to capture such tiny visual information in the raw data is a well behaved imaging system with a properly dithered ADCContinue reading Sub Bit Signal

Photographic Sensor Simulation

Physicists and mathematicians over the last few centuries have spent a lot of their time studying light and electrons, the key ingredients of digital photography.  In so doing they have left us with a wealth of theories to explain their behavior in nature and in our equipment.  In this article I will describe how to simulate the information generated by a uniformly illuminated imaging system using open source Octave (or equivalently Matlab) utilizing some of these theories.

Since as you will see the simulations are incredibly (to me) accurate, understanding how the simulator works goes a long way in explaining the inner workings of a digital sensor at its lowest levels; and simulated data can be used to further our understanding of photographic science without having to run down the shutter count of our favorite SLRs.  This approach is usually referred to as Monte Carlo simulation.

Continue reading Photographic Sensor Simulation

Information Transfer: Non ISO-Invariant Case

We’ve seen how information about a photographic scene is collected in the ISOless/invariant range of a digital camera sensor, amplified, converted to digital data and stored in a raw file.  For a given Exposure the best information quality (IQ) about the scene is available right at the photosites, only possibly degrading from there – but a properly designed** fully ISO invariant imaging system is able to store it in its entirety in the raw data.  It is able to do so because the information carrying capacity (photographers would call it the dynamic range) of each subsequent stage is equal to or larger than the previous one.   Cameras that are considered to be (almost) ISOless from base ISO include the Nikon D7000, D7200 and the Pentax K5.  All digital cameras become ISO invariant above a certain ISO, the exact value determined by design compromises.

ToneTransferISOless100
Figure 1: Simplified Scene Information Transfer in an ISO Invariant Imaging System at base ISO

In this article we’ll look at a class of imagers that are not able to store the whole information available at the photosites in one go in the raw file for a substantial portion of their working ISOs.  The photographer can in such a case choose out of the full information available at the photosites what smaller subset of it to store in the raw data by the selection of different in-camera ISOs.  Such cameras are sometimes improperly referred to as ISOful. Most Canon DSLRs fall into this category today.  As do kings of darkness such as the Sony a7S or Nikon D5.

Continue reading Information Transfer: Non ISO-Invariant Case

Image Quality: Raising ISO vs Pushing in Conversion

In the last few posts I have made the case that Image Quality in a digital camera is entirely dependent on the light Information collected at a sensor’s photosites during Exposure.  Any subsequent processing – whether analog amplification and conversion to digital in-camera and/or further processing in-computer – effectively applies a set of Information Transfer Functions to the signal  that when multiplied together result in the data from which the final photograph is produced.  Each step of the way can at best maintain the original Information Quality (IQ) but in most cases it will degrade it somewhat.

IQ: Only as Good as at Photosites’ Output

This point is key: in a well designed imaging system** the final image IQ is only as good as the scene information collected at the sensor’s photosites, independently of how this information is stored in the working data along the processing chain, on its way to being transformed into a pleasing photograph.  As long as scene information is properly encoded by the system early on, before being written to the raw file – and information transfer is maintained in the data throughout the imaging and processing chain – final photograph IQ will be virtually the same independently of how its data’s histogram looks along the way.

Continue reading Image Quality: Raising ISO vs Pushing in Conversion

Information Theory for Photographers

Ever since Einstein we’ve been able to say that humans ‘see’ because information about the scene is carried to the eyes by photons reflected by it.  So when we talk about Information in photography we are referring to information about the energy and distribution of photons arriving from the scene.   The more complete this information, the better we ‘see’.  No photons = no information = no see; few photons = little information = see poorly = poor IQ; more photons = more information = see better = better IQ.

Sensors in digital cameras work similarly, their output ideally being the energy and location of every photon incident on them during Exposure. That’s the full information ideally required to recreate an exact image of the original scene for the human visual system, no more and no less. In practice however we lose some of this information along the way during sensing, so we need to settle for approximate location and energy – in the form of photoelectron counts by pixels of finite area, often correlated to a color filter array.

Continue reading Information Theory for Photographers

How Many Bits to Fully Encode My Image

My camera sports a 14 stop Engineering Dynamic Range.  What bit depth do I need to safely fully encode all of the captured tones from the scene with a linear sensor?  As we will see the answer is not 14 bits because that’s the eDR, but it’s not too far from that either – for other reasons, as information science will show us in this article.

When photographers talk about grayscale ‘tones’ they typically refer to the number of distinct gray levels present in a displayed image.  They don’t want to see distinct levels in a natural slow changing gradient like a dark sky: if it’s smooth they want to perceive it as smooth when looking at their photograph.  So they want to make sure that all possible tonal  information from the scene has been captured and stored in the raw data by their imaging system.

Continue reading How Many Bits to Fully Encode My Image

Dynamic Range and Bit Depth

My camera has an engineering Dynamic Range of 14 stops, how many bits do I need to encode that DR?  Well, to encode the whole Dynamic Range 1 bit could suffice, depending on the content and the application.  The reason is simple, dynamic range is only concerned with the extremes, not with tones in between:

    \[ DR = \frac{Maximum Signal}{Minimum Signal} \]

So in theory we only need 1 bit to encode it: zero for minimum signal and one for maximum signal, like so

Continue reading Dynamic Range and Bit Depth

Engineering Dynamic Range in Photography

Dynamic Range (DR) in Photography usually refers to the linear working signal range, from darkest to brightest, that the imaging system is capable of capturing and/or displaying.  It is expressed as a ratio, in stops:

    \[ DR = log_2(\frac{Maximum Acceptable Signal}{Minimum Acceptable Signal}) \]

It is a key Image Quality metric because photography is all about contrast, and dynamic range limits the range of recordable/ displayable tones.  Different components in the imaging system have different working dynamic ranges and the system DR is equal to the dynamic range of the weakest performer in the chain.

Continue reading Engineering Dynamic Range in Photography

Determining Sensor IQ Metrics: RN, FWC, PRNU, DR, gain – 2

There are several ways to extract Sensor IQ metrics like read noise, Full Well Count, PRNU, Dynamic Range and others from mean and standard deviation statistics obtained from a uniform patch in a camera’s raw file.  In the last post we saw how to do it by using such parameters to make observed data match the measured SNR curve.  In this one we will achieve the same objective by fitting mean and  standard deviation data.  Since the measured data is identical, if the fit is good so should be the results.

Sensor Metrics from Measured Mean and Standard Deviation in DN

Continue reading Determining Sensor IQ Metrics: RN, FWC, PRNU, DR, gain – 2

Determining Sensor IQ Metrics: RN, FWC, PRNU, DR, gain – 1

We’ve seen how to model sensors and how to collect signal and noise statistics from the raw data of our digital cameras.  In this post I am going to pull both things together allowing us to estimate sensor IQ metrics: input-referred read noise, clipping/saturation/Full Well Count, Dynamic Range, Pixel Response Non-Uniformities and gain/sensitivity.

There are several ways to extract these metrics from signal and noise data obtained from a camera’s raw file.  I will show two related ones: via SNR in this post and via total noise N in the next.  The procedure is similar and the results are identical.

Continue reading Determining Sensor IQ Metrics: RN, FWC, PRNU, DR, gain – 1

Sensor IQ’s Simple Model

Imperfections in an imaging system’s capture process manifest themselves in the form of deviations from the expected signal.  We call these imperfections ‘noise’ because they introduce grain and artifacts in our images.   The fewer the imperfections, the lower the noise, the higher the image quality.

However, because the Human Visual System is adaptive within its working range, it’s not the absolute amount of noise that matters to perceived Image Quality (IQ) as much as the amount of noise relative to the signal – represented for instance by the Signal to Noise Ratio (SNR). That’s why to characterize the performance of a sensor in addition to signal and noise we also need to determine its sensitivity and the maximum signal it can detect.

In this series of articles I will describe how to use the Photon Transfer method and a spreadsheet to determine basic IQ performance metrics of a digital camera sensor.  It is pretty easy if we keep in mind the simple model of how light information is converted into raw data by digital cameras:

Sensor photons to DN A
Figure 1.

Continue reading Sensor IQ’s Simple Model

Olympus E-M5 II High-Res 64MP Shot Mode

Olympus just announced the E-M5 Mark II, an updated version of its popular micro Four Thirds E-M5 model, with an interesting new feature: its 16MegaPixel sensor, presumably similar to the one in other E-Mx bodies, has a high resolution mode where it gets shifted around by the image stabilization servos during exposure to capture, as they say in their press release

‘resolution that goes beyond full-frame DSLR cameras.  8 images are captured with 16-megapixel image information while moving the sensor by 0.5 pixel steps between each shot. The data from the 8 shots are then combined to produce a single, super-high resolution image, equivalent to the one captured with a 40-megapixel image sensor.’

A great idea that could give a welcome boost to the ‘sharpness’ of this handy system.  Preliminary tests show that the E-M5 mk II 64MP High-Res mode gives some advantage in MTF50 linear spatial resolution compared to the Standard Shot 16MP mode with the captures in this post.  Plus it apparently virtually eliminates the possibility of  aliasing and moiré.  Great stuff, Olympus.

Continue reading Olympus E-M5 II High-Res 64MP Shot Mode

Equivalence in Pictures: Sharpness/Spatial Resolution

So, is it true that a Four Thirds lens needs to be about twice as ‘sharp’ as its Full Frame counterpart in order to be able to display an image of spatial resolution equivalent to the larger format’s?

It is, because of the simple geometry I will describe in this article.  In fact with a few provisos one can generalize and say that lenses from any smaller format need to be ‘sharper’ by the ratio of their sensor diagonals in order to produce the same linear resolution on same-sized final images.

This is one of the reasons why Ansel Adams shot 4×5 and 8×10 – and I would too, were it not for logistical and pecuniary concerns.

Continue reading Equivalence in Pictures: Sharpness/Spatial Resolution

Equivalence in Pictures: Focal Length, f-number, diffraction, DOF

Equivalence – as we’ve discussed one of the fairest ways to compare the performance of two cameras of different physical formats, characteristics and specifications – essentially boils down to two simple realizations for digital photographers:

  1. metrics need to be expressed in units of picture height (or diagonal where the aspect ratio is significantly different) in order to easily compare performance with images displayed at the same size; and
  2. focal length changes proportionally to sensor size in order to capture identical scene content on a given sensor, all other things being equal.

The first realization should be intuitive (see next post).  The second one is the subject of this post: I will deal with it through a couple of geometrical diagrams.

Continue reading Equivalence in Pictures: Focal Length, f-number, diffraction, DOF

The Units of Spatial Resolution

Several sites for photographers perform spatial resolution ‘sharpness’ testing of a specific lens and digital camera set up by capturing a target.  You can also measure your own equipment relatively easily to determine how sharp your hardware is.  However comparing results from site to site and to your own can be difficult and/or misleading, starting from the multiplicity of units used: cycles/pixel, line pairs/mm, line widths/picture height, line pairs/image height, cycles/picture height etc.

This post will address the units involved in spatial resolution measurement using as an example readings from the popular slanted edge method, although their applicability is generic.

Continue reading The Units of Spatial Resolution

How to Measure the SNR Performance of Your Digital Camera

Determining the Signal to Noise Ratio (SNR) curves of your digital camera at various ISOs and extracting from them the underlying IQ metrics of its sensor can help answer a number of questions useful to photography.  For instance whether/when to raise ISO;  what its dynamic range is;  how noisy its output could be in various conditions; or how well it is likely to perform compared to other Digital Still Cameras.  As it turns out obtaining the relative data is a little  time consuming but not that hard.  All you need is your camera, a suitable target, a neutral density filter, dcraw or libraw or similar software to access the linear raw data – and a spreadsheet.

Continue reading How to Measure the SNR Performance of Your Digital Camera

Comparing Sensor SNR

We’ve seen how SNR curves can help us analyze digital camera IQ:

SNR-Photon-Transfer-Model-D610-4

In this post we will use them to help us compare digital cameras, independently of format size. Continue reading Comparing Sensor SNR

SNR Curves and IQ in Digital Cameras

In photography the higher the ratio of Signal to Noise, the less grainy the final image normally looks.  The Signal-to-Noise-ratio SNR is therefore a key component of Image Quality.  Let’s take a closer look at it. Continue reading SNR Curves and IQ in Digital Cameras

The Difference between Peak and Effective Quantum Efficiency

Effective Quantum Efficiency as I calculate it is an estimate of the probability that a visible photon  – from a ‘Daylight’ blackbody radiating source at a temperature of 5300K impinging on the sensor in question after making it through its IR filter, UV filter, AA low pass filter, microlenses, average Color Filter – will produce a photoelectron upon hitting silicon:

(1)   \begin{equation*} EQE = \frac{n_{e^-} \text{ produced by average pixel}}{n_{ph} \text{ incident on average pixel}} \end{equation*}

with n_{e^-} the signal in photoelectrons and n_{ph} the number of photons incident on the sensor at the given Exposure as shown below. Continue reading The Difference between Peak and Effective Quantum Efficiency

Equivalence and Equivalent Image Quality: Signal

One of the fairest ways to compare the performance of two cameras of different physical characteristics and specifications is to ask a simple question: which photograph would look better if the cameras were set up side by side, captured identical scene content and their output were then displayed and viewed at the same size?

Achieving this set up and answering the question is anything but intuitive because many of the variables involved, like depth of field and sensor size, are not those we are used to dealing with when taking photographs.  In this post I would like to attack this problem by first estimating the output signal of different cameras when set up to capture Equivalent images.

It’s a bit long so I will give you the punch line first:  digital cameras of the same generation set up equivalently will typically generate more or less the same signal in e^- independently of format.  Ignoring noise, lenses and aspect ratio for a moment and assuming the same camera gain and number of pixels, they will produce identical raw files. Continue reading Equivalence and Equivalent Image Quality: Signal

What is the Effective Quantum Efficiency of my Sensor?

Now that we know how to determine how many photons impinge on a sensor we can estimate its Effective Quantum Efficiency, that is the efficiency with which it turns such a photon flux (n_{ph}) into photoelectrons (n_{e^-} ), which will then be converted to raw data to be stored in the capture’s raw file:

(1)   \begin{equation*} EQE = \frac{n_{e^-} \text{ produced by average pixel}}{n_{ph} \text{ incident on average pixel}} \end{equation*}

I call it ‘Effective’, as opposed to ‘Absolute’, because it represents the probability that a photon arriving on the sensing plane from the scene will be converted to a photoelectron by a given pixel in a digital camera sensor.  It therefore includes the effect of microlenses, fill factor, CFA and other filters on top of silicon in the pixel.  Whether Effective or Absolute, QE is usually expressed as a percentage, as seen below in the specification sheet of the KAF-8300 by On Semiconductor, without IR/UV filters:

For instance if  an average of 100 photons per pixel were incident on a uniformly lit spot on the sensor and on average each pixel produced a signal of 20 photoelectrons we would say that the Effective Quantum Efficiency of the sensor is 20%.  Clearly the higher the EQE the better for Image Quality parameters such as SNR. Continue reading What is the Effective Quantum Efficiency of my Sensor?

How Many Photons on a Pixel at a Given Exposure

How many photons impinge on a pixel illuminated by a known light source during exposure?  To answer this question in a photographic context under daylight we need to know the effective area of the pixel, the Spectral Power Distribution of the illuminant and the relative Exposure.

We can typically estimate the pixel’s effective area and the Spectral Power Distribution of the illuminant – so all we need to determine is what Exposure the relative irradiance corresponds to in order to obtain the answer.

Continue reading How Many Photons on a Pixel at a Given Exposure

How Many Photons on a Pixel

How many visible photons hit a pixel on my sensor?  The answer depends on Exposure, Spectral power distribution of the arriving light and effective pixel area.  With a few simplifying assumptions it is not difficult to calculate that with a typical Daylight illuminant the number is roughly 11,760 photons per lx-s per \mu m^2.  Without the simplifying assumptions* it reduces to about 11,000. Continue reading How Many Photons on a Pixel

Nikon CFA Spectral Power Distribution

I measured the Spectral Photon Distribution of the three CFA filters of a Nikon D610 in ‘Daylight’ conditions with a cheap spectrometer.  Taking a cue from this post I pointed it at light from the sun reflected off a gray card  and took a raw capture of the spectrum it produced.

CFA Spectrum Spectrometer

An ImageJ plot did the rest.  I took a dozen captures at slightly different angles to catch the picture of the clearest spectrum.  Shown are the three spectral curves averaged over the two best opposing captures, each proportional to the number of photons let through by the respective Color Filter.   The units on the vertical axis are raw black-subtracted values from the raw file (DN), therefore the units on the vertical axis are proportional to the number of incident photons in each case.   The Photopic Eye Luminous Efficiency Function (2 degree, Sharpe et al 2005) is also shown for reference, scaled to the same maximum as the green curve (although in energy units, my bad). Continue reading Nikon CFA Spectral Power Distribution

Focus Tolerance and Format Size

The key variable as far as the tolerances required to position the lens for accurate focus are concerned (at least in a simplified ideal situation) is an appropriate approximate distance between the desired in-focus plane and the actual in-focus plane (which we are assuming is slightly out of focus). It is a distance in the direction perpendicular to the x-y plane normally used to describe position of the image on it, hence the designation delta z, or dz in this post.  The lens’ allowable focus tolerance is therefore  +/- dz, which we will show in this post to vary as the square of the format’s diagonal. Continue reading Focus Tolerance and Format Size

Exposure and ISO

The in-camera ISO dial is a ballpark milkshake of an indicator to help choose parameters that will result in a ‘good’ perceived picture. Key ingredients to obtain a ‘good’ perceived picture are 1) ‘good’ Exposure and 2) ‘good’ in-camera or in-computer processing. It’s easier to think about them as independent processes and that comes naturally to you because you shoot raw in manual mode and you like to PP, right? Continue reading Exposure and ISO