This article will set the stage for a discussion on how pleasing color is produced during raw conversion.  The easiest way to understand how a camera captures and processes ‘color’ is to start with an example of how the human visual system does it.

An Example: Green

Light from the sun strikes leaves on a tree.   The foliage of the tree absorbs some of the light and reflects the rest diffusely  towards the eye of a human observer.  The eye focuses the image of the foliage onto the retina at its back.  Near the center of the retina there is a small circular area called fovea centralis which is dense with light receptors of well defined spectral sensitivities called cones. Information from the cones is pre-processed by neurons and carried by nerve fibers via the optic nerve to the brain where, after some additional psychovisual processing, we recognize the color of the foliage as green^[1].

If we zoom in on the fovea we realize that cones come in three flavors depending on their spectral sensitivity, which may be more attuned to the Long, Medium or Short wavelengths of the visible spectrum;  they are denoted in turn L,M,S or ,  , cones because they loosely correspond to its red, green and blue portions  (I will use the latter notation here).

Follow the Photons

Let’s follow light from the illuminant along its path to the eye, with its spectral power distribution (SPD) expressed as a photon distribution.  This is accomplished  by dividing the relative energy contained within the small wavelength interval used in the distribution by the energy of the average photon within it.  The resulting energy is given by Planck’s relation:

(1)  

with Planck’s constant, the speed of light and the average wavelength within the interval used in the distribution^[2].

Upon diffuse reflection by the tree the relative Spectral Photon Distribution of the illuminant (above right) is effectively multiplied wavelength by wavelength by the spectral reflectance of the foliage (below left).

The resulting relative spectral distribution of photons (above right) arrives at the eye which focuses it onto the fovea where , and cones reside.  The cones respond to the incoming photons according to their respective spectral sensitivities^[3], which determine the likelihood that a photon of a given wavelength will be absorbed^[4] (below left, the red green and blue curves referring to , and cones respectively).

Therefore Figure 4b above right represents the relative number of photons per small wavelength interval absorbed by each typical , and human cone for the given ‘daylight’ D50 illuminant SPD reflected diffusely by such foliage.

Cornsweet^[4] suggests that if a photon is absorbed we can assume that it produces exactly the same effect on a cone type independently of wavelength.  Therefore the integrals of the , , curves in Figure 4b above right represent the whole input signal to the human visual system.  For this example of green foliage illuminated by daylight the integrals evaluate to the following relative values:

It is this cone-collected signal which, after some processing, we recognize as Green!  The units are not critical at this stage because the system as described is assumed to be linear under photopic conditions and has been normalized: within some adapted limits if the illuminant’s power or Exposure is doubled/halved, the values double/halve and we still recognize the color as green.  I like to think of them as a relative number of photons (or quanta as Cornsweet suggests).  For a given light intensity range and adapted state what really matters as far as color is concerned  is the relative proportion of photons absorbed in , , and cones.

Sounds familiar?  That’s exactly how things work in a digital camera, except that instead of cones there are pixels  and instead of Spectral Sensitivity Functions there are the Spectral Sensitivity Functions of filters in the Color Filter Array (CFA).  Of course the proportion, layout and sensitivity of the three cone types is different than for instance in an ‘rggb’ Bayer sensor, but the underlying photon-counting principle is the same.

Color Blindness

The fact that an absorbed photon produces the same effect on a cone type regardless of its wavelength is sometimes referred to as ‘color blindness’ and extremely important.  It is the basis upon which color science is built  because if we can find a way, any way,  to stimulate someone’s  cones to the relative tune of 238, 197 and 38 within an adapted photopic-conditions range – that person will perceive a ‘foliage’ green color^[5].

Next, why we need color matrices.

 

PS. This little TED short drives the point home visually.

 

Notes and References

<sub>1. Measuring Color. Third Edition. R.W.G. Hunt. Fountain Press, 1998. Visual Signal Transmission.
2. In practice one can forget about the constants and simply multiply the Spectral Power Distribution by the average wavelength within an interval since all figures need to be normalized.
3. CIE 2006 ‘physiologically-relevant’ LMS functions, UCL Institute of Ophthalmology, Linear Energy 1nm 2 degree functions.
4. Visual Perception. Tom N. Cornsweet. Academic Press Inc., 1970.
5. Lots of provisos and simplifications for clarity as always.  I am not a color scientist, so if you spot any mistakes please let me know.</sub>