Taking the Sharpness Model for a Spin

The series of articles starting here outlines a model of how the various physical components of a digital camera and lens can affect the ‘sharpness’ – that is the spatial resolution – of the images captured in the raw data. In this one we will pit the model against MTF curves obtained through the slanted edge method^[1]from real world raw captures both with and without an anti-aliasing filter.

With a few simplifying assumptions, which include ignoring aliasing and phase, the spatial frequency response (SFR or MTF) of a photographic digital imaging system near the center can be expressed as the product of the Modulation Transfer Function of each component in it. For a current digital camera these would typically be the main ones:

(1) $\begin{equation*} MTF_{sys} = MTF_{lens} (\cdot MTF_{AA}) \cdot MTF_{pixel} \end{equation*}$

all in two dimensions, where

$MTF_{lens}$ is the modulation transfer function of the lens, function of diffraction (itself function of aperture shape, the weighted mean wavelength of light $\lambda$ and effective f-number $N$ ) and aberrations, as outlined in this article for SA3 and Defocus;
$MTF_{AA}$ is the modulation transfer function of the Anti-Aliasing or Low Pass filter that if present sits on top of the sensor, as outlined in this article; and
$MTF_{pixel}$ is the modulation transfer function of the pixel aperture, function of pixel size, fill factor and shape as outlined in this article.

Missing from this equation is sampling pitch, which we will ignore in this post^[2]. The model will assume a lens with a circular aperture, a bi-refringent AA (if present) with first MTF0 null at spatial frequency $AA_0$ and contiguous square pixels of width $w$ with an effective fill factor of 100%.

Directional Linear Spatial Resolution

Although equation (1) works in two dimensions we will restrict the discussion to directional one dimensional results by taking a radial slice off the two dimensional system MTF, as shown in the example below:

mtf-and-radial-slice — Figure 1. 1a, Left: 2D Lens MTF displayed as a solid with a directional radial slice shown in red. 1b, Right: the one dimensional radial slice of the 2D MTF; the curve shown applies in the direction of the slice only. The x-axis represents linear spatial resolution (arbitrary units in this example).

The two dimensional MTF solid in figure 1a is also often shown from the top as the flat black and white image resulting from a 2D Discrete Fourier Transform.

It is useful to concentrate on the directional one dimensional MTF seen in figure 1b because the real world data that we’ll be getting comes from the slanted edge method, which produces a directional linear spatial frequency response like it^[3], making theoretical and real data directly comparable.

Model vs Real World MTF Curves – No AA

The latest iterations of open source MTF Mapper by Frans van den Bergh^[4] have been getting better and better. The radial MTF curves it produces from slanted edges captured with good technique are better corrected and less noisy than ever. In the example at hand the MTF curve of a Full Frame Bayer sensor D810 coupled with a Nikkor AP-S 85mm/1.4G lens was measured from the white-on-black slanted edge just below the center of this ISO64 f/5.6 raw capture by DPReview.com

Figure 2. DPReview.com Test Scene raw file DSC_7553.NEF, captured by a Nikon D810 mounting an AF-S 85mm:1.4G lens at ISO64 and F/5.6. Note the two black and white slanted edges just below and to the right of center.

Note that the printed edges are not exactly in the center of the field of view. The slanted edge method produces the two dimensional spectral frequency response of the system as a whole, including the edge, evaluated in the direction of the edge normal; since the slant in this case is about 9.39 degrees off the horizontal the resulting curve will refer to system MTF performance in a south-easterly direction, about 25 minutes on the clock.

The Output of the Model

MTF Mapper Version 0.5.8 produces the following MTF curve for the indicated camera and lens in the center of the field of view, from a region of interest 400 pixels wide centered on the edge:

d810 — Figure 3. Real world green channel SFR curve from a Nikon D810 mounting an AF-S 85mm:1.4G lens, measured off the slanted edge below center by MTF Mapper (dotted line); and theoretical MTF curve computed according to the model in equation (1) with the parameters listed in the figure (solid line).

The blue dots represent the spatial frequency response of the green channel measured by MTF Mapper while the solid line is the directional system MTF predicted by the model in equation (1) based on the parameters shown in the box. Refer to the links in the introduction to see how the MTF curves of the individual components are determined based on the shown parameters.

The linear spatial frequency x-axis is in cycles per pixel because we are assuming contiguous pixels with 100% fill factor. It can be expressed in cycles per picture height by multiplying it by 4920 px/ph – or in lp/mm by dividing the latter result by 24 mm/ph^[5].

For reference the green channel’s MTF50 occurs at 0.321 c/p or 1582 lp/ph or 65.9 lp/mm. Since the target is effectively neutral the Shannon-Nyquist frequency is assumed to be 0.5 c/p. The D810 clearly does not have an AA filter so the curve hits Nyquist at MTF27, leaving a lot of energy above that to generate aliasing and moiré.

The Model’s Components and Limitations

Recall that the simplified model in equation (1) assumes a perfect edge and attempts to account only for diffraction, third order spherical aberrations, defocus, AA (absent in this case) and pixel aperture. The MTF curve of each of these system components is shown in Figure 4 as dashed lines (extracting diffraction from $MTF_{lens}$ for additional detail). Multiplying together the relative MTF values at every spatial frequency yields the Modeled system MTF, the solid orange curve.

mtf-components — Figure 4. Main components of modeled System MTF.

These components should allow us to model to a decent approximation the performance of the imaging system in the center of the field of view of an in-spec lens, where other aberrations should be absent or at least at a minimum. Of course the slanted edges are not exactly in the center^[6] – and in-spec is a working range, not a precise number. There may also be a number of potential small errors that could have infiltrated the raw data at the time of capture (missed peak focus and motion/vibration blur at the top of the list). Plus of course the MTF of the edge itself is unknown and swept up into the only variable fit term here, SA3+Defocus, so these results have more relative than absolute value. Consider it a first trial run.

The Input to the Model

As seen in figure 3 the inputs to the model were

$\lambda$ = 0.535um, the presumed weighted average wavelength of light in the green channels as captured
$N$ = 5.6, the effective f-number
$w$ = 4.878um, the pixel width (also assumed to be sampling pitch here)
$W_{040}$ = 0.354 $\lambda$ , the coefficient of pupil radius to the fourth power in the model, associated with peak wavefront Optical Path Difference of third order Spherical Aberration
$W_{020}$ = -0.021 $\lambda$ , the coefficient of pupil radius squared in the model, associated with peak wavefront Optical Path Difference of Defocus
$AA_0$ = zero, meaning no antialiasing filter.

Estimated physical parameters $\lambda$ , $N$ and $w$ were entered manually into the model, keeping in mind that manufacturer indicated values for f-number can easily be off by 10% or more, especially at higher numbers. The $W_{0X0}$ OPD values were instead varied to make the measured and modeled curves look as similar as possible.

The $W_{0X0}$ OPD and $AA_0$ Fit

The fit to the theoretical MTF curve shape looks good. This happened less frequently until the third order spherical aberration term was introduced into the model. The main discrepancy occurs at higher frequencies where the MTF is lower than 0.1. When approaching zero MTF noise is rectified by the absolute math and it is unable to average itself out, hence measured data usually appears higher than the model at very low energy levels. The fact that incident light was polychromatic can affect MTF values near zero. And the D810’s effective pixel fill factor may be less than 100%.

The $W_{0X0}$ OPD coefficients were varied by Matlab’s Nelder-Mead simplex direct search algorithm until the difference between modeled and measured data hit a minimum. In this case there was no AA. The algorithm attempted the fit in the 0.1 to 0.75c/p range only (or 10%-80% of diffraction extinction, whichever is lower) to stay away from the noisy lower MTF values.

I am not sure whether 0.35 $\lambda$ SA3 OPD is reasonable for a single bayer channel, although I have read that it is fairly typical for in-spec photographic lenses to have a quarter to a half of SA3 OPD in the center. I’d be interested in the thoughts of someone in the know.

Keep in mind that here the MTF of the wavefront errors were effectively used as a plug. If, contrary to the assumption in this post, other errors or aberrations were present in the captured data (for instance an imperfectly printed edge, axial color or astigmatism) the absolute OPD values would lose some of their physical meaning. For instance I am sure that we are capturing an imperfect edge and some Axial/Longitudinal Chromatic Aberrations (Spherochromatism) under the SA3 and Defocus label here. Since the fits are so good, obviously their aggregate behavior is somewhat similar to that of the plugs.

Model vs Real World MTF Curves – with AA

Below is the same exercise performed on a camera with an anti-aliasing filter, a Canon 5D Mark IV mounting an EF85mm/1.8USM lens. The target is the same as for the D810 system above, the slanted edge below center captured in the raw data at ISO100 and f/5.6 by DPReview.com, green channel MTF curve shown:

5dm4 — Figure 5. Real world green channel SFR curve from a Canon 5D Mark IV mounting an EF85mm/1.8USM lens, measured off the slanted edge below center by MTF Mapper (dotted line); and theoretical MTF curve computed according to the model in equation (1) with the parameters listed in the figure (solid line).

The fit looks excellent here as well. In this case the Anti-Aliasing filter first zero $AA_0$ was allowed to vary in addition to the two $W_{0X0}$ OPD coefficients. The algorithm suggested a best fit with the null at about 0.8 c/p, at the weaker end of the range of AA strengths I have measured over the last couple of years. Because of the AA, Nyquist (0.5 c/p) is hit at MTF14, substantially lower than by the AAless D810 combination. This bodes well as far as minimizing the potential for aliasing and moiré in the final photograph – at the cost of some lost sharpness throughout the range.

Third order Spherical Aberration is very similar to the previous case above with $W_{040}$ of 0.346 $\lambda$ . Defocus is instead positive with $W_{020}$ = 0.07 $\lambda$ . Any help interpreting these results would be appreciated.

In the next post we will apply the model to a number of other DPReview.com raw captures stemming from cameras with different formats and pixel pitches.

Notes and References

_{1. See here for a description of the slanted edge method as used in this article and how you can obtain MTF curves for your equipment from it.

2. While in general it is reasonably safe to assume that pixel pitch is directly related to the pixel’s physical characteristics, newer technologies like pixel shift can effectively cause pitch to become independent of pixel size and shape.

3. The slanted edge method produces a directional radial slice of the 2D MTF of the 2D PSF of the imaging system. Its results represent the linear spectral frequency response of the imaging system in the direction of the slice.

4. Frans van den Bergh’s excellent blog is here; Open source MTF Mapper can be downloaded from here.

5. See here for a discussion of the units used in spatial resolution.

6. In fact the vertical edge to the right of center gives slightly different results, with MTF50 at 0.317 c/p or 1561 lp/ph or 65.0 lp/mm.

7. Contact me via the form in the About page top right if you would like a copy of the Matlab/Octave scripts used to generate some of the figures in this post.}

Strolls with my Dog