From Martin
Subject Canon 30D Raw Histograms
Date Mon, December 17, 2007 7:54


Hi!
I've read your page "Creating Raw Histograms From Digital Camera Raw Files"
and I have some (I think) interesting information to add:

The "combing" is really a scaling result. You already found out the strong
correlation between the combing and aperture size. Thanks for this research!

I did analyze the histograms of my photos, and I discovered that all photos
taken with the same ISO group and aperture have the same relative scaling
factor applied, see my tables below.
It was 100% consistent, not a single CR2 file was irregular!

Canon 30D uses integer scaling before writing raw data into .CR2:

Equation equivalent to: round( ((in-128)*scale/4096)+128 )
For ISO 3200, it is floor() instead of round().
This equation really gives all of the gaps/spikes at exactly the
same histrogram positions, that's how I found it out.
Sometimes scale/4096 could be simplified, like 8368/4096 = 523/256.

ISO 200ff means ISO 200, 400, 800, 1600
ISO 125ff means ISO 125, 250, 500, 1000
ISO 160ff means ISO 160, 320, 640, 1250

Scale [4096]:
aperture :        f/2.8+        f/2.5        f/2.2        f/2.0        f/1.8
ISO 100  :        4096        4141        4184        4250        4348  // same as ISO 200ff
ISO 200ff:        4096        4141        4184        4250        4348
ISO 125ff:        5311        5370        5426        5511        5639
ISO 160ff:         3158        3193        3226        3277        3352
ISO 3200 :        8192        8282        8368        8500        8696

Scale [%]:
aperture :        f/2.8+        f/2.5        f/2.2        f/2.0        f/1.8
ISO 100  :        100.00        101.10        102.15        103.76        106.15
ISO 200ff:        100.00        101.10        102.15        103.76        106.15
ISO 125ff:        129.66        131.10        132.47        134.55        137.67
ISO 160ff:         77.10        77.95        78.76        80.00        81.84
ISO 3200 :        200.00        202.20        204.30        207.52        212.30

Scale relative to f/2.8 [%]:  // only depends on aperture!
aperture :        f/2.8+        f/2.5        f/2.2        f/2.0        f/1.8
ISO 100  :        100.00        101.10        102.15        103.76        106.15
ISO 200ff:        100.00        101.10        102.15        103.76        106.15
ISO 125ff:        100.00        101.11        102.17        103.77        106.18
ISO 160ff:         100.00        101.11        102.15        103.77        106.14
ISO 3200 :        100.00        101.10        102.15        103.76        106.15

Clip value:
aperture :        f/2.8+        f/2.5        f/2.2        f/2.0        f/1.8
ISO 100  :        3398        3434        3468        3521        3599
ISO 200ff:        4071        4095        4095        4095        4095
ISO 125ff:        4095        4095        4095        4095        4095
ISO 160ff:         3168        3202        3233        3283        3355
ISO 3200 :        4095        4095        4095        4095        4095

True clip value before scaling:
aperture :        f/2.8+        f/2.5        f/2.2        f/2.0        f/1.8
ISO 100  :        3398        3398        3398        3398        3398  // scaling reversable w/o clipping loss
ISO 200ff:        4071        4052        4012        3951        3865
ISO 125ff:        3188        3154        3123        3077        3010
ISO 160ff:         4071        4071        4070        4071        4071  // same as ISO 200ff at f/2.8+
ISO 3200 :        2112        2090        2070        2040        1997

I tested with several lenses, some of them f/1.8 (Canon and Sigma), others
f/2.8 or less. I do not have a lens with aperture wider than f/1.8, that's
why I cannot give you the scaling factors for f/1.4 etc.

Looking at your table, you identified only 22-90% of the photos with
aperture f/1.8 or f/2.0 as being combed. I'm sorry to say, but the reason is
that your program rawhistogram.exe just does not recognize every scaled CR2.
I think, photos exposed very much to the left or right of the histogram,
are not recognized as being combed, although they are scaled just like the
others.
For strongly overexposed photos it is actually impossible to analyze the
histogram at all, but these should be sorted out rather than confusing the
statistics.
So, for the few of your photos, that have an aperture wider than f/2.8 and
that are not recognized by rawhistogram.exe as being combed, please have
a look at their histograms manually, and I think you will find them slightly
combed or under-/overexposed.

I think, digital sensors do register light rays the best when they arrive
perpendicular to the sensor surface, partly because of the micro lens array
in front of the pixels.
At very wide apertures, light rays arrive more inclined w.r.t. the
perpendicular, resulting in a slightly lower exposure. I think, the scaling
is intended to fix this. And maybe it is really a nice feature, as the raw
conversion software does not have to know the physical parameters of the
sensor.
And if the combing would be a problem for some application, one could
losslessly reverse the scaling (although for ISOs other than ISO 100,
the clipping would remain slightly earlier), only the downscaled ISOs
160ff are not reversable.

From your Canon 5D examples, I learned, that the 5D scales f/1.8 to 102.2%
and f/1.4 to 104.4%.

I have no idea about your combings with "EF24-85mm f/3.5-4.5 USM", maybe
there are special scalings for some lenses, or it is incorrectly recognized
as being combed.

Thanks for your work,
Martin

Note: Mark's last comment about the combings with "EF24-85mm f/3.5-4.5 USM" is easily explained - those frames were shot at intermediate (fake) ISOs.