Digital camera profiling with Raw Therapee and Argyll CMS

This article details a procedure for making simple matrix ICC profiles for digital cameras with Raw Therapee (RT) and Argyll CMS.

For this article I have profiled a Canon EOS 400D (Digital Rebel XTi) digital SLR (DSLR). I used an X-Rite ColorChecker Digital SG (CCSG) target. I also provide guidance for users of the X-Rite ColorChecker Classic (CC24) target. The illuminant was direct sunlight. And I have made worksheets for camera profiling with Argyll CMS that you might find helpful.

X-Rite ColorChecker Digital SG reference data

My CCSG target was not supplied with reference data and I do not have a spectrophotometer. I used generic reference spectral data, which can be obtained by downloading ProfileMaker 5 from X-Rite and temporarily installing it.

The X-Rite file first needs to be converted to Argyll .ti3 format. Then the spectral data can be converted to CIE XYZ and L*a*b*:
txt2ti3 ColorCheckerSG.txt ColorCheckerSG
spec2cie ColorCheckerSG.ti3 ColorCheckerSG.cie

The spec2cie defaults are appropriate: D50 illuminant and CIE 1931 2 degree observer. You can verify the results match the reference L*a*b* data on X-Rite’s support page. To save you time and effort, I have made available for download from my archives the reference spectral and CIE data.

Examining the target

Patches of particular interest on a profiling target are white and neutral grey. White is the brightest, neutral patch. Patch E5 on the CCSG reference data is white (L* = 96.52, a* = -0.9, b* = 0.6).

X-Rite CCSG target layout. One white (E5) patch and one middle-grey (H5) patch are circled. The CC24 patches are E2 to J5. The CCSG has a darker black (E6) than the CC24.

Neutral grey is a reference for white balance (WB). A perfectly neutral grey patch will have a* = 0 and b* = 0. Deviations can be evaluated by taking the square-root of a*-squared plus b*-squared (following the delta-E 76 formula). For the CCSG, there are 17 near-neutral (dE76 = 0.18-0.20), middle-grey (L* = 50) patches: A3, A6, A9, B1, B10, E1, E10, H1, H5, H10, K1, K10, M1, M10, N2, N5 and N8.

The CC24 grey scale is nearly the same as the CCSG, except that BabelColor’s average data shows that the third-brightest patch (G5 on the CCSG) is most neutral. This might not be true for individual charts however.

CCSG delta-E 76 CC24 delta-E 76
Patch X-Rite X-Rite Babel
Color
White 9.5 E5 1.1 1.3 2.3
Neutral 8 F5 0.4 0.7 0.6
Neutral 6.5 G5 0.3 0.9 0.4
Neutral 5 H5 0.2 0.3 0.7
Neutral 3.5 I5 0.3 1.3 0.7
Black 2 J5 1.3 1.0 0.4
CCSG black E6 0.3
Neutral errors (delta-E 76) for grey scale patches on X-Rite CCSG and CC24 camera profiling targets. For the CC24, two data sources are presented.

Selecting an image for profiling

I previously recommended to photograph the profiling target in morning sunlight and from a low angle to reduce glare.

My highlights strategy requires a photo around 1/3 or 2/3 stop overexposed. Load a candidate image in RT and check there is no clipping in the raw RGB histogram. Exposure will be checked again and fine-tuned with Argyll CMS.

Raw RGB histogram of profiling photograph in RT. The maximum linear device RGB is about 67 (on a 0-100 scale). For linear RGB, it is normal that most of the information is in the lower half of the graph.

Raw RGB histogram of profiling photograph in RT. The maximum linear device RGB is about 67 (on a 0-100 scale). For linear RGB, it is normal that most of the information is in the lower half of the graph.

Preparing the image in Raw Therapee

The selected image requires some pre-processing. Open the image in RT and load the neutral profile, which should switch off all adjustments. Then adjust the Color settings as follows:

  • Input Profile: No profile.
  • Working Profile: Irrelevant (no adjustments will be made in the working space and there is no Output Profile).
  • Output gamma: linear_g1.0 (this will disable the Output Profile).

Beware that the Working Profile can’t be disabled in RT and the colour values reported are for the RGB working space, not the linear RGB output. To inspect linear RGB values, we need to save the image and then use scanin from Argyll CMS.

Next, make a Custom WB with the Spot WB tool. For a quick result, white balance on a single patch (H5 or G5).

For direct morning sunlight, the colour temperature should be near 5000 K and the tint should be near 1.0. For this example, the final WB was temperature = 4749 K and tint = 0.980. RT colour temperature estimates are about 300 K low at 5000 K for Canon DSLRs.

Without individual or at least batch measurements, one can’t be sure which grey patches are closest to neutral. For the final profiling image, I spot WB all 17 middle-grey patches on the CCSG and compute the average. For the CC24, I suggest to spot WB the third- and fourth-brightest patches and compute the average.

Almost ready for profiling, crop to the target edges and save as a 16-bit uncompressed TIFF file. Alternatively, export a linear RGB image directly from RT (Save reference image for profiling in the Color tab) and crop the output in an image editor. I’ve tried exporting both ways and the patch values were identical.

Checking the image with Argyll CMS

Argyll CMS first has to read the patches in the TIFF file:
scanin -v -p -a -G1 -dipn Canon400D.tif ColorCheckerSG.cht ColorCheckerSG.cie
-v Verbose output.
-p Perspective correction.
-a Recognize chart in normal orientation (not upside-down).
-G Gamma for the TIFF file (1 = linear). Use to increase brightness for dark images. Do not use for light images.
-d ipn Generate diagnostic output. For checking the chart has been read correctly.
ColorCheckerSG.cht is the recognition template file from the Argyll CMS /ref folder.
ColorCheckerSG.cie was generated with spec2cie (detailed above).
Canon400D is the name of the input(.tif) and output (.ti3) files.

Always check the diag.tif diagnostic output to make sure patches have been identified correctly – this will save confusion later on. Cropping less or more tightly can solve chart recognition errors and misplaced patches.

Detailed quality control checks can be made from the .ti3 output. For this, I copy the patch data from the text file and paste it into a spreadsheet (space delimited). Here are some average patch readings from the .ti3 output (final results):

  • White (E5) = 66.7 (linear RGB, 0-100)
  • Grey (H5) = 14.5
  • Black (E6) = 1.1
  • Contrast Ratio = 60.0

If any patches are clipped (100) then choose another photo with a faster exposure. The above result is about 0.5 stops below clipping. Unfortunately, I had deleted the photos with positive exposure compensation (+1/3 and +2/3 stops).

Glare will cause black patches to read high. The table below provides expected linear RGB values and contrast ratios (white/black). For direct sunlight, the maximum contrast ratio I can achieve for the CCSG is about 60. Lower contrast suggests glare (try adjusting the shooting angle) or more diffuse lighting (e.g. clouds).

CCSG linear RGB CC24 linear RGB
Patch X-Rite X-Rite Babel
Color
White 9.5 E5 90.8 89.9 89.9
Neutral 8 F5 55.8 59.0 58.4
Neutral 6.5 G5 34.2 36.4 35.7
Neutral 5 H5 18.2 19.2 19.1
Neutral 3.5 I5 8.6 8.9 8.9
Black 2 J5 2.1 3.2 3.2
CCSG black E6 0.7
Contrast (W/B) 122 29 28
Linear RGB data and contrast ratios for X-Rite ColorChecker Digital SG (CCSG) and ColorChecker Classic (CC24) camera profiling targets. For the CC24, two data sources are presented. L*a*b* chart reference data were converted to XYZ and then to Wide Gamut RGB, which has the same D50 white point.

The middle-grey patches around the edges of the CCSG can be used to measure uniformity of lighting/reflectance from the .ti3 output. If brightness variations across the target are greater than 2 or 3%, try for a better image. I think X-Rite’s profiling software can use the edge patches to correct for non-uniformity across the chart.

Good uniformity example.

Bad uniformity example.

The X-Rite ColorChecker Passport is a small target and will be less affected by uneven illumination. However, I am not sure about impromptu use of the ColorChecker Passport in the field, as promoted by X-Rite. Without a careful setup, there could be unwanted reflections (e.g. green vegetation, bright clothes, buildings) that contaminate the colour of the illuminant and the field profile would then be inaccurate.

Making a camera profile in Argyll CMS

Next, Argyll CMS is used to fit a simple matrix model:
colprof -v -y -A “Canon” -M “EOS 400D” -D “Canon 400D sunlight matrix” -C “None” -qh -am -nc -U1.47 Canon400D

Profile check complete, peak err = 8.733370, avg err = 2.66556

-v Verbose output.
-y Detailed errors output.
-A Manufacturer (optional).
-M Model (optional).
-D Description (could be useful).
-C Copyright.
-q Quality.
-a m Algorithm, matrix.
-n c No input .ti3 data in the profile
-U Extrapolate white point.
Canon400D is the name of the input(.ti3) and output (.icc) files.

For matrix profiles, the parameter -U scales the linear RGB. Equivalently, I could have applied the same linear scaling to exposure in Raw Therapee and no scaling in Argyll CMS.

I estimated the white point scaling parameter from the scanin white patch reading: U = 100/66.7 = 1.50. I then made fine adjustments in colprof so that none of the RGB channels exceed 100 when plotting the neutral response of the profile:

xicclu -g -f b Canon400D.icc
-g Plot device values along the neutral axis.
-f Conversion (b = backwards).

Canon 400D matrix profile neutral response with 1.47× white point scaling, which has brought the maximum linear device RGB (vertical axis) very close to 100. The closeness of curves indicates the profile is very close to neutral.

Canon 400D matrix profile neutral response with 1.47× white point scaling, which has brought the maximum linear device RGB (vertical axis) very close to 100. The closeness of the RGB curves indicates the profile is neutral.

Canon 400D matrix profile neutral response with no white point scaling. Linear device RGB values above 67 (vertical axis) will be clipped by this profile.

Canon 400D matrix profile neutral response with no white point scaling. Linear device RGB values above 67 (vertical axis) will be clipped by this profile.

Canon 400D matrix profile neutral response with 1.50× white point scaling. While the maximum average linear device RGB is approximately 100, the white response is not perfectly neutral and the red channel value is clipped (> 100). The Argyll CMS automatic scaling parameter -u produced a nearly identical plot.

Canon 400D matrix profile neutral response with 1.50× white point scaling. While the maximum average linear device RGB is approximately 100, the red channel is clipped (> 100). The Argyll CMS automatic scaling parameter -u produced a similar plot.

Profile white point scaling is the key to retaining highlight details. My comparisons of different profiles has shown that white point scaling does not affect colour fidelity and Adobe DCP profiles appear to use similar white point scaling.

A simple matrix profile is a 3×3 matrix that transforms white-balanced linear device RGB to D50 XYZ. The white point and black point tags will not be used for colour management. So don’t worry about the white point lightness, which exceeds 100 for this profile:

iccdump -v 3 -t wtpt Canon400D.icc

XYZArray:
No. elements = 1
0:  1.236206, 1.278946, 1.033493    [Lab 109.914395, 0.446556, 1.485515]

-v Verbosity level.
-t Tag (wtpt = white point).

Results

For a second opinion on profiling errors, delta E 2000 can be calculated with profcheckDelta E 2000 is more realistic because human vision is more sensitive to hue differences than chroma and lightness.

profcheck -v 2 -k Canon400D.ti3 Canon400D.icc

Profile check complete, errors(CIEDE2000): max. = 4.483594, avg. = 1.761331, RMS = 2.065521

-v Verbosity level (2 = print each DE).
-k Report CIE DE2000 delta E values.
Canon400D is the name of the test data file (.ti3) and profile (.icc).

Profile errors are better summarised graphically, focusing here on the CC24 patches:

Delta-E 1976 error summary for Canon 400D sunlight profile.

Delta-E 1976 error summary for Canon 400D sunlight profile – pessimistic.

Delta-E 2000 error summary for Canon 400D sunlight profile.

Delta-E 2000 error summary for Canon 400D sunlight profile – realistic.

Only the black and some other very dark patches have large DE00 errors (> 4), caused by glare. Some moderate DE00 colour differences (> 2) could be apparent in side-by-side comparisons, but are not practically important. There are more substantial effects in image processing, such as white balance and tone curves.

A simple 2-dimensional plot can be made from the rXYZ, gXYZ and bXYZ tags in the profile. Also check that the profile estimated white point (wtpt tag) plots at D50.

Gamut plot for Canon 400D sunlight profile. RGB working space gamuts for comparison are Adobe RGB (smallest), Wide Gamut RGB and Pro Photo RGB (largest).

Gamut plot for Canon 400D sunlight profile. RGB working space gamuts for comparison are Adobe RGB (smallest), Wide Gamut RGB and Pro Photo RGB (largest). The device primaries are model estimates and were not measured.

The gamut of the CCSG target is small compared to the camera and plots mostly within Adobe RGB. Therefore, my profiles are best used with sRGB or Adobe RGB. Accuracy for very large working spaces has not been tested.

Dynamic range compression (tone curves)

The camera profile generated above outputs “scene-referred” images and they are too dark because of white-point scaling. Scene-referred images need to be translated into “output-referred” images where contrast is increased for displays and especally prints having limited brightness and dynamic range. Karl Lang has written a good introduction to rendering raw photos.

To compress the dynamic range of the scene-referred data and adjust brightness, I apply an RGB tone curve in RT. RGB tone curves are best because they do not cause hue shifts. A good default rendering is to mimic the the camera manufacturer’s JPEGs:

  1. Process the profiling raw photo in Canon Digital Photo Professional (exposure corrected, manual WB on neutral grey, output profile Adobe RGB, no other enhancements).
  2. Measure L* for each patch in the grey scale (patches E5, F5, G5, H5, I5, J5, E6 for the CCSG) in Photoshop or RT (with working space = Adobe RGB).
  3. Open the profiling raw photo in RT, apply the custom camera profile, Adobe RGB working space, WB on neutral grey and apply any exposure adjustment that might have been made before profiling.
  4. Make a parametric RGB tone curve in RT to match the grey scale to Canon’s L* response.

Using my profiles

You can download my Canon 400D and 350D camera profiles and RGB tone curves for RT from my archive.

To have RT automatically select a profile, match the file name to the EXIF Camera Model tag (e.g. Canon EOS DIGITAL REBEL XTi.icc). and copy to RT’s /iccprofiles/input folder. Then select Auto-matched camera-specific profile in the Color tab of RT. Do not blend highlights with matrix in RT, which results in clipping and false colours.

Here are some samples for visual comparisons.

A clear winter morning. Processed by RT with matrix ICC profile, grey card WB and RGB tone curve to mimic Canon response. These are genuine colours, I have not fiddled with saturation or anything else.

Processed by RT with camera standard profile (dcraw colour matrix), grey card WB and RGB tone curve to mimic Canon response. Red is weak!

Processed by RT with RT DCP profile, grey card WB and RGB tone curve to mimic Canon response. A good result and similar to my matrix profile.

Processed by Canon DPP with standard picture style and grey card WB. The sky looks yuck!

Processed by Canon DPP with landscape picture style and grey card WB. This result is more vibrant, but the sky colour has shifted towards magenta!

Advertisements

5 Responses to Digital camera profiling with Raw Therapee and Argyll CMS

  1. crgsen says:

    Is RT performance (algorithm) comparable to the ever improving versions of ACR or DXO? (eg. ACR has improved their highlight recovering from ACR 6, whereas DXO performs custom lab tests on specific cameras and lenses to correct; fine tune the camera or camera and lens profile.

    Something a user would have to be aware of and adjust accordingly in RT, (will be just as good or somewhat off?)

    Some forum users have complained about DXO’s choice of color space. (Adobe RGB, excluding ProPhoto) What are your thoughts? Output to an image processor later on to further adjust?

  2. Nick says:

    Could you explain how you produced those lighting uniformity charts? Was that a function in Argyll CMS or did you create them yourself from the .ti3 data? I’m trying to find an open-source way of measuring lighting uniformity on a Colorchecker SG card and those charts seem to be exactly what I’m looking for.

    • bulumakao says:

      The .ti3 is simply a text file.
      You can find the RGB values for each patch in this file, e.g. B1 = 18.3, 18.4, 18.4 and average = 18.4. Note that Argyll CMS scales the RGB values to 0-100.
      Then you can compare the average values for patches that are designed to be the same. I import the text file into a spreadsheet and all the computations are done automatically.
      Cheers.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: