Colour Correction Concepts for Monitors

Colour management for the desktop is a long standing issue not only for Linux. The following text will concentrate on colour correction of monitors. This means the experience, when you switch your computer or handheld on and look on the screen.

Why colour correct the desktop?
People tend to compensate for quite a lot of different types of monitors. They are most often able to adapt to one full screen and see colours as they are intended to look like. That’s fine as long as they are concentrated on the visual event on that one display. But in this article we want to discuss how to compensate the monitor colours to our human visual needs in environments with various displays side by side, as is the typical situation for more and more people today. They take pictures with mobile phones or other digital cameras and look at them on laptops, tablets, picture frames, TV sets and printouts often side by side. Or we look at them over the internet and want to share the same visual impression with other people. Many people with uncorrected systems see quite a difference between various colour devices and try to figure out how to conveniently synchronise them. Colour management should help accomplish that task.

What is used today on Linux for colour improvements?
Different display devices feature different colour gamuts and colour appearance. We are especially sensible to the gray balance. This is a important property for our visual experience. Gray balance is since long time maintained by placing three linear correction curves into the graphics card. They are known as VCGT tag. Properly done, this helps in maintaining a equally stepped gradient from black to white with neutral shades in between. This calibration technic is fast and deployed since a long time by tools like xcalib. Calibration means in ICC terms, to setup a device driver in order to deliver a good and stable colour response. Strictly it is not part of the ICC specification. But for practical reasons it is usually embedded in ICC profiles. This basic calibration with linear curves is still in use on Windows, osX and X11 systems.

Acer Extensa 5630EZ laptop monitor

Different primary colours become more and more an issue through evolving LCD display technology. The above and below CIE shoe diagrams show quite different gamuts between the three primary colours red, green and blue of a typical laptop above and a wide gamut monitor below.

HP LP2481zx wide gamut monitor

It is generally desirable to use a screen, which is able to show deeply saturated colours. But uncorrected images show much shifted colour primaries to better cover human visible colours. Compared to traditional monitor primaries green typical becomes more cyan and red is moved toward the purple line. Of course the same RGB number triple looks quite different on booth devices. A calibration technic like VCGT linear per channel curves can not correct in any way for these saturation and hue shifts. Colours look strange not only on side by side comparisons of such devices. We need a colorimetric description of the monitor to properly colour correct the given device response.

Historically we have seen three approaches to cover the colour correction of the full screen.

  • X11 systems are colour management wise much like Windows. They predate ICC colour management and therefore have no historical APIs to enforce throughout colour correction. Maintainance of gray balance through VCGT is possible. The X11 XCMS extension was to our knowledge never in real use. Never APIs can bring a requirement for colour management. Few do today. Differences between corrected and non corrected content on one screen can be very irritating.
  • A very rigorous answer to the situation of missed colour management APIs is a fixed colour correction in hardware. But this can limit the capabilities of monitors to sRGB and thus the experience of wide gamut monitor technology not available. On the other side it provides a way to synchronise tightly controlled display devices.
  • On osX the whole rendering pipe line is colour managed. Each colour must have a colour space assigned to or it is not accepted by the systems graphic APIs. As a result all content from applications is converted on the fly by Apples ColorSync to the according monitor and looks correct. ColorSync takes over the work to bring colours from a source colour space like LStar-RGB to a native destination colour space like from a monitor. This helps programmers to concentrate on their actual task, which is most often not colour management related at all.

What is done so far to get the colour correction ball rolling?
In 2008 we started a project for OpenICC to do colour correction on the GPU on top of X11. The project allows to colour correct all non colour characterised desktop areas. Non colour characterised means by definition all window regions, except actively marked regions for opt out of system colour management. If no colour managed application is running, this means a full screen colour correction. These non colour characterised regions or areas are handled similar to other non colour characterised content. Such colours are considered to be in sRGB In the printing world for instance. sRGB is the internets default colour space. This approach is a combination of the above outlined ones and an answer to the historical X11 specifics. The advantage is, that legacy APIs and applications can benefit from a colour corrected desktop, while updated applications will be able to opt-out of colour correction. Thus a situation like on osX, with all content clearly characterised, can be reached gradually. During the project was a protocol designed for server client communication. The belonging spec is the X Color Management spec and is maintained by me over OpenICC. The spec was previously called net-color spec, but was renamed due to name space conflicts. For a smooth transition of legacy applications, the implementation of the spec in libXcm contains the xcm tool. This tool allows to set colour regions manually as a workaround. LibXcm covers just the communication protocol for applications and toolkits to talk to a colour server. The colour server, for instance CompICC, is responsible for doing the colour correction on GPU. If you want to add a colour server to your favorite compositing window manager or have questions around the spec, get in contact with me.

Can the X Color Management spec do compositing?
During a discussion on a Qt email list came out the question about compositing and colour correction. Most obviously the spec misses a freely selectable compositing colour space to do full blown up compositing. this would mean blurry region borders and other technics where transparency is involved. That is simply out of scope for the X Color Management spec. It is logical in the responsibility of the compositor to decide about the compositing colour space. Compositing window managers on Linux use today just the plain monitor colour space for their compositing. That is not perfect but seems to be good enough for simple transparency effects. One correct way would be to meld the compositing of the window manager into the compositing of the client, which is in many cases the toolkit. But that is a bold adventure.taking the freedom of toolkits away and making traditionally modular stuff very monolithic. A architectural more sensible approach would be to use an intermediate blending space on top of the X Color Management spec. Thus the blending space is correctly selectable and modularity, interoperability and so on is much better. The toolkit can then rely on the compositing window manager to do all complicated transforms with the provided window and do colour correction as a final step. That would then essentially become a per window colour correction. From a today’s point of view the intermediate per window blending space needs a lot to do inside toolkits. It might take years to get there as toolkits just started about first steps. So the per region colour corrected desktop is still a valuable concept for today’s needs.

What is needed in the future?
Future developments have to take into account that the X11 architecture might change and other approaches become possible. The design of new architectures should cover colour management right from the beginning. With Wayland on the horizon and OpenGL-ES this is technically not a problem and would solve legacy issues like we see today in X11, on Windows and all major mobile platforms.

Colour management results of Google Summer of Code 2011

We are pleased to announce the final results of this years OpenICC participation in the Google Summer of Code program [1]. All students could reach successfully the project goals and obtained the second part of the Google stipends.

OpenICC mentored two students directly and one student through the collaboration with the openSUSE organisation. All three worked on colour management projects, which covered in parts new ground breaking technology and maintain existing one.

Yiannis Belias worked on the “API stabilization for Oyranos Colour Management System II” project [2]. The new classes, code generator improvements and tools will walk into the Oyranos master branch in the next months. This project helps in stablising the CMS core, which covers a great foundation of functionality. After the conversion, it will be much easier to provide stable and extensible APIs.

Joseph Simon realised the XCPD project. It’s goal was to implement the idea of a robust, standards conform printing dialog, which is based on the Common Printing Dialog (CPD) projects code. The handled and manipulated PDF follows the PDF/X specification for embedding user side colour managed content and remote printer configuration [3] [4].

Sebastian Oliva implemented a ICC device profile data base, called taxi. It is intented to share vendor and user created ICC profiles across platforms in a automated fashion. The online data base is designed to cover meta data about the device driver calibration status alongside the characterisation information in the belonging ICC profiles. The DB is online [5] [6]. The project slot was provided by the openSUSE distribution project and mentored by a OpenICC member.

Google Summer of Code is a global program that offers student developers stipends to write code for various open source software projects.

Many thanks to all students for their great work, all people how helped in shaping the basic ideas and discussing the projects in detail and Google for providing the stipends and openSUSE for addtionally inviting their students to the europe openSUSE Conference.

[1] http://code.google.com/soc/
[2] https://github.com/yiannis/Oyranos
[3] https://www.gitorious.org/google-summer-of-code-2011/xcpd
[4] http://jsimon3.wordpress.com/2011/08/23/gsoc%E2%80%9911-update-%E2%80%93-final/
[5] http://icc.opensuse.org
[6] http://openicc.git.sourceforge.net/git/gitweb.cgi?p=openicc/taxi

Oyranos 0.3.2

The new version of the colour management system Oyranos is released.Version 0.3.2 is a feature and bug fix release.

Changes Overview:

  • prefere normal ICC profiles over automatic generated ones
  • embedd and extract JSON device calibrations for ICC profiles
  • add oyranos-profile tool
  • add oyranos-profile-install script
  • prepare CUPS module for XCPD usage
  • harmonise monitor and libraw modules device DB keys
  • add example to embedd libraw device calibration into a ICC profile
  • switch to bzip2 compressed packages for RPM and other fixes
  • add man pages to all tools
  • add automatic generated help text in oyranos-policy -help

About:
Oyranos is a colour management system allowing to share various settings
across applications and services. The provided interfaces are the
C library, native graphical front ends and partitial access through
command line tools. The library is licensed under newBSD.

Known Applications using Oyranos:
ICC Examin, the KDE Kolor Manager and Synnefo configuration dialogs, the
CompIcc colour server and more.

ChangeLog:

http://www.oyranos.org/scm?p=oyranos.git;a=commitdiff;h=refs/heads/0.3.2

Thanks to all contributors and bug reporters.

Download:
git: git://www.oyranos.org/git/oyranos
git sha1: 238cb2dd27e624ef6167313e077d3b2a18a442f3
file: http://sourceforge.net/projects/oyranos/files/Oyranos/Oyranos%200.3/oyranos-0.3.2.tar.bz2/download
size: 1247164
md5sum: c80d142741c284b06cbea29b9d859933 oyranos-0.3.2.tar.bz2
sha1sum: a216254b8d0cd2563d3781fea57169d5b70a71e8 oyranos-0.3.2.tar.bz2
Linux RPM:http://download.opensuse.org/repositories/multimedia:/color_management/

dispcalGUI 0.7 @ rwx³

My yesterday held workshop for calibrating and profiling monitors using dispcalGUI+ArgyllCMS in Nuremberg was a nice experience. Around ten people from the openSUSE Conference gathered, all being eager to do something for their health of colour vision. We wanted to create ICC device profiles and collect them for later publishing. Following is a small report and review from the workshop.

We had available a i1display, a DTP94 and a i1pro as measurement devices, which no of the attendees owned privately. Installation on openSUSE-11.4 went pretty smooth. ArgyllCMS is in the multimedia:photo repository and dispcalGUI is in multimedia:color_management. Both are easily searchable through the http://software.opensuse.org URL. One hacker had all packages installed in advance and could start instantly with the i1display. With the default settings in dispcalGUI appeared a small terminal and required to adjust native monitor settings or continue with calibration. We pressed number 7 and continued with the calibration part. This lasted relatively long. dispcalGUI and in the background ArgyllCMS took quite some work to iterate over the calibration for four times with lots of “regression getting worse” style messages. This indicated to us, that it reached an end for improvement or the application where not satisfied according to each persons like. The profiling and installation finished after quite some time and the new calibration and profile could be used in colour managed applications. In Gimp the monitor profile was not used by default. The Colour Management tab in Gimp’s preferences needs to manually enable the system profile, which is a troublesome exercise to figure out. Instead colour managed applications should look first at the system profile. Overriding the system profile by default is dangerous for a good user experience.

The next people became ready and used the DTP94 in between. Unfortunately there where four people failing to use it. What looked like a defect DTP94 device cable of my best colorimeter, was observed by three users as continuous USB device ID switching. One could spot the reason to a conflicting UDEV rule of libmtp. I filled a bug report on SourceForge including the patch of that user. With that fix the DTP94 worked again. I am happy to can use the DTP94 in the future.

I figured out how we can speed up the calibration, by reducing the calibration quality, which was nice in such a big group. One attendee had a repeated black switching of his laptop monitor during the whole process in connection with the nv driver. That persisted even during the profiling stage, which makes no sense to me. I suggested him to install the nvidia proprietary driver especially for his installed Quadro card.

In several cases dispcalGUI crashed without providing further informations on the command line. We had then to repeat the process. Luckily a longer break allowed us to finalise most ICC profiles. I requested the monitors EDID data block from all people, to have the device information available alongside the profile. This can be done by installing the oyranos-monitor package, which brings relatively many dependencies, like a default profile set and several libraries and modules. oyranos-monitor -f edid -o monitor.edid could then be used to collect that data. Some installations did not work as expected and I have to look into the issue. The new profiles need now further preparation before they can be uploaded to icc.opensuse.org.

It was fun to see how most issues where resolved in the group, and that most people could get a device profile for their usage. Registration of a smaller number of people would be helpful to ensure no overload of capacities. Between, dispcalGUI has a homepage with further detailed informations.

Taxi @ rwx³

Sebastian Oliva presented today at the openSUSE conference the results of his ICC Device Profile Repository project to his mentoring organisation as participant of the Google Summer of Code 2011 program. As profile creation is expensive to most users, he emphasised the importance to easily share ICC profiles among these users. During the summer project, which is called now taxi, he developed a generic API to store and obtain ICC profiles through JSON requests.

X Color Management 0.3 DRAFT1

The net-color spec from the libXcm repository is renamed. During a great conversation with Martin Gräßlin from KWin at oSC rwx³ he pointed out, that the _NET_ prefix is reserved inside the Xorg atom name space. So I decided to rename all atoms in the applications known to me, which use the net-color spec. The renaming went on fast and all apps work again as expected in git.

The new spec is called X Color Management. It contains the color regions in Xorg description and the used device profile for Linux desktop colour servers.
Affected are libXcm, Xcm, Oyranos, CompICC and CinePaint.

CinePaint full screen

On my git repository with CinePaint patches is the full screen mode changed. It uses the GTK funtionality. That mode is now window manager based and will not work with twm. The old code is still ifdefed for users who rely on that and compile it. Putting a full fledged solution like in gqview inside CinePaint would have been too much work.

New is as well resizing of frames in the flip book, to allow me to load a bunch of cameraRAW images and watch them much like in a slide show. Colour regions are updated for the colour server.

And last but not least a fix in now available, to let CinePaint run on non KWin and non Compiz window managers. I tested on twm.

The package is patched in OBS.

Oyranos @ rwx³

rwx³ alias openSUSE Conference (oSC) will be from 11.09. - 14.09.2011 in Nuremberg / Germany. Im going the to openSUSE Conference! The conference for openSUSE - and Free Software enthusiasts, September in Nuremberg, Germany. Are you? We will meet there and can discuss ICC colour management for the openSUSE distribution, KDE and Qt.

Sebastian Oliva will be there too. I hope we can hack together a Oyranos connection to his newly created ICC DB. The ICC DB project, done during GSoC 2011, shall be used to search for ICC profiles by terms of colour device configurations. This means a printer can obtain a fitting ICC profile for a special driver without the need to have all the canned profiles packaged. Independent vendors can easily upload their ICC data and get their optimised profile selected automatically - if all works. Sebastian and Joseph Simon have done fair bits to get there in a clean way, without hacking the whole system.

I will have colour measurement equipment with me and we can create new profiles for actual used gear, like laptop monitors. Of course these shall walk into the ICC DB.

It would be cool to meet with Compiz and KWin maintainers/packagers to get the GPU colour managed desktop project further. The net-color-spec appears pretty robust. But we need a critical mass to support the concept of colour correcting all windows on the desktop in compositing window managers.

Will be great to meet you all.

Argyll V1.3.4 released

I’m pleased to announce the release of ArgyllCMS V1.3.4.
This update adds new
features as well as containing minor bug fixes. Amongst the changes are:

Support for the X-Rite i1 Display Pro and ColorMunki Display colorimeters (i1d3).
While these instruments seem to be quite good without specific display calibration,
they can benefit from it, so a new type of Argyll colorimeter calibration file has
been created (CCSS) to support the type of calibration these instruments use.
ccmxmake has been renamed to ccxxmake and support added for creating CCCS calibration
files using a spectrometer instrument. A new tool called i1d3ccss allows the calibration
files that come with the instruments to be translated into CCSS files and installed.
A CRT calibration is provided, something that is missing from the manufacturers set.
Non-1931 2 degree observer support has been added for these instruments, taking advantage
of the type of calibration they use.

The gamut finding code has been improved so as to deal more robustly with small
and odd shaped device gamuts.

The Spyder driver on Linux has been tweaked to work around device and Linux bugs,
and may work more reliably through USB hubs.

Printtarg has been modified to allow variable patch size for the DTP20.

An option has been added to colprof to allow setting the default profile rendering intent.

The full list of changes is in the log.txt file as usual.

Graeme Gill.