Watching org.libelektra with Qt

libelektra is a configuration library and tools set. It provides very many capabilities. Here I’d like to show how to observe data model changes from key/value manipulations outside of the actual application inside a user desktop. libelektra broadcasts changes as D-Bus messages. The Oyranos projects will use this method to sync the settings views of GUI’s, like qcmsevents, Synnefo and KDE’s KolorManager with libOyranos and it’s CLI tools in the next release.

Here a small example for connecting the org.libelektra interface over the QDBusConnection class with a class callback function:

Declare a callback function in your Qt class header:

public slots:
 void configChanged( QString msg );

Add the QtDBus API in your sources:

#include <QtDBus/QtDBus>

Wire the org.libelektra intereface to your callback in e.g. your Qt classes constructor:

if( QDBusConnection::sessionBus().connect( QString(), "/org/libelektra/configuration", "org.libelektra", QString(),
 this, SLOT( configChanged( QString ) )) )
 fprintf(stderr, "=================== Done connect\n" );

In your callback arrive the org.libelektra signals:

void Synnefo::configChanged( QString msg )
{
 fprintf( stdout, "config changed: %s\n", msg.toLocal8Bit().data() );
};

As the number of messages are not always known, it is useful to take the first message as a ping and update with a small timeout. Here a more practical code elaboration example:

// init a gate keeper in the class constructor:
acceptDBusUpdate = true;

void Synnefo::configChanged( QString msg )
{
  // allow the first message to ping
  if(acceptDBusUpdate == false) return;
  // block more messages
  acceptDBusUpdate = false;

  // update the view slightly later and avoid trouble
  QTimer::singleShot(250, this, SLOT( update() ));
};

void Synnefo::update()
{
  // clear the Oyranos settings cache (Oyranos CMS specific)
  oyGetPersistentStrings( NULL );

  // the data model reading from libelektra and GUI update
  // code ...

  // open the door for more messages to come
  acceptDBusUpdate = true;
}

The above code works for both Qt4 and Qt5.

Image Editing with 30-bit Monitors

Payable hardware for professionals is capable of 30-bit throughput since quite some years. And costs continue to go down. This means even budget setups are possible with this kind of gear. So lets follow the question why, who and how monitors capable of displaying 30-bit alias 10-bit per red, green and blue channel can be used. This blog article will first touch some basics, followed by technical aspects below.

Why is it useful to display graphics on a 30-bit monitor setup?
It is essential for graphical editing, to see what effect a editing step has. It is pretty common that low resolution monitors impose a barrier to reliably predict the intended output. This is true for geometrical resolution like for colour resolution and for gamut. The rule of thumb is, the graphics editor needs the most information available to do here/his job and spot artefacts and issues early in the process. This principle is deployed for print, web, film and video editing to reduce costs. You know, redoing something costs time and is part of the jobs calculation. More image information means as well more certainty to reach a graphical result. The typical artefact caused by low colour resolution is reduced tonal range. Colour conversions can reduce the tonal range further. So a sRGB image will look different on a 8-bit per channel monitor with a native gamma close to 2.2 compared to a pipeline with 10-bit per channel. The 8-bit output imposes a bottleneck resulting in loosing some tonal steps known as banding, which must not necessarily be present in the observed sRGB image. One very often read argument against higher bit depth is, that editing hardware shall be as close as possible to customers ones. But that is a illusion. The wide diversity of media and devices makes this nearly impossible. But simulation of end customer hardware is of course an issue and many graphics software has implemented simulation capabilities to address that concern.

Who is most interested in 30-bit colour editing on Linux?
Graphics professional and ambitious users closely observe Linux since many years and deploy it. Many block busters are produced and rendered on Linux machines. Web graphics is well supported since years and camera raw programs implemented a impressive level of features in the last years. So Linux is a potential content creation platform beside content consumption. The typical work flow for content creating people is to generate and edit their art work in high geometrical and colour resolution and down convert to lower resolutions as fits for the job, be that web, print, catalog previews and more flexible high quality delivery depending on actual contract. For instance many photographers archive their shootings in the cameras native format to preserve all available information for later editing or improved rendering. This is a important investment in the future for non short lived work, where old files can shine in new light. Motion picture productions are often rendered and color graded in floating point space by using the OpenEXR intermediate file format and output to 12 bits per component for playback in a cinema. Video production uses in parts raw workflows for advertisements. Medical-, scientific- and archival imaging are potentially interested too and require in parts 30-bit setups like in the DICOM standard. The benefit of 10-bit per channel versus 8-bit does not matter to everyone. Most consumers will not spot any difference while watching web video. But in more demanding areas it is a small but helpful improvement.

How to deploy 30-bit displays on Linux?
That feature was implemented by several companies starting on low level software components like X11, Cairo and pixman. However desktops where pretty slow to adapt software to the new needs and fix bugs. That was in part of the initially higher costs for the hardware. Only few developers in the open source community had early access to suitable gear. I do not write here about suitable graphic cards and monitor combinations. You should consult the web for this. Search for 30-bit monitor. Many early adopters observed psychedelic colours, not working graphics areas and more. Here the state on the well known KDE X11 desktop in release 4.11 on a openSUSE-13.1 . The login screen colours are broken. The splash screen after login looks correct and after some period the desktop becomes visible with again broken colours. To manually fix most of that, one have to tell that Qt shall use native colours. Create following text into a file called ~/.kde4/env/qtnative.sh .

$ kwrite ~/.kde4/env/qtnative.sh

#!/bin/sh
export QT_GRAPHICSSYSTEM=native

With the above variable the desktop should look reasonably in KWin, which is really great. Automating that in Qt would be appreciated.

However 30-bit monitors typical aim at high quality setups. Beside colour resolution they often enough offer a wider gamut than usual monitors. This results in partially heavily saturated colours, which burns in sensible eyes. Those people who do colour grading or photo editing are mostly affected, otherwise they can not easily play this work. So desktop colour correction is an other important feature to enable here. KWin supports ICC based colour correction through KolorManager, which would be useful for the colour saturation. But KWin disables all effects for 30-bit OpenGL visuals. The alternative Compiz-0.8 series has the CompIcc colour server plug-in, which provides the same ICC colour correction feature. To make use of it, one needs to install following packages: compizconfig-settings-manager, CompIcc-0.8.9. Unfortunedly the KDE decorator is no longer available. So use the Emerald decorator from X11:Compiz with the 30-bit-shadow.patch in order to avoid artefacts in the shadow code. Compiz can be used as a default window manager application. Use the system settings to switch to Compiz. Use ccsm to switch on Color Management if not done automatically. And voila the 30-bit desktop should be ready to explore.

What works and what not?
The Plasma desktop is fine including all menus. Dolphin, KWrite, and other applications work. Thunderbird shows some artefacts due to not properly supporting the R10G10B10A2 pixel format. The same is true for Firefox, which lets in parts shine through content behind the Firefox window. Gwenview and ShowFoto work fine within their 8-bit drawing. Only the preview is broken in ShowFoto. Krita supports with the OpenGL backend even native 10-bit per colour component. Menus in Sketch are black. Krita shows minimal artefacts through twice colour converting from image to sRGB by Krita and from sRGB to the monitor colour space by CompIcc. But this effect is much lesser visible than the improvements through its 30-bit support. Applications which try to code 24-bit colour themselves are broken like Konqueror. Gtk and hence Gnome applications with graphical areas do not work. They show black areas. VLC works fine. So daily work should be fine in with 30-bit in the KDE application family depending what you do, with some minor glitches. Valuable Gtk applications as is like Inkscape and most Gtk applications are unusable in a 30-bit setup, with Gimps drawing area being a exception. Thunderbird/Firefox are guessedly affected by the same Gtk bug for which a patch was created some time ago. A patched libgtk-2 is available for testing on openSUSE, which appears to have fixed the problem almost for me.

Beside the need to exchange a windowmanager, to patch a few components and do some manual settings, Linux appears almost there in support of 30-bit setups. Polishing of that feature needs testing of patches and finally acceptance for distribution. Your feedback about bugs, tests and patches can make a difference to developers.

Linux Color Management Hackfest Brno 2012 ended

Around a dozen people met inside the Red Hat offices in Brno last weekend. The attendees came from various distributions and projects to discuss and work on color management for Linux. Most people arrived at Thursday and we started immediately to brain storm ideas and share information.

I was quite shocked as I heard Dantii could not join us. Fortunately the last messages about him sound very encouraging. It is great that our community could in different ways help him and his family.

The basic concept we worked with during the hackfest, was the opt-out of colour management approach. That was visible in printing and in window manager colour management.

Printing people discussed the PDF/X OutputIntent. The concept was developed to overcome the current short commings in the cupsICCprofile, which is primarily a vendor solution for CUPS print servers and the colord user session hook inside CUPS server. The implementation of the concept was done inside libCmpx, which is basically a wrapper around Ghostscript, which does the majority of the work, and a interface to Oyranos. From the other side John Layt looked into that work and Krita new print colour management tab to understand the implications for the KDE/Qt print dialog. He discussed lively with Till Kamppeter, Richard Hughes, Chris Murphy and me on how to get forward with that. Richard wrote a proof of concept for on screen print simulation in GTK. Chris talked a lot about osX printing and did some testing there. His experience on other platforms than Linux helped us a lot to figure out, which path we want to go and way the make sense. I searched for some PDF’s showing the features we need. They can now be found on ColourWiki. Jaroslav Reznik printed them and Till tested them. Michael Vrhel from the Ghostscript project fixed already after the event all of the bugs, which Till worked on in Brno. We had the idea, that some PDF printers might be able to do the right thing with the OutputIntent themselves. While discussing on how to know about that capability, Till and Richard had a nice idea how to reduce code duplication inside the current set of Linux CUPS filters. In parts the Color Management Hackfest crossed over into a Printing Summit.

Jan Grulich started coding on KolorManager. He implemented a widget to show a 2D graph of a ICC profile inside the information tab. Sirko Kemter was not very happy about the colours inside the graph. So I adjusted them, but after the hackfest.

While working on that, Jan profiled his monitor using Dantii’s colord-kde. Yes, Lukáš Tinkl fixed it, so it can now create ICC profiles. We needed to hand massage the profile to get it into Taxi DB, and then thought, it would be good to download the fresh profile later through Oyranos. That worked fine on the command line. But inside KolorManager a selection that a profile is available for download from Taxi DB would be more appealing. Jan wanted to look into that, and I worked later on a API and code snippet for Oyranos.

By the way, the above screen shot is done using the new colour correction feature for KDE-4.10. You might see the strong colour cast in it. Dan Vrátil worked on undoing that cast inside KSnapshot using the actual monitor profile. The initial coding was fast. But he likes to get that working for multiple outputs too.

Casian Andrei, who did the KWin Color Correction project during this years GSoC, wrote some documentation about that newly added feature. While writing that and clarifying some points, we discussed the opt-out inside KWin and found that it is not yet present. Sig. But Casian had played with the idea already and said that per region opt-out would be trivial inside KWin and started to write on that feature during Sunday. In case that works out, it would be trivial to opt out inside existing applications. But we found as well, that for a perfect results only a blending in the correct colour space is needed during compositing. That can be implemented inside toolkits, which is not trivial, and can then be used together with the per window opt-out. That buys us some valuable time for the toolkits to become ready for full colour management support. Whether the same per region approach is easy enough to implement in Wayland needs to be seen.

Now back to profile distribution. Oyranos obtained a new backup tool for Taxi DB. And we counted already over 200 different ICC profiles in the online data base. Sirko Kemter and Daniel Jahre grabbed the taxi sources, installed MongoDB and worked on mostly basic stuff to add later more features. The online front end to the DB can be used on every platform for download and upload. Daniel and Sirko discussed how to temporarily store a ICC profile from the ColorHug LiveCD. We found that the data base can be used for very different things, e.g. distribution of spectral data sets for camera sensors.

Pippin worked since some time on improving the display of gradients on 8-bit driven monitors. He came up with a dithering approach and tested that using the Taxi DB profiles for the analysis of his implementation. That helped him to make the algorithm more robust even with strongly distorted monitor gamma curves. There is quite some interest inside the graphics designers community for his work to solve banding problems. We talked a bit about gegl and I found babl especially interesting. The small library does, what I call pixel layout conversions. Those are in part colour space conversions and encoding conversions. For better separating encoding, 8-bit versus 16-bit etc., from colour spaces, e.g. linear gamma + Rec709 primaries versus sRGB etc., we need better ICC profile analysis. Especially gamma analysis can be improved inside rendering pipelines. Beside discussing hardware stuff, spectral imaging, video processing and so on, he gave a small and very helpful introduction into colour theory, which was a eye opener for many of the colour management newcomers and thus very welcome.

We had a interesting discussion about financial implications of colour measurement hardware. My impression is the high costs and thus reduced availability for good colour measurement gear nags on the success and acceptance of ICC colour management in consumer and professional markets.

During the hackfest it was really a pleasure to have so many experts in the field in one room and work together in a highly productive atmosphere. Some of them I met the first time. So let me thank our sponsors Google, Red Hat and KDE for their generous support of the hackfest idea.

We all worked quite a lot and found such a event should not stay single. So we agreed already to arrange a one day track and one following day at LGM in Madrid / Spain 2013 under the OpenICC umbrella. I hope we will see then even more projects coming.

Linux Color Management Hackfest 2012

The hackfest in Brno is approaching fast. I wrote earlier this year about the idea. It will happen from 9th until 12th November 2012 inside the Red Hat Czech office. Talks with our local organiser and various sponsors went good so far. People will code in Brno on various topics around color management in Linux.

The main focus looks like to be at applications, desktop and library integration. For the printing system and Taxi DB was good interest too. As the event is organised as a framework for attendees, each one will decide, what is best to do. After a morning meeting, where we can coordinate, we will likely split in smaller groups according to a choosen topic or move around as needed. We hope that works for all attendees. There are specialists present for many Linux color topics for discussion and of course color management newbies can ask them to effectively improve their project.

OpenICC Google Summer of Code 2012 results

Google Summer of Code

Participation of the OpenICC group in the Google Summer of Code 2012 program was this year a great success. All projects reached their respective goals. Here a small summary:

Colour Management for Krita Printing
Joseph Simon worked on adaption and integration of his last years implementation for colour managed printing into Krita/Linux. The workflow is based on ICC profile injection into PDF through the means of a OutputIntent.

KWin Colour Correction
Casian Andrei’s KWin changes for ICC style colour correction in the GPU are reviewed upstream and his new code to the KolorManager code base waits just for approval. The concept follows the X Color Management spec. In contrast to the elder CompICC implementation is the KWin result highly modular and thus very flexible.

Simple Toolkit Abstraction
Nitin Chadas SimpleUI project for rendering a subset of XForms was written from
ground up and provides now backends for FLTK, Gtk and Qt. It needs a bit
of polishing to become useable.

Thanks to Google for providing the colour management and graphics community again a great chance to code and learn the open source way.

PDFassociation conference in Basel/Switzerland

PDF Association - The future of PDF

The PDF Technical Conference was organised by the independent PDFassociation and held this week March 27-28, 2012 in Basel, Switzerland in the rooms of Adobe. PDF experts from around the world meet there to talk about hot new stuff and to discuss technical details in a friendly atmosphere. In the following text I will highlight some of the talks.

Andreas Kraushaar from Fogra gave a talk about spectral imaging and why it is a good thing to support that inside PDF spot colours for packaging. Most of the time the number and kind of inks used for packaging is rapidly changing. So colour profiling the ICC way means a waste of too many time and material as it is not flexible enough for that. Embedded spectral data based recipes for rendering spot colours in supporting applications can improve speed and handling considerably.

Two talks elaborated on transparency in PDF. I found it amusing that the concept of a blending colour space is as well in the PDF community still a hot topic. But of course rendering to offscreen bitmaps instead of traditionally one final output buffer is quite different and developers agreed to find implementation sometimes not easy. And yes, there are many PDF viewers around, which blend Cmyk and RgbA together in one go ignoring any blending space requirement.

Florian Süßl from zipcon presented the well known ECI Altona Test Suite 2 now covering PDF/X-4 including transparencies. He elaborated on the work involved on how to create all the tests following the PDF-1.7 spec. He gave some examples where the various PDF renderers failed certain test. This is again a very valuable tool for developers of PDF software. It would be cool if such a test suite becomes part of the specification itself as is usual with other standards to help verifying implementations.

Following the topic of quality testing, David van Driessche explained the GWG test suite. I would find it really cool to embed GWG tests into a dedicated PDF document page for checking reproduction capabilities by layout applications like Scribus. That feature would be helpful for critical viewing and to easily test proofing software.

Bill McCoy from idpf presented a comparison of PDF and the HTML5/ePUB publication format. The later one is based on W3C standards like SVG and TTS with few modifications and has together with these a great potential depending on the involved people and organisations to further develop these technologies. ePUB was created to provide a independent platform for electronic publishers. And as the ePUB format comes out of open source technologies, it can as well be supported in the open source world by e.g. Mozilla and
Calibre logo.png.

The conference was beside sometimes very full rooms a well organised and interesting event.

KDE End to End Colour Management

The last blog posts about KDE and colour management might have been irritating about what actual happens on colour managed desktops. Here come some clarifications and thoughts from the Oyranos CMS maintainer. The project name starts with Oy (Oyranos like sky), hence my nick oy on IRC ;-)

Colour Management Systems (CMS) are a precondition to do colour correction of input and output devices. But this is not sufficient for having a colour corrected desktop. The claim was made, that Gnome is the first colour managed desktop on Linux. But Gnomes window manger mutter has no means to use ICC profiles. The same is true for all other window managers with an exception of old Compiz. A CMS selects only the needed ICC profile and does the configuration in that field. But the background, applications like the dock and most others are not colour corrected by standard ICC profiles mechanisms in Linux. The only thing users can do since many years on Linux is to do monitor calibration setup per single channel. This helps for better grayscale, but not for compensating of colour gamuts. Calibration is only a first step, but not sufficient for ICC colour correction. So Gnome users have today no colour corrected desktop like all other Linux users.

What is needed to get to a End to End colour corrected desktop in KDE? A more general Overview can be found here.

  1. KWin needs ICC support, in order to colour correct the KDE desktop in a reasonable time frame. That will help with the output side in a fast way by using the GPU during compositing while using few resources. If you feel it is time to do something, here is a  Google Summer of Code CM project idea for KWin. With my experience from the CompICC project, I would be glad to help any such project.
  2. An other project I would find really helpful is to provide colour correction to KDE’s primary image viewer gwenview. If people could help with a hackfest, that would be cool. We have such thing in mind and some ideas about, maybe you like to join us.
  3. Qt/KDE needs to explore how to do own fast colour correction of a complete window to be prepared for the future. Here are two project ideas.
  4. OpenICC did investigate to get print colour management right. There are currently two approaches who are promising. OpenICC has one project idea to introduce colour managed printing into Krita and one for user profile setup for colour managed print queues with KolorManager. These are two complementing, maintainable and robust paths for getting printing CM right.

Now some clarifications about Oyranos itself, as in the kde-planet where many wrong statements transported intermixed with half true claims.

  • Core is a toolkit independent library
  • KDE, Qt and FLTK front ends exist like KolorManager. Other native ones are possible.
  • The Elektra API and library is used for format independent configuration DB access.
  • Oyranos is planed to switch to a OpenICC JSON DB format to converge with ArgyllCMS and other interested CMS’es
  • Oyranos is a cross platform project
  • A DBus API would be welcome on top of the basic library but not in its core
  • Oyranos forces no one to use the CPU or prohibit to use the GPU :-D
  • The CMS provides means to do optional multi monitor colour correction and other conversions.
  • CompICC uses Oyranos and does colour correction on the GPU
  • Oyranos developers belief in collaboration :-)
  • Self containment in Oyranos results from adhering to and work on interoperable standards.
  • User configurations belong to users in Oyranos, so it needs no special root rights, which exposes security and privacy risks.
  • Oyranos provides optional policies for grouping single settings. That is a additional feature not a limitation.
  • Oyranos uses many advanced automatism’s to do it’s work successful
  • The CMS is designed to work with default settings.
  • Advanced manual configurations are supported and part of Oyranos’ user centrism.
  • Oyranos cares about quality and requires a careful selected and peer reviewed profile set that comes with no Fakes and no wrong colorimetry.
  • Licensing fits most open source and commercial projects with a newBSD style license.

Choice is a good thing for users. As a CMS author I have no problems, that an other CMS comes to KDE too on Linux. Many Linux CM standards I initiated or helped with allow for such interoperability, which is in the spirit of the ICC standard.

Oyranos @ rwx³

rwx³ alias openSUSE Conference (oSC) will be from 11.09. – 14.09.2011 in Nuremberg / Germany. Im going the to openSUSE Conference! The conference for openSUSE - and Free Software enthusiasts, September in Nuremberg, Germany. Are you? We will meet there and can discuss ICC colour management for the openSUSE distribution, KDE and Qt.

Sebastian Oliva will be there too. I hope we can hack together a Oyranos connection to his newly created ICC DB. The ICC DB project, done during GSoC 2011, shall be used to search for ICC profiles by terms of colour device configurations. This means a printer can obtain a fitting ICC profile for a special driver without the need to have all the canned profiles packaged. Independent vendors can easily upload their ICC data and get their optimised profile selected automatically – if all works. Sebastian and Joseph Simon have done fair bits to get there in a clean way, without hacking the whole system.

I will have colour measurement equipment with me and we can create new profiles for actual used gear, like laptop monitors. Of course these shall walk into the ICC DB.

It would be cool to meet with Compiz and KWin maintainers/packagers to get the GPU colour managed desktop project further. The net-color-spec appears pretty robust. But we need a critical mass to support the concept of colour correcting all windows on the desktop in compositing window managers.

Will be great to meet you all.

Cross Platform Toolkits and CM

Some readers might wonder, why ICC Examin has not yet a new osX package. There are several issues with FLTK. The most annoying is FLTK has no real colour management policy. But stop. Which cross platform toolkit has? Right – none. Qt has non, Gtk – don’t know? The situation dates from times where RGB was just native display values. But that is changing completely. osX has with SL colour server functionality on top of Quartz integrated. On Linux early prototypes date back to 2008 and start to be integrated as CompIcc project.

Application developers be warned. FLTK and Qt defaults to “Generic RGB” on osX SL. This means no sRGB primaries, alias a different Gamut, and a Gamma of 1.8.

Do not assume sRGB on all platforms. And worse, a application can not assume anything about the colour space of the toolkit. Trolltech and FLTK, and Gnomes Gtk too, can change the underlaying colour space at good will. So colour managed applications using these cross platform toolkits are merciless exposed to this situation.