IS&T's 2002 PICS Confeence Colo Encodings: srgb and Beyond Kevin E. Spaulding Eastman Kodak Company, Rocheste, New Yok Jack Holm Hewlett-Packad Company, Palo Alto, Califonia Abstact Most digital imaging poducts aimed at the open desktop maket ae designed to poduce and accept image data in the srgb colo encoding. srgb is defined with espect to the esponse of a efeence CRT display. As a esult, the image data can be easily intepeted, and can be diectly displayed on a typical CRT without the need fo additional colo tansfomations. Thus, srgb simplifies the wokflow fo softcopy-based viewing, editing, and image shaing. One intent of srgb is to standadize the way in which images ae stoed and communicated in consume digital imaging systems, theeby impoving the inteopeability of these systems. Howeve, fo non-crt-centic applications, limitations associated with cuent srgb-based wokflows can negatively impact pocess complexity and image quality. This pape will discuss the pos and cons of seveal diffeent appoaches that have been poposed to ovecome these limitations. Colo Gamut Consideations srgb Colo Gamut Issues One issue that is impotant in many digital imaging systems is the ability to fully and optimally utilize the colo gamut of the output media. Because the srgb colo encoding is specified elative to the esponse of a standad CRT, 1 the colos that can be epesented ae limited to those within the colo gamut of this standad display. As a esult, stoing images in srgb can limit the capability of an imaging system to accuately epoduce colos outside the srgb gamut. Film, scannes and digital cameas can both captue colos well beyond the srgb colo gamut. Theefoe, it is necessay in srgb-centic wokflows to colo ende the captued image data into the srgb gamut. Likewise, many output devices have colo gamut shapes diffeent fom that of srgb, with the device colo gamut extending beyond the srgb colo gamut in some egions of colo space, and the srgb gamut extending beyond the device gamut in othe egions. Fo example, as illustated by the gamut slices in Fig. 1, photogaphic pintes employed in digital photofinishing typically have a lage gamut fo dak colos. And a smalle gamut fo light colos. As a esult, when pinting srgb images it is necessay to gamut map fom the srgb gamut to the actual output device gamut. If the gamut mapping out of srgb is designed to be complementay to the colo endeing into srgb, this pactice can poduce acceptable esults. Howeve, in many applications, the input and output pocessing will be independent, and theefoe inconsistencies between popietay colo endeing and gamut mapping (o even gamut clipping) algoithms can often poduce less than optimal esults. Even in closed systems, whee both the input and output pocessing can be coodinated, this co-optimization of the colo endeing and gamut mapping can intoduce a significant amount of additional computation and complication. These gamut estictions ae even moe significant when colo enhancements ae applied to boost the colofulness of an image, o whee it is desied to accuately specify special colos (e.g., PANTONE colos) that ae outside the srgb colo gamut. L* 100 80 60 40 20 0 hue = 320º hue = 140º 100 50 0 50 100 C*ab Figue 1. Compaison of CIELAB hue leaf slices though srgb gamut (solid) and typical photogaphic pint gamut (dashed). Magenta is to the left, and geen is to the ight. (The black-point luminance fo the srgb gamut has been adjusted to match that of the pint media, as is common pio to gamut mapping.) PANTONE is a tademak of Pantone Inc. 167
IS&T's 2002 PICS Confeence Extended Colo-Gamut Though YC C b Colo Encoding Most cuent imaging devices and systems that poduce JPEG files nominally encode colo data accoding to the srgb colo encoding. Howeve, as pat of the JPEG compession pocess, these srgb images ae conveted to a colo encoding pio to the actual discete cosine tansfom (DCT) compession step to obtain a numbe of compession benefits. This is accomplished by simply applying a matix tansfomation to the nonlinea RGB values to detemine coesponding luminance-chominance values. (The paticula colo encoding that esults fom applying the standad JPEG colo tansfomation matix to srgb colo values is in the pocess of being standadized unde the name(s) srgb YCC o sycc. 2,3,4 ) This colo space has a substantially lage colo gamut than the srgb colo space. Theefoe, satuated colos that lie outside the srgb gamut can be stoed in a standad JPEG file by diectly encoding the image data as data, athe than fist clipping the image to the srgb gamut. One of the desiable featues of this appoach is that JPEG files containing extended-gamut data will be backwad compatible with existing JPEG file eades. Applications that know what to do with the extended-gamut image data can use it, while conventional file eades will simply clip off the extended-gamut infomation when the file is opened and use the srgb image. The applications that etain the extended-gamut image data can enable output devices, such as photogaphic pintes o inkjet pintes, to make the best use of thei colo epoduction capabilities. In addition, the need fo somewhat abitay gamut expansion on output is avoided. The idea of using extended-gamut image data is not new, although the full gamut has not commonly been used with JPEG files. Howeve, colo encodings have been used and standadized fo many applications, and most JPEG compession softwae and hadwae has suppoted the capability of dealing diectly with YC C b image data fo many yeas. As a esult, thee should be no baie to any digital camea/scanne ceating and stoing images. No should thee be a baie fo any pinting system to make use of the extended-gamut image data, egadless of the souce. Othe Extended-Gamut Colo Encodings Recently, seveal othe lage gamut colo encodings 3 have been poposed. One such colo encoding is e-srgb. This colo encoding is an extended vesion of srgb that allows the encoding of negative RGB values, as well as values lage than the srgb white point. To suppot the extended ange, the e-srgb colo encoding equies a minimum of 10-bits/pixel/colo in digital pecision. One desiable featue of this colo encoding is that srgb values coespond exactly to associated e-srgb values, and can be computed using a simple 1-D LUT tansfomation. This featue allows srgb images to be conveted to e-srgb and back without loss. Anothe extended-gamut colo encoding is ROMM 5 RGB. This colo encoding achieves an extended colo gamut by using a set of lage gamut colo pimaies chosen to encompass the gamut of eal wold suface colos and to have a numbe of othe desiable popeties. 6 ROMM RGB is well-suited fo both the stoage of lage-gamut digital images, as well as the application of common image manipulations such as tone scale modifications, colobalance adjustments, shapening, etc. One desiable featue of ROMM RGB is that it can be used in wokflows that ae limited to 8-bits/pixel/colo. The computation of an srgb peview image fom a ROMM RGB image is only slightly moe complicated than fo e-srgb, equiing a LUTmatix-LUT tansfomation. (Since thee is not a one-to-one coespondence between srgb code values and ROMM RGB code values, thee may be a small loss of pecision when going fom srgb to ROMM RGB and back, depending on the bit-depth.) Since both the e-srgb and ROMM RGB colo encodings equie that colo tansfomations be applied in ode to poduce a video peview image, they ae incompatible with many existing open system wokflows. Howeve, the ecently appoved JPEG2000 image file fomat standad seamlessly enables the use of these extended 2,7 gamut colo encodings though the use of ICC pofiles. (The suppoted types of ICC pofiles ae esticted to the Monochome Input and Thee-Component Matix-Based Input pofile classes.) Until such time that suppot fo the JPEG2000 file fomat is widely implemented, it is anticipated that the use of e-srgb and ROMM RGB will lagely be limited to closed systems and highe-end colo-managed applications. This implies that while these colo encodings may be useful fo the stoage and manipulation of the digital images within a paticula system, they will pobably not be used fo open image intechange in the shot tem. All of the extended-gamut colo encodings that have been mentioned thus fa ae output-efeed in that they ae intended to be epesentations of the colo of a endeed output image. In some applications, scene-efeed colo encodings have been found to offe a numbe of desiable featues. Scene-efeed colo encodings, such as the 6,8 ecently defined RIMM RGB, ae epesentations of the colo of an oiginal scene, and ae geneally designed to etain the full-dynamic ange of the oiginal scene captue. As a esult, they offe the geatest flexibility fo editing the image and specifying a pefeed endeing of the scene. The use of scene-efeed colo encodings is only pactical in open systems in conjunction with a file fomat like JPEG 2000, which suppots the specification of a default endeing tansfom that can be used to fom a coesponding output-efeed image. This default endeing tansfom can be used by devices and applications that ae not designed to wok with images in a native scene-efeed image state. Image Consistency Consideations One of the intentions in the standadization of srgb was to impove the inteopeability of digital images though the use of a common colo image encoding. In pactice, while thee has been elatively widespead acceptance of srgb as 168
IS&T's 2002 PICS Confeence a common exchange metic in consume digital imaging applications, srgb images poduced by diffeent souces have been obseved to vay significantly in thei chaacteistics (e.g., oveall bightness level, contast, etc.). While some of these diffeences esult fom intentional vaiations in the colo endeing aims used fo diffeent poducts, a significant potion of the vaiability can be attibuted to the fact that the srgb standad is somewhat ambiguous in seveal espects, and does not povide much useful guidance on how to colo ende an oiginal scene into srgb. This has left a lot of oom fo diffeent companies to define thei own intepetations of the srgb standad. One of the moe significant souces of vaiation can be taced to ambiguity concening the colo endeing state of an srgb image. The srgb standad 1 states that "the encoding tansfomations between CIE 1931 XYZ values and 8-bit RGB values povide unambiguous methods fo epesenting optimum image coloimety when viewed on the efeence display in the efeence viewing conditions by the efeence obseve." This sentence implies that the srgb image data is encoded in an output-efeed image state that shall be intepeted as being optimally colo endeed fo the srgb efeence display. A somewhat diffeent citeia is pesented in Annex B of the standad, which poposes that this desied appeaance can be poduced by simply captuing a scene accoding to ITU-R BT.709. 9 Futhemoe, Pat 9 of the IEC 61966 seies of standads 10 (of which the srgb standad is Pat 2) povides a method fo chaacteizing a digital camea to poduce scene coloimety adjusted to a D 65 white point, and Annex C ecommends that this scene-efeed coloimety be consideed srgb. These thee conflicting options diectly impact the ceation and intepetation of srgb image data. The coloimety of an oiginal scene is typically quite diffeent than the coloimety of a desiable output-efeed image ceated 6 fom that scene. As a esult, given the ambiguity in the image state of an srgb image, it is not supising that thee is substantial vaiation among implementations. The most visible manifestation of this vaiability will typically be diffeences in the oveall image contast since a boost in the luminance and chominance contast is usually an impotant featue of a well-designed colo endeing function. srgb images that ae ceated by encoding scene-efeed image coloimety will geneally be lowe in contast than those that ae ceated fom tue output-efeed coloimety. Anothe significant souce of vaiability is diffeing intepetations of the adapted white luminance that should be assumed when viewing srgb images. Some vendos have intepeted srgb to be a pint-like epesentation whee a white with a code-value of 255 coesponds to a vitual white piece of pape. This is consistent with the paadigm that the compute s monito is a vitual desktop, and that a document window open on that desktop is analogous to a physical piece of pape on a physical desktop. In this scenaio, pints made of a document that includes an srgb image would typically map the white with a code-value of 255 to the output media white, and would map neutals with code values less than 255 to popotionally dake neutals on the pint. This intepetation is also consistent with that assumed by most commonly available 11 srgb ICC pofiles. Othe vendos have intepeted the srgb white point to coespond to an ideal pefect white diffuse with a eflectance of 100%. In this case, it is assumed that the viewe will intepet a white with a code value of 255 as being somewhat bighte than a piece of pape. Still othe vendos have intepeted srgb to be a television-like video epesentation, whee the bightest colos ae eseved to ceate the appeaance of white-than-white specula highlights in the image. Images ceated using the pefect white diffuse paadigm will tend to be dake than those ceated using the pint-like paadigm, and images ceated using the television-like paadigm will tend to be dake still. The basic diffeence between these paadigms boils down to a diffeence in the assumption made about the adaptive white point of the obseve who is viewing the image. This, in tun, eflects diffeing assumptions made egading the typical wokflow and image pesentation. Fo example, if an srgb image wee to be pasted into a wod pocessing document whee the colo of the page was a white with a code value of 255, then most viewes would intepet the bightness of the image elative to a code value of 255 being a typical piece of white pape. On the othe hand, if the image wee pesented such that it filled the entie sceen and all visual bightness cues such as title bas wee eliminated, the viewe would lagely adapt to the bightness level of the image. In this case, images ceated using the television-like paadigm would have the same capacity to ceate the appeaance of specula highlights as they do on a conventional television. Appopiately contolling the image pesentation can contol the adapted white point of the obseve almost abitaily. The specification of the image backgound and suound in the srgb standad is not sufficient to clealy specify the adaptive white point luminance. This fact, combined with the ambiguity of the image state, has led diffeent vendos to intepet it accoding to thei view of an expected wokflow. Anothe souce of vaiability is the specification of an 2 unealistic value fo the srgb veiling glae. The 0.2 cd/m value specified in the standad is atypical fo eal CRT displays in the srgb viewing envionment, and CRT intenal flae is not accounted fo at all. The assumed viewe obseved black point will affect the appopiate colo endeing of an image, paticulaly in the shadows. By specifying an unealistically low black point, implementes must choose whethe to assume the specified value consistent with the standad, o a moe ealistic value that is consistent with wokflows whee the image data is sent without modification to eal CRT displays. Additionally, thee appeas to be a significant amount of vaiation in the way diffeent vendos use and account fo the efeence viewing envionment and display chaacteistics specified in the srgb standad. Consequently, when ceating a pint fom an srgb image, some output paths adjust the image chaacteistics to account fo diffeences in factos such as the oveall luminance level, the 169
IS&T's 2002 PICS Confeence viewing flae level and the image suound, while othes neglect these diffeences. Fo example, aside fom a chomatic adaptation step, the standad srgb ICC pofile, which is used in many colo-managed pinting paths, essentially neglects the diffeences between the ICC PCS efeence medium/viewing envionment and that associated with 11 srgb. To futhe complicate mattes, it appeas that some vendos may have ignoed the srgb specification altogethe, and have labeled thei images as srgb even though they may have been endeed to a diffeent RGB colo encoding. Fo example, some vendos have histoically ceated video RGB images fo CRTs with a diffeent gamma, white-point and/o RGB pimaies than those specified fo srgb. In some cases, these vendos have appaently labeled thei images as srgb even though they ae fundamentally inconsistent with the specification. In a digital imaging wokflow, any mismatch between the srgb intepetation assumed in the captue pocess and the pinting pocess can esult in sub-optimal esults fo the final image. Fo example, if the image state, adaptive white point, viewe obseved black point, o the gamma value assumed by the image souce is diffeent than that assumed by the output device, then the bightness and/o contast of the pinted image may be too low o too high. As a esult, consumes may need to edit thei images o manipulate pinte dive settings in ode to optimize esults fo a given pint path and to achieve consistent esults acoss a vaiety of devices. These same consistency poblems will cay ove into commecial digital photofinishing applications, and will apply to the extension of srgb as well. Thee ae a numbe of potential stategies that could be used to addess these issues when making pints of srgb images. Assume Nominal srgb Intepetation The pinting system can assume some nominal intepetation of srgb. The intepetation assumed in any paticula system may be designed to be consistent with a paticula input souce, o may be a compomise between some of the common input souces. This appoach may be able to poduce esults that ae acceptable fo many input souces, but will not be optimal fo any souce that diffes significantly fom the assumed intepetation. The majoity of cuent systems utilize this stategy. Utilize Souce-Dependent Pocessing Most othe appoaches will incease system complexity. One altenative is to adjust the pint path to account fo the souce-dependent image diffeences. Fo example, if all images fom a cetain vendo ae known to pint too dak, then images fom that vendo can be lightened befoe they ae pinted. The souce-dependent image diffeences can be chaacteized by the individual pinting systems. Altenatively, the diffeent image souce vendos can be asked to povide guidance about image adjustments that ae necessay to poduce optimal esults fo a specific pint path. This last concept is the basic appoach used in Epson s PRINT 12 Image Matching (PIM) technology whee digital camea vendos ae asked to stoe metadata in the JPEG image file indicating how PIM-enabled pint paths should alte the tone and colo chaacteistics of the image so that it is optimally endeed fo PIM-enabled output devices. The PIM metadata paametes include adjustments fo image attibutes such as bightness, contast, colo satua-tion, and colo balance that can be used to account fo diffeences in that camea s implementation of the srgb standad. While this appoach can educe the vaiability encounteed with a specific output path, it can be seen that soucedependent pocessing will actually incease the global inteopeability poblem, and fundamentally beaks the whatyou-see-is-what-you-get (WYSIWYG) paadigm that consumes have come to expect. Fo example, an image fom one camea vendo may look dake than an image fom anothe camea vendo when viewed on the use s CRT display, but when they ae pinted on a PIM-enabled pinte they might have a simila density level. Then, if these same images wee then sent to an Intenet pint fulfillment sevice, they might come out dake and lighte again. Consequently, the esults that ae obtained will not only be inconsistent between diffeent input devices, but now they will also be inconsistent between diffeent output devices. Related to this poblem is the fact that any image editing that a use pefoms on an image may be confounded with the image adjustments applied as pat of the soucedependent pocessing. Fo example, if a consume sees that an image fom a paticula camea is a little dak, he/she may edit the image to lighten it accodingly. Depending on whethe o not the image editing application peseves the metadata, the edited image may get lightened again when the image is pinted. Convesely, if the consume uses the image with an application that does not peseve the metadata, this would effectively disable the metadata-diven pocessing. Fo example, if the use wee to paste the image into a document, o even add text and save the image to a new file using an application that doesn t etain the image metadata, then the desied esults would not be obtained even with a pint path that is enabled to pefom soucedependent pocessing. Fundamentally, this appoach does not addess the oot cause of the inconsistency in pints made fom digital image files. Rathe, it actually legitimizes these diffeences by sending the message to camea vendos you can leave you images just the way they ae and we will fix them late. To pomote inteopeability of digital images, it is citical to ensue that metadata specifying modifications to image appeaance only be used in situations whee it can be guaanteed that it will be popely intepeted by all applications and output paths. Even if thee wee industy-wide ageement that the use of this type of metadata wee appopiate and should be standadized, it would be impactical to evise existing file fomat specifications such as Exif to equie the use of this metadata because thee is such a lage installed base of applications and pint paths that it would not be possible to update them all in a timely manne. The ecently appoved JPEG2000 file fomat standad does 170
IS&T's 2002 PICS Confeence povide a mechanism fo suppoting this featue though the 2 equied suppot fo esticted ICC pofiles. Utilize Image-Dependent Pocessing Anothe appoach that can be used to deal with the souce-dependent image vaiability is to individually adjust images as they come though the pinting system. This solution allows fo the coection of image-dependent poblems such as exposue eos, as well as the soucedependent vaiations associated with the diffeent intepetations of srgb. A bute foce implementation of this stategy would be to have an opeato manually adjust each image using a calibated softcopy display. Howeve, this would be time consuming and costly in any sot of highvolume wokflow. Altenatively, automatic image pocessing algoithms can be used to estimate and compensate fo the chaacteistics of individual images. The isk of this appoach is that automatic algoithms can sometimes be fooled by the image content. Fo example, is an image dak because the exposue was incoect, o because it is a pictue of a black cat in a coal bin? Reduce Vaiability In srgb Intepetation The ideal solution to the souce-dependent vaiability poblem is to minimize it by developing an industy consensus egading the intepetation of srgb/sycc (o any othe colo encoding metic that may become popula in the futue). While a detailed ecommendation fo a common intepetation is beyond the scope of this pape, seveal issues that should be addessed would include ageement on the intepetation of the srgb/sycc ambiguities mentioned peviously, and the documentation of a default/efeence means fo mapping an oiginal scene onto the efeence display. (It is ecognized that diffeent camea vendos will want to implement thei own pefeed endeing aims, and theefoe it would be inappopiate to foce a single look onto all cameas. Howeve, the definition of a efeence colo endeing function should educe the unintended vaiability by poviding a common baseline position.) Conclusions This pape has descibed two impotant issues associated with the use of consume srgb images in digital imaging wokflows. The fist issue is elated to the colo gamut of srgb, and the esulting implications fo accuately epoducing colos outside of the srgb gamut. One desiable solution to this poblem, which etains an extended colo gamut while peseving backwad compatibility and inteopeability, is the diect use of srgb YCC image data in JPEG image files. The second issue that was discussed elated to inconsistencies in the intepetation and implementation of the srgb specification. While souce-dependent pocessing can be used to addess this inconsistency, this appoach can actually esult in moe inteopeability poblems athe than less. A pefeed solution is to wok towad a common intepetation of srgb thoughout the digital imaging industy. Refeences 1. Multimedia Systems and Equipment Colou Measuement and Management Pat 2-1: Colou Management Default RGB Colou Space srgb, IEC 61966-2-1 (1999). 2. Infomation Technology JPEG 2000 Image Coding System Coe Coding System, Amendment 2, Inclusion of additional colo-space, ISO/IEC JTC 1/SC 29/WG1 N2223R, 10 July 2001. 3. Photogaphy Electonic still pictue imaging Extended srgb colo encoding e-srgb, PIMA 7667:2001. 4. Committee Daft fo Vote of Amendment 1 to Multimedia Systems and Equipment - Colou Measuement and Management Pat 2-1: Colou Management Default RGB Colou Space srgb, IEC 61966-2-1/A1/ Ed.1, 2001. 5. Photogaphy Electonic still pictue imaging Refeence Output Medium Metic RGB Colo encoding: ROMM-RGB, PIMA 7666:2001. 6. K. E. Spaulding, G. J. Woolfe and E. J. Giogianni, Optimized Extended Gamut Colo Encodings fo Scene- Refeed and Output-Refeed Image States, J. Imaging Sci. Technol. 45, 418-426 (2001). 7. C. Chistopoulos, et al., The JPEG2000 Still Image Coding System: An Oveview, IEEE Tans. Consume Electonics 46, 1103-1128 (2000). 8. Photogaphy Electonic still pictue imaging Refeence Input Medium Metic RGB Colo encoding: RIMM-RGB, PIMA 7466, Daft Vesion 1.0 (2001). 9. Basic paamete values fo the HDTV standad fo the studio and fo intenational pogamme exchange, Recommendation ITU-R BT.709 (fomely CCIR Recommendation 709). 10. Multimedia Systems and Equipment Colou Measuement and Management Pat 9: Digital Cameas, IEC 61966-9 (2000). 11. M. Nielsen and M. Stokes, The ceation of the srgb ICC pofile, in Sixth Colo Imaging Confeence: Colo Science, Systems and Applications, 253-257 (1998). 12. Bette pints fom digital still cameas with PRINT Image Matching, white pape available at http://www. pintimagematching.com (2001). Biogaphy Kevin Spaulding eceived a BS in Imaging Science fom Rocheste Institute of Technology in 1983, and MS and Ph.D. degees in Optical Engineeing fom the Univesity of Rocheste in 1988 and 1992, espectively. He has been with Eastman Kodak Company since 1983 whee he is cuently a Reseach Associate in the Imaging Science Division. He is also Technical Secetay fo the CIE TC8-05 committee, which is tasked with defining standads fo the unambiguous communication of colo infomation in images. His eseach inteests include digital colo encoding, colo epoduction, digital halftoning, image quality metics, and image pocessing algoithms fo digital camea and pintes. 171