Fucking colorspaces. How do they work?

Dear Lazyweb,

On one machine (Mac), "convert foo.png ... -colorspace RGB foo.jpg" works, but on another machine (Linux) that gives me an all-black image. Conversely, on the Mac, -colorspace sRGB gives me an all-black image, and on the Linux machine that works.

As far as I can tell, both versions of ImageMagick were built the same way. Both machines are 64 bit. Both have lcms2.

WTF?

All I want is a portable incantation for my "convert" command-lines that says "don't ever give me CMYK JPEGs, because most browsers display those wrong." If there are other subtleties between "RGB" and "sRGB", I'm sure I don't care about them.

What's the right chicken to wave at this shit?

Tags: , , ,

36 Responses:

  1. Sam says:

    I know very little about ImageMagick, and a fair bit about colorspace/profiles...

    Does it only happen going from CMYK->RGB? Or does it happen for any image you run the command on? For instance RGB->RGB, or Grayscale->RGB

    Is it embedding a color profile into the new file?

    When you are asking the software to convert, does it have base profiles it assigns to an image?

    For instance, Photoshop will always assign the working color profile to an open document, even if it has none embedded in the image. It is the only way that the software can do the math of going from one space to another. This way, even if you have not assigned an ICC profile to the document, it still knows how to display/convert it.

    Thought 2
    Have you tried converting everything to Lab Color then to RGB?

    Also: if you have a choice between RGB and sRGB, go with the latter. Short version, displays the best on most people's computers.

    [color profiles were the bane of my existence while in the print world, too often we would have to hire a guy with a whole different coop full of chicken's to come in and wave them around instead of the ones we had]

  2. Ben Brockert says:

    Unrelated: did you change something recently in the stylesheets or something? On the main page, and to a lesser extent in any single page, everything bounces up and down as the page loads; looks like space appears and disappears between the title and text of the posts. Firefox 9.0.1 on a Mac.

    • bbot says:

      That's the +1 buttons.

      • Nils says:

        I'm surprised there are Facebook "like" (and Google "+1") buttons on this site… after all, they're being used to systematically collect data on people who are logged out, or don't even have an account. It'd be more of a service to the readers to remove them...

        • Nick Lamb says:

          Using ShareNice would be the logical choice. Not only on this site but also on the DNA Lounge site itself. But it would be work, and Jamie has explained at length how much he dislikes work. Also Facebook is the new evil empire and maybe you haven't been paying attention but Jamie has some sort of Lando Calrissian thing going on when it comes to evil empires.

      • Ben Brockert says:

        Ah, it appears the creatures at Mozilla broke the tracker killing plugin on the last update.

        I wonder how much a programmer gets paid by Google to break the plugins at every release.

    • Adolf Osborne says:

      Also, it's slow. Takes forever to render/download/ejaculate. Can't even scroll the browser window while its busy doing whatever its doing.

      Not-very-special 32-bit Firefox release nine-bazillion-and-two-point-four (or whatever the current version-of-the-hour is) on a decently-fast x64 Windows 7 box.

  3. Pat Gunn says:

    It's a known bug, and is to be fixed in the next major release of ImageMagic.

    http://www.imagemagick.org/discourse-server/viewtopic.php?f=3&t=20231

    • jwz says:

      Looks to me like that bug just says "RGB and sRGB are synonyms" or something. Nothing about images going all black, or different behavior on different machines.

  4. John Bloom says:

    Is the behavior different with graphics magick?

  5. citral says:

    My ImageMagick (v6.3.7) under Linux (Debian Lenny) behaves like your Mac ImageMagick. Invoking it with "-colorspace sRGB" gives a JPEG that is significantly darker than the source PNG, although it is not completely black (please clarify this point). So either your Linux installation's ImageMagick is faulty, or the developers changed ImageMagick's behavior at some point. It would help to have the exact version numbers of your ImageMagick installations.

    • jwz says:

      I'm seeing 100% black, not just darker. Both are 6.7.5, though it was happening with slightly earlier versions too (I upgraded both of them to make sure that wasn't the problem).

  6. You shouldn't need to specify a colorspace at all for PNG images, as they don't support CMYK at all, only RGB. In fact, it looks like ImageMagick goes crazy when you use a colorspace; images processed with the colorspace flag look like this, whereas they look right with no colorspace flag at all. (This is on a Mac; don't have Linux.)

    Have you tried ditching the colorspace flag?

    • jwz says:

      Without -colorspace RGB, CMYK PDFs will result in CMYK JPEGs instead of RGB JPEGs, which don't work right on the web. So yeah, I could change my script to only use -colorspace if the original file was a PDF (or a CMYK JPEG, or...) but the whole fucking point of ImageMagick is that I'm not supposed to have to care what the input file type is! (I'm using this in set of scripts that takes "arbitrary shit some flyer designer mailed me" and turns it into "something I can use".)

      • charkes says:

        Oh, CMYK PDFs, why didn't you say... I spent hours working on converting CMYK PDFs to RGB JPGs for web and they never came out looking right. Sure, I'd fix it so some did but then another batch would be somehow different and look absolutely horrible. (An ImageMagick upgrade probably contributed to that.)

        My tried and true, hasn't-failed-so-far method is to let GhostScript tackle the PDF first. If you want higher quality JPGs, up the exported resolution in gs.


        gs -sDEVICE=jpeg -dNOPAUSE -dBATCH -dSAFER -r300 -sOutputFile=full_$file.jpg $file.pdf
        convert 'full_$file.jpg' -density 72 -colorspace RGB -geometry 800x2000 'view_$file.jpg'
        convert 'view_$file.jpg' -geometry 200x200 'thumb_$file.jpg'

        • Ian says:

          Yep, this is what I've done: let GhostScript do the conversion from PDF.

          I still haven't got over the way that ImageMagick's default state is 'do the most annoying thing you can with this'.

          • jwz says:

            I appreciate the cargo-culting effort, but with PDFs, ImageMagick runs GhostScript. So it's the same thing.

  7. jwz says:

    Well this is just awesome. I rebuilt the latest from source on both machines, with the same configure options, and they are exhibiting other randomly-different behavior, like:

    convert -version -verbose -size 320x -gravity east caption:'should be aligned right' foo.jpg
    Version: ImageMagick 6.7.5-7 2012-02-28 Q32 http://www.imagemagick.org
    Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
    Features: OpenMP OpenCL

    The text is aligned right on Linux, and aligned left on Mac. Fan fucking tastic.

  8. My ImageMagick conversion to RGB and to sRGB results in completely equal files, SHA1-checked. Original was Canon EOS photo, sRGB already. Converting to CMYK and from that CMYK to RGB and sRGB also works. No black images, all images perfectly viewable.

    estebano:~ solik$ brew info imagemagick
    imagemagick 6.7.1-1
    http://www.imagemagick.org
    Depends on: jpeg, libtiff, little-cms, jasper
    /usr/local/Cellar/imagemagick/6.7.1-1 (1389 files, 32M)

    Some tools will complain unless the ghostscript fonts are installed to:
    /usr/local/share/ghostscript/fonts

    http://github.com/mxcl/homebrew/commits/master/Library/Formula/imagemagick.rb

    estebano:~ solik$ brew info little-cms
    little-cms 1.19
    http://www.littlecms.com/
    Depends on: jpeg, libtiff
    /usr/local/Cellar/little-cms/1.19 (21 files, 872K)
    http://github.com/mxcl/homebrew/commits/master/Library/Formula/little-cms.rb

  9. Victor Bogado says:

    Does the output files are the same, it could be that the files could have an embedded profile and since Linux didn't care until recently about those ,and mac Os do calibrate it's display that could be making the difference in the perception.

    • Victor Bogado says:

      what I meant was that maybe the two files generated by linux and mac os are indeed the same but when they are being viewed the two different Oses are displaying them differently.

      I think that the idea got a little mangled since I didn't proof read the comment before hitting submit. :-P

  10. Max says:

    Have you tried -profile? The convert documentation implies that it follows a different path than -colorspace.

    • jwz says:

      That's because, as far as I can tell, profiles have nothing to do with colorspaces. They're basically gamma curves. "+profile \*" does not convert a CMYK image to RGB, and does not affect the behavior of any following -colorspace options.

      • Max says:

        I was thinking you could try applying an sRGB profile to the output. But that doesn't help any if there isn't already a profile associated with the image, so it doesn't solve your convert any/every thing problem.

      • Odd, because that's not what the ImageMagick docs say:

        For a more accurate color conversion to or from the RGB, CMYK, or grayscale colorspaces, use the -profile option.

        For what it's worth, "convert cmyk.file -profile AdobeRGB1998.icc rgb.file" does what you want.

      • Edouard says:

        No no, profiles are supersets of colourspaces, gamuts, calibrations, gammas and every other variable that goes into determining what objective colour your pixels (RGB, CYMK, etc) should represent.

        sRGB is strictly speaking a profile, as it defines both a gamma (well, gamma plus a linear cutoff point) whitepoint, colourspace, chromacities, etc. So that is worth a try, except of course for the myriad of programs out there that don't properly parse and display a PNG with an embedded profile. Because they only look for a gamma chunk, or an sRGB chunk. Or get confused when multiple tags representing the same thing are present. Or the programming writing the app know how to get it right, but choose to interpret the chunks according the way they think the majority of people are currently interpreting it wrong.

        It's a fucking train wreck, and sends me to tears every time I have to think about it seriously. In fact I think I'll go and sob heavily into my pillow right now.

        Good luck by the way.

  11. FYI, there's an obscure Mac command line image tool called 'sips' that might be useful.

  12. I know this is the common lazyweb response of "why don't you do something totally different than the stated problem", but if the goal is "convert random file to web-sane JPG", why not use Photoshop's "dropplets" for actions, which create an app/executable that runs the Action, and can be called from scripts. I'll grant that "must run on linux" is a compelling argument against this method - though "remove linux from workflow" might end up being better for long-term sanity.

    http://www.photoshoplab.com/droplets-for-the-lazy.html

    • Big says:

      While I mostly agree with your sentiment, my pragmatic side is thinking "Sure, if you've got a recent-ish copy of Photoshop, a machine to run it on, and an OS that supports it available - but if you're doing this for your business and you want to stay compliant with the various licenses required, that's probably 3 orders of magnitude more expensive than throwing the files at Mechanical Turk for a buck or two a time…"

      I suspect jwz is hoping one of us will know some magic --dont-be-brain-dead flag to pass to imagemagik that'll make it "just work", for a whole lot less time/money than a legal PS license. (And sorry, but I'm not "that guy" today…)

  13. John Bloom says:

    Alright, my interest is piqued. Do you have any samples of the problematic source image files that you could share?

    • jwz says:

      Yes, all of them. "convert rose: -colorspace RGB out.jpg" versus sRGB on Mac versus Linux. See also this, "convert -size 320x -gravity east caption:'should be aligned right' foo.jpg".