One of the things it's telling about this image on the DNA Lounge front page is, "Compressing and resizing could save 170.4KiB (90% reduction)." Um, really? That's a 360px wide JPEG, compressed with quality 95, which admittedly could probably be lower -- but to get that image down to the 17k that Google says should be possible, I end up with this. Um, no.
This looks pretty good.
This is crap. Crap, sir! What the hell are they talking about?
And what is considered a reasonable compression setting for JPEGs in ImageMagick? 70 looks pretty fuzzy to me. Is "-interlace plane" believed to actually do any good?
(PageSpeed is also really adamant that I minimize my CSS, which could save a whopping 4KB. Prior to transport compression. Which like... fits in the first HTTP packet. Who gives a shit.)
It's 170k, and this is 2017. You're fine.
I imagine jwz is engaging in this exercize not because he actually gives a shit, but in an attempt to avoid being punished by Daddy Google.
Pagespeed is routinely full of shit, and afaik there's no way 17K is achievable there without significant compression artifacts. That particular image simply isn't going to compress very well as a jpeg, and no amount of blind Google optimism will change that.
Quality of 95 is way overkill though. 65 is what I'd recommend as a default for photos, 75-ish for hard-to-compress illustrations like you're likely to encounter. If you're bothered by that, I'd stop bumping up quality at 85 or so. Not a whole lot of return for your KB after that.
Re: the interlace/progressive option, https://yuiblog.com/blog/2008/12/05/imageopt-4/ is the best reference I found last time I was caring about imagemagick jpeg compression, and it suggests you're likely to see ~5-10% improvement for your images. Personally I think 10% larger images is worth never having to read imagemagick docs ever again, but its probably fine to turn it on.
I think they want you to generate the jpeg with their new-fangled encoder, and ignore retina screens. At 166px wide, that gives me a 32KiB jpeg, at 320px wide I get a 138KiB jpg.
It doesn't do it at all anymore (for some reason), but Chrome on iOS used to let you turn on some sort of compression setting and yeah, that's basically how every JPEG looked. That's what Google thinks people can stomach.
(But wouldn't that particular image perform better as a PNG anyways?)
Chrome can still do this. It works by proxying through Google. It was good for saving mobile data, and for making shitty networks more usable.
It stopped working when everything-ish went https a few years ago.
Have you tried PNG and running it through tinypng? Sometimes that works better than JPG for illustrations.
Or just snap the rubber band against your wrist a few times and let Google suck it.
Yah, I was going to ask what about PNG? Only photos should be JPGs.
I've been using -quality 85 -sampling-factor 2x2 for years. Seems to be the best possible compromise.
So you say 85, Jacob up there says 65. Is there any actual consensus out in the world? I assume there's no technical way to analyze this, it being subjective and perceptual and dependent on what you're compressing.
It’s perceptual yes. But 65 seems very low to me. Trying different values out shouldn’t take very long, at least.
Well, yeah, at some point probably a decade ago I did that, which is how my scripts all ended up with "95" in them. So I dunno. My perception was, "sometimes it's fine and sometimes it's not, so fuck it, err on the high side."
"That's crazy talk," I keep hoping to hear someone say, "here's a study with math in it that says you should definitely be using "-qual 82.7 -flux sideways".
In theory, that’s the whole point of q values. You’re choosing a perceptual quality and more complex images will be larger. The useful range is about 50-95, outside that range and things get weird.
I may be wrong, but I seem to remember that part of the reason there is no hard and fast rule is that different apps implement the jpg quality scale differently. So 95 in one app is not necessarily 95 in another app. This is possible because the JPG spec was not just written under the influence of weed, it was written under the influence of weed plus mushrooms plus PCB.
It's not a consensus, but I've been using 85 because apple recommended it for epub:
https://help.apple.com/itc/booksassetguide/en.lproj/static.html#itc258e584c2
I was guessing they error on the pretty side, but who knows.
I ran it through ImageOptim to losslessly reduce the file size (strip metadata, etc), and it got like 15% smaller. Doing that is usually a pretty good idea and it doesn't hurt quality. I've also found that it'll shut PageSpeed up even if it isn't as small as it thinks it "should" be.
Alternatively, so what others said and convert to PNG. You can also probably get away with lowering the color depth to 256 or even 128 colors. There's a decent chance it'll still be bigger than the JPG version but it will also be tack sharp.
Well, one of the things I do not want to do is spend my life manually twiddling and tweaking every image on the site. If the process of posting a flyer is: looking at it, scratching my chin, and saying, "Hmmmmm! Maybe the one should be a PNG instead of a JPEG!", well I'm just not going to do that. Unless some script is going to make that decision for me, and guess right every time, they're just always gonna be JPEGs.
Yeah, I guess I could have my script compress every image several different ways and pick the one with the smallest file size -- but how can that script tell me which one looks best? How is my script going to compute whether lowering the color depth made it look like shit?
I think a human is usually required wisely to choose whether to use a JPG or PNG, but the machine is often able to take it from there. It's usually a pretty easy choice (is this a photo or an illustration? Does it have a lot of fine text that JPG will mangle? Does it have a lot of red, which JPG will make into a bunch of lego?)
If you pick PNG, 99% of the time you can safely lower the color depth to 256 colors, so that should probably be the default setting. Even smooth color gradients will look fine because the encoder will dither them for you.
For JPGs, just pick whatever compression setting seems sensible and use that every time. Maybe 85%.
Well, you could, in theory, plug your script into Amazon Mechanical Turk.
Building async publishing queues to enlist third world workerbees to select band flyer images is a great idea, and something that you should get right to work on.
Real-life images, continuous changes in intensity/color: JPG
Long runs of solid colors (tickets/flyers/text): PNG or GIF
PNG/GIF can do run-length encoding when the same color is repeated for many pixels.
JPG uses fourier transforms, so smooth transitions compress very well, abrupt ones don't.
Did... did I say something that suggested that I don't know how JPEGs and PNGs work?
You write screensavers, so I'd assume that you have it down cold, but then there's this thread, so...
If you're going to insult me, at least try harder.
"Web Guy".
Dude, I was just trying to help. I am not in a place where writing about RLE and DCT to someone is an insult. Boring maybe, but not an insult.
This is just the sort of condescending post about absolutely basic comp sci concepts I would expect from someone who goes unironically as "Web Guy."
To be fair, 'Computer Guy' is pretty long-in-the-tooth, and when he tried out ''Augmented Reality Guy" at the conference, people thought he was selling acid.
As a first approximation, you could ask ImageMagick to count unique colors and pick a threshold like 256 to route the file. That may be bad if shrinking for the thumbnail introduced additional shades. What format are you usually starting from?
Also what does ImageOptim do that ImageMagick and GD do not?
It removes metadata, color space information (because you are always using sRGB), etc. These don't affect image quality. Optionally it will compress JPGs and lower PNGs to 256 colors automatically.
ImageOptim is a GUI tool, but CLI programs like MoxJPEG, pngquant, Pngcrush, etc will do the trick as well since you need it to be scripted.
This answer sounds like "it does nothing that convert -strip -colors N does not do."
Is there any technical innovation on the compression in there, or is it just another UI on top of the same old compression libraries that everyone in the world uses?
It losslessly squeezed another 5% out of the JPG you posted, so it's doing something that your ImageMagick/GD setup isn't.
But yes, it's a GUI wrapper around a collection of image compression libraries.
I assume this just means "it's -quality setting has a different default".
But what that setting is I cannot guess, since there seems to be no documentation on what it's doing other than "more better!!11" Sigh.
Using the word "losslessly" with a JPEG requires technical details to be understood or believed. Do they mean "it probably looks kinda the same"? Do they mean "no really, it's bit for bit identical"? They don't specify.
After the lossy downsampling, DCT and quantization steps, JPEG has to perform a lossless entropy encoding step. This is normally Huffman encoding. It can be arithmetic coding instead, which will compress 5% better than Huffman on average.
Interesting, it looks like the latest ImageMagick also does coding table optimization and it's on by default. I guess it must be doing the same thing as the others, because I am not seeing the file size change when I run "jpegoptim -f" or "jpegtran -optimize" over its output.
You assume wrong. It uses JPEGOptim and jpegtran when you give it JPEGs (it’s just a frontend for a bunch of other separate optimisers). So its claims are to be taken literally.
To wit: the current 31-1-thumb.jpg is 191,858 bytes; when dropped into ImageOptim, 182,142 bytes come out the other side. I converted both images to BMP and got bit-for-bit identical files.
Huh. You're right, but I wish I could tell what it's actually doing. I'm seeing the file get smaller when using JPEGOptim with only jpegoptim and jpegtran enabled, and yet when I run those from the command line, I do not see a savings.
It must be doing
jpegoptim --strip-all --all-normalandjpegtran -optimize -copy none. I used the binaries from inside the ImageOptim.app bundle to check and got the same 182,142 figure. (Turns out it’s jpegtran doing all the work in this particular case.)It does actually do different things, but it is also close to black magic fuckery.
For PNGs, you can re-arrange the image chunks inside the file to get them to compress better. You can also use the zopfli compressor instead of standard gzip on the chunks, to make them smaller. There's a comparison of tools here:
https://css-ig.net/png-tools-overview
...but personally I've been piping the output of optipng into advdef. It's the best and simplest option 95% of the time.
There are similar methods to improve jpegs - jpegoptim or jpegtran if you want to extra-compress existing jpegs, or mozjpeg if you're making new ones. There's too much Math involved for me to fully understand how they work, but the results are indeed smaller.
Whether you care or not about the extra complexity in order to save a few bytes is of course up to you.
Yeah, who knows.
"Add this switch to your existing convert commands" - hey, why not, I'd do that for 1%.
"Re-tool your image-conversion pipeline to use something other than ImageMagick" -- I dunno, I guess I'd do that if I strongly believed I'd save 20% file size at the same visual quality. Otherwise I probably wouldn't bother.
(Though I'd also be tempted to assume that if I just wait six months, those features will land in ImageMagick.)
Anything that requires a human to eyeball the image before making decisions about settings is a non-starter. I'm just not ever going to take the time to do that.
If you're going to use JPG for everything (sensible enough choice), it doesn't seem like a huge deal to add another tool to the pipeline to squeeze another few percent out of it when it doesn't affect image quality.
If you're skeptical about it really being visually lossless, can't you test it out? Run an image through your usual ImageMagick process, and then run that through one of these tools (being careful not to enable any lossy compression). Convert both the images to some dumb archaic uncompressed format like TIFF and they should be the same size. Right?
Actually, nevermind, that test doesn't make any sense.
The Google does not approve of your lackadaisical attitude towards image workflows.
There is a free tool called jpeg-recompress (https://github.com/danielgtaylor/jpeg-archive) which does essentially the same thing as these GUI and web-backed lossy JPEG compressors (ImageOptim etc), and it's not in ImageMagick. The math is over my head, but the tool works to automatically determine a minimum JPEG quality for a given image without excessive loss of visual quality, and the amount of quality loss is a command-line parameter.
It can use different methods to detect quality change - the default is https://en.wikipedia.org/wiki/Structural_similarity
That said, this method works quite well on JPEGs that are photographs, but it doesn't do much for your sample image - as has been beaten to death, it's slightly better as a PNG. The best free tool for shrinking PNGs that I know of is pngquant -
https://pngquant.org/
Yeah, if I convert that image to a PNG with the highest compression, it doubles in size. If I quantize it to 128 colors, it's about 55%, but at that point I might as well have used GIF -- which in fact is almost exactly the same size.
WEBP?
Oh FFS go away.
I meant PageSpeed suggested using WEBP, by chance?
PageSpeed did not mention that. If it had, I would have immediately known the answer to "is PageSpeed just full of shit?"
Google research got me the following mathematical minimum
(min file size possible) = Shannon Entropy * (file size)
I should be possible to write, in less than 17K, a text description of the image so an artist can reproduce it. Writing an uncompression tool for this format is left as an exercise for the reader.
This reminds me of Kolmogorov complexity, which is a non-trivial exercise for the reader.
This is actually what PageSpeed is suggesting: provide a shitty compressed image which kind of hints at what the real image ought to look like, and assume that the viewer can reconstruct what the image should look like in their head. This of course fits well with Google's attitudes towards people: they're basically a free resource for it to exploit.
a: Yeah png was supposed to not fail; offer it an inexactness parameter and watch your stuff converge. -fuzz n (n is bitwise distance...on the sad 10bit displays of The Convexity here? So 23?)
b: WebP, because that is 4 color mono rip with matte redacted
c: Wait, what happened to DjVu then? Surely that's just compiled into all the stuff as an alternative to H.265+3/11ths prior to loading or refusing those blobs? It's the far future; fractal compression and posit maths all around! 6 instruction-set security processing supervision layers, each with nicer whistles, gayer shoes, and uninitialized.
d: Should the program return a figure of merit for Klown Soul Derangement on whole, or per bit? That's a .ecc file and you know it.
e: What, you don't fuzz all loaded pages? What kind of Computista doesn't keep a running missing codes monitor up? I bet your ethernet heatsinks haven't been changed in months? Look, the text on that pack is so close to being greeked, machine brains around are steaming buns trying to render serviceable Tamil and Gujarat Block type to match.
f: If Google doesn't totes gush about alternate renderings on the slightest provocation, they were just whistling "That Deprecated Optimization For Our Pages." Speaking of which, what cabaret songs come up in the new Blade Runner Priss Arc?
JPEG2000 would have insisted on a larger source. Since you are handmaking tiny pages that are delivered in little townhouses to readers, just turn on +ai.nebbish
With my apologies for maybe you wanted to workshop this another way.
I agree! Codec_ebooks is my favorite Working Group!
JPEG2000! The wavelets made me do it!
My wife did it, but it took her like nine months.
Can't the artist just supply it in SVG form? If it's not common yet, you could build a cloud application that lets them upload it in vector form, and try out different formats themselves.
..
.
That 'suggestion' by Google is the sort of thing you get when you 'test' stuff by having folks in countries where domestically made Samsung phones with 800 x 480 displays are the most popular devices.
This suggestion is only slightly more sane than "describe the image using a 140 character haiku, better yet, have someone else describe the image using a 140 character haiku".
Here's a suggestion: put your toe on the trigger of a shotgun and put a vent in your head. When somebody takes a picture of your gnarly corpse they can then decide whether JPG or PNG best captures the little bits of blood and brain on the wall behind you.
As a rule of thumb JPG would be best suited to that scene. Probably at q=80 or q=85. Somewhere in there.
That's it. They don't like resizing images in HTML (and ignore the resize-on-hover you're doing).
$ convert 31-1-thumb.jpg -resize 155x -quality 68 - | wc -c
17425
You should ignore PageSpeed in this case; you know better than it, in this particular circumstance. The other alternative is to ask your artist to produce something that compresses better at the expense of artistic quality.
fuck google pagespeed, use webpagetest.org.
it says to shrink it because it doesn't care about retina screens, so any image that is larger than the size it's being displayed at gets this warning. That being said, you should use srcset and sizes :)
95 quality is way too high, once you go above 80 you get small improvements at massive filesize increases - though i guess that number might be different for your tools, but every program i've ever used has been this way. I just resaved your jpeg as a jpeg at 60 and it looks pretty good to me at 67KB - it would look even better if i had the original material to save from. It's a hard image to compress with all the fine details. People saying to use png are insane, i don't see how this could compress smaller with png. 60 is my standard setting for jpegs, there is no consensus. I think WordPress uses 70 when resizing images.
And yeah removing metadata that you don't need always helps. ImageOptim removed 9KB from your image without changing image quality at all.
I'm using srcset on images in the jwz.org blog posts, but I don't think they would help with DNA flyers and photos, because there are really only two sizes at which those are ever displayed: "thumb" and "full". I have "thumb" rendered out at 360px wide, and there's no way srcset would ever pick a width less than that.
This is the full-sized version. What metadata were you able to strip from this? Both of those were already using convert -strip.
Here's what I get, shrinking the full-sized version to 360px wide and saving at 60 quality with Affinity Photo. Comes to 67KB. Here it is at 80 quality.
srcsetwould gladly pick a width less than that if you specifysizesproperly and if you have the right image sizes generated. Specify additional image sizes to be generated in your functions.php file with something like:add_image_size('teeny', 80);which means 80px wide, no max height, no cropping. And usewp_get_attachment_image_srcsetto output thesrcset. Gotta dosizesby hand.I'm not sure exactly what ImageOptim stripped from your original thumb, here's my settings for it.
Check out https://images.guide
Oh hey, guess what! The version of jpegtran that comes wth JPEGoptim, and which is the part of that package that does the actual heavy lifting, does what it does by virtue of being based on MozJPEG. And here's the shocking part: that's a drop-in API-compatible replacement for the ancient libjpeg, so you can just link ImageMagick against MozJPEG and boom, ImageMagick generates smaller and better JPEGs. Build ImageMagick with: CPPFLAGS=-I/opt/mozjpeg/include LDFLAGS=-L/opt/mozjpeg/lib64
Does that make it produce minimal JPEGs from the start? Or does it still throw in boilerplate crud that leaves work for jpegtran et al to do?
As far as I can tell, it makes "jpegtran -optimize -copy none" have nothing left to do.
Wow! [Eyes cross][Hairs cross][Barfs .scene assets for IT][Skin turns '70s tones a while]
Hey!
OK. One more drive through, for now. This disassmbler talk you guys sling so freely. makes me dread ever opening my mouth, and coming across as implicating deconstructions. So my LISPy "wishful-thinkings" of a northbound SDN api "photonic-s witch-fabric" will be kept to myself
There are binary search options to find the tradeoff point. https://codeascraft.com/2017/05/30/reducing-image-file-size-at-etsy/
It's always going to be particular to the image, there's no number one could give you.
I think your MozJPEG drop-in is probably enough of a win (though I definitely agree with earlier posters that 95 is overkill), but I've had impressive results with this:
mogrify -path OUTPUT_PATH -filter Triangle -define filter:support=2 -thumbnail OUTPUT_WIDTH -unsharp 0.25x0.25+8+0.065 -dither None -posterize 136 -quality 82 -strip -define jpeg:fancy-upsampling=off -define png:compression-filter=5 -define png:compression-level=9 -define png:compression-strategy=1 -define png:exclude-chunk=all -interlace none -colorspace sRGB INPUT_PATHFrom here: https://www.smashingmagazine.com/2015/06/efficient-image-resizing-with-imagemagick/#image-optimization
(The full article is neat in that he backs up his settings with tests* and explanations, but likely is a little dumbed down for where you're sitting. *heavy on filesize comparisons, but glossed over "visually similar" without doing the structural similarity tests)
I implemented this on a photo-heavy site recently and was really blown away by the filesizes and quality when looking at the output photos on their own. Direct and careful comparisons with the originals revealed some loss of depth and sharpness (obviously, with the posterizing and blurring), but the end result was a really neat balance, and the trade-off in filesize was more than worth it. I also spent an hour playing with settings and test images and couldn't do any better.