Regaining Hugo Image Processing
Table of Contents
So as of now, there’s about half as many LFS objects in this blog’s repository, the page size has fallen by, well, not exactly a rock, but by a noticeable amount, and as of now, all my images are around the same size again. So what gives?
Well like the trend is on this site, I offloaded some responsibility to something else. And by something, I once again mean Cloudflare. This time, not workers, but a feature standard with the Pro plan that I switched to a bit ago.
Polish is the name of the service that Cloudflare offers to Pro and above levels that includes dynamically compressing images for capable clients, passively.
It operates in one of two modes, lossy, or lossless (what I use), and has an option to enable conversion to WebP if the client supports it.
I’ve already talked about WebP, so go see that post for more information about the format, though for a brief rundown: WebP images are WAY lighter than PNGs, in my case, usually 0.870829417% as large?
That has to be an error, hold on…. (2372 / (2.66 x 1000 x 1000) ) x 100 = 0.870829417.
Yeah that checks out, the average file size is apparently under 1% that of PNG for my files.
As that post up there describes, I used the
<picture> HTML element to have the browser decide which of the two it wanted to load itself.
Well as of now, that system is out, and Cloudflare is passively re-compressing images on-the-fly as it sees fit.
Admittedly yes, lossless WebP are going to be larger, but even still, the format is designed specifically to be smaller than PNG, so even with the highest possible file size, it’s still a lot smaller for supporting devices. At least the nice thing is, I don’t have to worry about compatibility anymore, that’s on Cloudflare. If they messed up somewhere, well, quite frankly, that’s not my issue as I can’t really fix it.
You can tell when Polish ran on an image, because of the
cf-bgj (BackGround Job?) and
cf-polished response headers:
GET https://teknikaldomain.me/gallery/stiffening-up-the-x52-pro/00_clips_rendered_side_by_side.png HTTP/1.1 Host: teknikaldomain.me Accept: image/webp,image/apng,image/*,*/*;q=0.8 Accept-Encoding: gzip, deflate, br Accept-Language: en-US,en;q=0.9 Cache-Control: no-cache Cookie: __cfduid=d7b3271d684c95594353d20c9354ac7a31567472929 Pragma: no-cache Referer: https://teknikaldomain.me/ Sec-Ch-Ua: "Vivaldi 80" Sec-Fetch-Dest: image Sec-Fetch-Mode: no-cors Sec-Fetch-Site: same-origin User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.136 Safari/537.36 HTTP/1.1 200 OK Accept-Ranges: bytes Age: 101944 Alt-Svc: h3-27=":443"; ma=86400 Cache-Control: public, max-age=172800 CF-BGJ: imgq:100,h2pri CF-Cache-Status: HIT CF-Polished: origFmt=png, origSize=85721 CF-Ray: 5a321a48cdcdb9b6-ATL CF-Request-ID: 03532ec17c0000b9b60c1b0200000001 Content-Disposition: inline; filename="00_clips_rendered_side_by_side.webp" Content-Length: 56812 Content-Type: image/webp Date: Sun, 14 Jun 2020 06:47:55 GMT Etag: "5ee36e40-14ed9" Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" Last-Modified: Fri, 12 Jun 2020 12:00:00 GMT Referrer-Policy: strict-origin-when-cross-origin Server: cloudflare Strict-Transport-Security: max-age=31536000; includeSubDomains; preload Vary: Accept X-Content-Type-Options: nosniff X-Frame-Options: DENY X-Xss-Protection: 1; mode=block
We sent the
accept header indicating that
acceptable to receive, even though the URL file ends in
This is probably what allows it.
We also specify that
br compression is allowed in the next line, or Brotli compression, which is kinda cool but outside the scope of this post.
Cloudflare response with a
cf-bgj indicating the image quality of
100, or lossless, and
h2pri, which I can only interpret as “HTTP/2 Priority” (yes, this was actually an HTTP/2 request that I cleaned up because syntax highlighting reasons).
cf-polished tells us that the image was originally a PNG, of size 85,721 bytes.
content-length tells us that this is now 56,812 bytes, which is less than 50% savings, but that PNG was already small as it is.
Given that no PNG locally should exceed 1,000,000 bytes (1 MB), the values you see for this might be a little large in comparison, but the majority of images are on the CDN anyways, and that is something else to talk about.
Also, there is no header indicating compression, meaning that if this was transferred compressed somehow, you’d get even less data being sent.
The HTML for my pages is back to a pretty default
<img> tag, I almost don’t need to override the formatting.
The only differences are that I still center non-max-width images, and I still have a link to just the image alone if you click on it, to view it full-screen.
Polish does not work in resources returned from Workers though, meaning all the fuss I made about checking for WebP support and sending a 302 to the right one if it was possible?
That I still have to rely on, but I’m using what I assume to be the exact same point of comparison, the
What I find interesting though is that I had two choices: transparently swap out the image type in-flight, meaning you request a PNG but get a WebP, or I can issue an HTTP redirect to the proper one. Well I obviously went with the latter, and Cloudflare decided to go with the former on Polish. Not to say that this was a bad idea or a dumb one, no, but it does seem a little weird to have what you’re asking for just swapped out because it’s possible. And let me re-iterate, that is possible. If the browser says that it is capable of accepting something that it actually is incapable of accepting, then that is 100% the fault of the browser, and an indication of a poorly built one at that.
Hugo Image Processing
And this is (kinda) the reason behind the title of this post: Hugo has facilities in place to automatically handle image processing for, well, images that it has access to, like resizing them, changing quality, re-encoding, stuff like that. Well, the engine that Hugo uses to do this does not recognize the WebP format, so I had to disable that and do everything myself. As of now, while I probably still will to some degree to ensure optimum results, Hugo can do the majority of the work.
There is just one caveat: Hugo is a static content generator, and it only works with what it can see. Images that I’ve put into the CDN and deleted from disk it has no clue about, and those I have to do all the work myself. But for things like featured images, or image slideshows where the files have to be present on disk, then it can have the luxury of taking care that they’re in spec for the site, and Cloudflare takes care of choosing the best format between PNG and WebP to serve to you when it’s requested.
The only image processing this theme is set up to do is some basic resizing, so having the images be put through a processing pipeline twice before delivery isn’t going to cause that much of a big deal.