Regaining Hugo Image Processing

Table of Contents

So as of now, there’s about half as many LFS objects in this blog’s repository, the page size has fallen by, well, not exactly a rock, but by a noticeable amount, and as of now, all my images are around the same size again. So what gives?

Well like the trend is on this site, I offloaded some responsibility to something else. And by something, I once again mean Cloudflare. This time, not workers, but a feature standard with the Pro plan that I switched to a bit ago.

Polish

Polish is the name of the service that Cloudflare offers to Pro and above levels that includes dynamically compressing images for capable clients, passively. It operates in one of two modes, lossy, or lossless (what I use), and has an option to enable conversion to WebP if the client supports it. I’ve already talked about WebP, so go see that post for more information about the format, though for a brief rundown: WebP images are WAY lighter than PNGs, in my case, usually 0.870829417% as large? That has to be an error, hold on…. (2372 / (2.66 x 1000 x 1000) ) x 100 = 0.870829417. Yeah that checks out, the average file size is apparently under 1% that of PNG for my files. As that post up there describes, I used the <picture> HTML element to have the browser decide which of the two it wanted to load itself. Well as of now, that system is out, and Cloudflare is passively re-compressing images on-the-fly as it sees fit.

Admittedly yes, lossless WebP are going to be larger, but even still, the format is designed specifically to be smaller than PNG, so even with the highest possible file size, it’s still a lot smaller for supporting devices. At least the nice thing is, I don’t have to worry about compatibility anymore, that’s on Cloudflare. If they messed up somewhere, well, quite frankly, that’s not my issue as I can’t really fix it.

You can tell when Polish ran on an image, because of the cf-bgj (BackGround Job?) and cf-polished response headers:

GET https://teknikaldomain.me/gallery/stiffening-up-the-x52-pro/00_clips_rendered_side_by_side.png HTTP/1.1
Host: teknikaldomain.me
Accept: image/webp,image/apng,image/*,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cache-Control: no-cache
Cookie: __cfduid=d7b3271d684c95594353d20c9354ac7a31567472929
Pragma: no-cache
Referer: https://teknikaldomain.me/
Sec-Ch-Ua: "Vivaldi 80"
Sec-Fetch-Dest: image
Sec-Fetch-Mode: no-cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.136 Safari/537.36

HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 101944
Alt-Svc: h3-27=":443"; ma=86400
Cache-Control: public, max-age=172800
CF-BGJ: imgq:100,h2pri
CF-Cache-Status: HIT
CF-Polished: origFmt=png, origSize=85721
CF-Ray: 5a321a48cdcdb9b6-ATL
CF-Request-ID: 03532ec17c0000b9b60c1b0200000001
Content-Disposition: inline; filename="00_clips_rendered_side_by_side.webp"
Content-Length: 56812
Content-Type: image/webp
Date: Sun, 14 Jun 2020 06:47:55 GMT
Etag: "5ee36e40-14ed9"
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Last-Modified: Fri, 12 Jun 2020 12:00:00 GMT
Referrer-Policy: strict-origin-when-cross-origin
Server: cloudflare
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Vary: Accept
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block

We sent the accept header indicating that image/webp is acceptable to receive, even though the URL file ends in .png. This is probably what allows it. We also specify that br compression is allowed in the next line, or Brotli compression, which is kinda cool but outside the scope of this post. Cloudflare response with a cf-bgj indicating the image quality of 100, or lossless, and h2pri, which I can only interpret as “HTTP/2 Priority” (yes, this was actually an HTTP/2 request that I cleaned up because syntax highlighting reasons). And cf-polished tells us that the image was originally a PNG, of size 85,721 bytes. The new content-length tells us that this is now 56,812 bytes, which is less than 50% savings, but that PNG was already small as it is. Given that no PNG locally should exceed 1,000,000 bytes (1 MB), the values you see for this might be a little large in comparison, but the majority of images are on the CDN anyways, and that is something else to talk about. Also, there is no header indicating compression, meaning that if this was transferred compressed somehow, you’d get even less data being sent.

HTML

The HTML for my pages is back to a pretty default <img> tag, I almost don’t need to override the formatting. The only differences are that I still center non-max-width images, and I still have a link to just the image alone if you click on it, to view it full-screen.

CDN

Polish does not work in resources returned from Workers though, meaning all the fuss I made about checking for WebP support and sending a 302 to the right one if it was possible? That I still have to rely on, but I’m using what I assume to be the exact same point of comparison, the accept header.

What I find interesting though is that I had two choices: transparently swap out the image type in-flight, meaning you request a PNG but get a WebP, or I can issue an HTTP redirect to the proper one. Well I obviously went with the latter, and Cloudflare decided to go with the former on Polish. Not to say that this was a bad idea or a dumb one, no, but it does seem a little weird to have what you’re asking for just swapped out because it’s possible. And let me re-iterate, that is possible. If the browser says that it is capable of accepting something that it actually is incapable of accepting, then that is 100% the fault of the browser, and an indication of a poorly built one at that.

Hugo Image Processing

And this is (kinda) the reason behind the title of this post: Hugo has facilities in place to automatically handle image processing for, well, images that it has access to, like resizing them, changing quality, re-encoding, stuff like that. Well, the engine that Hugo uses to do this does not recognize the WebP format, so I had to disable that and do everything myself. As of now, while I probably still will to some degree to ensure optimum results, Hugo can do the majority of the work.

There is just one caveat: Hugo is a static content generator, and it only works with what it can see. Images that I’ve put into the CDN and deleted from disk it has no clue about, and those I have to do all the work myself. But for things like featured images, or image slideshows where the files have to be present on disk, then it can have the luxury of taking care that they’re in spec for the site, and Cloudflare takes care of choosing the best format between PNG and WebP to serve to you when it’s requested.

The only image processing this theme is set up to do is some basic resizing, so having the images be put through a processing pipeline twice before delivery isn’t going to cause that much of a big deal.