Optimizing images with the HTML <picture> tag

If you have a website, you probably want it to be fast. Websites that load fast are more fun. They’re also higher ranked on Google. If you make money from your website, chances are you’ll make more money the faster it loads. Faster websites also use less energy.

One common reason why websites aren’t fast is because they have a lot of images. Images take time to load. Generally, the browser will load your website, find out that it contains a bunch of images, and start loading those as well. You can optimize that with HTTP/2 Server Push. Another optimization is to simply make your images smaller, without sacrificing quality - that’s what this post is about.

MDN has a great primer on different image types if you’re interested in that. Tl;dr: PNG, JPEG and GIF are supported by pretty much every browser. But newer formats, like AVIF, WEBP, and JPEG XL are often more efficient (yeah, JPEG XL images are smaller than JPEG. don’t ask me.) The same picture can be 20%-90% smaller with AVIF compared to PNG, without sacrificing quality.

In my experience AVIF tends to perform best. But I’d recommend trying all the formats for each image and seeing which ends up with the smallest file size!

#1 Convert images to another format

I like using imagemagick for that, because it’s simple and cross-platform:

magick convert image.jpg image.avif

You can do that for every image/format combination. There’s also avif.io for AVIF specifically.

#2 Including multiple formats in HTML

This is important because some browser don’t support all the new image formats, and no, it’s not just Internet Explorer. For example, Safari currently doesn’t support AVIF and WebP only works on MacOS 11+. JPEG XL doesn’t seem to work anywhere unless you use special flags. I’d still recommend including JPEG XL versions of your images, because chances are that browser support will come eventually.

You can inspect the image above to see the <picture> element in action

Traditionally if you want to put an image in your HTML you use the <img> tag, with a single src attribute that points to the image URL. That’s no fun. We’ll use the <picture> tag instead, which lets us specify multiple source URLs with different MIME types. The browser will choose the first one it supports.

Here’s an example (taken from this post):

<picture>
<source srcset="/document.designmode.avif" type="image/avif">
<source srcset="/document.designmode.jxl" type="image/jxl">
<img src="/document.designmode.jpeg" alt="Screenshot on iOS with selected text, showing a popup with options to make text Bold, Italic, or Underlined" width="375" height="153" class="border">
</picture>

The first source URL is the AVIF format, because that’s the smallest one. Chrome and Firefox will use that one. Safari doesn’t support the image/avif MIME type, so it will skip that one. It will skip the JPEG XL version as well, and fallback to the last source, the JPEG version. Note that the last fallback URL is inside an <img> tag, while the previous one are inside <source> tags. The final <img> tag also specifies common attributes like width, height and alt text. Those attributes will apply regardless of which source the browser chooses! Browsers that don’t support the <picture> tag at all (can you guess which one?) will simply ignore it and treat the <img> tag like a normal image.

#3 Server Push for <picture> elements

Ok, it’s time to leave the realm of reasonable performance considerations and move on to crazy hyperoptimizations. Let’s say you followed the advice at the top and setup HTTP/2 Server Push for your images. How do you decide which image format to push to the browser, before it had an opportunity to parse your HTML on its own and decide which format it wants?

Most browser send an Accept header that indicates which format its supports. For example, Firefox Developer Edition Version 113.0b9 sends the following Accept header when loading a webpage:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8

To find out the supported formats, look for mime types starting with image/. And, importantly, ignore the */* at the end. In this case, the browser tells us it supports AVIF and WEBP images.

This approach isn't perfect - as you can see, Firefox doesn't tell us that it supports JPG and PNG, even though it does. The */* would indicate that it supports all formats, which is obviously not the case. An alternative (or additional) approach would be parsing the User-Agent header, comparing it with an up-to-date copy of the various caniuse tables linked above, and using that to guess which format the browser will request. I say guess because there are a lot of things that can go wrong here: Maybe the user changed their User-Agent string. Maybe they’re using a new browser version that’s not in your caniuse table. Maybe they’ve set custom flags in their browser configuration. You never know.

Here’s a simplified example in PHP so you know what I’m talking about:

$browser = get_browser(null, true);
$name = $browser['browser'];
$version = intval($browser['majorver']);

// Firefox >= 93 and Chrome >= 85 support AVIF
if ($name === 'Firefox' && $version >= 93 || $name === 'Chrome' && $version >= 85) {
header('Link: rel=preload; href=/document.designmode.avif; type=image/avif');
} else {
header('Link: rel=preload; href=/document.designmode.jpeg; type=image/jpeg');
}

If you’re going to do something like this, I’d recommend setting up server-side analytics so you know if (when) your pushed image format ends up being different from the one the browser requests. If you want to get real fancy, you could automatically update your caniuse table based on those analytics results as well (e.g., when a new Safari version is released that supports AVIF, your system picks it up after the first few requests and starts pushing AVIF to all new requests where the User-Agent indicates that Safari version.)