When we consolidated the Germany nodes as "hansels" (contended) and "gretels" (dedicated), I took the opportunity to upgrade the base OS from Ubuntu 22.04 to Ubuntu 24.04. While this change seemed to go off smoothly (and indeed, newer kernels are better at dealing with I/O deadlocks), on unintended side-effect was that our hardware transcoding stopped working!

Hardware transcoding fixzored

An observant Nazgul user noticed this when their Plex instance started maxxing out 13 out of 16 of their vCPUs, and we did some tests. Our first test was (as usual) to update the iGPU drivers, but this made no difference (GPU was detected, just Plex refused to play the media).

It turned out that the group ID for the system "render" group changed during the upgrade from 109 (22.04) to 993 (24.04), which prevented our pods from being able to utilize the iGPU (although it was still visible, and thus configurable in the streamers).

I pushed a second rollout through during the maintenance period to address this, so after a restart, the Hobbit / Ranger / Halfling and Nazgul streamers should once again have full hardware transcoding support!

Aaar MediaCovers now persist

The Aars (Radarr, Sonarr, etc) keep copies of media posters etc, in a folder under config called MediaCovers. In some cases, this folder has been observed to grow > 20GB. Some time ago, we switched this particular folder path to ephemeral storage, so that our precious, expensive, NVMe-backed network storage wouldn't get filled up with millions of 5MB cover art images!

The downside to the ephemeral storage is that the posters would be lost upon restart, and the Aaar would have to go and re-download them again, making for a poor user experience (no functional impact though)

As of tonight's update, we're storing these MediaCovers in a remote B2 bucket on an rclone mount, so that (1) they'll persist across restarts, and (2) they won't clog up our precious storage. After the restart, your covers will once again be gone, but once the Aarr re-downloads them, they should now persist across restarts!