Buy Me a Coffee

Buy Me a Coffee!

Friday, October 16, 2026

Does Plex Agree With IMDb? Auditing 919 Movie Runtimes with PowerShell

I have a lot of movies in Plex. I also have a nagging suspicion that some of them are wrong: a rip that got cut short, a theatrical cut when I thought I had the director's cut, a file that is really just a trailer wearing a movie's name. Plex will happily play all of them and never say a word.

So I decided to audit the whole library. The idea is simple: pull every movie's runtime from Plex, look up the runtime IMDb has for the same title, and flag anything that doesn't line up. It should take an afternoon.

It did not take an afternoon. It's a good example of how three small bugs can stack up behind each other, so I'm going to walk through the failures as well as the finished script.

The plan

The pieces are straightforward:

  1. Ask the Plex API for every item in my Movies library.
  2. Pull the IMDb ID out of each item's metadata (Plex stores these as imdb://tt... GUIDs).
  3. Ask the OMDb API for that IMDb ID and read back the runtime.
  4. Subtract, export to CSV, and sort by the biggest differences.

Plex reports duration in milliseconds, so that's a divide by 60,000. OMDb returns a string like 142 min, so that gets a regex. Nothing exotic.

First run: 919 movies, 919 "Missing Match"

The script ran cleanly, with no errors and a nice progress bar. Then I opened the CSV and every single row said Missing Match in the IMDb ID column and N/A for the IMDb runtime.

A script that fails loudly is a gift. A script that finishes happily and gives you nothing is the one that wastes your evening.

Bug 1: Plex doesn't hand out GUIDs by default

The library listing endpoint (/library/sections/{id}/all) leaves out the Guid array unless you ask for it. The fix is one query-string parameter:

$PlexApiUrl = "$PlexUrl/library/sections/$LibraryId/all?includeGuids=1"

Without includeGuids=1, my loop looking for imdb:// never matched anything, so no movie ever got looked up.

Bug 2: the OMDb URL had no query string

This one was sitting right behind the first. My OMDb call looked like this:

$OmdbUrl = "http://omdbapi.com"

No IMDb ID. No API key. The comment above it even said # FIXED: added required parameters, which tells you how much to trust comments. The URL had gotten mangled somewhere along the way, and I never noticed, because bug 1 meant that line never executed.

The corrected version:

$OmdbUrl = "https://www.omdbapi.com/?i=$ImdbId&apikey=$OmdbApiKey"

It also would have failed silently. I had wrapped the call in a try/catch with an empty catch block and a comment about "transient network issues." That's how you get a CSV full of N/A and no clue why. The catch block now logs a warning and records the reason in a Status column.

Bug 3: the response wasn't JSON, as far as PowerShell was concerned

With includeGuids=1 added, I expected victory. Instead I got this:

Found 0 movies. Comparing runtimes...

Plex had clearly answered, because the raw response started with "size":919. But Invoke-RestMethod handed me a plain string instead of an object, so $PlexResponse.MediaContainer.Metadata was empty.

The cause is a nasty little detail. With includeGuids=1, each movie has two keys that differ only by capitalization: guid (a single string like plex://movie/...) and Guid (the array of external IDs). PowerShell's JSON parsing is case-insensitive, so it sees a duplicate key, gives up, and returns the raw text. No exception, just the wrong type.

The fix is to parse the JSON myself into case-sensitive hashtables:

$Raw  = Invoke-WebRequest -Uri $PlexApiUrl -Headers $Headers -Method Get
$Json = if ($Raw.Content -is [byte[]]) { [Text.Encoding]::UTF8.GetString($Raw.Content) } else { $Raw.Content }
$PlexResponse = $Json | ConvertFrom-Json -AsHashtable
$Movies = @($PlexResponse['MediaContainer']['Metadata'] | Where-Object { $_ })

After that, fields come out with $Movie['title'] and $Movie['Guid'] instead of dot notation, and the two keys finally stay separate.

I also added a check that stops the script and prints diagnostics if Plex returns zero movies. That check is what showed me the response was a string, so it paid for itself immediately.

Three small things I'm glad I added

A cache. OMDb's free tier allows 1,000 requests a day. My library has 919 movies. Between debugging runs, I would have burned through the quota before the script even worked. Results now get stored in omdb_cache.json, so re-runs only pay for movies they haven't seen.

A -Limit switch. .\PlexLengthChecker.ps1 -Limit 10 processes ten movies. I used it for every test run.

The token in a header. I moved the Plex token out of the URL and into an X-Plex-Token header, so it doesn't end up in logs or in a screenshot for this post.

Results

  • 905 of 919 movies compared, 85 flagged as differing by more than 3 minutes
  • 14 movies with no IMDb ID in Plex at all
  • Most of the flagged were either extended versions, but I will check them out anyway.  I will be heading to the basement this weekend to pull my physical media and ripping them again.



Even before the comparison, the Plex side of the data had some things that caught my eye:

  • Clue (1985) shows up four times: one at about 97 minutes, and three at roughly 7 to 9 minutes each. That is where I added the film's alternate endings as separate library items.
  • A Bug's Life is listed at about 21 minutes, and Birdman at about 33. Those look like partial rips, which is exactly what I built this for.
  • Several titles appear twice with different runtimes, such as American Gangster, Stripes and Straight Outta Compton. That's usually two cuts of the same film, and it matters here because IMDb lists only one runtime per title. Those will show up as "mismatches" that aren't actually errors, so the flag column is a list of things to look at and not a list of confirmed problems.

What I'd tell myself at the start

  1. Test on ten items first. All three bugs would have been obvious in a ten-item run. A full run just made the failure bigger.
  2. Never leave a catch block empty. Mine hid bug 2 completely.
  3. Check the type of what came back. A string where you expected an object is the whole story of bug 3, and a one-line GetType() would have shown it sooner.
  4. Don't trust your own "FIXED" comments. Run the code.

The script

The full script is here: https://gist.github.com/lsmithmier/20b75e25cbed3971775c0112f7c16c17. You'll need a Plex token, your movie library's section ID, and a free OMDb API key. It requires PowerShell 7, because -AsHashtable isn't available in Windows PowerShell 5.1.

If you try it on your own library, I'd like to hear what it finds. Anyone who has ever ripped their own discs probably has at least one Clue hiding in there.

Sunday, September 20, 2026

Renaming the files ripped from Schoolhouse Rock Special 30th Anniversary Edition for Plex

So, I am going a little bit astray this time and talking about a little bash script I had to create to rename the files I ripped from Schoolhouse Rock Special 30th Anniversary Edition using MakeMKV so Plex can correctly ingest them.  It isn't much, but it is honest work.  

#!/usr/bin/env bash
#
# Rename Schoolhouse Rock! Special 30th Anniversary Edition (2002 DVD) MakeMKV rips
# into Plex-friendly filenames: "Schoolhouse Rock - S01E01 - Title.mkv"
#
# HOW TO USE:
#   1. Run with --dry-run first to see what would happen (default).
#   2. Fill in any "TBD_" entries below once you've identified them (see notes).
#   3. Run with --apply to actually rename the files.
#
# HOW TO IDENTIFY UNKNOWN FILES:
#   ffplay -ss 5 -t 8 -autoexit "DISC1/H3_t37.mkv"   # plays 8 seconds starting at 0:05
#   (or open in VLC/mpv - most of these songs are recognizable within a few seconds)

set -euo pipefail

MODE="dry-run"
if [[ "${1:-}" == "--apply" ]]; then
  MODE="apply"
fi

declare -A RENAME_MAP

# -- S01: Multiplication Rock
RENAME_MAP["DISC1/H4_t36.mkv"]="Schoolhouse Rock - S01E01 - Three Is a Magic Number.mkv"
RENAME_MAP["DISC1/H1_t34.mkv"]="Schoolhouse Rock - S01E02 - My Hero, Zero.mkv"
RENAME_MAP["DISC1/H2_t35.mkv"]="Schoolhouse Rock - S01E03 - Elementary, My Dear.mkv"
RENAME_MAP["DISC1/H3_t37.mkv"]="Schoolhouse Rock - S01E04 - The Four-Legged Zoo.mkv"
RENAME_MAP["DISC1/H5_t38.mkv"]="Schoolhouse Rock - S01E05 - Ready or Not, Here I Come.mkv"
RENAME_MAP["DISC1/H7_t39.mkv"]="Schoolhouse Rock - S01E06 - I Got Six.mkv"
RENAME_MAP["DISC1/H8_t40.mkv"]="Schoolhouse Rock - S01E07 - Lucky Seven Sampson.mkv"
RENAME_MAP["DISC1/H9_t41.mkv"]="Schoolhouse Rock - S01E08 - Figure Eight.mkv"
RENAME_MAP["DISC1/H10_t42.mkv"]="Schoolhouse Rock - S01E09 - Naughty Number Nine.mkv"
RENAME_MAP["DISC1/H6_t43.mkv"]="Schoolhouse Rock - S01E10 - The Good Eleven.mkv"
RENAME_MAP["DISC1/H11_t44.mkv"]="Schoolhouse Rock - S01E11 - Little Twelvetoes.mkv"

# -- S02: Grammar Rock
RENAME_MAP["DISC1/F7_t21.mkv"]="Schoolhouse Rock - S02E01 - A Noun Is a Person, Place or Thing.mkv"
RENAME_MAP["DISC1/F3_t20.mkv"]="Schoolhouse Rock - S02E02 - Verb - That's What's Happening.mkv"
RENAME_MAP["DISC1/F5_t17.mkv"]="Schoolhouse Rock - S02E03 - Conjunction Junction.mkv"
RENAME_MAP["DISC1/F4_t18.mkv"]="Schoolhouse Rock - S02E04 - Interjections!.mkv"
RENAME_MAP["DISC1/F1_t15.mkv"]="Schoolhouse Rock - S02E05 - Unpack Your Adjectives.mkv"
RENAME_MAP["DISC1/F9_t16.mkv"]="Schoolhouse Rock - S02E06 - Lolly, Lolly, Lolly, Get Your Adverbs Here.mkv"
RENAME_MAP["DISC1/F2_t19.mkv"]="Schoolhouse Rock - S02E07 - Rufus Xavier Sarsaparilla.mkv"
RENAME_MAP["DISC1/F8_t22.mkv"]="Schoolhouse Rock - S02E08 - Busy Prepositions.mkv"
RENAME_MAP["DISC1/F6_t23.mkv"]="Schoolhouse Rock - S02E09 - The Tale of Mr. Morton.mkv"


# -- S03: America Rock
RENAME_MAP["DISC1/E5_t04.mkv"]="Schoolhouse Rock - S03E01 - No More Kings.mkv"
RENAME_MAP["DISC1/E6_t06.mkv"]="Schoolhouse Rock - S03E02 - The Shot Heard 'Round the World.mkv"
RENAME_MAP["DISC1/E4_t07.mkv"]="Schoolhouse Rock - S03E03 - The Preamble.mkv"
RENAME_MAP["DISC1/E7_t11.mkv"]="Schoolhouse Rock - S03E04 - Sufferin' Till Suffrage.mkv"
RENAME_MAP["DISC1/E10_t12.mkv"]="Schoolhouse Rock - S03E05 - I'm Just a Bill.mkv"
RENAME_MAP["DISC1/E8_t09.mkv"]="Schoolhouse Rock - S03E06 - The Great American Melting Pot.mkv"
RENAME_MAP["DISC1/E2_t08.mkv"]="Schoolhouse Rock - S03E07 - Elbow Room.mkv"
RENAME_MAP["DISC1/E3_t05.mkv"]="Schoolhouse Rock - S03E08 - Fireworks.mkv"
RENAME_MAP["DISC1/E1_t10.mkv"]="Schoolhouse Rock - S03E09 - Mother Necessity.mkv"
RENAME_MAP["DISC1/E9_t13.mkv"]="Schoolhouse Rock - S03E10 - Three-Ring Government.mkv"
RENAME_MAP["DISC1/B6_t54.mkv"]="Schoolhouse Rock - S03E11 - I'm Gonna Send Your Vote to College.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S03E12 - Presidential Minute.mkv"

# -- S04: Science Rock
RENAME_MAP["DISC1/I5_t53.mkv"]="Schoolhouse Rock - S04E01 - A Victim of Gravity.mkv"
RENAME_MAP["DISC1/I2_t50.mkv"]="Schoolhouse Rock - S04E02 - Interplanet Janet.mkv"
RENAME_MAP["DISC1/I8_t46.mkv"]="Schoolhouse Rock - S04E03 - The Body Machine.mkv"
RENAME_MAP["DISC1/I3_t47.mkv"]="Schoolhouse Rock - S04E04 - Do the Circulation.mkv"
RENAME_MAP["DISC1/I7_t49.mkv"]="Schoolhouse Rock - S04E05 - The Energy Blues.mkv"
RENAME_MAP["DISC1/I1_t52.mkv"]="Schoolhouse Rock - S04E06 - The Not-So-Dry Bones.mkv"
RENAME_MAP["DISC1/I4_t48.mkv"]="Schoolhouse Rock - S04E07 - Electricity, Electricity.mkv"
RENAME_MAP["DISC1/I6_t51.mkv"]="Schoolhouse Rock - S04E08 - Telegraph Line.mkv"
RENAME_MAP["DISC2/N9_t04.mkv"]="Schoolhouse Rock - S04E09 - The Greatest Show on Earth.mkv"

# -- S05: Computer Rock
# RENAME_MAP[]="Schoolhouse Rock - S05E01 - Introduction - Scooter Computer And Mr Chips.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S05E02 - Hardware.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S05E03 - Software.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S05E04 - Number Cruncher.mkv"

# -- S06: Money Rock
RENAME_MAP["DISC1/G7_t25.mkv"]="Schoolhouse Rock - S06E01 - Dollars And Sense.mkv"
RENAME_MAP["DISC1/G2_t28.mkv"]="Schoolhouse Rock - S06E02 - Tax Man Max.mkv"
RENAME_MAP["DISC1/G5_t27.mkv"]="Schoolhouse Rock - S06E03 - Where the Money Goes.mkv"
RENAME_MAP["DISC1/G4_t26.mkv"]="Schoolhouse Rock - S06E04 - Making 7.50 Once a Week.mkv"
RENAME_MAP["DISC1/G3_t31.mkv"]="Schoolhouse Rock - S06E05 - Tyrannosaurus Debt.mkv"
RENAME_MAP["DISC1/G6_t30.mkv"]="Schoolhouse Rock - S06E06 - This For That.mkv"
RENAME_MAP["DISC1/G1_t29.mkv"]="Schoolhouse Rock - S06E07 - Walkin' On Wall Street.mkv"
RENAME_MAP["DISC1/G8_t32.mkv"]="Schoolhouse Rock - S06E08 - The Check's In The Mail.mkv"

# -- S07: Earth Rock
# RENAME_MAP[]="Schoolhouse Rock - S07E01 - Report from the North Pole.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S07E02 - The Little Things We Do.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S07E03 - The Trash Can Band.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S07E04 - You Oughta Be Savin' Water.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S07E05 - The Rainforest.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S07E06 - Save the Ocean.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S07E07 - FatCat Blue: The Clean Rivers Song.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S07E08 - A Tiny Urban Zoo.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S07E09 - Solar Power to the People.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S07E10 - Windy and the Windmills.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S07E11 - Don't Be a Carbon Sasquatch.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S07E12 - The 3 Rs.mkv"


# -- S08: Specials
RENAME_MAP["DISC2/Q1_t01.mkv"]="Schoolhouse Rock - S08E01 - Scooter Computer And Mr Chips.mkv"
RENAME_MAP["DISC2/Q2_t02.mkv"]="Schoolhouse Rock - S08E02 - Scooter Computer and Mr. Chips- Think Computer!.mkv"
RENAME_MAP["DISC2/Q3_t03.mkv"]="Schoolhouse Rock - S08E03 - Scooter Computer and Mr. Chips- Number Cruncher.mkv"
RENAME_MAP["DISC2/N6_t16.mkv"]="Schoolhouse Rock - S08E04 - Making of 'I'm Gonna Send Your Vote To College'.mkv"
RENAME_MAP["DISC2/N2_t22.mkv"]="Schoolhouse Rock - S08E05 - 'I'm Gonna Send Your Vote To College' In Surround.mkv"
RENAME_MAP["DISC2/N5_t17.mkv"]="Schoolhouse Rock - S08E06 - Emmy Awards Featurette.mkv"
# RENAME_MAP[]="Schoolhouse Rock - S08E07 - Nike Spot.mkv"
RENAME_MAP["DISC2/O1_t06.mkv"]="Schoolhouse Rock - S08E08 - Audio Commentary: Lolly, Lolly, Lolly Get Your Adverbs Here.mkv"
RENAME_MAP["DISC2/O2_t07.mkv"]="Schoolhouse Rock - S08E09 - Audio Commentary: Great American Melting Pot.mkv"
RENAME_MAP["DISC2/O3_t08.mkv"]="Schoolhouse Rock - S08E10 - Audio Commentary: Telegraph Line.mkv"
RENAME_MAP["DISC2/O4_t09.mkv"]="Schoolhouse Rock - S08E11 - Audio Commentary: Figure Eight.mkv"
RENAME_MAP["DISC2/O5_t10.mkv"]="Schoolhouse Rock - S08E12 - Audio Commentary: Interplanet Janet.mkv"
RENAME_MAP["DISC2/O6_t11.mkv"]="Schoolhouse Rock - S08E13 - Audio Commentary: Conjunction Junction.mkv"
RENAME_MAP["DISC2/O7_t12.mkv"]="Schoolhouse Rock - S08E14 - Audio Commentary: I'm Just A Bill.mkv"
RENAME_MAP["DISC2/O8_t13.mkv"]="Schoolhouse Rock - S08E15 - Audio Commentary: The Preamble.mkv"
RENAME_MAP["DISC2/O9_t14.mkv"]="Schoolhouse Rock - S08E16 - Audio Commentary: Interjections.mkv"
RENAME_MAP["DISC2/O10_t15.mkv"]="Schoolhouse Rock - S08E17 - Audio Commentary: A Noun is a Person, Place or Things.mkv"
RENAME_MAP["DISC2/P1_t18.mkv"]="Schoolhouse Rock - S08E18 - Deluxx Folk Implosion: I'm Just a Bill.mkv"
RENAME_MAP["DISC2/P2_t19.mkv"]="Schoolhouse Rock - S08E19 - Better Than Ezra: Conjunction Junction.mkv"
RENAME_MAP["DISC2/P3_t20.mkv"]="Schoolhouse Rock - S08E20 - The Lemonheads: My Hero, Zero.mkv"
RENAME_MAP["DISC2/P4_t21.mkv"]="Schoolhouse Rock - S08E21 - Goodness: Electricity, Electricity.mkv"

UNMAPPED=(

)

echo "=== Mode: $MODE ==="
echo

for src in "${!RENAME_MAP[@]}"; do
  dst_name="${RENAME_MAP[$src]}"
  dst_dir="$(dirname "$src")"
  dst="$dst_dir/$dst_name"
  if [[ ! -f "$src" ]]; then
    echo "SKIP (not found): $src"
    continue
  fi
  if [[ "$MODE" == "apply" ]]; then
    mv -v -- "$src" "$dst"
  else
    echo "WOULD RENAME: $src  ->  $dst"
  fi
done

echo
echo "=== ${#UNMAPPED[@]} files not yet identified (left untouched): ==="
for f in "${UNMAPPED[@]}"; do
  [[ -f "$f" ]] && echo "  $f"
done

Monday, August 17, 2026

Replacing hand-edited nginx with NixOS (and finding two dead config entries along the way)

I've been running nginx on a box I call VDO for years now - it's the reverse proxy in front of a handful of self-hosted services: VDO.ninja, a Palworld server, Vaultwarden, Leantime, and a few others that have come and gone. Like most homelab nginx setups, it grew the way these things always grow: a sites-enabled/ file added here, a Certbot cron job there, a proxy_pass tweaked at 11pm to fix something that broke. It worked. It also wasn't something I could hand to a future version of myself, let alone anyone else, and reconstruct with any confidence.

So I decided to rebuild it on NixOS, where the whole nginx + TLS setup lives in one declarative configuration.nix instead of a pile of hand-edited files and whatever Certbot happened to do to them over the years. This is the story of that migration - what worked, what broke, and what I found hiding in the config that I didn't expect.

Starting point: auditing what's actually there

Before touching anything, I pulled nginx -T off VDO to get the full, resolved configuration - every included file, flattened. That turned out to be the right first move, because it surfaced two things I would have otherwise just carried forward blindly:

  1. A whole stream {} block proxying Palworld's UDP port 8211 through nginx to the game server.
  2. A default vhost that silently fell through to my Leantime instance whenever a request didn't match any server_name - not something I'd ever consciously decided, just a side effect of vhost ordering.

Cross-referencing that against my router's port-forwarding table killed the first one immediately: Verizon Fios was port-forwarding Palworld's UDP 8211 straight to the game server's LAN IP, bypassing nginx entirely. The stream{} block had been dead code the whole time - nginx was never actually in that path. I also found a stray port-8080 forward pointing at VDO that didn't correspond to anything in the nginx config at all. A quick ss -tlnp on VDO confirmed nothing was even listening on it. Dead forward, dropped.

Small thing, but worth saying out loud: if you're about to rebuild something from scratch, audit what's actually running before you re-implement what you think is running. I would have faithfully ported over a UDP stream proxy that hadn't done anything in who knows how long.

While I was in there, I also killed the "default vhost falls through to Leantime" behavior and gave Leantime its own real server_name. The new default is a hard return 444 - connection just dies if nothing matches.

The plan: NixOS LXC, built alongside, cut over when ready

Rather than touch the live box, I built the replacement as a brand-new Proxmox LXC running NixOS, using nixos-generators to produce a proxmox-lxc-format template. Everything got built and tested side-by-side with the existing VDO nginx, with DNS still pointed at the old box the whole time. Certs for the new box came via Let's Encrypt DNS-01 challenges against Cloudflare (which already manages my DNS), specifically so the new box didn't need to be internet-facing yet to get real, working certificates. That let me curl --resolve against it and validate every vhost before DNS ever moved.

Where I got stuck (because I will absolutely forget this otherwise)

Nix's flake features aren't on by default. First run of nix run github:nix-community/nixos-generators failed with experimental Nix feature 'nix-command' is disabled. Flakes need both nix-command and flakes enabled - I'd only tried enabling one of them, and I'd also put the flag in the wrong place in the command (after the --, which routes it to the app being run, not to nix itself). Persisting it in ~/.config/nix/nix.conf made the problem go away for good.

A configuration.nix has to actually exist where you point. Obvious in hindsight, but I hit error: file 'nixos-config' was not found in the Nix search path because I hadn't actually put the file at the path I was telling nixos-generators to use yet - I'd mentally filed that as a later step and jumped ahead.

addSSL needs a certificate, even for a vhost whose whole job is dropping connections. My default catch-all vhost errored with services.nginx. virtualHosts._.sslCertificate' was accessed but has no value defined, because telling nginx to also listen on 443 means nginx needs something to present during the TLS handshake, "default" or not. Fix was pointing it at useACMEHost for one of my real certs - the same trick my old hand-written config had been quietly doing all along, I just hadn't carried it forward.

Option names aren't stable across nixpkgs versions, and the error messages are genuinely helpful about it. security.acme.defaults.credentialsFile doesn't exist on the nixpkgs revision I was pinned to - it's environmentFile now. The error output actually suggested the fix directly ("Did you mean ... environmentFile?"), which was a nice surprise after years of cryptic build tool errors elsewhere.

nixos-generators is on its way out. Mid-build I got a deprecation warning - as of NixOS 25.05, this functionality has been folded into nixos-rebuild build-image directly. It still works today and isn't going away suddenly, so I kept going rather than switch tools mid-troubleshoot, but it's worth knowing about if you're starting this today.

First boot has no PATH. Entering the fresh LXC via pct enter and running mkdir got me command not found - the shell exists, the binaries exist in the Nix store, but nothing had wired up PATH yet. export PATH="/run/current-system/sw/bin:$PATH" (or just . /etc/profile) sorted it out.

None of these were hard once I knew what they were. All of them would have been mildly maddening to hit cold.

Sizing the box

For resources, I ended up on 2 vCPUs, 4GB RAM, no swap (disk is on NVMe, so no speed argument for swap - but I bumped RAM up specifically because I dropped swap, since without it there's no cushion if a nixos-rebuild evaluation spikes), and 16GB of disk. That last number isn't about nginx's footprint, which is tiny - it's about giving the Nix store room to hold a few generations before garbage collection kicks in. I added nix.gc to the config to run weekly and prune anything older than 30 days, so that stays bounded going forward instead of creeping.

Where it landed

nixos-rebuild switch on the new box completed clean - no errors, just the expected (and harmless) warning about /boot not existing, which is just NixOS's tooling checking for a bootloader partition that an LXC has no concept of. From there, curl --resolve against every vhost on the new box returned the same content as the equivalent request against the live VDO. Cutover is just a DNS change away at this point.

What I'd tell past-me

  • Pull the real, resolved config (nginx -T, not just the files you remember editing) before you start porting anything. You will find dead config you forgot about.
  • Cross-reference against what's actually reachable from the outside - your router's port-forwarding table and ss -tlnp on the box will tell you the truth faster than the app config will.
  • Build the replacement in parallel and validate with DNS-01 + --resolve before ever touching DNS. There's no reason to take the old thing down to test the new thing.
  • NixOS's error messages, when they involve a renamed option, will usually just tell you the new name. Read the whole error before reaching for a search engine.

Friday, August 14, 2026

Create a live Linux USB drive to get started with Linux

 

  1. Download a LiveCD from the distribution you choose to try. I am keen on openSUSE so I go to https://get.opensuse.org/tumbleweed/?type=desktop#download and download the KDE LiveCD


  2. Download balenaEtcher from https://etcher.balena.io/


  3. Launch balenaEtcher and choose the ISO and USB disk and click the Flash! button 

  4. Boot from the USB!

Wiring Up Zabbix in the Homelab: Geomaps, Broken Repos, and the 7.4 Auth Change That Bit Me

I've been rolling Zabbix into the homelab for monitoring, and like most homelab projects, "install the agent and watch the dashboard fill up" turned into an afternoon of chasing three separate problems. None of them were hard once I understood what was actually happening, but none of them were obvious from the error messages either. Writing this up mostly for future-me, but hopefully it saves you a step or two.

Problem 1: I Didn't Want to Hand-Enter Lat/Long for Every Host

Zabbix's Geomap dashboard widget is genuinely nice once it's populated - it plots your hosts on an actual map using the location_lat and location_lon inventory fields. The catch is that nothing populates those fields for you. Add a new host, and it just doesn't show up on the map until you go type in coordinates by hand.

For a homelab where most of my boxes live in the same rack, that's annoying busywork. What I wanted was a template-level default: any host using the template gets a fallback pin, and I can override it later for anything that actually moves (laptops, remote nodes, whatever).

The trick is combining user macros with script items:

Step 1 - Define the macros on the template.

{$DEFAULT_LAT} → 40.6892
{$DEFAULT_LON} → -74.0467

(Coordinates are decimal degrees - latitude -90 to 90, longitude -180 to 180.)

Step 2 - Create a script item for latitude.

  • Name/Key: Default Latitude / default.latitude
  • Type: Script
  • Type of information: Numeric (float)
  • Update interval: 1d - it's static data, no reason to poll it constantly
  • Script:
return '{$DEFAULT_LAT}';
  • Populates host inventory field: Location latitude

Step 3 - Repeat for longitude, swapping in {$DEFAULT_LON} and mapping to Location longitude.

Step 4 - the part that'll trip you up: the script items only get to overwrite inventory data if the host's Inventory mode is set to Automatic. If it's left on Manual or Disabled, the item runs fine, collects the value, and just... doesn't write it anywhere. No error, no warning, it just silently doesn't work. I burned more time than I'd like to admit on this before checking the inventory mode.

One small bonus while I was in the Geomap widget: if you're tired of the map resetting to some default zoom level every time the dashboard loads, pan/zoom to the view you actually want, right-click the map, and pick "Set this view as default." Small thing, but it stuck with me.

Problem 2: Agent Install Failed on Dependencies

Next up, installing zabbix-agent2 on an OpenMediaVault VM (which runs on Proxmox in my case) blew up with:

Depends: libc6 (>= 2.38) but 2.36-9+deb12u9 is to be installed
Depends: libssl3t64 (>= 3.0.13) but it is not installable

The root cause was embarrassingly simple once I looked at it: the VM is on Debian 12 (Bookworm), which tops out at libc6 2.36. I'd grabbed the Zabbix repo package built for Debian 13 (Trixie) / Ubuntu 24.04, which expects newer system libraries that Bookworm doesn't have and isn't going to get. Classic case of downloading the wrong release artifact and not noticing until apt complained.

Fix was a clean purge-and-reinstall with the correct release package:

Purge the bad repo, keys, and cache:

sudo rm -f /etc/apt/sources.list.d/zabbix.list
sudo rm -f /etc/apt/trusted.gpg.d/zabbix*
sudo rm -f /etc/apt/keyrings/zabbix*
sudo apt-get clean

Install the Debian 12–correct release package, then refresh and install:

sudo dpkg -i zabbix-release_latest_7.0+debian12_all.deb
sudo apt update
sudo apt install zabbix-agent2 zabbix-agent2-plugin-*
sudo systemctl enable --now zabbix-agent2

Lesson: always double-check the release package matches the actual OS version on the box, not the OS version you assume it's running. This is exactly the kind of gap that's easy to hit when you're juggling a handful of VMs with slightly different base images.

Problem 3: The Zabbix 7.4 API Auth Change Nobody Warns You About

This one cost me the most confusion. I wanted to query the API directly (Postman/cURL) to pull inventory data, and ran into two separate issues stacked on top of each other.

First: a 404 on the endpoint. Most guides show the API living at /zabbix/api_jsonrpc.php, but depending on how you installed (this applies to a lot of default and Docker-based setups), the frontend is served straight from the domain root. If you're getting a flat 404, check whether your install actually uses the /zabbix/ subdirectory at all before you go further down a rabbit hole.

Second, and the real gotcha: Zabbix 7.4 changed how authentication works. The traditional pattern - sticking "auth": "<token>" inside the JSON-RPC body - is exactly what every older tutorial shows you, and it's exactly what 7.4 now rejects:

{"jsonrpc":"2.0","error":{"code":-32600,"message":"Invalid request.","data":"Invalid parameter \"/\": unexpected parameter \"auth\"."},"id":1}

The error message itself doesn't really tell you why - it just says the parameter is unexpected, which reads like a typo, not a breaking API change. As of 7.4, the API key has to be passed as a standard HTTP header instead of inside the request body:

Authorization: Bearer <your_actual_api_key_here>

And the JSON body gets simplified back down to just the method call:

{
    "jsonrpc": "2.0",
    "method": "host.get",
    "params": {
        "output": ["hostid", "name"],
        "limit": 5
    },
    "id": 1
}

If you're following an older blog post or Postman collection and getting this error, this is almost certainly why - check the Zabbix version before assuming your payload is malformed.

Problem 4: Inventory Fields Come Back Empty

Once the auth was sorted, the API calls worked - but host.get with inventory fields requested (os, software, os_full, etc.) came back mostly blank, except for os, which was a raw, ugly kernel string like Linux version 6.17.2-2-pve.... Not exactly what I wanted for tracking which boxes need OS updates.

Turns out this is by design, not a bug. Zabbix defaults new hosts to Inventory mode: Disabled. Even when a template - like the stock "Linux by Zabbix agent active" template - is actively collecting an item that could populate inventory (system.sw.os in this case), the item's "Populates host inventory field" setting ships blank. Zabbix apparently treats this as a deliberate opt-in, presumably to avoid inventory-table overhead at scale, but for a homelab it just means everything's empty until you flip a few switches.

Three things fixed it:

1. Set the default going forward: Administration > General > Other > Default host inventory mode → Automatic

2. Backfill existing hosts: Data collection > Hosts → select the hosts → Mass update → Inventory tab → set mode to Automatic

3. Actually map the item to a field: Data collection > Templates > Linux by Zabbix agent active > Items → find Operating system (system.sw.os) → set Populates host inventory field to OS (or OS (Full)) → Update

If you want cleaner data than the raw kernel string - something like Ubuntu 24.04 LTS instead of the full uname output - you can add a custom item that regexes it straight out of /etc/os-release:

Key: vfs.file.regexp[/etc/os-release,"PRETTY_NAME=(.*)",,,,\1]
Inventory Field: OS (Full)

Wrap-Up

None of these were individually hard problems, but each one had an unhelpful or misleading surface-level symptom: a silent no-op on the Geomap fields, a dependency error that pointed at the wrong root cause, an auth error that read like a bad payload instead of a breaking version change, and empty inventory that looked like a bug rather than a default setting. If you're standing up Zabbix 7.4 fresh, checking inventory mode and the auth header format up front will save you the loop I went through.

Thursday, August 6, 2026

Installing VMware Workstation Pro on openSUSE: A Battle Report

I just wrapped up the VMware Workstation Pro chapter for the book I'm writing, and it was, without question, the longest and most frustrating install I've documented so far. Long enough that I figured it deserved its own post here, separate from the step-by-step version - this one's more about what actually went wrong and what I learned fighting through it.

Why bother with VMware at all?

VMware Workstation Pro is a paid, proprietary hypervisor, which puts it in an odd spot on Linux where free options like KVM/QEMU and libvirt already do most of the job. But Workstation Pro earns its keep with tight kernel integration, a genuinely polished GUI, snapshot/clone workflows that make experimentation cheap, OVF/OVA export for sharing VMs, and support for nested virtualization - handy if you want to run Docker or Kubernetes inside a guest. On openSUSE Tumbleweed specifically, none of that comes easy.

The install itself: death by a thousand dependencies

The first sign this wasn't going to be a zypper install afternoon was the dependency list. VMware needs kernel headers and a C++ toolchain to build its kernel modules against your running kernel - fine, expected. But it also wants a handful of specific library versions, and openSUSE Tumbleweed, being a rolling release, is usually ahead of what VMware's installer expects. In my case, that meant faking an older libxml2 with a symlink:

sudo ln -s /usr/lib64/libxml2.so.16 /usr/lib64/libxml2.so.2
export VMWARE_USE_SHIPPED_LIBS='yes'

That second line matters as much as the symlink - it tells VMware to fall back to the libraries it ships internally rather than fighting your system's newer versions. Without both pieces together, the installer gets confused in ways that are genuinely hard to diagnose from the error messages alone.

The installer finishes... and then quietly fails anyway

Here's the part that really tested my patience. After working through the download (Broadcom now gatekeeps VMware downloads behind a login, a whole separate minor annoyance), making the bundle executable, and running the installer - accepting license agreements, telemetry prompts, all the usual dance - the first launch of VMware itself failed. Not with a helpful error, but with this buried in the logs:

[Apploader] Cannot get library dependencies. (10c)
[AppLoader] Fallback to use all shipped libraries.

The fix wasn't in any VMware documentation I could find. It came down to manually running the setup helper script the installer should have run correctly on its own:

sudo /usr/lib/vmware/bin/vmware-setup-helper -e -o -u yes -c yes

After that, VMware launched cleanly. I want to be honest about how I found this - it wasn't buried deep in a forum thread after hours of searching, it was AI-assisted troubleshooting that got me there fast. I've said this before and I'll say it again here: for these obscure, distro-specific integration failures, an AI that can reason through error messages and suggest specific diagnostic commands is genuinely one of the best tools in your kit now. I don't think I'd have found that exact invocation on my own nearly as quickly.

And then the hardware plot twist

Just when I thought I was through it, I hit a wall that no amount of troubleshooting could fix: my CPU. I'm running an Intel Xeon X5670 in this box, and it turns out every VMware Workstation Pro release after 17.0.2 requires XSAVE instruction set support - which this chip simply doesn't have. Not a driver issue, not a config issue. The silicon itself doesn't support it.

That leaves two options, neither of which is great: switch to different hardware, or downgrade to 17.0.2. And the downgrade path isn't clean either - the kernel modules VMware 17.0.2 expects were built against older kernel APIs, and Tumbleweed's rolling kernel has moved well past that point. So it becomes its own compatibility puzzle, layered on top of everything else.

What I'd tell someone else attempting this

A few honest takeaways, in the spirit of documenting the failures alongside the wins:

  • Take VM snapshots liberally while you work through this. I can't overstate how much easier this made recovering from bad states along the way. Without that safety net, this install would have been genuinely miserable rather than just long.
  • Check your CPU's instruction set support before you start, not after. lscpu will tell you what you're working with - if XSAVE isn't listed and you're on older enterprise-grade hardware like mine, save yourself the afternoon and either plan on 17.0.2 from the start or reconsider the hardware.
  • Rolling-release distros and vendor installers built for "stable" Linux baselines are going to fight each other. This isn't the first time I've hit this in this book - Docker Desktop and a couple of other tools have had their own version of this same friction on Tumbleweed. If you're on a rolling release, budget extra time for exactly this kind of gap.
  • Don't be afraid to manually run what the installer should have run for you. The vmware-setup-helper fix wasn't documented anywhere obvious, but the installer's own failure message was specific enough to point toward it once I stopped assuming the GUI installer was the only supported path.

The full step-by-step version of this - with every screenshot and command in sequence - will be in the book. This post is more the "here's what it actually felt like" version. If you're fighting the same install right now: it does end, and it does eventually work. Good luck.

Friday, July 31, 2026

Setting Up a Palworld Dedicated Server (Or: Everything I Got Wrong Along the Way)

I run a homelab, I've got a Proxmox cluster with more hardware than any reasonable person needs, and my kid wanted a Palworld server. This felt like an easy weekend project. It was not an easy weekend project. But it was a genuinely useful one, because almost every mistake I made along the way was the kind of mistake that teaches you something about how your own infrastructure actually behaves - not just how you assume it behaves.

This post is the honest version of that journey, bugs and bad assumptions included.

Step one: SteamCMD doesn't just install on Debian 13

First stumble, and it happened before I'd even touched Palworld. apt install steamcmd on a fresh Debian 13 (trixie) box just fails - no candidate. SteamCMD lives in the non-free component, and it's also a 32-bit binary, so you need i386 architecture support enabled on what's otherwise a clean 64-bit system:

sudo dpkg --add-architecture i386
sudo sed -i '/^Components:/ s/$/ contrib non-free non-free-firmware/' /etc/apt/sources.list.d/debian.sources
sudo apt update
sudo apt install steamcmd

I initially tried chasing this by pointing apt at unstable, which is exactly the wrong move - trixie doesn't have an unstable suite to pull from, and all that did was generate a pile of warnings and waste twenty minutes. Lesson: when a package "doesn't exist," check which component it lives in before you start reaching for bigger hammers.

Auto-starting the server: systemd, not sysctl

Small but worth naming, because I hear people conflate these constantly (I did too, once, years ago): sysctl is for kernel parameters. What you want for "run this on boot and restart it if it dies" is a systemd unit:

[Unit]
Description=Palworld Dedicated Server
After=network.target

[Service]
Type=simple
User=palworld
Group=palworld
WorkingDirectory=/home/palworld/.local/share/Steam/steamapps/common/PalServer
ExecStart=/home/palworld/.local/share/Steam/steamapps/common/PalServer/PalServer.sh
Restart=on-failure
RestartSec=10
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target

That LimitNOFILE bump matters - Palworld servers are known to want a higher open-file limit than the systemd default gives you.

The SMB mount that wasn't allowed to happen

I wanted to pull some save files in from a Samba share on my Proxmox host. Straightforward, I thought - install cifs-utils, mount the share, done. Except every mount attempt came back permission denied, even though smbclient proved the credentials were completely correct.

The tell was that dmesg itself failed with "Operation not permitted," even under sudo. That's the fingerprint of an unprivileged LXC container - it doesn't get the kernel capabilities mount.cifs needs, full stop, no matter how correct your credentials are. No amount of fiddling with vers=, sec=ntlmssp, or credentials file permissions was ever going to fix it, because the problem wasn't the mount command - it was the container.

The right fix was to stop trying to mount inside the container entirely: mount the CIFS share on the Proxmox host, then bind it into the container as a mount point:

# on the host
mount -t cifs //192.168.1.12/public/shared/PalWorld /mnt/palworld-share -o credentials=/etc/samba/creds-palworld

# then
pct set 100 -mp0 /mnt/palworld-share,mp=/mnt/palworld-share
pct reboot 100

No capability loosening on the container needed. This is one of those things where the actual fix is almost embarrassingly simple once you know it, but the error message gives you zero hint that "unprivileged container" is the root cause.

The UDP odyssey (this is the big one)

This is where most of my week went, and it's the part I want to actually walk through honestly, because I got it wrong more than once.

I only have one public IP and one exposed host through my DMZ, so I front everything with nginx and route by hostname. That works great for HTTPS - SNI does the routing for you. It does not work for a raw UDP game port, because there's no hostname to route on at the UDP layer. Palworld talks straight to an IP:port.

nginx's stream module handles this - it's a completely separate config block from the http one, living at the top level of nginx.conf, not inside it:

stream {
    upstream palworld_udp {
        server 192.168.1.90:8211;
    }

    server {
        listen 8211 udp;
        proxy_pass palworld_udp;
        proxy_timeout 60s;
    }
}

I got this working internally almost immediately. Externally, nothing. Classic "works on my machine" - except "my machine" was my own LAN, which is a much easier test than I gave it credit for.

Mistake #1: proxy_timeout 1s. My first pass at this config had a one-second timeout on the UDP proxy session. Fine on a LAN with near-zero latency. Fatal over the real internet, where a one-second gap between packets during the connection handshake is completely normal - nginx would tear the session down before the handshake even finished. Bumped it to 60s. Progress, but still not fixed.

Mistake #2, and the one that actually embarrasses me a little: proxy_responses 0. I'd set this thinking it was a reasonable default. It is not. That directive tells nginx's UDP proxy to not relay responses back to the client - it's meant for fire-and-forget protocols like syslog, where nothing is expected in return. Palworld is a request/response protocol through and through. With that setting in place, nginx would faithfully forward the client's packet to the game server, receive the game server's reply... and just drop it on the floor instead of sending it back out. I confirmed this with tcpdump - three lines of traffic (client in, forward to backend, reply from backend) and then silence where a fourth line should have been.

stream {
    upstream palworld_udp {
        server 192.168.1.90:8211;
    }
    server {
        listen 8211 udp;
        proxy_pass palworld_udp;
        proxy_timeout 60s;
        # no proxy_responses line - the default relays replies, which is what you want
    }
}

Once that was gone, the round trip actually completed. tcpdump is not optional equipment for this kind of debugging - without watching the packets literally stop at one specific hop, I'd have kept guessing at the router.

And a red herring worth naming, because it wasted real time: online "open port checker" tools are TCP-only. Running one against my UDP game port and seeing "closed" told me nothing useful - it wasn't even testing the right protocol. If you're debugging UDP, those tools are close to useless; tcpdump while attempting a real connection is the only test that actually tells you the truth.

The REST API and the setting that wouldn't stick

Palworld ships a REST admin API - player counts, kick/ban, clean shutdown, all over HTTP on port 8212. I wanted it for a simple status page. Enabling it should be a one-line ini edit:

RESTAPIEnabled=True
RESTAPIPort=8212
AdminPassword="something-reasonably-long"

I set it. Restarted the service. Confirmed the file was correct with grep. Still got Unauthorized (AdminPassword is empty) on every request, including with a perfectly valid password. Confirmed the ini's contents byte-for-byte with xxd in case of some invisible character - clean. No duplicate keys. No parse errors. The server was, somehow, just not reading a setting that was sitting right there in the file it had definitely loaded.

Turns out there's a second file: WorldOption.sav. Once a Palworld world has been generated, the server starts reading most world settings - apparently including AdminPassword - from this binary save file instead of the ini, and it silently overrides anything you put in PalWorldSettings.ini from that point forward. The ini edit was completely correct. It just wasn't being consulted anymore.

The fix: stop the server, delete WorldOption.sav, restart. That forces one clean read from the ini, and the server regenerates a fresh .sav from whatever's actually in the ini at that point. Worth knowing this isn't a permanent fix so much as a "you get exactly one clean boot" fix - change AdminPassword again later, and you'll be deleting that file again.

I would not have found this without just searching for the literal error message alongside "PalWorldSettings.ini" - sometimes that's still the fastest path, even after you've checked everything that seems checkable.

DMZ vs. port forwarding - the belated correction

I'd been running everything through my router's DMZ setting, pointed at my nginx box. It works, but it means every port on that box is exposed, whether anything's listening there or not. Once I had the actual list of ports something was really using - 80, 443, and 8211/udp - switching to explicit port-forward rules and turning DMZ off was a five-minute change that meaningfully shrinks what's actually reachable from the internet. I should have done this from day one instead of reaching for DMZ because it was the first thing I found.

The last gotcha: testing over cellular lies to you

Even after all of the above was fixed, I still saw intermittent "connection timed out" while testing from my phone's hotspot - mid-session, after successfully connecting and starting to build a character. Packet capture showed the server responding just fine, continuously, into total silence from the client side.

This one wasn't my infrastructure at all - it's carrier-grade NAT on the mobile network expiring the connection's NAT mapping during a quiet stretch (like sitting on a character-creation screen not sending anything). The server has no way to know the client's public-side mapping just vanished underneath it.

Confirming this took setting up a free WireGuard tunnel through an Oracle Cloud always-free instance and routing my test traffic through that instead of the cellular hotspot. WireGuard's PersistentKeepalive keeps its own NAT mapping alive regardless of game activity, which sidesteps the whole problem. Once I tested through the tunnel, everything was rock solid - confirming the server-side setup had been correct for a while, and cellular hotspots are just an unreliable way to validate a UDP game server.

What I'd tell past-me

  • Check which apt component a package lives in before assuming it's missing entirely.
  • sysctl and systemd are not the same thing, and confusing them wastes your own time.
  • Unprivileged LXC containers can't mount CIFS shares directly - mount on the host, bind into the container.
  • For UDP proxying: watch the packets with tcpdump, don't guess. TCP-only port checkers will actively mislead you.
  • proxy_responses 0 is not a safe default for anything request/response-shaped.
  • If a config value looks right in the file but the app insists otherwise, check whether there's a second source of truth (a save file, a cache, a database) that's silently taking precedence.
  • Test UDP game servers from a real home connection, not a cellular hotspot - carrier NAT will produce failures that have nothing to do with your setup.

None of these were exotic problems. Every single one had a boring, findable explanation once I stopped assuming and started watching the actual traffic. That's most of what debugging is, honestly - it's just occasionally really annoying to remember in the moment.