From Trust to Proof

Last December I wrote about how my homelab had grown from a playground into something I actually depend on. The theme was trust — trust that my services would be there, trust that my data was safe, trust that I could stop babysitting infrastructure and enjoy the results of it.

Rereading that post a year later, I notice what I left out. Trust isn't something you decide to have. It's something you earn with evidence, and at the end of 2025 I didn't have much. I had a stack that felt reliable. I couldn't prove it was.

So 2026 was the year I went looking for proof.

Here's the part that surprised me: I didn't buy anything. Not a single piece of hardware changed all year. Same gateway, same switch, same mini PC, same NAS. The rack looks identical to the photo on last year's post, and underneath it's almost unrecognizable.

Visibility: The Year I Stopped Guessing

I work in security, and it still took me embarrassingly long to take my own logs seriously. In January I finally stood up Graylog and started shipping everything into it — the reverse proxy, the firewall, the NAS, every Linux host in the lab.

The first month was mostly humbling.

I found things that had been quietly broken for months. Not down — broken. That distinction turned out to be the whole point of the exercise. A dead service trips a monitor and wakes you up at 2am. A service that returns a perfectly valid response several seconds late, every single time, will sit there for as long as you let it. One of mine did exactly that, on this very blog, while my uptime monitor stayed a cheerful green the entire time.

I also found an alerting channel that hadn't delivered a message in months. I'd built it, tested it, watched a test notification land on my phone, and never checked it again. A network change downstream had quietly severed it. Every alert since had been shouting into a closed socket.

Metrics came next — Prometheus for the numbers, Grafana for the picture, and the exporters pushed out across the fleet by Ansible instead of installed by hand one box at a time. Uptime Kuma stayed on for the simple "is it answering" checks, because not everything needs a dashboard.

The real work wasn't installing any of that. It was normalization. Every log source names things differently: the client IP is one field name in the proxy logs, another in the firewall's, another again on the NAS. Until I forced all of them into a common vocabulary, I had four separate search engines that happened to share a login page. Afterward I had one place where a single query spans the proxy, the firewall, the NAS, and every Linux host at once.

That find-and-replace work is the least interesting thing I did all year and probably the most valuable.

Segmentation: Default Deny

The second half of the year went to the network, and it started with an audit I didn't enjoy.

My VLANs had been, if I'm honest, mostly decorative. Traffic was tagged and separated on paper, but almost everything could still reach almost everything else. I'd built the walls and left every door propped open.

So I redesigned it around zones and rebuilt the policy from a default of no. Services don't get general internet access anymore — each host gets a narrow allowlist for exactly the destinations it actually needs, and nothing else. Cameras and IoT devices live somewhere they can't reach anything interesting. Admin interfaces sit behind single sign-on rather than whatever login screen the app shipped with. Encrypted DNS is blocked at the edge, because filtering that a device can trivially bypass isn't filtering.

Two lessons from that project that I'd hand to anyone doing the same thing.

A dropped packet doesn't look like an error. It looks like slowness. Every single time I tightened a rule too far, the symptom wasn't a connection refused message in a log somewhere. It was an app that hung, returned an empty result, and looked like a bug in the app. I chased three of those before I recognized the pattern. If a service starts behaving like it's slow or empty right after you touch the firewall, go look at the firewall.

Verify what's actually on a network before you quarantine it. During the audit I found a VLAN I was confident was unused, checked the client list, saw nothing, and very nearly locked it down. It wasn't empty. My own network gear was managed through it. Clients and infrastructure are two different lists, and I only looked at one.

Automation: Do It Once, Then Stop Doing It

Ansible was on last year's list, which was generous of me, because it wasn't really doing anything. This year it became the thing that actually runs the place.

There's a base playbook every host gets — hardening, keys, log shipping, the monitoring agent — so a new container inherits the whole standard on day one instead of whenever I remember. Patching runs weekly on a schedule and pushes the result to my phone whether it succeeded or not. SSH keys rotate on a timer, with the rotation built to abort safely and leave both keys working rather than lock me out of my own fleet. My container images build on a self-hosted Git server with CI, so deploying is a pull and a script instead of me typing commands from memory at 11pm.

The one that earned its place was post-upgrade verification. A routine package upgrade replaced a symlink and stranded a service's plugins — every dashboard went blank at once, and nothing anywhere logged so much as a warning. It was a clean, silent, total failure. Now a playbook checks package integrity across the fleet after every patch run and tells me when something doesn't match.

That's become the operating rule for the lab: if a failure can happen silently, something has to go look for it on purpose. Nothing gets to depend on me noticing.

Off-site backup also finally stopped being a to-do item. Versioned backups now run to cloud storage on a schedule, and I've actually tested a restore, which is the half of "having backups" that everyone skips including me, for about three years.

What I Chose Not to Build

A short list, because it's as much a part of the year as the rest.

I seriously evaluated Home Assistant, AdGuard Home, Paperless, and Immich, and installed none of them. Not because they're bad — they're all excellent. I don't own enough smart home gear to justify the first, DNS filtering is already handled upstream, I don't generate enough paper for the third, and my photos are already somewhere I trust. I also decommissioned two boxes that existed purely because I'd built them once and never questioned them since.

2025 me would have installed all four in a weekend just to see. Turning things down is a feature I had to learn.

Services I Run on Proxmox

A mix of containers and a single VM, covering both infrastructure and everyday life:

  • Ansible – automation and fleet configuration
  • Docker – containerization
  • Gitea – self-hosted Git and CI runners
  • Ghost – hosting this blog
  • Grafana – metrics dashboards
  • Graylog – log collection, parsing, and alerting
  • Jellyfin – media server and client
  • Keycloak – single sign-on for admin interfaces
  • Mealie – recipe management
  • MCP servers – tool endpoints for Claude Code, which got its own post
  • Nginx – reverse proxy for self-hosted apps
  • ntfy – push notifications
  • Ollama – local LLM inference
  • Prometheus – metrics collection
  • Prowlarr – indexer management
  • Radarr – movie management
  • Sonarr – TV show management
  • qBittorrent – media downloads
  • Uptime Kuma – uptime monitoring
  • Vaultwarden – password and secrets management

Plus a scheduled job on the NAS that pulls family photos into the main library every night. It is not remotely impressive engineering and it is the single most appreciated thing I have ever built.

Current Hardware

  • Gateway / Firewall: UniFi Dream Machine Pro Max
  • Switch: UniFi USW Pro XG 10 PoE
  • Access Point: UniFi U7 Pro XGS
  • Proxmox Server: Geekom IT11 Mini PC (64 GB DDR4)
  • Storage: Synology DS925+ and DS218+
  • UPS: APC Back-UPS 1500

That's the same list as last year. That's the point.

Closing Thoughts

Last year I said my homelab was about trust. I still think that's right, but I'd add something to it now.

Trust without evidence is just optimism with a rack. Every genuinely bad surprise I had this year was something that had been broken for a long time in a way nothing was set up to notice — a request that was slow but successful, an alert channel that had gone quiet, a plugin directory that quietly emptied itself during an upgrade. None of it showed up as an outage. All of it showed up the moment I built something whose job was to look.

The lab is quieter than it was a year ago. Fewer things to poke at, more things that just run. I still tinker, but now most of my evenings go to asking better questions of a system that can finally answer them, rather than logging into six boxes to find out whether something is on fire.

That's not the upgrade I expected to be writing about. It's the one that mattered.