Hermes SEG v260731 is out. It fixes a fault that had been quietly disabling four scheduled background jobs. And, awkwardly, the update checker that would have told you about it.
What went wrong
The file that tells Hermes SEG’s scheduler which jobs to run is generated from the database by the admin console. It was also, mistakenly, committed to our repository as a fixed copy taken from one machine, and Docker mounts that copy directly over the live file.
Nothing regenerated it during installation, and the upgrade process overwrote the live file with the repository copy on every run. A system that had accidentally repaired itself (saving any SPF, DKIM, DMARC or certificate page triggers a regeneration) was silently returned to the broken state at its next upgrade.
That frozen copy predated four of the ten scheduled jobs, and still pointed the update check at a script we’d removed several releases earlier.
The impact
On affected systems, these never ran:
- ClamAV third-party malware signature refresh
- DMARC aggregate report generation
- Mail queue health alerts
- Authelia log rotation
Core mail flow was unaffected throughout: filtering, spam and virus scanning against ClamAV’s official feed, encryption and delivery all continued working.
The update checker was the compounding problem. With it dead, affected systems couldn’t learn that new releases existed, and no notification e-mails went out. Four releases shipped that many operators never heard about.
Why it took us so long to notice
Our development server is updated file-by-file rather than through the release process, so it never performed a fresh install and never ran the upgrade path that caused the overwrite. The one machine we looked at every day was the one machine structurally incapable of showing the fault.
Our automated health check didn’t catch it either. It counted scheduled jobs in the database rather than checking what the scheduler had actually loaded, so it reported “all critical checks passed” on a system with four dead jobs.
What’s changed
- The schedule is now generated, not shipped, rendered from the database during installation and during every upgrade, which permanently closes the overwrite problem.
- The version number is derived from the installed release rather than hardcoded, so it can’t drift.
- The health check now fails loudly when the scheduled jobs and the database disagree, naming the jobs that aren’t running, and verifies the update check points somewhere real.
- The dashboard now distinguishes “we haven’t checked yet” from “the checker is broken”, with an explanation and a Check now action.
What to do
cd /path/to/hermes-seg
sudo ./scripts/system_update_docker.shOne run applies every release you’ve missed.
If your system has been showing UPDATE CHECK PENDING, this is why, and this update fixes it.