Mediabot’s Debian 13 readiness work has now crossed the systemd boundary all the way to a real runtime validation.
MB684 proved the fresh configuration/bootstrap path.
MB685 proved fresh MariaDB installation.
MB686 proved the stable 3.3 → current database migration path.
MB687 makes the published systemd service contract installable, testable, fail-closed — and validates it on the real development service through restart, database connectivity, IRC login and channel joins.
MB687 adds:
install/systemd_install.sh
The helper installs Mediabot’s published multi-instance systemd template and optionally creates an instance environment file.
It is intentionally conservative:
identical existing file
→ accepted
different existing file
→ refused
different file + explicit replacement
→ backup + controlled replacement
symlink target
→ refused
Supported replacement controls include:
--replace-template
--replace-instance
--template-only
The installer does not implicitly start, restart, stop, or enable Mediabot.
When live systemd files change, its automatic systemd action is limited to:
systemctl daemon-reload
Operational activation remains an administrator decision.
The published template carries the lifecycle previously proven through the MB645 development-instance override:
ExitType=cgroup
Environment=MEDIABOT_SYSTEMD_UPDATE_SAFE=1
Restart=always
SuccessExitStatus=75
RestartPreventExitStatus=75
RestartSec=10s
This allows updater work to remain inside the service cgroup while still preserving an explicit final shutdown through exit status 75.
Because ExitType=cgroup requires systemd 250 or newer, the installer refuses unsupported live systemd versions instead of pretending the lifecycle is portable.
MB687 adds:
t/cases/889_mb687_debian13_systemd_install_gate.t
The Debian 13 gate exercises the real installer against an isolated filesystem tree and validates:
systemd-analyze verify.Validation completed successfully:
Targeted : 218 / 218
Fast : 5992 / 5992
Full : 15605 / 15605
Commit:
596ab5e 🛠️ Teach Debian 13 the Systemd Installation Charm
Version:
3.4dev-20260822_101942
Before changing the live system, the production host showed two different effective contracts.
The shared template still used the old lifecycle:
Restart=on-failure
ExitType=main
The development instance had already received the MB645 lifecycle through:
/etc/systemd/system/mediabot@dev.service.d/mb645-test.conf
while the Undernet instance had no override and therefore still used the legacy shared template.
That meant replacing the shared template could not honestly be called a “dev-only” change.
The live rollout therefore preserved Undernet’s old lifecycle explicitly before replacing the shared template.
A temporary compatibility drop-in was installed for Undernet:
/etc/systemd/system/mediabot@undernet.service.d/mb687-legacy-preserve.conf
It preserves:
ExitType=main
Restart=on-failure
SuccessExitStatus=
RestartPreventExitStatus=
UnsetEnvironment=MEDIABOT_SYSTEMD_UPDATE_SAFE
Only after that protection was active and verified did the rollout install the new shared template.
The obsolete development-only MB645 test drop-in was then removed.
The first live rollout attempt stopped because a validation rule incorrectly treated the merged Environment= property as the final process environment.
The script failed closed and automatically restored:
No bot was inadvertently restarted.
The corrected V2 rollout then completed successfully.
This means MB687 did not only prove its happy path: the rollback path was exercised on the real machine before the successful deployment.
Before migration:
DEV PID : 3626224
Undernet PID : 2899306
After installing the shared template and removing the obsolete development drop-in, neither process had restarted.
DEV alone was then explicitly restarted:
DEV PID before : 3626224
DEV PID after : 3664795
Undernet remained:
Undernet PID : 2899306
unchanged throughout the operation.
Final DEV systemd contract:
ExitType=cgroup
Restart=always
SuccessExitStatus=75
RestartPreventExitStatus=75
Environment=MEDIABOT_SYSTEMD_UPDATE_SAFE=1
DropInPaths=
ActiveState=active
SubState=running
Final Undernet systemd contract:
ExitType=main
Restart=on-failure
SuccessExitStatus=
RestartPreventExitStatus=
Environment=MEDIABOT_SYSTEMD_UPDATE_SAFE=1
UnsetEnvironment=MEDIABOT_SYSTEMD_UPDATE_SAFE
DropInPaths=/etc/systemd/system/mediabot@undernet.service.d/mb687-legacy-preserve.conf
ActiveState=active
SubState=running
The final smoke was performed several minutes after the restart.
Systemd remained stable:
MainPID=3664795
NRestarts=0
ExecMainStatus=0
ActiveState=active
SubState=running
DropInPaths=
The process was the expected Mediabot runtime:
/usr/bin/perl /home/mediabot/mediabot_v3/mediabot.pl
--conf=/home/mediabot/mediabot_v3/mediabot.conf
The process had live TCP connections to:
127.0.0.1:3306
for MariaDB, and:
[::1]:15432
for the local IRC endpoint.
The runtime logs confirmed the full application startup path:
Achievements database persistence enabled
Metrics endpoint listening on 127.0.0.1:9108
Schedulers registered
Trying to connect to localhost:15432
Connected to irc server localhost
Logging to x@channels.undernet.org
Joining console channel #teuk
Now talking in #teuk
Now talking in #quebec
Now talking in #gabou
Now talking in #radiocapsule
Now talking in #test
Now talking in #miaw
Now talking in #boulets
The fatal/runtime error scan returned:
OK: no obvious fatal/runtime connection error
After the DEV smoke:
Undernet PID : 2899306
was still unchanged.
The running Undernet process was also inspected directly through /proc/<PID>/environ.
Result:
OK: running Undernet process does NOT have
MEDIABOT_SYSTEMD_UPDATE_SAFE=1
So the temporary compatibility drop-in is not merely visible in systemd metadata: it is preserving the intended legacy process environment.
The DEV restart produced the expected controlled shutdown:
Received SIGTERM. Initiating clean shutdown.
Cleaning and exiting...
Achievements: final save() before exit
mediabot@dev.service: Deactivated successfully.
Stopped mediabot@dev.service
Started mediabot@dev.service
The systemd migration therefore reached the live boundary without forcing an unclean Mediabot termination.
systemd installer CI ✅
fail-closed replacement ✅
idempotency ✅
systemd template verification ✅
real shared-template deployment ✅
real rollback path ✅
DEV clean restart ✅
DEV database connectivity ✅
DEV IRC connectivity ✅
DEV channel joins ✅
DEV scheduler/application startup ✅
Undernet isolation during DEV migration ✅
Undernet itself has not yet been migrated to the modern lifecycle.
Its compatibility drop-in must remain until a separate validated migration is performed.
A reliable deployment spell is not the one that changes everything at once. It is the one that proves exactly what changed — and proves just as carefully what did not. 🪄
You must be logged in to reply.