Mediabot v3 continues its 3.4 development cycle with a major observability pass covering database health, event-loop stalls, latency distributions, script envelopes and partyline diagnostics.
This round does not add one isolated feature. It connects several parts of the bot so that slowdowns, reconnects and stale state can be measured, displayed and explained instead of guessed.
The database layer now exposes dedicated health metrics:
mediabot_db_up
mediabot_db_reconnects_total
mediabot_db_slow_pings_total
They reflect the same reconnect logic already used by Mediabot.
A failed reconnect sets the database gauge to 0, while successful reconnects and slow pings are counted separately.
The metric path remains best-effort: if the Metrics subsystem is unavailable, database behavior stays unchanged.
The important rule is simple: the graph must not claim the vault is open when the connection is dead.
Mediabot now detects delays in its own event loop.
The five-second tick measures how late it arrived. When the delay exceeds the configured tolerance, Mediabot records a line such as:
event loop stalled ~9.02s (tick expected every 5s)
It also increments:
mediabot_loop_stalls_total
and stores the latest detected stall for the partyline status display.
This catches more than database delays. A synchronous DNS lookup, disk pause or any other blocking work inside the event loop can now leave evidence.
The moving staircases may still stop, but a portrait will report when they did and for how long. 🪜
The Metrics module now supports Prometheus histograms.
Instead of only logging when a command exceeds one second, Mediabot records the full duration distribution for:
mediabot_privmsg_processing_seconds
mediabot_scriptbridge_run_seconds{origin}
Prometheus exports cumulative buckets, +Inf, _sum and _count, while Grafana can calculate useful percentiles such as p50 and p95.
This means the operator can distinguish:
The final review also fixed an easy-to-miss edge case: very short Time::HiRes values may be written in scientific notation, such as 3e-06. Those observations are now accepted instead of silently disappearing from the histograms.
Every grain reaches the hourglass, including the smallest ones. ⏳
The main overview dashboard now includes infrastructure panels for:
The script-bridge dashboard also gains latency percentiles by run origin.
As with the previous dashboard work, the JSON remains tied to code through tests: metrics referenced by Grafana must exist, and declared series must remain represented.
No painted needles. No imaginary counters.
External scripts now receive a read-only network snapshot under:
data.network
Available fields include:
users
users_max
channels
servers
operators
age_seconds
The field is absent until real LUSERS data has been received.
The timing model is deliberately asymmetric:
data.config = snapshot captured when the timer was armed
data.network = fresh state rebuilt when the script actually runs
A delayed callback can therefore keep its original route configuration while seeing the current network population.
That is exactly what an operational script usually needs: yesterday’s recipe, today’s Gazette. 📰
The final review tightened the data.network contract.
Callers cannot inject their own structured network field and accidentally leave a null or stale value in the payload.
The runner now removes any caller-provided value and rebuilds the field exclusively from the trusted LUSERS cache.
If no trusted network state exists, the field is absent—exactly as documented.
The partyline .status output now includes:
DB: up
Loop: no stall detected
or, after an incident:
Loop: last stall 9.02s at <date>
The final hardening made an important correction: .status no longer performs a synchronous database ping or reconnect just to render the page.
It reads the already maintained health state instead.
A diagnostic command should report a freeze, not cause one.
The partyline dispatcher now applies the same timing discipline as the IRC PRIVMSG path.
Any command taking more than one second can produce:
SLOW PARTYLINE: .command took 2.41s
The observability triptych is now complete:
PRIVMSG processing
event-loop stalls
partyline commands
No user-facing path should be able to remain slow in silence.
Manual and periodic LUSERS requests share a throttle.
The final review corrected the failure path so the throttle timestamp advances only after the IRC send succeeds.
A connection race can therefore no longer make Mediabot wait a full refresh interval after a request that never left the process.
This matters now that scripts and dashboards consume fresh network data.
The completed pass was validated with:
10881/10881 tests passed
Security audit: GO — all 11 invariants hold
Startup integrity: OK
41/41 modules loaded
84/84 cross-module methods resolved
197/197 dispatch handlers resolved
The final state includes the three new test files for database health, histograms and network-envelope/status behavior.
No database schema change.
No mandatory live configuration change.
Existing installations keep their current behavior unless Metrics, dashboards or the new script fields are used.
This round makes Mediabot’s observability considerably more trustworthy:
Hogwarts now has clocks, portraits, hourglasses and newspapers that agree with one another.
When Gringotts slows down or the staircases stop moving, Mediabot no longer shrugs—it records the delay, graphs the evidence and tells the operator exactly where to look. ⏳🏦📊🪄
You must be logged in to reply.