Forum teuk.org

⏳ Mediabot 3.4dev — Every Hogwarts clock now tells the whole truth 📊🪄

in Mediabot · started by TeuK · 4w ago

TeuK · 4w ago

Mediabot v3 continues its 3.4 development cycle with a major observability pass covering database health, event-loop stalls, latency distributions, script envelopes and partyline diagnostics.

This round does not add one isolated feature. It connects several parts of the bot so that slowdowns, reconnects and stale state can be measured, displayed and explained instead of guessed.

🏦 Gringotts health reaches Prometheus

The database layer now exposes dedicated health metrics:

mediabot_db_up
mediabot_db_reconnects_total
mediabot_db_slow_pings_total

They reflect the same reconnect logic already used by Mediabot.

A failed reconnect sets the database gauge to 0, while successful reconnects and slow pings are counted separately.

The metric path remains best-effort: if the Metrics subsystem is unavailable, database behavior stays unchanged.

The important rule is simple: the graph must not claim the vault is open when the connection is dead.

🪜 The moving staircases can no longer freeze silently

Mediabot now detects delays in its own event loop.

The five-second tick measures how late it arrived. When the delay exceeds the configured tolerance, Mediabot records a line such as:

event loop stalled ~9.02s (tick expected every 5s)

It also increments:

mediabot_loop_stalls_total

and stores the latest detected stall for the partyline status display.

This catches more than database delays. A synchronous DNS lookup, disk pause or any other blocking work inside the event loop can now leave evidence.

The moving staircases may still stop, but a portrait will report when they did and for how long. 🪜

🧪 Latency becomes a distribution

The Metrics module now supports Prometheus histograms.

Instead of only logging when a command exceeds one second, Mediabot records the full duration distribution for:

mediabot_privmsg_processing_seconds
mediabot_scriptbridge_run_seconds{origin}

Prometheus exports cumulative buckets, +Inf, _sum and _count, while Grafana can calculate useful percentiles such as p50 and p95.

This means the operator can distinguish:

  • one rare ten-second pause;
  • a general rise in latency;
  • command scripts becoming slower than event or timer scripts;
  • a healthy median hiding a bad tail.

The final review also fixed an easy-to-miss edge case: very short Time::HiRes values may be written in scientific notation, such as 3e-06. Those observations are now accepted instead of silently disappearing from the histograms.

Every grain reaches the hourglass, including the smallest ones. ⏳

📊 Grafana gains an infrastructure row

The main overview dashboard now includes infrastructure panels for:

  • current database availability;
  • reconnects over 24 hours;
  • slow database pings;
  • event-loop stalls;
  • database events over time;
  • stall durations and frequency.

The script-bridge dashboard also gains latency percentiles by run origin.

As with the previous dashboard work, the JSON remains tied to code through tests: metrics referenced by Grafana must exist, and declared series must remain represented.

No painted needles. No imaginary counters.

📰 Fresh network data inside script envelopes

External scripts now receive a read-only network snapshot under:

data.network

Available fields include:

users
users_max
channels
servers
operators
age_seconds

The field is absent until real LUSERS data has been received.

The timing model is deliberately asymmetric:

data.config  = snapshot captured when the timer was armed
data.network = fresh state rebuilt when the script actually runs

A delayed callback can therefore keep its original route configuration while seeing the current network population.

That is exactly what an operational script usually needs: yesterday’s recipe, today’s Gazette. 📰

🛡️ Reserved means reserved

The final review tightened the data.network contract.

Callers cannot inject their own structured network field and accidentally leave a null or stale value in the payload.

The runner now removes any caller-provided value and rebuilds the field exclusively from the trusted LUSERS cache.

If no trusted network state exists, the field is absent—exactly as documented.

🧙 Partyline status without blocking the castle

The partyline .status output now includes:

DB: up
Loop: no stall detected

or, after an incident:

Loop: last stall 9.02s at <date>

The final hardening made an important correction: .status no longer performs a synchronous database ping or reconnect just to render the page.

It reads the already maintained health state instead.

A diagnostic command should report a freeze, not cause one.

🕰️ Slow partyline commands are now visible

The partyline dispatcher now applies the same timing discipline as the IRC PRIVMSG path.

Any command taking more than one second can produce:

SLOW PARTYLINE: .command took 2.41s

The observability triptych is now complete:

PRIVMSG processing
event-loop stalls
partyline commands

No user-facing path should be able to remain slow in silence.

📨 Failed LUSERS requests no longer delay the next retry

Manual and periodic LUSERS requests share a throttle.

The final review corrected the failure path so the throttle timestamp advances only after the IRC send succeeds.

A connection race can therefore no longer make Mediabot wait a full refresh interval after a request that never left the process.

This matters now that scripts and dashboards consume fresh network data.

✅ Validation

The completed pass was validated with:

10881/10881 tests passed
Security audit: GO — all 11 invariants hold
Startup integrity: OK
41/41 modules loaded
84/84 cross-module methods resolved
197/197 dispatch handlers resolved

The final state includes the three new test files for database health, histograms and network-envelope/status behavior.

No database schema change.
No mandatory live configuration change.
Existing installations keep their current behavior unless Metrics, dashboards or the new script fields are used.

🧙 Summary

This round makes Mediabot’s observability considerably more trustworthy:

  • database health reaches Prometheus;
  • reconnects and slow pings are counted;
  • event-loop stalls are detected;
  • PRIVMSG and script latencies become real distributions;
  • Grafana displays infrastructure health and p95 values;
  • scripts receive fresh network state;
  • partyline status remains non-blocking;
  • failed LUSERS sends do not poison the refresh throttle;
  • scientific-notation timings are no longer lost;
  • every diagnostic path reports state instead of changing it.

Hogwarts now has clocks, portraits, hourglasses and newspapers that agree with one another.

When Gringotts slows down or the staircases stop moving, Mediabot no longer shrugs—it records the delay, graphs the evidence and tells the operator exactly where to look. ⏳🏦📊🪄

You must be logged in to reply.