Forum teuk.org

🔭 Mediabot v3 — The Plugin Observatory and the Pensieve

in Mediabot · started by TeuK · 3w ago

TeuK · 3w ago

Development chronicle — mb598 to mb602 · Metrics, time events, configuration and persistent state

The plugin v2 gallery is no longer only alive. It can now be observed, configured and given a memory.

This development arc adds Prometheus metrics for plugin activity, detailed partyline inspection, network-wide nick and quit events, a cron-like event inspired by Eggdrop’s bind time, per-plugin configuration, and bounded persistent JSON storage controlled entirely by Mediabot.

A final pre-commit review then hardened every boundary where operator input, plugin metadata, local files and external script results meet.

No database schema change was required.


🦉 mb598 — Plugin activity enters the observatory

Four Prometheus metrics now describe the live plugin system:

mediabot_plugin_command_total{plugin,command}
mediabot_plugin_command_denied_total{plugin,command}
mediabot_plugin_event_total{plugin,event}
mediabot_plugin_script_failure_total{plugin,kind}

Their semantics are deliberately separate.

  • command_total counts authorized dispatches.
  • command_denied_total counts access-control refusals.
  • event_total counts routed plugin events.
  • script_failure_total counts command or event execution failures.

An authorized command that later fails inside its script therefore contributes both to usage and failure visibility. An authorization refusal contributes only to the denied metric.

Typical Grafana queries can now answer useful questions:

sum by (plugin) (
  rate(mediabot_plugin_command_total[5m])
)
sum by (plugin, kind) (
  increase(mediabot_plugin_script_failure_total[1h])
)
sum by (plugin, command) (
  increase(mediabot_plugin_command_denied_total[1h])
)

.plugins info

Operators also gain a detailed read-only plugin card:

.plugins info fortune

Example output:

Plugin 'fortune' [enabled] kind=script api=2 version=1.0
  source: examples-v2/fortune.pl
  description: Random aphorisms by category
  commands:
    fortune   level=public calls=42 fortune [code|irc|life]
    fortunes  level=Master calls=3 fortunes - list categories
  events: (none)

The command reports metadata, commands, access levels, calls, effective configuration, event subscriptions and storage information without loading, unloading or reconfiguring the plugin.

The existing Owner and Master gates for mutating verbs remain unchanged.


🕰️ mb599 — Nick, quit and the return of bind time

The EventBus routing surface now includes network-wide identity changes.

Nick changes

A script subscribed to the nick event receives:

  • the previous nickname;
  • the new nickname;
  • whether the change concerns the bot itself.

Real quits

Quit events include the nickname and quit reason.

They deliberately do not claim a channel, because an IRC quit belongs to the network rather than one channel.

Mediabot does not emit plugin quit events during a detected netsplit. Starting external scripts for a burst of split-generated departures would defeat the existing protections designed to suppress expensive work during that exact incident.

plugin_cron_observed

A new periodic event brings an Eggdrop-style bind time model to plugin v2:

{
  "api": 2,
  "name": "announcements",
  "version": "1.0",
  "events": [
    "plugin_cron_observed"
  ]
}

The existing five-second timer emits at most once per minute and only when a listener exists.

The script receives:

minute
hour
dow
mday
month

A daily announcement at 09:00 therefore remains a simple script-side decision rather than another scheduler configuration language.


🖼️ mb600 — Each plugin gets an adjustable frame

A sidecar can now declare configuration defaults:

{
  "api": 2,
  "name": "greeter",
  "version": "1.0",
  "events": [
    "channel_join_observed"
  ],
  "config": {
    "GREETING": ""
  }
}

The bot configuration may override those values:

[plugins]
greeter.GREETING="Welcome %s, the cider is cold."

The effective configuration is delivered to the script through:

data.config

The greeter is the living example:

.plugins reload greeter

A joining user can then receive the operator-defined text without editing the Tcl script.

Configuration is snapshot-based:

  • sidecar defaults are validated at load;
  • operator overrides are merged at load;
  • reload reads both again;
  • malformed operator overrides are ignored with a trace;
  • a bad local override does not prevent the plugin from loading.

This keeps the author contract fail-closed while preventing an operator typo from unnecessarily taking a working plugin offline.


🧠 mb601 — The bot holds the memory

External scripts still do not receive arbitrary filesystem access.

To persist state, a script emits a protocol action:

{
  "type": "store",
  "data": {
    "counts": {
      "SlaY": 3
    },
    "total": 3
  }
}

Mediabot applies the action through an injected storage sink.

On the next command or event dispatch, the current state is returned through:

data.storage

The contract is bounded:

  • the stored value must be a JSON object;
  • maximum depth: 3;
  • maximum keys: 256;
  • maximum key length: 64 characters;
  • maximum canonical serialized size: 16 KiB;
  • one store action per run;
  • the first store wins and additional stores are errors.

The new document replaces the previous document. Read-modify-write logic remains explicit in the script.

Atomic bot-side writes

Storage is written under the configured plugin data directory:

<DATA_DIR>/<plugin>.json

Writes use a temporary file followed by an atomic rename.

The directory is private and created with restrictive permissions only when a real store operation needs it.

Fresh state on every dispatch

Unlike configuration, storage is read afresh for every dispatch.

That distinction is intentional:

data.config   → snapshot taken when the plugin loads
data.storage  → current state read for each invocation

Concurrent runs follow a documented last-writer-wins model.

Partyline operations

.plugins info shows whether data exists and its bounded size.

An Owner may purge one plugin’s state:

.plugins cleardata counter

The command removes only the validated plugin slug’s own storage file.


🛡️ mb602 — The boundary ward

The final review found places where the implementation did not yet fully match the guarantees promised by mb598–mb601.

They are now closed.

Storage names cannot escape their directory

Every storage operation revalidates the plugin slug before constructing a path.

Partyline input such as traversal components cannot turn cleardata into a generic file-deletion primitive.

Inspection remains truly read-only

.plugins info no longer creates the data directory merely to calculate a possible storage path.

A read command now leaves the filesystem unchanged.

JSON booleans are valid data

true and false are accepted as JSON leaves instead of being mistaken for unsupported arbitrary Perl objects.

The same limits apply everywhere

The storage contract is enforced during:

planning → writing → reading

A hand-created or damaged local file cannot bypass the depth, key-count, key-length or size limits that apply to script-produced data.

Invalid files and unsafe symbolic-link cases are ignored and logged rather than crashing dispatch.

Sensitive configuration is redacted

.plugins info still shows useful effective configuration, but values behind names resembling secrets are displayed as:

[redacted]

Examples include keys containing concepts such as:

PASSWORD
TOKEN
SECRET
API_KEY
CLIENT_SECRET

Descriptions, help text and displayed configuration values are also normalized to bounded single-line text so metadata cannot forge partyline output.

Literal percent signs are safe in the greeter

The configurable Tcl greeting no longer treats the operator’s whole message as a Tcl format program.

The %s placeholder is replaced literally, while unrelated percent characters survive:

100% welcome, %s

becomes:

100% welcome, SlaY

Failures are visible and counted consistently

Useful diagnostics from script responses and action application are preserved when safe.

Action-application failures now contribute to the plugin failure metric for both command and event execution paths.

Malformed diagnostics receive a generic fallback and cannot crash the bot.


đź§Ş Validation

The complete local suite passed after the mb602 hardening:

11969/11969 PASSED

The dedicated coverage includes:

  • Prometheus plugin metrics;
  • authorized and denied command accounting;
  • command and event failure accounting;
  • read-only .plugins info;
  • nick and quit event envelopes;
  • netsplit suppression;
  • minute-level periodic events;
  • sidecar configuration defaults and overrides;
  • real Tcl greeter configuration;
  • bot-side persistent state;
  • command and event storage round trips;
  • storage gates and atomic-write structure;
  • path traversal rejection;
  • read-only filesystem behavior;
  • JSON boolean support;
  • depth, key and size enforcement on every boundary;
  • secret redaction;
  • literal percent handling;
  • safe diagnostics.

✨ What plugin authors gain

A sidecar plugin may now declare four complete dimensions:

commands
events
config
persistent state

The bot supplies the surrounding controls:

  • manifest validation;
  • authentication and authorization;
  • EventBus routing;
  • periodic events;
  • metrics;
  • operator inspection;
  • configuration merging;
  • bounded storage;
  • atomic writes;
  • redaction;
  • reload and rollback;
  • complete unload.

The script remains a small, replaceable worker. Mediabot remains the trusted keeper of policy, files, credentials and lifecycle.


🪄 Final parchment

The castle’s portraits now have a registry in the owlery, clocks in the tower, adjustable frames and private shelves in the Pensieve room.

They may remember, but they never hold the keys to the cupboard.

They may report their activity, but they do not reveal the operator’s secrets.

And every path from the script world back into the castle passes through a measured, guarded door.

You must be logged in to reply.