Forum teuk.org

Mediabot 3.6dev: opening the plugin corridors, one guarded door at a time 🗺️🪄

in Mediabot · started by TeuK · 14h ago

TeuK · 14h ago

On September 17, we stepped back from the steady flow of feature work and asked a broader question: what should Mediabot become next?

The answer was not another isolated command, another integration bolted onto the core, or a rewrite in a fashionable new language. Mediabot already had plugin mechanisms, but most of its real product behavior still lived in large modules and historical command tables. The extension points existed; the architecture around them was not yet strong enough to make plugins the normal, safe way to grow the bot.

That observation became a roadmap: preserve the proven IRC core, draw a strict boundary around it, and build a new plugin platform incrementally. Five milestones later, MB740 through MB744 are now committed on the 3.6dev line. The castle has not been rebuilt. We have mapped it, placed one authoritative door at the entrance, and added guarded passages through which future features can travel. 🏰

Why revisit the plugin architecture?

Mediabot already supported trusted in-process Perl modules and external sidecar scripts. Those systems remain useful, but they had several limits:

  • trusted in-process plugins could see the complete bot object;
  • sidecar API v2 was safer and declarative, but mostly global and centered on public commands;
  • built-in commands and plugins did not yet converge on one authoritative catalogue;
  • shared services such as events, scheduling, HTTP, storage and approved data access did not have one stable capability boundary;
  • enabling a package and deciding where it may act were not separate enough.

Continuing to add features along every historical route would create more authorities for dispatch, access control and documentation. A rewrite, on the other hand, would throw away years of tested IRC behavior, security rules and operational knowledge.

We therefore chose an incremental microkernel direction:

  • the core keeps IRC connectivity, identity, authorization, wire safety, registries, migrations, secrets, lifecycle and observability;
  • plugins receive narrow services instead of the complete bot;
  • built-ins and plugins use one command catalogue;
  • events and jobs are versioned, copied, bounded and core-owned;
  • activation is explicit for both the package and each channel;
  • API v2 remains supported while API v3 grows behind a compatibility boundary;
  • low-risk features move first, with rollback preserved at every step.

The effective permission model can now be summarized as:

manifest request
  INTERSECT instance grant
  INTERSECT channel policy
  INTERSECT runtime user authorization

A plugin can act only when every gate agrees. No single manifest, configuration switch or global enable operation is enough to bypass the others.

The five completed milestones

Milestone Purpose Result
MB740 Map the existing architecture Accepted boundary, API v2 freeze and deterministic inventory
MB741 Give commands one front door CommandRegistry became authoritative for built-ins and plugins
MB742 Make API v3 executable Strict packages, capability-scoped context and explicit lifecycle
MB743 Add observation and time Versioned events, backpressure and scheduler-owned jobs
MB744 Decide where a plugin may act Typed channel configuration and off / observe / on policy

These milestones deliberately build on one another. MB742 would have created a second command authority without MB741. MB743 would have exposed unsafe global activity without MB742’s capability boundary. MB744 would have been a cosmetic switch without the event, job and output paths already passing through the core.

MB740: drawing the Marauder’s Map 🗺️🧩

Commit: cd812d6

The first milestone changed no IRC behavior. Its job was to replace assumptions with an exact map.

MB740 recorded an architecture decision describing what belongs in the core, what future plugins may own, and which alternatives were rejected. It also froze API v2 as a compatibility contract: existing plugins remain supported and security repairs continue, but new privileged services will be designed for API v3.

A deterministic architecture inventory was added to examine the existing command surfaces. It covered the 245 current built-in help entries and exposed how the registry, public dispatch and private dispatch related to one another. A machine-readable API v2 contract was also committed so implementation and documentation drift can be detected automatically.

The important result was not new runtime code. It was an agreed boundary and a sequence that could be tested one milestone at a time.

MB741: one enchanted doorway for every command 🪄🚪

Commit: 6680606

Before extracting features, command lookup needed one source of truth. MB741 made CommandRegistry authoritative for all built-in commands:

  • 238 public command entries;
  • 94 private command entries;
  • syntax, access level and descriptions captured as catalogue metadata;
  • built-ins and plugins resolved through the same lookup path.

The historical public and private handler hashes still exist, but only as frozen implementation adapters. They are no longer alternative discovery or fallback paths. Adding or removing an adapter without updating the catalogue now fails the deterministic inventory and contract tests.

This was a structural change with intentionally unchanged command behavior. Existing handlers still execute, but every caller must enter through the same front door. Future commands are registry-native by construction.

MB742: a guarded Portkey into API v3 🗝️🪄

Commit: 192dcc2

MB742 turned the design into an executable API v3 foundation.

An API v3 plugin is now a package directory with a strict plugin.json manifest. Discovery reads and validates metadata without loading executable code. Unknown fields, oversized manifests, unsafe paths, symlinks, invalid entrypoints, malformed commands, aliases, events or capability requests fail closed.

Loading and enabling became two distinct operations:

  1. discovery validates the package;
  2. explicit loading constructs the runtime but leaves it disabled;
  3. explicit enabling starts its lifecycle;
  4. disabling and unloading stop it and remove its registrations.

API v3 code receives Mediabot::PluginContext and copied invocation data. It does not receive the Mediabot object, raw command context, IRC socket, database handle or configuration object. Replies and notices cross core-owned, capability-checked sinks.

An inert hello-v3 reference package was added to exercise the contract without activating anything. A read-only v2 adapter describes older plugins for migration tooling, but does not silently run them through v3 or expand their permissions.

One nuance matters: API v3 Perl packages remain trusted in-process code. This is a strong architectural and policy boundary, not an operating-system sandbox against deliberately hostile Perl. Its purpose is to prevent accidental coupling, centralize authority and make behavior testable and revocable.

MB743: events, backpressure and a guarded Time-Turner ⏳🪄

Commit: 933607c

Commands alone do not make a useful plugin platform. Plugins also need to observe the bot and perform recurring work without owning the event loop.

MB743 published eight versioned event schemas:

  • public command observation;
  • channel join, part, topic and kick;
  • nick change and user quit;
  • the scheduler minute event.

The core translates existing observations into field-whitelisted, detached envelopes. Unknown fields, references and out-of-range integers are discarded. Event handlers never run inside the original core EventBus callback.

Each plugin owns a deferred queue with explicit backpressure:

  • at most 32 pending envelopes;
  • at most eight envelopes drained per turn;
  • the newest event is dropped when the queue is full;
  • loss is logged and counted;
  • disable or unload clears pending work and invalidates deferred callbacks.

MB743 also introduced declarative periodic jobs owned by the central scheduler. A plugin may declare up to eight jobs, with bounded intervals and delays. Loading reserves granted jobs without starting them; enabling starts them transactionally; disabling stops them; unloading removes them. A start failure rolls back the work already started.

Events require events.subscribe. Jobs require scheduler.jobs. Neither service activates a package automatically.

MB744: one guarded Room Key per channel 🔑🏰

Commit: b9c0bcf

MB744 completed the first foundation arc by separating global package lifecycle from channel activation.

Every channel now has one of three core-owned modes:

Mode Handler execution IRC output
off No No
observe Yes, within normal limits Suppressed
on Yes, within normal limits Allowed only with the required capability

off is unconditional and remains the default. Enabling a package globally does not opt any channel in. Channel names use IRC RFC1459 casemapping and a plugin may hold at most 128 channel policies.

The manifest’s config_schema is now an executable core contract rather than a documentation hint. It supports bounded string, integer and boolean fields with typed defaults, required values, string lengths, enums and numeric bounds. The core rejects unknown schema properties, unknown configured values and implicit string-to-number or string-to-boolean coercion before plugin code runs.

Additional limits keep the contract predictable:

  • at most 32 configuration fields;
  • at most 4,096 encoded bytes in one effective channel configuration;
  • detached snapshots for commands, events and jobs;
  • no mutation of stored policy through a plugin’s local copy.

The late checks are as important as the initial ones. Policy and configuration are read again when a deferred event drains and when output is emitted. If an operator switches a channel to off, queued work is revoked. If the channel moves to observe, a delayed reply cannot escape onto IRC. The Room Key is checked at the door, not merely when the journey begins.

What remained deliberately unchanged

This work is a platform foundation, not a surprise production rollout.

  • API v1 and API v2 runtime behavior remains intact.
  • No API v3 plugin is activated automatically.
  • No channel is opted in automatically.
  • The hello-v3 package remains an inert witness.
  • No private configuration was changed.
  • No database schema or stored data was changed.
  • No plugin migration or remote installation occurs at startup.
  • IRC transport, authentication, authorization and flood protection remain core responsibilities.

The default state is therefore quiet and reversible. The new corridors exist, but no portrait swings open by itself at midnight.

The gates did their job

The implementation followed the same operational rule throughout: syntax, targeted tests and the fast lane during development; the complete suite exactly once, immediately before the final commit.

Milestone Targeted assertions Fast-lane assertions Final full suite
MB740 75 7,167 947 files / 19,353 assertions
MB741 555 7,190 948 files / 19,377 assertions
MB742 381 7,260 952 files / 19,447 assertions
MB743 849 7,324 955 files / 19,542 assertions
MB744 889 7,358 958 files / 19,589 assertions

Every final suite passed. Each commit was made only after the staged source was hashed, the worktree was frozen, and the running development service was confirmed unchanged throughout the full suite.

The guards also stopped imperfect candidates before promotion. Whitespace at end of file, a stale source-inspection assumption and a package-directory checksum mistake were all caught before a final commit. Corrections were made in new, verified artifacts instead of weakening the checks. That is exactly what fail-closed delivery is supposed to achieve.

Across the five final commits, the full-suite surface grew from 947 to 958 test files and from 19,353 to 19,589 assertions while preserving the established runtime.

What this unlocks

Mediabot now has the foundations needed for independently testable and reversible first-party plugins:

  • one authoritative command registry;
  • strict package discovery;
  • explicit lifecycle and capability grants;
  • copied command, event and job inputs;
  • bounded event delivery with backpressure;
  • scheduler-owned periodic work;
  • typed configuration per channel;
  • safe observe mode before visible activation;
  • late revocation of queued work and IRC output;
  • machine-readable contracts and deterministic documentation checks.

This does not make every existing feature a plugin overnight. It makes future extraction possible without inventing a new security and lifecycle model for each feature.

Next stop: MB745, the first visible proof 🧹✨

The next milestone will prove the platform with deliberately low-risk behavior on one development channel.

The current plan is to move the simple fun-command pack first:

  • roll;
  • flip;
  • choose;
  • 8ball;
  • morse;
  • abbrev.

MB745 will also introduce one new autonomous channel ritual as a native API v3 plugin. The rollout will begin in observe, compare behavior and diagnostics, then move to on only after the old path, permissions, output limits and rollback have been demonstrated.

Richer extractions come later. Shared HTTP and data facades must exist before external content or database-backed features move. AI conversation, radio, central moderation, authentication, updating and IRC transport are explicitly not first-wave candidates: they are too coupled and too important to use as platform experiments.

Later milestones will add those mediated services, improve author and operator tooling, extract more first-party features, and retire duplicate compatibility dispatch only after parity and rollback are proven.

Closing the first chapter

The most important result of MB740 through MB744 is not the number of new modules. It is the reduction in implicit authority.

A plugin no longer needs the whole castle to light one room. It requests a small set of capabilities, receives copied data, runs behind bounded queues and owned jobs, and acts only in channels whose policy permits it. The core remains responsible for the dangerous doors.

That was the promise of the roadmap. Five milestones later, the promise is now executable, documented and backed by 19,589 passing assertions.

The map is drawn. The front door is guarded. The Room Keys are ready. Now we can finally start moving the furniture. 🗺️🔑🪄


  • Current development version: 3.6dev-20260918_191643
  • Foundation milestones: MB740–MB744
  • Next milestone: MB745, first development-channel plugin rollout

You must be logged in to reply.