Forum teuk.org

MB746 — Owl Post Gets a Guarded Window and a Tiny Notebook 🦉📦

in Mediabot · started by TeuK · 8h ago

TeuK · 8h ago

Mediabot’s plugin roadmap has reached an important threshold.

The first API v3 foundations taught plugins how to declare commands, receive versioned events, schedule bounded jobs and obey a per-channel policy. MB745 then proved that a visible feature could move behind that boundary without breaking its historical behaviour.

MB746 opens the next passage: plugins can now fetch carefully bounded external content and retain a small amount of state — without receiving a network client, a socket, a filesystem path, a database handle or the bot itself.

This is not a secret tunnel through the castle walls. It is a guarded Owl Post window owned by the core. 🪟🦉

Why this mattered

A useful plugin eventually needs more than a command callback. It may need to read a JSON endpoint, cache a result or remember a small counter. Letting each plugin build its own HTTP and storage stack would recreate the coupling that API v3 is intended to remove:

  • different timeout and TLS rules;
  • duplicated SSRF protections;
  • unbounded responses or redirects;
  • direct knowledge of local paths and database internals;
  • callbacks that can still speak after a plugin has been disabled;
  • state writes with no conflict detection.

MB746 keeps those responsibilities in Mediabot’s core and exposes only two explicit capabilities:

  • http.fetch for asynchronous, policy-controlled HTTPS GET requests;
  • storage.kv for small namespaced and revisioned documents.

A package must request each capability in its manifest, and an operator must grant it while loading the package. Nothing is inherited accidentally.

A shared HTTPS service with castle walls

The new HTTP service performs outbound work away from the IRC event loop. The plugin submits a bounded request and later receives an immutable response; it never handles the transport itself.

The core enforces one consistent security and resource policy:

  • HTTPS only, on port 443;
  • normal TLS hostname verification;
  • DNS validation and connection pinning;
  • redirect revalidation, with at most two redirects;
  • rejection of loopback, private, link-local, documentation, multicast and other reserved destinations;
  • no credentials, fragments, arbitrary headers, cookies, request bodies or inherited proxy configuration;
  • a maximum timeout of ten seconds;
  • a maximum response body of 64 KiB;
  • at most two concurrent requests per plugin.

Successful results may enter a 128-entry, plugin-scoped process cache. The caller chooses a TTL, capped at one hour. This is why repeated live #short tests during the five-minute pilot window correctly returned the same advice: the second call used the configured cache instead of creating needless outbound traffic.

Three transport, rate-limit or server failures open a 60-second circuit. A disabled or unloaded package loses its owned requests, and late completions are discarded through the generation boundary. The current channel policy is also checked again before a callback may produce visible output.

In other words, even an owl arriving late must still show the right badge at the door. 🪪✨

A tiny notebook, not a database key

storage.kv gives each plugin its own small repository without revealing where or how it is stored.

A snapshot contains a monotonic revision and at most 64 scalar values. Each encoded value is limited to 2048 bytes. A commit supplies its expected revision and a bounded set of changes or deletions; if another write has already moved the revision forward, the stale commit receives a conflict instead of silently overwriting newer state.

The core still owns the existing atomic 0600 temporary-file-and-rename boundary. Plugin code sees no pathname, filehandle, SQL connection or arbitrary query surface.

Channel policy remains authoritative:

  • off blocks the feature;
  • observe permits reads and execution, but suppresses IRC output and writes;
  • on permits the bounded visible result and repository commit.

This is intentionally a small notebook for compact plugin state. Richer domain data will later use approved data.<domain> facades rather than turning storage.kv into a disguised general-purpose database.

short-content-v3: a real but deliberately small proof

MB746 ships an inactive proof package named short-content-v3. It contributes one public command, short, and requests exactly three capabilities:

http.fetch, irc.reply, storage.kv

Its channel policy selects a trusted HTTPS JSON endpoint, a dotted scalar path such as slip.advice, a language, a cache duration, a maximum length and a prefix. The plugin then:

  1. asks the shared HTTP service for one JSON document;
  2. reads one scalar through a path of at most four object keys;
  3. emits at most one bounded IRC line;
  4. stores only the last parsed value and a served counter.

The proof package is discovered but never loaded or enabled at startup. It has no default endpoint, and no channel becomes active automatically.

The live #test pilot

The package was loaded explicitly through the authenticated Owner Partyline, granted only its three declared capabilities and first placed in observe on #test.

After inspection, the same complete typed policy was switched to on with the trusted Advice Slip endpoint and slip.advice JSON path. The end-to-end IRC test then produced:

<Te[u]K> #short
<mediabotv3> Conseil:Learn to handle criticism.

The repeated identical response within 300 seconds was expected cache behaviour, not a stuck command. The bot remained connected, the package call counter advanced only on real command dispatch, and the state write created the expected namespaced plugin data.

Rollback remains immediate and does not require a database migration or a private configuration edit:

.plugins policy short-content-v3 #test off
.plugins disable short-content-v3
.plugins unload short-content-v3

off blocks new work, disable cancels owned work and invalidates late callbacks, and unload removes the command and runtime entry.

Validation — and one useful trap caught by the full suite

The MB746 development gates completed successfully before the final commit gate:

  • 501/501 targeted tests passed;
  • 7404/7404 fast-lane tests passed;
  • syntax, API contract, generated inventory and file-ending checks passed;
  • the development service restarted cleanly and reconnected to IRC;
  • the live Partyline and IRC pilot passed on #test.

The first complete suite then stopped at 19678/19679 assertions. The failure was not in HTTP, storage or runtime delivery. The successful live pilot had legitimately created plugin-data, while an older offline-tool test still assumed that the repository-wide directory could never exist.

That was exactly the kind of environmental assumption a final gate should catch. No commit or push occurred. The correction moves the offline tool test into its own temporary working directory and checks for side effects there, without deleting or modifying the real plugin data created by the pilot. The corrected gate now covers 26 MB746 files and reruns targeted tests, the fast lane and the complete suite before publication is allowed.

Where the roadmap now stands

The path from MB740 to MB746 is no longer just an architectural sketch:

  • MB740 mapped the legacy surface and froze API v2;
  • MB741 established one authoritative command catalogue;
  • MB742 introduced strict API v3 manifests, capabilities and lifecycle;
  • MB743 added versioned events, backpressure and shared jobs;
  • MB744 added typed configuration and per-channel off / observe / on;
  • MB745 migrated the first visible playful command pack with exact fallback;
  • MB746 adds shared HTTPS and namespaced compare-and-swap storage.

API v3 remains experimental, explicit and off by default. It is a capability boundary for trusted in-process Perl packages, not an operating-system sandbox. Its purpose is to prevent accidental coupling, centralize policy and make every activation observable and reversible.

The next chapters can now focus on approved domain-data facades, richer first-party migrations, developer tooling and the gradual retirement of duplicate dispatch paths. The doors are opening one at a time — with the keys, wards and rollback spell already in place. 🗝️🏰

MB746 gives plugins a guarded window onto the outside world and a tiny notebook of their own. Mediabot still decides who may open the window, what may pass through it, and when the spell must stop. 🦉📦🪄

You must be logged in to reply.