Forum teuk.org

🏰 The Gatekeeper, the Owl Post and the Atlas

in Mediabot · started by TeuK · 23h ago

TeuK · 23h ago

Mediabot MB897–MB899 · Three dedicated modules, thirteen preserved Helpers entry points, and 30,251 passing assertions across the same 1,156 test files.

The previous article, Clearer Corridors, Bounded Buffers and the Book of Admittance, ended at MB896. Browser execution, social-provider handling, channel-history reads and version grammar had acquired clearer owners. Two concrete runtime defects had also been corrected along the way.

The next three rounds returned to Mediabot::Helpers. Three responsibilities still shared that general utility room: deciding whether a command could proceed, checking a remote version, and finding the country associated with a hostname or IPv4 address.

They now have their own doors:

  • Mediabot::IRC::CommandThrottle owns the existing command-admission guards.
  • Mediabot::Version::Remote owns remote VERSION retrieval and its worker adapter.
  • Mediabot::External::Country owns country lookup and its existing asynchronous adapter.

Across these rounds, Helpers falls from 4,722 to 3,848 lines. Its historical entry points remain available. The latest development-host validation passes 30,251 / 30,251 assertions, all 37 security invariants, and startup integrity.

Mediabot remains on 3.6dev. The database schema and SQL are unchanged. This chapter improves ownership and the evidence protecting compatibility; it introduces no new user command or configuration setting.

The gatekeeper has a watchtower, the owls have a sorting office, and the atlas has a reading room. The work was making sure the old corridors still reach them. 🦉

🚪 MB897 — Give command admission its own watchtower

Command throttling answers several related questions. Has this command been used too recently in this channel? Is a channel sending too many commands within its window? Has one nickname accumulated enough traffic or strikes to be temporarily muted?

Those decisions form a coherent responsibility. MB897 moves four functions into Mediabot::IRC::CommandThrottle: the antiflood integer-configuration reader, command cooldown checker, channel-flood checker and nickname-flood checker.

The new module offers explicit imports and qualified calls, with no default exports. Helpers keeps forwarding wrappers under all four old names. Existing command and Partyline call sites continue using their established interfaces.

Helpers drops from 4,722 to 4,530 lines. Its 96 other function bodies and default exports are byte-identical. The configuration reader moves unchanged; the three guards retain their bodies apart from eight configuration calls routed through a compatibility relay.

Keep the gatekeeper’s notebook in the parent

The counters and timestamps remain in the same bot object. No admission check moves into a child process, and no new timer, worker or global state is introduced.

That placement matters for commands doing their expensive work in disposable workers. A cooldown recorded only inside such a worker would disappear when the worker exited. The next request could then look like the first request again. Existing callers continue checking admission and retaining its state in the parent.

Alias handling also keeps its established meaning. The news aliases share one cooldown bucket. Alternating between actualites, news, actu and actualite therefore does not create four independent allowances. A request through the historical Helpers name and a later request through the new module see the same bot-owned state.

Configuration precedence remains explicit override, then channel configuration, then the existing command default. Commands retain their individual durations, including the longer windows for archive-heavy aggregation. The extraction introduces no new default or threshold.

Preserve the edges of each window

Flood control is sensitive to details that a normal successful command does not exercise. The channel guard retains its sliding window, strict over-limit comparison, silence expiry, notification cooldown, metrics and warn-only mode. Warn-only continues recording the event without silencing the channel.

The nickname guard retains its own window and state cleanup, notice cadence, strike counter and 300-second mute after three strikes. Expired mutes are removed as before. Notifications and metrics keep their existing calls and exception behaviour.

Historical target and case conventions remain too. These guards retain their existing #-channel restriction and lowercase transformations. Other IRC utilities may recognise additional target forms or support richer case mapping, but adopting those rules here would change which requests share or enter a throttle bucket. That policy change is outside this extraction.

The original whole-second clock and boundary comparisons remain. Tests control time before module compilation, so expiry checks can exercise exact boundaries without waiting for real seconds to pass.

Admission and outgoing delivery keep separate owners

The outgoing checkAntiFlood implementation, its database query, parameter caches and deferred delivery remain where they were. Its shared integer getter reaches the same algorithm through the retained wrapper.

This distinction keeps two different decisions clear: admitting work and scheduling messages produced by that work. Extracting the first does not retune the output queue or change when an IRC message is sent.

Configuration and notification hooks remain resolved at call time. Calculations with intercepted notifications can run without loading Helpers, HTTP or DBI. Actual notices still use the established sending helpers and their runtime dependencies; the extraction does not make real IRC output a pure calculation.

Seven existing cases follow the new implementation owner. The nickname-flood case grows from 9 to 15 assertions, covering fresh load orders, controlled clocks, configuration and notification hooks, shared alias state, strikes, mute expiry and cleanup.

An external probe executes the real implementations over 107 scenarios. Its event sequences include 619 guard calls, with state copied after each step. Returns, state, encoding flags and bytes, configuration reads, logs, metrics, notices, errors and relevant SQL traces agree across the reference, compatibility wrappers and direct module API.

A deliberately altered copy bypasses configuration hooks and removes notices. Five assertions reject it; the candidate passes all fifteen. The watchtower’s tests check that the gatekeeper still consults the right notebook and still sends the required warning.

🦉 MB898 — Put remote version checks in the Owl Post Office

MB896 had already separated version grammar and ordering into Mediabot::Version::Format. Remote retrieval still lived in Helpers alongside unrelated utilities.

MB898 completes another part of that separation. Mediabot::Version::Remote becomes the owner of seven functions: URL selection, per-request timeout, worker budget, remote fetch, synchronous check, worker-error translation and asynchronous check.

Helpers keeps all seven historical names as wrappers. Its 93 other function bodies and default exports remain byte-identical. It shrinks from 4,530 to 4,137 lines; the new owner has 533 lines, including compatibility bridges and comments.

The resulting responsibilities are easier to distinguish:

Owner Responsibility retained there
Version::Format Recognise, parse and compare Mediabot versions
Helpers’ local identity code Resolve and read local VERSION, retain its cache, and coordinate existing command/startup decisions
Version::Remote Select remote sources, fetch and validate their response, and adapt version checks to the shared worker
AsyncWorker Manage the shared child-process lifecycle used by the remote adapter

Local identity-file resolution stays in its established location. Moving that code casually could change which directory becomes the project root. Immediate command replies and update decisions also keep their current implementations.

A route override remains a route override

Remote fetching preserves the existing source-selection policy. Normally, the checker has its two established version URLs. An explicit configured URL replaces that list; it does not become a preliminary attempt followed by an unsolicited return to the default sources.

The default list now has one definition in Remote. Helpers retains its historical mutable list, initialized from those defaults. When Helpers is present, the canonical code reads that legacy list, including an intentionally empty one. Existing callers and fixtures can still replace it without acquiring a second, stale copy of the route policy.

The request timeout keeps its eight-second default and thirty-second cap. The worker budget continues accounting for the selected route count: per-route timeout multiplied by the number of routes, with one second of margin and the existing absolute cap of sixty-five seconds. With the two default routes and default request timeout, that means a seventeen-second worker budget.

Those are carried-forward rules. MB898 does not introduce a new retry strategy or extend an operator’s configured route selection.

The envelope carries the reason as well as the version

Transport still uses the shared HTTP client factory and its TLS policy. The 64 KiB HTTP response bound, version-response validation, whitespace/BOM handling and single-line grammar checks remain. The adapter retains its separate 1,024-byte worker-output limit.

Keeping those limits distinct matters: a download bound, accepted version value and inter-process result envelope protect different stages of the operation.

Failure reasons continue crossing the worker boundary. A timeout, unavailable HTTPS support, failed request, malformed response or worker failure must retain its established diagnostic meaning. The extraction does not turn those cases into an unexplained missing version.

The silent child logger moves with its exact body and historical class name. It still prevents duplicate child-side logging while the parent receives the result and reason. Three-argument callbacks retain their shape and single-completion behaviour. The no-loop synchronous compatibility path remains available.

Remote’s load and configuration calculations make no network request. Operations requiring local identity or the shared HTTP factory still resolve those dependencies when needed. This makes the owner more focused without claiming every full version-check operation is independent of Helpers.

Follow replacements installed after loading

An existing hook may be replaced after both modules have loaded. Binding permanently to its original function during import could silently bypass that replacement.

Remote therefore resolves the established URL, timeout, fetch, check and reason hooks at call time. Parsing and comparison retain their Helpers/Format bridges. Local identity and cache access reuse the established Helpers owner. The new module exposes seven optional imports and adds no default imports to its caller.

Ten existing cases follow the implementation’s new location. The worker-adapter case grows from 59 to 65 assertions, including three fresh-interpreter load orders and checks for mutable route lists, late HTTP/parser hooks, local identity, comparison, worker budget, reason translation and quiet child logging.

The differential probe records 420 identical scenarios across the source API, candidate wrappers and canonical owner. They cover configuration, HTTP retrieval, synchronous checks, terminal worker reasons and asynchronous adaptation. A mutant bypassing legacy route and policy hooks fails all six new assertions.

Six more scenarios run in 18 fresh Perl processes with real IO::Async loops and the actual AsyncWorker: success, child exception, timeout, signal death, fallback and callback exception. The producer is controlled to avoid live network dependence. Results agree, callbacks occur once, child logging stays quiet and children are collected.

The Owl Post Office now owns fetching the letter. The Book of Admittance still owns reading its version, and the established local desk still knows which copy of Mediabot is running.

🗺️ MB899 — Give country lookup its own atlas room

The next coherent block in Helpers was country lookup. whereis resolves a hostname or IPv4 address and asks country.is for its country. whereis_async runs the potentially blocking work through the existing child/pipe/timer adapter.

Both now belong to Mediabot::External::Country. Helpers retains both entry points and its default exports. Its 98 other function bodies remain byte-identical, and its size falls from 4,137 to 3,848 lines.

The new owner has 339 lines and two optional exports. It can load without Helpers, the External facade or DBI, and loading it makes no request.

Move lookup policy without rewriting the WHOIS conversation

The synchronous function body is unchanged. Invalid or missing input still produces N/A. The existing Undernet hidden-host exception and suffix matching remain. IPv4 handling uses the same Socket operations; hostname handling uses the existing resolver. IPv6 support is not added by moving the implementation.

The country.is request retains HTTPS, explicit certificate verification and its three-second HTTP timeout. Response-shape, JSON and country-value guards retain their existing treatment of missing or structured data.

The WHOIS command continues capturing its original caller, requested nickname and reply target before asynchronous completion. Formatting and delivery remain in their established owners. A private request still follows the existing private reply route.

The asynchronous function changes only two internal lookup calls to a compatibility relay. That relay uses the historical Helpers hook when present, otherwise the canonical lookup. Its original child process, pipe, timers, wait handling and termination logic travel with it.

Timeout normalization retains the five-second default and 0.1–15 second bounds, with 0.2 seconds between TERM and escalation. The child retains its 128-character truncation and existing encoding; the parent retains at most 256 bytes. The refactor does not reinterpret those byte and character policies or add a new decoder.

Catch a Perl prototype trap before delivery

Socket and JSON symbols historically imported into Helpers can be replaced by callers or fixtures. The new owner keeps that late resolution too.

During qualification, a new hook assertion exposed a subtle forwarding mistake. A prototyped JSON decoder called with @_ through an ordinary named relay could receive the array’s element count instead of the original JSON argument. The lookup might then fall back to N/A, hiding the extraction defect behind an otherwise legitimate failure value.

The relay now forwards the original argument list through an ampersand call. Fresh load-order tests exercise both the direct JSON path and the historical Helpers hook. Reintroducing the faulty forwarding in a separate mutant causes three failures out of thirty assertions; the candidate passes all thirty.

This was an extraction-time defect caught and corrected before delivery, rather than a newly claimed fix to the original lookup policy. The test needed a successful country result and observed arguments to distinguish correct forwarding from a plausible fallback.

Record the worker’s existing limit honestly

MB899 keeps the bespoke asynchronous lifecycle. It does not migrate this adapter to AsyncWorker during the ownership change.

Native qualification exposed an existing limitation in the local execution environment: even a producer returning FR could end with N/A. Diagnostic tracing showed the pipe contained the expected text, but the explicit wait reported no child. Pipe closure collecting the child is a likely cause; it was not established as a complete root-cause diagnosis.

The reference and candidate show the same observation. It remains documented and unfixed in this round. The permanent native assertions check timeout recovery, including a producer ignoring TERM, single callbacks and child collection. Positive synchronous and no-loop results are checked separately.

A green full suite therefore does not establish successful live country delivery under every scheduling order. The new owner makes a future lifecycle repair easier to isolate, while this extraction keeps its behavioural claim bounded.

Five existing cases follow Country’s implementation. The old async case replaces its extracted-and-compiled copy of the function with calls to the loaded Helpers API, so its runtime checks exercise the implementation actually used by callers. Six additional assertions cover load orders, late hooks and the two native timeout paths.

The external comparison records 477 identical scenarios: synchronous host/HTTP/JSON combinations, resolver and request errors, late hooks, no-loop results, timeout options and callback boundaries. Ten further scenarios run in 30 fresh Perl processes, comparing real child/pipe/timer behaviour through the reference, wrappers and direct owner. Each completes once and leaves no uncollected child according to the final wait check.

🛡️ Move the security inspection with the code

One security invariant explicitly checks that the country.is HTTPS call verifies certificates. Once the implementation leaves Helpers, an inspection still reading only Helpers would be looking in the wrong room.

The actual audit now reads Country for that same expression. Its TLS requirement and failure behaviour remain. An initial stale-owner rejection during development was corrected by following the implementation, without relaxing the invariant or reducing the audit matrix.

Qualification challenges that change with three altered trees: certificate verification disabled, Country absent, and Country absent while the old valid-looking implementation is present in Helpers. All three produce NO-GO for the country.is invariant. The healthy tree passes all 37 checks.

The last mutation matters particularly after a refactor. An obsolete copy of good code must not reassure an audit when the implementation reached by current callers has gone missing.

đź§Ş More useful evidence, with the test-file catalogue held steady

Each round adds six focused assertions to an existing case. The generic module checks add another three assertions per new module. That accounts for 18 focused assertions plus nine automatic checks: 27 more assertions overall.

Milestone Existing case extended Focused assertions before → after Added numbered files
MB897 Nickname flood and compatibility 9 → 15 0
MB898 Version worker adapter 59 → 65 0
MB899 Asynchronous country lookup 24 → 30 0

The larger differential probes remain qualification tools. Their 1,004 scenarios across the three rounds are not 1,004 permanent test additions. They exercise the actual modules and compare state and effects beyond a visible reply: bytes, UTF-8 flags, configuration, logs, notifications, errors, callbacks and process outcomes.

Fresh interpreters check load order and isolate native lifecycle probes from unrelated process-global state. Mutation checks establish that the assertions reject deliberately broken compatibility routes. Source-body comparisons identify which surrounding implementations remain untouched.

Together those checks support the refactor more strongly than a catalogue of successful example commands. They remain sampled evidence, with documented adaptations and limits, rather than a proof covering every Perl value, extension or scheduling interleaving.

Local full-suite results are retained honestly. The updater rehearsal MB786 fails in the local environment on reference and candidate runs. For MB897 and MB899, the candidate full run also records a Doctor process-identity failure that did not appear in the corresponding reference full run. Those full-run failure sets are not described as identical.

Separate diagnostics reproduce the process-identity limitation with both codes: a numeric PID can be alive according to kill 0 while its numeric /proc entry is unavailable in that execution view. The unchanged Doctor cannot extract an owner from an absent process directory. Separate case runs and the actual probes help distinguish that environmental observation from the refactor’s behaviour.

No failing case was excluded, cached as passed or waived on the target host. All three delivered installation paths still require the complete unfiltered suite.

📜 The O.W.L. ledger

The reported development-host runs all pass their complete suites, followed by security GO37 and startup integrity. The catalogue remains at 1,156 test files throughout.

Milestone Main change Targeted assertions passed Full-suite assertions passed Full elapsed
MB897 IRC command-admission ownership 1,108 / 1,108 30,233 / 30,233 186 s
MB898 Remote VERSION ownership 503 / 503 30,242 / 30,242 187 s
MB899 Country lookup ownership and TLS-audit location 416 / 416 30,251 / 30,251 187 s

The last article ended at 30,224 passing assertions. This chapter ends at 30,251, explaining the 27-assertion increase without adding a numbered test file. The targeted groups overlap and differ in scope; their counts are not a sum of distinct coverage.

The recorded times are observations from successive runs. These rounds do not establish a speedup. Moving source into clearer owners and exercising failure paths are the measured work here.

Installation remains a separate part of qualification. Guarded packages check expected contents before writing, preserve original bytes, modes and absent-file states, and reject incompatible edits. Local rehearsals exercise real application, payload verification, repeat application without rewriting, and guarded restoration. Restoration is checked again in a subsequent command.

The normal server path requires syntax checks, the targeted group, perl t/test_commands.pl --progress, security and startup integrity. Local environment limitations do not weaken that path. These are accepted development-host validations, with no claim of a new stable release or deployment to every production instance.

🏡 Three new rooms, with the old doors still connected

The change to Helpers can now be read as a sequence of responsibilities finding their owners:

Stage Helpers lines Newly focused owner
After MB896, the previous article’s endpoint 4,722 Version grammar already belongs to Version::Format
After MB897 4,530 IRC::CommandThrottle
After MB898 4,137 Version::Remote
After MB899 3,848 External::Country

That is 874 fewer lines in Helpers, with thirteen established entry points still available there. The new owners include dependency bridges and compatibility code, so the reduction in Helpers is not presented as a reduction of the same size in total project code.

The useful change is where the next maintainer can look. Command admission has a dedicated state-and-policy owner. Remote version transport sits beside the version domain while sharing its existing worker. Country lookup has a named external-service owner and an explicit record of its lifecycle limit.

Late hooks still reach their established implementations. State stays with the bot that owns it. Local identity, WHOIS replies, outgoing delivery and database work retain their existing boundaries. The security audit follows the moved code, and the full development-host suite remains green.

There is still work behind the atlas room’s door. It now has a precise address, a recorded observation and tests defining what must survive the next repair. The castle is becoming easier to maintain one responsibility at a time. 🦉

You must be logged in to reply.