Forum teuk.org
Login Register

🪄 Mischief Managed! — Mediabot v3 Goes Green Across Static and Live Tests

in Mediabot · started by TeuK · 1mo ago

TeuK · 1mo ago

Overview

This one feels good.

After a long series of hardening rounds, schema cleanups, live-test fixes, Debian 13 compatibility work, and a few very stubborn edge cases, Mediabot v3 is now passing the full static test suite and the full live IRC test suite.

No hand-waving. No “it starts on my machine”. No “the bot seems fine”.

This round validates the project from several angles:

  • static regression tests;
  • live IRC behavior against a real local IRC server;
  • database schema consistency;
  • fresh-install readiness;
  • Partyline restart behavior;
  • command dispatch stability;
  • migration documentation;
  • Debian 13 compatibility.

Mischief managed. The bot lives, answers, restarts, and survives the tests. ✨

✅ What passed

The latest full validation run reached the target we wanted:

100% of the relevant static and live tests passed.

The live test suite validates real bot behavior, not just isolated code paths. During the run, Mediabot successfully:

  • connected to the local IRC server;
  • joined the test channel;
  • answered public commands;
  • handled NOTICE-based help output;
  • authenticated a test operator;
  • processed private commands;
  • handled user and channel commands;
  • added, searched, displayed and deleted quotes;
  • executed database-backed commands;
  • survived unknown commands;
  • performed an IRC restart through Partyline;
  • kept Partyline alive during restart;
  • reconnected to IRC;
  • answered again after reconnect.

That last part matters a lot: the bot is not merely passing static checks, it is behaving correctly as a long-running IRC service.

🧪 Static tests: cleaned and aligned

Several static tests were failing not because the bot was broken, but because the tests had fallen behind the code.

This round fixed that properly instead of weakening coverage.

Claude / Partyline callback tests

The Claude callback test now checks the right file for the Partyline command implementation.

_cmd_ai lives in:

Mediabot/Partyline.pm

not in:

Mediabot/External.pm

The test now reflects the actual architecture:

  • claudeAI() lives in External.pm;
  • Partyline command routing lives in Partyline.pm;
  • the callback-based output path remains verified;
  • the old monkey-patching behavior remains banned.

Karma history test

The karmahist regression test was also adjusted to match the real implementation.

The code correctly keeps a per-channel in-memory ring buffer capped at 20 entries:

splice @$klog, 0, @$klog - 20 if @$klog > 20;

The test now validates that behavior directly instead of looking for an obsolete textual shape.

TMDB and YouTube tests

A few media-related tests were updated to match recent improvements:

  • TMDB search now passes the logger into get_tmdb_info();
  • YouTube search result count is now configurable through main.YT_SEARCH_RESULTS;
  • the default remains 3 results;
  • the configured value is guarded to a safe range.

That keeps the tests meaningful while allowing the implementation to evolve.

🛰️ Live tests: made robust instead of lucky

The most important work was in the live test suite.

The bot had already been behaving correctly, but the tests were still assuming older routing behavior in a few places.

Help output now uses NOTICE

Mediabot now routes help output by NOTICE to avoid flooding channels.

The live tests were still expecting some !help responses as public PRIVMSG channel output. That was wrong.

The tests now validate the intended behavior:

!help → NOTICE to the requesting nick

This matches how the bot actually behaves and keeps public channels cleaner.

Multi-line replies are drained properly

Commands such as:

!q
userinfo
whoami
help foobar

can produce multiple NOTICE lines.

The live tests now drain those lines properly before moving to the next assertion. This prevents one command’s leftover output from polluting the next test.

That was one of the main causes of false negatives.

Partyline restart is now validated correctly

The Partyline restart test now confirms the full restart cycle:

  1. connect to Partyline;
  2. authenticate;
  3. issue .restart;
  4. observe the bot leaving IRC;
  5. verify Partyline remains alive;
  6. verify Partyline still answers commands;
  7. wait for IRC reconnect;
  8. verify the bot answers again in-channel.

This is a serious runtime test. It validates restart behavior in conditions close to real operation.

🧬 Schema and migrations: no more silent drift

The project now has a cleaner database story.

The reference schema is:

install/mediabot.sql

Existing installations should use migrations from:

install/migrations/

and validate the result with:

perl tools/check_schema_drift.pl --conf=mediabot.conf --strict

The checker was also made more Debian 13 friendly:

  • supports DBD::mysql when available;
  • falls back to DBD::MariaDB when that is what Debian provides;
  • avoids invalid port= usage with DBD::MariaDB and localhost;
  • handles charset through SET NAMES utf8mb4 instead of fragile driver-specific attributes.

This means the schema drift checker now works in the kind of environment a fresh Debian 13 install actually provides.

🧱 Live test schema: one source of truth

The old live-test schema file has been removed from the workflow.

Instead of maintaining a second schema just for tests, live tests now load:

install/mediabot.sql

then apply a small fixture file:

t/live/fixtures.sql

That is the right model:

  • the real schema stays the single source of truth;
  • fixtures contain only test data;
  • new tables automatically reach the live test environment;
  • schema drift between install and tests becomes much harder.

This already helped with recent additions such as:

  • REMINDERS;
  • BOT_ALIAS;
  • KARMA;
  • USER_SEEN;
  • CHANNEL_BAN;
  • the Claude chanset.

🧙 Claude / AI integration status

The Claude integration is now better framed for real use:

  • [anthropic] is documented in mediabot.sample.conf;
  • the Claude chanset exists in the main schema;
  • a migration exists for existing databases;
  • Partyline output uses a callback instead of monkey-patching;
  • rate limiting and prompt/persona behavior are covered by tests.

This prevents the AI command from being a mysterious hidden feature. It is now part of the documented configuration and schema model.

📦 Files and areas touched

This round touched several important areas:

Area Result
t/cases/ Static tests aligned with current code
t/live/ Live tests made robust against NOTICE output and multi-line replies
t/test_live.pl Uses the real schema plus fixtures
tools/check_schema_drift.pl Debian 13 / MariaDB compatibility hardened
install/mediabot.sql Schema aligned with current features
install/migrations/ Existing DB upgrade path documented
mediabot.sample.conf Anthropic / Claude configuration documented
Partyline restart test Validates real restart and reconnect behavior

🧾 Why this matters

This is more than “some tests passed”.

For a long-lived IRC bot, the hard parts are not always the flashy features. The hard parts are:

  • clean installs;
  • safe upgrades;
  • predictable database migrations;
  • avoiding schema drift;
  • handling reconnects;
  • keeping admin interfaces alive;
  • preventing tests from lying;
  • making sure a fresh machine can reproduce the setup.

This round moves Mediabot v3 much closer to that standard.

The bot now has a stronger base for future work:

  • less duplicated schema logic;
  • better live coverage;
  • cleaner migration documentation;
  • more realistic Debian 13 support;
  • stronger confidence before release work.

Final note

This was one of those rounds where the last few failures were not dramatic, but they mattered.

A fragile test suite can hide real bugs. A noisy test suite can also waste time chasing ghosts.

Now the tests match the actual bot behavior much better.

Mediabot connects, answers, authenticates, restarts, reconnects, and passes.

Finite Incantatem. Mischief managed. 🪄

You must be logged in to reply.