Hey everyone,

I’ve spent the last few sessions building a proper test framework for Mediabot v3, and I wanted to share what’s now in place because it covers quite a bit of ground.

Phase 1 — Unit test framework (no IRC required)

The first phase is a fully offline test suite. It uses a set of Perl mocks (MockBot, MockIRC, MockMessage, MockUser) to simulate the bot environment without connecting to any IRC server or database.

It currently runs 115 tests across 4 files and covers:

• Context.pm and Command.pm logic
• mbCommandPublic / mbCommandPrivate dispatch table
• Auth hierarchy (Owner / Master / Administrator / User)
• Mock framework integrity

Running it is straightforward:

perl t/test_commands.pl
perl t/test_commands.pl --verbose
perl t/test_commands.pl --filter 03

Phase 2 — Live IRC end-to-end framework

The second phase goes much further: it spins up a real bot subprocess connected to an IRC server, alongside a spy client that monitors the channel and checks the bot’s responses.

Here’s what test_live.pl does automatically on every run:

1. Checks IRC connectivity
2. Creates a dedicated mediabot_test MariaDB database and loads the full schema with reference data
3. Generates a temporary test.conf from a template
4. Connects the spy client, joins the test channel
5. Forks and launches the actual mediabot.pl process with the test config
6. Waits for the bot to join the channel
7. Runs the test suites
8. Tears everything down (SIGTERM the bot, drop the DB, disconnect the spy)

The test suites currently cover:

• 01_connect.t — bot joins the channel, responds to !version, WHOIS confirms the nick
• 02_commands.t — !version, !date, !help, !whoami without auth, survival after unknown command
• 03_auth_live.t — !die without auth is refused, login with unknown user is refused, !whoami notice without auth, !die without sufficient level is refused, bot still alive after all that

All 13 tests pass in about 22 seconds against a local ircu server.

[ 01_connect.t ]   3/3
[ 02_commands.t ]  5/5
[ 03_auth_live.t ] 5/5
============================================================
PASSED : 13/13  (22s)
============================================================

A notable bug fixed along the way

While setting up the live framework, I discovered that populateChannels() in Mediabot.pm was not including auto_join in its SELECT query — meaning the field was never passed to Channel->new(), so get_auto_join() always returned undef and the bot never auto-joined any channel on startup. This was silently broken. Fixed by adding auto_join to the SELECT and to the Channel->new() call.

Running it yourself

The live framework is designed to work against a local ircu server for speed and reliability (but the default is a test on Libera) :

perl t/test_live.pl --server localhost --port 6667 --channel '#mbtest' --verbose

or with default settings :

perl t/test_live.pl --verbose

It requires sudo mysql access via Unix socket (no password needed) and all the usual Perl deps from ./configure.

Full documentation is in the updated README.md.

Cheers, TeuK

Fixing Mediabot v3 on Libera.Chat: Net::Async::IRC CAP Negotiation and the is_connected Race Condition

Background

Mediabot v3 uses Net::Async::IRC as its IRC backend, running inside an IO::Async event loop. It has been working flawlessly on Undernet (ircu) for years. When we built a live IRC test framework to validate commands against a real server, everything passed 13/13 on the local ircu instance — but the same tests would systematically fail on Libera.Chat, with the bot never joining the target channel.

Symptoms

Running the test suite against Libera.Chat:

Waiting for bot to join ##mbtest (timeout=45s)...
ERROR: Bot did not join ##mbtest within 45s.

The bot log showed a normal connection sequence — ident check, MOTD, on_login() callback firing, “Trying to join ##mbtest” — but no “[LIVE] * Now talking in ##mbtest”, and the process exited cleanly (exit code 0) about 5-6 seconds after login.

Manual testing with a raw Perl socket confirmed that joining ##mbtest on Libera works fine with no restrictions (channel modes +Cnst, no +r or +i). The problem was specific to how Mediabot interacted with Net::Async::IRC on a Solanum ircd.

Root Cause

mediabot.pl runs an IO::Async::Timer::Periodic with a 5-second interval that starts in on_login(). Its first tick arrives 5 seconds after login. The tick callback checks:

unless ($irc->is_connected) {
    if ($mediabot->getQuit()) {
        $mediabot->clean_and_exit(0);
    } else {
        $loop->stop;   # <-- triggers reconnect logic
        ...
    }
}

On ircu (Undernet), there is no CAP negotiation. The connection is established immediately, is_connected returns true from the first tick onward, and everything works.

On Libera.Chat (Solanum), the server initiates a CAP negotiation exchange at connection time. Net::Async::IRC handles this transparently, but internally is_connected returns false during or just after the CAP handshake — even though on_login() has already fired and the bot has sent the JOIN command.

The result: the first timer tick fires at T+5s, is_connected is still false, getQuit() is 0, so the code calls $loop->stop — which cleanly terminates the event loop. The process exits with code 0, no error is logged, and the JOIN confirmation from the server is never processed.

The JOIN was actually sent and Libera had accepted it — but the bot was already gone before receiving the server’s response.

The Fix

A 15-second grace period after on_login() is all that is needed. During this window, the is_connected check is skipped entirely, giving Net::Async::IRC time to complete the CAP negotiation and stabilize its internal state.

setConnectionTimestamp(time) is already called in on_login(), so we simply use it:

# In on_timer_tick(), before the is_connected check:

# Grace period of 15s after login to let Net::Async::IRC finish CAP negotiation
my $grace = (time - ($mediabot->getConnectionTimestamp() // 0)) < 15;
unless ($grace || $irc->is_connected) {
    if ($mediabot->getQuit()) {
        $mediabot->{logger}->log(0, "Disconnected from server");
        $mediabot->clean_and_exit(0);
    } else {
        $loop->stop;
        ...
    }
}

Additionally, the $login->get call (which blocks until the login Future resolves) was wrapped in an eval block to surface any Future failures in the log rather than silently dying:

eval { $login->get };
if ($@) {
    my $err = $@; $err =~ s/\n/ /g;
    $mediabot->{logger}->log(0, "Login Future failed: $err");
    $mediabot->clean_and_exit(1);
}

Two minor fixes were also made to the test framework (test.conf.tpl):

• RADIO_PUB=900 and RANDOM_QUOTE=900 to suppress spam-guard warnings during tests
• test.conf is now preserved alongside the database when running with --keep-db

Results

After the fix, the live test suite passes 13/13 on both servers:

# Libera.Chat
perl t/test_live.pl --verbose
PASSED : 13/13  (8s)

# Local ircu (Undernet)
perl t/test_live.pl --verbose --server localhost --port 6667 --channel '#mbtest'
PASSED : 13/13  (21s)

Takeaway

Net::Async::IRC’s is_connected method does not necessarily return true immediately after on_login() fires on servers that perform CAP negotiation (e.g. Solanum/Libera.Chat). If your bot runs periodic connectivity checks starting from on_login, add a short grace period to avoid a false-negative disconnect on modern IRC networks.