Forum teuk.org

🧪 Protego Testum: a Stronger Test Framework for Mediabot v3

in Mediabot · started by TeuK · 1w ago

TeuK · 1w ago

🧪 Protego Testum: a Stronger Test Framework for Mediabot v3

A new quality milestone has been reached for Mediabot v3.

After the recent work on the Partyline, DCC CHAT, CTCP CHAT, persistent bans, timed kickbans, and ban levels, the project now has a much stronger test framework to keep those features stable.

The goal is simple: stop relying only on manual IRC testing and logs, and start catching regressions earlier.


What changed

The test runner has been improved:

perl t/test_commands.pl --verbose

It now gives a clearer view of the test suite, with grouped test files and readable [OK] output.

Current result:

PASSED : 243/243

That is an important checkpoint.


Test files now covered

The current test suite covers:

01_context.t
02_dispatch.t
03_auth.t
04_mockbot.t
05_metrics.t
06_channel_ban.t
07_partyline_unit.t
08_auth_module.t

This gives coverage over several important Mediabot areas:

command context
public/private dispatch
authentication levels
mock bot behavior
metrics
persistent channel bans
Partyline internals
authentication module

Channel ban tests

A new 06_channel_ban.t test file validates the sensitive logic around the new persistent ban system.

It tests:

parse_duration()
parse_ban_level()
validate_mask()
normalize_mask()
mask_from_hostmask()
looks_like_duration()
looks_like_level()
min_ban_level()

This matters because ban handling is dangerous if done badly.

The tests now check that broad masks are refused:

*!*@*
*!*@*.*
*

and that valid masks are accepted:

*!*user@hostname.com
*!*user@192.168.1.1
*!*user@*.wanadoo.fr
nick!user@host

This is exactly the kind of logic that should be tested before touching a live IRC channel.


Partyline tests

A new 07_partyline_unit.t test file covers important Partyline internals.

It checks:

nick display formatting
telnet IAC stripping
DCC offer keys
DCC offer registration
pending offer detection
offer removal
broadcast behavior
partyline port handling

This is useful after the recent DCC and CTCP CHAT work.

It helps protect the Partyline from regressions in the code that manages DCC offers and connected sessions.


Auth module tests

A new 08_auth_module.t test file validates basic behavior of the authentication module.

It checks:

module construction
AUTOLOGIN method presence
invalid credential handling
undefined argument handling
verify_credentials availability

This is still a modest test, but it creates a clean starting point for extending authentication coverage later.


MockBot and MockIRC improvements

The test framework uses mocks to avoid needing a live IRC connection for every test.

The mocked objects can capture:

privmsg replies
notice replies
raw IRC messages
user levels
authenticated/non-authenticated users
channel/private messages

This makes it possible to test more command behavior safely.

It also prepares the project for the next step: testing actual ban commands such as:

ban
kickban
kb
unban
bans

without requiring a real IRC server for every regression check.


Why this matters

Mediabot v3 is becoming more complex.

It now includes:

DCC CHAT
CTCP CHAT
Partyline login
password masking
persistent channel bans
timed bans
kickban
ban levels
automatic expiration
mbweb
Prometheus metrics

That is a lot of moving parts.

A better test framework gives the project a safety net.

It makes it easier to refactor, add features, and fix bugs without accidentally breaking something that was working yesterday.


Current result

The current test run reports:

PASSED : 243/243

This is a strong checkpoint.

It does not mean the project is perfect, but it means the core pieces now have a much better safety net than before.


Next useful step

The next logical test improvement would be a dedicated command-level test file:

09_channel_ban_commands.t

It could test:

user level 74 cannot ban
user level 75 can ban
dangerous masks are refused
ban level above actor level is refused
ban sends MODE +b
kickban sends MODE +b then KICK
unban sends MODE -b
bans lists active bans

That would move the test suite from helper-level validation to command-level regression protection.


Commit

Suggested commit message:

🧪 Protego Testum: strengthen Mediabot command and Partyline test coverage

Mediabot v3 now has a better foundation for safe future changes.

You must be logged in to reply.