Version: 3.4dev-20260820_054605
Commit: 392c988
Commit message: ⚡ Let the Time-Turner Speed Up the Test Suite Without Losing a Spell
MB674 did not add a user-facing feature.
It did something just as important for the project: it made the complete Mediabot test suite dramatically faster without removing a single test, assertion, or safety check.
Before MB674, the complete test suite was healthy but expensive:
PASSED : 14090/14090 (768s)
That is roughly:
12 minutes 48 seconds
for every full validation cycle.
With more than 700 test files, that cost was becoming a real development bottleneck.
The goal of MB674 was therefore deliberately strict:
same tests
same assertions
same failure detection
no application-code changes
much less time
No parallelism trick, no skipped coverage, no relaxed validation.
The first MB674 pass changed nothing.
The existing suite profiler was used on all:
741 test files
and immediately showed that the runtime was not evenly distributed.
The slowest tests included:
36.418s 541_mb319_trivia_async_fetch.t
35.755s 612_mb394_trivia_rate_limit_retry.t
32.812s 613_mb395_trivia_process_watch_diagnostics.t
24.093s 614_mb396_trivia_stage_protocol_deadline.t
20.522s 239_usercommands_wave4_subs.t
19.283s 535_mb313_partyline_async_reverse_dns.t
18.344s 236_usercommands_karmahist.t
17.457s 335_antiflood_chanflood_public_only.t
16.659s 190_seen_respects_channel_scope.t
16.657s 192_seen_persisted_respects_channel_scope.t
16.656s 707_mb497_seen_enriched.t
Those first eleven files alone represented more than four minutes.
That was the clue.
Many source-contract tests need to extract Perl subroutines from large source files.
A common pattern was:
open source with <:encoding(UTF-8)
read the complete file
scan it character by character
use substr(..., offset, 1)
track { and } depth
This looks harmless.
But large decoded UTF-8 Perl strings combined with repeated character-offset substr() operations are extremely expensive.
Some tests were scanning files such as:
Mediabot/UserCommands.pm
which is several hundred kilobytes long.
The tests were not actually waiting for Trivia, networking or retries.
They were spending most of their time walking decoded Perl source character by character.
Before touching the repository, MB674 tested the hypothesis on a disposable copy.
Only eleven of the worst offenders were changed from:
<:encoding(UTF-8)
to:
<:raw
The result was immediate:
Before : 254.656s
After : 3.000s
with:
PASSED : 236/236
That is:
98.8% reduction
84.89x faster
for that selected group.
A static scan found:
139
test files using the same source-scanner pattern.
A full disposable run with those 139 source reads converted to raw byte reads produced:
WALL_SECONDS=219
instead of:
768
The first disposable run appeared to contain 38 fewer assertions:
14052 instead of 14090
MB674 deliberately stopped there rather than accepting a green RC=0.
The missing assertions were investigated.
The difference had nothing to do with :raw.
The disposable test tree had been created with:
git archive HEAD
and therefore intentionally did not contain the local-only:
commit.sh
One existing test behaves differently depending on whether that private maintainer script exists:
519_mb297_commit_secret_scanner_precision.t
With local commit.sh:
39 assertions
Without it:
1 assertion
Difference:
38 assertions
Exactly the discrepancy observed.
A second disposable test copied the local commit.sh privately into the temporary test tree solely to reproduce the maintainer environment.
Result:
PASSED : 14090/14090 (220s)
Full assertion parity restored.
commit.sh remains strictly local and was never added to Git.
The real MB674 change is intentionally boring.
Exactly:
139 test files
were modified.
Each file contains exactly one semantic change:
- <:encoding(UTF-8)
+ <:raw
Only tests using the proven character-by-character Perl source scanner were touched.
No Mediabot runtime file was changed.
Validation of the patch itself:
FILES=139
NON_TEST_FILES=0
NUMSTAT=OK
Meaning:
139 files changed
0 application files changed
1 line removed + 1 line added per file
All modified tests were executed together:
PASSED : 2130/2130 (7s)
Previously, many of these tests individually took several seconds or even tens of seconds.
After the change, the slowest transformed test was approximately:
0.597s
The final proof was performed on the real modified worktree, not a laboratory copy.
Before:
PASSED : 14090/14090 (768s)
After:
PASSED : 14090/14090 (220s)
So MB674 achieved:
Time saved : 548 seconds
Reduction : 71.4%
Speed-up : 3.49x
In human terms:
~12m48 → ~3m40
No tests were disabled.
No assertions were removed.
No timeout was shortened simply to make the suite look faster.
No application code was modified.
No parallel execution was introduced.
No test classification was treated as proof of parallel safety.
The complete suite still validates:
14090/14090
assertions.
The improvement comes entirely from removing an inefficient way of scanning source files.
Commit : 392c988
Version : 3.4dev-20260820_054605
⚡ Let the Time-Turner Speed Up the Test Suite Without Losing a Spell
Git summary:
140 files changed
140 insertions(+)
140 deletions(-)
The 140th file is the normal VERSION update.
MB674 turns the full test suite from something expensive enough to postpone into something practical to run frequently.
before : 12m48
after : 3m40
Same suite.
Same 14,090 assertions.
Same safety net.
Just a much faster trip through time. ⚡⏳
You must be logged in to reply.