Commit: d6585dc
Version: 3.4dev-20260818_032740
MB661 completes an important part of the validation-efficiency roadmap: Mediabot now has a deterministic fast validation lane that is deliberately different from — and explicitly not equivalent to — the full test suite.
The goal is not to make tests disappear.
The goal is to stop paying the full 14–15 minute cost after every small development round while preserving a strong global checkpoint.
--fast validation modeThe test runner now supports:
perl t/test_commands.pl --fast
The final reviewed lane contains:
303 / 726 test files
It combines:
all eligible primary PURE tests
+ mandatory cross-cutting sentinels
- explicitly profiled slow cases
The result is deterministic and documented.
PURE does not mean quickThe first MB661 pass selected 317 files.
It worked:
PASSED : 5619/5619
but still took:
427s
The profiler showed why.
Some tests classified as dependency-light were still very slow:
612_mb394_trivia_rate_limit_retry.t ~55s
239_usercommands_wave4_subs.t ~27s
236_usercommands_karmahist.t ~24s
707_mb497_seen_enriched.t ~20s
190_seen_respects_channel_scope.t ~20s
192_seen_persisted_respects_channel_scope.t ~20s
So MB661 was adjusted based on measured data instead of assuming that classification and speed were the same concept.
The final fast lane has an explicit manifest of profiler-confirmed slow tests.
This is not a hidden skip mechanism.
The exclusions are:
reviewable
deterministic
documented
covered by the full suite
And there is an important precedence rule:
mandatory sentinel > slow exclusion
So if an essential cross-cutting contract is slow, it still remains in --fast.
For example:
556_mb335_test_runner_contract_isolation.t
remains mandatory even though it costs roughly ten seconds.
MB661 keeps 11 cross-cutting sentinel tests in the fast lane.
They cover areas such as:
test-runner crash protection
TAP/process isolation
dispatch/export integrity
module structure
MB650 profiler contract
MB660 classifier contract
MB661 fast-lane contract itself
The sentinel list is fail-closed.
If an expected sentinel is renamed or removed, the runner refuses to silently execute a weakened fast lane.
--fast is not the full suiteThis is a core part of the contract.
MB661 explicitly documents and reports that:
--fast != full suite
The default command remains unchanged:
perl t/test_commands.pl
and still means the complete suite.
MB661 introduces no automatic replacement of full validation.
After the profiler-informed tuning, the real server produced:
Profiled 303 test file(s)
PASSED : 5175/5175
Runtime:
177s
That is approximately:
2 minutes 57 seconds
The latest complete suite had taken approximately:
872 seconds
So the new development-validation lane is nearly five times faster while still exercising more than five thousand assertions and the mandatory cross-cutting contracts.
The intended development workflow is now:
Small development round
syntax checks
+ source guards
+ targeted regression tests
+ --fast
Cross-cutting checkpoint
full suite
Stable release checkpoint
full suite mandatory
The fast lane is therefore an additional layer, not a weaker definition of “all tests passed”.
The work also exposed a broader test-engineering issue.
More tests are not automatically better if their cost becomes high enough that developers avoid running them.
Likewise, removing tests merely to improve a timing number would be the wrong optimisation.
MB661 instead starts separating three questions:
What dependency does this test have?
How much does this test cost?
When does this test need to run?
Those are different properties.
That distinction should guide future work on the suite.
Representative runner and fast-lane regression:
PASSED : 271/271
RC=0
Corrected structural guard:
Fast lane : 303/726
Sentinels : 11
Slow exclusions : 15
OK: default full-suite behaviour remains untouched
OK: 11 cross-cutting sentinels are fail-closed
OK: 15 profiler-confirmed slow cases are explicit
OK: mandatory sentinels override slow exclusions
OK: dominant Trivia timing test is outside default --fast
OK: mandatory runner sentinel remains inside --fast
OK: no --jobs/--parallel executor exists
OK: README documents PURE != quick and --fast != full
MB661_SOURCE_GUARD=OK
RC=0
Final real-host fast lane:
PASSED : 5175/5175
RC=0
Runtime: 177s
A full suite was intentionally not rerun for this commit.
MB661 introduces:
no database schema change
no migration
no runtime configuration change
no service change
no systemd change
no production deployment change
no parallel test executor
Committed scope:
CHANGELOG.md
README.md
VERSION
t/test_commands.pl
t/lib/FastValidation.pm
t/cases/843_mb661_fast_validation_lane.t
d6585dc
⚡ Give Mediabot a Fast Validation Lane Without Weakening the Full Suite
Version:
3.4dev-20260818_032740
MB650 taught us where the suite spent its time.
MB660 taught us what kind of tests we had.
MB661 finally gives us a practical way to validate everyday changes without waiting a quarter of an hour every time — while keeping the full suite exactly where it belongs.
You must be logged in to reply.