Forum teuk.org

⚑ Mediabot v3 β€” Safe Parallel Fast Validation, Without Touching the Main Runner

in Mediabot Β· started by TeuK Β· 5d ago

TeuK Β· 5d ago

Commit: 6153cee
Version: 3.4dev-20260818_040611

MB662 closes the current validation-performance chapter with a deliberately conservative experiment: parallelise only the part of the fast lane that is safe enough to overlap, while leaving the main test runner and the serial full-suite contract untouched.

The result is a meaningful speedup without turning the test system into a new source of uncertainty.


πŸ§ͺ The starting point

MB661 introduced a deterministic serial fast lane:

303 test files
5175 assertions
177 seconds

That already reduced everyday validation from roughly 14–15 minutes to about 3 minutes.

MB662 asks the next question:

Can part of that fast lane run in parallel
without weakening the existing contracts?

The answer is yes β€” but only under strict conditions.


βš™οΈ Opt-in parallel pilot

MB662 adds a separate pilot runner:

perl t/fast_parallel.pl --jobs 2

It does not replace:

perl t/test_commands.pl

and it does not change:

perl t/test_commands.pl --fast

The main runner remains byte-identical.


🧭 Source of truth stays in MB661

The pilot does not invent another fast-lane definition.

It asks the existing runner for the selected files using:

--fast --list-selected

That means MB661 remains the authoritative policy for what belongs in the fast lane.

MB662 only decides which of those already-selected files may overlap.


πŸ”’ Conservative split

The reviewed two-job plan is:

Selected fast files : 303
Parallel PURE files : 292
Serial sentinels    : 11
Parallel jobs       : 2

Only non-sentinel tests whose primary class is PURE are eligible for overlap.

The 11 cross-cutting sentinels always run afterward in a separate serial stage.


πŸ›‘οΈ Sentinels always win

Even if a mandatory sentinel would otherwise qualify for parallel execution, it remains serial.

The rule is:

sentinel > parallel eligibility

This keeps runner, classifier, structure and fast-lane contracts isolated from concurrent execution.


🚫 Main validation infrastructure remains untouched

MB662 explicitly leaves these files byte-identical:

t/test_commands.pl
t/lib/FastValidation.pm
t/lib/TestClassifier.pm

Verified hashes:

t/test_commands.pl
4a572f2b57dcb1680974d6c1904c6ae9cac31c141c3243b283969ad3ad18d4f3

t/lib/FastValidation.pm
fa6ee75dd691e3a02e2dd45b478a322245f2e14fdd56c7021b5a1fe7e967fa24

t/lib/TestClassifier.pm
fff3400c1b93e953e8f4d295aa2882e3b36f0a43feea0bba1485358c8e61bab4

So the experiment is operationally isolated from the established serial runner.


πŸ§ͺ Focused validation

The MB660 / MB661 / MB662 regression passed:

PASSED : 227/227
RC=0

The invariants also passed:

OK: committed test_commands.pl remains byte-identical
OK: committed FastValidation.pm remains byte-identical
OK: committed TestClassifier.pm remains byte-identical
OK: parallel pilot reuses MB661 --fast --list-selected as source of truth
OK: only non-sentinel primary PURE files may overlap
OK: all 11 sentinels remain in a separate serial stage
OK: jobs are conservatively limited to 2..4
OK: no shell system() delegation
OK: deterministic reviewed plan = 292 parallel + 11 serial = 303
MB662_INVARIANTS=OK

⚑ Real-host two-job result

The real server produced:

parallel shard 1
  files      = 146
  assertions = 2266/2266
  runner     = 81s

parallel shard 2
  files      = 146
  assertions = 2543/2543
  runner     = 77s

serial sentinels
  files      = 11
  assertions = 366/366
  runner     = 31s

Final result:

Files       : 303
Assertions  : 5175/5175
Jobs        : 2
Wall time   : 111.882s
Verdict     : FAST_PARALLEL_PILOT=OK

πŸ“‰ Practical gain

Reference serial fast lane:

177s

Two-job pilot:

111.882s

That is roughly:

36.8% less wall-clock time

while reproducing exactly the same:

5175/5175 assertions

No coverage count changed.


🧠 Why stop at two workers?

The pilot supports a conservative range:

2 <= jobs <= 4

but MB662 was validated with two workers only.

There was no attempt to chase the smallest possible benchmark number.

The purpose of this round was to prove that safe overlap is possible, not to turn validation into a throughput contest.


🧱 Operational scope

MB662 introduces:

no database schema change
no migration
no runtime configuration change
no service change
no systemd change
no change to the main test runner
no change to serial --fast
no change to the full-suite contract

Committed scope:

CHANGELOG.md
README.md
VERSION
t/fast_parallel.pl
t/cases/844_mb662_fast_parallel_pilot.t

✨ Commit

6153cee
⚑ Pilot Safe Parallel Fast Validation Without Touching the Main Runner

Version:

3.4dev-20260818_040611

MB650 showed us where the time went.

MB660 classified the suite.

MB661 gave everyday development a usable fast lane.

MB662 proves that even that lane can be shortened safely β€” without rewriting the foundations underneath it.

You must be logged in to reply.