The previous chapter brought Mediabot closer to a future 3.3 release: private data stayed out of the logs, version reporting became deterministic, reconnects survived a second outage, PID ownership became atomic, core commands stopped flooding IRC, and the CPAN-only installer finally included the real MariaDB driver.
Then one very small command refused to cooperate:
m trivia
Instead of a question, the channel received only:
Trivia: could not fetch question.
That message was technically true and operationally useless.
The failure could have been anywhere between Open Trivia DB, DNS, TLS, the HTTP client, JSON parsing, the forked worker, the pipe, process collection or the final IRC callback. Mediabot did not know which door had jammed — and neither did its administrator.
MB394 through MB396 turned that opaque failure into a bounded, observable and working asynchronous pipeline.
The project remains on the 3.2dev line.
The stable 3.3 release has not been declared.
As always:
No database schema was changed.
Open Trivia DB may return API response code 5 when several requests share the same public IP inside its five-second window. An HTTP 429 can represent the same condition.
The old code collapsed that situation into the generic failure shown above.
MB394 added one bounded retry:
response_code 5 or HTTP 429
wait slightly longer than five seconds
retry once
never sleep in the IRC event loop
The wait happens inside the child worker, not inside the main bot process.
The final result now distinguishes:
rate limiting
HTTP failure
invalid API response
JSON parsing failure
The channel also receives a useful rate-limit message instead of being told only that something went wrong.
This solved one real API behaviour, but the live command still failed.
The Trivia worker ran in a child process while the parent used IO::Async.
Its completion path still polled the PID with:
waitpid($child_pid, POSIX::WNOHANG())
That gave two pieces of code an interest in collecting the same child:
IO::Async process handling
Trivia's manual waitpid polling
A valid worker result could therefore be lost if the loop collected the PID first and the Trivia poll later saw no child to reap.
MB395 replaced the polling with:
$loop->watch_process($child_pid, sub { ... })
It also added correlated diagnostics carrying only bounded metadata:
channel
requesting nick
token
worker PID
stage
HTTP status
API response code
payload byte count
elapsed time
exit code
signal
Question text, answer text and raw remote JSON are never written to the logs.
The first live test after MB395 finally exposed the next problem:
trivia worker start
24 seconds without completion
TERM sent
KILL sent
At the same time:
m check
still answered normally.
That observation mattered. The main IRC loop was healthy. Only the Trivia child and its completion path were stuck.
MB395 still used the special Perl form:
open(my $pipe, '-|')
while also asking IO::Async to watch the resulting child PID.
MB396 removed that ambiguous ownership model.
The worker now uses three explicit operations:
pipe(...)
fork()
$loop->watch_process(...)
Responsibilities are clear:
ordinary pipe → transports worker records
IO::Async stream → reads those records
watch_process → owns child collection
The child no longer stays silent until the final JSON result.
It emits bounded progress records such as:
http_client_start
http_client_ready
http_get_start
http_get_done
http_get_timeout
rate_limit_wait_start
rate_limit_wait_done
api_parse_start
api_parse_ok
api_parse_failed
The parent records the last stage together with safe metadata:
attempt
elapsed milliseconds
HTTP status
content bytes
response code
parse class
retry delay
This makes a future failure diagnosable without leaking Trivia content.
The HTTP client timeout alone did not provide a reliable wall-clock guarantee for every possible stall.
MB396 adds a hard deadline around each complete request with Time::HiRes::alarm().
A blocked request becomes an explicit result:
error=http_timeout
stage=http_get
last_stage=http_get_timeout
The public command receives:
Trivia: the question service request timed out. Details were logged.
The previous timeout path still waited for child_done before invoking the command callback.
The live logs proved that this notification could fail to arrive even after TERM and KILL.
The new sequence is:
outer timeout expires
TERM is sent and delivery is logged
KILL follows after 0.5 second if required
forced finalisation follows after another 1.5 seconds if required
The pending Trivia request is always cleared.
The channel always receives a final result.
A failed child can no longer leave m trivia permanently occupied or vanish after two signal lines.
The child resets inherited handlers for:
TERM
INT
HUP
A timeout signal can therefore terminate the child normally instead of accidentally entering the parent bot’s shutdown handler inside the forked process.
The first MB396 installer generated code with the pipe handles declared inside the unless condition:
unless (pipe(my $pipe, my $child_write)) {
Under use strict, those lexicals were not visible to the later fork branches.
The mandatory syntax check rejected the patch before any restart:
Global symbol "$pipe" requires explicit package name
Global symbol "$child_write" requires explicit package name
The automatic rollback restored the previous working tree.
The corrected revision declares the handles in the enclosing scope:
my ($pipe, $child_write);
unless (pipe($pipe, $child_write)) {
This was not a runtime regression. It was a failed installation safely stopped by the guardrails that were designed for exactly this situation.
The final focused Trivia selection reports:
154/154
It covers:
existing Trivia command guards
asynchronous worker behaviour
word-boundary answer matching
Open Trivia DB rate-limit retry
IO::Async process ownership and diagnostics
progress protocol
hard HTTP deadline
forced timeout completion
The corrected installer also completed its real syntax check on the development server before the bot was restarted.
The decisive live result is simple:
m trivia works again
The bot remained responsive throughout the investigation, and the final implementation now provides enough stage information to diagnose a future network or API failure precisely.
The validation groups from MB394, MB395 and MB396 overlap and must not be added into one artificial total.
Mediabot/UserCommands.pm
t/cases/541_mb319_trivia_async_fetch.t
t/cases/612_mb394_trivia_rate_limit_retry.t
t/cases/613_mb395_trivia_process_watch_diagnostics.t
t/cases/614_mb396_trivia_stage_protocol_deadline.t
The older MB319 test was realigned with the explicit pipe() + fork() contract.
None.
0 new tables
0 altered columns
0 migrations
0 configuration changes
0 changes to commit.sh
0 declaration of version 3.3
The current public line remains:
3.2dev-*
Mediabot Trivia now:
IO::Async sole ownership of child-process collection;The Goblet was not empty.
It was waiting behind a rate limit, a child-process ownership dispute and a timeout path that never finished its spell.
Now the worker has one collector, every stage leaves a safe footprint, every timeout has an exit, and the question finally reaches the channel.
You must be logged in to reply.