Forum teuk.org

🪄 Mediabot v3 — The Final Charms Before 3.3

in Mediabot · started by TeuK · 1mo ago

TeuK · 1mo ago

Development status: still on the 3.2dev-* line
Stable 3.3: not released yet
Release authority: no stable release, tag, or version switch without Christophe’s explicit approval


🏰 A quieter kind of magic

Not every major step toward a release arrives with a spectacular new command.

Sometimes the most important work happens behind the castle walls: fixing the staircases before they move, checking every potion label twice, making sure the Room of Requirement really contains what the map says it contains, and preventing the Sorting Hat from declaring victory while half the database is still missing.

This development cycle did both.

It polished several visible community features, completed the documentation for the future 3.3 release, aligned fresh installations with upgraded databases, and hardened the entire database validation workflow so that Mediabot can no longer mistake an incomplete schema for a healthy one.

The result is not yet Mediabot 3.3.

But it is a much more convincing release candidate.


🧙 Chapter I — The Channel Engagement Spellbook

🎯 !milestone learns some manners

The new channel milestone command was useful, but it could still be invoked repeatedly and trigger unnecessary database work.

A per-user cooldown was added, using the same protective pattern already applied to other recent analytics commands.

The command now behaves politely under repeated use:

!milestone
!milestone
→ milestone: please wait a few seconds before asking again

No change was made to older historical commands such as !stats or !heatmap. Their established behavior remains untouched unless real production abuse proves that a cooldown is necessary.


🏆 !topquote and !halloffame find their proper house

The quote hall-of-fame commands worked, but their help-category routing was wrong.

They could appear under unrelated categories instead of alongside quotes, games, and community fun.

That sorting mistake was fixed.

These commands now live where users naturally expect them:

!help commands ai_fun
!help topquote
!help halloffame

The command behavior itself remains unchanged.


📈 !milestone remembers the last great achievement

The channel milestone display now includes the last milestone already crossed.

Example:

#teuk: 98,750 public messages logged · last passed 95,000

And when the channel lands exactly on a milestone:

#teuk: 100,000 public messages logged · just hit 100,000!

This requires no extra database query. The value is calculated from the total already loaded by the command.

A small detail, perhaps—but one worthy of the Great Hall.


🧹 !recap ai receives an anti-flood charm

The recap command already limited its time window and the amount of raw channel data sent for summarisation.

One final opening remained: an unusually long AI response could still produce too many IRC lines.

The output is now capped:

maximum summary lines: 12

When the limit is exceeded, Mediabot adds a clean truncation notice instead of flooding the channel.

The short summaries remain unchanged.


📚 Chapter II — The 3.3 Library Opens

📜 A real CHANGELOG.md

Mediabot now has a proper changelog for the future 3.3 release.

The section remains explicitly unreleased, because 3.3 has not yet received its final approval.

The changelog documents the major work delivered on the development line, including:

!tell and delayed messages
DidYouMean command suggestions
persistent channel factoids
?keyword shortcuts
offline unit conversion
achievements inside !stats
on-this-day channel history
daily history digests
richer !seen and !mood commands
quote hall of fame
channel milestones
faster and richer URL previews
security and integrity hardening
database migrations and upgrade requirements

It also points readers to the authoritative migration documentation rather than pretending that a short release-note list is a complete upgrade plan.


📖 README improvements

The main README now gives the engagement features proper visibility.

A newcomer can discover that Mediabot is not only a traditional IRC administration bot. It also remembers channel history, stores community knowledge, delivers messages, analyses activity, celebrates milestones, ranks popular quotes, and enriches pasted links.

The database validation examples were updated as well, so the README now reflects the real release workflow.


🧪 Documentation that tests itself

Release documentation is no longer treated as decorative prose.

New regression tests verify that:

CHANGELOG.md exists
the future 3.3 section exists
important user-facing features are mentioned
referenced migration files really exist
README version wording remains coherent
fresh schema and migration expectations agree
database workflow documentation stays aligned

In other words, the library books are now protected by their own enchantments.


🧱 Chapter III — Fresh Install and Upgrade Must Tell the Same Story

🏛️ The QUOTES index mismatch

The quote hall of fame introduced:

QUOTES.hits

and an index designed for ranking quotes efficiently:

idx_quotes_channel_hits (id_channel, hits)

The migration for existing databases included that index.

The fresh-install schema originally included the column but not the index.

That created two possible worlds:

upgraded installation → column + index
fresh installation    → column without index

This inconsistency was corrected.

The official fresh schema and the official migration now describe the same database structure.

No new schema feature was introduced. This was a consistency repair for an already approved migration.


🔍 The database drift checker learns about indexes

The schema drift tool already compared:

tables
columns
column types
required reference data

It now supports:

--indexes

This allows Mediabot to verify required indexes from the reference schema.

Example:

perl tools/check_schema_drift.pl \
  --conf=mediabot.conf \
  --strict \
  --types \
  --indexes

The behavior is deliberately conservative:

missing required indexes are reported
safe ADD INDEX statements may be generated for review
extra DBA-created indexes are ignored
no automatic DROP INDEX is generated
no destructive replacement is generated

The tool was validated against the real development database and successfully identified and added the missing indexes.

Kudos were awarded. Quite rightly.


🗺️ Chapter IV — The Migration Map Stops Lying

⏳ Migrations in real chronological order

The configuration wizard described its migration list as:

oldest first

But the list was sorted only by filename.

One historical migration contains its date near the end of its filename:

mediabot_fun_commands_migration_20260512.sql

A simple lexical sort could therefore place it after migrations from July 2026.

The wizard now extracts the embedded YYYYMMDD date and sorts by:

1. migration date
2. filename

The Marauder’s Map now shows the staircases in the order they actually appeared.


🧭 Consistent index-aware documentation

After index support was added, a contradiction remained in several documents:

command examples used --indexes
nearby prose still claimed that indexes were not checked

The README, database migration guide, migration README, tests, and tool help were aligned.

The documentation now says what the tool really does:

required reference indexes are checked when --indexes is supplied
extra live-only indexes are intentionally ignored

🛡️ Chapter V — The Configuration Wizard Learns to Fail Closed

This was the most important bug found during the final review.

🐍 The subtle return-code trap

The configuration wizard expected the database checker to behave like this:

0 → database clean
1 → drift detected
other → operational error

However, the first call was made without strict mode.

In report mode, the drift checker can display missing columns or indexes and still return success.

That meant the wizard could say:

Database schema is in sync

while the database was not actually in sync.

A missing column, type mismatch, or missing index could therefore be visible in the output but ignored by the control flow.

That is exactly the kind of cheerful but dangerous prophecy Professor Trelawney might deliver.


🔒 MB509 closes the door

The integrated workflow now uses the complete validation dimensions.

Initial check:

perl tools/check_schema_drift.pl \
  --conf=mediabot.conf \
  --strict \
  --types \
  --indexes

Migration-plan generation:

perl tools/check_schema_drift.pl \
  --conf=mediabot.conf \
  --generate-migration \
  --types \
  --indexes

Final check:

perl tools/check_schema_drift.pl \
  --conf=mediabot.conf \
  --strict \
  --types \
  --indexes

If required drift remains unresolved, configure now exits with a failure code.

It no longer completes successfully while quietly leaving the database incomplete.

For configuration-only maintenance where database validation is intentionally not wanted, the explicit escape paths remain available:

./configure --sync-only
./configure --skip-db

This preserves flexibility without silently weakening a normal installation or upgrade.


🕵️ Chapter VI — The Secret Scanner and the Phantom Password

Just before the commit, the local commit assistant stopped the operation with this warning:

tools/check_schema_drift.pl
literal credential assignment
key pass

There was no credential.

The line was a Getopt::Long declaration:

'pass=s' => \$opt{pass},

The scanner had mistaken the = inside the option specification for a literal password assignment.

The local scanner was corrected narrowly:

Getopt::Long declarations are recognised
real PASS=secret assignments remain blocked
private keys and known tokens remain blocked
secret values are never printed

The same local improvement also makes CHANGELOG.md a recommended project file while keeping these protected:

commit.sh
mp3/
live configuration
credentials
logs
backups
snapshots

This tool remains strictly local and is not part of the public repository.

Its mention here is simply part of the development chronicle: even the gatekeeper needed one final pair of spectacles.


✅ Chapter VII — The Closing Tests

The focused release-preparation matrix now protects:

milestone behavior
help-category consistency
recap output limits
CHANGELOG truth
fresh-schema and migration agreement
index-aware drift checks
migration ordering
configure strict-mode behavior
fail-closed database validation
documentation consistency

Focused results reached:

159 / 159

The security audit remained:

GO

The full project suite must still be executed on the real project host, where all required Perl runtime dependencies are installed.

That result—not a reduced analysis container—remains the authority.


🚂 The Next Stop: Debian 13

The feature doors are now closed for this candidate.

No new command, no new URL service, and no new schema evolution should be added before the installation proof.

The next journey is operational:

1. run the complete test suite on teuk.org
2. commit the pre-release hardening
3. generate a fresh private snapshot
4. build the installation artifact from the committed HEAD
5. perform a clean Debian 13 installation
6. validate database tables, types, references, and indexes
7. verify startup integrity with a manifest
8. test Owner registration and authentication
9. test IRC, database, and Scheduler lifecycle
10. run the user-facing smoke-test matrix

The smoke tests include:

!help and categories
!tell
factoids and ?keyword
!convert
!onthisday
!mood
!seen
!topquote
!milestone
!recap ai
X, Facebook, Instagram, and Apple Music URLs
clean start, stop, and restart

🦉 Final Word from the Owlery

Since the previous commit, Mediabot gained visible polish, better historical and community features, stronger documentation, a real changelog, consistent fresh-install and upgrade schemas, index-aware drift checks, chronological migration handling, and a database configuration workflow that finally fails closed.

The stable 3.3 spell has not been cast.

But the wand is inspected, the incantation is documented, the potion bottles are labelled, and the castle doors now lock when something is missing.

That is exactly where a serious release should be before boarding the train.

You must be logged in to reply.