Forum teuk.org

📰 Mediabot v3 — The Daily Prophet Enters the Common Room

in Mediabot · started by TeuK · 2w ago

TeuK · 2w ago

Development chronicle — mb613 to mb616 · Live news search, multilingual summaries and hardened async delivery

Mediabot can now search current news, summarise it through Claude, and answer in the language of the IRC channel.

The feature started as a port of the historical news_teuk.tcl idea, but the implementation now follows the modern Mediabot architecture: Tavily performs the search, Claude produces the short synthesis, CommandAsync keeps network work outside the event loop, and the existing channel-language API decides how the result should speak.

No database schema change is required.


🗞️ A real actualites command

The following forms are supported:

m actualites
m actualites intelligence artificielle
m actualités politique française
m actualite bitcoin
m actu OpenAI
m news cybersecurity

Without a topic, Mediabot searches the important headlines of the day.

With a topic, the query is used directly.

The accented and ASCII spellings deliberately reach the same command.


🌍 The Common Room chooses the language

News uses the same language resolver as ai summary and recap ai.

Conceptually:

explicit en|fr|es / lang=xx
            │
            ▼
channel +LangFR / +LangES
            │
            ▼
main.LANG
            │
            ▼
English fallback

Examples:

m actualites fr
m news lang=en OpenAI
m actualites es elecciones

So a French channel can simply use:

m actualites

and receive the search notices and Claude synthesis in French.

The language policy lives in one shared implementation rather than being copied into the news module.


🔎 Tavily performs the search

News search is configured through:

[tavily]
API_KEY=...

The public sample contains only a commented placeholder. No real API key belongs in the repository.

Mediabot uses different default searches for English, French and Spanish and progressively broadens the search window when there is not enough recent material.

Noise-heavy social/video domains are excluded from the search input.


⏳ Freshness before volume

Mediabot prefers recent, dated articles.

The normal target is at least two results that can actually be proven to be no more than seven days old.

The search therefore works like this:

narrow search
     │
     ├── enough fresh material ──► summarise
     │
     └── not enough
              │
              ▼
         wider search
              │
              ├── enough fresh material ──► summarise
              │
              └── final wider window

An undated article may still be useful as a fallback, but it is no longer counted as “fresh” merely because its date is unknown.

This distinction matters: unknown age is not the same thing as recent.


🧠 Claude writes the short synthesis

Tavily results provide:

title
domain
publication date
snippet

Claude receives that material with a constrained prompt:

  • factual summary;
  • resolved channel language;
  • short IRC-oriented output;
  • preference for corroborated facts;
  • explicit attribution when only one source supports a point;
  • no invented dates or sources.

If synthesis fails, Mediabot can still fall back to useful search-result titles rather than returning nothing.


📚 Sources are not invented by the model

The final source line is built deterministically from Tavily results, not from Claude’s prose.

Example:

Sources: lemonde.fr (08/08), reuters.com (08/08), bbc.com (07/08)

Domains are deduplicated and the number displayed is bounded.

If Tavily did not provide a parseable publication date, Mediabot displays:

(?)

instead of making one up.


⚡ mb614 — Accented commands finally reach the dispatch table

The first production test exposed a classic Perl/IRC encoding mismatch.

The source file contained a real Unicode key:

actualités

while IRC delivered UTF-8 bytes.

They looked identical to a human but were different values to Perl, so the command silently missed the dispatch table.

The repair was made at the command-dispatch boundary rather than as a special news hack.

Command names are now folded to lowercase ASCII before lookup:

actualités  ──► actualites
actualité   ──► actualite
ACTUALITES  ──► actualites

The dispatch table itself stays ASCII-only.

That makes the fix reusable for future accented command aliases too.


📨 mb615 — Async workers can speak through every helper path

The next production test found a subtler async issue.

The news worker successfully started but produced no IRC response.

The reason was that worker output interception covered imported UserCommands aliases but not fully-qualified calls such as:

Mediabot::Helpers::botPrivmsg(...)

External::News uses the qualified form.

The worker therefore attempted to use a path that should belong to the parent process, and its output disappeared when the child exited.

The worker facade now captures both forms:

imported alias
     │
     ├──────────────┐
     │              ▼
qualified Helper ─► intent collector
                    │
                    ▼
                  pipe
                    │
                    ▼
                  parent
                    │
                    ▼
             real IRC helpers

privmsg, notice and action all follow the same rule.

The temporary helper overrides are restored before parent-side replay.


🛡️ mb616 — The last news guardrails

The final pre-commit review found two small promises that the implementation did not yet fully keep.

One cooldown means one cooldown

The four aliases all declared a 45-second parent-side cooldown:

actualites
actualite
actu
news

But the state key still used the exact alias typed by the user.

That made this possible:

actualites
news
actu
actualite

— four paid Tavily/Claude calls inside what was supposed to be one protected interval.

All aliases now resolve to the same cooldown bucket:

actualites:#channel

The same canonical name is used for per-channel cooldown configuration.

MIN_FRESH now means what it says

The news module declared a minimum of two fresh results, but the request loop previously stopped at the first non-empty Tavily response.

It now truly continues to the wider window until enough dated fresh material exists or the final search window is reached.


🧪 Regression coverage

The news tests cover:

ASCII and accented command aliases
channel-language resolution
default headline searches
Tavily search parameters
freshness selection
undated-result behavior
deterministic source lines
missing API-key behavior
async worker output collection
worker helper restoration
shared cooldown across every alias

The news test was also renumbered to keep the test sequence unambiguous: the earlier achievement async guard already occupied number 796.


🔐 Operational configuration

Tavily belongs in the private runtime configuration only:

[tavily]
API_KEY=tvly-...

The Anthropic configuration remains the one already used by Mediabot’s Claude integration.

Neither key is stored in source, sample configuration or public documentation.


🏰 Final parchment

The Daily Prophet has found its way into the castle.

The owl searches outside, the portrait reads the dispatches, and the Common Room hears the story in its own language.

Accents no longer confuse the doorkeeper. Workers return their messages to the Great Hall instead of whispering into an empty corridor. And changing the name of the spell no longer buys four turns of the hourglass.

Current news is now a first-class Mediabot command rather than a script bolted onto the side.

You must be logged in to reply.