Forum teuk.org

📰 Mediabot v3 — The Daily Prophet Finally Gets a Newsroom

in Mediabot · started by TeuK · 2w ago

TeuK · 2w ago

Development chronicle — mb613 to mb619 · Live news, multilingual output, async-safe delivery and editorially relevant sources

Mediabot’s news command has moved from a working prototype to a proper IRC news feature.

The first version could search Tavily and ask Claude for a short synthesis. The following rounds then fixed what only real IRC use could reveal: accented command names, worker output that vanished in a fork, cooldown bypasses, austere source rendering, vague links, stale headlines, and finally the mismatch between what Claude summarized and what the user could actually click.

The result is now much closer to a small newsroom than a search wrapper.

No database schema change was required.


🗞️ actualites becomes a first-class command

The following forms reach the same implementation:

m actualites
m actualités
m actualite
m actualité
m actu
m news

A subject is optional:

m actualités OpenAI
m news cybersécurité

With no subject, Mediabot returns current general headlines.

With a subject, it searches recent coverage of that topic.


🌍 One language rule everywhere

News uses the same language resolver already shared by:

ai summary
recap ai

The resolution order is:

explicit en|fr|es or lang=xx
              │
              ▼
channel +LangFR / +LangES
              │
              ▼
main.LANG
              │
              ▼
English fallback

So:

m actualités

on a French channel naturally answers in French, while:

m news lang=en OpenAI

forces English without changing the channel configuration.

Search notices, errors and the Claude synthesis all follow that same resolved language.


🔎 Tavily is the research desk

Tavily remains the factual research layer.

It provides material such as:

title
source domain
publication date
snippet

That material is used to corroborate and enrich the synthesis.

The API key stays in the private runtime configuration:

[tavily]
API_KEY=...

No secret belongs in the public sample configuration or repository.


🧠 Claude is the editor — not the source list

Claude receives a bounded news context and is asked for a short factual IRC-oriented synthesis.

The prompt requires:

  • the resolved channel language;
  • concrete names, dates, places and figures;
  • preference for facts corroborated by multiple sources;
  • explicit attribution when only one source supports a claim;
  • no invented fact, date or source;
  • at most two short synthesis lines.

Crucially, Claude does not invent the clickable references.

Those are built deterministically by Mediabot.


⚡ mb614 — Accented commands reach the dispatcher

The first real-world issue appeared immediately:

m actualités

looked right to a human but did not match the command table.

The source contained a Unicode é, while IRC delivered UTF-8 bytes. Perl therefore saw different values.

The fix was made at the command-dispatch boundary rather than as a special case for news.

Command names are now folded before lookup:

actualités  → actualites
actualité   → actualite
ACTUALITES  → actualites

The dispatch table remains ASCII-only.

That also protects future commands with accented aliases.


📨 mb615 — Async workers can actually speak

News performs multiple network operations, so it belongs in CommandAsync.

A production test exposed a subtle hole: the worker facade intercepted imported helper aliases, but the news module called the fully qualified form:

Mediabot::Helpers::botPrivmsg(...)

The child therefore tried to write through the wrong path and its output disappeared when the worker exited.

The facade now intercepts both:

imported helper alias
        │
        ├──────────────┐
        │              ▼
qualified helper ───► intent collector
                       │
                       ▼
                     pipe
                       │
                       ▼
                     parent
                       │
                       ▼
                    real IRC

The same applies to privmsg, notice and action.


🛡️ mb616 — Cooldowns and freshness mean what they say

All aliases advertise the same news command, so they now share one cooldown bucket:

actualites
actualite
actu
news
    │
    └──► actualites:#channel

Changing the spelling no longer buys another paid Tavily/Claude request.

The search widening logic was also corrected so MIN_FRESH is real rather than decorative.


🎨 mb617 — IRC gets its visual charter back

The initial Mediabot output was technically useful but visually flat:

Actu <summary>
Actu Sources: source1, source2, source3

The Windrop implementation had already shown a better IRC presentation.

Mediabot now uses:

badge             → white on red
date + publisher  → grey
separator |       → orange
URL               → blue + underlined

Only the first synthesis line carries the colored badge.

Up to three article references follow, packed into IRC-safe lines.

URLs are shortened through TinyURL when possible, with automatic fallback to the original URL.

TinyURL has its own short timeout so link presentation cannot consume the async worker’s entire time budget.


🔗 mb618 — Clickable links finally describe real articles

The next issue was not visual.

Tavily can return useful factual pages whose titles are actually generic sections:

Journaux d'information
Actualités Île-de-France
Actualités à la une

Those links were clickable but did not tell an IRC user what article they were about to open.

The architecture was therefore aligned with the mature Windrop script:

Tavily
   │
   └──► factual context / corroboration

Google News RSS
   │
   └──► precise article title
        publisher
        publication date
        clickable URL

Google News provides the visible article list.

Tavily remains the research material.

If RSS fails, the existing Tavily results remain available as a fallback, so Google News is not a hard dependency for the command.


🕒 mb619 — “Today’s news” finally means today

A final live test exposed the most important editorial problem.

A generic Google News search could rank pages from:

17/07
01/04

above articles published on August 8.

That is unacceptable for:

m actualités

The no-subject command now uses the localized Google News Top Stories feed rather than searching for a sentence such as “important news today in France”.

For the default headline command, a visible article must:

  • have a parseable publication date;
  • be no more than 36 hours old;
  • not be dated implausibly in the future;
  • not be a generic continuous-news or live-index page.

Examples of titles intentionally rejected from the clickable showcase include patterns equivalent to:

info en continu
journal des informations
live updates
top stories
actualité du vendredi

🎯 Topic searches widen only when required

A named subject has a different freshness strategy.

For:

m actualités OpenAI

Mediabot searches:

1 day
  │
  ├── enough useful material → stop
  │
  └── otherwise
          ▼
       3 days
          │
          ├── enough → stop
          │
          └── otherwise
                  ▼
               7 days

The command therefore remains useful for narrower subjects without polluting the normal “headlines” view with old articles.


🔒 The summary and links now describe the same stories

This is the key mb619 editorial rule.

Earlier, Tavily could make Claude summarize subjects A and B while Google News happened to display links C, D and E.

That produced a polished-looking but incoherent answer.

The prompt is now explicitly anchored on the selected clickable headlines.

Conceptually:

Top Stories / topic RSS
          │
          ├──► 3 visible articles
          │
          └──► anchor topics for Claude
                         ▲
                         │
                 Tavily context

Tavily supplies corroboration and detail for those stories rather than independently deciding what the user sees.

What Mediabot says and what the user can click are now the same editorial package.


🧪 Regression coverage

The news arc is protected by focused tests covering:

797  accented command folding
798  async worker output facade
799  base news command and Tavily contract
800  IRC visual charter and clickable links
801  precise RSS article extraction
802  fresh and relevant headline selection

The latest guards specifically verify:

  • localized Top Stories feeds;
  • real publication dates;
  • 36-hour default freshness;
  • topic widening 1d → 3d → 7d;
  • rejection of stale and generic live/index pages;
  • article-title and publisher parsing;
  • TinyURL presentation fallback;
  • diversity of visible sources;
  • Claude prompt alignment with the same clickable stories.

✨ What an IRC user gets now

The intended result is no longer:

Actu Sources: site1.fr (?), site2.fr (?), site3.fr (?)

It is closer to:

Actu <short factual synthesis of the stories below>
<optional second synthesis line>

08/08 France Culture <precise current headline> https://tinyurl.com/...
08/08 Le Monde <precise current headline> https://tinyurl.com/... | 07/08 Franceinfo <precise headline> https://tinyurl.com/...

The reader can understand the story, identify the publisher, see how recent it is and know exactly what they are about to open.

That is the point where a news command stops being a demo and becomes something people may actually use.


🏰 Final parchment

The Daily Prophet no longer arrives as a pile of anonymous clippings.

The owl brings today’s papers.

The editor reads the same stories that are pinned to the notice board.

Each clipping has a date, a publisher, a proper headline and a door you can actually open.

And yesterday’s forgotten parchment no longer gets presented as this morning’s news.

You must be logged in to reply.