The RSS work in Mediabot v3 is now complete.
What started as a fairly simple idea — “let channels subscribe to RSS feeds” — turned into three deliberately separated rounds:
The important part is not only that RSS now works.
It is that it has been tested all the way from a real public feed to a real IRC announcement, without cheating the runtime or replaying database rows.
MB692 introduced the native RSS foundation.
Mediabot now has durable RSS storage with:
The initial command set was added around:
m rss list
m rss info <feed>
m rss add <feed name> <url> [interval=30] [max=5]
m rss del <feed name>
m rss set <feed name> <interval|max|enabled> <value>
m rss probe <url>
m rss show <feed>
Feed names may contain spaces, and channel-level management follows the existing Mediabot permission model.
RSS fetching was intentionally treated as an outbound-network security problem, not just an XML parsing problem.
The fetcher now enforces, among other things:
This also fixed a few subtle implementation issues discovered during testing, including DNS rebinding exposure, IPv6 classification precedence and charset handling.
MB692 closed with the full suite green.
Commit:
8a94ea4 🦉 Give Channels a Safe Native RSS Owl Post
MB693 turned the RSS foundation into a real background service.
The major change was the introduction of native Scheduler-driven polling.
A dedicated task is now registered:
rss_poll_dispatch
The Scheduler itself does not perform HTTP requests.
RSS network and database work runs in isolated workers, keeping the main IRC loop responsive.
The runtime includes safeguards such as:
The first successful poll of a newly added feed is intentionally silent.
Existing feed history is stored and marked as already handled.
That means subscribing to a feed containing 30 articles does not dump 30 old headlines into the channel.
Only content discovered after the baseline becomes eligible for automatic announcement.
Items are stored in RSS_ITEM and deduplicated durably.
New items remain pending until delivery succeeds.
Once Mediabot accepts the IRC output path, the item receives its durable:
announced_at
This means an item is not considered delivered merely because it was fetched.
MB693 also introduced proper HTTP cache validators:
ETag
If-None-Match
Last-Modified
If-Modified-Since
HTTP 304
Unchanged feeds therefore avoid unnecessary parsing and processing.
RSS announcements use the same compact visual style as the existing news output:
* mediabotv3 - news : [Source] Title - https://tinyurl.com/...
TinyURL failure is non-fatal: Mediabot simply falls back to the original URL.
Multiple new items are queued on the parent side and spaced before delivery.
A controlled three-item live proof showed:
12:28:46 item 1
12:28:48 item 2
12:28:50 item 3
So delivery spacing is actually enforced by the runtime, not just asserted in unit tests.
MB693 closed with:
15908/15908 tests PASS
Commit:
ba4d924 🦉 Let RSS Owls Deliver Only Fresh Posts
Once the runtime was solid, MB694 deliberately avoided adding new architecture.
This round was about making RSS understandable from IRC.
m rss listThe list now exposes useful operational information instead of only feed names.
Example:
Flux RSS #boulets : [MB694 UX] 5 min/max3 · 25 items · OK
Feed state can now be shown as:
OK
WAITING FIRST POLL
PENDING
ERROR
PAUSED
m rss infoA feed can now be inspected without touching the database:
RSS [MB694 UX] on #boulets — ON · OK
URL: https://example.org/feed
Polling: every 5 min | max 3 | next: in 4 min
Items: 25 stored | 0 pending
Last poll: ... | success: ...
Last error: none
Paused feeds clearly show:
OFF · PAUSED
next: paused
A few examples now return useful explanations:
RSS max must be between 1 and 10 articles per poll.
and:
RSS URL rejected: private, loopback or reserved IP addresses are not allowed.
The add command also explicitly tells the operator that:
first poll is silent
which matters when testing a feed for the first time.
Before declaring RSS finished, I wanted one stronger proof.
Not a unit test.
Not a mocked HTTP response.
Not a database replay.
A real public RSS feed receiving genuinely new content after subscription.
The test used the existing public forum feed:
https://teuk.org/forum/rss
Mediabot subscribed normally:
m rss add MB694 Forum https://teuk.org/forum/rss interval=5 max=3
The first poll stored the existing history:
30 items
0 pending
0 historical announcements
The journal confirmed the intended behavior:
rss_poll_dispatch: silent baseline ... inserted=30
Then a new forum topic was actually published:
MB694 LIVE RSS GATE 20260823-072221-8ad857
Nothing in RSS_ITEM was edited.
The Scheduler was not accelerated.
last_poll_at was not touched.
The normal five-minute polling cycle was left alone.
Mediabot discovered the new article at:
13:34:33
and automatically announced it at:
13:34:34
with:
* mediabotv3 - news : [MB694 Forum] MB694 LIVE RSS GATE 20260823-072221-8ad857 - https://tinyurl.com/25q6yjxz
The resulting database state confirmed:
announcement count : 1
pending : 0
announced_at : set
That was the final piece I wanted before calling the RSS subsystem complete.
MB694 finished with:
789/789 test files
15940/15940 tests PASS
No RSS runtime architecture was changed during the UX round.
No schema migration was required.
The existing background polling implementation remained stable throughout the live tests.
Final MB694 commit:
983b72e ✨ Polish the RSS Noticeboard Before the Feast
Current development version:
3.4dev-20260823_073937
With MB692, MB693 and MB694 closed, Mediabot v3 is now entering feature freeze.
That means no new major functionality for the remaining 3.5 preparation work.
The remaining roadmap is deliberately less exciting — and that is exactly what I want before a release:
MB695 DB drift / upgrade compatibility
MB696 security / secrets
MB697 runtime cleanup
MB698 documentation / release contract
MB699 readiness report
After that, and only after an explicit final GO, the project can move from:
3.4dev-*
to:
3.5
The owls are flying.
Now it is time to make sure the castle is ready for release. 🏰
You must be logged in to reply.