Skip to content

Add aggregation utils

plata requested to merge utils into main

aggregate.py parses a list of sources (currently only RSS), filters for a given time period and assigns each entry to a category. The result is written to a .csv file.
For example:

python3 aggregate.py --begin 2022-08-29 --end 2022-09-05

Every source can be assigned to a category. If no category is specified, there's a simple heuristic based on keywords.

csv_to_md.py converts this .csv file to <year>-<version>.md in the format:

+++
title = "Weekly GNU-like Mobile Linux Update (<version>/<year>)"
date = "date time"
draft = true
[taxonomies]
tags = []
categories = ["weekly update"]
[extra]
author = "auto"
+++

# category
- source: link
- source: link

# category
...

The version is fetched from the LINMOB.net feed.

This could run in CI automatically (e.g. once a week) and open an MR which can serve as a base line for the article.

Of course, the result is not perfect but it should still help. Also, some feeds are not filled properly. For example, Megi's blog does not contain a timestamp (only in the title). Therefore, always all articles in the feed are collected.

Edited by plata

Merge request reports