[ptxdist] [PATCH] scripts: add update_check script

["Sven Püschel" <s.pueschel@pengutronix.de>]

Add an update_check python script as an easy check which PTXdist
rules can be bumped in their versions. It uses the report file generated
from an existing PTXdist BSP as a basis to determine the package
versions. Then it compares these package versions with the package
versions against the newest release versions reported by
release-monitoring.org .

release-monitoring.org is a deployment of an open-source release
monitoring project named Anitya [1]. It allows to define a Project
which is some software you want to keep track of updates. A given
software can have multiple Projects. An example is GTK+, where you want
to have separate projects for GTK+ 3 and GTK+ 4. A Project can then
define a mapping to a package name in a Distribution.

I've created the PTXdist distribution [2] and created Project mappings
for a good chunk of the PTXdist rules. I've excluded certain rules with
user-defined versions (e.g. the linux kernel) and projects where it was
unclear if PTXdist should track the latest major version or use the
latest version of the current major version. Also I've ignored rules,
where no Project existed yet. Furthermore I've skipped the `python-`
rules, as they can probably be added in an automated fashion (given the
pypy name can be used for matching purposes).

This python script uses the unauthenticated API to get all current
versions for the Projects with a mapped PTXdist name. It then caches
these versions locally (updating only if the cache is older then 24h)
and compares the versions against the versions in the report file.
It displays all rules that can be updated and also counts the number of
up-to-date rules and rules missing a mapping. With a command line flag
the latter two can also be displayed.

Note that this isn't perfect and just provides an approximation to the
actual truth. There are a number of imperfections:
- PTXdist using non-stable versions (e.g. iftop 1.0pre4 instead of 0.17)
- Versions with different separators (dashes/underscores vs. dots)
- Rules not intended to be tracked (e.g. cross-dummy-strip)

Also there are some pitfalls in this design:
- Version numbers are based on the BSP
  - Updated rules might be already present in a newer PTXdist release
  - Rules might be overwritten with an older version locally
- BSP only rules (custom applications) will show up as untracked
- Versions may lag behind a bit (Anitya cron job and rate-limits)

[1] https://github.com/fedora-infra/anitya
[2] https://release-monitoring.org/distro/PTXdist/

Signed-off-by: Sven Püschel <s.pueschel@pengutronix.de>
---
 scripts/update_check.py | 132 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 132 insertions(+)
 create mode 100755 scripts/update_check.py

diff --git a/scripts/update_check.py b/scripts/update_check.py
new file mode 100755
index 000000000..94ffe83bc
--- /dev/null
+++ b/scripts/update_check.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+#
+# Copyright (C) 2026 by Sven Püschel <s.pueschel@pengutronix.de>
+#
+# For further information about the PTXdist project and license conditions
+# see the README file.
+#
+
+import argparse
+import itertools
+import json
+import os
+import time
+from urllib import request
+
+import yaml
+
+API_ENDPOINT = 'https://release-monitoring.org/api/v2'
+ITEMS_PER_PAGE = 25
+
+XDG_CACHE_DIR = os.getenv('XDG_CACHE_HOME')
+if not XDG_CACHE_DIR:
+    XDG_CACHE_DIR = os.path.join(os.getenv('HOME'), '.cache')
+
+VERSION_CACHE_FILENAME = 'upstream_pkg_versions.json'
+VERSION_CACHE_EXPIRE_SEC = 24 * 60 * 60
+VERSION_CACHE_DIR = os.path.join(XDG_CACHE_DIR, 'ptxdist')
+VERSION_CACHE_FILE = os.path.join(VERSION_CACHE_DIR, VERSION_CACHE_FILENAME)
+
+def download_pkg_version_list():
+    versions = []
+
+    for page in itertools.count(1):
+        with request.urlopen(request.Request(
+                f'{API_ENDPOINT}/packages/?distribution=PTXdist&page={page}&items_per_page={ITEMS_PER_PAGE}',
+                headers={ 'User-Agent': 'PTXdist update_check script' }
+            )) as url:
+            data = json.load(url)
+            assert data['page'] == page
+            assert data['items_per_page'] == ITEMS_PER_PAGE
+            total = data['total_items']
+            versions.extend(data['items'])
+            print(f'\rDownloading current package versions ({len(versions)}/{total})', end='')
+            if len(versions) >= total:
+                break
+    print()
+    return versions
+
+parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter)
+parser.add_argument('--show-untracked', help='Show rules where no updated version is known', action='store_true')
+parser.add_argument('--show-up-to-date', help='Show rules which are already at the newest versions', action='store_true')
+parser.add_argument('report_file', help='report file generated by `ptxdist fast-bsp-report`')
+
+args = parser.parse_args()
+
+versions = None
+try:
+    if os.stat(VERSION_CACHE_FILE).st_mtime > time.time() - VERSION_CACHE_EXPIRE_SEC:
+        print('Using cached version list')
+        with open(VERSION_CACHE_FILE) as f:
+            versions = json.load(f)
+except FileNotFoundError:
+    # No cache exists, just download it
+    pass
+except Exception as e:
+    print(f'Unexpected error trying to read the version cache: {e}')
+
+if not versions:
+    versions = download_pkg_version_list()
+    try:
+        os.makedirs(VERSION_CACHE_DIR, exist_ok=True)
+        with open(VERSION_CACHE_FILE, 'w') as f:
+            json.dump(versions, f, indent=1)
+            f.write("\n")
+    except Exception as e:
+        print(f"Couldn't write package version cache: {e}")
+print()
+
+stable_versions = {}
+for package in versions:
+    stable_versions[package['name']] = package['stable_version']
+
+with open(args.report_file) as f:
+    report = yaml.safe_load(f)
+
+latest_version = []
+needs_update = []
+untracked = []
+
+max_pkgname_len = 0
+max_version_len = 0
+
+for pkg in report['packages'].values():
+    if not 'rulefile' in pkg or not 'version' in pkg:
+        continue
+
+    max_pkgname_len = max(max_pkgname_len, len(pkg['name']))
+    max_version_len = max(max_version_len, len(pkg['version']))
+
+    version = stable_versions.get(pkg['name'])
+    if version is None:
+        version = stable_versions.get(pkg['name'].removeprefix('host-'))
+
+    if version is None:
+        untracked.append((pkg['name'], pkg['version']))
+    else:
+        if version != pkg['version']:
+            needs_update.append((pkg['name'], pkg['version'], version))
+        else:
+            latest_version.append((pkg['name'], pkg['version']))
+
+if args.show_up_to_date:
+    print('Up-to-date rules')
+    print('================')
+    for pkg in latest_version:
+        print(f"{pkg[0].ljust(max_pkgname_len)} {pkg[1].rjust(max_version_len)}")
+    print()
+
+print('Updateable rules')
+print('================')
+for pkg in needs_update:
+    print(f"{pkg[0].ljust(max_pkgname_len)} {pkg[1].rjust(max_version_len)} -> {pkg[2]}")
+print()
+
+if args.show_untracked:
+    print('Untracked rules')
+    print('===============')
+    for pkg in untracked:
+        print(f"{pkg[0].ljust(max_pkgname_len)} {pkg[1].rjust(max_version_len)}")
+    print()
+
+print(f"Updateable: {len(needs_update)} Up-to-date: {len(latest_version)} Untracked: {len(untracked)}")
-- 
2.47.3