From mboxrd@z Thu Jan 1 00:00:00 1970 Delivery-date: Fri, 23 Oct 2026 11:01:24 +0200 Received: from metis.whiteo.stw.pengutronix.de ([2a0a:edc0:2:b01:1d::104]) by lore.white.stw.pengutronix.de with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1xKB9Q-002CAe-08 for lore@lore.pengutronix.de; Fri, 23 Oct 2026 11:01:24 +0200 Received: from [127.0.0.1] (helo=metis.whiteo.stw.pengutronix.de) by metis.whiteo.stw.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1xKB9P-0008Fh-KU; Fri, 23 Oct 2026 11:01:23 +0200 Received: from dude04.red.stw.pengutronix.de ([2a0a:edc0:0:1101:1d::ac]) by metis.whiteo.stw.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1xKB97-0008FM-OM; Fri, 23 Oct 2026 11:01:05 +0200 From: =?UTF-8?q?Sven=20P=C3=BCschel?= To: ptxdist@pengutronix.de Date: Fri, 23 Oct 2026 11:00:25 +0200 Message-ID: <20261023090104.3420495-1-s.pueschel@pengutronix.de> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Subject: [ptxdist] [PATCH] scripts: add update_check script X-BeenThere: ptxdist@pengutronix.de X-Mailman-Version: 2.1.29 Precedence: list List-Id: PTXdist Development Mailing List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: ptxdist@pengutronix.de Cc: =?UTF-8?q?Sven=20P=C3=BCschel?= Sender: "ptxdist" X-SA-Exim-Connect-IP: 127.0.0.1 X-SA-Exim-Mail-From: ptxdist-bounces@pengutronix.de X-SA-Exim-Scanned: No (on metis.whiteo.stw.pengutronix.de); SAEximRunCond expanded to false Add an update_check python script as an easy check which PTXdist rules can be bumped in their versions. It uses the report file generated from an existing PTXdist BSP as a basis to determine the package versions. Then it compares these package versions with the package versions against the newest release versions reported by release-monitoring.org . release-monitoring.org is a deployment of an open-source release monitoring project named Anitya [1]. It allows to define a Project which is some software you want to keep track of updates. A given software can have multiple Projects. An example is GTK+, where you want to have separate projects for GTK+ 3 and GTK+ 4. A Project can then define a mapping to a package name in a Distribution. I've created the PTXdist distribution [2] and created Project mappings for a good chunk of the PTXdist rules. I've excluded certain rules with user-defined versions (e.g. the linux kernel) and projects where it was unclear if PTXdist should track the latest major version or use the latest version of the current major version. Also I've ignored rules, where no Project existed yet. Furthermore I've skipped the `python-` rules, as they can probably be added in an automated fashion (given the pypy name can be used for matching purposes). This python script uses the unauthenticated API to get all current versions for the Projects with a mapped PTXdist name. It then caches these versions locally (updating only if the cache is older then 24h) and compares the versions against the versions in the report file. It displays all rules that can be updated and also counts the number of up-to-date rules and rules missing a mapping. With a command line flag the latter two can also be displayed. Note that this isn't perfect and just provides an approximation to the actual truth. There are a number of imperfections: - PTXdist using non-stable versions (e.g. iftop 1.0pre4 instead of 0.17) - Versions with different separators (dashes/underscores vs. dots) - Rules not intended to be tracked (e.g. cross-dummy-strip) Also there are some pitfalls in this design: - Version numbers are based on the BSP - Updated rules might be already present in a newer PTXdist release - Rules might be overwritten with an older version locally - BSP only rules (custom applications) will show up as untracked - Versions may lag behind a bit (Anitya cron job and rate-limits) [1] https://github.com/fedora-infra/anitya [2] https://release-monitoring.org/distro/PTXdist/ Signed-off-by: Sven Püschel --- scripts/update_check.py | 132 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 132 insertions(+) create mode 100755 scripts/update_check.py diff --git a/scripts/update_check.py b/scripts/update_check.py new file mode 100755 index 000000000..94ffe83bc --- /dev/null +++ b/scripts/update_check.py @@ -0,0 +1,132 @@ +#!/usr/bin/env python3 +# +# Copyright (C) 2026 by Sven Püschel +# +# For further information about the PTXdist project and license conditions +# see the README file. +# + +import argparse +import itertools +import json +import os +import time +from urllib import request + +import yaml + +API_ENDPOINT = 'https://release-monitoring.org/api/v2' +ITEMS_PER_PAGE = 25 + +XDG_CACHE_DIR = os.getenv('XDG_CACHE_HOME') +if not XDG_CACHE_DIR: + XDG_CACHE_DIR = os.path.join(os.getenv('HOME'), '.cache') + +VERSION_CACHE_FILENAME = 'upstream_pkg_versions.json' +VERSION_CACHE_EXPIRE_SEC = 24 * 60 * 60 +VERSION_CACHE_DIR = os.path.join(XDG_CACHE_DIR, 'ptxdist') +VERSION_CACHE_FILE = os.path.join(VERSION_CACHE_DIR, VERSION_CACHE_FILENAME) + +def download_pkg_version_list(): + versions = [] + + for page in itertools.count(1): + with request.urlopen(request.Request( + f'{API_ENDPOINT}/packages/?distribution=PTXdist&page={page}&items_per_page={ITEMS_PER_PAGE}', + headers={ 'User-Agent': 'PTXdist update_check script' } + )) as url: + data = json.load(url) + assert data['page'] == page + assert data['items_per_page'] == ITEMS_PER_PAGE + total = data['total_items'] + versions.extend(data['items']) + print(f'\rDownloading current package versions ({len(versions)}/{total})', end='') + if len(versions) >= total: + break + print() + return versions + +parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter) +parser.add_argument('--show-untracked', help='Show rules where no updated version is known', action='store_true') +parser.add_argument('--show-up-to-date', help='Show rules which are already at the newest versions', action='store_true') +parser.add_argument('report_file', help='report file generated by `ptxdist fast-bsp-report`') + +args = parser.parse_args() + +versions = None +try: + if os.stat(VERSION_CACHE_FILE).st_mtime > time.time() - VERSION_CACHE_EXPIRE_SEC: + print('Using cached version list') + with open(VERSION_CACHE_FILE) as f: + versions = json.load(f) +except FileNotFoundError: + # No cache exists, just download it + pass +except Exception as e: + print(f'Unexpected error trying to read the version cache: {e}') + +if not versions: + versions = download_pkg_version_list() + try: + os.makedirs(VERSION_CACHE_DIR, exist_ok=True) + with open(VERSION_CACHE_FILE, 'w') as f: + json.dump(versions, f, indent=1) + f.write("\n") + except Exception as e: + print(f"Couldn't write package version cache: {e}") +print() + +stable_versions = {} +for package in versions: + stable_versions[package['name']] = package['stable_version'] + +with open(args.report_file) as f: + report = yaml.safe_load(f) + +latest_version = [] +needs_update = [] +untracked = [] + +max_pkgname_len = 0 +max_version_len = 0 + +for pkg in report['packages'].values(): + if not 'rulefile' in pkg or not 'version' in pkg: + continue + + max_pkgname_len = max(max_pkgname_len, len(pkg['name'])) + max_version_len = max(max_version_len, len(pkg['version'])) + + version = stable_versions.get(pkg['name']) + if version is None: + version = stable_versions.get(pkg['name'].removeprefix('host-')) + + if version is None: + untracked.append((pkg['name'], pkg['version'])) + else: + if version != pkg['version']: + needs_update.append((pkg['name'], pkg['version'], version)) + else: + latest_version.append((pkg['name'], pkg['version'])) + +if args.show_up_to_date: + print('Up-to-date rules') + print('================') + for pkg in latest_version: + print(f"{pkg[0].ljust(max_pkgname_len)} {pkg[1].rjust(max_version_len)}") + print() + +print('Updateable rules') +print('================') +for pkg in needs_update: + print(f"{pkg[0].ljust(max_pkgname_len)} {pkg[1].rjust(max_version_len)} -> {pkg[2]}") +print() + +if args.show_untracked: + print('Untracked rules') + print('===============') + for pkg in untracked: + print(f"{pkg[0].ljust(max_pkgname_len)} {pkg[1].rjust(max_version_len)}") + print() + +print(f"Updateable: {len(needs_update)} Up-to-date: {len(latest_version)} Untracked: {len(untracked)}") -- 2.47.3