From mboxrd@z Thu Jan 1 00:00:00 1970 Delivery-date: Fri, 23 Oct 2026 11:29:24 +0200 Received: from metis.whiteo.stw.pengutronix.de ([2a0a:edc0:2:b01:1d::104]) by lore.white.stw.pengutronix.de with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1xKBaW-002CnJ-1H for lore@lore.pengutronix.de; Fri, 23 Oct 2026 11:29:24 +0200 Received: from [127.0.0.1] (helo=metis.whiteo.stw.pengutronix.de) by metis.whiteo.stw.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1xKBaV-0002fj-UK; Fri, 23 Oct 2026 11:29:23 +0200 Received: from mx1.white.stw.pengutronix.de ([2a0a:edc0:0:b01:1d::107]) by metis.whiteo.stw.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1xKBaO-0002fc-K1 for ptxdist@pengutronix.de; Fri, 23 Oct 2026 11:29:16 +0200 Received: from mail-ot1-x32f.google.com (mail-ot1-x32f.google.com [IPv6:2607:f8b0:4864:20::32f]) by mx1.white.stw.pengutronix.de (Postfix) with ESMTPS id 14A6C201E0D for ; Fri, 23 Oct 2026 11:29:16 +0200 (CEST) ARC-Seal: i=2; s=20260414; d=pengutronix.de; t=1792747756; a=rsa-sha256; cv=pass; b=AeQeYHJnidjHifjtIKU50homGFrTKgaRBCtMjRGE9cUdYVcJFCzq9iEta3EyikRcQQTbKi MonDvUQx8hHR+ZpvjWy+/9srdfWMpG+sw1oCWgErWBZcQRVkz/a+2+4p1Ww6Ti/2YgIKgK DVpchM5VXcrNiP9OUgLz9NKjHpUl1ugMHx8AVpLdZQtPseer2uS8+fSzWhjirCNCltvjWo x1NDaLeXwPOIS5lpwineiUF+6napFVZbZsMQqh/3MNgbqnSh31NyWi7rEVoD94ayAJ1wq7 ClstxfNF/D9K0LU463BN3lG0veclC9i/hcw8MOxfxl0//CNb7iEVmPra+VUrGg== ARC-Authentication-Results: i=2; mx1.white.stw.pengutronix.de; dkim=pass header.d=gmail.com header.s=20251104 header.b="gHb/ILlW"; spf=pass (mx1.white.stw.pengutronix.de: domain of bruno.thomsen@gmail.com designates 2607:f8b0:4864:20::32f as permitted sender) smtp.mailfrom=bruno.thomsen@gmail.com; dmarc=pass (policy=none) header.from=gmail.com; arc=pass ("google.com:s=arc-20260327:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=pengutronix.de; s=20260414; t=1792747756; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j+ZTQ8Js6lGRTs0jzEmd3dDWesfo0rQfo5gqADOg4aM=; b=Mh6nwIHEgDbL0x+5P2miSsoLSI5tQBSryScMviTxXmEUpKAo5zmKqj1NLXGGWfdqs2pl5W +GecmxuU+CSKoD0p7FT6C7A534xLMsRfSdB2U8e9x8Ntd1e7OznWJfwdcxN2oskpz1hm91 CmtmnYSU//RCqPEt/YJC4FOf5uWjK4onEJUqP+oTwyPdBQcEcB5eJdvBXS9zYamy4Rlgmp 7HqfOPOSu+CkVygMJKDQDdZ/GI5mSM3djWP02zASzSB7gjm/z8ZZFB820Vnadh7o1vjWn2 hdgI7lySfbIXOIu+7G8yo1E2cs4JSd97oH5PWfb2brWKYyVwWFAttw3Px/GVJw== Received: by mail-ot1-x32f.google.com with SMTP id 46e09a7af769-8446af403b5so422760a34.2 for ; Fri, 23 Oct 2026 02:29:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1792747754; cv=none; d=google.com; s=arc-20260327; b=LVJKNB6435XnLX8ycYg9VEpdJbOgOvcT86INq4su0eXCF4ijnZajwKjSHPmLyM7agj PRn+G03wxsbAA/0JdenlN33zjOBiN2TWPQGGVOWHYtlcXSbU+Gcj7csyZ/4NxAd5y268 3w1UsFmmrY7ecwmvY59HW2edgYqUL0XRBVoS88CkxzxX/RrAcY70LnqQcpAHuil2+w7O P0zdbCF/A9IKIdcrT9YDFng2HrxDJihzC+rI1TucHFssOkewco/bQI4JPvMGekr1pwXh /ghGeWjDzNH6dzD6GFBg/KeTzs/T9+K+fAfFnIOGSPSL/163bEWMtvGO2nbS3rSmdGyt Frmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=j+ZTQ8Js6lGRTs0jzEmd3dDWesfo0rQfo5gqADOg4aM=; fh=sUG++IJmTX2XtS7LYEwbt3L1khwMCRhwD9TBGTZPSEs=; b=hhY+l2+HApYytVvNfYfkw8wc1M9clUi4uw9bn1yOo096fnfafNbVZ+PStifk/Gey8a dVpjUANmSYQLD/yA6KYW40t9//2sYxpFzCbIigV4kFVH8picAR1NkazEZF8hU95vU0Ab HmK5lfVrl+927vSgHMlUAvff9SBWzrQZi0tv2bW56V24847uaI/skS5Avc1UXJDYsgfZ 1KbIaGMRXgzjIGT4VGTcom3k+TsN4BNOI9WWa1lpVUh2H/314jT8MBdDU5FWuCzrxLW4 CtKQGVDKSt51zmlbLvHKazq28XWBhPDYXQVIjiNkUgXoWJpvPHzsZdZZ4VE1ufjKBs9L Aesw==; darn=pengutronix.de ARC-Authentication-Results: i=1; mx.google.com; arc=none X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1792747754; x=1793352554; h=content-transfer-encoding:content-type:cc:to:subject:message-id :date:from:in-reply-to:references:mime-version:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=j+ZTQ8Js6lGRTs0jzEmd3dDWesfo0rQfo5gqADOg4aM=; b=DRfugqaFUBj5pIfA0magL5jKjTY91D8nU98iCjoqxYDKhttUSoTaxbDOVgPyGFXvz0 uXPV7ie8g4kzJi4TXvYzW+Mqm6vuMpi6vb7c3R9S+YrRCN/cJpBYw6GBvb2YJ9VEd9Rv kyIyV/UYozDK+IOfFixP/20Yda091JnL9Z5HtE+JT20GJsI7HDOYexGLtlUvu9uZnCyT 03oGlpEFQ2Xlc8C2gl73LrEMPO/fCOCIdWAlqGHecFfRPD+SxO9aUjY5LnhE6MLZbEpY iaUxUlBc7LqeZaw1G/7lm0c0gViy/eeREfyh1DbaqoppLLAzxFwrLfDEkYN98eQHY6gs vwFQ== X-Gm-Message-State: AFq9FYKVztxOO3PK+woko0GSWE05eRq6ydHRKkWapLCzc4LM0J/Ugfyy dFMEJ2jX3dJSVDJW5gJ4S2aD39NdQ0WrFaUAjBTjhSshdNU+37nA4cl/+xQXnzVx3IEFOXppi4S fm2nCI8815l+LkClPsL10wFFrBwbcD3VM9MT4 X-Gm-Gg: AYBFou1B56iZFlIRnATWn/Tl3gz27WR1eso7yqzPk95B7OjFCVA7S3BKmNLSz55/rWV 4dPhMpG8+fnk3R6ZtDvuGmDNS19CseWGM4L6+Vyk9SuD/YV6Bovv/TjJSRfCI+Rq7kQHYdI/+UH nnVktojajl1SARtiY0iocMW37LaZgxsyBPoN6oeO+h8aZPbb7v0i9IGcBGCct2WW3k++nxQ6lZ7 xE3SIwWePnz70VTqocg6L888vU6MbgR+fbODIwSEqswcEbC00j5XeQMY9FcVjg9R5r2Prpst8t/ XoEhdg3lWRnJsQVkFOFQLqMl9zJ69UgrWHbVKch3nnEM1+iJ1jkVlcQrb/ukvVPFoVNHoY8YL71 mX9Sf7eKnf13jPgZHT0MJnNamPdFYKhhbKEj6 X-Received: by 2002:a05:6808:4445:b0:4d6:90d3:e184 with SMTP id 5614622812f47-55be85b85famr950625b6e.45.1792747754322; Fri, 23 Oct 2026 02:29:14 -0700 (PDT) MIME-Version: 1.0 References: <20261023090104.3420495-1-s.pueschel@pengutronix.de> In-Reply-To: <20261023090104.3420495-1-s.pueschel@pengutronix.de> From: Bruno Thomsen Date: Fri, 23 Oct 2026 11:28:58 +0200 X-Gm-Features: AclHuK-uzZCaisgnr3lf6UlQkEo5NlLoBVZ-iX_QadWH4dx_RDtUm19FLR26ahI Message-ID: To: ptxdist@pengutronix.de Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: mx1 X-Rspamd-Queue-Id: 14A6C201E0D X-Spamd-Result: default: False [-5.00 / 15.00]; BAYES_HAM(-3.00)[99.99%]; ARC_ALLOW(-1.00)[google.com:s=arc-20260327:i=1]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20251104]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4864::/56]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_TLS_LAST(0.00)[]; TAGGED_FROM(0.00)[]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_FROM(0.00)[gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MISSING_XM_UA(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[ptxdist@pengutronix.de]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::32f:from]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; NEURAL_HAM(-0.00)[-0.616]; RCVD_COUNT_ONE(0.00)[1]; ARC_SIGNED(0.00)[pengutronix.de:s=20260414:i=2] X-Rspamd-Action: no action X-Stat-Signature: ifz8mzwayrg4n8g8sz51cosdp1pesmak Subject: Re: [ptxdist] [PATCH] scripts: add update_check script X-BeenThere: ptxdist@pengutronix.de X-Mailman-Version: 2.1.29 Precedence: list List-Id: PTXdist Development Mailing List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: ptxdist@pengutronix.de Cc: =?UTF-8?Q?Sven_P=C3=BCschel?= Sender: "ptxdist" X-SA-Exim-Connect-IP: 127.0.0.1 X-SA-Exim-Mail-From: ptxdist-bounces@pengutronix.de X-SA-Exim-Scanned: No (on metis.whiteo.stw.pengutronix.de); SAEximRunCond expanded to false Den fre. 23. okt. 2026 kl. 11.01 skrev Sven P=C3=BCschel : > > Add an update_check python script as an easy check which PTXdist > rules can be bumped in their versions. It uses the report file generated > from an existing PTXdist BSP as a basis to determine the package > versions. Then it compares these package versions with the package > versions against the newest release versions reported by > release-monitoring.org . > > release-monitoring.org is a deployment of an open-source release > monitoring project named Anitya [1]. It allows to define a Project > which is some software you want to keep track of updates. A given > software can have multiple Projects. An example is GTK+, where you want > to have separate projects for GTK+ 3 and GTK+ 4. A Project can then > define a mapping to a package name in a Distribution. > > I've created the PTXdist distribution [2] and created Project mappings > for a good chunk of the PTXdist rules. I've excluded certain rules with > user-defined versions (e.g. the linux kernel) and projects where it was > unclear if PTXdist should track the latest major version or use the > latest version of the current major version. Also I've ignored rules, > where no Project existed yet. Furthermore I've skipped the `python-` > rules, as they can probably be added in an automated fashion (given the > pypy name can be used for matching purposes). Nice that you have added PTXdist to release-monitoring.org, you can also see it in Repology[1] where it has lists of outdated packages[2], potentially vulnerable packages[3], and packages with problems[4] like dead URLs or permanent redirects. Information is tracking the master branch= . /Bruno [1] https://repology.org/repository/ptxdist [2] https://repology.org/projects/?inrepo=3Dptxdist&outdated=3D1&families_n= ewest=3D2- [3] https://repology.org/projects/?inrepo=3Dptxdist&vulnerable=3D1 [4] https://repology.org/repository/ptxdist/problems > > This python script uses the unauthenticated API to get all current > versions for the Projects with a mapped PTXdist name. It then caches > these versions locally (updating only if the cache is older then 24h) > and compares the versions against the versions in the report file. > It displays all rules that can be updated and also counts the number of > up-to-date rules and rules missing a mapping. With a command line flag > the latter two can also be displayed. > > Note that this isn't perfect and just provides an approximation to the > actual truth. There are a number of imperfections: > - PTXdist using non-stable versions (e.g. iftop 1.0pre4 instead of 0.17) > - Versions with different separators (dashes/underscores vs. dots) > - Rules not intended to be tracked (e.g. cross-dummy-strip) > > Also there are some pitfalls in this design: > - Version numbers are based on the BSP > - Updated rules might be already present in a newer PTXdist release > - Rules might be overwritten with an older version locally > - BSP only rules (custom applications) will show up as untracked > - Versions may lag behind a bit (Anitya cron job and rate-limits) > > [1] https://github.com/fedora-infra/anitya > [2] https://release-monitoring.org/distro/PTXdist/ > > Signed-off-by: Sven P=C3=BCschel > --- > scripts/update_check.py | 132 ++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 132 insertions(+) > create mode 100755 scripts/update_check.py > > diff --git a/scripts/update_check.py b/scripts/update_check.py > new file mode 100755 > index 000000000..94ffe83bc > --- /dev/null > +++ b/scripts/update_check.py > @@ -0,0 +1,132 @@ > +#!/usr/bin/env python3 > +# > +# Copyright (C) 2026 by Sven P=C3=BCschel > +# > +# For further information about the PTXdist project and license conditio= ns > +# see the README file. > +# > + > +import argparse > +import itertools > +import json > +import os > +import time > +from urllib import request > + > +import yaml > + > +API_ENDPOINT =3D 'https://release-monitoring.org/api/v2' > +ITEMS_PER_PAGE =3D 25 > + > +XDG_CACHE_DIR =3D os.getenv('XDG_CACHE_HOME') > +if not XDG_CACHE_DIR: > + XDG_CACHE_DIR =3D os.path.join(os.getenv('HOME'), '.cache') > + > +VERSION_CACHE_FILENAME =3D 'upstream_pkg_versions.json' > +VERSION_CACHE_EXPIRE_SEC =3D 24 * 60 * 60 > +VERSION_CACHE_DIR =3D os.path.join(XDG_CACHE_DIR, 'ptxdist') > +VERSION_CACHE_FILE =3D os.path.join(VERSION_CACHE_DIR, VERSION_CACHE_FIL= ENAME) > + > +def download_pkg_version_list(): > + versions =3D [] > + > + for page in itertools.count(1): > + with request.urlopen(request.Request( > + f'{API_ENDPOINT}/packages/?distribution=3DPTXdist&page= =3D{page}&items_per_page=3D{ITEMS_PER_PAGE}', > + headers=3D{ 'User-Agent': 'PTXdist update_check script' = } > + )) as url: > + data =3D json.load(url) > + assert data['page'] =3D=3D page > + assert data['items_per_page'] =3D=3D ITEMS_PER_PAGE > + total =3D data['total_items'] > + versions.extend(data['items']) > + print(f'\rDownloading current package versions ({len(version= s)}/{total})', end=3D'') > + if len(versions) >=3D total: > + break > + print() > + return versions > + > +parser =3D argparse.ArgumentParser(formatter_class=3Dargparse.RawDescrip= tionHelpFormatter) > +parser.add_argument('--show-untracked', help=3D'Show rules where no upda= ted version is known', action=3D'store_true') > +parser.add_argument('--show-up-to-date', help=3D'Show rules which are al= ready at the newest versions', action=3D'store_true') > +parser.add_argument('report_file', help=3D'report file generated by `ptx= dist fast-bsp-report`') > + > +args =3D parser.parse_args() > + > +versions =3D None > +try: > + if os.stat(VERSION_CACHE_FILE).st_mtime > time.time() - VERSION_CACH= E_EXPIRE_SEC: > + print('Using cached version list') > + with open(VERSION_CACHE_FILE) as f: > + versions =3D json.load(f) > +except FileNotFoundError: > + # No cache exists, just download it > + pass > +except Exception as e: > + print(f'Unexpected error trying to read the version cache: {e}') > + > +if not versions: > + versions =3D download_pkg_version_list() > + try: > + os.makedirs(VERSION_CACHE_DIR, exist_ok=3DTrue) > + with open(VERSION_CACHE_FILE, 'w') as f: > + json.dump(versions, f, indent=3D1) > + f.write("\n") > + except Exception as e: > + print(f"Couldn't write package version cache: {e}") > +print() > + > +stable_versions =3D {} > +for package in versions: > + stable_versions[package['name']] =3D package['stable_version'] > + > +with open(args.report_file) as f: > + report =3D yaml.safe_load(f) > + > +latest_version =3D [] > +needs_update =3D [] > +untracked =3D [] > + > +max_pkgname_len =3D 0 > +max_version_len =3D 0 > + > +for pkg in report['packages'].values(): > + if not 'rulefile' in pkg or not 'version' in pkg: > + continue > + > + max_pkgname_len =3D max(max_pkgname_len, len(pkg['name'])) > + max_version_len =3D max(max_version_len, len(pkg['version'])) > + > + version =3D stable_versions.get(pkg['name']) > + if version is None: > + version =3D stable_versions.get(pkg['name'].removeprefix('host-'= )) > + > + if version is None: > + untracked.append((pkg['name'], pkg['version'])) > + else: > + if version !=3D pkg['version']: > + needs_update.append((pkg['name'], pkg['version'], version)) > + else: > + latest_version.append((pkg['name'], pkg['version'])) > + > +if args.show_up_to_date: > + print('Up-to-date rules') > + print('=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D') > + for pkg in latest_version: > + print(f"{pkg[0].ljust(max_pkgname_len)} {pkg[1].rjust(max_versio= n_len)}") > + print() > + > +print('Updateable rules') > +print('=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D') > +for pkg in needs_update: > + print(f"{pkg[0].ljust(max_pkgname_len)} {pkg[1].rjust(max_version_le= n)} -> {pkg[2]}") > +print() > + > +if args.show_untracked: > + print('Untracked rules') > + print('=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D') > + for pkg in untracked: > + print(f"{pkg[0].ljust(max_pkgname_len)} {pkg[1].rjust(max_versio= n_len)}") > + print() > + > +print(f"Updateable: {len(needs_update)} Up-to-date: {len(latest_version)= } Untracked: {len(untracked)}") > -- > 2.47.3 > >