[ptxdist] [PATCH v2] ptxdist: add repology json output support

[ptxdist] [PATCH v2] ptxdist: add repology json output support

[Bruno Thomsen]

Improve ptxdist project visibility by adding it to repology.
https://repology.org/

This also allows components like rauc to showcase supported
distributions and current packaged version.

It also gives ptxdist users and maintainers an overview of packages
with known CVEs that should be priotized for version bumpes.

Generate JSON output about almost all packages in ptxdist for repology.
Below is an output example for alsa-lib:

{
    "name": "alsa-lib",
    "version": "1.2.13",
    "upstream_urls": "https://www.alsa-project.org/files/pub/lib/alsa-lib-1.2.13.tar.bz2",
    "license": "LGPL-2.1-or-later"
},

Add PTXdist support #1487:
https://github.com/repology/repology-updater/issues/1487

This have been running for the past few months here:
https://github.com/baxeno/ptxdist-repology/blob/main/repology.json

Using the following command with DistroKit:
$ ptxdist repology | grep -v "make: Nothing to be done for" > repology.json

Signed-off-by: Bruno Thomsen <bruno.thomsen@gmail.com>
Co-developed-by: Ladislav Michl <ladis@triops.cz>
---
v1-v2:
- Keep Makefile implementation as Python implementation did not work
- Filter out packages with empty URL
- Filter out packages with empty or undefined version
- Reduced number of hard-coded ignored packages from 33 to 2
- View parsed output: https://repology.org/repository/ptxdist

PATCH v1:
https://lore.ptxdist.org/ptxdist/20250422193456.26040-1-bruno.thomsen@gmail.com/

PATCH RFC:
https://lore.ptxdist.org/ptxdist/20250418113357.23654-1-bruno.thomsen@gmail.com/

 bin/ptxdist                        |  6 ++++
 rules/post/ptxd_make_repology.make | 49 ++++++++++++++++++++++++++++++
 scripts/bash_completion            |  2 +-
 3 files changed, 56 insertions(+), 1 deletion(-)
 create mode 100644 rules/post/ptxd_make_repology.make

diff --git a/bin/ptxdist b/bin/ptxdist
index 0e7af6142..868eb17b3 100755
--- a/bin/ptxdist
+++ b/bin/ptxdist
@@ -799,6 +799,7 @@ Misc:
 				variable and print the contents of this variable
   licensecheck			check md5sums of license files for all packages
   lint				run some basic checks for the bsp and PTXdist
+  repology			generate repology JSON for PTXdist
   list-packages			print a list of all selected packages
   local-src <pkg> [<directory>]	overwrite a package source with a locally provided
 				directory containing the sourcecode.
@@ -1925,6 +1926,11 @@ EOF
 				ptxd_bailout "lint checks failed"
 			fi
 			;;
+		repology)
+			check_premake_compiler &&
+			ptxd_make_log ptxdist-repology
+			exit
+			;;
 		list-packages)
 			check_config &&
 			check_deps &&
diff --git a/rules/post/ptxd_make_repology.make b/rules/post/ptxd_make_repology.make
new file mode 100644
index 000000000..50c913738
--- /dev/null
+++ b/rules/post/ptxd_make_repology.make
@@ -0,0 +1,49 @@
+# -*-makefile-*-
+#
+# Copyright (C) 2025 by Bruno Thomsen <bruno.thomsen@gmail.com>
+#               2025 by Ladislav Michl <ladis@triops.cz>
+#
+# For further information about the PTXdist project and license conditions
+# see the README file.
+#
+
+PTX_PACKAGES_IGNORE	:= \
+	cross-llvm \
+	nodejs_packages
+
+PHONY += ptxdist-repology
+
+ptx_packages_reported:=$(sort $(filter-out host-system-%,$(filter-out host-%,$(filter-out image-%,$(PTX_PACKAGES_ALL)))))
+ptx_packages_reported:=$(filter-out $(PTX_PACKAGES_IGNORE),$(ptx_packages_reported))
+ptx_package_last:=$(lastword $(ptx_packages_reported))
+ptx_packages_reported:=$(filter-out $(ptx_package_last),$(ptx_packages_reported))
+
+ptx/repology-print = \
+	$(info $(space)   "name": "$(1)"$(comma)) \
+	$(info $(space)   "version": "$($(PTX_MAP_TO_PACKAGE_$(1))_VERSION)"$(comma)) \
+	$(info $(space)   "upstream_urls": "$($(PTX_MAP_TO_PACKAGE_$(1))_URL)"$(comma)) \
+	$(info $(space)   "license": "$($(PTX_MAP_TO_PACKAGE_$(1))_LICENSE)")
+
+ptx/repology-item = \
+	$(info $(space) {) \
+	$(call ptx/repology-print,$(1)) \
+	$(info $(space) }$(comma))
+
+ptx/repology-lastitem = \
+	$(info $(space) {) \
+	$(call ptx/repology-print,$(1)) \
+	$(info $(space) })
+
+ptx/repology = \
+	$(info [) \
+	$(foreach pkg,$(ptx_packages_reported), \
+		$(if $(subst undefined,,$($(PTX_MAP_TO_PACKAGE_$(pkg))_VERSION)), \
+			$(if $($(PTX_MAP_TO_PACKAGE_$(pkg))_URL), \
+				$(call ptx/repology-item,$(pkg))$(ptx/nl),),)) \
+	$(call ptx/repology-lastitem,$(ptx_package_last)) \
+	$(info ])
+
+ptxdist-repology:
+	@$(call ptx/repology)
+
+# vim: syntax=make
diff --git a/scripts/bash_completion b/scripts/bash_completion
index 4bff2d4b6..2bb21a848 100644
--- a/scripts/bash_completion
+++ b/scripts/bash_completion
@@ -123,7 +123,7 @@ _ptxdist_completion()
 	clean)
 		COMPREPLY=( $( compgen -W root -- $cur ) )
 		;&
-	get|extract|prepare|compile|install|targetinstall|tags|urlcheck|licensecheck|package-info|cargosync)
+	get|extract|prepare|compile|install|targetinstall|tags|urlcheck|licensecheck|repology|package-info|cargosync)
 		COMPREPLY+=( $( compgen -W "$(__ptxdist_completion_packages)" -- $cur ) )
 		;;
 	drop)

base-commit: de3afe806e398349482e92e812f6f69467070ec4
-- 
2.49.0

Re: [ptxdist] [PATCH v2] ptxdist: add repology json output support

[Roland Hieber]

On Wed, Oct 01, 2025 at 09:17:26PM +0200, Bruno Thomsen wrote:
> Improve ptxdist project visibility by adding it to repology.
> https://repology.org/
> 
> This also allows components like rauc to showcase supported
> distributions and current packaged version.
> 
> It also gives ptxdist users and maintainers an overview of packages
> with known CVEs that should be priotized for version bumpes.
> 
> Generate JSON output about almost all packages in ptxdist for repology.
> Below is an output example for alsa-lib:
> 
> {
>     "name": "alsa-lib",
>     "version": "1.2.13",
>     "upstream_urls": "https://www.alsa-project.org/files/pub/lib/alsa-lib-1.2.13.tar.bz2",
>     "license": "LGPL-2.1-or-later"
> },
> 
> Add PTXdist support #1487:
> https://github.com/repology/repology-updater/issues/1487
> 
> This have been running for the past few months here:
> https://github.com/baxeno/ptxdist-repology/blob/main/repology.json
> 
> Using the following command with DistroKit:
> $ ptxdist repology | grep -v "make: Nothing to be done for" > repology.json
> 
> Signed-off-by: Bruno Thomsen <bruno.thomsen@gmail.com>
> Co-developed-by: Ladislav Michl <ladis@triops.cz>
> ---
> v1-v2:
> - Keep Makefile implementation as Python implementation did not work
> - Filter out packages with empty URL
> - Filter out packages with empty or undefined version
> - Reduced number of hard-coded ignored packages from 33 to 2
> - View parsed output: https://repology.org/repository/ptxdist
> 
> PATCH v1:
> https://lore.ptxdist.org/ptxdist/20250422193456.26040-1-bruno.thomsen@gmail.com/
> 
> PATCH RFC:
> https://lore.ptxdist.org/ptxdist/20250418113357.23654-1-bruno.thomsen@gmail.com/
> 
>  bin/ptxdist                        |  6 ++++
>  rules/post/ptxd_make_repology.make | 49 ++++++++++++++++++++++++++++++
>  scripts/bash_completion            |  2 +-
>  3 files changed, 56 insertions(+), 1 deletion(-)
>  create mode 100644 rules/post/ptxd_make_repology.make
> 
> diff --git a/bin/ptxdist b/bin/ptxdist
> index 0e7af6142..868eb17b3 100755
> --- a/bin/ptxdist
> +++ b/bin/ptxdist
> @@ -799,6 +799,7 @@ Misc:
>  				variable and print the contents of this variable
>    licensecheck			check md5sums of license files for all packages
>    lint				run some basic checks for the bsp and PTXdist
> +  repology			generate repology JSON for PTXdist
>    list-packages			print a list of all selected packages
>    local-src <pkg> [<directory>]	overwrite a package source with a locally provided
>  				directory containing the sourcecode.
> @@ -1925,6 +1926,11 @@ EOF
>  				ptxd_bailout "lint checks failed"
>  			fi
>  			;;
> +		repology)
> +			check_premake_compiler &&
> +			ptxd_make_log ptxdist-repology
> +			exit
> +			;;
>  		list-packages)
>  			check_config &&
>  			check_deps &&
> diff --git a/rules/post/ptxd_make_repology.make b/rules/post/ptxd_make_repology.make
> new file mode 100644
> index 000000000..50c913738
> --- /dev/null
> +++ b/rules/post/ptxd_make_repology.make
> @@ -0,0 +1,49 @@
> +# -*-makefile-*-
> +#
> +# Copyright (C) 2025 by Bruno Thomsen <bruno.thomsen@gmail.com>
> +#               2025 by Ladislav Michl <ladis@triops.cz>
> +#
> +# For further information about the PTXdist project and license conditions
> +# see the README file.
> +#
> +
> +PTX_PACKAGES_IGNORE	:= \
> +	cross-llvm \
> +	nodejs_packages
> +
> +PHONY += ptxdist-repology
> +
> +ptx_packages_reported:=$(sort $(filter-out host-system-%,$(filter-out host-%,$(filter-out image-%,$(PTX_PACKAGES_ALL)))))
> +ptx_packages_reported:=$(filter-out $(PTX_PACKAGES_IGNORE),$(ptx_packages_reported))
> +ptx_package_last:=$(lastword $(ptx_packages_reported))
> +ptx_packages_reported:=$(filter-out $(ptx_package_last),$(ptx_packages_reported))
> +
> +ptx/repology-print = \
> +	$(info $(space)   "name": "$(1)"$(comma)) \
> +	$(info $(space)   "version": "$($(PTX_MAP_TO_PACKAGE_$(1))_VERSION)"$(comma)) \
> +	$(info $(space)   "upstream_urls": "$($(PTX_MAP_TO_PACKAGE_$(1))_URL)"$(comma)) \
> +	$(info $(space)   "license": "$($(PTX_MAP_TO_PACKAGE_$(1))_LICENSE)")
> +
> +ptx/repology-item = \
> +	$(info $(space) {) \
> +	$(call ptx/repology-print,$(1)) \
> +	$(info $(space) }$(comma))
> +
> +ptx/repology-lastitem = \
> +	$(info $(space) {) \
> +	$(call ptx/repology-print,$(1)) \
> +	$(info $(space) })
> +
> +ptx/repology = \
> +	$(info [) \
> +	$(foreach pkg,$(ptx_packages_reported), \
> +		$(if $(subst undefined,,$($(PTX_MAP_TO_PACKAGE_$(pkg))_VERSION)), \
> +			$(if $($(PTX_MAP_TO_PACKAGE_$(pkg))_URL), \
> +				$(call ptx/repology-item,$(pkg))$(ptx/nl),),)) \
> +	$(call ptx/repology-lastitem,$(ptx_package_last)) \
> +	$(info ])
> +
> +ptxdist-repology:
> +	@$(call ptx/repology)

If you add a "@touch $(PTXDIST_TEMPDIR)/repology.done" here, you don't need the
'grep -v "make: Nothing to be done for"' in the shell command line.

 - Roland

> +
> +# vim: syntax=make
> diff --git a/scripts/bash_completion b/scripts/bash_completion
> index 4bff2d4b6..2bb21a848 100644
> --- a/scripts/bash_completion
> +++ b/scripts/bash_completion
> @@ -123,7 +123,7 @@ _ptxdist_completion()
>  	clean)
>  		COMPREPLY=( $( compgen -W root -- $cur ) )
>  		;&
> -	get|extract|prepare|compile|install|targetinstall|tags|urlcheck|licensecheck|package-info|cargosync)
> +	get|extract|prepare|compile|install|targetinstall|tags|urlcheck|licensecheck|repology|package-info|cargosync)
>  		COMPREPLY+=( $( compgen -W "$(__ptxdist_completion_packages)" -- $cur ) )
>  		;;
>  	drop)
> 
> base-commit: de3afe806e398349482e92e812f6f69467070ec4
> -- 
> 2.49.0
> 
> 
> 

-- 
Roland Hieber, Pengutronix e.K.          | rhi@pengutronix.de          |
Steuerwalder Str. 21                     | https://www.pengutronix.de/ |
31137 Hildesheim, Germany                | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686         | Fax:   +49-5121-206917-5555 |