Re: [ptxdist] [RFC PATCH] ptxdist: add repology sub command

[Ladislav Michl <oss-lists@triops.cz>]

Hello Bruno,

On Fri, Apr 18, 2025 at 01:33:57PM +0200, Bruno Thomsen wrote:
> Improve ptxdist project visibility by adding it to repology.

This is just cool :) A little remark bellow...

> Generate JSON output about all packages in ptxdist for repology.
> 
> ptxdist repology | tail -n +7 | jq
> 
> This is just a POC for integrating ptxdist with repology. 
> 
> Add PTXdist support #1487:
> https://github.com/repology/repology-updater/issues/1487
> 
> I have run this patch against ptxdist-2024.12.0 and got this output:
> https://github.com/baxeno/ptxdist-repology/blob/main/repology.json
> 
> Signed-off-by: Bruno Thomsen <bruno.thomsen@gmail.com>
> ---
> Any feedback is welcome as this is a very rough implementation.
> 
>  bin/ptxdist                             |  6 +++
>  rules/post/ptxd_make_repology.make      | 20 +++++++++
>  scripts/bash_completion                 |  2 +-
>  scripts/lib/ptxd_make_world_repology.sh | 54 +++++++++++++++++++++++++
>  4 files changed, 81 insertions(+), 1 deletion(-)
>  create mode 100644 rules/post/ptxd_make_repology.make
>  create mode 100644 scripts/lib/ptxd_make_world_repology.sh
> 
> diff --git a/bin/ptxdist b/bin/ptxdist
> index 1f561fdf6..24a8e2ed9 100755
> --- a/bin/ptxdist
> +++ b/bin/ptxdist
> @@ -799,6 +799,7 @@ Misc:
>  				variable and print the contents of this variable
>    licensecheck			check md5sums of license files for all packages
>    lint				run some basic checks for the bsp and PTXdist
> +  repology			generate repology JSON for PTXdist
>    list-packages			print a list of all selected packages
>    local-src <pkg> [<directory>]	overwrite a package source with a locally provided
>  				directory containing the sourcecode.
> @@ -1916,6 +1917,11 @@ EOF
>  				ptxd_bailout "lint checks failed"
>  			fi
>  			;;
> +		repology)
> +			check_premake_compiler &&
> +			ptxd_make_log ptxdist-repology
> +			exit
> +			;;
>  		list-packages)
>  			check_config &&
>  			check_deps &&
> diff --git a/rules/post/ptxd_make_repology.make b/rules/post/ptxd_make_repology.make
> new file mode 100644
> index 000000000..b243ed03e
> --- /dev/null
> +++ b/rules/post/ptxd_make_repology.make
> @@ -0,0 +1,20 @@
> +# -*-makefile-*-
> +#
> +# Copyright (C) 2025 by Bruno Thomsen <bruno.thomsen@gmail.com>
> +#
> +# For further information about the PTXdist project and license conditions
> +# see the README file.
> +#
> +
> +PHONY += ptxdist-repology
> +
> +ptx/repology = \
> +	$(call ptx/env) \
> +	ptx_dgen_rulesfiles_make="$(PTX_DGEN_RULESFILES_MAKE)" \
> +	ptxd_make_world_repology

Here would be probably better to pass $(PTX_PACKAGES_INSTALL) and possibly sort it.

> +ptxdist-repology:
> +	@$(call targetinfo)
> +	@$(call ptx/repology)
> +
> +# vim: syntax=make
> diff --git a/scripts/bash_completion b/scripts/bash_completion
> index 4bff2d4b6..2bb21a848 100644
> --- a/scripts/bash_completion
> +++ b/scripts/bash_completion
> @@ -123,7 +123,7 @@ _ptxdist_completion()
>  	clean)
>  		COMPREPLY=( $( compgen -W root -- $cur ) )
>  		;&
> -	get|extract|prepare|compile|install|targetinstall|tags|urlcheck|licensecheck|package-info|cargosync)
> +	get|extract|prepare|compile|install|targetinstall|tags|urlcheck|licensecheck|repology|package-info|cargosync)
>  		COMPREPLY+=( $( compgen -W "$(__ptxdist_completion_packages)" -- $cur ) )
>  		;;
>  	drop)
> diff --git a/scripts/lib/ptxd_make_world_repology.sh b/scripts/lib/ptxd_make_world_repology.sh
> new file mode 100644
> index 000000000..2b3a0a098
> --- /dev/null
> +++ b/scripts/lib/ptxd_make_world_repology.sh
> @@ -0,0 +1,54 @@
> +#!/bin/bash
> +#
> +# Copyright (C) 2025 by Bruno Thomsen <bruno.thomsen@gmail.com>
> +#
> +# For further information about the PTXdist project and license conditions
> +# see the README file.
> +#
> +
> +ptxd_make_world_repology() {
> +	local filefd file
> +	local first
> +	ptxd_make_world_init
> +
> +	echo "Locating all target packages ..."
> +
> +	first=1
> +	exec {filefd}< <(ptxd_make_world_lint_makefiles)
> +	while read file <&${filefd}; do
> +	filename="${file##*/}"
> +
> +	case "${filename}" in
> +		host-system-*|host-*|cross-*|image-*)
> +		continue
> +		;;
> +		*)
> +		;;
> +	esac
> +	case "${file}" in
> +		*/rules/post/*|*/rules/pre/*)
> +			continue
> +			;;
> +		*)
> +			;;
> +	esac
> +	grep -q '^[^ 	]*_VERSION[ 	:]*=' "${file}" || continue
> +
> +	pkg_name=$(grep '^PACKAGES-$(PTXCONF_' "${file}" | cut -d '=' -f 2 | xargs)
> +	pkg_version=$(grep '^[^ 	]*_VERSION[ 	:]*=' "${file}" | cut -d '=' -f 2 | xargs)
> +	pkg_license=$(grep '^[^ 	]*_LICENSE[ 	:]*=' "${file}" | cut -d '=' -f 2 | xargs)

...and here use that list as this loop fails for valgring, which lists:
PACKAGES-$(PTXCONF_ARCH_X86)-$(PTXCONF_VALGRIND) += valgrind
PACKAGES-$(PTXCONF_ARCH_PPC)-$(PTXCONF_VALGRIND) += valgrind
PACKAGES-$(PTXCONF_ARCH_ARM)-$(PTXCONF_VALGRIND) += valgrind
PACKAGES-$(PTXCONF_ARCH_ARM64)-$(PTXCONF_VALGRIND) += valgrind

> +	[[ "$pkg_license" == *"call remove_quotes"* ]] && continue
> +	[[ "$pkg_version" == *"call ptx/"* ]] && continue
> +
> +	if [ $first -eq 1 ]; then
> +		first=0
> +		echo "["
> +	else
> +		echo ","
> +	fi
> +	echo "{\"name\": \"${pkg_name}\", \"version\": \"${pkg_version}\", \"license\": \"${pkg_license}\"}"
> +	done
> +	echo "]"
> +}
> +export -f ptxd_make_world_repology
> 
> base-commit: f61905c23240642dec3e5390ee15e83cf1e016e9
> -- 
> 2.49.0
>