Re: [ptxdist] [RFC PATCH] ptxdist: add repology sub command

[Bruno Thomsen <bruno.thomsen@gmail.com>]

Den lør. 26. apr. 2025 kl. 12.13 skrev Michael Olbrich
<m.olbrich@pengutronix.de>:
>
> On Fri, Apr 18, 2025 at 01:33:57PM +0200, Bruno Thomsen wrote:
> > Improve ptxdist project visibility by adding it to repology.
> >
> > Generate JSON output about all packages in ptxdist for repology.
> >
> > ptxdist repology | tail -n +7 | jq
> >
> > This is just a POC for integrating ptxdist with repology.
> >
> > Add PTXdist support #1487:
> > https://github.com/repology/repology-updater/issues/1487
> >
> > I have run this patch against ptxdist-2024.12.0 and got this output:
> > https://github.com/baxeno/ptxdist-repology/blob/main/repology.json
> >
> > Signed-off-by: Bruno Thomsen <bruno.thomsen@gmail.com>
> > ---
> > Any feedback is welcome as this is a very rough implementation.
>
> Hmm, I think it makes more sense to add this to the report stuff in
> scripts/report/. That should be a pretty simple python script.
> You can still add a command and make target to call this but there is no
> need for manual parsing.

Welcome back :)
There is a newer patch version that removes the bash script and does it all
in makefile. Maybe it should be converted to a python script. This "report"
is a bit different since it includes all ptxdist packages and not just
the packages
used in a project.

/Bruno

>
> Michael
>
> >
> >  bin/ptxdist                             |  6 +++
> >  rules/post/ptxd_make_repology.make      | 20 +++++++++
> >  scripts/bash_completion                 |  2 +-
> >  scripts/lib/ptxd_make_world_repology.sh | 54 +++++++++++++++++++++++++
> >  4 files changed, 81 insertions(+), 1 deletion(-)
> >  create mode 100644 rules/post/ptxd_make_repology.make
> >  create mode 100644 scripts/lib/ptxd_make_world_repology.sh
> >
> > diff --git a/bin/ptxdist b/bin/ptxdist
> > index 1f561fdf6..24a8e2ed9 100755
> > --- a/bin/ptxdist
> > +++ b/bin/ptxdist
> > @@ -799,6 +799,7 @@ Misc:
> >                               variable and print the contents of this variable
> >    licensecheck                       check md5sums of license files for all packages
> >    lint                               run some basic checks for the bsp and PTXdist
> > +  repology                   generate repology JSON for PTXdist
> >    list-packages                      print a list of all selected packages
> >    local-src <pkg> [<directory>]      overwrite a package source with a locally provided
> >                               directory containing the sourcecode.
> > @@ -1916,6 +1917,11 @@ EOF
> >                               ptxd_bailout "lint checks failed"
> >                       fi
> >                       ;;
> > +             repology)
> > +                     check_premake_compiler &&
> > +                     ptxd_make_log ptxdist-repology
> > +                     exit
> > +                     ;;
> >               list-packages)
> >                       check_config &&
> >                       check_deps &&
> > diff --git a/rules/post/ptxd_make_repology.make b/rules/post/ptxd_make_repology.make
> > new file mode 100644
> > index 000000000..b243ed03e
> > --- /dev/null
> > +++ b/rules/post/ptxd_make_repology.make
> > @@ -0,0 +1,20 @@
> > +# -*-makefile-*-
> > +#
> > +# Copyright (C) 2025 by Bruno Thomsen <bruno.thomsen@gmail.com>
> > +#
> > +# For further information about the PTXdist project and license conditions
> > +# see the README file.
> > +#
> > +
> > +PHONY += ptxdist-repology
> > +
> > +ptx/repology = \
> > +     $(call ptx/env) \
> > +     ptx_dgen_rulesfiles_make="$(PTX_DGEN_RULESFILES_MAKE)" \
> > +     ptxd_make_world_repology
> > +
> > +ptxdist-repology:
> > +     @$(call targetinfo)
> > +     @$(call ptx/repology)
> > +
> > +# vim: syntax=make
> > diff --git a/scripts/bash_completion b/scripts/bash_completion
> > index 4bff2d4b6..2bb21a848 100644
> > --- a/scripts/bash_completion
> > +++ b/scripts/bash_completion
> > @@ -123,7 +123,7 @@ _ptxdist_completion()
> >       clean)
> >               COMPREPLY=( $( compgen -W root -- $cur ) )
> >               ;&
> > -     get|extract|prepare|compile|install|targetinstall|tags|urlcheck|licensecheck|package-info|cargosync)
> > +     get|extract|prepare|compile|install|targetinstall|tags|urlcheck|licensecheck|repology|package-info|cargosync)
> >               COMPREPLY+=( $( compgen -W "$(__ptxdist_completion_packages)" -- $cur ) )
> >               ;;
> >       drop)
> > diff --git a/scripts/lib/ptxd_make_world_repology.sh b/scripts/lib/ptxd_make_world_repology.sh
> > new file mode 100644
> > index 000000000..2b3a0a098
> > --- /dev/null
> > +++ b/scripts/lib/ptxd_make_world_repology.sh
> > @@ -0,0 +1,54 @@
> > +#!/bin/bash
> > +#
> > +# Copyright (C) 2025 by Bruno Thomsen <bruno.thomsen@gmail.com>
> > +#
> > +# For further information about the PTXdist project and license conditions
> > +# see the README file.
> > +#
> > +
> > +ptxd_make_world_repology() {
> > +     local filefd file
> > +     local first
> > +     ptxd_make_world_init
> > +
> > +     echo "Locating all target packages ..."
> > +
> > +     first=1
> > +     exec {filefd}< <(ptxd_make_world_lint_makefiles)
> > +     while read file <&${filefd}; do
> > +     filename="${file##*/}"
> > +
> > +     case "${filename}" in
> > +             host-system-*|host-*|cross-*|image-*)
> > +             continue
> > +             ;;
> > +             *)
> > +             ;;
> > +     esac
> > +     case "${file}" in
> > +             */rules/post/*|*/rules/pre/*)
> > +                     continue
> > +                     ;;
> > +             *)
> > +                     ;;
> > +     esac
> > +     grep -q '^[^    ]*_VERSION[     :]*=' "${file}" || continue
> > +
> > +     pkg_name=$(grep '^PACKAGES-$(PTXCONF_' "${file}" | cut -d '=' -f 2 | xargs)
> > +     pkg_version=$(grep '^[^         ]*_VERSION[     :]*=' "${file}" | cut -d '=' -f 2 | xargs)
> > +     pkg_license=$(grep '^[^         ]*_LICENSE[     :]*=' "${file}" | cut -d '=' -f 2 | xargs)
> > +
> > +     [[ "$pkg_license" == *"call remove_quotes"* ]] && continue
> > +     [[ "$pkg_version" == *"call ptx/"* ]] && continue
> > +
> > +     if [ $first -eq 1 ]; then
> > +             first=0
> > +             echo "["
> > +     else
> > +             echo ","
> > +     fi
> > +     echo "{\"name\": \"${pkg_name}\", \"version\": \"${pkg_version}\", \"license\": \"${pkg_license}\"}"
> > +     done
> > +     echo "]"
> > +}
> > +export -f ptxd_make_world_repology
> >
> > base-commit: f61905c23240642dec3e5390ee15e83cf1e016e9
> > --
> > 2.49.0
> >
> >
> >
>
> --
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |