Re: [ptxdist] [PATCH 2/2] scripts/lib: also create CSV lists on "ptxdist make license-report"

[Michael Olbrich <m.olbrich@pengutronix.de>]

On Wed, Mar 28, 2018 at 10:23:07PM +0200, Andreas Pretzsch wrote:
> When calling "ptxdist make license-report", license reports for
> both the target- and the host-tools-packages are created.
> 
> Based upon the license info files created during the regular build
> process, LaTeX files are generated, which are then converted into
> matching PDFs, containing package informations, license tags as well
> as the licenses themselves.
> These are called "report/license-report.pdf" respectively
> "report/license-report-tools.pdf".
> 
> Extend this process to also create a list of these packages with the
> most basic fields as CSV:
>     "name"|"version"|"licenses"|"flags"
>     "somepackage"|"1.2.3"|"Apache-2.0"|"attribution"
>     [...]
> 
> These are called "report/license-report.csv" respectively
> "report/license-report-tools.csv".
> 
> The compliance report and distribution creation, trigger via
> "ptxdist make license-compliance-distribution", is not modified,
> as it essentially contains just a subset of the above packages,
> namely all the Free and Open Source Software.
> This could be extended by a CSV file, too, but it will not aid
> in any verification of the overall distribution, as this has to
> be checked in total, not only with respect to the FOSS parts.
> 
> Signed-off-by: Andreas Pretzsch <apr@cn-eng.de>
> ---
> Tested with PTXdist 2016.01.0, but given the minimal differences
> against PTXdist 2018.03.0, I assume this is still fine.
> 
>  scripts/lib/ptxd_make_license_report.sh | 21 ++++++++++++
>  scripts/lib/ptxd_make_world_license.sh  | 59 +++++++++++++++++++++++++++++++++
>  2 files changed, 80 insertions(+)
> 
> diff --git a/scripts/lib/ptxd_make_license_report.sh b/scripts/lib/ptxd_make_license_report.sh
> index a6410d1..d6570ca 100644
> --- a/scripts/lib/ptxd_make_license_report.sh
> +++ b/scripts/lib/ptxd_make_license_report.sh
> @@ -194,6 +194,27 @@ ptxd_make_license_report() {
>  	ptxd_make_license_report_footer
>      ) > "${ptx_license_target_tex}"
>  
> +    ptx_license_target_csv="${ptx_report_dir}/${pkg_section}/$(basename "${ptx_license_target%.pdf}.csv")"
> +    (
> +	local pkg_csv_file
> +	ptxd_make_world_license_write_csv_header
> +	for pkg in ${@}; do
> +		pkg_lic="${ptxd_package_license_association[${pkg}]}"
> +		if [ -z ${pkg_lic} ]; then
> +			continue
> +		fi
> +		pkg_lic="${pkg_lic}/${pkg}"
> +
> +		pkg_csv_file="${ptx_report_dir}/${pkg_lic}/license-report.csv"
> +		if [ -r "${pkg_csv_file}" ]; then
> +			cat "${pkg_csv_file}"
> +		else
> +			ptxd_bailout "no $(ptxd_print_path "${pkg_csv_file}")"
> +		fi
> +	done
> +	ptxd_make_world_license_write_csv_footer
> +    ) > "${ptx_license_target_csv}"
> +

Please create separate ptxd_make_license_report_tex /
ptxd_make_license_report_csv functions that are called by
ptxd_make_license_report.

>      ptxd_make_license_report_build
>  #    mv "${ptx_license_target_tex%.tex}.pdf" "${ptx_license_target}"
>  }
> diff --git a/scripts/lib/ptxd_make_world_license.sh b/scripts/lib/ptxd_make_world_license.sh
> index bba8ca3..39c696c 100644
> --- a/scripts/lib/ptxd_make_world_license.sh
> +++ b/scripts/lib/ptxd_make_world_license.sh
> @@ -209,6 +209,63 @@ ptxd_make_world_license_write_tex() {
>  }
>  export -f ptxd_make_world_license_write_tex
>  
> +#
> +# echo header for generated CSV:
> +# "name"|"version"|"licenses"|"flags"
> +#
> +ptxd_make_world_license_write_csv_header() {
> +	local -r STRING_QUOTE='"'
> +	local -r CSV_SEPARATOR='|'

Lower case for local variables. Also, I think a helper to write one line
makes sense. Untested:

helper() {
	string_quote='"'
	csv_separator='|'
	while [ ${#} -gt 0 ]; do
		if [ -n "${1}" ]; then
			echo -n "${string_quote}${1}${string_quote}"
		fi
		shift
		if [ [ ${#} -gt 0 ]; do
			echo -n ${csv_separator}"
		fi
	done
	echo
}

use it here...

> +	local l
> +	l="${STRING_QUOTE}name${STRING_QUOTE}"
> +	l="${l}${CSV_SEPARATOR}"
> +	l="${l}${STRING_QUOTE}version${STRING_QUOTE}"
> +	l="${l}${CSV_SEPARATOR}"
> +	l="${l}${STRING_QUOTE}licenses${STRING_QUOTE}"
> +	l="${l}${CSV_SEPARATOR}"
> +	l="${l}${STRING_QUOTE}flags${STRING_QUOTE}"
> +	l="${l}\n"
> +	echo -ne "${l}"
> +}
> +export -f ptxd_make_world_license_write_csv_header
> +#
> +# generate a CSV for the package:
> +# "somepackage"|"1.2.3"|"Apache-2.0"|"attribution"
> +#
> +ptxd_make_world_license_write_csv() {
> +	local -r STRING_QUOTE='"'
> +	local -r CSV_SEPARATOR='|'
> +	local -a flags=( "${!pkg_license_flags[@]}" )
> +	local l
> +
> +	# not required, seems to be filtered before:
> +	#case "${pkg_license}" in
> +	#*ignore*)
> +	#	# ignore this package, e.g. do not list it in the report
> +	#	return 0
> +	#	;;
> +	#esac
> +
> +	l="${STRING_QUOTE}${pkg_label}${STRING_QUOTE}"
> +	l="${l}${CSV_SEPARATOR}"
> +	[ -n "${pkg_version}" ]	&& l="${l}${STRING_QUOTE}${pkg_version}${STRING_QUOTE}"
> +	l="${l}${CSV_SEPARATOR}"
> +	[ -n "${pkg_license}" ]	&& l="${l}${STRING_QUOTE}${pkg_license}${STRING_QUOTE}"
> +	l="${l}${CSV_SEPARATOR}"
> +	[ -n "${flags[*]}" ]	&& l="${l}${STRING_QUOTE}${flags[*]}${STRING_QUOTE}"
> +	l="${l}\n"
> +	echo -ne "${l}"

... and here.


Michael

> +}
> +export -f ptxd_make_world_license_write_csv
> +#
> +# echo footer for generated CSV
> +#
> +ptxd_make_world_license_write_csv_footer() {
> +	# nothing to to
> +	/bin/true
> +}
> +export -f ptxd_make_world_license_write_csv_footer
> +
>  # Copy all patches according to the series file
>  # $1 full path to the series file
>  # $2 source directory
> @@ -465,6 +522,8 @@ changed: ${md5} -> $(md5sum "${lic}" | sed 's/ .*//')
>          sed -e 's/%/\\%/g' > "${pkg_license_dir}/license-report.tex" &&
>      check_pipe_status &&
>  
> +    ptxd_make_world_license_write_csv > "${pkg_license_dir}/license-report.csv" &&
> +
>      echo "${pkg_license}" > "${pkg_license_dir}/license-name" &&
>      if [ "${#pkg_license_flags[@]}" -gt 0 ]; then
>  	echo "${!pkg_license_flags[@]}" > "${pkg_license_dir}/license-flags"
> -- 
> 2.9.3
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de