Re: [ptxdist] [APPLIED] image-root-cpio: Add custom compressor and compressor options.

[Michael Olbrich <m.olbrich@pengutronix.de>]

Thanks, applied as fbdd1b19b83569cd13d3899391473a6d4ad24494.

Michael

[sent from post-receive hook]

On Mon, 14 Mar 2022 11:14:44 +0100, Christian Melki <christian.melki@t2data.com> wrote:
> * Add possibility for custom compression.
> * Options are now issued with the compressor util.
> * Fill in some more help regarding the methods.
> * xz needs crc32 instead of crc64 for the kernel cpio.
> * Set zstd compression level to the same as for the kernel (19).
> * Use multithreading for zstd (deterministic, afaiu).
> 
> Actively decided against -T0 for xz, as compression
> is not deterministic when number of threads varies.
> 
> Signed-off-by: Christian Melki <christian.melki@t2data.com>
> Message-Id: <20220313065837.2103087-1-christian.melki@t2data.com>
> Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
> 
> diff --git a/platforms/image-root-cpio.in b/platforms/image-root-cpio.in
> index d3090dc298e9..0338175a0882 100644
> --- a/platforms/image-root-cpio.in
> +++ b/platforms/image-root-cpio.in
> @@ -15,6 +15,8 @@ menuconfig IMAGE_ROOT_CPIO
>  	  When including a cpio archive inside the kernel blob,
>  	  the kernel build system can manage compressing the cpio
>  	  archive itself.
> +	  Beware, not all compressors are inherently deterministic
> +	  when using them in a multithreaded fashion.
>  
>  if IMAGE_ROOT_CPIO
>  
> @@ -32,42 +34,57 @@ config IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ
>  	bool
>  	prompt "gz"
>  	help
> -	  Compress root.cpio with gz
> +	  Compress root.cpio with gz.
> +	  Uses default compression level.
>  
>  config IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD
>  	bool
>  	prompt "zstd"
>  	help
> -	  Compress root.cpio with zstd
> +	  Compress root.cpio with zstd.
> +	  Uses multithreaded, compression level 19.
>  
>  config IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ
>  	bool
>  	prompt "xz"
>  	help
> -	  Compress root.cpio with xz
> +	  Compress root.cpio with xz.
> +	  Uses default compression level and
> +	  crc32 checksum for kernel cpio compatibility.
>  
>  config IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP
>  	bool
>  	prompt "lzop"
>  	help
> -	  Compress root.cpio with lzop
> +	  Compress root.cpio with lzop.
> +	  Uses default compression level.
>  
>  endchoice
>  
> +config IMAGE_ROOT_CPIO_CUSTOM_COMPRESSION
> +	bool
> +	prompt "customize compression command"
> +	help
> +	  This allows specifying additional arguments to the compressor
> +	  command. Note that the command itself should not be changed.
> +	  Otherwise ptxdist cannot ensure that the tool is available.
> +
>  config IMAGE_ROOT_CPIO_COMPRESSION_SUFFIX
>  	string
> -	default ""		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_NONE
> -	default ".gz"		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ
> -	default ".zst"		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD
> -	default ".xz"		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ
> -	default ".lzo"		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP
> +	prompt "compression suffix"	if IMAGE_ROOT_CPIO_CUSTOM_COMPRESSION
> +	default ""			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_NONE
> +	default ".gz"			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ
> +	default ".zst"			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD
> +	default ".xz"			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ
> +	default ".lzo"			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP
>  
>  config IMAGE_ROOT_CPIO_COMPRESSION_UTIL
>  	string
> -	default ""		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_NONE
> -	default "gzip"		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ
> -	default "zstd"		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD
> -	default "xz"		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ
> -	default "lzop"		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP
> +	prompt "compression tool"	if IMAGE_ROOT_CPIO_CUSTOM_COMPRESSION
> +	default ""			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_NONE
> +	default "gzip"			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ
> +	default "zstd -T0 -19"		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD
> +	default "xz --check=crc32"	if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ
> +	default "lzop"			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP
>  
>  endif
> diff --git a/rules/image-root-cpio.make b/rules/image-root-cpio.make
> index fdb0ccfff1cd..447a649c8245 100644
> --- a/rules/image-root-cpio.make
> +++ b/rules/image-root-cpio.make
> @@ -26,7 +26,7 @@ IMAGE_ROOT_CPIO_CONFIG	:= cpio.config
>  
>  IMAGE_ROOT_CPIO_ENV := \
>  	FORMAT="newc" \
> -	COMPRESS=$(call remove_quotes, $(PTXCONF_IMAGE_ROOT_CPIO_COMPRESSION_UTIL))
> +	COMPRESS=$(PTXCONF_IMAGE_ROOT_CPIO_COMPRESSION_UTIL)
>  
>  ifdef PTXCONF_IMAGE_ROOT_CPIO
>  $(IMAGE_ROOT_CPIO_IMAGE):

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de