mailarchive of the ptxdist mailing list
 help / color / mirror / Atom feed
From: Christian Melki <christian.melki@t2data.com>
To: ptxdist@pengutronix.de
Subject: [ptxdist] [PATCH] image-root-cpio.in: Use multithreaded XZ (revisited).
Date: Sun, 25 Feb 2024 15:49:06 +0100	[thread overview]
Message-ID: <20240225144906.2406989-1-christian.melki@t2data.com> (raw)

So, after asking questions about the default multithread mode
of xz 5.6.0 and determinism, it turned out that there
is a common misconception about determinism in xz.

XZ compression varies between the two compressors. Ie, the single
thread and multi thread variant. T0 always uses the multithread
one, even if used on one core. But T1 uses the singlethread
compressor.

This has caused a very common misconception about determinism
variability with number of threads. I've seen this discussion
on a lot of places. So albeit true, it only applies to the
distinct compressors, not how many threads are used in
multithread mode.

Now after giving this some thought I've decided to reenable
threaded XZ, using T0. T0 should produce the same results always,
on any target. Even unicore machines. And with the multithread
compressor now being the default, users should end up with the
same results unless they are actively using T1.

Signed-off-by: Christian Melki <christian.melki@t2data.com>
---
 platforms/image-root-cpio.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/platforms/image-root-cpio.in b/platforms/image-root-cpio.in
index 0338175a0..546dda7fb 100644
--- a/platforms/image-root-cpio.in
+++ b/platforms/image-root-cpio.in
@@ -84,7 +84,7 @@ config IMAGE_ROOT_CPIO_COMPRESSION_UTIL
 	default ""			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_NONE
 	default "gzip"			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ
 	default "zstd -T0 -19"		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD
-	default "xz --check=crc32"	if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ
+	default "xz -T0 --check=crc32"	if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ
 	default "lzop"			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP
 
 endif
-- 
2.34.1




             reply	other threads:[~2024-02-25 14:49 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-02-25 14:49 Christian Melki [this message]
2024-03-07 16:04 ` Michael Olbrich
2024-03-08 18:04   ` Christian Melki

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240225144906.2406989-1-christian.melki@t2data.com \
    --to=christian.melki@t2data.com \
    --cc=ptxdist@pengutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox