mailarchive of the ptxdist mailing list
 help / color / mirror / Atom feed
* [ptxdist] [PATCH] image-root-cpio.in: Use multithreaded XZ (revisited).
@ 2024-02-25 14:49 Christian Melki
  2024-03-07 16:04 ` Michael Olbrich
  0 siblings, 1 reply; 3+ messages in thread
From: Christian Melki @ 2024-02-25 14:49 UTC (permalink / raw)
  To: ptxdist

So, after asking questions about the default multithread mode
of xz 5.6.0 and determinism, it turned out that there
is a common misconception about determinism in xz.

XZ compression varies between the two compressors. Ie, the single
thread and multi thread variant. T0 always uses the multithread
one, even if used on one core. But T1 uses the singlethread
compressor.

This has caused a very common misconception about determinism
variability with number of threads. I've seen this discussion
on a lot of places. So albeit true, it only applies to the
distinct compressors, not how many threads are used in
multithread mode.

Now after giving this some thought I've decided to reenable
threaded XZ, using T0. T0 should produce the same results always,
on any target. Even unicore machines. And with the multithread
compressor now being the default, users should end up with the
same results unless they are actively using T1.

Signed-off-by: Christian Melki <christian.melki@t2data.com>
---
 platforms/image-root-cpio.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/platforms/image-root-cpio.in b/platforms/image-root-cpio.in
index 0338175a0..546dda7fb 100644
--- a/platforms/image-root-cpio.in
+++ b/platforms/image-root-cpio.in
@@ -84,7 +84,7 @@ config IMAGE_ROOT_CPIO_COMPRESSION_UTIL
 	default ""			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_NONE
 	default "gzip"			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ
 	default "zstd -T0 -19"		if IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD
-	default "xz --check=crc32"	if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ
+	default "xz -T0 --check=crc32"	if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ
 	default "lzop"			if IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP
 
 endif
-- 
2.34.1




^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2024-03-08 18:07 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-02-25 14:49 [ptxdist] [PATCH] image-root-cpio.in: Use multithreaded XZ (revisited) Christian Melki
2024-03-07 16:04 ` Michael Olbrich
2024-03-08 18:04   ` Christian Melki

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox