From mboxrd@z Thu Jan 1 00:00:00 1970 Delivery-date: Sun, 13 Mar 2022 07:59:31 +0100 Received: from metis.ext.pengutronix.de ([2001:67c:670:201:290:27ff:fe1d:cc33]) by lore.white.stw.pengutronix.de with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nTICR-001FmZ-GT for lore@lore.pengutronix.de; Sun, 13 Mar 2022 07:59:31 +0100 Received: from localhost ([127.0.0.1] helo=metis.ext.pengutronix.de) by metis.ext.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1nTICQ-0005TK-9t; Sun, 13 Mar 2022 07:59:30 +0100 Received: from mail-eopbgr140071.outbound.protection.outlook.com ([40.107.14.71] helo=EUR01-VE1-obe.outbound.protection.outlook.com) by metis.ext.pengutronix.de with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nTIBj-0005TA-Kn for ptxdist@pengutronix.de; Sun, 13 Mar 2022 07:58:49 +0100 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Uiuf+SySz/JeVgmXPC8UebZ0LJNL3V5WEdfEFIXzKL7OhrEY/dfUivR6Ckjbrs9m5BOaGepADdsFqxx0j6FpTb6wSCh4uYE99slPgWNFifAztNgBTZ1gkNMiMmL3Qw5kza3QhoHDVeYWJJWaIgep2dP9U4XOioa1rUMciP0Bdesa6bktwfJDZ0fGCxe/dl+DcqLRjGd64QoK/XEFK7SZnGY7RfkDwkFUrojmtNE9PTjJbfTD9IxtdgpLNOcgDTUqzteeJ0GlYpWRybwQHqVUzR8Vx3kfFmYf1e+0n4ILxWDpuOVg8xR/P4MkkwqsUy5F8L4AZ6L19iIcMWVMaHIQ3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bTi9Lg2wkkulaNnpP0lPVuNRdiUZ7iZmQiTRAe7GaY0=; b=I/nd+WQ7gs4ec+Sfs5wR+Rs8OI6F42SM0RL7qKMzg2STfGYbSEtYJ3/Z3zIDWmbP/H2o5Hf/bLn0yDHV/+FYGlLlJdqxjpY2XMTEpoTDNGXn0gsz54AhY5TdWt9jNkrAPHV6T2LgGNQP2m1dXPpJpJLDYNKp8IfiPzKfZwbjAt7rfDU7TlcRFz7/L0LTrWECMnxMzUamnYOF5M8/8lvbkjTzcTGWefcOn/TCeMTxHTeNfztKDfLh9Wy/WWU2zQLPxc1j8FNVcNR61qiQurJ+bjfVau/JfKnNWJXSIS4gW1qq+Oqh/LMAl0W4sr4Z6eZtNSVlmDZGC4D+t5NvEEgjnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=t2data.com; dmarc=pass action=none header.from=t2data.com; dkim=pass header.d=t2data.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=t2datacom.onmicrosoft.com; s=selector1-t2datacom-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bTi9Lg2wkkulaNnpP0lPVuNRdiUZ7iZmQiTRAe7GaY0=; b=CqMkT2rSAqSadAzAQl/SFKTCL6bMNDc+bvRTUczFjmF82FZf9P/flC7aaEiTXPbNbxylwcyF6oITv4CHrjuDmW8t53IOEoDmWqqX/dOq0DafvV88VfH6YRVCdfvD0BFUa94T/IFrkbIXpIOV+i8pvFWyPscyLeud8M03yywTMnE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=t2data.com; Received: from DB9P251MB0618.EURP251.PROD.OUTLOOK.COM (2603:10a6:10:334::22) by AS8P251MB0326.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:404::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5061.26; Sun, 13 Mar 2022 06:58:44 +0000 Received: from DB9P251MB0618.EURP251.PROD.OUTLOOK.COM ([fe80::1cba:3835:dd1e:63d3]) by DB9P251MB0618.EURP251.PROD.OUTLOOK.COM ([fe80::1cba:3835:dd1e:63d3%7]) with mapi id 15.20.5061.026; Sun, 13 Mar 2022 06:58:44 +0000 From: Christian Melki To: ptxdist@pengutronix.de Date: Sun, 13 Mar 2022 07:58:37 +0100 Message-Id: <20220313065837.2103087-1-christian.melki@t2data.com> X-Mailer: git-send-email 2.32.0 X-ClientProxiedBy: GV3P280CA0009.SWEP280.PROD.OUTLOOK.COM (2603:10a6:150:b::25) To DB9P251MB0618.EURP251.PROD.OUTLOOK.COM (2603:10a6:10:334::22) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1c8d6463-a578-4181-47db-08da04bee9ea X-MS-TrafficTypeDiagnostic: AS8P251MB0326:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hV44cK11NZgUrun/ct0YhKq98/hYtt5ToLRodqb6hscQzyEoOVLn1nCB8vTunFXODH1kLZZDbALTWqB6Eby/EJ9+vGv6NuiGS8q6RQToKdSdvNrtNMMHTEzI6Fn84vfIZ7LXyuIrIaeANSfbepOryc52Xl3RogEaKb/cZDME0+dx9Tck5WLPTi7e046zApcc7HgbsUW5FfRJ7Wr1CEJU7rrEo9hnIOBdZjnGaLm0FX9aUMyTVm7c9uZv86Ik9TekvptJSpSVpE2lalOzB7pLUSq85vru4NRIFoigq38MyKjCZJDBrzcsVw/pl8px9fUKwZBnM5tdAjB7LhYPFQkQsIDff9loITbhy1G/Fn8dX4jYw20GrMHoE31bh45TxHhhLULz1MzX+ZzwCIFZoCxsS2D3u6H7KSsVXhFVRiFrf5EZCgAb+WdGuPHU4n6RPum3hLgpqTmRj/nPhUIdf8PXgev47CXxoyIztR/VlohFojDCr1WNdnlmyvSXFGDxOVqkFJKaYWfnvKlejr627oRmNaxxuusR2pzQf4ZsR7S+t9CjBO4DgZsg7remgVAkf8jTI1relIZI5cIJJVSdy2pZkfgP6gnKFNV24NsAiHrI3fNpjvIGC41AvSA7+wXnIcdj7yjo0U8ijY7kwSQukDJQfRpsgHytYoMZAu0jH6Qt9JnyRDU9CiWGRKpINfRI/4FYFTLiCzDzm66MWhXTnXFUhw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB9P251MB0618.EURP251.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230001)(346002)(366004)(39830400003)(376002)(136003)(396003)(66556008)(66946007)(38100700002)(38350700002)(2906002)(36756003)(86362001)(5660300002)(8936002)(8676002)(66476007)(44832011)(26005)(52116002)(6666004)(6506007)(6512007)(186003)(83380400001)(508600001)(6916009)(316002)(1076003)(6486002)(2616005); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?KNS0AjOYEgGsvxlNZTXqMqwPNPPARgYAyHp4bFQvZ+nCQgHCm8EvjK1tukEG?= =?us-ascii?Q?JEnDvPn59VnthqIKeMrF72A9oCdXsGFVQP8YYbqeLfiGeXlrTkzMJNWyLLEJ?= =?us-ascii?Q?/tQVatEISJy5fcq7a6vmbv7fgBGK76xTxPtyPu7b7v5bTraJ9nXoE7y4qU78?= =?us-ascii?Q?fiSZ3l6Z0sYdLssfCZMKaex58J9tP8bhEvp3hgVCPkOn/jPWg9BKe0aS7uXs?= =?us-ascii?Q?0BQwGHTYtOP9otdEksnfIbIQ4d+fRGndvNlJY1nSrzlkLoNg+08w4fCTRZT6?= =?us-ascii?Q?K8Pi8uq9LSeOu0Ldbjy2nyT6tPhATk+1gFozeXwx7OewLcOOCjx8L3KOZeHh?= =?us-ascii?Q?GF+/ppIkPC1GImOcG/epg4jlcU3ip/9BzmFuJIs7CDMG4OQ0RKLwjKzQpZFu?= =?us-ascii?Q?FT/CMgy/VCaVS6MGTITFUQzEY9ULvqdzTIujO9IVAO54dWoSf7KMs5hoeCJN?= =?us-ascii?Q?tBfEiCrHMSvF8ThcpsWZWbszEDfDQ0RVWtfYRo3iw+dqhCRIHrJpKmf4Kin3?= =?us-ascii?Q?FGU26rBGiD1CaAYF1HwYxdm1NXtiZHPXknZ1jKslUQWDPeVSQbWwJgID73Fl?= =?us-ascii?Q?3sL8eUyPpCu+fOnKdPN5l8r07xxf0imWA3c9aC8Z8Z4iAsGD6l2C25+3e/YQ?= =?us-ascii?Q?0axXzLgF6w8rHXGCiaaZCSUGcLLkCfvQHyIKzFGSChSLYzllwQ2qN7oSpmgO?= =?us-ascii?Q?F7B3VcqSmBtAqNZtzRt0mO2appiMpVTQbrEvP5jlg/1LBaikOWt6ea2FGmzf?= =?us-ascii?Q?3oz76zOnzZLVy1PCQLfYU9QAzevxzW4ry7IKUD/ql0DWOUW6sZD9PXDEvaqO?= =?us-ascii?Q?VYp7jSech0XSx4eWpFYoZtmWmiJEqJUVavmnmbLGZR7CgrpR0ge4ZRapj1aC?= =?us-ascii?Q?JP5AuTbTJS2asW1WyAPOjpnIuXI/x4u4DXAd2ETJsifLGSKKpsNy8vNUrJsg?= =?us-ascii?Q?fV3Na+8SxSFL0++tT9ZHGfN0lpostB3fVdSvUWMPTK0B0lwNI+BteoNGupaa?= =?us-ascii?Q?Vde07bZuT36pStut7GAz+KJMvTsUoPs7TA+ycZBpU70obqEGNYqFNeqlQUAg?= =?us-ascii?Q?7LtJcCbX8ZGQCe9GKAeiQM87MNPGQybkP3GUyDhmE3ugPjSCqfSFUB2w1yuj?= =?us-ascii?Q?u39GkuUoyKZrY3ML6aCaWHHlJaKQgDkl6AWUItJqczFY//k28czSLWGIHDJz?= =?us-ascii?Q?L2k5kI4qyFDk3JjhsVD1HBbkiF0PyMn9E7dpPca7FTrEkf4fqfhosddF6/Hw?= =?us-ascii?Q?GBR2kQTqFpOzKBQ1OohyPvyoxF8pMDHXmMBFqFSVNhatfqjJk1JzW6hP5AtS?= =?us-ascii?Q?IvnWG61MvChknB8WRvQ9nAIblWtbMGOV7Y/RYnsDdJMzUxo1U3mhexrZpNqw?= =?us-ascii?Q?CffP28YYej6HLSrxNLc5jc1no+3aLusSjyAorer5ht302xHIhnIRLwuJE35D?= =?us-ascii?Q?gTetYPKVFydSAExgIuub71C19rqxCYGPhFjif6KgNE3YR5L9YjI8UQ=3D=3D?= X-OriginatorOrg: t2data.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1c8d6463-a578-4181-47db-08da04bee9ea X-MS-Exchange-CrossTenant-AuthSource: DB9P251MB0618.EURP251.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Mar 2022 06:58:44.3287 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 27928da5-aacd-4ba1-9566-c748a6863e6c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7YWi15wr5Av517MR+wopCBSfgZFZA6QcNZfdll3ixAJiQts6VspozyNyvcHV6SjOMrHi5li17XSE03Dhnbg0+rnLlHG9LJ6bQbYpkBsK1qw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8P251MB0326 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on metis.ext.pengutronix.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=4.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.2 Subject: [ptxdist] [PATCH v3] image-root-cpio: Add custom compressor and compressor options. X-BeenThere: ptxdist@pengutronix.de X-Mailman-Version: 2.1.29 Precedence: list List-Id: PTXdist Development Mailing List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: ptxdist@pengutronix.de Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "ptxdist" X-SA-Exim-Connect-IP: 127.0.0.1 X-SA-Exim-Mail-From: ptxdist-bounces@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false * Add possibility for custom compression. * Options are now issued with the compressor util. * Fill in some more help regarding the methods. * xz needs crc32 instead of crc64 for the kernel cpio. * Set zstd compression level to the same as for the kernel (19). * Use multithreading for zstd (deterministic, afaiu). Actively decided against -T0 for xz, as compression is not deterministic when number of threads varies. Signed-off-by: Christian Melki --- platforms/image-root-cpio.in | 45 +++++++++++++++++++++++++----------- rules/image-root-cpio.make | 2 +- 2 files changed, 32 insertions(+), 15 deletions(-) diff --git a/platforms/image-root-cpio.in b/platforms/image-root-cpio.in index d3090dc29..0338175a0 100644 --- a/platforms/image-root-cpio.in +++ b/platforms/image-root-cpio.in @@ -15,6 +15,8 @@ menuconfig IMAGE_ROOT_CPIO When including a cpio archive inside the kernel blob, the kernel build system can manage compressing the cpio archive itself. + Beware, not all compressors are inherently deterministic + when using them in a multithreaded fashion. if IMAGE_ROOT_CPIO @@ -32,42 +34,57 @@ config IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ bool prompt "gz" help - Compress root.cpio with gz + Compress root.cpio with gz. + Uses default compression level. config IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD bool prompt "zstd" help - Compress root.cpio with zstd + Compress root.cpio with zstd. + Uses multithreaded, compression level 19. config IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ bool prompt "xz" help - Compress root.cpio with xz + Compress root.cpio with xz. + Uses default compression level and + crc32 checksum for kernel cpio compatibility. config IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP bool prompt "lzop" help - Compress root.cpio with lzop + Compress root.cpio with lzop. + Uses default compression level. endchoice +config IMAGE_ROOT_CPIO_CUSTOM_COMPRESSION + bool + prompt "customize compression command" + help + This allows specifying additional arguments to the compressor + command. Note that the command itself should not be changed. + Otherwise ptxdist cannot ensure that the tool is available. + config IMAGE_ROOT_CPIO_COMPRESSION_SUFFIX string - default "" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_NONE - default ".gz" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ - default ".zst" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD - default ".xz" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ - default ".lzo" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP + prompt "compression suffix" if IMAGE_ROOT_CPIO_CUSTOM_COMPRESSION + default "" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_NONE + default ".gz" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ + default ".zst" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD + default ".xz" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ + default ".lzo" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP config IMAGE_ROOT_CPIO_COMPRESSION_UTIL string - default "" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_NONE - default "gzip" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ - default "zstd" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD - default "xz" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ - default "lzop" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP + prompt "compression tool" if IMAGE_ROOT_CPIO_CUSTOM_COMPRESSION + default "" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_NONE + default "gzip" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_GZ + default "zstd -T0 -19" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_ZSTD + default "xz --check=crc32" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_XZ + default "lzop" if IMAGE_ROOT_CPIO_COMPRESSION_MODE_LZOP endif diff --git a/rules/image-root-cpio.make b/rules/image-root-cpio.make index fdb0ccfff..447a649c8 100644 --- a/rules/image-root-cpio.make +++ b/rules/image-root-cpio.make @@ -26,7 +26,7 @@ IMAGE_ROOT_CPIO_CONFIG := cpio.config IMAGE_ROOT_CPIO_ENV := \ FORMAT="newc" \ - COMPRESS=$(call remove_quotes, $(PTXCONF_IMAGE_ROOT_CPIO_COMPRESSION_UTIL)) + COMPRESS=$(PTXCONF_IMAGE_ROOT_CPIO_COMPRESSION_UTIL) ifdef PTXCONF_IMAGE_ROOT_CPIO $(IMAGE_ROOT_CPIO_IMAGE): -- 2.32.0 _______________________________________________ ptxdist mailing list ptxdist@pengutronix.de To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de