From mboxrd@z Thu Jan 1 00:00:00 1970 Delivery-date: Tue, 08 Nov 2022 13:51:10 +0100 Received: from metis.ext.pengutronix.de ([2001:67c:670:201:290:27ff:fe1d:cc33]) by lore.white.stw.pengutronix.de with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1osO4M-001N2l-9O for lore@lore.pengutronix.de; Tue, 08 Nov 2022 13:51:10 +0100 Received: from localhost ([127.0.0.1] helo=metis.ext.pengutronix.de) by metis.ext.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1osO4L-0004oQ-9L; Tue, 08 Nov 2022 13:51:09 +0100 Received: from mail.thorsis.com ([92.198.35.195]) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1osO42-0004ns-GN; Tue, 08 Nov 2022 13:50:50 +0100 Date: Tue, 8 Nov 2022 13:50:45 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=thorsis.com; s=default; t=1667911849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:content-type:content-type:in-reply-to:in-reply-to: references:references; bh=9qnMya6sU0YkZT1/5ri3Fkx9K8DCqrd0IoVEHdOrSkc=; b=pDmvqhbi0IajByZ8fITs/4FWVzFY2nwBLM9qMOSAhdb8pDOUzw6PLrJCgxJxmY7aVAlskv HgpTCpPAbEOskb+lGf+fiFFs52h31Ho4zUTBtXPmEcz6d2GCbb2aDrNIlqUXPR8fN/9+F1 wB6YNDeTVyoz0hZqcNT+HP5flVIXD8/2J9rkOqMj9CqkvudIiIPhefcvJzdnRumQoTJMZX rI3noN+AeFW7C/nYgvXiME2Rbo4anF6Jty9dYOJe4iogIDk/GcsJoteSsX5mXhnQTUNrJ/ cVhU63FT8RZ0NEmXRD6qMpkWE60f7lds3U0+1N0AD8JUkfQ3Y3hF1nZAVhW1wA== From: Alexander Dahl To: Christian Melki Message-ID: Mail-Followup-To: Christian Melki , post@lespocky.de, CG@eks-engel.de, Michael Olbrich , ptxdist@pengutronix.de References: <75f0b7f568884d33af8f5c6bc23bd5dd@eks-engel.de> <2aa3b38a-9025-d88e-8364-fea4ada10f3a@t2data.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2aa3b38a-9025-d88e-8364-fea4ada10f3a@t2data.com> X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on metis.ext.pengutronix.de X-Spam-Level: X-Spam-Status: No, score=-2.8 required=4.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_PASS,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 Subject: Re: [ptxdist] Speed up targetinstall of certain packages X-BeenThere: ptxdist@pengutronix.de X-Mailman-Version: 2.1.29 Precedence: list List-Id: PTXdist Development Mailing List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: ptxdist@pengutronix.de Cc: Michael Olbrich , post@lespocky.de, ptxdist@pengutronix.de, CG@eks-engel.de Sender: "ptxdist" X-SA-Exim-Connect-IP: 127.0.0.1 X-SA-Exim-Mail-From: ptxdist-bounces@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false Hello everyone, Am Tue, Nov 08, 2022 at 11:13:44AM +0100 schrieb Christian Melki: > > > On 11/4/22 8:12 PM, Alexander Dahl wrote: > > Not sure how that should behave. However if you want to speed up the > > build: make sure you call ptxdist with -q or --quiet parameter. The > > output on screen takes suprisingly much time, even with modern > > terminals, and especially when doing targetinstall of many many files > > (as usually the case with web frontends. been there, done that.) > > > > Greets > > Alex > > > > I have a slight disagreement here. I don't think the console is slow. You're right. As already discussed in IRC yesterday: I did some benchmarks yesterday on the targetinstall stage of a package using install_tree to install a folder containing 900+ files. Turns out: no significant difference between running with or without --quiet. Greets Alex > So I did some investigation, mostly since the slowness bugs me too. > I did a: > export PS4='+[${EPOCHREALTIME}][${BASH_SOURCE}:${LINENO}]: ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'; set -x; > as time measurement and trace in the shellscript in question (mostly scripts/lib/ptxd_make_xpkg_pkg.sh). > This pretty linear progressive cpu consumption, albeit the big chunks were due to calling of external binaries. > > Then I did a timezone package clean and targetinstall. > When targetinstalling a zone there was calls to several binaries. > > For one zone I had external calls to (in order): > echo, mkdir, printf(?), flock, ls, rm, mkdir, flock, mkdir, flock, mkdir, flock, mkdir, flock, mkdir, flock, mkdir, mkdir, mkdir, mkdir, mkdir, install, install, chmod, chmod, chown, echo. > > Some of these could be internal builtins, but the consumed time suggested otherwise. > Either way. Each install took about 26 ms and I could account the majority of that time in forking external programs and waiting for them to return. > > So my conclusion is: The whole thing is a bit slow and bash doesn't help. > > Regards, > Christian > > > > > > > Thanks for any feedback. > > > BR, > > > Christian > > >