Re: [ptxdist] Speed up targetinstall of certain packages

[Alexander Dahl <ada@thorsis.com>]

Hello everyone,

Am Tue, Nov 08, 2022 at 11:13:44AM +0100 schrieb Christian Melki:
> 
> 
> On 11/4/22 8:12 PM, Alexander Dahl wrote:
> > Not sure how that should behave.  However if you want to speed up the
> > build: make sure you call ptxdist with -q or --quiet parameter.  The
> > output on screen takes suprisingly much time, even with modern
> > terminals, and especially when doing targetinstall of many many files
> > (as usually the case with web frontends. been there, done that.)
> > 
> > Greets
> > Alex
> > 
> 
> I have a slight disagreement here. I don't think the console is slow.

You're right.  As already discussed in IRC yesterday: I did some
benchmarks yesterday on the targetinstall stage of a package using
install_tree to install a folder containing 900+ files.  Turns out: no
significant difference between running with or without --quiet.

Greets
Alex

> So I did some investigation, mostly since the slowness bugs me too.
> I did a:
> export PS4='+[${EPOCHREALTIME}][${BASH_SOURCE}:${LINENO}]: ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'; set -x;
> as time measurement and trace in the shellscript in question (mostly scripts/lib/ptxd_make_xpkg_pkg.sh).
> This pretty linear progressive cpu consumption, albeit the big chunks were due to calling of external binaries.
> 
> Then I did a timezone package clean and targetinstall.
> When targetinstalling a zone there was calls to several binaries.
> 
> For one zone I had external calls to (in order):
> echo, mkdir, printf(?), flock, ls, rm, mkdir, flock, mkdir, flock, mkdir, flock, mkdir, flock, mkdir, flock, mkdir, mkdir, mkdir, mkdir, mkdir, install, install, chmod, chmod, chown, echo.
> 
> Some of these could be internal builtins, but the consumed time suggested otherwise.
> Either way. Each install took about 26 ms and I could account the majority of that time in forking external programs and waiting for them to return.
> 
> So my conclusion is: The whole thing is a bit slow and bash doesn't help.
> 
> Regards,
> Christian
> 
> > > 
> > > Thanks for any feedback.
> > > BR,
> > > Christian
> > 
>