Re: [ptxdist] Loss of speedup for subsequent different -but still similar- images build

[Michael Olbrich <m.olbrich@pengutronix.de>]

Hi,

On Thu, Sep 23, 2021 at 01:02:20PM +0000, Henzer, Kevin wrote:
> Thanks for adding me to the mailing list, and thank for this nice tool that PTXdist is !
> 
> I'm new to PTXdist, and I have inherited a build setup that creates two different but quite similar system images, following two different global configurations (config_1 and config_2). 
> One is the rescue update image of the system, the other the full-featured image. It is using a script to build them both in a row. 
> 
> Before a recent update, calling the build process successively on those two configurations gave a big speed-up for the second build. The system was reusing most of the packages.
> 
> ./ptxdist go --ptxconfig=configs/config_1 --platformconfig=configs/platformconfig 	# takes time only the first time, else quite fast
> ./ptxdist images --ptxconfig=configs/ config_1  --platformconfig=configs/platformconfig  # quite fast
> 
> ./ptxdist go --ptxconfig=configs/config_2 --platformconfig=configs/platformconfig 	# quite fast
> ./ptxdist images --ptxconfig=configs/ config_2  --platformconfig=configs/platformconfig  # quite fast
> 
> 
> 
> But after updating PTXdist to newer version (now using PTXdist version 2021.03.0), and updating the config files, it seems something has changed, and the outcome is different. 
> 
> ./ptxdist go --ptxconfig=configs/config_1 --platformconfig=configs/platformconfig 	# takes time at each build
> ./ptxdist images --ptxconfig=configs/ config_1  --platformconfig=configs/platformconfig  # quite fast
> 
> ./ptxdist go --ptxconfig=configs/config_2 --platformconfig=configs/platformconfig 	# takes time at each build, seems the system is rebuilding everything
> 
> 
> And I got this as first output when switching from one configuration to the other:
> 
> -----------------------------------------------------------------
> target: host-system-perl.[some hash].cfghash
> -----------------------------------------------------------------
> 
> Configuration changed! Reconfiguring...

This is most likely the cause. Just about everything in the BSP depends
(indirectly) on this.

> I checked this package and seems to me it is just ensuring that Perl is installed on the host and that the right features are available.
> I checked all config options that I think are linked to this package (I searched in rules/host-system-perl.make) : 
> 
> PTXCONF_HOST_SYSTEM_PERL_BUILD
> PTXCONF_HOST_SYSTEM_PERL_JSON
> PTXCONF_HOST_SYSTEM_PERL_LOCALE_PO
> PTXCONF_HOST_SYSTEM_PERL_XMLPARSER
> PTXCONF_HOST_SYSTEM_PERL_YAPP
> 
> And they are set the same in both config files.
> 
> 
> Still, the hash of this package is changing between each call with different config. 
> 
> So, I was wondering:
> - Can it be that the reconfiguration and "rebuild" of this package is
> what triggers the full image rebuild?

Yes, definitely.

> - If yes, what could trigger this host-system-perl reconfiguration, in your opinion ? 

I'm not sure. It look like it shouldn't change, so lets dig deeper.

> And most important for me, how are those .cfghash computed ? If you have a quick link to the relevant part of the doc, I'll be very happy !

So, there is no documentation and it changed a bit over time. PTXdist
creates a file for this and calculates the md5. The file contains:
 - the content of host-system-perl.make and host-system-perl.in
 - all PTXCONF_HOST_SYSTEM_PERL* options

While PTXdist is running, this file can be found at
/tmp/ptxdist.XXXXXX/pkghash-HOST_SYSTEM_PERL (XXXXXX is some random string
here). Compare the two versions to see what's different.

Michael


-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de