Re: [ptxdist] [RFC 2/2] ptxd_lib_dgen: recompile when certain global kconfig options change

[Roland Hieber <rhi@pengutronix.de>]

On Fri, Jan 15, 2021 at 08:02:57AM +0100, Michael Olbrich wrote:
> On Thu, Jan 14, 2021 at 11:51:49PM +0100, Roland Hieber wrote:
> > There are certain variables in the menu which influence the build
> > environment for all packages:
> > 
> > * enabling dev packages
> 
> I wouldn't do this here. It has no effect on the resulting target files,
> and I can always to a clean build if I want dev packages for everything.
> 
> > * enabling debug packages
> 
> Only the targetinstall stage needs to be rebuilt for this.
> 
> > * changing the reproducible date (only influences target packages)
> > * (probably more, but this is all which I could come up with right now)
> 
> Any GLOBAL_* option. But those don't affect all packages. And so far I
> don't have a good idea to figure out the affected ones.

Yeah, ideally we would calculate a hash of the script that is executed
in each make stage with all variables expanded, and then only rebuild
the stages whose script checksum has changed, but that's hard to do in
pure make…

 - Roland

> 
> > Add those settings to the respective package hashes, so the hash changes
> > when those variables are changed, and the respective packages are
> > rebuilt.
> > 
> > Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> > ---
> > A similar case exists for GLOBAL_IPV6_OPTION, but it only affects the
> > packages that are using it, and not the whole userspace.
> > ---
> >  scripts/lib/ptxd_lib_dgen.awk     | 22 ++++++++++++++++++++++
> >  scripts/lib/ptxd_make_pkghash.awk | 13 +++++++++++++
> >  2 files changed, 35 insertions(+)
> > 
> > diff --git a/scripts/lib/ptxd_lib_dgen.awk b/scripts/lib/ptxd_lib_dgen.awk
> > index 17748111b5ea..2fe358a6825e 100644
> > --- a/scripts/lib/ptxd_lib_dgen.awk
> > +++ b/scripts/lib/ptxd_lib_dgen.awk
> > @@ -202,6 +202,28 @@ $1 ~ /^PTX_MAP_._SOURCE/ {
> >  
> >  #
> >  # parse the ptx- and platformconfig
> > +#
> > +$1 ~ /^PTXCONF_PROJECT_CREATE_DEVPKGS/ {
> > +	if (PTXDIST_OLD_MAKE)
> > +		print "VAR: " $0 >> PTXDIST_HASHLIST;
> > +	else {
> > +		print "ifdef PTXDIST_SETUP_ONCE"					> DGEN_DEPS_POST;
> > +		print "$(file >>" PTXDIST_HASHLIST ",VAR: " $1 "=$(" $1 "))"		> DGEN_DEPS_POST;
> > +		print "endif"								> DGEN_DEPS_POST;
> > +	}
> > +}
> > +
> > +$1 ~ /^PTXCONF_(DEBUG_PACKAGES|REPRODUCIBLE_TIMESTAMP)/ {
> > +	if (PTXDIST_OLD_MAKE)
> > +		print "TARGETVAR: " $0 >> PTXDIST_HASHLIST;
> > +	else {
> > +		print "ifdef PTXDIST_SETUP_ONCE"					> DGEN_DEPS_POST;
> > +		print "$(file >>" PTXDIST_HASHLIST ",TARGETVAR: " $1 "=$(" $1 "))"	> DGEN_DEPS_POST;
> > +		print "endif"								> DGEN_DEPS_POST;
> > +	}
> > +}
> 
> This can all be 'print "...." > PTXDIST_HASHLIST;'. The '$(file ....)'
> stuff is only needed for late evaluation. e.g. if the value of the kconfig
> variable is a string that references make variables. That's not the case
> for any of those.
> 
> > +
> > +#
> >  # record yes and module packages
> >  #
> >  $1 ~ /^PTXCONF_/ {
> > diff --git a/scripts/lib/ptxd_make_pkghash.awk b/scripts/lib/ptxd_make_pkghash.awk
> > index 2ecae47b48f5..461bb152318c 100755
> > --- a/scripts/lib/ptxd_make_pkghash.awk
> > +++ b/scripts/lib/ptxd_make_pkghash.awk
> > @@ -9,6 +9,8 @@
> >  BEGIN {
> >  	PTXDIST_TEMPDIR		= ENVIRON["PTXDIST_TEMPDIR"];
> >  	dirs = ""
> > +	vars = ""
> > +	targetvars = ""
> >  }
> >  
> >  $1 == "PATCHES:" {
> > @@ -30,6 +32,14 @@ $1 == "RULES:" {
> >  	rules[pkg] = rules[pkg] " " rule
> >  }
> >  
> > +$1 == "VAR:" {
> > +	vars = $2 " " vars
> > +}
> > +
> > +$1 == "TARGETVAR:" {
> > +	targetvars = $2 " " targetvars
> > +}
> > +
> >  function dump_file(src, dst, tmp) {
> >  	if (!src)
> >  		return
> > @@ -46,6 +56,9 @@ function dump_file(src, dst, tmp) {
> >  END {
> >  	for (pkg in rules) {
> >  		f1 = PTXDIST_TEMPDIR "/pkghash-" pkg
> > +		printf vars "\n" >> f1
> > +		if ( pkg !~ /^HOST_/ )
> 
> What about CROSS_*?
> 
> Michael
> 
> > +			printf targetvars "\n" >> f1
> >  		n = split(rules[pkg], cfgs)
> >  		for (rule = 1; rule <= n; rule++) {
> >  			dump_file(cfgs[rule], f1)
> > -- 
> > 2.30.0
> > 
> > 
> > _______________________________________________
> > ptxdist mailing list
> > ptxdist@pengutronix.de
> > To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> > 
> 
> -- 
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> 

-- 
Roland Hieber, Pengutronix e.K.          | r.hieber@pengutronix.de     |
Steuerwalder Str. 21                     | https://www.pengutronix.de/ |
31137 Hildesheim, Germany                | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686         | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de