mailarchive of the ptxdist mailing list
 help / color / mirror / Atom feed
* [ptxdist] [RFC 1/2] ptxdist: print a note when calling ptxdist inside 'ptxdist bash'
@ 2021-01-14 22:51 Roland Hieber
  2021-01-14 22:51 ` [ptxdist] [RFC 2/2] ptxd_lib_dgen: recompile when certain global kconfig options change Roland Hieber
                   ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Roland Hieber @ 2021-01-14 22:51 UTC (permalink / raw)
  To: ptxdist; +Cc: Roland Hieber

For debugging purposes it might be useful to run ptxdist inside a
'ptxdist bash' session, but under normal conditions this will not work
out of the box. Mention the fact to prevent confusion and useless
debugging sessions:

    $ ptxdist
    FATAL didn't find /scripts/ptxdist_vars.sh
    Note: PTXdist was called inside 'ptxdist bash' – things may be broken.

Signed-off-by: Roland Hieber <rhi@pengutronix.de>
---
 bin/ptxdist | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/bin/ptxdist b/bin/ptxdist
index 0a252e4ab22e..da3adb134962 100755
--- a/bin/ptxdist
+++ b/bin/ptxdist
@@ -1695,6 +1695,7 @@ parse_second()
 . ~/.bashrc
 PS1="[ptx] \${PS1}"
 PATH="${PATH}"
+export INSIDE_PTXDIST_BASH=1
 alias ptxsudo='sudo env PATH=$PATH'
 EOF
 				"${BASH}" --init-file "${bashrc}"
@@ -2111,7 +2112,7 @@ setup_env() {
 	unset $({
 		export -p  | sed -n 's/^declare -x \([^=]*\).*$/\1/p'
 		export -fp | sed -n 's/^declare -fx \([^=]*\).*$/\1/p'
-		} | egrep -v "^(PTXDIST_PTXRC|PTX_AUTOBUILD_DESTDIR|CCACHE_.*|PWD|HOME|USER|PATH|TERM|COLUMNS|LINES|DISPLAY|TMPDIR|KCONFIG_ALLCONFIG|KCONFIG_SEED|http_proxy|https_proxy|ftp_proxy|no_proxy${whitelist})$")
+		} | egrep -v "^(INSIDE_PTXDIST_BASH|PTXDIST_PTXRC|PTX_AUTOBUILD_DESTDIR|CCACHE_.*|PWD|HOME|USER|PATH|TERM|COLUMNS|LINES|DISPLAY|TMPDIR|KCONFIG_ALLCONFIG|KCONFIG_SEED|http_proxy|https_proxy|ftp_proxy|no_proxy${whitelist})$")
 
 	######## the environment is clean now ########
 
@@ -2134,6 +2135,10 @@ ptxdist_trap_exit_handler() {
 	local retval="${?}"
 	local file
 
+	if [ -n "${INSIDE_PTXDIST_BASH}" ]; then
+		echo "Note: PTXdist was called inside 'ptxdist bash' – things may be broken." >/dev/tty
+	fi
+
 	if [ -n "${PTXDIST_TEMPDIR}" -a -d "${PTXDIST_TEMPDIR}" ]; then
 		rm -fr -- "${PTXDIST_TEMPDIR}"
 	fi
-- 
2.30.0


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [ptxdist] [RFC 2/2] ptxd_lib_dgen: recompile when certain global kconfig options change
  2021-01-14 22:51 [ptxdist] [RFC 1/2] ptxdist: print a note when calling ptxdist inside 'ptxdist bash' Roland Hieber
@ 2021-01-14 22:51 ` Roland Hieber
  2021-01-15  7:02   ` Michael Olbrich
  2021-01-15  6:35 ` [ptxdist] [RFC 1/2] ptxdist: print a note when calling ptxdist inside 'ptxdist bash' Michael Olbrich
  2021-06-06 13:53 ` [ptxdist] [PATCH v1] " Roland Hieber
  2 siblings, 1 reply; 8+ messages in thread
From: Roland Hieber @ 2021-01-14 22:51 UTC (permalink / raw)
  To: ptxdist; +Cc: Roland Hieber

There are certain variables in the menu which influence the build
environment for all packages:

* enabling dev packages
* enabling debug packages
* changing the reproducible date (only influences target packages)
* (probably more, but this is all which I could come up with right now)

Add those settings to the respective package hashes, so the hash changes
when those variables are changed, and the respective packages are
rebuilt.

Signed-off-by: Roland Hieber <rhi@pengutronix.de>
---
A similar case exists for GLOBAL_IPV6_OPTION, but it only affects the
packages that are using it, and not the whole userspace.
---
 scripts/lib/ptxd_lib_dgen.awk     | 22 ++++++++++++++++++++++
 scripts/lib/ptxd_make_pkghash.awk | 13 +++++++++++++
 2 files changed, 35 insertions(+)

diff --git a/scripts/lib/ptxd_lib_dgen.awk b/scripts/lib/ptxd_lib_dgen.awk
index 17748111b5ea..2fe358a6825e 100644
--- a/scripts/lib/ptxd_lib_dgen.awk
+++ b/scripts/lib/ptxd_lib_dgen.awk
@@ -202,6 +202,28 @@ $1 ~ /^PTX_MAP_._SOURCE/ {
 
 #
 # parse the ptx- and platformconfig
+#
+$1 ~ /^PTXCONF_PROJECT_CREATE_DEVPKGS/ {
+	if (PTXDIST_OLD_MAKE)
+		print "VAR: " $0 >> PTXDIST_HASHLIST;
+	else {
+		print "ifdef PTXDIST_SETUP_ONCE"					> DGEN_DEPS_POST;
+		print "$(file >>" PTXDIST_HASHLIST ",VAR: " $1 "=$(" $1 "))"		> DGEN_DEPS_POST;
+		print "endif"								> DGEN_DEPS_POST;
+	}
+}
+
+$1 ~ /^PTXCONF_(DEBUG_PACKAGES|REPRODUCIBLE_TIMESTAMP)/ {
+	if (PTXDIST_OLD_MAKE)
+		print "TARGETVAR: " $0 >> PTXDIST_HASHLIST;
+	else {
+		print "ifdef PTXDIST_SETUP_ONCE"					> DGEN_DEPS_POST;
+		print "$(file >>" PTXDIST_HASHLIST ",TARGETVAR: " $1 "=$(" $1 "))"	> DGEN_DEPS_POST;
+		print "endif"								> DGEN_DEPS_POST;
+	}
+}
+
+#
 # record yes and module packages
 #
 $1 ~ /^PTXCONF_/ {
diff --git a/scripts/lib/ptxd_make_pkghash.awk b/scripts/lib/ptxd_make_pkghash.awk
index 2ecae47b48f5..461bb152318c 100755
--- a/scripts/lib/ptxd_make_pkghash.awk
+++ b/scripts/lib/ptxd_make_pkghash.awk
@@ -9,6 +9,8 @@
 BEGIN {
 	PTXDIST_TEMPDIR		= ENVIRON["PTXDIST_TEMPDIR"];
 	dirs = ""
+	vars = ""
+	targetvars = ""
 }
 
 $1 == "PATCHES:" {
@@ -30,6 +32,14 @@ $1 == "RULES:" {
 	rules[pkg] = rules[pkg] " " rule
 }
 
+$1 == "VAR:" {
+	vars = $2 " " vars
+}
+
+$1 == "TARGETVAR:" {
+	targetvars = $2 " " targetvars
+}
+
 function dump_file(src, dst, tmp) {
 	if (!src)
 		return
@@ -46,6 +56,9 @@ function dump_file(src, dst, tmp) {
 END {
 	for (pkg in rules) {
 		f1 = PTXDIST_TEMPDIR "/pkghash-" pkg
+		printf vars "\n" >> f1
+		if ( pkg !~ /^HOST_/ )
+			printf targetvars "\n" >> f1
 		n = split(rules[pkg], cfgs)
 		for (rule = 1; rule <= n; rule++) {
 			dump_file(cfgs[rule], f1)
-- 
2.30.0


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ptxdist] [RFC 1/2] ptxdist: print a note when calling ptxdist inside 'ptxdist bash'
  2021-01-14 22:51 [ptxdist] [RFC 1/2] ptxdist: print a note when calling ptxdist inside 'ptxdist bash' Roland Hieber
  2021-01-14 22:51 ` [ptxdist] [RFC 2/2] ptxd_lib_dgen: recompile when certain global kconfig options change Roland Hieber
@ 2021-01-15  6:35 ` Michael Olbrich
  2021-01-15  6:43   ` Michael Olbrich
  2021-06-06 13:53 ` [ptxdist] [PATCH v1] " Roland Hieber
  2 siblings, 1 reply; 8+ messages in thread
From: Michael Olbrich @ 2021-01-15  6:35 UTC (permalink / raw)
  To: ptxdist

On Thu, Jan 14, 2021 at 11:51:48PM +0100, Roland Hieber wrote:
> For debugging purposes it might be useful to run ptxdist inside a
> 'ptxdist bash' session, but under normal conditions this will not work
> out of the box. Mention the fact to prevent confusion and useless
> debugging sessions:
> 
>     $ ptxdist
>     FATAL didn't find /scripts/ptxdist_vars.sh
>     Note: PTXdist was called inside 'ptxdist bash' – things may be broken.

I think we should just fail. So 'ptxd_bailout' instead of echo. Calling
ptxdist here is just not supported.

Michael

> 
> Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> ---
>  bin/ptxdist | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/bin/ptxdist b/bin/ptxdist
> index 0a252e4ab22e..da3adb134962 100755
> --- a/bin/ptxdist
> +++ b/bin/ptxdist
> @@ -1695,6 +1695,7 @@ parse_second()
>  . ~/.bashrc
>  PS1="[ptx] \${PS1}"
>  PATH="${PATH}"
> +export INSIDE_PTXDIST_BASH=1
>  alias ptxsudo='sudo env PATH=$PATH'
>  EOF
>  				"${BASH}" --init-file "${bashrc}"
> @@ -2111,7 +2112,7 @@ setup_env() {
>  	unset $({
>  		export -p  | sed -n 's/^declare -x \([^=]*\).*$/\1/p'
>  		export -fp | sed -n 's/^declare -fx \([^=]*\).*$/\1/p'
> -		} | egrep -v "^(PTXDIST_PTXRC|PTX_AUTOBUILD_DESTDIR|CCACHE_.*|PWD|HOME|USER|PATH|TERM|COLUMNS|LINES|DISPLAY|TMPDIR|KCONFIG_ALLCONFIG|KCONFIG_SEED|http_proxy|https_proxy|ftp_proxy|no_proxy${whitelist})$")
> +		} | egrep -v "^(INSIDE_PTXDIST_BASH|PTXDIST_PTXRC|PTX_AUTOBUILD_DESTDIR|CCACHE_.*|PWD|HOME|USER|PATH|TERM|COLUMNS|LINES|DISPLAY|TMPDIR|KCONFIG_ALLCONFIG|KCONFIG_SEED|http_proxy|https_proxy|ftp_proxy|no_proxy${whitelist})$")
>  
>  	######## the environment is clean now ########
>  
> @@ -2134,6 +2135,10 @@ ptxdist_trap_exit_handler() {
>  	local retval="${?}"
>  	local file
>  
> +	if [ -n "${INSIDE_PTXDIST_BASH}" ]; then
> +		echo "Note: PTXdist was called inside 'ptxdist bash' – things may be broken." >/dev/tty
> +	fi
> +
>  	if [ -n "${PTXDIST_TEMPDIR}" -a -d "${PTXDIST_TEMPDIR}" ]; then
>  		rm -fr -- "${PTXDIST_TEMPDIR}"
>  	fi
> -- 
> 2.30.0
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ptxdist] [RFC 1/2] ptxdist: print a note when calling ptxdist inside 'ptxdist bash'
  2021-01-15  6:35 ` [ptxdist] [RFC 1/2] ptxdist: print a note when calling ptxdist inside 'ptxdist bash' Michael Olbrich
@ 2021-01-15  6:43   ` Michael Olbrich
  0 siblings, 0 replies; 8+ messages in thread
From: Michael Olbrich @ 2021-01-15  6:43 UTC (permalink / raw)
  To: ptxdist

On Fri, Jan 15, 2021 at 07:35:22AM +0100, Michael Olbrich wrote:
> On Thu, Jan 14, 2021 at 11:51:48PM +0100, Roland Hieber wrote:
> > For debugging purposes it might be useful to run ptxdist inside a
> > 'ptxdist bash' session, but under normal conditions this will not work
> > out of the box. Mention the fact to prevent confusion and useless
> > debugging sessions:
> > 
> >     $ ptxdist
> >     FATAL didn't find /scripts/ptxdist_vars.sh
> >     Note: PTXdist was called inside 'ptxdist bash' – things may be broken.
> 
> I think we should just fail. So 'ptxd_bailout' instead of echo. Calling
> ptxdist here is just not supported.
> 
> Michael
> 
> > 
> > Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> > ---
> >  bin/ptxdist | 7 ++++++-
> >  1 file changed, 6 insertions(+), 1 deletion(-)
> > 
> > diff --git a/bin/ptxdist b/bin/ptxdist
> > index 0a252e4ab22e..da3adb134962 100755
> > --- a/bin/ptxdist
> > +++ b/bin/ptxdist
> > @@ -1695,6 +1695,7 @@ parse_second()
> >  . ~/.bashrc
> >  PS1="[ptx] \${PS1}"
> >  PATH="${PATH}"
> > +export INSIDE_PTXDIST_BASH=1
> >  alias ptxsudo='sudo env PATH=$PATH'
> >  EOF
> >  				"${BASH}" --init-file "${bashrc}"
> > @@ -2111,7 +2112,7 @@ setup_env() {
> >  	unset $({
> >  		export -p  | sed -n 's/^declare -x \([^=]*\).*$/\1/p'
> >  		export -fp | sed -n 's/^declare -fx \([^=]*\).*$/\1/p'
> > -		} | egrep -v "^(PTXDIST_PTXRC|PTX_AUTOBUILD_DESTDIR|CCACHE_.*|PWD|HOME|USER|PATH|TERM|COLUMNS|LINES|DISPLAY|TMPDIR|KCONFIG_ALLCONFIG|KCONFIG_SEED|http_proxy|https_proxy|ftp_proxy|no_proxy${whitelist})$")
> > +		} | egrep -v "^(INSIDE_PTXDIST_BASH|PTXDIST_PTXRC|PTX_AUTOBUILD_DESTDIR|CCACHE_.*|PWD|HOME|USER|PATH|TERM|COLUMNS|LINES|DISPLAY|TMPDIR|KCONFIG_ALLCONFIG|KCONFIG_SEED|http_proxy|https_proxy|ftp_proxy|no_proxy${whitelist})$")
> >  
> >  	######## the environment is clean now ########
> >  
> > @@ -2134,6 +2135,10 @@ ptxdist_trap_exit_handler() {
> >  	local retval="${?}"
> >  	local file
> >  
> > +	if [ -n "${INSIDE_PTXDIST_BASH}" ]; then

And I think we can check for an existing variable. Maybe PTXDIST_TOPDIR?

Michael

> > +		echo "Note: PTXdist was called inside 'ptxdist bash' – things may be broken." >/dev/tty
> > +	fi
> > +
> >  	if [ -n "${PTXDIST_TEMPDIR}" -a -d "${PTXDIST_TEMPDIR}" ]; then
> >  		rm -fr -- "${PTXDIST_TEMPDIR}"
> >  	fi
> > -- 
> > 2.30.0
> > 
> > 
> > _______________________________________________
> > ptxdist mailing list
> > ptxdist@pengutronix.de
> > To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> 
> -- 
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ptxdist] [RFC 2/2] ptxd_lib_dgen: recompile when certain global kconfig options change
  2021-01-14 22:51 ` [ptxdist] [RFC 2/2] ptxd_lib_dgen: recompile when certain global kconfig options change Roland Hieber
@ 2021-01-15  7:02   ` Michael Olbrich
  2021-01-15 10:09     ` Roland Hieber
  0 siblings, 1 reply; 8+ messages in thread
From: Michael Olbrich @ 2021-01-15  7:02 UTC (permalink / raw)
  To: ptxdist; +Cc: Roland Hieber

On Thu, Jan 14, 2021 at 11:51:49PM +0100, Roland Hieber wrote:
> There are certain variables in the menu which influence the build
> environment for all packages:
> 
> * enabling dev packages

I wouldn't do this here. It has no effect on the resulting target files,
and I can always to a clean build if I want dev packages for everything.

> * enabling debug packages

Only the targetinstall stage needs to be rebuilt for this.

> * changing the reproducible date (only influences target packages)
> * (probably more, but this is all which I could come up with right now)

Any GLOBAL_* option. But those don't affect all packages. And so far I
don't have a good idea to figure out the affected ones.

> Add those settings to the respective package hashes, so the hash changes
> when those variables are changed, and the respective packages are
> rebuilt.
> 
> Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> ---
> A similar case exists for GLOBAL_IPV6_OPTION, but it only affects the
> packages that are using it, and not the whole userspace.
> ---
>  scripts/lib/ptxd_lib_dgen.awk     | 22 ++++++++++++++++++++++
>  scripts/lib/ptxd_make_pkghash.awk | 13 +++++++++++++
>  2 files changed, 35 insertions(+)
> 
> diff --git a/scripts/lib/ptxd_lib_dgen.awk b/scripts/lib/ptxd_lib_dgen.awk
> index 17748111b5ea..2fe358a6825e 100644
> --- a/scripts/lib/ptxd_lib_dgen.awk
> +++ b/scripts/lib/ptxd_lib_dgen.awk
> @@ -202,6 +202,28 @@ $1 ~ /^PTX_MAP_._SOURCE/ {
>  
>  #
>  # parse the ptx- and platformconfig
> +#
> +$1 ~ /^PTXCONF_PROJECT_CREATE_DEVPKGS/ {
> +	if (PTXDIST_OLD_MAKE)
> +		print "VAR: " $0 >> PTXDIST_HASHLIST;
> +	else {
> +		print "ifdef PTXDIST_SETUP_ONCE"					> DGEN_DEPS_POST;
> +		print "$(file >>" PTXDIST_HASHLIST ",VAR: " $1 "=$(" $1 "))"		> DGEN_DEPS_POST;
> +		print "endif"								> DGEN_DEPS_POST;
> +	}
> +}
> +
> +$1 ~ /^PTXCONF_(DEBUG_PACKAGES|REPRODUCIBLE_TIMESTAMP)/ {
> +	if (PTXDIST_OLD_MAKE)
> +		print "TARGETVAR: " $0 >> PTXDIST_HASHLIST;
> +	else {
> +		print "ifdef PTXDIST_SETUP_ONCE"					> DGEN_DEPS_POST;
> +		print "$(file >>" PTXDIST_HASHLIST ",TARGETVAR: " $1 "=$(" $1 "))"	> DGEN_DEPS_POST;
> +		print "endif"								> DGEN_DEPS_POST;
> +	}
> +}

This can all be 'print "...." > PTXDIST_HASHLIST;'. The '$(file ....)'
stuff is only needed for late evaluation. e.g. if the value of the kconfig
variable is a string that references make variables. That's not the case
for any of those.

> +
> +#
>  # record yes and module packages
>  #
>  $1 ~ /^PTXCONF_/ {
> diff --git a/scripts/lib/ptxd_make_pkghash.awk b/scripts/lib/ptxd_make_pkghash.awk
> index 2ecae47b48f5..461bb152318c 100755
> --- a/scripts/lib/ptxd_make_pkghash.awk
> +++ b/scripts/lib/ptxd_make_pkghash.awk
> @@ -9,6 +9,8 @@
>  BEGIN {
>  	PTXDIST_TEMPDIR		= ENVIRON["PTXDIST_TEMPDIR"];
>  	dirs = ""
> +	vars = ""
> +	targetvars = ""
>  }
>  
>  $1 == "PATCHES:" {
> @@ -30,6 +32,14 @@ $1 == "RULES:" {
>  	rules[pkg] = rules[pkg] " " rule
>  }
>  
> +$1 == "VAR:" {
> +	vars = $2 " " vars
> +}
> +
> +$1 == "TARGETVAR:" {
> +	targetvars = $2 " " targetvars
> +}
> +
>  function dump_file(src, dst, tmp) {
>  	if (!src)
>  		return
> @@ -46,6 +56,9 @@ function dump_file(src, dst, tmp) {
>  END {
>  	for (pkg in rules) {
>  		f1 = PTXDIST_TEMPDIR "/pkghash-" pkg
> +		printf vars "\n" >> f1
> +		if ( pkg !~ /^HOST_/ )

What about CROSS_*?

Michael

> +			printf targetvars "\n" >> f1
>  		n = split(rules[pkg], cfgs)
>  		for (rule = 1; rule <= n; rule++) {
>  			dump_file(cfgs[rule], f1)
> -- 
> 2.30.0
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ptxdist] [RFC 2/2] ptxd_lib_dgen: recompile when certain global kconfig options change
  2021-01-15  7:02   ` Michael Olbrich
@ 2021-01-15 10:09     ` Roland Hieber
  0 siblings, 0 replies; 8+ messages in thread
From: Roland Hieber @ 2021-01-15 10:09 UTC (permalink / raw)
  To: ptxdist

On Fri, Jan 15, 2021 at 08:02:57AM +0100, Michael Olbrich wrote:
> On Thu, Jan 14, 2021 at 11:51:49PM +0100, Roland Hieber wrote:
> > There are certain variables in the menu which influence the build
> > environment for all packages:
> > 
> > * enabling dev packages
> 
> I wouldn't do this here. It has no effect on the resulting target files,
> and I can always to a clean build if I want dev packages for everything.
> 
> > * enabling debug packages
> 
> Only the targetinstall stage needs to be rebuilt for this.
> 
> > * changing the reproducible date (only influences target packages)
> > * (probably more, but this is all which I could come up with right now)
> 
> Any GLOBAL_* option. But those don't affect all packages. And so far I
> don't have a good idea to figure out the affected ones.

Yeah, ideally we would calculate a hash of the script that is executed
in each make stage with all variables expanded, and then only rebuild
the stages whose script checksum has changed, but that's hard to do in
pure make…

 - Roland

> 
> > Add those settings to the respective package hashes, so the hash changes
> > when those variables are changed, and the respective packages are
> > rebuilt.
> > 
> > Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> > ---
> > A similar case exists for GLOBAL_IPV6_OPTION, but it only affects the
> > packages that are using it, and not the whole userspace.
> > ---
> >  scripts/lib/ptxd_lib_dgen.awk     | 22 ++++++++++++++++++++++
> >  scripts/lib/ptxd_make_pkghash.awk | 13 +++++++++++++
> >  2 files changed, 35 insertions(+)
> > 
> > diff --git a/scripts/lib/ptxd_lib_dgen.awk b/scripts/lib/ptxd_lib_dgen.awk
> > index 17748111b5ea..2fe358a6825e 100644
> > --- a/scripts/lib/ptxd_lib_dgen.awk
> > +++ b/scripts/lib/ptxd_lib_dgen.awk
> > @@ -202,6 +202,28 @@ $1 ~ /^PTX_MAP_._SOURCE/ {
> >  
> >  #
> >  # parse the ptx- and platformconfig
> > +#
> > +$1 ~ /^PTXCONF_PROJECT_CREATE_DEVPKGS/ {
> > +	if (PTXDIST_OLD_MAKE)
> > +		print "VAR: " $0 >> PTXDIST_HASHLIST;
> > +	else {
> > +		print "ifdef PTXDIST_SETUP_ONCE"					> DGEN_DEPS_POST;
> > +		print "$(file >>" PTXDIST_HASHLIST ",VAR: " $1 "=$(" $1 "))"		> DGEN_DEPS_POST;
> > +		print "endif"								> DGEN_DEPS_POST;
> > +	}
> > +}
> > +
> > +$1 ~ /^PTXCONF_(DEBUG_PACKAGES|REPRODUCIBLE_TIMESTAMP)/ {
> > +	if (PTXDIST_OLD_MAKE)
> > +		print "TARGETVAR: " $0 >> PTXDIST_HASHLIST;
> > +	else {
> > +		print "ifdef PTXDIST_SETUP_ONCE"					> DGEN_DEPS_POST;
> > +		print "$(file >>" PTXDIST_HASHLIST ",TARGETVAR: " $1 "=$(" $1 "))"	> DGEN_DEPS_POST;
> > +		print "endif"								> DGEN_DEPS_POST;
> > +	}
> > +}
> 
> This can all be 'print "...." > PTXDIST_HASHLIST;'. The '$(file ....)'
> stuff is only needed for late evaluation. e.g. if the value of the kconfig
> variable is a string that references make variables. That's not the case
> for any of those.
> 
> > +
> > +#
> >  # record yes and module packages
> >  #
> >  $1 ~ /^PTXCONF_/ {
> > diff --git a/scripts/lib/ptxd_make_pkghash.awk b/scripts/lib/ptxd_make_pkghash.awk
> > index 2ecae47b48f5..461bb152318c 100755
> > --- a/scripts/lib/ptxd_make_pkghash.awk
> > +++ b/scripts/lib/ptxd_make_pkghash.awk
> > @@ -9,6 +9,8 @@
> >  BEGIN {
> >  	PTXDIST_TEMPDIR		= ENVIRON["PTXDIST_TEMPDIR"];
> >  	dirs = ""
> > +	vars = ""
> > +	targetvars = ""
> >  }
> >  
> >  $1 == "PATCHES:" {
> > @@ -30,6 +32,14 @@ $1 == "RULES:" {
> >  	rules[pkg] = rules[pkg] " " rule
> >  }
> >  
> > +$1 == "VAR:" {
> > +	vars = $2 " " vars
> > +}
> > +
> > +$1 == "TARGETVAR:" {
> > +	targetvars = $2 " " targetvars
> > +}
> > +
> >  function dump_file(src, dst, tmp) {
> >  	if (!src)
> >  		return
> > @@ -46,6 +56,9 @@ function dump_file(src, dst, tmp) {
> >  END {
> >  	for (pkg in rules) {
> >  		f1 = PTXDIST_TEMPDIR "/pkghash-" pkg
> > +		printf vars "\n" >> f1
> > +		if ( pkg !~ /^HOST_/ )
> 
> What about CROSS_*?
> 
> Michael
> 
> > +			printf targetvars "\n" >> f1
> >  		n = split(rules[pkg], cfgs)
> >  		for (rule = 1; rule <= n; rule++) {
> >  			dump_file(cfgs[rule], f1)
> > -- 
> > 2.30.0
> > 
> > 
> > _______________________________________________
> > ptxdist mailing list
> > ptxdist@pengutronix.de
> > To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> > 
> 
> -- 
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> 

-- 
Roland Hieber, Pengutronix e.K.          | r.hieber@pengutronix.de     |
Steuerwalder Str. 21                     | https://www.pengutronix.de/ |
31137 Hildesheim, Germany                | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686         | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [ptxdist] [PATCH v1] ptxdist: print a note when calling ptxdist inside 'ptxdist bash'
  2021-01-14 22:51 [ptxdist] [RFC 1/2] ptxdist: print a note when calling ptxdist inside 'ptxdist bash' Roland Hieber
  2021-01-14 22:51 ` [ptxdist] [RFC 2/2] ptxd_lib_dgen: recompile when certain global kconfig options change Roland Hieber
  2021-01-15  6:35 ` [ptxdist] [RFC 1/2] ptxdist: print a note when calling ptxdist inside 'ptxdist bash' Michael Olbrich
@ 2021-06-06 13:53 ` Roland Hieber
  2021-06-16 10:35   ` [ptxdist] [APPLIED] " Michael Olbrich
  2 siblings, 1 reply; 8+ messages in thread
From: Roland Hieber @ 2021-06-06 13:53 UTC (permalink / raw)
  To: ptxdist; +Cc: Roland Hieber

For debugging purposes it might be useful to run ptxdist inside a
'ptxdist bash' session, but under normal conditions this will not work
out of the box. Mention the fact to prevent confusion and useless
debugging sessions:

    $ ptxdist
    FATAL didn't find /scripts/ptxdist_vars.sh
    Note: PTXdist was called inside 'ptxdist bash' – things may be broken.

Signed-off-by: Roland Hieber <rhi@pengutronix.de>
---
RFC -> PATCH v1:
 - bailout instead of printing a notice (Michael Olbrich)
 - check for PTXDIST_TOPDIR directly in main because it will already be
   defined in ptxdist_trap_exit_handler (and also I don't know anymore
   what significance ptxdist_trap_exit_handler holds that I used it
   previously…)

RFC was at https://lore.ptxdist.org/ptxdist/20210114225149.3181-1-rhi@pengutronix.de
---
 bin/ptxdist | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/bin/ptxdist b/bin/ptxdist
index 13f81674f4fd..c41065311f0c 100755
--- a/bin/ptxdist
+++ b/bin/ptxdist
@@ -2641,6 +2641,10 @@ setup_export() {
 ########################################################################
 
 main() {
+	if [ -n "${PTXDIST_TOPDIR}" ]; then
+		ptxd_bailout "Calling PTXdist inside 'ptxdist bash' is not supported."
+	fi
+
 	check_uid &&
 	check_path &&
 
-- 
2.29.2


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [ptxdist] [APPLIED] ptxdist: print a note when calling ptxdist inside 'ptxdist bash'
  2021-06-06 13:53 ` [ptxdist] [PATCH v1] " Roland Hieber
@ 2021-06-16 10:35   ` Michael Olbrich
  0 siblings, 0 replies; 8+ messages in thread
From: Michael Olbrich @ 2021-06-16 10:35 UTC (permalink / raw)
  To: ptxdist; +Cc: Roland Hieber

Thanks, applied as d93df30ca039207a49664c1d83d83a8efe06a3f9.

Michael

[sent from post-receive hook]

On Wed, 16 Jun 2021 12:35:28 +0200, Roland Hieber <rhi@pengutronix.de> wrote:
> For debugging purposes it might be useful to run ptxdist inside a
> 'ptxdist bash' session, but under normal conditions this will not work
> out of the box. Mention the fact to prevent confusion and useless
> debugging sessions:
> 
>     $ ptxdist
>     FATAL didn't find /scripts/ptxdist_vars.sh
>     Note: PTXdist was called inside 'ptxdist bash' – things may be broken.
> 
> Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> Message-Id: <20210606135350.22150-1-rhi@pengutronix.de>
> Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
> 
> diff --git a/bin/ptxdist b/bin/ptxdist
> index 13f81674f4fd..c41065311f0c 100755
> --- a/bin/ptxdist
> +++ b/bin/ptxdist
> @@ -2641,6 +2641,10 @@ setup_export() {
>  ########################################################################
>  
>  main() {
> +	if [ -n "${PTXDIST_TOPDIR}" ]; then
> +		ptxd_bailout "Calling PTXdist inside 'ptxdist bash' is not supported."
> +	fi
> +
>  	check_uid &&
>  	check_path &&
>  

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-06-16 10:36 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-14 22:51 [ptxdist] [RFC 1/2] ptxdist: print a note when calling ptxdist inside 'ptxdist bash' Roland Hieber
2021-01-14 22:51 ` [ptxdist] [RFC 2/2] ptxd_lib_dgen: recompile when certain global kconfig options change Roland Hieber
2021-01-15  7:02   ` Michael Olbrich
2021-01-15 10:09     ` Roland Hieber
2021-01-15  6:35 ` [ptxdist] [RFC 1/2] ptxdist: print a note when calling ptxdist inside 'ptxdist bash' Michael Olbrich
2021-01-15  6:43   ` Michael Olbrich
2021-06-06 13:53 ` [ptxdist] [PATCH v1] " Roland Hieber
2021-06-16 10:35   ` [ptxdist] [APPLIED] " Michael Olbrich

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox