[ptxdist] [PATCH 0/5] RAUC: update to v1.7 and add some new options

[ptxdist] [PATCH 0/5] RAUC: update to v1.7 and add some new options

[Enrico Jorns]

RAUC v1.7 comes with two major new features:

* The new 'crypt' bundle format for encrypted bundles
* Built-in HTTP(S) bundle streaming support based on NBD

This adds options for these in 'rauc' and 'image-rauc' recipe.

Also since v1.3, the option --disable-create allows to strip code for
bundle generation from target rauc where this is normally not required.
This adds an option that disable bundle generation on target by default.

Also default to 'verity' bundle format to guide ptxdist users that are
unsure which bundle format to use.

Enrico Jorns (5):
  RAUC: version bump 1.5.1 -> 1.7
  RAUC: add option to enable streaming support
  RAUC: leverage --disable-create option to shrink size for target RAUC
  image-rauc: add support for crypt bundles
  image-rauc: default to 'verity' bundle format

 platforms/image-rauc.in |  9 +++++++++
 rules/host-rauc.make    |  1 +
 rules/image-rauc.make   | 12 +++++++++++-
 rules/rauc.in           | 21 +++++++++++++++++++++
 rules/rauc.make         |  6 ++++--
 5 files changed, 46 insertions(+), 3 deletions(-)

-- 
2.30.2

[ptxdist] [PATCH 1/5] RAUC: version bump 1.5.1 -> 1.7

[Enrico Jorns]

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 rules/rauc.make | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/rules/rauc.make b/rules/rauc.make
index f74f6b62c..46d196bed 100644
--- a/rules/rauc.make
+++ b/rules/rauc.make
@@ -14,8 +14,8 @@ PACKAGES-$(PTXCONF_RAUC) += rauc
 #
 # Paths and names
 #
-RAUC_VERSION	:= 1.5.1
-RAUC_MD5	:= e1c1fad12d81b955784968c37130d272
+RAUC_VERSION	:= 1.7
+RAUC_MD5	:= aa31fc2a457ba2e3fca8206d9fb9f126
 RAUC		:= rauc-$(RAUC_VERSION)
 RAUC_SUFFIX	:= tar.xz
 RAUC_URL	:= https://github.com/rauc/rauc/releases/download/v$(RAUC_VERSION)/$(RAUC).$(RAUC_SUFFIX)
-- 
2.30.2

[ptxdist] [PATCH 2/5] RAUC: add option to enable streaming support

[Enrico Jorns]

Enable it by default so that default installation can benefit from this.
Can be deactivated for saving space or for deactivating network
functionality.
Web server must support HTTP range requests for this and NBD support
must be enabled in kernel.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 rules/rauc.in   | 11 +++++++++++
 rules/rauc.make |  1 +
 2 files changed, 12 insertions(+)

diff --git a/rules/rauc.in b/rules/rauc.in
index aee162b83..832967b44 100644
--- a/rules/rauc.in
+++ b/rules/rauc.in
@@ -13,6 +13,7 @@ menuconfig RAUC
 	select BUSYBOX_FEATURE_SEAMLESS_GZ	if BUSYBOX_TAR
 	select DBUS				if RAUC_SERVICE
 	select LIBCURL				if RAUC_NETWORK
+	select LIBNL3				if RAUC_STREAMING
 	select JSON_GLIB			if RAUC_JSON
 	select UTIL_LINUX_NG_LIBFDISK		if RAUC_GPT
 	select SQUASHFS_TOOLS			if RUNTIME
@@ -44,6 +45,16 @@ config RAUC_NETWORK
 	  Note that network is primarily designed to be used for RAUC'S casync
 	  capabilities, not for fetching full bundles.
 
+config RAUC_STREAMING
+	bool
+	prompt "Enable streaming support"
+	default y
+	select RAUC_NETWORK
+	help
+	  Enables built-in HTTP(S) streaming support that allows RAUC to
+	  access remote bundles via http/https and write images without the
+	  need for intermediate storage on the target.
+
 config RAUC_JSON
 	bool
 	prompt "JSON support"
diff --git a/rules/rauc.make b/rules/rauc.make
index 46d196bed..820583e59 100644
--- a/rules/rauc.make
+++ b/rules/rauc.make
@@ -43,6 +43,7 @@ RAUC_CONF_OPT	:= \
 	--disable-valgrind \
 	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
 	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \
+	--$(call ptx/endis,PTXCONF_RAUC_STREAMING)-streaming \
 	--$(call ptx/endis,PTXCONF_RAUC_JSON)-json \
 	--$(call ptx/endis,PTXCONF_RAUC_GPT)-gpt \
 	--with-systemdunitdir=/usr/lib/systemd/system \
-- 
2.30.2

[ptxdist] [PATCH 3/5] RAUC: leverage --disable-create option to shrink size for target RAUC

[Enrico Jorns]

We do not need bundle creation capabilities on the target in most cases,
thus make this optional (and disable by default).

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 rules/host-rauc.make |  1 +
 rules/rauc.in        | 10 ++++++++++
 rules/rauc.make      |  1 +
 3 files changed, 12 insertions(+)

diff --git a/rules/host-rauc.make b/rules/host-rauc.make
index c78a518f6..1851aee9a 100644
--- a/rules/host-rauc.make
+++ b/rules/host-rauc.make
@@ -23,6 +23,7 @@ HOST_RAUC_CONF_OPT	:= \
 	$(HOST_AUTOCONF) \
 	--disable-code-coverage \
 	--disable-valgrind \
+	--enable-create \
 	--disable-service \
 	--disable-network \
 	--disable-json
diff --git a/rules/rauc.in b/rules/rauc.in
index 832967b44..e43692846 100644
--- a/rules/rauc.in
+++ b/rules/rauc.in
@@ -27,6 +27,16 @@ menuconfig RAUC
 
 if RAUC
 
+config RAUC_CREATE
+	bool
+	prompt "Enable bundle creation support on target"
+	default n
+	help
+	  Enables support for creating bundles on target.
+	  Disabling this will remove functionality required for bundle
+	  creation which is not required on the target in most cases and thus
+	  allows to save space.
+
 config RAUC_SERVICE
 	bool
 	prompt "Enable service (D-Bus) support"
diff --git a/rules/rauc.make b/rules/rauc.make
index 820583e59..e001956a5 100644
--- a/rules/rauc.make
+++ b/rules/rauc.make
@@ -41,6 +41,7 @@ RAUC_CONF_OPT	:= \
 	$(GLOBAL_LARGE_FILE_OPTION) \
 	--disable-code-coverage \
 	--disable-valgrind \
+	--$(call ptx/endis,PTXCONF_RAUC_CREATE)-create \
 	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
 	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \
 	--$(call ptx/endis,PTXCONF_RAUC_STREAMING)-streaming \
-- 
2.30.2

[ptxdist] [PATCH 4/5] image-rauc: add support for crypt bundles

[Enrico Jorns]

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 platforms/image-rauc.in |  8 ++++++++
 rules/image-rauc.make   | 12 +++++++++++-
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/platforms/image-rauc.in b/platforms/image-rauc.in
index 3835e0718..7772fd95b 100644
--- a/platforms/image-rauc.in
+++ b/platforms/image-rauc.in
@@ -39,6 +39,14 @@ config IMAGE_RAUC_BUNDLE_FORMAT_VERITY
 	help
 	  The improved 'verity' bundle format
 
+config IMAGE_RAUC_BUNDLE_FORMAT_CRYPT
+	bool "crypt"
+	help
+	  The 'crypt' bundle format for fully encryptable bundles
+
+	  Note that you must run 'rauc encrypt' on a crypt bundle to generate a
+	  fully encrypted bundle.
+
 endchoice
 
 config IMAGE_RAUC_INTERMEDIATE
diff --git a/rules/image-rauc.make b/rules/image-rauc.make
index f7bed6e49..23d8d83b5 100644
--- a/rules/image-rauc.make
+++ b/rules/image-rauc.make
@@ -26,10 +26,20 @@ IMAGE_RAUC_CONFIG	:= rauc.config
 
 ifdef PTXCONF_IMAGE_RAUC
 
+ifdef PTXCONF_IMAGE_RAUC_BUNDLE_FORMAT_PLAIN
+IMAGE_RAUC_BUNDLE_FORMAT := "plain"
+endif
+ifdef PTXCONF_IMAGE_RAUC_BUNDLE_FORMAT_VERITY
+IMAGE_RAUC_BUNDLE_FORMAT := "verity"
+endif
+ifdef PTXCONF_IMAGE_RAUC_BUNDLE_FORMAT_CRYPT
+IMAGE_RAUC_BUNDLE_FORMAT := "crypt"
+endif
+
 IMAGE_RAUC_ENV	= \
 	$(CODE_SIGNING_ENV) \
 	RAUC_BUNDLE_COMPATIBLE="$(call remove_quotes,$(PTXCONF_RAUC_COMPATIBLE))" \
-	RAUC_BUNDLE_FORMAT="$(call ptx/ifdef,PTXCONF_IMAGE_RAUC_BUNDLE_FORMAT_VERITY,verity,plain)" \
+	RAUC_BUNDLE_FORMAT=$(IMAGE_RAUC_BUNDLE_FORMAT) \
 	RAUC_BUNDLE_VERSION="$(call remove_quotes, $(PTXCONF_RAUC_BUNDLE_VERSION))" \
 	RAUC_BUNDLE_BUILD=$(call ptx/sh, date +%FT%T%z) \
 	RAUC_BUNDLE_DESCRIPTION=$(PTXCONF_IMAGE_RAUC_DESCRIPTION) \
-- 
2.30.2

[ptxdist] [PATCH 5/5] image-rauc: default to 'verity' bundle format

[Enrico Jorns]

The 'plain' format is legacy and 'crypt' must only be used when
encryption is really required. Thus 'verity' is the default to choose.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 platforms/image-rauc.in | 1 +
 1 file changed, 1 insertion(+)

diff --git a/platforms/image-rauc.in b/platforms/image-rauc.in
index 7772fd95b..391956a42 100644
--- a/platforms/image-rauc.in
+++ b/platforms/image-rauc.in
@@ -28,6 +28,7 @@ config IMAGE_RAUC_DESCRIPTION
 
 choice
 	prompt "Bundle format"
+	default IMAGE_RAUC_BUNDLE_FORMAT_VERITY
 
 config IMAGE_RAUC_BUNDLE_FORMAT_PLAIN
 	bool "plain"
-- 
2.30.2

Re: [ptxdist] [PATCH 3/5] RAUC: leverage --disable-create option to shrink size for target RAUC

[Alexander Dahl]

Hello Enrico,

this might be nitpicking, but still …

Am Wed, Jun 15, 2022 at 04:22:15PM +0200 schrieb Enrico Jorns:
> We do not need bundle creation capabilities on the target in most cases,
> thus make this optional (and disable by default).
> 
> Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> ---
>  rules/host-rauc.make |  1 +
>  rules/rauc.in        | 10 ++++++++++
>  rules/rauc.make      |  1 +
>  3 files changed, 12 insertions(+)
> 
> diff --git a/rules/host-rauc.make b/rules/host-rauc.make
> index c78a518f6..1851aee9a 100644
> --- a/rules/host-rauc.make
> +++ b/rules/host-rauc.make
> @@ -23,6 +23,7 @@ HOST_RAUC_CONF_OPT	:= \
>  	$(HOST_AUTOCONF) \
>  	--disable-code-coverage \
>  	--disable-valgrind \
> +	--enable-create \

>From the cover letter I understood this is a new post 1.5 option,
right?

>  	--disable-service \
>  	--disable-network \
>  	--disable-json
> diff --git a/rules/rauc.in b/rules/rauc.in
> index 832967b44..e43692846 100644
> --- a/rules/rauc.in
> +++ b/rules/rauc.in
> @@ -27,6 +27,16 @@ menuconfig RAUC
>  
>  if RAUC
>  
> +config RAUC_CREATE
> +	bool
> +	prompt "Enable bundle creation support on target"
> +	default n
> +	help
> +	  Enables support for creating bundles on target.
> +	  Disabling this will remove functionality required for bundle
> +	  creation which is not required on the target in most cases and thus
> +	  allows to save space.
> +
>  config RAUC_SERVICE
>  	bool
>  	prompt "Enable service (D-Bus) support"
> diff --git a/rules/rauc.make b/rules/rauc.make
> index 820583e59..e001956a5 100644
> --- a/rules/rauc.make
> +++ b/rules/rauc.make
> @@ -41,6 +41,7 @@ RAUC_CONF_OPT	:= \
>  	$(GLOBAL_LARGE_FILE_OPTION) \
>  	--disable-code-coverage \
>  	--disable-valgrind \
> +	--$(call ptx/endis,PTXCONF_RAUC_CREATE)-create \

Same here. This should have set explicitly to a sensible default with
the package update. Making this optional would just change that line
then instead of adding it.

Greets
Alex

>  	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
>  	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \
>  	--$(call ptx/endis,PTXCONF_RAUC_STREAMING)-streaming \
> -- 
> 2.30.2
> 
> 

Re: [ptxdist] [PATCH 3/5] RAUC: leverage --disable-create option to shrink size for target RAUC

[Enrico Jörns]

Hi Alex,

Am Mittwoch, dem 15.06.2022 um 17:48 +0200 schrieb Alexander Dahl:
> Hello Enrico,
> 
> this might be nitpicking, but still …
> 
> Am Wed, Jun 15, 2022 at 04:22:15PM +0200 schrieb Enrico Jorns:
> > We do not need bundle creation capabilities on the target in most cases,
> > thus make this optional (and disable by default).
> > 
> > Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> > ---
> >  rules/host-rauc.make |  1 +
> >  rules/rauc.in        | 10 ++++++++++
> >  rules/rauc.make      |  1 +
> >  3 files changed, 12 insertions(+)
> > 
> > diff --git a/rules/host-rauc.make b/rules/host-rauc.make
> > index c78a518f6..1851aee9a 100644
> > --- a/rules/host-rauc.make
> > +++ b/rules/host-rauc.make
> > @@ -23,6 +23,7 @@ HOST_RAUC_CONF_OPT	:= \
> >  	$(HOST_AUTOCONF) \
> >  	--disable-code-coverage \
> >  	--disable-valgrind \
> > +	--enable-create \
> 
> From the cover letter I understood this is a new post 1.5 option,
> right?

almost, as the cover-letter explicitly notes this is a post 1.3 option. ;)

And this line is only for completeness. The added option in RAUC is semantically
--disable-create and --enable-create is the default.

> >  	--disable-service \
> >  	--disable-network \
> >  	--disable-json
> > diff --git a/rules/rauc.in b/rules/rauc.in
> > index 832967b44..e43692846 100644
> > --- a/rules/rauc.in
> > +++ b/rules/rauc.in
> > @@ -27,6 +27,16 @@ menuconfig RAUC
> >  
> >  if RAUC
> >  
> > +config RAUC_CREATE
> > +	bool
> > +	prompt "Enable bundle creation support on target"
> > +	default n
> > +	help
> > +	  Enables support for creating bundles on target.
> > +	  Disabling this will remove functionality required for bundle
> > +	  creation which is not required on the target in most cases and thus
> > +	  allows to save space.
> > +
> >  config RAUC_SERVICE
> >  	bool
> >  	prompt "Enable service (D-Bus) support"
> > diff --git a/rules/rauc.make b/rules/rauc.make
> > index 820583e59..e001956a5 100644
> > --- a/rules/rauc.make
> > +++ b/rules/rauc.make
> > @@ -41,6 +41,7 @@ RAUC_CONF_OPT	:= \
> >  	$(GLOBAL_LARGE_FILE_OPTION) \
> >  	--disable-code-coverage \
> >  	--disable-valgrind \
> > +	--$(call ptx/endis,PTXCONF_RAUC_CREATE)-create \
> 
> Same here. This should have set explicitly to a sensible default with
> the package update. Making this optional would just change that line
> then instead of adding it.

See above. The ptxdist default for RAUC_CREATE ("n") leads to --disable-create
which changes the RAUC default (which is --enable-create).


Regards, Enrico

> Greets
> Alex
> 
> >  	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
> >  	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \
> >  	--$(call ptx/endis,PTXCONF_RAUC_STREAMING)-streaming \
> > -- 
> > 2.30.2
> > 
> > 
> 
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

Re: [ptxdist] [PATCH 3/5] RAUC: leverage --disable-create option to shrink size for target RAUC

[Alexander Dahl]

[-- Attachment #1: Type: text/plain, Size: 5248 bytes --]

Hello Enrico,

On Wed, Jun 15, 2022 at 11:03:21PM +0200, Enrico Jörns wrote:
> Hi Alex,
> 
> Am Mittwoch, dem 15.06.2022 um 17:48 +0200 schrieb Alexander Dahl:
> > Hello Enrico,
> > 
> > this might be nitpicking, but still …
> > 
> > Am Wed, Jun 15, 2022 at 04:22:15PM +0200 schrieb Enrico Jorns:
> > > We do not need bundle creation capabilities on the target in most cases,
> > > thus make this optional (and disable by default).
> > > 
> > > Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> > > ---
> > >  rules/host-rauc.make |  1 +
> > >  rules/rauc.in        | 10 ++++++++++
> > >  rules/rauc.make      |  1 +
> > >  3 files changed, 12 insertions(+)
> > > 
> > > diff --git a/rules/host-rauc.make b/rules/host-rauc.make
> > > index c78a518f6..1851aee9a 100644
> > > --- a/rules/host-rauc.make
> > > +++ b/rules/host-rauc.make
> > > @@ -23,6 +23,7 @@ HOST_RAUC_CONF_OPT	:= \
> > >  	$(HOST_AUTOCONF) \
> > >  	--disable-code-coverage \
> > >  	--disable-valgrind \
> > > +	--enable-create \
> > 
> > From the cover letter I understood this is a new post 1.5 option,
> > right?
> 
> almost, as the cover-letter explicitly notes this is a post 1.3 option. ;)
> 
> And this line is only for completeness. The added option in RAUC is semantically
> --disable-create and --enable-create is the default.

make rules in ptxdist pin down _all_ configure options, so if that was
missing for 1.5 package it was already forgotten to set when upgrading
from 1.3 or 1.4.

> > >  	--disable-service \
> > >  	--disable-network \
> > >  	--disable-json
> > > diff --git a/rules/rauc.in b/rules/rauc.in
> > > index 832967b44..e43692846 100644
> > > --- a/rules/rauc.in
> > > +++ b/rules/rauc.in
> > > @@ -27,6 +27,16 @@ menuconfig RAUC
> > >  
> > >  if RAUC
> > >  
> > > +config RAUC_CREATE
> > > +	bool
> > > +	prompt "Enable bundle creation support on target"
> > > +	default n
> > > +	help
> > > +	  Enables support for creating bundles on target.
> > > +	  Disabling this will remove functionality required for bundle
> > > +	  creation which is not required on the target in most cases and thus
> > > +	  allows to save space.
> > > +
> > >  config RAUC_SERVICE
> > >  	bool
> > >  	prompt "Enable service (D-Bus) support"
> > > diff --git a/rules/rauc.make b/rules/rauc.make
> > > index 820583e59..e001956a5 100644
> > > --- a/rules/rauc.make
> > > +++ b/rules/rauc.make
> > > @@ -41,6 +41,7 @@ RAUC_CONF_OPT	:= \
> > >  	$(GLOBAL_LARGE_FILE_OPTION) \
> > >  	--disable-code-coverage \
> > >  	--disable-valgrind \
> > > +	--$(call ptx/endis,PTXCONF_RAUC_CREATE)-create \
> > 
> > Same here. This should have set explicitly to a sensible default with
> > the package update. Making this optional would just change that line
> > then instead of adding it.
> 
> See above. The ptxdist default for RAUC_CREATE ("n") leads to --disable-create
> which changes the RAUC default (which is --enable-create).

I understand RAUC has some opinion what should be default. Some
embedded build systems like buildroot stick with those defaults (for
all options not explicitly set through menu). Others like ptxdist set
all configure options. The script 'scripts/configure_helper.py' helps
determining missing options not set. See example output for package
host-rauc on a ptxdist 2021.06.0 based BSP:

	% ~/src/ptxdist/scripts/configure_helper.py -p host-rauc
	--- rules/host-rauc.make
	+++ rauc-1.5.1
	@@ -1,6 +1,17 @@
			--prefix=
	+       --enable-debug=yes/info/profile/no
	+       --disable-largefile
	+       --enable-compile-warnings=no/yes/error
	+       --disable-Werror
			--disable-code-coverage
			--disable-valgrind
			--disable-service
	+       --disable-create
			--disable-network
			--disable-json
	+       --enable-gpt
	+       --with-gcov=GCOV
	+       --with-systemdunitdir=DIR
	+       --with-dbuspolicydir=DIR
	+       --with-dbussystemservicedir=DIR
	+       --with-dbusinterfacesdir=DIR

You see --disable-create is not the only option not set in
rules/host-rauc.make and I guess it might look similar for rauc.make?

Pinning down the old options should go to a patch before bumping the
version.

Greets
Alex

> 
> Regards, Enrico
> 
> > Greets
> > Alex
> > 
> > >  	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
> > >  	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \
> > >  	--$(call ptx/endis,PTXCONF_RAUC_STREAMING)-streaming \
> > > -- 
> > > 2.30.2
> > > 
> > > 
> > 
> > 
> 
> -- 
> Pengutronix e.K.                           | Enrico Jörns                |
> Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
> Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
> 31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

-- 
/"\ ASCII RIBBON | »With the first link, the chain is forged. The first
\ / CAMPAIGN     | speech censured, the first thought forbidden, the
 X  AGAINST      | first freedom denied, chains us all irrevocably.«
/ \ HTML MAIL    | (Jean-Luc Picard, quoting Judge Aaron Satie)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [ptxdist] [PATCH 3/5] RAUC: leverage --disable-create option to shrink size for target RAUC

[Enrico Jörns]

Hello Alex,

Am Donnerstag, dem 16.06.2022 um 08:16 +0200 schrieb Alexander Dahl:
> > > > 
> > > > diff --git a/rules/host-rauc.make b/rules/host-rauc.make
> > > > index c78a518f6..1851aee9a 100644
> > > > --- a/rules/host-rauc.make
> > > > +++ b/rules/host-rauc.make
> > > > @@ -23,6 +23,7 @@ HOST_RAUC_CONF_OPT	:= \
> > > >  	$(HOST_AUTOCONF) \
> > > >  	--disable-code-coverage \
> > > >  	--disable-valgrind \
> > > > +	--enable-create \
> > > 
> > > From the cover letter I understood this is a new post 1.5 option,
> > > right?
> > 
> > almost, as the cover-letter explicitly notes this is a post 1.3 option. ;)
> > 
> > And this line is only for completeness. The added option in RAUC is semantically
> > --disable-create and --enable-create is the default.
> 
> make rules in ptxdist pin down _all_ configure options, so if that was
> missing for 1.5 package it was already forgotten to set when upgrading
> from 1.3 or 1.4.

yes, there have been no adaptions since the initial recipe submission.

Seems as if I've won an extra price for this round. \o/

> > > >  	--disable-service \
> > > >  	--disable-network \
> > > >  	--disable-json
> > > > diff --git a/rules/rauc.in b/rules/rauc.in
> > > > index 832967b44..e43692846 100644
> > > > --- a/rules/rauc.in
> > > > +++ b/rules/rauc.in
> > > > @@ -27,6 +27,16 @@ menuconfig RAUC
> > > >  
> > > >  if RAUC
> > > >  
> > > > +config RAUC_CREATE
> > > > +	bool
> > > > +	prompt "Enable bundle creation support on target"
> > > > +	default n
> > > > +	help
> > > > +	  Enables support for creating bundles on target.
> > > > +	  Disabling this will remove functionality required for bundle
> > > > +	  creation which is not required on the target in most cases and thus
> > > > +	  allows to save space.
> > > > +
> > > >  config RAUC_SERVICE
> > > >  	bool
> > > >  	prompt "Enable service (D-Bus) support"
> > > > diff --git a/rules/rauc.make b/rules/rauc.make
> > > > index 820583e59..e001956a5 100644
> > > > --- a/rules/rauc.make
> > > > +++ b/rules/rauc.make
> > > > @@ -41,6 +41,7 @@ RAUC_CONF_OPT	:= \
> > > >  	$(GLOBAL_LARGE_FILE_OPTION) \
> > > >  	--disable-code-coverage \
> > > >  	--disable-valgrind \
> > > > +	--$(call ptx/endis,PTXCONF_RAUC_CREATE)-create \
> > > 
> > > Same here. This should have set explicitly to a sensible default with
> > > the package update. Making this optional would just change that line
> > > then instead of adding it.
> > 
> > See above. The ptxdist default for RAUC_CREATE ("n") leads to --disable-create
> > which changes the RAUC default (which is --enable-create).
> 
> I understand RAUC has some opinion what should be default. Some
> embedded build systems like buildroot stick with those defaults (for
> all options not explicitly set through menu). Others like ptxdist set
> all configure options. The script 'scripts/configure_helper.py' helps
> determining missing options not set. See example output for package
> host-rauc on a ptxdist 2021.06.0 based BSP:

Since I know some of them, I'd tend to say the group of 'Others' exclusively
consists of ptxdist to the best of my knowledge.
But yes, in theory I am aware of this design decision ;)

Thanks for the helper script, I'll give it a try.

> 	% ~/src/ptxdist/scripts/configure_helper.py -p host-rauc
> 	--- rules/host-rauc.make
> 	+++ rauc-1.5.1
> 	@@ -1,6 +1,17 @@
> 			--prefix=
> 	+       --enable-debug=yes/info/profile/no
> 	+       --disable-largefile
> 	+       --enable-compile-warnings=no/yes/error
> 	+       --disable-Werror
> 			--disable-code-coverage
> 			--disable-valgrind
> 			--disable-service
> 	+       --disable-create
> 			--disable-network
> 			--disable-json
> 	+       --enable-gpt
> 	+       --with-gcov=GCOV
> 	+       --with-systemdunitdir=DIR
> 	+       --with-dbuspolicydir=DIR
> 	+       --with-dbussystemservicedir=DIR
> 	+       --with-dbusinterfacesdir=DIR
> 
> You see --disable-create is not the only option not set in
> rules/host-rauc.make and I guess it might look similar for rauc.make?

It does, yes (with a little bit less diff lines).

> Pinning down the old options should go to a patch before bumping the
> version.

Give me a second..


Best regards

Enrico

> 
> Greets
> Alex
> 
> > 
> > Regards, Enrico
> > 
> > > Greets
> > > Alex
> > > 
> > > >  	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
> > > >  	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \
> > > >  	--$(call ptx/endis,PTXCONF_RAUC_STREAMING)-streaming \
> > > > -- 
> > > > 2.30.2
> > > > 
> > > > 
> > > 
> > > 
> > 
> > -- 
> > Pengutronix e.K.                           | Enrico Jörns                |
> > Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
> > Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
> > 31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

Re: [ptxdist] [PATCH 2/5] RAUC: add option to enable streaming support

[Michael Olbrich]

On Wed, Jun 15, 2022 at 04:22:14PM +0200, Enrico Jorns wrote:
> Enable it by default so that default installation can benefit from this.
> Can be deactivated for saving space or for deactivating network
> functionality.
> Web server must support HTTP range requests for this and NBD support
> must be enabled in kernel.
> 
> Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> ---
>  rules/rauc.in   | 11 +++++++++++
>  rules/rauc.make |  1 +
>  2 files changed, 12 insertions(+)
> 
> diff --git a/rules/rauc.in b/rules/rauc.in
> index aee162b83..832967b44 100644
> --- a/rules/rauc.in
> +++ b/rules/rauc.in
> @@ -13,6 +13,7 @@ menuconfig RAUC
>  	select BUSYBOX_FEATURE_SEAMLESS_GZ	if BUSYBOX_TAR
>  	select DBUS				if RAUC_SERVICE
>  	select LIBCURL				if RAUC_NETWORK
> +	select LIBNL3				if RAUC_STREAMING
>  	select JSON_GLIB			if RAUC_JSON
>  	select UTIL_LINUX_NG_LIBFDISK		if RAUC_GPT
>  	select SQUASHFS_TOOLS			if RUNTIME
> @@ -44,6 +45,16 @@ config RAUC_NETWORK
>  	  Note that network is primarily designed to be used for RAUC'S casync
>  	  capabilities, not for fetching full bundles.
>  
> +config RAUC_STREAMING
> +	bool
> +	prompt "Enable streaming support"
> +	default y

Why 'default y'? I think most users won't need this.

Michael

> +	select RAUC_NETWORK
> +	help
> +	  Enables built-in HTTP(S) streaming support that allows RAUC to
> +	  access remote bundles via http/https and write images without the
> +	  need for intermediate storage on the target.
> +
>  config RAUC_JSON
>  	bool
>  	prompt "JSON support"
> diff --git a/rules/rauc.make b/rules/rauc.make
> index 46d196bed..820583e59 100644
> --- a/rules/rauc.make
> +++ b/rules/rauc.make
> @@ -43,6 +43,7 @@ RAUC_CONF_OPT	:= \
>  	--disable-valgrind \
>  	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
>  	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \
> +	--$(call ptx/endis,PTXCONF_RAUC_STREAMING)-streaming \
>  	--$(call ptx/endis,PTXCONF_RAUC_JSON)-json \
>  	--$(call ptx/endis,PTXCONF_RAUC_GPT)-gpt \
>  	--with-systemdunitdir=/usr/lib/systemd/system \
> -- 
> 2.30.2
> 
> 
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] [PATCH 2/5] RAUC: add option to enable streaming support

[Enrico Jörns]

Am Freitag, dem 17.06.2022 um 08:28 +0200 schrieb Michael Olbrich:
> On Wed, Jun 15, 2022 at 04:22:14PM +0200, Enrico Jorns wrote:
> > Enable it by default so that default installation can benefit from this.
> > Can be deactivated for saving space or for deactivating network
> > functionality.
> > Web server must support HTTP range requests for this and NBD support
> > must be enabled in kernel.
> > 
> > Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> > ---
> >  rules/rauc.in   | 11 +++++++++++
> >  rules/rauc.make |  1 +
> >  2 files changed, 12 insertions(+)
> > 
> > diff --git a/rules/rauc.in b/rules/rauc.in
> > index aee162b83..832967b44 100644
> > --- a/rules/rauc.in
> > +++ b/rules/rauc.in
> > @@ -13,6 +13,7 @@ menuconfig RAUC
> >  	select BUSYBOX_FEATURE_SEAMLESS_GZ	if BUSYBOX_TAR
> >  	select DBUS				if RAUC_SERVICE
> >  	select LIBCURL				if RAUC_NETWORK
> > +	select LIBNL3				if RAUC_STREAMING
> >  	select JSON_GLIB			if RAUC_JSON
> >  	select UTIL_LINUX_NG_LIBFDISK		if RAUC_GPT
> >  	select SQUASHFS_TOOLS			if RUNTIME
> > @@ -44,6 +45,16 @@ config RAUC_NETWORK
> >  	  Note that network is primarily designed to be used for RAUC'S casync
> >  	  capabilities, not for fetching full bundles.
> >  
> > +config RAUC_STREAMING
> > +	bool
> > +	prompt "Enable streaming support"
> > +	default y
> 
> Why 'default y'? I think most users won't need this.

I am not convinced that streaming is not required for most users. I guess that
being able to download bundles form HTTP directly is a use case that is more and
more common. With network and streaming enabled, one can download bundles only
externally or install from USB sticks etc.

In Yocto we have this enabled by default, too. If it's ptxdist's philosophy to
ship as little features as possible by default (i.e. focus on small size), we
can also make this default n.


Enrico

> Michael
> 
> > +	select RAUC_NETWORK
> > +	help
> > +	  Enables built-in HTTP(S) streaming support that allows RAUC to
> > +	  access remote bundles via http/https and write images without the
> > +	  need for intermediate storage on the target.
> > +
> >  config RAUC_JSON
> >  	bool
> >  	prompt "JSON support"
> > diff --git a/rules/rauc.make b/rules/rauc.make
> > index 46d196bed..820583e59 100644
> > --- a/rules/rauc.make
> > +++ b/rules/rauc.make
> > @@ -43,6 +43,7 @@ RAUC_CONF_OPT	:= \
> >  	--disable-valgrind \
> >  	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
> >  	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \
> > +	--$(call ptx/endis,PTXCONF_RAUC_STREAMING)-streaming \
> >  	--$(call ptx/endis,PTXCONF_RAUC_JSON)-json \
> >  	--$(call ptx/endis,PTXCONF_RAUC_GPT)-gpt \
> >  	--with-systemdunitdir=/usr/lib/systemd/system \
> > -- 
> > 2.30.2
> > 
> > 
> > 
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

Re: [ptxdist] [PATCH 2/5] RAUC: add option to enable streaming support

[Michael Olbrich]

On Fri, Jun 17, 2022 at 11:47:19AM +0200, Enrico Jörns wrote:
> Am Freitag, dem 17.06.2022 um 08:28 +0200 schrieb Michael Olbrich:
> > On Wed, Jun 15, 2022 at 04:22:14PM +0200, Enrico Jorns wrote:
> > > Enable it by default so that default installation can benefit from this.
> > > Can be deactivated for saving space or for deactivating network
> > > functionality.
> > > Web server must support HTTP range requests for this and NBD support
> > > must be enabled in kernel.
> > > 
> > > Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> > > ---
> > >  rules/rauc.in   | 11 +++++++++++
> > >  rules/rauc.make |  1 +
> > >  2 files changed, 12 insertions(+)
> > > 
> > > diff --git a/rules/rauc.in b/rules/rauc.in
> > > index aee162b83..832967b44 100644
> > > --- a/rules/rauc.in
> > > +++ b/rules/rauc.in
> > > @@ -13,6 +13,7 @@ menuconfig RAUC
> > >  	select BUSYBOX_FEATURE_SEAMLESS_GZ	if BUSYBOX_TAR
> > >  	select DBUS				if RAUC_SERVICE
> > >  	select LIBCURL				if RAUC_NETWORK
> > > +	select LIBNL3				if RAUC_STREAMING
> > >  	select JSON_GLIB			if RAUC_JSON
> > >  	select UTIL_LINUX_NG_LIBFDISK		if RAUC_GPT
> > >  	select SQUASHFS_TOOLS			if RUNTIME
> > > @@ -44,6 +45,16 @@ config RAUC_NETWORK
> > >  	  Note that network is primarily designed to be used for RAUC'S casync
> > >  	  capabilities, not for fetching full bundles.
> > >  
> > > +config RAUC_STREAMING
> > > +	bool
> > > +	prompt "Enable streaming support"
> > > +	default y
> > 
> > Why 'default y'? I think most users won't need this.
> 
> I am not convinced that streaming is not required for most users. I guess that
> being able to download bundles form HTTP directly is a use case that is more and
> more common. With network and streaming enabled, one can download bundles only
> externally or install from USB sticks etc.
> 
> In Yocto we have this enabled by default, too. If it's ptxdist's philosophy to
> ship as little features as possible by default (i.e. focus on small size), we
> can also make this default n.

The thing is, nobody changes the defaults unless they notice something is
not working as expected. And enabling things in PTXdist is easy.

How does this work anyways? If I call 'rauc install <some-url>', will this
use streaming implicitly or does it require an explicit option?

How about 'default RAUC_NETWORK'? This way those that use networking also
get streaming by default.

I can do a fixup here if this makes sense.

Michael


> > > +	select RAUC_NETWORK
> > > +	help
> > > +	  Enables built-in HTTP(S) streaming support that allows RAUC to
> > > +	  access remote bundles via http/https and write images without the
> > > +	  need for intermediate storage on the target.
> > > +
> > >  config RAUC_JSON
> > >  	bool
> > >  	prompt "JSON support"
> > > diff --git a/rules/rauc.make b/rules/rauc.make
> > > index 46d196bed..820583e59 100644
> > > --- a/rules/rauc.make
> > > +++ b/rules/rauc.make
> > > @@ -43,6 +43,7 @@ RAUC_CONF_OPT	:= \
> > >  	--disable-valgrind \
> > >  	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
> > >  	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \
> > > +	--$(call ptx/endis,PTXCONF_RAUC_STREAMING)-streaming \
> > >  	--$(call ptx/endis,PTXCONF_RAUC_JSON)-json \
> > >  	--$(call ptx/endis,PTXCONF_RAUC_GPT)-gpt \
> > >  	--with-systemdunitdir=/usr/lib/systemd/system \
> > > -- 
> > > 2.30.2
> > > 
> > > 
> > > 
> > 
> 
> -- 
> Pengutronix e.K.                           | Enrico Jörns                |
> Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
> Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
> 31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] [PATCH 2/5] RAUC: add option to enable streaming support

[Michael Olbrich]

On Fri, Jun 17, 2022 at 12:27:50PM +0200, Michael Olbrich wrote:
> On Fri, Jun 17, 2022 at 11:47:19AM +0200, Enrico Jörns wrote:
> > Am Freitag, dem 17.06.2022 um 08:28 +0200 schrieb Michael Olbrich:
> > > On Wed, Jun 15, 2022 at 04:22:14PM +0200, Enrico Jorns wrote:
> > > > Enable it by default so that default installation can benefit from this.
> > > > Can be deactivated for saving space or for deactivating network
> > > > functionality.
> > > > Web server must support HTTP range requests for this and NBD support
> > > > must be enabled in kernel.
> > > > 
> > > > Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> > > > ---
> > > >  rules/rauc.in   | 11 +++++++++++
> > > >  rules/rauc.make |  1 +
> > > >  2 files changed, 12 insertions(+)
> > > > 
> > > > diff --git a/rules/rauc.in b/rules/rauc.in
> > > > index aee162b83..832967b44 100644
> > > > --- a/rules/rauc.in
> > > > +++ b/rules/rauc.in
> > > > @@ -13,6 +13,7 @@ menuconfig RAUC
> > > >  	select BUSYBOX_FEATURE_SEAMLESS_GZ	if BUSYBOX_TAR
> > > >  	select DBUS				if RAUC_SERVICE
> > > >  	select LIBCURL				if RAUC_NETWORK
> > > > +	select LIBNL3				if RAUC_STREAMING
> > > >  	select JSON_GLIB			if RAUC_JSON
> > > >  	select UTIL_LINUX_NG_LIBFDISK		if RAUC_GPT
> > > >  	select SQUASHFS_TOOLS			if RUNTIME
> > > > @@ -44,6 +45,16 @@ config RAUC_NETWORK
> > > >  	  Note that network is primarily designed to be used for RAUC'S casync
> > > >  	  capabilities, not for fetching full bundles.
> > > >  
> > > > +config RAUC_STREAMING
> > > > +	bool
> > > > +	prompt "Enable streaming support"
> > > > +	default y
> > > 
> > > Why 'default y'? I think most users won't need this.
> > 
> > I am not convinced that streaming is not required for most users. I guess that
> > being able to download bundles form HTTP directly is a use case that is more and
> > more common. With network and streaming enabled, one can download bundles only
> > externally or install from USB sticks etc.
> > 
> > In Yocto we have this enabled by default, too. If it's ptxdist's philosophy to
> > ship as little features as possible by default (i.e. focus on small size), we
> > can also make this default n.
> 
> The thing is, nobody changes the defaults unless they notice something is
> not working as expected. And enabling things in PTXdist is easy.
> 
> How does this work anyways? If I call 'rauc install <some-url>', will this
> use streaming implicitly or does it require an explicit option?
> 
> How about 'default RAUC_NETWORK'? This way those that use networking also
> get streaming by default.

Grr, kconfig does not allow that. So it'll be disabled by default.

Hmmm, or maybe just use the same option? What would the consequences be?

Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] [PATCH 2/5] RAUC: add option to enable streaming support

[Alexander Dahl]

[-- Attachment #1: Type: text/plain, Size: 5016 bytes --]

Moin,

On Fri, Jun 17, 2022 at 12:27:50PM +0200, Michael Olbrich wrote:
> On Fri, Jun 17, 2022 at 11:47:19AM +0200, Enrico Jörns wrote:
> > Am Freitag, dem 17.06.2022 um 08:28 +0200 schrieb Michael Olbrich:
> > > On Wed, Jun 15, 2022 at 04:22:14PM +0200, Enrico Jorns wrote:
> > > > Enable it by default so that default installation can benefit from this.
> > > > Can be deactivated for saving space or for deactivating network
> > > > functionality.
> > > > Web server must support HTTP range requests for this and NBD support
> > > > must be enabled in kernel.
> > > > 
> > > > Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> > > > ---
> > > >  rules/rauc.in   | 11 +++++++++++
> > > >  rules/rauc.make |  1 +
> > > >  2 files changed, 12 insertions(+)
> > > > 
> > > > diff --git a/rules/rauc.in b/rules/rauc.in
> > > > index aee162b83..832967b44 100644
> > > > --- a/rules/rauc.in
> > > > +++ b/rules/rauc.in
> > > > @@ -13,6 +13,7 @@ menuconfig RAUC
> > > >  	select BUSYBOX_FEATURE_SEAMLESS_GZ	if BUSYBOX_TAR
> > > >  	select DBUS				if RAUC_SERVICE
> > > >  	select LIBCURL				if RAUC_NETWORK
> > > > +	select LIBNL3				if RAUC_STREAMING
> > > >  	select JSON_GLIB			if RAUC_JSON
> > > >  	select UTIL_LINUX_NG_LIBFDISK		if RAUC_GPT
> > > >  	select SQUASHFS_TOOLS			if RUNTIME
> > > > @@ -44,6 +45,16 @@ config RAUC_NETWORK
> > > >  	  Note that network is primarily designed to be used for RAUC'S casync
> > > >  	  capabilities, not for fetching full bundles.
> > > >  
> > > > +config RAUC_STREAMING
> > > > +	bool
> > > > +	prompt "Enable streaming support"
> > > > +	default y
> > > 
> > > Why 'default y'? I think most users won't need this.
> > 
> > I am not convinced that streaming is not required for most users. I guess that
> > being able to download bundles form HTTP directly is a use case that is more and
> > more common. With network and streaming enabled, one can download bundles only
> > externally or install from USB sticks etc.
> > 
> > In Yocto we have this enabled by default, too. If it's ptxdist's philosophy to
> > ship as little features as possible by default (i.e. focus on small size), we
> > can also make this default n.
> 
> The thing is, nobody changes the defaults unless they notice something is
> not working as expected. And enabling things in PTXdist is easy.

Despite how the default should be if one enables RAUC the first time,
how does this behave when upgrading?  I mean if I have RAUC enabled
and running with some method, then this update comes along.  Does
ptxdist enable that option on migrate?  I yes I would opt against
making this default y, because that would just add a then unused
feature.

A

> 
> How does this work anyways? If I call 'rauc install <some-url>', will this
> use streaming implicitly or does it require an explicit option?
> 
> How about 'default RAUC_NETWORK'? This way those that use networking also
> get streaming by default.
> 
> I can do a fixup here if this makes sense.
> 
> Michael
> 
> 
> > > > +	select RAUC_NETWORK
> > > > +	help
> > > > +	  Enables built-in HTTP(S) streaming support that allows RAUC to
> > > > +	  access remote bundles via http/https and write images without the
> > > > +	  need for intermediate storage on the target.
> > > > +
> > > >  config RAUC_JSON
> > > >  	bool
> > > >  	prompt "JSON support"
> > > > diff --git a/rules/rauc.make b/rules/rauc.make
> > > > index 46d196bed..820583e59 100644
> > > > --- a/rules/rauc.make
> > > > +++ b/rules/rauc.make
> > > > @@ -43,6 +43,7 @@ RAUC_CONF_OPT	:= \
> > > >  	--disable-valgrind \
> > > >  	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
> > > >  	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \
> > > > +	--$(call ptx/endis,PTXCONF_RAUC_STREAMING)-streaming \
> > > >  	--$(call ptx/endis,PTXCONF_RAUC_JSON)-json \
> > > >  	--$(call ptx/endis,PTXCONF_RAUC_GPT)-gpt \
> > > >  	--with-systemdunitdir=/usr/lib/systemd/system \
> > > > -- 
> > > > 2.30.2
> > > > 
> > > > 
> > > > 
> > > 
> > 
> > -- 
> > Pengutronix e.K.                           | Enrico Jörns                |
> > Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
> > Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
> > 31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |
> > 
> 
> -- 
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

-- 
/"\ ASCII RIBBON | »With the first link, the chain is forged. The first
\ / CAMPAIGN     | speech censured, the first thought forbidden, the
 X  AGAINST      | first freedom denied, chains us all irrevocably.«
/ \ HTML MAIL    | (Jean-Luc Picard, quoting Judge Aaron Satie)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [ptxdist] [PATCH 2/5] RAUC: add option to enable streaming support

[Enrico Jörns]

Am Freitag, dem 17.06.2022 um 12:48 +0200 schrieb Michael Olbrich:
> On Fri, Jun 17, 2022 at 12:27:50PM +0200, Michael Olbrich wrote:
> > On Fri, Jun 17, 2022 at 11:47:19AM +0200, Enrico Jörns wrote:
> > > Am Freitag, dem 17.06.2022 um 08:28 +0200 schrieb Michael Olbrich:
> > > > On Wed, Jun 15, 2022 at 04:22:14PM +0200, Enrico Jorns wrote:
> > > > > Enable it by default so that default installation can benefit from this.
> > > > > Can be deactivated for saving space or for deactivating network
> > > > > functionality.
> > > > > Web server must support HTTP range requests for this and NBD support
> > > > > must be enabled in kernel.
> > > > > 
> > > > > Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> > > > > ---
> > > > >  rules/rauc.in   | 11 +++++++++++
> > > > >  rules/rauc.make |  1 +
> > > > >  2 files changed, 12 insertions(+)
> > > > > 
> > > > > diff --git a/rules/rauc.in b/rules/rauc.in
> > > > > index aee162b83..832967b44 100644
> > > > > --- a/rules/rauc.in
> > > > > +++ b/rules/rauc.in
> > > > > @@ -13,6 +13,7 @@ menuconfig RAUC
> > > > >  	select BUSYBOX_FEATURE_SEAMLESS_GZ	if BUSYBOX_TAR
> > > > >  	select DBUS				if RAUC_SERVICE
> > > > >  	select LIBCURL				if RAUC_NETWORK
> > > > > +	select LIBNL3				if RAUC_STREAMING
> > > > >  	select JSON_GLIB			if RAUC_JSON
> > > > >  	select UTIL_LINUX_NG_LIBFDISK		if RAUC_GPT
> > > > >  	select SQUASHFS_TOOLS			if RUNTIME
> > > > > @@ -44,6 +45,16 @@ config RAUC_NETWORK
> > > > >  	  Note that network is primarily designed to be used for RAUC'S casync
> > > > >  	  capabilities, not for fetching full bundles.
> > > > >  
> > > > > +config RAUC_STREAMING
> > > > > +	bool
> > > > > +	prompt "Enable streaming support"
> > > > > +	default y
> > > > 
> > > > Why 'default y'? I think most users won't need this.
> > > 
> > > I am not convinced that streaming is not required for most users. I guess that
> > > being able to download bundles form HTTP directly is a use case that is more and
> > > more common. With network and streaming enabled, one can download bundles only
> > > externally or install from USB sticks etc.
> > > 
> > > In Yocto we have this enabled by default, too. If it's ptxdist's philosophy to
> > > ship as little features as possible by default (i.e. focus on small size), we
> > > can also make this default n.
> > 
> > The thing is, nobody changes the defaults unless they notice something is
> > not working as expected. And enabling things in PTXdist is easy.

Indeed.

> > How does this work anyways? If I call 'rauc install <some-url>', will this
> > use streaming implicitly or does it require an explicit option?

No explicit option required. Streaming will be used by default. But required NBD
support to be enabled in the kernel.

Without streaming enabled (only network enabled), the download size of RAUC is
limited by default to allow only casync bundles that do not have any image
inside (only index files).

> > How about 'default RAUC_NETWORK'? This way those that use networking also
> > get streaming by default.
> 
> Grr, kconfig does not allow that. So it'll be disabled by default.
> 
> Hmmm, or maybe just use the same option? What would the consequences be?

As up to now the use case for RAUC_NETWORK was primarily casync support, I would
assume that most of these existing users do not require/benefit from built-in
streaming.

Thus maybe just make it 'n' then.

Note that I have to rebase this series anyway in case the other rauc series I
sent (for pinning configure options) is applied.


Regards, Enrico

> Michael
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

Re: [ptxdist] [PATCH 2/5] RAUC: add option to enable streaming support

[Michael Olbrich]

On Fri, Jun 17, 2022 at 01:47:14PM +0200, Enrico Jörns wrote:
> Am Freitag, dem 17.06.2022 um 12:48 +0200 schrieb Michael Olbrich:
> > On Fri, Jun 17, 2022 at 12:27:50PM +0200, Michael Olbrich wrote:
> > > On Fri, Jun 17, 2022 at 11:47:19AM +0200, Enrico Jörns wrote:
> > > > Am Freitag, dem 17.06.2022 um 08:28 +0200 schrieb Michael Olbrich:
> > > > > On Wed, Jun 15, 2022 at 04:22:14PM +0200, Enrico Jorns wrote:
> > > > > > Enable it by default so that default installation can benefit from this.
> > > > > > Can be deactivated for saving space or for deactivating network
> > > > > > functionality.
> > > > > > Web server must support HTTP range requests for this and NBD support
> > > > > > must be enabled in kernel.
> > > > > > 
> > > > > > Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> > > > > > ---
> > > > > >  rules/rauc.in   | 11 +++++++++++
> > > > > >  rules/rauc.make |  1 +
> > > > > >  2 files changed, 12 insertions(+)
> > > > > > 
> > > > > > diff --git a/rules/rauc.in b/rules/rauc.in
> > > > > > index aee162b83..832967b44 100644
> > > > > > --- a/rules/rauc.in
> > > > > > +++ b/rules/rauc.in
> > > > > > @@ -13,6 +13,7 @@ menuconfig RAUC
> > > > > >  	select BUSYBOX_FEATURE_SEAMLESS_GZ	if BUSYBOX_TAR
> > > > > >  	select DBUS				if RAUC_SERVICE
> > > > > >  	select LIBCURL				if RAUC_NETWORK
> > > > > > +	select LIBNL3				if RAUC_STREAMING
> > > > > >  	select JSON_GLIB			if RAUC_JSON
> > > > > >  	select UTIL_LINUX_NG_LIBFDISK		if RAUC_GPT
> > > > > >  	select SQUASHFS_TOOLS			if RUNTIME
> > > > > > @@ -44,6 +45,16 @@ config RAUC_NETWORK
> > > > > >  	  Note that network is primarily designed to be used for RAUC'S casync
> > > > > >  	  capabilities, not for fetching full bundles.
> > > > > >  
> > > > > > +config RAUC_STREAMING
> > > > > > +	bool
> > > > > > +	prompt "Enable streaming support"
> > > > > > +	default y
> > > > > 
> > > > > Why 'default y'? I think most users won't need this.
> > > > 
> > > > I am not convinced that streaming is not required for most users. I guess that
> > > > being able to download bundles form HTTP directly is a use case that is more and
> > > > more common. With network and streaming enabled, one can download bundles only
> > > > externally or install from USB sticks etc.
> > > > 
> > > > In Yocto we have this enabled by default, too. If it's ptxdist's philosophy to
> > > > ship as little features as possible by default (i.e. focus on small size), we
> > > > can also make this default n.
> > > 
> > > The thing is, nobody changes the defaults unless they notice something is
> > > not working as expected. And enabling things in PTXdist is easy.
> 
> Indeed.
> 
> > > How does this work anyways? If I call 'rauc install <some-url>', will this
> > > use streaming implicitly or does it require an explicit option?
> 
> No explicit option required. Streaming will be used by default. But required NBD
> support to be enabled in the kernel.
> 
> Without streaming enabled (only network enabled), the download size of RAUC is
> limited by default to allow only casync bundles that do not have any image
> inside (only index files).
> 
> > > How about 'default RAUC_NETWORK'? This way those that use networking also
> > > get streaming by default.
> > 
> > Grr, kconfig does not allow that. So it'll be disabled by default.
> > 
> > Hmmm, or maybe just use the same option? What would the consequences be?
> 
> As up to now the use case for RAUC_NETWORK was primarily casync support, I would
> assume that most of these existing users do not require/benefit from built-in
> streaming.
> 
> Thus maybe just make it 'n' then.
> 
> Note that I have to rebase this series anyway in case the other rauc series I
> sent (for pinning configure options) is applied.

Not needed. It applies just fine on-top of that. No need for a resent.
Btw, 'default n' is the default, so need to add that (I've removed it in
one of the patches).

Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] [APPLIED] RAUC: version bump 1.5.1 -> 1.7

[Michael Olbrich]

Thanks, applied as ae96c2a86af3035fb7d8b00e128fcb39a8e4290a.

Michael

[sent from post-receive hook]

On Mon, 20 Jun 2022 08:19:57 +0200, Enrico Jorns <ejo@pengutronix.de> wrote:
> Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> Message-Id: <20220615142217.990960-2-ejo@pengutronix.de>
> Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
> 
> diff --git a/rules/rauc.make b/rules/rauc.make
> index 41b76de7ca7e..a1be8b1b2f89 100644
> --- a/rules/rauc.make
> +++ b/rules/rauc.make
> @@ -14,8 +14,8 @@ PACKAGES-$(PTXCONF_RAUC) += rauc
>  #
>  # Paths and names
>  #
> -RAUC_VERSION	:= 1.5.1
> -RAUC_MD5	:= e1c1fad12d81b955784968c37130d272
> +RAUC_VERSION	:= 1.7
> +RAUC_MD5	:= aa31fc2a457ba2e3fca8206d9fb9f126
>  RAUC		:= rauc-$(RAUC_VERSION)
>  RAUC_SUFFIX	:= tar.xz
>  RAUC_URL	:= https://github.com/rauc/rauc/releases/download/v$(RAUC_VERSION)/$(RAUC).$(RAUC_SUFFIX)

Re: [ptxdist] [APPLIED] RAUC: add option to enable streaming support

[Michael Olbrich]

Thanks, applied as 7c378e97df58c372cdef473fbd2b0ee613dfeff9.

Michael

[sent from post-receive hook]

On Mon, 20 Jun 2022 08:19:58 +0200, Enrico Jorns <ejo@pengutronix.de> wrote:
> Enable it by default so that default installation can benefit from this.
> Can be deactivated for saving space or for deactivating network
> functionality.
> Web server must support HTTP range requests for this and NBD support
> must be enabled in kernel.
> 
> Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> Message-Id: <20220615142217.990960-3-ejo@pengutronix.de>
> [mol: change default to n]
> Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
> 
> diff --git a/rules/rauc.in b/rules/rauc.in
> index aee162b83ac4..c6c697df7e9d 100644
> --- a/rules/rauc.in
> +++ b/rules/rauc.in
> @@ -13,6 +13,7 @@ menuconfig RAUC
>  	select BUSYBOX_FEATURE_SEAMLESS_GZ	if BUSYBOX_TAR
>  	select DBUS				if RAUC_SERVICE
>  	select LIBCURL				if RAUC_NETWORK
> +	select LIBNL3				if RAUC_STREAMING
>  	select JSON_GLIB			if RAUC_JSON
>  	select UTIL_LINUX_NG_LIBFDISK		if RAUC_GPT
>  	select SQUASHFS_TOOLS			if RUNTIME
> @@ -44,6 +45,15 @@ config RAUC_NETWORK
>  	  Note that network is primarily designed to be used for RAUC'S casync
>  	  capabilities, not for fetching full bundles.
>  
> +config RAUC_STREAMING
> +	bool
> +	prompt "Enable streaming support"
> +	select RAUC_NETWORK
> +	help
> +	  Enables built-in HTTP(S) streaming support that allows RAUC to
> +	  access remote bundles via http/https and write images without the
> +	  need for intermediate storage on the target.
> +
>  config RAUC_JSON
>  	bool
>  	prompt "JSON support"
> diff --git a/rules/rauc.make b/rules/rauc.make
> index a1be8b1b2f89..78f6c894b73e 100644
> --- a/rules/rauc.make
> +++ b/rules/rauc.make
> @@ -46,6 +46,7 @@ RAUC_CONF_OPT	:= \
>  	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
>  	--disable-create \
>  	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \
> +	--$(call ptx/endis,PTXCONF_RAUC_STREAMING)-streaming \
>  	--$(call ptx/endis,PTXCONF_RAUC_JSON)-json \
>  	--$(call ptx/endis,PTXCONF_RAUC_GPT)-gpt \
>  	--with-gcov=gcov \

Re: [ptxdist] [APPLIED] RAUC: leverage --disable-create option to shrink size for target RAUC

[Michael Olbrich]

Thanks, applied as c2ad273712cfb89c292dd5a72e8e1ee87a997ab7.

Michael

[sent from post-receive hook]

On Mon, 20 Jun 2022 08:19:59 +0200, Enrico Jorns <ejo@pengutronix.de> wrote:
> We do not need bundle creation capabilities on the target in most cases,
> thus make this optional (and disable by default).
> 
> Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> Message-Id: <20220615142217.990960-4-ejo@pengutronix.de>
> Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
> 
> diff --git a/rules/host-rauc.make b/rules/host-rauc.make
> index e5190bb8e60a..f0716da042b6 100644
> --- a/rules/host-rauc.make
> +++ b/rules/host-rauc.make
> @@ -27,6 +27,7 @@ HOST_RAUC_CONF_OPT	:= \
>  	--disable-Werror \
>  	--disable-code-coverage \
>  	--disable-valgrind \
> +	--enable-create \
>  	--disable-service \
>  	--enable-create \
>  	--disable-network \
> diff --git a/rules/rauc.in b/rules/rauc.in
> index c6c697df7e9d..fa24fa61d318 100644
> --- a/rules/rauc.in
> +++ b/rules/rauc.in
> @@ -27,6 +27,15 @@ menuconfig RAUC
>  
>  if RAUC
>  
> +config RAUC_CREATE
> +	bool
> +	prompt "Enable bundle creation support on target"
> +	help
> +	  Enables support for creating bundles on target.
> +	  Disabling this will remove functionality required for bundle
> +	  creation which is not required on the target in most cases and thus
> +	  allows to save space.
> +
>  config RAUC_SERVICE
>  	bool
>  	prompt "Enable service (D-Bus) support"
> diff --git a/rules/rauc.make b/rules/rauc.make
> index 78f6c894b73e..6a2d49343105 100644
> --- a/rules/rauc.make
> +++ b/rules/rauc.make
> @@ -43,6 +43,7 @@ RAUC_CONF_OPT	:= \
>  	--disable-Werror \
>  	--disable-code-coverage \
>  	--disable-valgrind \
> +	--$(call ptx/endis,PTXCONF_RAUC_CREATE)-create \
>  	--$(call ptx/endis,PTXCONF_RAUC_SERVICE)-service \
>  	--disable-create \
>  	--$(call ptx/endis,PTXCONF_RAUC_NETWORK)-network \

Re: [ptxdist] [APPLIED] image-rauc: add support for crypt bundles

[Michael Olbrich]

Thanks, applied as 5bb0db5266e4d087b98dd0ca39a5c62ad9fe7124.

Michael

[sent from post-receive hook]

On Mon, 20 Jun 2022 08:20:00 +0200, Enrico Jorns <ejo@pengutronix.de> wrote:
> Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> Message-Id: <20220615142217.990960-5-ejo@pengutronix.de>
> Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
> 
> diff --git a/platforms/image-rauc.in b/platforms/image-rauc.in
> index 3835e071892d..7772fd95b21b 100644
> --- a/platforms/image-rauc.in
> +++ b/platforms/image-rauc.in
> @@ -39,6 +39,14 @@ config IMAGE_RAUC_BUNDLE_FORMAT_VERITY
>  	help
>  	  The improved 'verity' bundle format
>  
> +config IMAGE_RAUC_BUNDLE_FORMAT_CRYPT
> +	bool "crypt"
> +	help
> +	  The 'crypt' bundle format for fully encryptable bundles
> +
> +	  Note that you must run 'rauc encrypt' on a crypt bundle to generate a
> +	  fully encrypted bundle.
> +
>  endchoice
>  
>  config IMAGE_RAUC_INTERMEDIATE
> diff --git a/rules/image-rauc.make b/rules/image-rauc.make
> index f7bed6e49d99..23d8d83b5251 100644
> --- a/rules/image-rauc.make
> +++ b/rules/image-rauc.make
> @@ -26,10 +26,20 @@ IMAGE_RAUC_CONFIG	:= rauc.config
>  
>  ifdef PTXCONF_IMAGE_RAUC
>  
> +ifdef PTXCONF_IMAGE_RAUC_BUNDLE_FORMAT_PLAIN
> +IMAGE_RAUC_BUNDLE_FORMAT := "plain"
> +endif
> +ifdef PTXCONF_IMAGE_RAUC_BUNDLE_FORMAT_VERITY
> +IMAGE_RAUC_BUNDLE_FORMAT := "verity"
> +endif
> +ifdef PTXCONF_IMAGE_RAUC_BUNDLE_FORMAT_CRYPT
> +IMAGE_RAUC_BUNDLE_FORMAT := "crypt"
> +endif
> +
>  IMAGE_RAUC_ENV	= \
>  	$(CODE_SIGNING_ENV) \
>  	RAUC_BUNDLE_COMPATIBLE="$(call remove_quotes,$(PTXCONF_RAUC_COMPATIBLE))" \
> -	RAUC_BUNDLE_FORMAT="$(call ptx/ifdef,PTXCONF_IMAGE_RAUC_BUNDLE_FORMAT_VERITY,verity,plain)" \
> +	RAUC_BUNDLE_FORMAT=$(IMAGE_RAUC_BUNDLE_FORMAT) \
>  	RAUC_BUNDLE_VERSION="$(call remove_quotes, $(PTXCONF_RAUC_BUNDLE_VERSION))" \
>  	RAUC_BUNDLE_BUILD=$(call ptx/sh, date +%FT%T%z) \
>  	RAUC_BUNDLE_DESCRIPTION=$(PTXCONF_IMAGE_RAUC_DESCRIPTION) \

Re: [ptxdist] [APPLIED] image-rauc: default to 'verity' bundle format

[Michael Olbrich]

Thanks, applied as 11abb1016c88d00337895925e1dee4bf383e3e5d.

Michael

[sent from post-receive hook]

On Mon, 20 Jun 2022 08:20:02 +0200, Enrico Jorns <ejo@pengutronix.de> wrote:
> The 'plain' format is legacy and 'crypt' must only be used when
> encryption is really required. Thus 'verity' is the default to choose.
> 
> Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
> Message-Id: <20220615142217.990960-6-ejo@pengutronix.de>
> Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
> 
> diff --git a/platforms/image-rauc.in b/platforms/image-rauc.in
> index 7772fd95b21b..391956a42f81 100644
> --- a/platforms/image-rauc.in
> +++ b/platforms/image-rauc.in
> @@ -28,6 +28,7 @@ config IMAGE_RAUC_DESCRIPTION
>  
>  choice
>  	prompt "Bundle format"
> +	default IMAGE_RAUC_BUNDLE_FORMAT_VERITY
>  
>  config IMAGE_RAUC_BUNDLE_FORMAT_PLAIN
>  	bool "plain"