[ptxdist] [PATCH] ptxd_make_fit_image: pad FIT image to block size of 4096 bytes

[ptxdist] [PATCH] ptxd_make_fit_image: pad FIT image to block size of 4096 bytes

[Roland Hieber]

Trying to flash a FIT image over fastboot often results in errors like
this:

    Invalid sparse file format at header magic
    error: write_sparse_skip_chunk: don't care size 18394488 is not a multiple of the block size 4096
    error: write_sparse_skip_chunk: don't care size 10009976 is not a multiple of the block size 4096
    error: write_sparse_skip_chunk: don't care size 1625464 is not a multiple of the block size 4096
    [...]
    FAILED (remote: 'writing sparse image: Invalid argument')

The FIT image knows about its own size anyway, so to get around this
issue, we can simply pad the image to a multiple of 4 kiB to make it
work with the fastboot sparse magic.

Suggested-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
Signed-off-by: Roland Hieber <rhi@pengutronix.de>
---
 scripts/lib/ptxd_make_fit_image.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/ptxd_make_fit_image.sh b/scripts/lib/ptxd_make_fit_image.sh
index dd0f63b7b72d..660dd236f241 100644
--- a/scripts/lib/ptxd_make_fit_image.sh
+++ b/scripts/lib/ptxd_make_fit_image.sh
@@ -139,6 +139,7 @@ ptxd_make_image_fit() {
 	echo "Generated device-tree for the fit image:"
 	cat "${its}"
     fi &&
-    ptxd_exec mkimage -N pkcs11 -f "${its}" "${image_image}" -r "${sign_args[@]}"
+    ptxd_exec mkimage -N pkcs11 -f "${its}" "${image_image}.tmp" -r "${sign_args[@]}" &&
+    ptxd_exec dd if="${image_image}.tmp" of="${image_image}" conv=sync bs=4k
 }
 export -f ptxd_make_image_fit
-- 
2.39.2

[ptxdist] [PATCH v2] ptxd_make_fit_image: pad FIT image to block size of 4096 bytes

[Roland Hieber]

Trying to flash a FIT image over fastboot often results in errors like
this:

    Invalid sparse file format at header magic
    error: write_sparse_skip_chunk: don't care size 18394488 is not a multiple of the block size 4096
    error: write_sparse_skip_chunk: don't care size 10009976 is not a multiple of the block size 4096
    error: write_sparse_skip_chunk: don't care size 1625464 is not a multiple of the block size 4096
    [...]
    FAILED (remote: 'writing sparse image: Invalid argument')

The FIT image knows about its own size anyway, so to get around this
issue, we can simply pad the image to a multiple of 4 kiB to make it
work with the fastboot sparse magic.

Suggested-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
Signed-off-by: Roland Hieber <rhi@pengutronix.de>
---
v2: clean up the temp image too
---
 scripts/lib/ptxd_make_fit_image.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/lib/ptxd_make_fit_image.sh b/scripts/lib/ptxd_make_fit_image.sh
index dd0f63b7b72d..8162195fdfa3 100644
--- a/scripts/lib/ptxd_make_fit_image.sh
+++ b/scripts/lib/ptxd_make_fit_image.sh
@@ -139,6 +139,8 @@ ptxd_make_image_fit() {
 	echo "Generated device-tree for the fit image:"
 	cat "${its}"
     fi &&
-    ptxd_exec mkimage -N pkcs11 -f "${its}" "${image_image}" -r "${sign_args[@]}"
+    ptxd_exec mkimage -N pkcs11 -f "${its}" "${image_image}.tmp" -r "${sign_args[@]}" &&
+    ptxd_exec dd if="${image_image}.tmp" of="${image_image}" conv=sync bs=4k &&
+    ptxd_exec rm "${image_image}.tmp"
 }
 export -f ptxd_make_image_fit
-- 
2.39.2

Re: [ptxdist] [PATCH v2] ptxd_make_fit_image: pad FIT image to block size of 4096 bytes

[Roland Hieber]

On Wed, Aug 23, 2023 at 12:49:59PM +0200, Roland Hieber wrote:
> Trying to flash a FIT image over fastboot often results in errors like
> this:
> 
>     Invalid sparse file format at header magic
>     error: write_sparse_skip_chunk: don't care size 18394488 is not a multiple of the block size 4096
>     error: write_sparse_skip_chunk: don't care size 10009976 is not a multiple of the block size 4096
>     error: write_sparse_skip_chunk: don't care size 1625464 is not a multiple of the block size 4096
>     [...]
>     FAILED (remote: 'writing sparse image: Invalid argument')
> 
> The FIT image knows about its own size anyway, so to get around this
> issue, we can simply pad the image to a multiple of 4 kiB to make it
> work with the fastboot sparse magic.
> 
> Suggested-by: Ahmad Fatoum <a.fatoum@pengutronix.de>

For reference, Ahmad also posted a more detailed explanation and a patch
addressing the situation on the barebox list:
<https://lore.barebox.org/barebox/20230825172250.2857448-1-a.fatoum@pengutronix.de/>

(However the problem still persists on other bootloaders.)

 - Roland

> Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> ---
> v2: clean up the temp image too
> ---
>  scripts/lib/ptxd_make_fit_image.sh | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/scripts/lib/ptxd_make_fit_image.sh b/scripts/lib/ptxd_make_fit_image.sh
> index dd0f63b7b72d..8162195fdfa3 100644
> --- a/scripts/lib/ptxd_make_fit_image.sh
> +++ b/scripts/lib/ptxd_make_fit_image.sh
> @@ -139,6 +139,8 @@ ptxd_make_image_fit() {
>  	echo "Generated device-tree for the fit image:"
>  	cat "${its}"
>      fi &&
> -    ptxd_exec mkimage -N pkcs11 -f "${its}" "${image_image}" -r "${sign_args[@]}"
> +    ptxd_exec mkimage -N pkcs11 -f "${its}" "${image_image}.tmp" -r "${sign_args[@]}" &&
> +    ptxd_exec dd if="${image_image}.tmp" of="${image_image}" conv=sync bs=4k &&
> +    ptxd_exec rm "${image_image}.tmp"
>  }
>  export -f ptxd_make_image_fit
> -- 
> 2.39.2
> 
> 
> 

-- 
Roland Hieber, Pengutronix e.K.          | r.hieber@pengutronix.de     |
Steuerwalder Str. 21                     | https://www.pengutronix.de/ |
31137 Hildesheim, Germany                | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686         | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] [PATCH v2] ptxd_make_fit_image: pad FIT image to block size of 4096 bytes

[Michael Olbrich]

On Wed, Aug 23, 2023 at 12:49:59PM +0200, Roland Hieber wrote:
> Trying to flash a FIT image over fastboot often results in errors like
> this:
> 
>     Invalid sparse file format at header magic
>     error: write_sparse_skip_chunk: don't care size 18394488 is not a multiple of the block size 4096
>     error: write_sparse_skip_chunk: don't care size 10009976 is not a multiple of the block size 4096
>     error: write_sparse_skip_chunk: don't care size 1625464 is not a multiple of the block size 4096
>     [...]
>     FAILED (remote: 'writing sparse image: Invalid argument')
> 
> The FIT image knows about its own size anyway, so to get around this
> issue, we can simply pad the image to a multiple of 4 kiB to make it
> work with the fastboot sparse magic.

This looks suspicious. The FIT image is not a sparse file, so why would you
get errors from the sparse file handling code?

Michael

> 
> Suggested-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
> Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> ---
> v2: clean up the temp image too
> ---
>  scripts/lib/ptxd_make_fit_image.sh | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/scripts/lib/ptxd_make_fit_image.sh b/scripts/lib/ptxd_make_fit_image.sh
> index dd0f63b7b72d..8162195fdfa3 100644
> --- a/scripts/lib/ptxd_make_fit_image.sh
> +++ b/scripts/lib/ptxd_make_fit_image.sh
> @@ -139,6 +139,8 @@ ptxd_make_image_fit() {
>  	echo "Generated device-tree for the fit image:"
>  	cat "${its}"
>      fi &&
> -    ptxd_exec mkimage -N pkcs11 -f "${its}" "${image_image}" -r "${sign_args[@]}"
> +    ptxd_exec mkimage -N pkcs11 -f "${its}" "${image_image}.tmp" -r "${sign_args[@]}" &&
> +    ptxd_exec dd if="${image_image}.tmp" of="${image_image}" conv=sync bs=4k &&
> +    ptxd_exec rm "${image_image}.tmp"
>  }
>  export -f ptxd_make_image_fit
> -- 
> 2.39.2
> 
> 
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] [PATCH v2] ptxd_make_fit_image: pad FIT image to block size of 4096 bytes

[Roland Hieber]

On Fri, Sep 01, 2023 at 08:31:58AM +0200, Michael Olbrich wrote:
> On Wed, Aug 23, 2023 at 12:49:59PM +0200, Roland Hieber wrote:
> > Trying to flash a FIT image over fastboot often results in errors like
> > this:
> > 
> >     Invalid sparse file format at header magic
> >     error: write_sparse_skip_chunk: don't care size 18394488 is not a multiple of the block size 4096
> >     error: write_sparse_skip_chunk: don't care size 10009976 is not a multiple of the block size 4096
> >     error: write_sparse_skip_chunk: don't care size 1625464 is not a multiple of the block size 4096
> >     [...]
> >     FAILED (remote: 'writing sparse image: Invalid argument')
> > 
> > The FIT image knows about its own size anyway, so to get around this
> > issue, we can simply pad the image to a multiple of 4 kiB to make it
> > work with the fastboot sparse magic.
> 
> This looks suspicious. The FIT image is not a sparse file, so why would you
> get errors from the sparse file handling code?

Sorry, I have no idea. As far as I understood Ahmad's barebox patch,
fastboot tries to sparse it on-the-fly…?

 - Roland

> Michael
> 
> > 
> > Suggested-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
> > Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> > ---
> > v2: clean up the temp image too
> > ---
> >  scripts/lib/ptxd_make_fit_image.sh | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> > 
> > diff --git a/scripts/lib/ptxd_make_fit_image.sh b/scripts/lib/ptxd_make_fit_image.sh
> > index dd0f63b7b72d..8162195fdfa3 100644
> > --- a/scripts/lib/ptxd_make_fit_image.sh
> > +++ b/scripts/lib/ptxd_make_fit_image.sh
> > @@ -139,6 +139,8 @@ ptxd_make_image_fit() {
> >  	echo "Generated device-tree for the fit image:"
> >  	cat "${its}"
> >      fi &&
> > -    ptxd_exec mkimage -N pkcs11 -f "${its}" "${image_image}" -r "${sign_args[@]}"
> > +    ptxd_exec mkimage -N pkcs11 -f "${its}" "${image_image}.tmp" -r "${sign_args[@]}" &&
> > +    ptxd_exec dd if="${image_image}.tmp" of="${image_image}" conv=sync bs=4k &&
> > +    ptxd_exec rm "${image_image}.tmp"
> >  }
> >  export -f ptxd_make_image_fit
> > -- 
> > 2.39.2
> > 
> > 
> > 
> 
> -- 
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> 
> 

-- 
Roland Hieber, Pengutronix e.K.          | r.hieber@pengutronix.de     |
Steuerwalder Str. 21                     | https://www.pengutronix.de/ |
31137 Hildesheim, Germany                | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686         | Fax:   +49-5121-206917-5555 |