Re: [ptxdist] [PATCH 2/3] barebox: add option for building barebox reproducibly

[Michael Olbrich <m.olbrich@pengutronix.de>]

On Fri, Oct 12, 2018 at 12:21:14PM +0000, Baeuerle, Florian wrote:
> Am Freitag, den 12.10.2018, 14:01 +0200 schrieb Michael Olbrich:
> > On Thu, Oct 11, 2018 at 02:19:17PM +0000, Baeuerle, Florian wrote:
> > > Inject fake timestamps into the build via faketime to produce the exact
> > > same binaries.
> > > 
> > > Signed-off-by: Florian Bäuerle <florian.baeuerle@allegion.com>
> > > ---
> > >  platforms/barebox.in | 10 ++++++++++
> > >  rules/barebox.make   | 16 +++++++++++++++-
> > >  2 files changed, 25 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/platforms/barebox.in b/platforms/barebox.in
> > > index 645366e93..5bc2f08b6 100644
> > > --- a/platforms/barebox.in
> > > +++ b/platforms/barebox.in
> > > @@ -13,6 +13,7 @@ menuconfig BAREBOX
> > >  	select HOST_OPENSSL if BAREBOX_NEEDS_HOST_OPENSSL
> > >  	select HOST_IMX_CST if BAREBOX_NEEDS_HOST_IMX_CST
> > >  	select HOST_LZOP if BAREBOX_NEEDS_HOST_LZOP
> > > +	select HOST_FAKETIME if BAREBOX_REPRODUCIBLE
> > >  	prompt "barebox                       "
> > >  	bool
> > >  	help
> > > @@ -50,6 +51,15 @@ config BAREBOX_EXTRA_ENV_PATH
> > >  	  Directories and files to add to the builtin barebox environment.
> > >  	  Relative paths must be relative to the platformconfigdir.
> > >  
> > > +config BAREBOX_REPRODUCIBLE
> > > +	prompt "build reproducibly"
> > > +	bool
> > > +	default n
> > > +	help
> > > +	  With this, fake timestamps are injected to the barebox build, so the
> > > +	  resulting barebox images are identical when the sources haven't been
> > > +	  changed.
> > > +
> > >  config BAREBOX_BAREBOXENV
> > >  	prompt "install 'bareboxenv'"
> > >  	bool
> > > diff --git a/rules/barebox.make b/rules/barebox.make
> > > index a6ec3c394..ad160b3ba 100644
> > > --- a/rules/barebox.make
> > > +++ b/rules/barebox.make
> > > @@ -45,6 +45,20 @@ BAREBOX_ENV := \
> > >  	KCONFIG_NOTIMESTAMP=1 \
> > >  	pkg_wrapper_blacklist="$(BAREBOX_WRAPPER_BLACKLIST)"
> > >  
> > > +ifdef PTXCONF_BAREBOX_REPRODUCIBLE
> > > +BAREBOX_FAKE_TIMESTAMP := \
> > > +	$(shell echo $(BAREBOX_VERSION) | \
> > > +		sed -E "s/([0-9]{4})\.([0-9]{1,2}).*/\1-\2-01 00:00:00/")
> > > +BAREBOX_FAKETIME := faketime -f "$(BAREBOX_FAKE_TIMESTAMP)"
> > > +BAREBOX_ENV := $(BAREBOX_ENV) \
> > > +	KBUILD_BUILD_TIMESTAMP="$(BAREBOX_FAKE_TIMESTAMP)" \
> > > +	KBUILD_BUILD_VERSION="0" \
> > > +	KBUILD_BUILD_USER="ptxdist" \
> > > +	KBUILD_BUILD_HOST="ptxdist"
> > 
> > What is not covered by the KBUILD_BUILD_* variables? faketime is a big
> > hammer that can cause other problems. I'd like to understand what happens
> > here first.
> 
> According to my observations, only the compression. A few compressors are
> capable of not including timestamps (e.g. gzip: --no-name), but others are not
> (lzop).
> 
> To fix that, there are a few options I do not like:
> 
> 1. fix the in the barebox build system and implement ugly hacks so this works
> for lzop (overwriting an lzo header field) and other compressors that do not
> support this out of the box.
> 
> 2. fix this in ptxdist, probably by doing the same ugly stuff before copying
> images to IMAGEDIR
> 
> 3. fix the problems in lzop and other compressors that do not support faking or
> skipping those header fields.
> 
> The thirds option is still the best, but it will take ages to reach
> distributions.

I've looked at this a bit. For gzip, barebox already uses '-n', so that
should work correctly. Most other compressors don't add a timestamp, at
least in my quick tests. What remains is lzop.

We already have a BAREBOX_NEEDS_HOST_LZOP option, so the only thing needed
is a patch. Using SOURCE_DATE_EPOCH for mtime_low/mtime_high if available
(instead of the file time) should be trivial.

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de