[ptxdist] kernel prepare stage creates extra series file

[ptxdist] kernel prepare stage creates extra series file

[Alexander Dahl]

Hei hei,

I'm currently investigating a minor annoying issue. 

For our BSP we set PTXCONF_PLATFORM to something like 'at91foo'. Recently we 
added some kernel patches (we didn't have those before when using a pure 
vanilla kernel, this is no option anymore) and we put those to $
(PTXDIST_WORKSPACE)/patches (where our other patches are) and created a file 
'linux-4.9.47/series.at91foo' in the subdirectory matching the currently used 
kernel version. This works so far, build is successful, image is created and 
runs on target.

To those patches: we track our patches in a separate Git repository and use it 
as a Git submodule in more than one BSP, not below the 
PTXDIST_PLATFORMCONFIGDIR (where the platform specific linux patches maybe 
should go), but below PTXDIST_WORKSPACE, because it's a shared patches folder 
anyway. Maybe also not the best decision, but it's like that currently.

The linux kernel patch stack was created with 'git ptx-patches', but on a 
clean build I deactivated PTXCONF_SETUP_PATCHIN_GIT so patches are applied 
without git, this saves quite some time on extract stage.

Now after a `ptxdist clean` and a `ptxdist prepare kernel` a new 'series' file 
appears in $(PTXDIST_WORKSPACE)/patches/linux-4.9.47 and it's only named 
'series' and put besides the already present 'series.at91foo' which is tracked 
in Git. In the patches folder Git lists this as untracked file. In the parent 
folder, which is PTXDIST_WORKSPACE aka the BSP using the patches folder as a 
Git submodule and tracked in a different Git repo, on `git status` I see this:

	modified:   patches (untracked content)

And with `git describe --tags --dirty` this:

	v2017.03.0-275-gd8e18cb-dirty

We use this somewhere in our firmware. Now on a clean build, we must abort in 
between, remove this additional series file, and proceed the build to get a 
clean version string without '-dirty' added. :-(

From my intuition I would say ptxdist should not create this additional series 
file, or delete it again, if it's just temporary. I tried to find the spot in 
ptxdist sources, but up to now with no luck.

Is this a bug in ptxdist? Should we organize our BSPs and patches somehow 
different to not run into this? Are there any other possibilities or 
suggestions? This behaviour is problematic for fully automatic builds. :-/

Greets
Alex


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] kernel prepare stage creates extra series file

[Michael Olbrich]

Hi,

On Fri, Oct 13, 2017 at 01:53:13PM +0200, Alexander Dahl wrote:
> I'm currently investigating a minor annoying issue. 
> 
> For our BSP we set PTXCONF_PLATFORM to something like 'at91foo'. Recently we 
> added some kernel patches (we didn't have those before when using a pure 
> vanilla kernel, this is no option anymore) and we put those to $
> (PTXDIST_WORKSPACE)/patches (where our other patches are) and created a file 
> 'linux-4.9.47/series.at91foo' in the subdirectory matching the currently used 
> kernel version. This works so far, build is successful, image is created and 
> runs on target.
> 
> To those patches: we track our patches in a separate Git repository and use it 
> as a Git submodule in more than one BSP, not below the 
> PTXDIST_PLATFORMCONFIGDIR (where the platform specific linux patches maybe 
> should go), but below PTXDIST_WORKSPACE, because it's a shared patches folder 
> anyway. Maybe also not the best decision, but it's like that currently.
> 
> The linux kernel patch stack was created with 'git ptx-patches', but on a 
> clean build I deactivated PTXCONF_SETUP_PATCHIN_GIT so patches are applied 
> without git, this saves quite some time on extract stage.
> 
> Now after a `ptxdist clean` and a `ptxdist prepare kernel` a new 'series' file 
> appears in $(PTXDIST_WORKSPACE)/patches/linux-4.9.47 and it's only named 
> 'series' and put besides the already present 'series.at91foo' which is tracked 
> in Git. In the patches folder Git lists this as untracked file. In the parent 
> folder, which is PTXDIST_WORKSPACE aka the BSP using the patches folder as a 
> Git submodule and tracked in a different Git repo, on `git status` I see this:
> 
> 	modified:   patches (untracked content)
> 
> And with `git describe --tags --dirty` this:
> 
> 	v2017.03.0-275-gd8e18cb-dirty
> 
> We use this somewhere in our firmware. Now on a clean build, we must abort in 
> between, remove this additional series file, and proceed the build to get a 
> clean version string without '-dirty' added. :-(
> 
> From my intuition I would say ptxdist should not create this additional series 
> file, or delete it again, if it's just temporary. I tried to find the spot in 
> ptxdist sources, but up to now with no luck.
> 
> Is this a bug in ptxdist? Should we organize our BSPs and patches somehow 
> different to not run into this? Are there any other possibilities or 
> suggestions? This behaviour is problematic for fully automatic builds. :-/

The only place ptxdist creates a series file, is, when it cannot find one
(This is in scripts/lib/ptxd_make_world_patchin.sh line 282). In that case,
you should see a warning, when it is created. Maybe PTXCONF_KERNEL_SERIES
is not set correctly?

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] kernel prepare stage creates extra series file

[Alexander Dahl]

Hi,

(What I didn't mention before: the BSP is on ptxdist 2017.09.0.)

Am Montag, 16. Oktober 2017, 14:40:51 CEST schrieb Michael Olbrich:
> The only place ptxdist creates a series file, is, when it cannot find one
> (This is in scripts/lib/ptxd_make_world_patchin.sh line 282). In that case,
> you should see a warning, when it is created. Maybe PTXCONF_KERNEL_SERIES
> is not set correctly?

This is set to: series${PTXDIST_PLATFORMSUFFIX}

So:

	$ p print PTXDIST_PLATFORMSUFFIX 
	.at91foo
	$ p print PTXCONF_KERNEL_SERIES 
	"series.at91foo"
	$ ls -l patches/linux-4.9.47/series.at91foo 
	-rw-r--r-- 1 adahl adahl 18229 Sep 29 11:33 patches/linux-4.9.47/
series.at91foo

If I just drop the kernel prepare stage and execute it again, this 'series' 
file is not created.

But, what I found, after you mentioned this warning, which should come:

	-----------------------------
	target: kernel-header.extract
	-----------------------------

	extract: pkg_src=/home/adahl/[…]/linux-4.9.47.tar.xz
	extract: pkg_extract_dir=XXX/platform-at91foo/build-target/kernel-
header-4.9.47
	ptxdist: error: series file for 'XXX/patches/linux-4.9.47' is missing

There's no option to the set the series file for the kernel header package, 
like for the kernel package.

As a workaround: I looked which selects the kernel-header package here and it 
seems it is only oprofile, which I only need for debugging. So for now I 
deactivate the kernel header package, which circumvents my initial problem, 
this also saves quite some build time. :-)

Would still be interesting to look, why the kernel-header package ignores the 
right series file somehow, and creates a new one which applies the same 
patches.

Greets
Alex


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] kernel prepare stage creates extra series file

[Michael Olbrich]

Hi,

On Mon, Oct 16, 2017 at 03:26:35PM +0200, Alexander Dahl wrote:
> (What I didn't mention before: the BSP is on ptxdist 2017.09.0.)
> 
> Am Montag, 16. Oktober 2017, 14:40:51 CEST schrieb Michael Olbrich:
> > The only place ptxdist creates a series file, is, when it cannot find one
> > (This is in scripts/lib/ptxd_make_world_patchin.sh line 282). In that case,
> > you should see a warning, when it is created. Maybe PTXCONF_KERNEL_SERIES
> > is not set correctly?
> 
> This is set to: series${PTXDIST_PLATFORMSUFFIX}
> 
> So:
> 
> 	$ p print PTXDIST_PLATFORMSUFFIX 
> 	.at91foo
> 	$ p print PTXCONF_KERNEL_SERIES 
> 	"series.at91foo"
> 	$ ls -l patches/linux-4.9.47/series.at91foo 
> 	-rw-r--r-- 1 adahl adahl 18229 Sep 29 11:33 patches/linux-4.9.47/
> series.at91foo
> 
> If I just drop the kernel prepare stage and execute it again, this 'series' 
> file is not created.
> 
> But, what I found, after you mentioned this warning, which should come:
> 
> 	-----------------------------
> 	target: kernel-header.extract
> 	-----------------------------
> 
> 	extract: pkg_src=/home/adahl/[…]/linux-4.9.47.tar.xz
> 	extract: pkg_extract_dir=XXX/platform-at91foo/build-target/kernel-
> header-4.9.47
> 	ptxdist: error: series file for 'XXX/patches/linux-4.9.47' is missing
> 
> There's no option to the set the series file for the kernel header package, 
> like for the kernel package.
> 
> As a workaround: I looked which selects the kernel-header package here and it 
> seems it is only oprofile, which I only need for debugging. So for now I 
> deactivate the kernel header package, which circumvents my initial problem, 
> this also saves quite some build time. :-)
> 
> Would still be interesting to look, why the kernel-header package ignores the 
> right series file somehow, and creates a new one which applies the same 
> patches.

For the most part, the kernel-header is intended to provide new versions of
the kernel headers if those from the toolchain are too old. So it should be
mostly platform independent. That's why you can set the version separately
from the kernel.
The kernel ABI is pretty stable, so a older version than the kernel is
usually not a problem.

In your case I can see two solutions:
1. Move the patches to configs/platform-XXX/patches/linux-4.9.47/ and just
   use 'series'.
2. Use a slightly different version for the kernel-header package(e.g.
   4.9.46) so it wont share the patches. Unless your patches change
   something needed for oprofile?

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] kernel prepare stage creates extra series file

[Alexander Dahl]

Hi,

Am Montag, 16. Oktober 2017, 15:42:37 CEST schrieb Michael Olbrich:
> In your case I can see two solutions:
> 1. Move the patches to configs/platform-XXX/patches/linux-4.9.47/ and just
>    use 'series'.

Yes, I already knew that one.

> 2. Use a slightly different version for the kernel-header package(e.g.
>    4.9.46) so it wont share the patches. Unless your patches change
>    something needed for oprofile?

For now I deactivated oprofile. If I should ever need it again, I probably try 
4.9.0. However I just read oprofile doesn't work with the PREEMPT RT stack 
anyway, so profiling on the target won't work anymore on this target.

Thanks for your suggestions
Alex


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de