[ptxdist] PTXdist usescases

[ptxdist] PTXdist usescases

[Erwin Rol]

Hello all,

as a long time ptxdist users I ended up with some (weird) ptxdist uses
cases, some of wich simply are from before ptxdist offered something
similar, some others I am not so sure if they are possible with "plain"
ptxdist. 

So I thought to ask here to see if others have similar uses cases and
how they solved them.

To not get a huge mail with 1000 and 1 topics I start with my most
important uses case, that is to be able to build separate (squashfs)
images and also version (via git) those images. 

For example I build a rootfs image with things like webkit and Qt5
(both take serious amounts of CPU power and time to build) and version
that. From ptxdist point of view that is a normal build. 

But than I use the sysroot-* dirs inside a second ptxdist projects to
build an "appfs". That only has all software written by my client.
Those client packages have no ptxdist-DEPS on ptxdist packages like Qt,
because we know for sure those packages are already build because we
first build the rootfs.

This means we end up with two (git versioned) images that will be
overlaymounted to give the full/final "root" filesystem on the target.

There are several reasons why I started doing this. 

- Build times, the rootfs doesn't have to be rebuild when client
software is changed.
- Target software update images are smaller when only client software
(appfs) has to be updated (was important in a time with expensive GSM
modems, and it is still useful to reduce in field update times)
- Reuse, the rootfs kan be reused for different projects. 

It is a bit like the yocto layer system, just that the (two) layers are
build separately and give their own filesystem images. 

It has some problems (where acceptable at the time);
- copying the systemroot-* dirs is a bad hack, and still needs the
original systemroot-* dirs because of all the absolute paths in the
files. 
- the host tools (like cmake) are build twice (in rootfs and appfs)
which also takes a considerable amount of time. 

So is there someone that has similar uses cases and if yes how did you
solve it?

And would it be an interesting feature for upstream ptxdist (not my
implementation) ? 

- Erwin



_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Robert Schwebel]

Hi Erwin,

On Mon, Aug 13, 2018 at 09:28:09AM +0200, Erwin Rol wrote:
> [layer system]
> 
> So is there someone that has similar uses cases and if yes how did you
> solve it?
> 
> And would it be an interesting feature for upstream ptxdist (not my
> implementation) ? 

Michael Olbrich prototyped a layering system for ptxdist¹ just before
leaving for vacation; we are currently testing the prototype and mol
will post it here on the mailing list afterwards.

My impression is that it will be able to solve your usecases.

rsc

¹ Of course better² than yocto's layering system!
² For whatever value of "better" ... :-)
-- 
Pengutronix e.K.                           | Dipl.-Ing. Robert Schwebel  |
Industrial Linux Solutions                 | https://www.pengutronix.de/ |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Erwin Rol]

On Mon, 2018-08-13 at 12:19 +0200, Robert Schwebel wrote:

> Michael Olbrich prototyped a layering system for ptxdist¹ just before
> leaving for vacation; 

So who allowed him to go on vacation ? ;-P 

> we are currently testing the prototype and mol
> will post it here on the mailing list afterwards.
> 
> My impression is that it will be able to solve your usecases.
> 

OK sounds interesting, I'll keep an eye out for the mail.

- Erwin
 


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Robert Schwebel]

On Mon, Aug 13, 2018 at 02:16:20PM +0200, Erwin Rol wrote:
> > Michael Olbrich prototyped a layering system for ptxdist¹ just before
> > leaving for vacation; 
> 
> So who allowed him to go on vacation ? ;-P 

I don't know :-)
 
> > we are currently testing the prototype and mol
> > will post it here on the mailing list afterwards.
> > 
> > My impression is that it will be able to solve your usecases.
> 
> OK sounds interesting, I'll keep an eye out for the mail.

Yup.

rsc
-- 
Pengutronix e.K.                           | Dipl.-Ing. Robert Schwebel  |
Industrial Linux Solutions                 | https://www.pengutronix.de/ |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Michael Olbrich]

On Mon, Aug 13, 2018 at 02:40:14PM +0200, Robert Schwebel wrote:
> On Mon, Aug 13, 2018 at 02:16:20PM +0200, Erwin Rol wrote:
> > > Michael Olbrich prototyped a layering system for ptxdist¹ just before
> > > leaving for vacation; 
> > 
> > So who allowed him to go on vacation ? ;-P 
> 
> I don't know :-)

:-)

> > > we are currently testing the prototype and mol
> > > will post it here on the mailing list afterwards.
> > > 
> > > My impression is that it will be able to solve your usecases.
> > 
> > OK sounds interesting, I'll keep an eye out for the mail.

The dev packages[1] might be interresting for you too. They should be
relocatable.

Michael

[1] https://www.ptxdist.org/doc/daily_work_section.html?highlight=dev#using-pre-built-archives

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Erwin Rol]

Hey Michael,

On Mon, 2018-08-27 at 18:40 +0200, Michael Olbrich wrote:
> On Mon, Aug 13, 2018 at 02:40:14PM +0200, Robert Schwebel wrote:
> > On Mon, Aug 13, 2018 at 02:16:20PM +0200, Erwin Rol wrote:
> > > > Michael Olbrich prototyped a layering system for ptxdist
> > > > we are currently testing the prototype and mol
> > > > will post it here on the mailing list afterwards.
> > > > 

What I am really looking for is to be able to make two (or more)
filesystem images that I can build (and version) independently, and
where the next "layer" does not have to rebuild the previous layer.

For example;

- A rootfs layer that generates a rootfs image, which holds the minimal
setup for the hardware (bash, systemd, etc).

- A qt5fs, which holds QT5 and gui stuff, but that does not need to
build the stuff that is already in the rootfs image (like bash and
systemd).

- A appfs that holds own applications that use QT5 but again without
building qt5fs/rootfs.

On the target those 3 filesystems will than overlay mounted so
everything is at its expected place.

After rootfs and qt5fs are versioned and build, the appfs can be
quickly changed with minimal build time, and it can also be completely
different for example two different projects that use the same base
hardware. Than you just have to version and manage 1 rootfs/qt5fs and 2
appfs projects. 

That is different from the yocto layer system where you build up your
one-and-only rootfs out of different layers. The caching might reduce
build times but the versioning is still a problem.

Also the updating of the target is easier when only a 4MB appfs has to
be distributed than when 250MB rootfs has to be distributed. 

Robert mentioned you where working on some sort of layering system, and
I was hoping I maybe could use that as a base :-)

- Erwin


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Baeuerle, Florian]

Am Dienstag, den 28.08.2018, 17:27 +0200 schrieb Erwin Rol:
> Hey Michael,
> 
> On Mon, 2018-08-27 at 18:40 +0200, Michael Olbrich wrote:
> > On Mon, Aug 13, 2018 at 02:40:14PM +0200, Robert Schwebel wrote:
> > > On Mon, Aug 13, 2018 at 02:16:20PM +0200, Erwin Rol wrote:
> > > > > Michael Olbrich prototyped a layering system for ptxdist
> > > > > we are currently testing the prototype and mol
> > > > > will post it here on the mailing list afterwards.
> > > > > 
> 
> What I am really looking for is to be able to make two (or more)
> filesystem images that I can build (and version) independently, and
> where the next "layer" does not have to rebuild the previous layer.
> 
> For example;
> 
> - A rootfs layer that generates a rootfs image, which holds the minimal
> setup for the hardware (bash, systemd, etc).
> 
> - A qt5fs, which holds QT5 and gui stuff, but that does not need to
> build the stuff that is already in the rootfs image (like bash and
> systemd).
> 
> - A appfs that holds own applications that use QT5 but again without
> building qt5fs/rootfs.

The advantages of such a scheme would be smaller updates.

On the other hand, you have to keep the upper layers (rootfs and qt5fs)
stable. This would probably lead to a reduced frequency of updates in those
layers.

Anyway, this might actually be possible with image packages[1], if you set
IMAGE_XPKG_EXTRA_ARGS to "--nodeps" (or find a different way to pass that
option to opkg). You can basically build an image with just the Qt5-package.

But then you'd have to make sure on your own, that the rootfs-image contains
all the dependencies required. Otherwise, you could pick Qt5s dependencies that
are not already included in the rootfs-image. This is where it gets hairy I
think.


I personally prefer just having one rootfs, because its simple and robust.
The problem is the size of updates, but the largest project's rootfs is only
roughly 50 MiB (including qt5, noto fonts and application). Other projects'
rootfs images are below 20 MiB (includes the kernel).

You can just update ptxdist and get new versions of everything without
thinking about it too much.

Regarding build times: You usually only build the whole thing once, and
then just recompile only the application anyway.
In our CI, a full build from scratch takes roughly 8 minutes for the largest
rootfs image.


1: https://www.ptxdist.org/doc/ref_manual.html#image-packages


Regards
Florian
_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Michael Olbrich]

Hi,

On Wed, Aug 29, 2018 at 08:14:43AM +0000, Baeuerle, Florian wrote:
> Am Dienstag, den 28.08.2018, 17:27 +0200 schrieb Erwin Rol:
> > On Mon, 2018-08-27 at 18:40 +0200, Michael Olbrich wrote:
> > > On Mon, Aug 13, 2018 at 02:40:14PM +0200, Robert Schwebel wrote:
> > > > On Mon, Aug 13, 2018 at 02:16:20PM +0200, Erwin Rol wrote:
> > > > > > Michael Olbrich prototyped a layering system for ptxdist
> > > > > > we are currently testing the prototype and mol
> > > > > > will post it here on the mailing list afterwards.
> > > > > > 
> > 
> > What I am really looking for is to be able to make two (or more)
> > filesystem images that I can build (and version) independently, and
> > where the next "layer" does not have to rebuild the previous layer.

I'm not sure what you mean with 'version'.

> > For example;
> > 
> > - A rootfs layer that generates a rootfs image, which holds the minimal
> > setup for the hardware (bash, systemd, etc).
> > 
> > - A qt5fs, which holds QT5 and gui stuff, but that does not need to
> > build the stuff that is already in the rootfs image (like bash and
> > systemd).
> > 
> > - A appfs that holds own applications that use QT5 but again without
> > building qt5fs/rootfs.
> 
> The advantages of such a scheme would be smaller updates.
> 
> On the other hand, you have to keep the upper layers (rootfs and qt5fs)
> stable. This would probably lead to a reduced frequency of updates in those
> layers.
> 
> Anyway, this might actually be possible with image packages[1], if you set
> IMAGE_XPKG_EXTRA_ARGS to "--nodeps" (or find a different way to pass that
> option to opkg). You can basically build an image with just the Qt5-package.

I think we could add a image specific variable similar to IMAGE_XPKG_EXTRA_ARGS.

> But then you'd have to make sure on your own, that the rootfs-image contains
> all the dependencies required. Otherwise, you could pick Qt5s dependencies that
> are not already included in the rootfs-image. This is where it gets hairy I
> think.

I think this can be done with collections:

- ptxconfig with all packages
- one collection with everything base + qt
- one collection with base

This will take care of the dependencies.

The base image uses
IMAGE_BASE_PKGS = $(call ptx/collection, /path/to/base_collection)

The Qt image uses
IMAGE_QT_PKGS = $(filter-out $(IMAGE_BASE_PKGS),$(call ptx/collection, /path/to/qt_collection))

And the app image uses
IMAGE_APP_PKGS = $(filter-out $(IMAGE_QT_PKGS) $(IMAGE_BASE_PKGS),$(PTX_PACKAGES_INSTALL))

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Baeuerle, Florian]

Hi,

the way we use ptxdist might be interesting for you as well.

We use one ptxdist directory with two platforms and three applications,
let's call the platforms X and Y, and the Applications A, B and C

Application A with two variants on one platform
 - release [platform X]
 - devel [platform X]
Application B with four variants on two platforms
 - variant-1-release [platform X, Y]
 - variant-1-devel [platform X, Y]
 - variant-2-release [platform X, Y]
 - variant-2-devel [platform X, Y]
Application C with two variants on one platform
 - release [platform X]
 - devel [platform X]

The devel variant is release + gdb, a few other tools and root login.

Application A, B, C each have their own ptxconfig. Each variant has its
collectionconfig. We build the images (squashfs + rauc bundles) with 
image packages under each platform directory.

so for building all images for Application B on Platform X, we do this:

ptxdist images \
    --ptxconfig=configs/app-B/ptxconfig \
    --platformconfig=configs/plat-X/platformconfig

The platformconfig for Platform X has all image packages enabled, but of
course we do not want to build image-platform-X-app-A with app-B's ptxconfig,
so we put some ifs in the image package makefile, where we check for the
currently selected PTXCONF_PROJECT.

What ptxdist misses for our use-case is some sort of collection-like feature
for platform-specific things (such as image packages, or platform-specific
graphics drivers). With the image packages, we use that workaround, but there
could be other use-cases.

Let's say I want to build App D on two platforms, each requiring different
graphic drivers (SGX is a nightmare), I wouldn't really know how to do that
properly.

The problem otherwise could be solved by using one ptxconfig that selects many
many packages as module. That, however, turned out to be a bit too complex,
because there are quite a few differences between the dependencies of App A, B
and C. Further, there are things that are bool instead of tristate that you
might not need/want.


Best Regards
Florian

Am Montag, den 13.08.2018, 09:28 +0200 schrieb Erwin Rol:
> Hello all,
> 
> as a long time ptxdist users I ended up with some (weird) ptxdist uses
> cases, some of wich simply are from before ptxdist offered something
> similar, some others I am not so sure if they are possible with "plain"
> ptxdist. 
> 
> So I thought to ask here to see if others have similar uses cases and
> how they solved them.
> 
> To not get a huge mail with 1000 and 1 topics I start with my most
> important uses case, that is to be able to build separate (squashfs)
> images and also version (via git) those images. 
> 
> For example I build a rootfs image with things like webkit and Qt5
> (both take serious amounts of CPU power and time to build) and version
> that. From ptxdist point of view that is a normal build. 
> 
> But than I use the sysroot-* dirs inside a second ptxdist projects to
> build an "appfs". That only has all software written by my client.
> Those client packages have no ptxdist-DEPS on ptxdist packages like Qt,
> because we know for sure those packages are already build because we
> first build the rootfs.
> 
> This means we end up with two (git versioned) images that will be
> overlaymounted to give the full/final "root" filesystem on the target.
> 
> There are several reasons why I started doing this. 
> 
> - Build times, the rootfs doesn't have to be rebuild when client
> software is changed.
> - Target software update images are smaller when only client software
> (appfs) has to be updated (was important in a time with expensive GSM
> modems, and it is still useful to reduce in field update times)
> - Reuse, the rootfs kan be reused for different projects. 
> 
> It is a bit like the yocto layer system, just that the (two) layers are
> build separately and give their own filesystem images. 
> 
> It has some problems (where acceptable at the time);
> - copying the systemroot-* dirs is a bad hack, and still needs the
> original systemroot-* dirs because of all the absolute paths in the
> files. 
> - the host tools (like cmake) are build twice (in rootfs and appfs)
> which also takes a considerable amount of time. 
> 
> So is there someone that has similar uses cases and if yes how did you
> solve it?
> 
> And would it be an interesting feature for upstream ptxdist (not my
> implementation) ? 
> 
> - Erwin
> 
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Michael Olbrich]

Hi,

On Tue, Aug 28, 2018 at 10:07:45AM +0000, Baeuerle, Florian wrote:
> the way we use ptxdist might be interesting for you as well.
> 
> We use one ptxdist directory with two platforms and three applications,
> let's call the platforms X and Y, and the Applications A, B and C
> 
> Application A with two variants on one platform
>  - release [platform X]
>  - devel [platform X]
> Application B with four variants on two platforms
>  - variant-1-release [platform X, Y]
>  - variant-1-devel [platform X, Y]
>  - variant-2-release [platform X, Y]
>  - variant-2-devel [platform X, Y]
> Application C with two variants on one platform
>  - release [platform X]
>  - devel [platform X]
> 
> The devel variant is release + gdb, a few other tools and root login.
> 
> Application A, B, C each have their own ptxconfig. Each variant has its
> collectionconfig. We build the images (squashfs + rauc bundles) with 
> image packages under each platform directory.
> 
> so for building all images for Application B on Platform X, we do this:
> 
> ptxdist images \
>     --ptxconfig=configs/app-B/ptxconfig \
>     --platformconfig=configs/plat-X/platformconfig
> 
> The platformconfig for Platform X has all image packages enabled, but of
> course we do not want to build image-platform-X-app-A with app-B's ptxconfig,
> so we put some ifs in the image package makefile, where we check for the
> currently selected PTXCONF_PROJECT.
> 
> What ptxdist misses for our use-case is some sort of collection-like feature
> for platform-specific things (such as image packages, or platform-specific
> graphics drivers). With the image packages, we use that workaround, but there
> could be other use-cases.

The layer stuff I'm working on may help here.

The idea is to basically stack multiple BSPs. Files and rules are searched
last to first much like we do this with BSP/platform/PTXdist already.

ptxconfig/platformconfig/kernelconfig etc. in the later 'BSPs' are
more or less a 'diff' to the same config on the next layer.

> Let's say I want to build App D on two platforms, each requiring different
> graphic drivers (SGX is a nightmare), I wouldn't really know how to do that
> properly.

Hmm, we have something like this already: The 'platform-opengl' package
defaults to Mesa, but you can a platform specific rule in each platform. I
used this in the past for Mesa vs. Vivante.

> The problem otherwise could be solved by using one ptxconfig that selects many
> many packages as module. That, however, turned out to be a bit too complex,
> because there are quite a few differences between the dependencies of App A, B
> and C. Further, there are things that are bool instead of tristate that you
> might not need/want.

The layers might help here too.

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Baeuerle, Florian]

Hello Michael,

Am Mittwoch, den 29.08.2018, 16:48 +0200 schrieb Michael Olbrich:
> Hi,
> 
> On Tue, Aug 28, 2018 at 10:07:45AM +0000, Baeuerle, Florian wrote:
> > the way we use ptxdist might be interesting for you as well.
> > 
> > We use one ptxdist directory with two platforms and three applications,
> > let's call the platforms X and Y, and the Applications A, B and C
> > 
> > Application A with two variants on one platform
> >  - release [platform X]
> >  - devel [platform X]
> > Application B with four variants on two platforms
> >  - variant-1-release [platform X, Y]
> >  - variant-1-devel [platform X, Y]
> >  - variant-2-release [platform X, Y]
> >  - variant-2-devel [platform X, Y]
> > Application C with two variants on one platform
> >  - release [platform X]
> >  - devel [platform X]
> > 
> > The devel variant is release + gdb, a few other tools and root login.
> > 
> > Application A, B, C each have their own ptxconfig. Each variant has its
> > collectionconfig. We build the images (squashfs + rauc bundles) with 
> > image packages under each platform directory.
> > 
> > so for building all images for Application B on Platform X, we do this:
> > 
> > ptxdist images \
> >     --ptxconfig=configs/app-B/ptxconfig \
> >     --platformconfig=configs/plat-X/platformconfig
> > 
> > The platformconfig for Platform X has all image packages enabled, but of
> > course we do not want to build image-platform-X-app-A with app-B's ptxconfig,
> > so we put some ifs in the image package makefile, where we check for the
> > currently selected PTXCONF_PROJECT.
> > 
> > What ptxdist misses for our use-case is some sort of collection-like feature
> > for platform-specific things (such as image packages, or platform-specific
> > graphics drivers). With the image packages, we use that workaround, but there
> > could be other use-cases.
> 
> The layer stuff I'm working on may help here.
> 
> The idea is to basically stack multiple BSPs. Files and rules are searched
> last to first much like we do this with BSP/platform/PTXdist already.
> 
> ptxconfig/platformconfig/kernelconfig etc. in the later 'BSPs' are
> more or less a 'diff' to the same config on the next layer.


Sounds interesting, even though I think I do not yet properly understand how it
will work. Is it really a diff, or more like an extension? Can I add additional
packages to an upper layer or can I also deselect them?

+-------------------------+-------------------------+ 
|                         |                         |
|       Platform A        |      Platform B         |
|                         |                         |
+-----------------+-------+-------+-----------------+ 
|                 |               |                 | <- ptxconfig for each
|      App A      |     App B     |      App C      |    application
|                 |               |                 |
|·················+···············+·················+
|                 ·               ·                 | <- not really shared, even 
|       Common packages selected in ptxconfig       |    though the packages are
|                 ·               ·                 |    supposed to be 95% the same
+--------+--------+---+---+---+---+--------+--------+
|        |        | 1 | 1 | 2 | 2 |        |        | <- collections for a bunch of
|  dev   |  rls   | d | r | d | r |  dev   |  rls   |    variants
|        |        | e | l | e | l |        |        |
|        |        | v | s | v | s |        |        |
+--------+--------+---+---+---+---+--------+--------+

I would be happy if I could actually share the third layer in above ASCII drawing.

That basically is the common stuff that should end up on all images (systemd, busybox
configuration, etc.). It's supposed to be the same everywhere, but they cannot be
trivially kept in sync.

> 
> > Let's say I want to build App D on two platforms, each requiring different
> > graphic drivers (SGX is a nightmare), I wouldn't really know how to do that
> > properly.
> 
> Hmm, we have something like this already: The 'platform-opengl' package
> defaults to Mesa, but you can a platform specific rule in each platform. I
> used this in the past for Mesa vs. Vivante.

That would, however, only work if it would be one package?
For am335x with gpu, I need three packages: the OOT kernel module, a franken-patched
libgbm and the blobs (libEGL, GLES, libpvr*, etc.).

When i override platform-opengl through configs/platform-a, then these three packages'
symbols would end up in my ptxconfig, so my ptxconfig is more or less tied to my platform.

Besides: is there a way to just "by default" select and install a package for a given
platform (without having the symbol in the ptxconfig)?

> 
> > The problem otherwise could be solved by using one ptxconfig that selects many
> > many packages as module. That, however, turned out to be a bit too complex,
> > because there are quite a few differences between the dependencies of App A, B
> > and C. Further, there are things that are bool instead of tristate that you
> > might not need/want.
> 
> The layers might help here too.

I'm curious, when can we expect it to see them in the ptxdist repository?



Florian
_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Michael Olbrich]

On Thu, Aug 30, 2018 at 08:05:04AM +0000, Baeuerle, Florian wrote:
> Am Mittwoch, den 29.08.2018, 16:48 +0200 schrieb Michael Olbrich:
> > On Tue, Aug 28, 2018 at 10:07:45AM +0000, Baeuerle, Florian wrote:
> > > the way we use ptxdist might be interesting for you as well.
> > > 
> > > We use one ptxdist directory with two platforms and three applications,
> > > let's call the platforms X and Y, and the Applications A, B and C
> > > 
> > > Application A with two variants on one platform
> > >  - release [platform X]
> > >  - devel [platform X]
> > > Application B with four variants on two platforms
> > >  - variant-1-release [platform X, Y]
> > >  - variant-1-devel [platform X, Y]
> > >  - variant-2-release [platform X, Y]
> > >  - variant-2-devel [platform X, Y]
> > > Application C with two variants on one platform
> > >  - release [platform X]
> > >  - devel [platform X]
> > > 
> > > The devel variant is release + gdb, a few other tools and root login.
> > > 
> > > Application A, B, C each have their own ptxconfig. Each variant has its
> > > collectionconfig. We build the images (squashfs + rauc bundles) with 
> > > image packages under each platform directory.
> > > 
> > > so for building all images for Application B on Platform X, we do this:
> > > 
> > > ptxdist images \
> > >     --ptxconfig=configs/app-B/ptxconfig \
> > >     --platformconfig=configs/plat-X/platformconfig
> > > 
> > > The platformconfig for Platform X has all image packages enabled, but of
> > > course we do not want to build image-platform-X-app-A with app-B's ptxconfig,
> > > so we put some ifs in the image package makefile, where we check for the
> > > currently selected PTXCONF_PROJECT.
> > > 
> > > What ptxdist misses for our use-case is some sort of collection-like feature
> > > for platform-specific things (such as image packages, or platform-specific
> > > graphics drivers). With the image packages, we use that workaround, but there
> > > could be other use-cases.
> > 
> > The layer stuff I'm working on may help here.
> > 
> > The idea is to basically stack multiple BSPs. Files and rules are searched
> > last to first much like we do this with BSP/platform/PTXdist already.
> > 
> > ptxconfig/platformconfig/kernelconfig etc. in the later 'BSPs' are
> > more or less a 'diff' to the same config on the next layer.
> 
> 
> Sounds interesting, even though I think I do not yet properly understand how it
> will work. Is it really a diff, or more like an extension? Can I add additional
> packages to an upper layer or can I also deselect them?

It's a diff. If you run e.g. a menuconfig in on layer the it will basically
take the config from the lower layer, applies the diff before actually
running menuconfig. When you're done, a new diff is created.

So each layer can add new packages and enable or disable existing packages.

> +-------------------------+-------------------------+ 
> |                         |                         |
> |       Platform A        |      Platform B         |
> |                         |                         |
> +-----------------+-------+-------+-----------------+ 
> |                 |               |                 | <- ptxconfig for each
> |      App A      |     App B     |      App C      |    application
> |                 |               |                 |
> |·················+···············+·················+
> |                 ·               ·                 | <- not really shared, even 
> |       Common packages selected in ptxconfig       |    though the packages are
> |                 ·               ·                 |    supposed to be 95% the same
> +--------+--------+---+---+---+---+--------+--------+
> |        |        | 1 | 1 | 2 | 2 |        |        | <- collections for a bunch of
> |  dev   |  rls   | d | r | d | r |  dev   |  rls   |    variants
> |        |        | e | l | e | l |        |        |
> |        |        | v | s | v | s |        |        |
> +--------+--------+---+---+---+---+--------+--------+
> 
> I would be happy if I could actually share the third layer in above ASCII drawing.
> 
> That basically is the common stuff that should end up on all images (systemd, busybox
> configuration, etc.). It's supposed to be the same everywhere, but they cannot be
> trivially kept in sync.

That should be easy with layers: The base layer contains all the common
stuff (for platforms, ptxconfig etc.). After changing something in the base
layer you just need to run oldconfig in each App layer.

> > > Let's say I want to build App D on two platforms, each requiring different
> > > graphic drivers (SGX is a nightmare), I wouldn't really know how to do that
> > > properly.
> > 
> > Hmm, we have something like this already: The 'platform-opengl' package
> > defaults to Mesa, but you can a platform specific rule in each platform. I
> > used this in the past for Mesa vs. Vivante.
> 
> That would, however, only work if it would be one package?
> For am335x with gpu, I need three packages: the OOT kernel module, a franken-patched
> libgbm and the blobs (libEGL, GLES, libpvr*, etc.).
> 
> When i override platform-opengl through configs/platform-a, then these three packages'
> symbols would end up in my ptxconfig, so my ptxconfig is more or less tied to my platform.

Add platform-opengl and the extra packages to the platformconfig and add
the dependencies there. If platform-opengl is enabled in both configs, then
any dependencies will be handled correctly.

> Besides: is there a way to just "by default" select and install a package for a given
> platform (without having the symbol in the ptxconfig)?

Add the symbol to the platformconfig.

> > > The problem otherwise could be solved by using one ptxconfig that selects many
> > > many packages as module. That, however, turned out to be a bit too complex,
> > > because there are quite a few differences between the dependencies of App A, B
> > > and C. Further, there are things that are bool instead of tristate that you
> > > might not need/want.
> > 
> > The layers might help here too.
> 
> I'm curious, when can we expect it to see them in the ptxdist repository?

My current plan is to get this ready for the October release. So if all
goes according to plan then the code should hit the repository sometime in
September.

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Baeuerle, Florian]

Hi,

Am Donnerstag, den 30.08.2018, 11:41 +0200 schrieb Michael Olbrich:
> On Thu, Aug 30, 2018 at 08:05:04AM +0000, Baeuerle, Florian wrote:
> > Am Mittwoch, den 29.08.2018, 16:48 +0200 schrieb Michael Olbrich:
> > > On Tue, Aug 28, 2018 at 10:07:45AM +0000, Baeuerle, Florian wrote:
> > > > the way we use ptxdist might be interesting for you as well.
> > > > 
> > > > We use one ptxdist directory with two platforms and three applications,
> > > > let's call the platforms X and Y, and the Applications A, B and C
> > > > 
> > > > Application A with two variants on one platform
> > > >  - release [platform X]
> > > >  - devel [platform X]
> > > > Application B with four variants on two platforms
> > > >  - variant-1-release [platform X, Y]
> > > >  - variant-1-devel [platform X, Y]
> > > >  - variant-2-release [platform X, Y]
> > > >  - variant-2-devel [platform X, Y]
> > > > Application C with two variants on one platform
> > > >  - release [platform X]
> > > >  - devel [platform X]
> > > > 
> > > > The devel variant is release + gdb, a few other tools and root login.
> > > > 
> > > > Application A, B, C each have their own ptxconfig. Each variant has its
> > > > collectionconfig. We build the images (squashfs + rauc bundles) with 
> > > > image packages under each platform directory.
> > > > 
> > > > so for building all images for Application B on Platform X, we do this:
> > > > 
> > > > ptxdist images \
> > > >     --ptxconfig=configs/app-B/ptxconfig \
> > > >     --platformconfig=configs/plat-X/platformconfig
> > > > 
> > > > The platformconfig for Platform X has all image packages enabled, but of
> > > > course we do not want to build image-platform-X-app-A with app-B's ptxconfig,
> > > > so we put some ifs in the image package makefile, where we check for the
> > > > currently selected PTXCONF_PROJECT.
> > > > 
> > > > What ptxdist misses for our use-case is some sort of collection-like feature
> > > > for platform-specific things (such as image packages, or platform-specific
> > > > graphics drivers). With the image packages, we use that workaround, but there
> > > > could be other use-cases.
> > > 
> > > The layer stuff I'm working on may help here.
> > > 
> > > The idea is to basically stack multiple BSPs. Files and rules are searched
> > > last to first much like we do this with BSP/platform/PTXdist already.
> > > 
> > > ptxconfig/platformconfig/kernelconfig etc. in the later 'BSPs' are
> > > more or less a 'diff' to the same config on the next layer.
> > 
> > 
> > Sounds interesting, even though I think I do not yet properly understand how it
> > will work. Is it really a diff, or more like an extension? Can I add additional
> > packages to an upper layer or can I also deselect them?
> 
> It's a diff. If you run e.g. a menuconfig in on layer the it will basically
> take the config from the lower layer, applies the diff before actually
> running menuconfig. When you're done, a new diff is created.
> 
> So each layer can add new packages and enable or disable existing packages.
> 
> > +-------------------------+-------------------------+ 
> > >                         |                         |
> > >       Platform A        |      Platform B         |
> > >                         |                         |
> > 
> > +-----------------+-------+-------+-----------------+ 
> > >                 |               |                 | <- ptxconfig for each
> > >      App A      |     App B     |      App C      |    application
> > >                 |               |                 |
> > > ·················+···············+·················+
> > >                 ·               ·                 | <- not really shared, even 
> > >       Common packages selected in ptxconfig       |    though the packages are
> > >                 ·               ·                 |    supposed to be 95% the same
> > 
> > +--------+--------+---+---+---+---+--------+--------+
> > >        |        | 1 | 1 | 2 | 2 |        |        | <- collections for a bunch of
> > >  dev   |  rls   | d | r | d | r |  dev   |  rls   |    variants
> > >        |        | e | l | e | l |        |        |
> > >        |        | v | s | v | s |        |        |
> > 
> > +--------+--------+---+---+---+---+--------+--------+
> > 
> > I would be happy if I could actually share the third layer in above ASCII drawing.
> > 
> > That basically is the common stuff that should end up on all images (systemd, busybox
> > configuration, etc.). It's supposed to be the same everywhere, but they cannot be
> > trivially kept in sync.
> 
> That should be easy with layers: The base layer contains all the common
> stuff (for platforms, ptxconfig etc.). After changing something in the base
> layer you just need to run oldconfig in each App layer.
> 
> > > > Let's say I want to build App D on two platforms, each requiring different
> > > > graphic drivers (SGX is a nightmare), I wouldn't really know how to do that
> > > > properly.
> > > 
> > > Hmm, we have something like this already: The 'platform-opengl' package
> > > defaults to Mesa, but you can a platform specific rule in each platform. I
> > > used this in the past for Mesa vs. Vivante.
> > 
> > That would, however, only work if it would be one package?
> > For am335x with gpu, I need three packages: the OOT kernel module, a franken-patched
> > libgbm and the blobs (libEGL, GLES, libpvr*, etc.).
> > 
> > When i override platform-opengl through configs/platform-a, then these three packages'
> > symbols would end up in my ptxconfig, so my ptxconfig is more or less tied to my platform.
> 
> Add platform-opengl and the extra packages to the platformconfig and add
> the dependencies there. If platform-opengl is enabled in both configs, then
> any dependencies will be handled correctly.
> 
> > Besides: is there a way to just "by default" select and install a package for a given
> > platform (without having the symbol in the ptxconfig)?
> 
> Add the symbol to the platformconfig.

Is there a way to do this so that the (manually?) added symbol survives a
"ptxdist menuconfig platform"?

> 
> > > > The problem otherwise could be solved by using one ptxconfig that selects many
> > > > many packages as module. That, however, turned out to be a bit too complex,
> > > > because there are quite a few differences between the dependencies of App A, B
> > > > and C. Further, there are things that are bool instead of tristate that you
> > > > might not need/want.
> > > 
> > > The layers might help here too.
> > 
> > I'm curious, when can we expect it to see them in the ptxdist repository?
> 
> My current plan is to get this ready for the October release. So if all
> goes according to plan then the code should hit the repository sometime in
> September.
> 

Cool, waiting eagerly for it :)


Florian
_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Michael Olbrich]

On Thu, Aug 30, 2018 at 01:06:57PM +0000, Baeuerle, Florian wrote:
> Am Donnerstag, den 30.08.2018, 11:41 +0200 schrieb Michael Olbrich:
> > On Thu, Aug 30, 2018 at 08:05:04AM +0000, Baeuerle, Florian wrote:
> > > Besides: is there a way to just "by default" select and install a package for a given
> > > platform (without having the symbol in the ptxconfig)?
> > 
> > Add the symbol to the platformconfig.
> 
> Is there a way to do this so that the (manually?) added symbol survives a
> "ptxdist menuconfig platform"?

Create a .in file in platforms/. Pick a section that exists for packages in
the platformconfig. If it's always on then define a symbol without prompt
and 'default y'.

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Baeuerle, Florian]

Am Donnerstag, den 30.08.2018, 11:41 +0200 schrieb Michael Olbrich:
> > > > Let's say I want to build App D on two platforms, each requiring different
> > > > graphic drivers (SGX is a nightmare), I wouldn't really know how to do that
> > > > properly.
> > > 
> > > Hmm, we have something like this already: The 'platform-opengl' package
> > > defaults to Mesa, but you can a platform specific rule in each platform. I
> > > used this in the past for Mesa vs. Vivante.
> > 
> > That would, however, only work if it would be one package?
> > For am335x with gpu, I need three packages: the OOT kernel module, a franken-patched
> > libgbm and the blobs (libEGL, GLES, libpvr*, etc.).
> > 
> > When i override platform-opengl through configs/platform-a, then these three packages'
> > symbols would end up in my ptxconfig, so my ptxconfig is more or less tied to my platform.
> 
> Add platform-opengl and the extra packages to the platformconfig and add
> the dependencies there. If platform-opengl is enabled in both configs, then
> any dependencies will be handled correctly.
> 

I added the three packages to

$platform_a/platforms

and an override for platform-opengl, selecting the three packages in

$platform_a/rules/platform-opengl.in

as a result, I have the three symbols in my ptxconfig, which I do not want.

If i add it in $platform_a/platforms/platform-opengl.in with the project_version
section, defaulting to y, "ptxdist menuconfig platform" spits out a bunch of
errors due to references to undefined symbols (the dependencies of the three
packages).

What I'd like is:

ptxconfig + platform a -> installs ti-sgx-drivers
ptxconfig + platform b -> installs other platform-dependent driver

That would mean some sort of dependency resolving that happens at some later
point. If I understand correctly, that happens during menuconfig, resulting in
a {ptx,platform}config.

I think I am missing something. Googling, trial and error and the commit logs
did not get my anywhere. Further help greatly appreciated, I'm volunteering to
write documentation on this, if desired.


Florian
_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Michael Olbrich]

On Fri, Aug 31, 2018 at 08:41:40AM +0000, Baeuerle, Florian wrote:
> Am Donnerstag, den 30.08.2018, 11:41 +0200 schrieb Michael Olbrich:
> > > > > Let's say I want to build App D on two platforms, each requiring different
> > > > > graphic drivers (SGX is a nightmare), I wouldn't really know how to do that
> > > > > properly.
> > > > 
> > > > Hmm, we have something like this already: The 'platform-opengl' package
> > > > defaults to Mesa, but you can a platform specific rule in each platform. I
> > > > used this in the past for Mesa vs. Vivante.
> > > 
> > > That would, however, only work if it would be one package?
> > > For am335x with gpu, I need three packages: the OOT kernel module, a franken-patched
> > > libgbm and the blobs (libEGL, GLES, libpvr*, etc.).
> > > 
> > > When i override platform-opengl through configs/platform-a, then these three packages'
> > > symbols would end up in my ptxconfig, so my ptxconfig is more or less tied to my platform.
> > 
> > Add platform-opengl and the extra packages to the platformconfig and add
> > the dependencies there. If platform-opengl is enabled in both configs, then
> > any dependencies will be handled correctly.
> > 
> 
> I added the three packages to
> 
> $platform_a/platforms
> 
> and an override for platform-opengl, selecting the three packages in
> 
> $platform_a/rules/platform-opengl.in
> 
> as a result, I have the three symbols in my ptxconfig, which I do not want.
> 
> If i add it in $platform_a/platforms/platform-opengl.in with the project_version
> section, defaulting to y, "ptxdist menuconfig platform" spits out a bunch of
> errors due to references to undefined symbols (the dependencies of the three
> packages).
> 
> What I'd like is:
> 
> ptxconfig + platform a -> installs ti-sgx-drivers
> ptxconfig + platform b -> installs other platform-dependent driver
> 
> That would mean some sort of dependency resolving that happens at some later
> point. If I understand correctly, that happens during menuconfig, resulting in
> a {ptx,platform}config.
> 
> I think I am missing something. Googling, trial and error and the commit logs
> did not get my anywhere. Further help greatly appreciated, I'm volunteering to
> write documentation on this, if desired.

You need:
$platform_a/rules/platform-opengl.in with no dependencies

$platform_a/platforms/platform-opengl.in that depends on ti-sgx-drivers
$platform_a/platforms/*.in for ti-sgx-drivers

These must have a section that exists for the platformconfig, e.g.
'architecture_options'. Don't use prompts and set 'default y' for
platform-opengl here.

Then the symbols will be in the platformconfig.

Do the samge for platform b.

At build-time it does not matter if the package is selected in the
ptxconfig or the platformconfig.

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] PTXdist usescases

[Baeuerle, Florian]

Am Freitag, den 31.08.2018, 12:17 +0200 schrieb Michael Olbrich:
> You need:
> $platform_a/rules/platform-opengl.in with no dependencies
> 
> $platform_a/platforms/platform-opengl.in that depends on ti-sgx-drivers
> $platform_a/platforms/*.in for ti-sgx-drivers
> 
> These must have a section that exists for the platformconfig, e.g.
> 'architecture_options'. Don't use prompts and set 'default y' for
> platform-opengl here.

It feels overall hacky and it does not seem to work for dependencies of the
ti-sgx-packages. So my options are:

- managing the ti-sgx- dependencies in the ptxconfig/or a collection manually.
- define dependant symbols the same way (including the symbols the dependencies
  depend on...), so they end up in the platformconfig

One more thing that I missed pointing out is that this combination should
(ideally) work as well:

ptxconfig without dependency on platform-opengl + platform a
 -> installs no ti-sgx-drivers
> 
> Then the symbols will be in the platformconfig.
> 
> Do the samge for platform b.
> 
> At build-time it does not matter if the package is selected in the
> ptxconfig or the platformconfig.


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de