[ptxdist] [RFC PATCH] doc: Add section on creating new layers

[ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Alexander Dahl]

After discussing these topics in IRC, it seemed reasonable to add some
explanation on how to create a new layer from scratch.

Signed-off-by: Alexander Dahl <ada@thorsis.com>
---
 doc/dev_manual.rst | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/doc/dev_manual.rst b/doc/dev_manual.rst
index d79ebdba7..746f3d695 100644
--- a/doc/dev_manual.rst
+++ b/doc/dev_manual.rst
@@ -1631,6 +1631,8 @@ convenient way to crate simple templates. It is also possible to create
 more files. For examples, the builtin ``genimage`` template creates a extra
 config file for the new package.
 
+.. _layers:
+
 Layers in PTXdist
 -----------------
 
@@ -1740,3 +1742,32 @@ layer.
 
 The make macros that provide these features are currently used by the
 barebox and kernel packages and templates.
+
+Creating a new Layer
+~~~~~~~~~~~~~~~~~~~~
+
+Adding new layers on top of an already existing BSP is easy:
+
+#. Create an empty folder, e.g. next to the existing BSP.
+#. The lower layer is just a folder named ``base`` (see layers_ above). You
+   may do one of the following (or something else you consider convenient):
+
+   - Create a symbolic link ``base`` to an outside laying BSP folder. For
+     an easy start, you can try this method with DistroKit_ for example:
+     ``$ ln -s ../DistroKit base``
+   - Use a Git submodule with a BSP in a folder ``base``.
+   - Copy over your whole existing BSP to a new subfolder ``base``.
+
+#. If needed select **ptxconfig** and/or **platformconfig** after copying
+   that file from the lower layer to your new layer:
+
+.. code-block:: text
+
+   $ mkdir -p configs/platform-v7a
+   $ cp base/configs/platform-v7a/platformconfig configs/platform-v7a/
+   $ ptxdist platform configs/platform-v7a/platformconfig
+
+#. Use ``ptxdist menu`` like usual.
+#. Put your new layer into Git and add configs and delta configs to it.
+
+.. _DistroKit: https://www.pengutronix.de/de/software/distrokit.html
-- 
2.20.1


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Roland Hieber]

On Tue, Sep 03, 2019 at 01:50:25PM +0200, Alexander Dahl wrote:
> After discussing these topics in IRC, it seemed reasonable to add some
> explanation on how to create a new layer from scratch.

\o/ someone else did a task on my overlong todo list :D

> Signed-off-by: Alexander Dahl <ada@thorsis.com>
> ---
>  doc/dev_manual.rst | 31 +++++++++++++++++++++++++++++++
>  1 file changed, 31 insertions(+)
> 
> diff --git a/doc/dev_manual.rst b/doc/dev_manual.rst
> index d79ebdba7..746f3d695 100644
> --- a/doc/dev_manual.rst
> +++ b/doc/dev_manual.rst
> @@ -1631,6 +1631,8 @@ convenient way to crate simple templates. It is also possible to create
>  more files. For examples, the builtin ``genimage`` template creates a extra
>  config file for the new package.
>  
> +.. _layers:
> +
>  Layers in PTXdist
>  -----------------
>  
> @@ -1740,3 +1742,32 @@ layer.
>  
>  The make macros that provide these features are currently used by the
>  barebox and kernel packages and templates.
> +
> +Creating a new Layer
> +~~~~~~~~~~~~~~~~~~~~
> +
> +Adding new layers on top of an already existing BSP is easy:
> +
> +#. Create an empty folder, e.g. next to the existing BSP.
> +#. The lower layer is just a folder named ``base`` (see layers_ above). You
> +   may do one of the following (or something else you consider convenient):
> +
> +   - Create a symbolic link ``base`` to an outside laying BSP folder. For
> +     an easy start, you can try this method with DistroKit_ for example:
> +     ``$ ln -s ../DistroKit base``

We have done that too sometimes in the past, but if the config files in
the base layer are not in the right versions, the MD5 sum in the config
diffs in the inherited layer will not match. I think this should be
mentioned here.

> +   - Use a Git submodule with a BSP in a folder ``base``.
> +   - Copy over your whole existing BSP to a new subfolder ``base``.

Subtree merges are another possibility, although I've never done one.

> +
> +#. If needed select **ptxconfig** and/or **platformconfig** after copying
> +   that file from the lower layer to your new layer:
> +
> +.. code-block:: text
> +
> +   $ mkdir -p configs/platform-v7a
> +   $ cp base/configs/platform-v7a/platformconfig configs/platform-v7a/
> +   $ ptxdist platform configs/platform-v7a/platformconfig

I think you should be able to select the config from base/ directly
without copying them? mol?

> +
> +#. Use ``ptxdist menu`` like usual.

menuconfig? platformconfig? 

> +#. Put your new layer into Git and add configs and delta configs to it.

nit: s/Git/SCM/, or something like that

> +
> +.. _DistroKit: https://www.pengutronix.de/de/software/distrokit.html
> -- 
> 2.20.1
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> 

-- 
Roland Hieber                     | r.hieber@pengutronix.de     |
Pengutronix e.K.                  | https://www.pengutronix.de/ |
Peiner Str. 6-8, 31137 Hildesheim | Phone: +49-5121-206917-5086 |
Amtsgericht Hildesheim, HRA 2686  | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Alexander Dahl]

Hello Roland,

Am Dienstag, 3. September 2019, 14:00:17 CEST schrieb Roland Hieber:
> On Tue, Sep 03, 2019 at 01:50:25PM +0200, Alexander Dahl wrote:
> > After discussing these topics in IRC, it seemed reasonable to add some
> > explanation on how to create a new layer from scratch.
> 
> \o/ someone else did a task on my overlong todo list :D

Yes, and I just saw you already wrote a piece on that topic, but on your 
website, not in ptxdist documentation:

https://pengutronix.de/en/blog/2019-06-18-DistroKitLayer.html

o.O

> > +
> > +Creating a new Layer
> > +~~~~~~~~~~~~~~~~~~~~
> > +
> > +Adding new layers on top of an already existing BSP is easy:
> > +
> > +#. Create an empty folder, e.g. next to the existing BSP.
> > +#. The lower layer is just a folder named ``base`` (see layers_ above).
> > You +   may do one of the following (or something else you consider
> > convenient): +
> > +   - Create a symbolic link ``base`` to an outside laying BSP folder. For
> > +     an easy start, you can try this method with DistroKit_ for example:
> > +     ``$ ln -s ../DistroKit base``
> 
> We have done that too sometimes in the past, but if the config files in
> the base layer are not in the right versions, the MD5 sum in the config
> diffs in the inherited layer will not match. I think this should be
> mentioned here.

Sorry, I do not understand. :-/

> > +   - Use a Git submodule with a BSP in a folder ``base``.
> > +   - Copy over your whole existing BSP to a new subfolder ``base``.
> 
> Subtree merges are another possibility, although I've never done one.

Same here.

> > +
> > +#. If needed select **ptxconfig** and/or **platformconfig** after copying
> > +   that file from the lower layer to your new layer:
> > +
> > +.. code-block:: text
> > +
> > +   $ mkdir -p configs/platform-v7a
> > +   $ cp base/configs/platform-v7a/platformconfig configs/platform-v7a/
> > +   $ ptxdist platform configs/platform-v7a/platformconfig
> 
> I think you should be able to select the config from base/ directly
> without copying them? mol?

I did not try that. What is possible to select would be indeed an interesting 
piece in the docs! That question came up multiple times in IRC in the last 
days. ;-)

> > +
> > +#. Use ``ptxdist menu`` like usual.
> 
> menuconfig? platformconfig?

You reach both from `p menu` … but I don't mind changing this.

> > +#. Put your new layer into Git and add configs and delta configs to it.
> 
> nit: s/Git/SCM/, or something like that

Yes.

So you already wrote on that topic. I wouldn't mind if you take my patch as 
base for your own additions. O:-)

Greets
Alex


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Michael Olbrich]

On Tue, Sep 03, 2019 at 02:22:04PM +0200, Alexander Dahl wrote:
> Hello Roland,
> 
> Am Dienstag, 3. September 2019, 14:00:17 CEST schrieb Roland Hieber:
> > On Tue, Sep 03, 2019 at 01:50:25PM +0200, Alexander Dahl wrote:
> > > After discussing these topics in IRC, it seemed reasonable to add some
> > > explanation on how to create a new layer from scratch.
> > 
> > \o/ someone else did a task on my overlong todo list :D
> 
> Yes, and I just saw you already wrote a piece on that topic, but on your 
> website, not in ptxdist documentation:
> 
> https://pengutronix.de/en/blog/2019-06-18-DistroKitLayer.html
> 
> o.O
> 
> > > +
> > > +Creating a new Layer
> > > +~~~~~~~~~~~~~~~~~~~~
> > > +
> > > +Adding new layers on top of an already existing BSP is easy:
> > > +
> > > +#. Create an empty folder, e.g. next to the existing BSP.
> > > +#. The lower layer is just a folder named ``base`` (see layers_ above).
> > > You +   may do one of the following (or something else you consider
> > > convenient): +
> > > +   - Create a symbolic link ``base`` to an outside laying BSP folder. For
> > > +     an easy start, you can try this method with DistroKit_ for example:
> > > +     ``$ ln -s ../DistroKit base``
> > 
> > We have done that too sometimes in the past, but if the config files in
> > the base layer are not in the right versions, the MD5 sum in the config
> > diffs in the inherited layer will not match. I think this should be
> > mentioned here.
> 
> Sorry, I do not understand. :-/
> 
> > > +   - Use a Git submodule with a BSP in a folder ``base``.
> > > +   - Copy over your whole existing BSP to a new subfolder ``base``.
> > 
> > Subtree merges are another possibility, although I've never done one.
> 
> Same here.
> 
> > > +
> > > +#. If needed select **ptxconfig** and/or **platformconfig** after copying
> > > +   that file from the lower layer to your new layer:
> > > +
> > > +.. code-block:: text
> > > +
> > > +   $ mkdir -p configs/platform-v7a
> > > +   $ cp base/configs/platform-v7a/platformconfig configs/platform-v7a/
> > > +   $ ptxdist platform configs/platform-v7a/platformconfig
> > 
> > I think you should be able to select the config from base/ directly
> > without copying them? mol?
> 
> I did not try that. What is possible to select would be indeed an interesting 
> piece in the docs! That question came up multiple times in IRC in the last 
> days. ;-)

If there is just on platform, then ptxdist will just use that on if no
selected_platformconfig exists. This is especially useful with layers,
because it will then just pick the config from the first layer that has
one.

This means, that you can start with just the base link. Then change
something with 'ptxdist platformconfig'. PTXdist will create the
platformconfig in your new layer and use it automatically.

The same works for the ptxconfig as well.

Note: ptxdist will delete the config if it is identical to the layer below.
It does not touch the selected_* links you might need to change those if it
happens.

Michael

> > > +
> > > +#. Use ``ptxdist menu`` like usual.
> > 
> > menuconfig? platformconfig?
> 
> You reach both from `p menu` … but I don't mind changing this.



> > > +#. Put your new layer into Git and add configs and delta configs to it.
> > 
> > nit: s/Git/SCM/, or something like that
> 
> Yes.
> 
> So you already wrote on that topic. I wouldn't mind if you take my patch as 
> base for your own additions. O:-)
> 
> Greets
> Alex
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Roland Hieber]

On Tue, Sep 03, 2019 at 03:08:08PM +0200, Michael Olbrich wrote:
> On Tue, Sep 03, 2019 at 02:22:04PM +0200, Alexander Dahl wrote:
> > Hello Roland,
> > 
> > Am Dienstag, 3. September 2019, 14:00:17 CEST schrieb Roland Hieber:
> > > On Tue, Sep 03, 2019 at 01:50:25PM +0200, Alexander Dahl wrote:
> > > > After discussing these topics in IRC, it seemed reasonable to add some
> > > > explanation on how to create a new layer from scratch.
> > > 
> > > \o/ someone else did a task on my overlong todo list :D
> > 
> > Yes, and I just saw you already wrote a piece on that topic, but on your 
> > website, not in ptxdist documentation:
> > 
> > https://pengutronix.de/en/blog/2019-06-18-DistroKitLayer.html
> > 
> > o.O

Uhm. Thanks for reminding me ... ^^ Chris did most of the work though, I
only proof-read it.

> > > > +
> > > > +Creating a new Layer
> > > > +~~~~~~~~~~~~~~~~~~~~
> > > > +
> > > > +Adding new layers on top of an already existing BSP is easy:
> > > > +
> > > > +#. Create an empty folder, e.g. next to the existing BSP.
> > > > +#. The lower layer is just a folder named ``base`` (see layers_ above).
> > > > You +   may do one of the following (or something else you consider
> > > > convenient): +
> > > > +   - Create a symbolic link ``base`` to an outside laying BSP folder. For
> > > > +     an easy start, you can try this method with DistroKit_ for example:
> > > > +     ``$ ln -s ../DistroKit base``
> > > 
> > > We have done that too sometimes in the past, but if the config files in
> > > the base layer are not in the right versions, the MD5 sum in the config
> > > diffs in the inherited layer will not match. I think this should be
> > > mentioned here.
> > 
> > Sorry, I do not understand. :-/
> > 
> > > > +   - Use a Git submodule with a BSP in a folder ``base``.
> > > > +   - Copy over your whole existing BSP to a new subfolder ``base``.
> > > 
> > > Subtree merges are another possibility, although I've never done one.
> > 
> > Same here.
> > 
> > > > +
> > > > +#. If needed select **ptxconfig** and/or **platformconfig** after copying
> > > > +   that file from the lower layer to your new layer:
> > > > +
> > > > +.. code-block:: text
> > > > +
> > > > +   $ mkdir -p configs/platform-v7a
> > > > +   $ cp base/configs/platform-v7a/platformconfig configs/platform-v7a/
> > > > +   $ ptxdist platform configs/platform-v7a/platformconfig
> > > 
> > > I think you should be able to select the config from base/ directly
> > > without copying them? mol?
> > 
> > I did not try that. What is possible to select would be indeed an interesting 
> > piece in the docs! That question came up multiple times in IRC in the last 
> > days. ;-)
> 
> If there is just on platform, then ptxdist will just use that on if no
> selected_platformconfig exists. This is especially useful with layers,
> because it will then just pick the config from the first layer that has
> one.
> 
> This means, that you can start with just the base link. Then change
> something with 'ptxdist platformconfig'. PTXdist will create the
> platformconfig in your new layer and use it automatically.
> 
> The same works for the ptxconfig as well.

Hmm, this explanation does not match my experience...

    rhi@dude test-bsp $ p version
    2019.08.0
    
    rhi@dude test-bsp $ l
    lrwxrwxrwx  1 rhi ptx   10 Sep  3 14:58 base -> DistroKit/
    drwxr-xr-x 11 rhi ptx  232 Sep  3 15:03 DistroKit
    
    rhi@dude test-bsp $ l DistroKit/selected_*
    lrwxrwxrwx 1 rhi ptx  35 Sep  3 15:03 DistroKit/selected_platformconfig -> configs/platform-v7a/platformconfig
    lrwxrwxrwx 1 rhi ptx 118 Sep  3 15:03 DistroKit/selected_toolchain -> /opt/OSELAS.Toolchain-2018.12.0/arm-v7a-linux-gnueabihf/gcc-8.2.1-glibc-2.28-binutils-2.31.1-kernel-4.19-sanitized/bin
    
    rhi@dude test-bsp $ p menuconfig platform
    configuration written to .config
    
    *** End of the configuration.
    *** Execute 'ptxdist go' to build the project or try 'ptxdist help'.
    
    Saving 'test-bsp/DistroKit/configs/platform-v7a/platformconfig'.

Same for ptxdist menuconfig, which changes test-bsp/DistroKit/configs/ptxconfig
instead of configs/ptxconfig in the current layer. However, when I copy
base/configs/ptxconfig to configs/ptxconfig, and then do a menuconfig,
it is changed in the current layer.

 - Roland

-- 
Roland Hieber                     | r.hieber@pengutronix.de     |
Pengutronix e.K.                  | https://www.pengutronix.de/ |
Peiner Str. 6-8, 31137 Hildesheim | Phone: +49-5121-206917-5086 |
Amtsgericht Hildesheim, HRA 2686  | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Michael Olbrich]

On Tue, Sep 03, 2019 at 03:38:31PM +0200, Roland Hieber wrote:
> On Tue, Sep 03, 2019 at 03:08:08PM +0200, Michael Olbrich wrote:
> > On Tue, Sep 03, 2019 at 02:22:04PM +0200, Alexander Dahl wrote:
> > > Hello Roland,
> > > 
> > > Am Dienstag, 3. September 2019, 14:00:17 CEST schrieb Roland Hieber:
> > > > On Tue, Sep 03, 2019 at 01:50:25PM +0200, Alexander Dahl wrote:
> > > > > After discussing these topics in IRC, it seemed reasonable to add some
> > > > > explanation on how to create a new layer from scratch.
> > > > 
> > > > \o/ someone else did a task on my overlong todo list :D
> > > 
> > > Yes, and I just saw you already wrote a piece on that topic, but on your 
> > > website, not in ptxdist documentation:
> > > 
> > > https://pengutronix.de/en/blog/2019-06-18-DistroKitLayer.html
> > > 
> > > o.O
> 
> Uhm. Thanks for reminding me ... ^^ Chris did most of the work though, I
> only proof-read it.
> 
> > > > > +
> > > > > +Creating a new Layer
> > > > > +~~~~~~~~~~~~~~~~~~~~
> > > > > +
> > > > > +Adding new layers on top of an already existing BSP is easy:
> > > > > +
> > > > > +#. Create an empty folder, e.g. next to the existing BSP.
> > > > > +#. The lower layer is just a folder named ``base`` (see layers_ above).
> > > > > You +   may do one of the following (or something else you consider
> > > > > convenient): +
> > > > > +   - Create a symbolic link ``base`` to an outside laying BSP folder. For
> > > > > +     an easy start, you can try this method with DistroKit_ for example:
> > > > > +     ``$ ln -s ../DistroKit base``
> > > > 
> > > > We have done that too sometimes in the past, but if the config files in
> > > > the base layer are not in the right versions, the MD5 sum in the config
> > > > diffs in the inherited layer will not match. I think this should be
> > > > mentioned here.
> > > 
> > > Sorry, I do not understand. :-/
> > > 
> > > > > +   - Use a Git submodule with a BSP in a folder ``base``.
> > > > > +   - Copy over your whole existing BSP to a new subfolder ``base``.
> > > > 
> > > > Subtree merges are another possibility, although I've never done one.
> > > 
> > > Same here.
> > > 
> > > > > +
> > > > > +#. If needed select **ptxconfig** and/or **platformconfig** after copying
> > > > > +   that file from the lower layer to your new layer:
> > > > > +
> > > > > +.. code-block:: text
> > > > > +
> > > > > +   $ mkdir -p configs/platform-v7a
> > > > > +   $ cp base/configs/platform-v7a/platformconfig configs/platform-v7a/
> > > > > +   $ ptxdist platform configs/platform-v7a/platformconfig
> > > > 
> > > > I think you should be able to select the config from base/ directly
> > > > without copying them? mol?
> > > 
> > > I did not try that. What is possible to select would be indeed an interesting 
> > > piece in the docs! That question came up multiple times in IRC in the last 
> > > days. ;-)
> > 
> > If there is just on platform, then ptxdist will just use that on if no
> > selected_platformconfig exists. This is especially useful with layers,
> > because it will then just pick the config from the first layer that has
> > one.
> > 
> > This means, that you can start with just the base link. Then change
> > something with 'ptxdist platformconfig'. PTXdist will create the
> > platformconfig in your new layer and use it automatically.
> > 
> > The same works for the ptxconfig as well.
> 
> Hmm, this explanation does not match my experience...
> 
>     rhi@dude test-bsp $ p version
>     2019.08.0
>     
>     rhi@dude test-bsp $ l
>     lrwxrwxrwx  1 rhi ptx   10 Sep  3 14:58 base -> DistroKit/
>     drwxr-xr-x 11 rhi ptx  232 Sep  3 15:03 DistroKit
>     
>     rhi@dude test-bsp $ l DistroKit/selected_*
>     lrwxrwxrwx 1 rhi ptx  35 Sep  3 15:03 DistroKit/selected_platformconfig -> configs/platform-v7a/platformconfig
>     lrwxrwxrwx 1 rhi ptx 118 Sep  3 15:03 DistroKit/selected_toolchain -> /opt/OSELAS.Toolchain-2018.12.0/arm-v7a-linux-gnueabihf/gcc-8.2.1-glibc-2.28-binutils-2.31.1-kernel-4.19-sanitized/bin
>     
>     rhi@dude test-bsp $ p menuconfig platform
>     configuration written to .config
>     
>     *** End of the configuration.
>     *** Execute 'ptxdist go' to build the project or try 'ptxdist help'.
>     
>     Saving 'test-bsp/DistroKit/configs/platform-v7a/platformconfig'.
> 
> Same for ptxdist menuconfig, which changes test-bsp/DistroKit/configs/ptxconfig
> instead of configs/ptxconfig in the current layer. However, when I copy
> base/configs/ptxconfig to configs/ptxconfig, and then do a menuconfig,
> it is changed in the current layer.

I cannot reproduce this. It works as expected for me. The last log message
may be a clue, I would expect

"Saving 'test-bsp/configs/platform-v7a/platformconfig'."
or
"Saving 'test-bsp/base/configs/platform-v7a/platformconfig'."

but not "test-bsp/DistroKit/....". So maybe the symlink handling does not
work correctly.

What's the output of 'p migrate'?
What happens if 'base' is a symlink to '../DistroKit'?

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Roland Hieber]

On Tue, Sep 03, 2019 at 04:12:02PM +0200, Michael Olbrich wrote:
> I cannot reproduce this. It works as expected for me. The last log message
> may be a clue, I would expect
> 
> "Saving 'test-bsp/configs/platform-v7a/platformconfig'."
> or
> "Saving 'test-bsp/base/configs/platform-v7a/platformconfig'."
> 
> but not "test-bsp/DistroKit/....". So maybe the symlink handling does not
> work correctly.
> 
> What's the output of 'p migrate'?

See in the log below.

> What happens if 'base' is a symlink to '../DistroKit'?

I tried three different things:

1.  clean DistroKit checked out as base/ directory inside the BSP:
    => works as intended.

    rhi@dude test-bsp $ git clone ../DistroKit base
    Cloning into 'base'...
    done.

    rhi@dude test-bsp $ ll
    total 0
    drwxr-xr-x 10 rhi ptx  212 Sep  3 15:55 base

    rhi@dude test-bsp $ p menuconfig
    error:	'selected_platformconfig' is missing
    	try 'ptxdist platform <platformconfig>'

    rhi@dude test-bsp $ cd base/; p platform configs/platform-v7a/platformconfig; cd ..
    info: selected platformconfig:
          'configs/platform-v7a/platformconfig'

    found and using toolchain:
    '/opt/OSELAS.Toolchain-2018.12.0/arm-v7a-linux-gnueabihf/gcc-8.2.1-glibc-2.28-binutils-2.31.1-kernel-4.19-sanitized/bin'

    rhi@dude test-bsp $ p menuconfig
    [... change project name ...]
    #
    # configuration written to .config
    #
    configuration written to .config

    *** End of the configuration.
    *** Execute 'ptxdist go' to build the project or try 'ptxdist help'.

    Saving 'test-bsp/configs/ptxconfig'.
    Saving 'test-bsp/configs/ptxconfig.diff'.

    rhi@dude test-bsp $ ll
    total 0
    drwxr-xr-x 10 rhi ptx  212 Sep  3 15:55 base
    drwxr-xr-x  2 rhi ptx   66 Sep  4 10:59 configs
    drwxr-xr-x  2 rhi ptx   78 Sep  4 10:59 platform-v7a

2.  DistroKit checked out as DistroKit inside the BSP, symlink base/:
    => writes to the config in the wrong layer.

    rhi@dude test-bsp $ git clone ../DistroKit DistroKit
    Cloning into 'DistroKit'...
    done.

    rhi@dude test-bsp $ ln -s DistroKit/ base

    rhi@dude test-bsp $ ll
    total 0
    lrwxrwxrwx  1 rhi ptx  10 Sep  4 11:55 base -> DistroKit/
    drwxr-xr-x 10 rhi ptx 155 Sep  4 11:55 DistroKit

    rhi@dude test-bsp $ p menuconfig
    error:	'selected_platformconfig' is missing
    	try 'ptxdist platform <platformconfig>'

    rhi@dude test-bsp $ cd base/; p platform configs/platform-v7a/platformconfig; cd ..
    info: selected platformconfig:
          'configs/platform-v7a/platformconfig'

    found and using toolchain:
    '/opt/OSELAS.Toolchain-2018.12.0/arm-v7a-linux-gnueabihf/gcc-8.2.1-glibc-2.28-binutils-2.31.1-kernel-4.19-sanitized/bin'

    rhi@dude test-bsp $ p menuconfig
    [... change project name ...]
    configuration written to .config

    *** End of the configuration.
    *** Execute 'ptxdist go' to build the project or try 'ptxdist help'.

    Saving 'test-bsp/DistroKit/configs/ptxconfig'.

    rhi@dude test-bsp $ ll
    total 0
    lrwxrwxrwx  1 rhi ptx  10 Sep  4 11:55 base -> DistroKit/
    drwxr-xr-x 10 rhi ptx 212 Sep  4 12:14 DistroKit
    drwxr-xr-x  2 rhi ptx  78 Sep  4 12:14 platform-v7a

    rhi@dude test-bsp $ p migrate
    Updating 'test-bsp/DistroKit/configs/ptxconfig'...
    #
    # configuration written to .config
    #

    Updating 'test-bsp/DistroKit/configs/platform-v7a/platformconfig'...
    #
    # configuration written to .config
    #

    rhi@dude test-bsp $ ll
    total 0
    lrwxrwxrwx  1 rhi ptx  10 Sep  4 12:25 base -> DistroKit/
    drwxr-xr-x 10 rhi ptx 212 Sep  4 12:25 DistroKit
    drwxr-xr-x  2 rhi ptx  78 Sep  4 12:26 platform-v7a

3.  DistroKit cloned outside the BSP, symlink base/
    => works as intended.

    rhi@dude test-bsp $ git clone ../distrokit ../DistroKit
    Cloning into '../DistroKit'...
    done.

    rhi@dude test-bsp $ ln -s ../DistroKit base

    rhi@dude test-bsp $ ll
    total 0
    lrwxrwxrwx 1 rhi ptx 12 Sep  4 12:19 base -> ../DistroKit

    rhi@dude test-bsp $ p menuconfig
    error:	'selected_platformconfig' is missing
    	try 'ptxdist platform <platformconfig>'

    rhi@dude test-bsp $ cd base/; p platform configs/platform-v7a/platformconfig; cd ..
    info: selected platformconfig:
          'configs/platform-v7a/platformconfig'

    found and using toolchain:
    '/opt/OSELAS.Toolchain-2018.12.0/arm-v7a-linux-gnueabihf/gcc-8.2.1-glibc-2.28-binutils-2.31.1-kernel-4.19-sanitized/bin'

    rhi@dude test-bsp $ p menuconfig
    [... change project name ...]
    #
    # configuration written to .config
    #
    configuration written to .config

    *** End of the configuration.
    *** Execute 'ptxdist go' to build the project or try 'ptxdist help'.

    Saving 'test-bsp/configs/ptxconfig'.
    Saving 'test-bsp/configs/ptxconfig.diff'.

    rhi@dude dude test-bsp $ ll
    total 0
    lrwxrwxrwx 1 rhi ptx 12 Sep  4 12:19 base -> ../DistroKit
    drwxr-xr-x 2 rhi ptx 66 Sep  4 12:20 configs
    drwxr-xr-x 2 rhi ptx 78 Sep  4 12:19 platform-v7a

So it seems I hit a special case there in my last mail…

-- 
Roland Hieber                     | r.hieber@pengutronix.de     |
Pengutronix e.K.                  | https://www.pengutronix.de/ |
Peiner Str. 6-8, 31137 Hildesheim | Phone: +49-5121-206917-5086 |
Amtsgericht Hildesheim, HRA 2686  | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Roland Hieber]

On Wed, Sep 04, 2019 at 12:30:58PM +0200, Roland Hieber wrote:
> On Tue, Sep 03, 2019 at 04:12:02PM +0200, Michael Olbrich wrote:
> > I cannot reproduce this. It works as expected for me. The last log message
> > may be a clue, I would expect
> > 
> > "Saving 'test-bsp/configs/platform-v7a/platformconfig'."
> > or
> > "Saving 'test-bsp/base/configs/platform-v7a/platformconfig'."
> > 
> > but not "test-bsp/DistroKit/....". So maybe the symlink handling does not
> > work correctly.
> > 
> > What's the output of 'p migrate'?
> 
> See in the log below.
> 
> > What happens if 'base' is a symlink to '../DistroKit'?
> 
> I tried three different things:
> 
[...]
> 2.  DistroKit checked out as DistroKit inside the BSP, symlink base/:
>     => writes to the config in the wrong layer.

It turned out this was an edge case with symlinks, mol just fixed it
with commit c20eaea9bc4c85b65389 ("ptxd_normalize_config: handle
symlinks in the workspace path correctly") in ptxdist-2019.09.0.

 - Roland

-- 
Roland Hieber                     | r.hieber@pengutronix.de     |
Pengutronix e.K.                  | https://www.pengutronix.de/ |
Peiner Str. 6-8, 31137 Hildesheim | Phone: +49-5121-206917-5086 |
Amtsgericht Hildesheim, HRA 2686  | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Alexander Dahl]

Hei hei,

Am Dienstag, 3. September 2019, 15:08:08 CEST schrieb Michael Olbrich:
> On Tue, Sep 03, 2019 at 02:22:04PM +0200, Alexander Dahl wrote:
> > Am Dienstag, 3. September 2019, 14:00:17 CEST schrieb Roland Hieber:
> > > On Tue, Sep 03, 2019 at 01:50:25PM +0200, Alexander Dahl wrote:
> > > > +
> > > > +#. If needed select **ptxconfig** and/or **platformconfig** after
> > > > copying
> > > > +   that file from the lower layer to your new layer:
> > > > +
> > > > +.. code-block:: text
> > > > +
> > > > +   $ mkdir -p configs/platform-v7a
> > > > +   $ cp base/configs/platform-v7a/platformconfig
> > > > configs/platform-v7a/
> > > > +   $ ptxdist platform configs/platform-v7a/platformconfig
> > > 
> > > I think you should be able to select the config from base/ directly
> > > without copying them? mol?
> > 
> > I did not try that. What is possible to select would be indeed an
> > interesting piece in the docs! That question came up multiple times in
> > IRC in the last days. ;-)
> 
> If there is just on platform, then ptxdist will just use that on if no
> selected_platformconfig exists. This is especially useful with layers,
> because it will then just pick the config from the first layer that has
> one.

So this is not different to ptxdist behaviour without layers. Maybe DistroKit 
is no good example, because it contains more than one platform and you must 
select one to get going?

> This means, that you can start with just the base link. Then change
> something with 'ptxdist platformconfig'. PTXdist will create the
> platformconfig in your new layer and use it automatically.
> 
> The same works for the ptxconfig as well.
> 
> Note: ptxdist will delete the config if it is identical to the layer below.

I think this is very convenient. This makes creating a new layer even easier, 
at least for BSPs with exactly one ptxconfig and exactly one platformconfig.

> It does not touch the selected_* links you might need to change those if it
> happens.

So the question remains. If there is more than one platform or config, where 
should it be selected best and how?

Greets
Alex


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Michael Olbrich]

On Tue, Sep 03, 2019 at 03:48:17PM +0200, Alexander Dahl wrote:
> Am Dienstag, 3. September 2019, 15:08:08 CEST schrieb Michael Olbrich:
> > If there is just on platform, then ptxdist will just use that on if no
> > selected_platformconfig exists. This is especially useful with layers,
> > because it will then just pick the config from the first layer that has
> > one.
> 
> So this is not different to ptxdist behaviour without layers. Maybe DistroKit 
> is no good example, because it contains more than one platform and you must 
> select one to get going?

Correct. You need to select one (or run ptxdist with --platformconfig=...).
Once ptxdist has crated a platformconfig for the new layer, you need to
select that one. PTXdist will complain if you forget this step.

> > This means, that you can start with just the base link. Then change
> > something with 'ptxdist platformconfig'. PTXdist will create the
> > platformconfig in your new layer and use it automatically.
> > 
> > The same works for the ptxconfig as well.
> > 
> > Note: ptxdist will delete the config if it is identical to the layer below.
> 
> I think this is very convenient. This makes creating a new layer even easier, 
> at least for BSPs with exactly one ptxconfig and exactly one platformconfig.

That's the most common use case, so I made sure that this is easy.

> > It does not touch the selected_* links you might need to change those if it
> > happens.
> 
> So the question remains. If there is more than one platform or config, where 
> should it be selected best and how?

Only selections in the current layer are used. Anything else is ignored.
And PTXdist enforces that the selected config is from the top layer (if the
config exists there) or the next layer that contains it.

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Alexander Dahl]

[-- Attachment #1.1: Type: text/plain, Size: 3759 bytes --]

Hello Michael,

I tried a little more:

    mkdir DistroKit-rpi1
    cd DistroKit-rpi1
    ln -s ../DistroKit base

The new layer is almost empty so far, see below.

On Tue, Sep 03, 2019 at 04:22:55PM +0200, Michael Olbrich wrote:
> On Tue, Sep 03, 2019 at 03:48:17PM +0200, Alexander Dahl wrote:
> > Am Dienstag, 3. September 2019, 15:08:08 CEST schrieb Michael Olbrich:
> > > If there is just on platform, then ptxdist will just use that on if no
> > > selected_platformconfig exists. This is especially useful with layers,
> > > because it will then just pick the config from the first layer that has
> > > one.
> > 
> > So this is not different to ptxdist behaviour without layers. Maybe DistroKit 
> > is no good example, because it contains more than one platform and you must 
> > select one to get going?
> 
> Correct. You need to select one (or run ptxdist with --platformconfig=...).
> Once ptxdist has crated a platformconfig for the new layer, you need to
> select that one. PTXdist will complain if you forget this step.

Indeed! Because there was no platformconfig in my new layer, I
selected the platformconfig from the base layer:

    ptxdist platform base/configs/platform-rpi/platformconfig

> > > It does not touch the selected_* links you might need to change those if it
> > > happens.
> > 
> > So the question remains. If there is more than one platform or config, where 
> > should it be selected best and how?
> 
> Only selections in the current layer are used. Anything else is ignored.
> And PTXdist enforces that the selected config is from the top layer (if the
> config exists there) or the next layer that contains it.

"Enforces" does not mean it changes the platfrom selection by itself
automatically, but just checks and bails out in case. For example,
after selecting the platformconfig in the base layer before, I changed
something in the platformconfig, which leads to ptxdist creating a
platformconfig file in my new layer. Now ptxdist stops like this:


    alex@lemmy ~/src/DistroKit-rpi1 % p -q -j6 go

    ptxdist: error: The selected config file:
    ptxdist: error:
    DistroKit-rpi1/base/configs/platform-rpi/platformconfig
    ptxdist: error: is overwritten by:
    ptxdist: error: DistroKit-rpi1/configs/platform-rpi/platformconfig
    ptxdist: error: The config file in the outer layer must be used!

    error: error during generation of dependencies


    ptxdist: error: error in dgen


The error message is clear, I have to select the platform in my upper
layer now. Okay, did that.

Now I changed something in my platformconfig to make it equal to the
platformconfig of the lower layer again. This leads to ptxdist
removing the platformconfig from the upper layer (just keeping
platformconfig.old), I have a symlink 'selected_platform' to a non
existent file and ptxdist bails out like this:


    alex@lemmy ~/src/DistroKit-rpi1 % p -q -j6 go                                   
    error:  'selected_platformconfig' is missing
            try 'ptxdist platform <platformconfig>'


All in all, one has to take care a little and reset the selected
platform if ptxdist complains, but ptxdist either seems to build or
print what's wrong, so it's not that hard to find out, which platform
to select.

I'll see what I can do with that information, to improve the
documentation or the patch, if nobody else wants to jump in.

Greets
Alex

-- 
/"\ ASCII RIBBON | »With the first link, the chain is forged. The first
\ / CAMPAIGN     | speech censured, the first thought forbidden, the
 X  AGAINST      | first freedom denied, chains us all irrevocably.«
/ \ HTML MAIL    | (Jean-Luc Picard, quoting Judge Aaron Satie)

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 92 bytes --]

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [RFC PATCH] doc: Add section on creating new layers

[Michael Olbrich]

Hi,

On Tue, Sep 03, 2019 at 08:49:12PM +0200, Alexander Dahl wrote:
> I tried a little more:
> 
>     mkdir DistroKit-rpi1
>     cd DistroKit-rpi1
>     ln -s ../DistroKit base
> 
> The new layer is almost empty so far, see below.
> 
> On Tue, Sep 03, 2019 at 04:22:55PM +0200, Michael Olbrich wrote:
> > On Tue, Sep 03, 2019 at 03:48:17PM +0200, Alexander Dahl wrote:
> > > Am Dienstag, 3. September 2019, 15:08:08 CEST schrieb Michael Olbrich:
> > > > If there is just on platform, then ptxdist will just use that on if no
> > > > selected_platformconfig exists. This is especially useful with layers,
> > > > because it will then just pick the config from the first layer that has
> > > > one.
> > > 
> > > So this is not different to ptxdist behaviour without layers. Maybe DistroKit 
> > > is no good example, because it contains more than one platform and you must 
> > > select one to get going?
> > 
> > Correct. You need to select one (or run ptxdist with --platformconfig=...).
> > Once ptxdist has crated a platformconfig for the new layer, you need to
> > select that one. PTXdist will complain if you forget this step.
> 
> Indeed! Because there was no platformconfig in my new layer, I
> selected the platformconfig from the base layer:
> 
>     ptxdist platform base/configs/platform-rpi/platformconfig
> 
> > > > It does not touch the selected_* links you might need to change those if it
> > > > happens.
> > > 
> > > So the question remains. If there is more than one platform or config, where 
> > > should it be selected best and how?
> > 
> > Only selections in the current layer are used. Anything else is ignored.
> > And PTXdist enforces that the selected config is from the top layer (if the
> > config exists there) or the next layer that contains it.
> 
> "Enforces" does not mean it changes the platfrom selection by itself
> automatically, but just checks and bails out in case. For example,
> after selecting the platformconfig in the base layer before, I changed
> something in the platformconfig, which leads to ptxdist creating a
> platformconfig file in my new layer. Now ptxdist stops like this:
> 
> 
>     alex@lemmy ~/src/DistroKit-rpi1 % p -q -j6 go
> 
>     ptxdist: error: The selected config file:
>     ptxdist: error:
>     DistroKit-rpi1/base/configs/platform-rpi/platformconfig
>     ptxdist: error: is overwritten by:
>     ptxdist: error: DistroKit-rpi1/configs/platform-rpi/platformconfig
>     ptxdist: error: The config file in the outer layer must be used!
> 
>     error: error during generation of dependencies
> 
> 
>     ptxdist: error: error in dgen
> 
> 
> The error message is clear, I have to select the platform in my upper
> layer now. Okay, did that.
> 
> Now I changed something in my platformconfig to make it equal to the
> platformconfig of the lower layer again. This leads to ptxdist
> removing the platformconfig from the upper layer (just keeping
> platformconfig.old), I have a symlink 'selected_platform' to a non
> existent file and ptxdist bails out like this:
> 
> 
>     alex@lemmy ~/src/DistroKit-rpi1 % p -q -j6 go                                   
>     error:  'selected_platformconfig' is missing
>             try 'ptxdist platform <platformconfig>'
> 
> 
> All in all, one has to take care a little and reset the selected
> platform if ptxdist complains, but ptxdist either seems to build or
> print what's wrong, so it's not that hard to find out, which platform
> to select.

Right, this error message could be a bit better. We should handle existing
symlinks that point to non-existing paths differently.

So, I've considered handling this automatically when the config is created
or deleted. However, the code handling this stuff is quite complex, with
all the different possibilities when it comes to symlinks. I still need to
track down why things happen differently for Roland :-/. So I'm reluctant
to add more complexity and change the links automatically.

And I don't want to magically use a file other and the one the link points
to (e.g. use configs/ptxconfig when the link points to
base/configs/ptxconfig). Someone might open the file directly and things
get confusing.

But maybe we could accept --ptxconfig=configs/ptxconfig as a command-line
option, even if only base/configs/ptxconfig exists (and the same for the
platformconfig of course). Would that make sense to you?

> I'll see what I can do with that information, to improve the
> documentation or the patch, if nobody else wants to jump in.

Please do so. I'll be happy to review the result.

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de