[ptxdist] [PATCH 0/5] Add code-signing-provider template, add code signing docs

[ptxdist] [PATCH 0/5] Add code-signing-provider template, add code signing docs

[Bastian Krause]

The code signing infrastructure is available since 2019 in PTXdist. Now
it's time to document it, especially since RAUC also uses it.
Unfortunately this did not make it into ptxdist-2020.06.0.

To make things easier introduce a code-signing-provider template.

Bastian Krause (5):
  package templates: add code-signing-provider template
  doc: dev_manual: split up into multiple files
  doc: move code signing docs from scripts/ into doc/
  doc: dev_code_signing: rework and extend code signing section
  doc: introduce ref_code_signing_helpers

 doc/dev_add_bin_only_files.rst                |  105 +
 doc/dev_add_new_pkgs.rst                      | 1339 +++++++++++++
 doc/dev_code_signing.rst                      |  129 ++
 doc/dev_create_new_pkg_templates.rst          |   77 +
 doc/dev_dir_hierarchy.rst                     |  108 +
 doc/dev_layers_in_ptxdist.rst                 |  111 ++
 doc/dev_manual.rst                            | 1765 +----------------
 doc/ref_code_signing_helpers.inc              |  246 +++
 doc/ref_manual.rst                            |    1 +
 .../code-signing-provider/ptxdist-set-keys.sh |   96 +
 .../template-code-signing-provider-choice-in  |    5 +
 .../template-code-signing-provider-in         |   16 +
 .../template-code-signing-provider-make       |   41 +
 scripts/lib/ptxd_lib_code_signing.sh          |   32 +-
 scripts/lib/ptxd_lib_template.sh              |   16 +
 15 files changed, 2302 insertions(+), 1785 deletions(-)
 create mode 100644 doc/dev_add_bin_only_files.rst
 create mode 100644 doc/dev_add_new_pkgs.rst
 create mode 100644 doc/dev_code_signing.rst
 create mode 100644 doc/dev_create_new_pkg_templates.rst
 create mode 100644 doc/dev_dir_hierarchy.rst
 create mode 100644 doc/dev_layers_in_ptxdist.rst
 create mode 100644 doc/ref_code_signing_helpers.inc
 create mode 100755 rules/templates/code-signing-provider/ptxdist-set-keys.sh
 create mode 100644 rules/templates/template-code-signing-provider-choice-in
 create mode 100644 rules/templates/template-code-signing-provider-in
 create mode 100644 rules/templates/template-code-signing-provider-make

-- 
2.27.0


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

[ptxdist] [PATCH 1/5] package templates: add code-signing-provider template

[Bastian Krause]

A ptxdist code signing provider is a package which selects the required
host tools needed for the code signing helpers to work. A shell script
is needed to define roles, set PKCS#11 URIs and import keys if SoftHSM
is used. In order to simplify its creation provide a template along with
an example script.

Signed-off-by: Bastian Krause <bst@pengutronix.de>
---
 .../code-signing-provider/ptxdist-set-keys.sh | 96 +++++++++++++++++++
 .../template-code-signing-provider-choice-in  |  5 +
 .../template-code-signing-provider-in         | 16 ++++
 .../template-code-signing-provider-make       | 41 ++++++++
 scripts/lib/ptxd_lib_template.sh              | 16 ++++
 5 files changed, 174 insertions(+)
 create mode 100755 rules/templates/code-signing-provider/ptxdist-set-keys.sh
 create mode 100644 rules/templates/template-code-signing-provider-choice-in
 create mode 100644 rules/templates/template-code-signing-provider-in
 create mode 100644 rules/templates/template-code-signing-provider-make

diff --git a/rules/templates/code-signing-provider/ptxdist-set-keys.sh b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
new file mode 100755
index 000000000..040a61534
--- /dev/null
+++ b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
@@ -0,0 +1,96 @@
+#!/bin/bash
+
+set -e
+
+set_fit_keys() {
+	local r="image-kernel-fit"
+	cs_define_role "${r}"
+
+	# HSM use case
+	cs_set_uri "${r}" "pkcs11:token=foo;object=kernel-fit"
+}
+
+import_fit_keys() {
+	local fit_cert_dir=fit
+	local r="image-kernel-fit"
+	cs_define_role "${r}"
+
+	cs_import_cert_from_der "${r}" "${fit_cert_dir}/fit-4096-development.crt"
+	cs_import_pubkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
+	cs_import_privkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
+}
+
+set_rauc_keys() {
+	local r="update"
+	cs_define_role "${r}"
+	cs_set_uri "${r}" "pkcs11:token=foo;object=rauc"
+	cs_append_ca_from_uri "${r}"
+}
+
+import_rauc_keys() {
+	local rauc_cert_dir=rauc
+	local r="update"
+	cs_define_role "${r}"
+
+	# SoftHSM use case
+	cs_import_cert_from_pem "${r}" "${rauc_cert_dir}/rauc.cert.pem"
+	cs_import_pubkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
+	cs_import_privkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
+
+	cs_append_ca_from_uri "${r}"
+}
+
+set_imx_habv4_keys() {
+	# HSM use case, assuming it contains only 1st CSF/IMG key
+	for i in 1 2 3 4; do
+		r="imx-habv4-srk${i}"
+		cs_define_role "${r}"
+		cs_set_uri "${r}" "pkcs11:token=foo;object=srk-release${i}"
+		cs_append_ca_from_uri "${r}"
+	done
+
+	r="imx-habv4-csf1"
+	cs_define_role ${r}
+	cs_set_uri "${r}" "pkcs11:token=foo;object=csf1"
+
+	r="imx-habv4-img1"
+	cs_define_role ${r}
+	cs_set_uri "${r}" "pkcs11:token=foo;object=img1"
+}
+
+import_imx_habv4_keys() {
+	local imx_habv4_key_dir="habv4"
+	local crts="${imx_habv4_key_dir}/crts"
+	local keys="${imx_habv4_key_dir}/keys"
+	local OPENSSL_KEYPASS="${imx_habv4_key_dir}/keys/key_pass.txt"
+
+	for i in 1 2 3 4; do
+		r="imx-habv4-srk${i}"
+		cs_define_role "${r}"
+		cs_import_cert_from_der "${r}" "${crts}/SRK${i}_sha256_4096_65537_v3_ca_crt.der"
+		cs_import_key_from_pem "${r}" "${keys}/SRK${i}_sha256_4096_65537_v3_ca_key.pem"
+		cs_append_ca_from_uri "${r}"
+
+		r="imx-habv4-csf${i}"
+		cs_define_role "${r}"
+		cs_import_cert_from_der "${r}" "${crts}/CSF${i}_1_sha256_4096_65537_v3_usr_crt.der"
+		cs_import_key_from_pem "${r}" "${keys}/CSF${i}_1_sha256_4096_65537_v3_usr_key.pem"
+
+		r="imx-habv4-img${i}"
+		cs_define_role "${r}"
+		cs_import_cert_from_der "${r}" "${crts}/IMG${i}_1_sha256_4096_65537_v3_usr_crt.der"
+		cs_import_key_from_pem "${r}" "${keys}/IMG${i}_1_sha256_4096_65537_v3_usr_key.pem"
+	done
+}
+
+
+# HSM use case
+#set_fit_keys
+#set_rauc_keys
+#set_imx_habv4_keys
+
+# or: SoftHSM use case
+#cs_init_softhsm
+#import_fit_keys
+#import_rauc_keys
+#import_imx_habv4_keys
diff --git a/rules/templates/template-code-signing-provider-choice-in b/rules/templates/template-code-signing-provider-choice-in
new file mode 100644
index 000000000..e2108f870
--- /dev/null
+++ b/rules/templates/template-code-signing-provider-choice-in
@@ -0,0 +1,5 @@
+## SECTION=code_signing_provider
+
+config CODE_SIGNING_PROVIDER_@PACKAGE@
+	bool
+	prompt "@package@"
diff --git a/rules/templates/template-code-signing-provider-in b/rules/templates/template-code-signing-provider-in
new file mode 100644
index 000000000..a0c61e6ef
--- /dev/null
+++ b/rules/templates/template-code-signing-provider-in
@@ -0,0 +1,16 @@
+## SECTION=code_signing
+
+config CODE_SIGNING
+	select HOST_@PACKAGE@_CODE_SIGNING if CODE_SIGNING_PROVIDER_@PACKAGE@
+
+config CODE_SIGNING_PROVIDER
+	default "@package@" if CODE_SIGNING_PROVIDER_@PACKAGE@
+
+config HOST_@PACKAGE@_CODE_SIGNING
+	bool
+	select HOST_OPENSC
+	select HOST_LIBP11
+	select HOST_OPENSSL
+	#select HOST_SOFTHSM
+	#select HOST_OPENSC_PCSC
+	#select HOST_EXTRACT_CERT
diff --git a/rules/templates/template-code-signing-provider-make b/rules/templates/template-code-signing-provider-make
new file mode 100644
index 000000000..94830d92e
--- /dev/null
+++ b/rules/templates/template-code-signing-provider-make
@@ -0,0 +1,41 @@
+# -*-makefile-*-
+#
+# Copyright (C) @YEAR@ by @AUTHOR@
+#
+# For further information about the PTXdist project and license conditions
+# see the README file.
+#
+
+#
+# We provide this package
+#
+HOST_PACKAGES-$(PTXCONF_HOST_@PACKAGE@_CODE_SIGNING) += host-@package@-code-signing
+
+#
+# Paths and names
+#
+HOST_@PACKAGE@_CODE_SIGNING_VERSION	:= @VERSION@
+HOST_@PACKAGE@_CODE_SIGNING		:= @package@-code-signing-$(HOST_@PACKAGE@_CODE_SIGNING_VERSION)
+HOST_@PACKAGE@_CODE_SIGNING_URL		:= file://local_src/@package@-code-signing
+HOST_@PACKAGE@_CODE_SIGNING_DIR		:= $(HOST_BUILDDIR)/$(HOST_@PACKAGE@_CODE_SIGNING)
+
+HOST_@PACKAGE@_CODE_SIGNING_CONF_TOOL	:= NO
+
+# ----------------------------------------------------------------------------
+# Compile
+# ----------------------------------------------------------------------------
+
+HOST_@PACKAGE@_CODE_SIGNING_MAKE_ENV	:= \
+	$(CODE_SIGNING_ENV)
+
+$(STATEDIR)/host-@package@-code-signing.compile:
+	@$(call targetinfo)
+	@$(call world/execute, HOST_@PACKAGE@_CODE_SIGNING, \
+		./ptxdist-set-keys.sh)
+	@$(call touch)
+
+$(STATEDIR)/host-@package@-code-signing.install:
+	@$(call targetinfo)
+	@$(call touch)
+
+# vim: syntax=make
diff --git a/scripts/lib/ptxd_lib_template.sh b/scripts/lib/ptxd_lib_template.sh
index f39e6e033..b89981f45 100644
--- a/scripts/lib/ptxd_lib_template.sh
+++ b/scripts/lib/ptxd_lib_template.sh
@@ -460,3 +460,19 @@ ptxd_template_new_blspec_entry() {
 export -f ptxd_template_new_blspec_entry
 ptxd_template_help_list[${#ptxd_template_help_list[@]}]="blspec-entry"
 ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create package for a bootloader spec entry"
+
+ptxd_template_new_code_signing_provider() {
+    export class="host-"
+    ptxd_template_read_basic &&
+    ptxd_template_read_author &&
+    package_filename="${package_filename}-code-signing"
+    ptxd_template_write_platform_rules
+    local template_file="$(ptxd_template_file "${template}-choice-in")"
+    local filename="${PTXDIST_PLATFORMCONFIGDIR}/platforms/${class}${package_filename}-choice.in"
+    ptxd_template_filter "${template_file}" "${filename}"
+    package="${package}-code-signing"
+    ptxd_template_write_src
+}
+export -f ptxd_template_new_code_signing_provider
+ptxd_template_help_list[${#ptxd_template_help_list[@]}]="code-signing-provider"
+ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create package for a code signing provider"
-- 
2.27.0


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

[ptxdist] [PATCH 2/5] doc: dev_manual: split up into multiple files

[Bastian Krause]

Split the lengthy developer's manual into multiple files to ease
navigation when editing.

No further change to the content.

Signed-off-by: Bastian Krause <bst@pengutronix.de>
---
 doc/dev_add_bin_only_files.rst       |  105 ++
 doc/dev_add_new_pkgs.rst             | 1339 +++++++++++++++++++
 doc/dev_create_new_pkg_templates.rst |   77 ++
 doc/dev_dir_hierarchy.rst            |  108 ++
 doc/dev_layers_in_ptxdist.rst        |  111 ++
 doc/dev_manual.rst                   | 1764 +-------------------------
 6 files changed, 1749 insertions(+), 1755 deletions(-)
 create mode 100644 doc/dev_add_bin_only_files.rst
 create mode 100644 doc/dev_add_new_pkgs.rst
 create mode 100644 doc/dev_create_new_pkg_templates.rst
 create mode 100644 doc/dev_dir_hierarchy.rst
 create mode 100644 doc/dev_layers_in_ptxdist.rst

diff --git a/doc/dev_add_bin_only_files.rst b/doc/dev_add_bin_only_files.rst
new file mode 100644
index 000000000..9031e437c
--- /dev/null
+++ b/doc/dev_add_bin_only_files.rst
@@ -0,0 +1,105 @@
+.. _adding_files:
+
+Adding Binary Only Files
+------------------------
+
+Sometimes a few binary files have to be added into the root filesystem.
+Or - to be more precise - some files, that do not need to be built in
+any way.
+
+On the other hand, sometimes files should be included that are not
+covered by any open source license and so, should not be shipped in the
+source code format.
+
+Add Binary Files File by File
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Doing to on a file by file base can happen by just using the ``install_copy``
+macro in the *targetinstall* stage in our own customized rules file.
+
+.. code-block:: none
+
+    @$(call install_copy, binary_example, 0, 0, 0644, \
+       </path/to/some/file/>ptx_logo.png, \
+       /example/ptx_logo.png)
+
+It copies the file ``ptx_logo.png`` from some location to target’s root
+filesystem. Refer :ref:`install_copy` for further information about using the
+``install_copy`` macro.
+
+The disadvantage of this method is: if we want to install more than one
+file, we need one call to the ``install_copy`` macro per file. This is
+even harder if not only a set of files is to be installed, but a whole
+directory tree with files instead.
+
+Add Binary Files via an Archive
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a whole tree of files is to be installed, working with a *tar* based
+archive could make life easier. In this case the archive itself provides
+all the required information the files are needing to be installed in a
+correct manner:
+
+-  the file itself and its name
+
+-  the directory structure and the final location of every file in this
+   structure
+
+-  user and group ID on a per file base
+
+.. code-block:: none
+
+    @$(call install_archive, binary_example, -, -, \
+       </path/to/an/>archive.tgz, /)
+
+Refer :ref:`install_archive` for further information about using the
+``install_archive`` macro.
+
+Using an archive can be useful to install parts of the root filesystem
+that are not covered by any open source license. Its possible to ship
+the binaries within the regular BSP, without the need for their sources.
+However it is possible for the customer to re-create everything required
+from the BSP to get their target up and running again.
+
+Another use case for the archive method could be the support for
+different development teams. One team provides a software component in
+the archive format, the other team does not need to build it but can use
+it in the same way than every other software component.
+
+Creating a Rules File
+~~~~~~~~~~~~~~~~~~~~~
+
+Let PTXdist create one for us.
+
+.. code-block:: text
+
+    $ ptxdist newpackage file
+
+    ptxdist: creating a new 'file' package:
+
+    ptxdist: enter package name.......: my_binfiles
+    ptxdist: enter version number.....: 1
+    ptxdist: enter package author.....: My Name <me@my-org.com>
+    ptxdist: enter package section....: rootfs
+
+Now two new files are present in the BSP:
+
+#. ``rules/my_binfiles.in`` The template for the menu
+
+#. ``rules/my_binfiles.make`` The rules template
+
+Both files now must be customized to meet our requirements. Due to the
+answer *rootfs* to the “``enter package section``” question, we will
+find the new menu entry in:
+
+.. code-block:: text
+
+    Root Filesystem --->
+    	< > my_binfiles (NEW)
+
+Enabling this new entry will also run our stages in
+``rules/my_binfiles.make`` the next time we enter:
+
+.. code-block:: text
+
+    $ ptxdist go
diff --git a/doc/dev_add_new_pkgs.rst b/doc/dev_add_new_pkgs.rst
new file mode 100644
index 000000000..4ae2765c2
--- /dev/null
+++ b/doc/dev_add_new_pkgs.rst
@@ -0,0 +1,1339 @@
+.. _adding_new_packages:
+
+Adding New Packages
+-------------------
+
+PTXdist provides a huge amount of applications sufficient for the most
+embedded use cases. But there is still need for some fancy new packages.
+This section describes the steps and the background on how to integrate
+new packages into the project.
+
+At first a summary about possible application types which PTXdist can
+handle:
+
+-  **host type**: This kind of package is built to run on the build
+   host. Most of the time such a package is needed if another
+   target-relevant package needs to generate some data. For example the
+   *glib* package depends on its own to create some data. But if it is
+   compiled for the target, it can’t do so. That’s why a host glib
+   package is required to provide these utilities runnable on the build
+   host. It sounds strange to build a host package, even if on the build
+   host such utilities are already installed. But this way ensures that
+   there are no dependencies regarding the build host system.
+
+-  **target type**: This kind of package is built for the target.
+
+-  **cross type**: This kind of package is built for the build host, but
+   creates architecture specific data for the target.
+
+-  **src-autoconf-prog**: This kind of package is built for the target.
+   It is intended for development, as it does not handle a released
+   archive but a plain source project instead. Creating such a package
+   will also create a small autotools based source template project on
+   demand to give the developer an easy point to start. This template is
+   prepared to build a single executable program. For further details refer
+   section :ref:`adding_src_autoconf_exec`.
+
+-  **src-autoconf-lib**: This kind of package is built for the target.
+   It is intended for development, as it does not handle a released
+   archive but a plain source project instead. Creating such a package
+   will also create a small autotools/libtool based source template
+   project on demand to give the developer an easy point to start. This
+   template is prepared to build a single shared library. For further
+   details refer section :ref:`adding_src_autoconf_lib`.
+
+-  **src-autoconf-proglib**: This kind of package is built for the
+   target. It is intended for development, as it does not handle a
+   released archive but a plain source project instead. Creating such a
+   package will also create a small autotools/libtool based template
+   project on demand to give the developer an easy point to start. This
+   template is prepared to build a single shared library and a single
+   executable program. The program will be linked against the shared
+   library. For further details refer section :ref:`adding_src_autoconf_exec_lib`.
+
+-  **file**: This kind of package is intended to add a few simple files
+   into the build process. We assume these files do not need any
+   processing, they are ready to use and must only be present in the
+   build process or at run-time (HTML files for example). Refer to the
+   section :ref:`adding_files` for further details on how to use
+   it.
+
+-  **src-make-prog**: This kind of package is built for the target. It’s
+   intended for development, as it does not handle a released archive
+   but a plain source project instead. Creating such a package will also
+   create a simple makefile-based template project the developer can use
+   as a starting point for development.
+
+-  **src-cmake-prog**: This kind of package is built for the target.
+   It’s intended for developments based on the *cmake* buildsystem.
+   Various projects are using *cmake* instead of *make* and can be built
+   with this package type. PTXdist will prepare it to compile sources in
+   accordance to the target libraries and their settings. Creating such
+   a package will also create a simple template project to be used as a
+   starting point for development.
+
+-  **src-qmake-prog**: This kind of package is built for the target.
+   It’s intended for developments based on the *qmake* buildsystem. If
+   the developer is going to develop a QT based application, this rule
+   is prepared to compile sources in accordance to the target libraries
+   and their settings. Creating such a package will also create a simple
+   template project to be used as a starting point for development.
+
+-  **src-meson-prog**: This kind of package is built for the target.
+   It’s intended for developments based on the *meson* buildsystem.
+   Various projects are using *meson* today and can be built
+   with this package type. PTXdist will prepare it to compile sources in
+   accordance to the target libraries and their settings. Creating such
+   a package will also create a simple template project to be used as a
+   starting point for development.
+
+-  **font**: This package is a helper to add X font files to the root
+   filesystem. This package does not create an additional IPKG, instead
+   it adds the font to the existing font IPKG. This includes the
+   generation of the directory index files, required by the Xorg
+   framework to recognize the font file.
+
+-  **src-linux-driver**: This kind of package builds an out of tree
+   kernel driver. It also creates a driver template to give the
+   developer an easy point to start.
+
+-  **kernel**: PTXdist comes with the ability to handle one kernel in its
+   platform. This type of package enables us to handle more than one kernel in
+   the project.
+
+-  **barebox**: PTXdist comes with the ability to handle one bootloader in its
+   platform. This type of package enables us to handle more than one bootloader
+   in the project.
+
+-  **image-tgz**: This kind of package creates a tar ball from a list of
+   packages. It is often uses as an input for other image packages.
+
+-  **image-genimage**: This kind of package can handle all kind of image
+   generation for almost every target independent of its complexity.
+
+-  **blspec-entry**: PTXdist comes with the ability to handle one bootspec in its
+   platform. This type of package enables us to handle more than one bootspec
+   in the project.
+
+.. _foo_example:
+
+Rule File Creation
+~~~~~~~~~~~~~~~~~~
+
+To create such a new package, we create a project local ``rules/``
+directory first. Then we run
+
+.. code-block:: text
+
+    $ ptxdist newpackage <package type>
+
+If we omit the <``package type``\ >, PTXdist will list all available
+package types.
+
+In our first example, we want to add a new target type archive package.
+When running the
+
+.. code-block:: text
+
+    $ ptxdist newpackage target
+
+command, PTXdist asks a few questions about this package. This
+information is the basic data PTXdist must know about the package.
+
+.. code-block:: text
+
+    ptxdist: creating a new 'target' package:
+
+    ptxdist: enter package name.......: foo
+    ptxdist: enter version number.....: 1.1.0
+    ptxdist: enter URL of basedir.....: http://www.foo.com/download/src
+    ptxdist: enter suffix.............: tar.gz
+    ptxdist: enter package author.....: My Name <me@my-org.com>
+    ptxdist: enter package section....: project_specific
+
+What we have to answer:
+
+-  **package name**: As this kind of package handles a source archive,
+   the correct answer here is the basename of the archive’s file name.
+   If its full name is ``foo-1.1.0.tar.gz``, then ``foo`` is the
+   basename to enter here.
+
+-  **version number**: Most source archives are using a release or
+   version number in their file name. If its full name is
+   ``foo-1.1.0.tar.gz``, then ``1.1.0`` is the version number to enter
+   here.
+
+-  **URL of basedir**: This URL tells PTXdist where to download the
+   source archive from the web (if not already done). If the full URL to
+   download the archive is
+   ``http://www.foo.com/download/src/foo-1.1.0.tar.gz``, the basedir
+   part ``http://www.foo.com/download/src`` is to be entered here.
+
+-  **suffix**: Archives are using various formats for distribution.
+   PTXdist uses the *suffix* entry to select the matching extraction
+   tool. If the archive’s full name is ``foo-1.1.0.tar.gz``, then
+   ``tar.gz`` is the suffix to enter here.
+
+-  **package author**: If we intend to contribute this new package to
+   PTXdist mainline, we should add our name here. This name will be used
+   in the copyright note of the rule file and will also be added to the
+   generated ipkg. When you run ``ptxdist setup`` prior to this call,
+   you can enter your name and your email address, so PTXdist will use
+   it as the default (very handy if you intend to add many new
+   packages).
+
+-  **package section**: We can enter here the menu section name where
+   our new package menu entry should be listed. In the first step we can
+   leave the default name unchanged. It’s a string in the menu file
+   only, so changing it later on is still possible.
+
+Make it Work
+~~~~~~~~~~~~
+
+Generating the rule file is only one of the required steps to get a new
+package. The next steps to make it work are to check if all stages are
+working as expected and to select the required parts to get them
+installed in the target root filesystem. Also we must find a reasonable
+location where to add our new menu entry to configure the package.
+
+The generated skeleton starts to add the new menu entry in the main
+configure menu (if we left the section name unchanged). Running
+``ptxdist menuconfig`` will show it on top of all other menus entries.
+
+.. important:: 
+  To be able to implement and test all the other required steps for adding
+  a new package, we first must enable the package for building. (Fine
+  tuning the menu can happen later on.)
+
+
+The rule file skeleton still lacks some important information. Let’s
+take a look into some of the top lines of the generated rule file
+``./rules/foo.make``:
+
+.. code-block:: make
+
+    FOO_VERSION	:= 1.1.0
+    FOO_MD5	:=
+    FOO		:= foo-$(FOO_VERSION)
+    FOO_SUFFIX	:= tar.gz
+    FOO_URL	:= http://www.foo.com/download/src/$(FOO).$(FOO_SUFFIX)
+    FOO_SOURCE	:= $(SRCDIR)/$(FOO).$(FOO_SUFFIX)
+    FOO_DIR	:= $(BUILDDIR)/$(FOO)
+    FOO_LICENSE	:= unknown
+
+We can find these lines with different content in most or all of the
+other rule files PTXdist comes with. Up to the underline character is
+always the package name and after the underline character is always
+PTXdist specific. What does it mean:
+
+-  ``*_VERSION`` brings in the version number of the release and is used
+   for the download and IPKG/OPKG package generation.
+
+-  ``*_MD5`` to be sure the correct package has been downloaded, PTXdist
+   checks the given MD5 sum against the archive content. If both sums do
+   not match, PTXdist rejects the archive and fails the currently
+   running build.
+
+-  ``*_SUFFIX`` defines the archive type, to make PTXdist choosing the
+   correct extracting tool.
+
+-  ``*_URL`` defines the full qualified URL into the web for download. If
+   alternative download locations are known, they can be listed in this
+   variable, delimiter character is the space.
+
+-  ``*_SOURCE`` tells PTXdist where to store the downloaded package.
+
+-  ``*_DIR`` points to the directory this package will be built later on
+   by PTXdist.
+
+-  ``*_LICENSE`` enables the user to get a list of licenses she/he is
+   using in her/his project (licenses of the enabled packages).
+
+After enabling the menu entry, we can start to check the *get* and
+*extract* stages, calling them manually one after another.
+
+.. note:: The shown commands below expect that PTXdist downloads the
+  archives to a global directory named ``global_src``. This is not the
+  default setting, but we recommend to use a global directory to share all
+  archives between PTXdist based projects. Advantage is every download
+  happens only once. Refer to the ``setup`` command PTXdist provides.
+
+.. code-block:: text
+
+    $ ptxdist get foo
+
+    ---------------------------
+    target: foo-1.1.0.tar.gz
+    ---------------------------
+
+    --2009-12-21 10:54:45--  http://www.foo.com/download/src/foo-1.1.0.tar.gz
+    Length: 291190 (284K) [application/x-gzip]
+    Saving to: `/global_src/foo-1.1.0.tar.gz.XXXXOGncZA'
+
+    100%[======================================>] 291,190      170K/s   in 1.7s
+
+    2009-12-21 10:54:48 (170 KB/s) - `/global_src/foo-1.1.0.tar.gz' saved [291190/291190]
+
+This command should start to download the source archive. If it fails,
+we should check our network connection, proxy setup or if the given URL
+in use is correct.
+
+.. note:: Sometimes we do not know the content of all the other variables in
+  the rule file. To get an idea what content a variable has, we can ask
+  PTXdist about it:
+
+.. code-block:: text
+
+    $ ptxdist print FOO_URL
+    http://www.foo.com/download/src/foo-1.1.0.tar.gz
+
+The next step would be to extract the archive. But as PTXdist checks the
+MD5 sum in this case, this step will fail, because the ``FOO_MD5``
+variable is still empty. Let’s fill it:
+
+.. code-block:: text
+
+    $ md5sum /global_src/foo-1.1.0.tar.gz
+    9a09840ab775a139ebb00f57a587b447
+
+This string must be assigned to the FOO_MD5 in our new ``foo.make``
+rule file:
+
+.. code-block:: text
+
+    FOO_MD5		:= 9a09840ab775a139ebb00f57a587b447
+
+We are now prepared for the next step:
+
+.. code-block:: text
+
+    $ ptxdist extract foo
+
+    -----------------------
+    target: foo.extract
+    -----------------------
+
+    extract: archive=/global_src/foo-1.1.0.tar.gz
+    extract: dest=/home/jbe/my_new_prj/build-target
+    PATCHIN: packet=foo-1.1.0
+    PATCHIN: dir=/home/jbe/my_new_prj/build-target/foo-1.1.0
+    PATCHIN: no patches for foo-1.1.0 available
+    Fixing up /home/jbe/my_new_prj/build-target/foo-1.1.0/configure
+    finished target foo.extract
+
+In this example we expect an autotoolized source package. E.g. to
+prepare the build, the archive comes with a ``configure`` script. This
+is the default case for PTXdist. So, there is no need to modify the rule
+file and we can simply run:
+
+.. code-block:: text
+
+    $ ptxdist prepare foo
+
+    -----------------------
+    target: foo.prepare
+    -----------------------
+
+    [...]
+
+    checking build system type... i686-host-linux-gnu
+    checking host system type... |ptxdistCompilerName|
+    checking whether to enable maintainer-specific portions of Makefiles... no
+    checking for a BSD-compatible install... /usr/bin/install -c
+    checking whether build environment is sane... yes
+    checking for a thread-safe mkdir -p... /bin/mkdir -p
+    checking for gawk... gawk
+    checking whether make sets $(MAKE)... yes
+    checking for |ptxdistCompilerName|-strip... |ptxdistCompilerName|-strip
+    checking for |ptxdistCompilerName|-gcc... |ptxdistCompilerName|-gcc
+    checking for C compiler default output file name... a.out
+
+    [...]
+
+    configure: creating ./config.status
+    config.status: creating Makefile
+    config.status: creating ppa_protocol/Makefile
+    config.status: creating config.h
+    config.status: executing depfiles commands
+    finished target foo.prepare
+
+At this stage things can fail:
+
+-  A wrong or no MD5 sum was given
+
+-  The ``configure`` script is not cross compile aware
+
+-  The package depends on external components (libraries for example)
+
+If the ``configure`` script is not cross compile aware, we are out of
+luck. We must patch the source archive in this case to make it work.
+Refer to the section :ref:`configure_rebuild` on how to use
+PTXdist’s features to simplify this task.
+If the package depends on external components, these components might
+be already part of PTXdist. In this case we just have to add this
+dependency into the menu file and we are done. But if PTXdist cannot
+fulfill this dependency, we also must add it as a separate package
+first.
+
+If the *prepare* stage has finished successfully, the next step is to
+compile the package.
+
+.. code-block:: text
+
+    $ ptxdist compile foo
+
+    -----------------------
+    target: foo.compile
+    -----------------------
+
+    make[1]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+    make  all-recursive
+    make[2]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+    make[3]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+
+    [...]
+
+    make[3]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+    make[2]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+    make[1]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+    finished target foo.compile
+
+At this stage things can fail:
+
+-  The build system is not cross compile aware (it tries to execute just
+   created target binaries for example)
+
+-  The package depends on external components (libraries for example)
+   not detected by ``configure``
+
+-  Sources are ignoring the endianness of some architectures or using
+   header files from the build host system (from ``/usr/include`` for
+   example)
+
+-  The linker uses libraries from the build host system (from
+   ``/usr/lib`` for example) by accident
+
+In all of these cases we must patch the sources to make them work. Refer
+to section :ref:`patching_packages` on how to use PTXdist’s
+features to simplify this task.
+
+In this example we expect the best case: everything went fine, even for
+cross compiling. So, we can continue with the next stage: *install*
+
+.. code-block:: text
+
+    $ ptxdist install foo
+
+    -----------------------
+    target: foo.install
+    -----------------------
+
+    make[1]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+    make[2]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+    make[3]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+    test -z "/usr/bin" || /bin/mkdir -p "/home/jbe/my_new_prj/build-target/foo-1.1.0/usr/bin"
+      /usr/bin/install -c 'foo' '/home/jbe/my_new_prj/build-target/foo-1.1.0/usr/bin/foo'
+    make[3]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+    make[2]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+    make[1]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
+    finished target foo.install
+
+    ----------------------------
+    target: foo.install.post
+    ----------------------------
+
+    finished target foo.install.post
+
+This *install* stage does not install anything to the target root
+filesystem. It is mostly intended to install libraries and header files
+other programs should link against later on.
+
+The last stage – *targetinstall* – is the one that defines the package’s
+components to be forwarded to the target’s root filesystem. Due to the
+absence of a generic way, this is the task of the developer. So, at this
+point of time we must run our favourite editor again and modify our new
+rule file ``./rules/foo.make``.
+
+The skeleton for the *targetinstall* stage looks like this:
+
+.. code-block:: make
+
+    # ----------------------------------------------------------------------------
+    # Target-Install
+    # ----------------------------------------------------------------------------
+
+    $(STATEDIR)/foo.targetinstall:
+    	@$(call targetinfo)
+
+    	@$(call install_init,  foo)
+    	@$(call install_fixup, foo,PACKAGE,foo)
+    	@$(call install_fixup, foo,PRIORITY,optional)
+    	@$(call install_fixup, foo,VERSION,$(FOO_VERSION))
+    	@$(call install_fixup, foo,SECTION,base)
+    	@$(call install_fixup, foo,AUTHOR,"My Name <me@my-org.com>")
+    	@$(call install_fixup, foo,DEPENDS,)
+    	@$(call install_fixup, foo,DESCRIPTION,missing)
+
+    	@$(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/foobar, /dev/null)
+
+    	@$(call install_finish, foo)
+    	@$(call touch)
+
+The “header” of this stage defines some information IPKG needs. The
+important part that we must modify is the call to the ``install_copy``
+macro (refer to section :ref:`reference_macros` for more details
+about this kind of macros). This call instructs PTXdist to include the
+given file (with UID, GID and permissions) into the IPKG, which means to
+install this file to the target’s root filesystem.
+
+From the previous *install* stage we know this package installs an
+executable called ``foo`` to location ``/usr/bin``. We can do the same
+for our target by changing the *install\_copy* line to:
+
+.. code-block:: none
+
+    @$(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/foo, /usr/bin/foo)
+
+To check it, we just run:
+
+.. code-block:: text
+
+    $ ptxdist targetinstall foo
+
+    -----------------------------
+    target: foo.targetinstall
+    -----------------------------
+
+    install_init:   preparing for image creation...
+    install_init:   @ARCH@ -> i386 ... done
+    install_init:   preinst not available
+    install_init:   postinst not available
+    install_init:   prerm not available
+    install_init:   postrm not available
+    install_fixup:  @PACKAGE@ -> foo ... done.
+    install_fixup:  @PRIORITY@ -> optional ... done.
+    install_fixup:  @VERSION@ -> 1.1.0 ... done.
+    install_fixup:  @SECTION@ -> base ... done.
+    install_fixup:  @AUTHOR@ -> "My Name <me\@my-org.com>" ... done.
+    install_fixup:  @DESCRIPTION@ -> missing ... done.
+    install_copy:
+      src=/home/jbe/my_new_prj/build-target/foo-1.1.0/foo
+      dst=/usr/bin/foo
+      owner=0
+      group=0
+      permissions=0755
+    xpkg_finish:    collecting license (unknown) ... done.
+    xpkg_finish:    creating ipkg package ... done.
+    finished target foo.targetinstall
+
+    ----------------------------------
+    target: foo.targetinstall.post
+    ----------------------------------
+
+    finished target foo.targetinstall.post
+
+After this command, the target’s root filesystem contains a file called
+``/usr/bin/foo`` owned by root, its group is also root and everyone has
+execution permissions, but only the user root has write permissions.
+
+One last task of this port is still open: A reasonable location for
+the new menu entry in PTXdist’s menu hierarchy. PTXdist arranges its
+menus on the meaning of each package. Is it a network related tool? Or
+a scripting language? Or a graphical application?
+Each of these global meanings has its own submenu, where we can add
+our new entry to. We just have to edit the head of our new menu file
+``./rules/foo.in`` to add it to a specific global menu. If our new
+package is a network related tool, the head of the menu file should
+look like:
+
+.. code-block:: kconfig
+
+    ## SECTION=networking
+
+We can grep through the other menu files from the PTXdist main
+installation ``rules/`` directory to get an idea what section names are
+available:
+
+.. code-block:: text
+
+    rules/ $ find . -name \*.in | xargs grep "## SECTION"
+    ./acpid.in:## SECTION=shell_and_console
+    ./alsa-lib.in:## SECTION=system_libraries
+    ./alsa-utils.in:## SECTION=multimedia_sound
+    ./apache2.in:## SECTION=networking
+    ./apache2_mod_python.in:## SECTION=networking
+    [...]
+    ./xkeyboard-config.in:## SECTION=multimedia_xorg_data
+    ./xorg-app-xev.in:## SECTION=multimedia_xorg_app
+    ./xorg-app-xrandr.in:## SECTION=multimedia_xorg_app
+    ./host-eggdbus.in:## SECTION=hosttools_noprompt
+    ./libssh2.in:## SECTION=networking
+
+Porting a new package to PTXdist is (almost) finished now.
+
+To check it right away, we simply run these two commands:
+
+.. code-block:: text
+
+    $ ptxdist clean foo
+    rm -rf /home/jbe/my_new_prj/state/foo.*
+    rm -rf /home/jbe/my_new_prj/packages/foo_*
+    rm -rf /home/jbe/my_new_prj/build-target/foo-1.1.0
+    $ ptxdist targetinstall foo
+
+    [...]
+
+.. important:: Discover somehow hidden dependencies with one more last check!
+
+Up to this point all the development of the new package was done in an already
+built BSP. Doing so sometimes somehow hidden dependencies cannot be seen:
+everything seems fine, the new package builds always successfully and the
+results are working on the target.
+
+So to check for this kind of dependencies there is still one more final check
+to do (even if its boring and takes time):
+
+.. code-block:: text
+
+    $ ptxdist clean
+    [...]
+    $ ptxdist targetinstall foo
+    [...]
+
+This will re-start with a **clean** BSP and builds exactly the new package and
+its (known) dependencies. If this builds successfully as well we are really done
+with the new package.
+
+Some Notes about Licenses
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The already mentioned rule variable ``*_LICENSE`` (e.g. ``FOO_LICENSE`` in our
+example) is very important and must be filled by the developer of the package.
+Many licenses bring in obligations using the corresponding package (*attribution*
+for example). To make life easier for everybody the license for a package must
+be provided. *SPDX* license identifiers unify the license names and are used
+in PTXdist to identify license types and obligations.
+
+If a package comes with more than one license, all of their SPDX identifiers
+must be listed and connected with the keyword ``AND``. If your package comes
+with GPL-2.0 and LGPL-2.1 licenses, the definition should look like this:
+
+.. code-block:: make
+
+   FOO_LICENSE := GPL-2.0 AND LGPL-2.1
+
+One specific obligation cannot be detected examining the SPDX license identifiers
+by PTXdist: *the license choice*. In this case all licenses of choice must be
+listed and connected by the keyword ``OR``.
+
+If, for example, your obligation is to select one of the licenses *GPL-2.0* **or**
+*GPL-3.0*, the ``*_LICENSE`` variable should look like this:
+
+.. code-block:: make
+
+   FOO_LICENSE := GPL-2.0 OR GPL-3.0
+
+SPDX License Identifiers
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+A list of SPDX license identifiers can be found here:
+
+   https://spdx.org/licenses/
+
+Help to Detect the Correct License
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+License identification isn't trivial. A help in doing so can be the following
+repository and its content. It contains a list of known licenses based on their
+SPDX identifier. The content is without formatting to simplify text search.
+
+   https://github.com/spdx/license-list-data/tree/master/text
+
+Advanced Rule Files
+~~~~~~~~~~~~~~~~~~~
+
+The previous example on how to create a rule file sometimes works as
+shown above. But most of the time source archives are not that simple.
+In this section we want to give the user a more detailed selection how
+the package will be built.
+
+Adding Static Configure Parameters
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``configure`` scripts of various source archives provide additional
+parameters to enable or disable features, or to configure them in a
+specific way.
+
+We assume the ``configure`` script of our ``foo`` example (refer to
+section :ref:`foo_example`) supports two additional parameters:
+
+-  **--enable-debug**: Make the program more noisy. It’s disabled by
+   default.
+
+-  **--with-bar**: Also build the special executable **bar**. Building
+   this executable is also disabled by default.
+
+We now want to forward these options to the ``configure`` script when it
+runs in the *prepare* stage. To do so, we must again open the rule file
+with our favourite editor and navigate to the *prepare* stage entry.
+
+PTXdist uses the variable ``FOO_CONF_OPT`` as the list of parameters to
+be given to ``configure``.
+
+Currently this variable is commented out and defined to:
+
+.. code-block:: make
+
+    # FOO_CONF_OPT := $(CROSS_AUTOCONF_USR)
+
+The variable ``CROSS_AUTOCONF_USR`` is predefined by PTXdist and
+contains all basic parameters to instruct ``configure`` to prepare for a
+**cross** compile environment.
+
+To use the two additional mentioned ``configure`` parameters, we comment
+in this line and supplement this expression as follows:
+
+.. code-block:: make
+
+    FOO_CONF_OPT := \
+        $(CROSS_AUTOCONF_USR) \
+        --enable-debug \
+        --with-bar
+
+.. note:: We recommend to use this format with each parameter on a line of
+ its own. This format is easier to read and a diff shows more exactly any
+ change.
+
+To do a fast check if this addition was successful, we run:
+
+.. code-block:: text
+
+    $ ptxdist print FOO_CONF_OPT
+    --prefix=/usr --sysconfdir=/etc --host=|ptxdistCompilerName| --build=i686-host-linux-gnu --enable-debug --with-bar
+
+.. note:: It depends on the currently selected platform and its architecture
+ what content this variable will have. The content shown above is an
+ example for a target.
+
+Or re-build the package with the new settings:
+
+.. code-block:: text
+
+    $ ptxdist drop foo prepare
+    $ ptxdist targetinstall foo
+
+Adding Dynamic Configure Parameters
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Sometimes it makes sense to add this kind of parameters on demand only;
+especially a parameter like ``--enable-debug``. To let the user decide
+if this parameter is to be used or not, we must add a menu entry. So,
+let’s expand our menu. Here is its current content:
+
+.. code-block:: kconfig
+
+    ## SECTION=project_specific
+
+    config FOO
+            tristate
+            prompt "foo"
+            help
+              FIXME
+
+We’ll add two menu entries, one for each optional parameter we want to
+add on demand to the ``configure`` parameters:
+
+.. code-block:: kconfig
+
+    ## SECTION=project_specific
+
+    config FOO
+           tristate
+           prompt "foo"
+           help
+             FIXME
+
+    if FOO
+    config FOO_DEBUG
+           bool
+           prompt "add debug noise"
+
+    config FOO_BAR
+           bool
+           prompt "build bar"
+
+    endif
+
+.. important:: Always follow the rule to extend the base name by a suboption
+  name as the trailing part of the variable name. This gives PTXdist the ability
+  to detect a change in the package’s settings (via menuconfig) to force its
+  rebuild on demand.
+
+To make usage of the new menu entries, we must check them in the rule
+file and add the correct parameters:
+
+.. code-block:: make
+
+    #
+    # autoconf
+    #
+    FOO_CONF_OPT := \
+        $(CROSS_AUTOCONF_USR) \
+        --$(call ptx/endis, PTXCONF_FOO_DEBUG)-debug \
+        --$(call ptx/wwo, PTXCONF_FOO_BAR)-bar
+
+.. important:: Please note the leading ``PTXCONF_`` for each define. While Kconfig is
+  using ``FOO_BAR``, the rule file must use ``PTXCONF_FOO_BAR`` instead.
+
+.. note:: Refer :ref:`Rule File Macro Reference <param_macros>` for further
+   details about these special kind of option macros (e.g. ``ptx/...``).
+
+It is a good practice to always add both settings, e.g. ``--disable-debug``
+even if this is the default case. Sometimes ``configure`` tries to guess
+something and the binary result might differ depending on the build
+order. For example some kind of package would also build some X related
+tools, if X libraries are found. In this case it depends on the build
+order, if the X related tools are built or not. All the autocheck
+features are problematic here. So, if we do not want ``configure`` to
+guess its settings we **must disable everything we do not want**.
+
+To support this process, PTXdist supplies a helper script, located at
+``/path/to/ptxdist/scripts/configure-helper.py`` that compares the configure
+output with the settings from ``FOO_CONF_OPT``:
+
+.. code-block:: text
+
+    $ /opt/ptxdist-2017.06.0/scripts/configure-helper.py -p libsigrok
+    --- rules/libsigrok.make
+    +++ libsigrok-0.5.0
+    @@ -4,3 +4,74 @@
+     	--libdir=/usr/lib
+     	--build=x86_64-host-linux-gnu
+     	--host=arm-v7a-linux-gnueabihf
+    +	--enable-warnings=min|max|fatal|no
+    +	--disable-largefile
+    +	--enable-all-drivers
+    +	--enable-agilent-dmm
+    [...]
+    +	--enable-ruby
+    +	--enable-java
+    +	--without-libserialport
+    +	--without-libftdi
+    +	--without-libusb
+    +	--without-librevisa
+    +	--without-libgpib
+    +	--without-libieee1284
+    +	--with-jni-include-path=DIR-LIST
+
+In this example, many configure options from libsigrok (marked with ``+``)
+are not yet present in ``LIBSIGROK_CONF_OPT`` and must be added, possibly also
+by providing more dynamic options in the package definition.
+
+If some parts of a package are built on demand only, they must also be
+installed on demand only. Besides the *prepare* stage, we also must
+modify our *targetinstall* stage:
+
+.. code-block:: make
+
+    	@$(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/foo, /usr/bin/foo)
+
+    ifdef PTXCONF_FOO_BAR
+    	@$(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/bar, /usr/bin/bar)
+    endif
+
+    	@$(call install_finish, foo)
+    	@$(call touch)
+
+Now we can play with our new menu entries and check if they are working
+as expected:
+
+.. code-block:: text
+
+    $ ptxdist menuconfig
+    $ ptxdist targetinstall foo
+
+Whenever we change a *FOO* related menu entry, PTXdist should detect it
+and re-build the package when a new build is started.
+
+.. _external_dependencies:
+
+Managing External Compile Time Dependencies
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+While running the prepare stage, it could happen that it fails due to a
+missing external dependency.
+
+For example:
+
+.. code-block:: text
+
+    checking whether zlib exists....failed
+
+In this example, our new package depends on the compression library
+*zlib*. PTXdist comes with a target *zlib*. All we need to do in this
+case is to declare that our new package *foo* depends on *zlib*. This
+kind of dependency is managed in the menu file of our new package by
+simply adding the ``select ZLIB`` line. After this addition our menu
+file looks like:
+
+.. code-block:: kconfig
+
+    ## SECTION=project_specific
+
+    config FOO
+           tristate
+           select ZLIB
+           prompt "foo"
+           help
+             FIXME
+
+    if FOO
+    config FOO_DEBUG
+           bool
+           prompt "add debug noise"
+
+    config FOO_BAR
+           bool
+           prompt "build bar"
+
+    endif
+
+PTXdist now builds the *zlib* first and our new package thereafter.
+
+Refer :ref:`external_dependencies_variants` for more specific dependency
+description.
+
+Managing External Compile Time Dependencies on Demand
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It is good practice to add only those dependencies that are really
+required for the current configuration of the package. If the package
+provides the features *foo* and *bar* and its ``configure`` provides
+switches to enable/disable them independently, we can also add
+dependencies on demand. Let’s assume feature *foo* needs the compression
+library *libz* and *bar* needs the XML2 library *libxml2*. These
+libraries are only required at run-time if the corresponding feature is
+enabled. To add these dependencies on demand, the menu file looks like:
+
+.. code-block:: kconfig
+
+    ## SECTION=project_specific
+
+    config FOO
+           tristate
+           select ZLIB if FOO_FOO
+           select LIBXML2 if FOO_BAR
+           prompt "foo"
+           help
+             FIXME
+
+    if FOO
+    config FOO_DEBUG
+           bool
+           prompt "add debug noise"
+
+    config FOO_FOO
+           bool
+           prompt "build foo"
+
+    config FOO_BAR
+           bool
+           prompt "build bar"
+
+    endif
+
+.. important:: Do not add these ``select`` statements to the corresponding menu entry.
+  They must belong to the main menu entry of the package to ensure that
+  the calculation of the dependencies between the packages is done in a
+  correct manner.
+
+Managing External Runtime Dependencies
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Some packages are building all of their components and also installing
+them into the target’s sysroot. But only their *targetinstall* stage
+decides which parts are copied to the root filesystem. So, compiling and
+linking of our package will work, because everything required is found
+in the target’s sysroot.
+
+In our example there is a hidden dependency to the math library
+``libm``. Our new package was built successfully, because the linker was
+able to link our binaries against the ``libm`` from the toolchain. But
+in this case the ``libm`` must also be available in the target’s root
+filesystem to fulfil the run-time dependency: We have to force PTXdist to
+install ``libm``. ``libm`` is part of the *glibc* package, but is not
+installed by default (to keep the root filesystem small). So, it **does
+not** help to select the ``GLIBC`` symbol, to get a ``libm`` at run-time.
+
+The correct solution here is to add a ``select LIBC_M`` to our menu
+file. With all the additions above it now looks like:
+
+.. code-block:: kconfig
+
+    ## SECTION=project_specific
+
+    config FOO
+           tristate
+           select ZLIB if FOO_FOO
+           select LIBXML2 if FOO_BAR
+           select LIBC_M
+           prompt "foo"
+           help
+             FIXME
+
+    if FOO
+    config FOO_DEBUG
+           bool
+           prompt "add debug noise"
+
+    config FOO_FOO
+           bool
+           prompt "build foo"
+
+    config FOO_BAR
+           bool
+           prompt "build bar"
+
+    endif
+
+.. note:: There are other packages around, that do not install everything by
+  default. If our new package needs something special, we must take a look
+  into the menu of the other package how to force the required components
+  to be installed and add the corresponding ``selects`` to our own menu
+  file. In this case it does not help to enable the required parts in our
+  project configuration, because this has no effect on the build order!
+
+Managing Plain Makefile Packages
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Many packages are still coming with a plain ``Makefile``. The user has
+to adapt it to make it work in a cross compile environment as well.
+PTXdist can also handle this kind of packages. We only have to specify
+a special *prepare* and *compile* stage.
+
+Such packages often have no special need for any kind of preparation. In
+this we must instruct PTXdist to do nothing in the *prepare* stage:
+
+.. code-block:: make
+
+    FOO_CONF_TOOL := NO
+
+To compile the package, we can use ``make``\ ’s feature to overwrite
+variables used in the ``Makefile``. With this feature we can still use
+the original ``Makefile`` but with our own (cross compile) settings.
+
+Most of the time the generic compile rule can be used, only a few
+settings are required. For a well defined ``Makefile`` it is sufficient to
+set up the correct cross compile environment for the *compile* stage:
+
+.. code-block:: make
+
+    FOO_MAKE_ENV := $(CROSS_ENV)
+
+``make`` will be called in this case with:
+
+``$(FOO_MAKE_ENV) $(MAKE) -C $(FOO_DIR) $(FOO_MAKE_OPT)``
+
+So, in the rule file only the two variables ``FOO_MAKE_ENV`` and
+``FOO_MAKE_OPT`` must be set, to forward the required settings to the
+package’s buildsystem. If the package cannot be built in parallel, we
+can also add the ``FOO_MAKE_PAR := NO``. ``YES`` is the default.
+
+Managing CMake/QMake/Meson Packages
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Building packages that use ``cmake``, ``qmake`` or ``meson`` is much like
+building packages with an autotools based buildsystem. We need to specify
+the configuration tool:
+
+.. code-block:: make
+
+    FOO_CONF_TOOL := cmake
+
+or
+
+.. code-block:: make
+
+    FOO_CONF_TOOL := qmake
+
+or respectively
+
+.. code-block:: make
+
+    FOO_CONF_TOOL := meson
+
+And provide the correct configuration options. The syntax is different so
+PTXdist provides additional macros to simplify configurable features.
+For ``cmake`` the configuration options typically look like this:
+
+.. code-block:: make
+
+    FOO_CONF_OPT := \
+    	$(CROSS_CMAKE_USR) \
+    	-DBUILD_TESTS:BOOL=OFF \
+    	-DENABLE_BAR:BOOL=$(call ptx/onoff, PTXCONF_FOO_BAR)
+
+For ``qmake`` the configuration options typically look like this:
+
+.. code-block:: make
+
+    FOO_CONF_OPT := \
+    	$(CROSS_QMAKE_OPT) \
+    	PREFIX=/usr
+
+And for ``meson`` the configuration options typically look like this:
+
+.. code-block:: make
+
+    FOO_CONF_OPT := \
+    	$(CROSS_MESON_USR) \
+    	-Dbar=$(call ptx/truefalse,PTXCONF_FOO_BAR)
+
+Please note that currently only host and target ``cmake``\/``meson`` packages
+and only target ``qmake`` packages are supported.
+
+Managing Python Packages
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+As with any other package, the correct configuration tool must be selected
+for Python packages:
+
+.. code-block:: make
+
+    FOO_CONF_TOOL := python
+
+.. note:: For Python3 packages the value must be ``python3``.
+
+No Makefiles are used when building Python packages so the usual ``make``
+and ``make install`` for the *compile* and *install* stages cannot be used.
+PTXdist will call ``python setup.py build`` and ``python setup.py install``
+instead.
+
+.. note:: *FOO* is still the name of our example package. It must be
+  replaced by the real package name.
+
+
+.. _patching_packages:
+
+Patching Packages
+~~~~~~~~~~~~~~~~~
+
+There can be various reasons why a package must be patched:
+
+-  Package is broken for cross compile environments
+
+-  Package is broken within a specific feature
+
+-  Package is vulnerable and needs some fixes
+
+-  or anything else (this case is the most common one)
+
+Ideally, those problems should be addressed in the original project,
+so any patches you add to your BSP or to PTXdist should also be submitted upstream.
+The upstream project can often provide better feedback, they can integrate your
+patch into a new release, and also maintain your changes as part of the project.
+This way we make sure that all advantages of the open source idea work for us;
+and your patch can be removed again later when a new release of the project is
+integrated into your BSP or into PTXdist.
+
+PTXdist handles patching automatically.
+After extracting the archive of a package, PTXdist checks for the existence of
+a patch directory named like its ``<PKG>_PATCHES`` variable, or, if this variable
+is not set, like its ``<PKG>`` variable.
+The patch directory is then searched in all locations listed by the
+``PTXDIST_PATH_PATCHES`` variable, and the first one found is used.
+Take an exemplary package ``foo`` with version ``1.1.0``:
+The variable ``FOO`` will have the value ``foo-1.1.0``, so PTXdist will look for
+a patch directory named ``foo-1.1.0`` in the following locations:
+
+#. the current layer:
+
+   a. project (``./patches/foo-1.1.0``)
+   b. platform (``./configs/|ptxdistPlatformConfigDir|/patches/foo-1.1.0``)
+
+#. any :ref:`base layers <layers-in-ptxdist>`,
+   applying the same search order as above for each layer recursively
+
+#. ptxdist (``<ptxdist/installation/path>/patches/foo-1.1.0``)
+
+The patches from the first location found are used. Note: Due to this
+search order, a PTXdist project can replace global patches from the
+PTXdist installation. This can be useful if a project sticks to a
+specific PTXdist revision but fixes from a more recent revision of
+PTXdist should be used.
+
+PTXdist uses the utilities *git*, *patch* or *quilt* to work with
+patches or patch series. We recommend *git*, as it can manage patch
+series in a very easy way.
+
+Creating a Patch Series for a Package
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To create a patch series for the first time, we can run the following
+steps. We are still using our *foo-1.1.0* example package here:
+
+Using Quilt
+"""""""""""
+
+We create a special directory for the patch series in the local project
+directory:
+
+.. code-block:: text
+
+    $ mkdir -p patches/foo-1.1.0
+
+PTXdist expects a ``series`` file in the patch directory and at least
+one patch. Otherwise it fails. Due to the fact that we do not have any
+patch content yet, we’ll start with a dummy entry in the ``series`` file
+and an empty ``patch`` file.
+
+.. code-block:: text
+
+    $ touch patches/foo-1.1.0/dummy
+    $ echo dummy > patches/foo-1.1.0/series
+
+Next is to extract the package (if already done, we must remove it
+first):
+
+.. code-block:: text
+
+    $ ptxdist extract foo
+
+This will extract the archive and create a symbolic link in the build
+directory pointing to our local patch directory. Working this way will
+ensure that we do not lose our created patches if we enter
+``ptxdist clean foo`` by accident. In our case the patches are still
+present in ``patches/foo-1.1.0`` and can be used the next time we
+extract the package again.
+
+All we have to do now is to do the modification we need to make the
+package work. We change into the build directory and use quilt_ to
+create new patches, add files to respective patches, modify these files
+and refresh the patches to save our changes.
+See the *quilt* documentation (``man 1 quilt``) for more information.
+
+.. note:: For patches that are intended for PTXdist upstream use the git
+  workflow described below to get proper patch headers.
+
+.. _quilt: http://savannah.nongnu.org/projects/quilt
+
+Using Git
+"""""""""
+
+Create the patch directory like above for *quilt*,
+but only add an empty series file:
+
+.. code-block:: text
+
+    $ mkdir -p patches/foo-1.1.0
+    $ touch patches/foo-1.1.0/series
+
+Then extract the package with an additional command line switch:
+
+.. code-block:: text
+
+    $ ptxdist --git extract foo
+
+The empty series file makes PTXdist create a Git repository in the
+respective package build directory,
+and import the package source as the first commit.
+
+.. note:: Optionally, you can enable the setting *Developer Options →
+  use git to apply patches* in `ptxdist setup` to get this behaviour
+  as a default for every package.
+  However, note that this setting is meant for development only, and can lead
+  to failures – some packages try to determine if they are being compiled from
+  a Git source tree, and behave differently in that case.
+
+Then you can change into the package build directory
+(``platform-<name>/build-target/foo-1.1.0``),
+patch the required source files,
+and make Git commits on the way.
+The Git history should now look something like this:
+
+.. code-block:: text
+
+    $ git log --oneline --decorate
+    * df343e821851 (HEAD -> master) Makefile: don't build the tests
+    * 65a360c2bd60 strfry.c: frobnicate the excusator
+    * fdc315f6844c (tag: foobar-1.1.0, tag: base) initial commit
+
+Finally, call ``git ptx-patches`` to transform those Git commits into the patch
+series in the ``patches/foo-1.1.0`` folder.
+This way they don't get lost when cleaning the package.
+
+.. note:: PTXdist will only create a Git repository for packages with
+  patches.  To use Git to generate the first patch, create an empty series
+  file ``patches/foobar-1.1.0/series`` before extracting the packages. This
+  will tell PTXdist to use Git anyways and ``git ptx-patches`` will put the
+  patches there.
+
+Both approaches (Git and quilt) are not suitable for modifying files
+that are autogenerated in autotools-based buildsystems.
+Refer to the section :ref:`configure_rebuild` on how PTXdist can
+handle this special task.
+
+Adding More Patches to a Package
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If we want to add more patches to an already patched package, we can use
+nearly the same way as creating patches for the first time. But if the
+patch series comes from the PTXdist main installation, we do not have
+write permissions to these directories (do NEVER work on the main
+installation directories, NEVER, NEVER, NEVER). Due to the search order
+in which PTXdist searches for patches for a specific package, we can
+copy the global patch series to our local project directory. Now we have
+the permissions to add more patches or modify the existing ones. Also
+*quilt* and *git* are our friends here to manage the patch series.
+
+If we think that our new patches are valuable also for others, or they
+fix an error, it could be a good idea to send these patches to PTXdist
+mainline, and to the upstream project too.
+
+
+.. _configure_rebuild:
+
+Modifying Autotoolized Packages
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Autotoolized packages are very picky when automatically generated files
+get patched. The patch order is very important in this case and
+sometimes it even fails and nobody knows why.
+
+To improve a package’s autotools-based build system, PTXdist comes with
+its own project local autotools to regenerate the autotools template
+files, instead of patching them. With this feature, only the template
+files must be patched, the required ``configure`` script and the
+``Makefile.in`` files are regenerated in the final stages of the
+*prepare* step.
+
+This feature works like the regular patching mechanism. The only
+difference is the additional ``autogen.sh`` file in the patch directory.
+If it exists and has execution permissions, it will be called after the
+package was patched (while the *extract* stage is running).
+
+Its content depends on developer needs; for the most simple case the
+content can be:
+
+.. code-block:: bash
+
+    #!/bin/bash
+
+    aclocal $ACLOCAL_FLAGS
+
+    libtoolize \
+            --force \
+            --copy
+
+    autoreconf \
+            --force \
+            --install \
+            --warnings=cross \
+            --warnings=syntax \
+            --warnings=obsolete \
+            --warnings=unsupported
+
+.. note:: In this way not yet autotoolized package can be autotoolized. We
+  just have to add the common autotool template files (``configure.ac``
+  and ``Makefile.am`` for example) via a patch series to the package
+  source and the ``autogen.sh`` to the patch directory.
diff --git a/doc/dev_create_new_pkg_templates.rst b/doc/dev_create_new_pkg_templates.rst
new file mode 100644
index 000000000..d7c2927b6
--- /dev/null
+++ b/doc/dev_create_new_pkg_templates.rst
@@ -0,0 +1,77 @@
+Creating New Package Templates
+------------------------------
+
+For larger projects it can be convenient to have project specific package
+templates. This can be achieved by either modifying existing templates or
+by creating completely new templates.
+
+Modifying a Template
+~~~~~~~~~~~~~~~~~~~~
+
+A template can be modified by providing new input files. This is easier
+than creating a new template but does not allow to specify new variables to
+substitute in the input files.
+
+PTXdist looks for template files the same way it looks for rules files. The
+only difference is, that it searches in the ``templates/`` subdirectory.
+So a modified ``./rules/templates/template-target-make`` can be used to
+tweak the ``target`` template.
+
+Creating a New Template
+~~~~~~~~~~~~~~~~~~~~~~~
+
+For a completely new template, some bash scripting is required. All shell
+code must be placed in a file named like this:
+``./scripts/lib/ptxd_lib_*.sh``.
+
+The minimum requirement for a new template is:
+-  a shell function that creates the new package
+-  registering the new template
+
+.. code-block:: sh
+
+    ptxd_template_new_mypkg() {
+        # create the package here
+    }
+    export -f ptxd_template_new_mypkg
+    ptxd_template_help_list[${#ptxd_template_help_list[@]}]="mypkg"
+    ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create awesome mypkg package"
+
+PTXdist provides several helper functions to simplify the template.
+Using those functions, the package creation process is split into two
+parts:
+
+-  query the user for input and export variables.
+-  create the new package files from the template source files by
+   substituting all instances of ``@<variable>@`` with the value of the
+   corresponding variable.
+
+A simple template function could look like this:
+
+.. code-block:: sh
+
+    ptxd_template_new_mypkg() {
+        ptxd_template_read_basic &&
+        ptxd_template_read "enter download section" DL_SECTION "foobar"
+        ptxd_template_read_author &&
+        export section="local_${dlsection}" &&
+        ptxd_template_write_rules
+    }
+
+This template requires ``rules/templates/template-mypkg-make`` and
+``rules/templates/template-mypkg-in`` as source files. They could be
+derived from the ``target`` template with a simple modification:
+
+.. code-block:: make
+
+    @PACKAGE@_SUFFIX	:= tar.xz
+    @PACKAGE@_URL	:= http://dl.my-company.local/downloads/@DL_SECTION@/$(@PACKAGE@).$(@PACKAGE@_SUFFIX)
+
+The helper functions that are used in the example above are defined in
+``scripts/lib/ptxd_lib_template.sh`` in the PTXdist source tree.
+
+The template is a normal shell function. Arbitrary things can be done here
+to create the new package. The helper functions are just the most
+convenient way to crate simple templates. It is also possible to create
+more files. For examples, the builtin ``genimage`` template creates a extra
+config file for the new package.
diff --git a/doc/dev_dir_hierarchy.rst b/doc/dev_dir_hierarchy.rst
new file mode 100644
index 000000000..bd1ad40d0
--- /dev/null
+++ b/doc/dev_dir_hierarchy.rst
@@ -0,0 +1,108 @@
+.. _directory_hierarchy:
+
+PTXdist’s Directory Hierarchy
+-----------------------------
+
+.. note:: Referenced directories are meant relative to the PTXdist main
+  installation location (if not otherwise stated). If not configured
+  differently, this main path is ``/usr/local/lib/ptxdist-|ptxdistVendorVersion|``
+
+Rule Files
+~~~~~~~~~~
+
+When building a single package, PTXdist needs the information on how to
+handle the package, i.e. on how to get it from the source up to what the
+target needs at run-time. This information is provided by a rule file per
+package.
+
+PTXdist collects all rule files in its ``rules/`` directory. Whenever
+PTXdist builds something, all these rule files are scanned at once.
+These rule files are global rule files, valid for all projects. PTXdist
+uses a mechanism to be able to add or replace specific rule files on a
+per project base. If a ``rules/`` directory exists in the current
+project, its content is scanned too. These project local rule files are
+used in addition to the global rule files or – if they are using the
+same name as a global rule file – **replacing** the global rule file.
+
+The replacing mechanism can be used to extend or adapt packages for
+specific project requirements. Or it can be used for bug fixing by
+backporting rule files from more recent PTXdist revisions to projects
+that are stuck to an older PTXdist revision for maintenance only.
+
+Patch Series
+~~~~~~~~~~~~
+
+There are many packages in the wild that are not cross build aware. They
+fail compiling some files, use wrong include paths or try to link
+against host libraries. To be successful in the embedded world, these
+types of failures must be fixed. If required, PTXdist provides such
+fixes per package. They are organized in *patch series* and can be found
+in a ``patches/`` directory within a subdirectory using the same name
+as the package itself.
+
+PTXdist uses the utility ``patch`` or ``quilt`` (or ``git`` on demand) to apply
+an existing patch series after extracting the archive. So, every patch series
+contains a set of patches and one ``series`` file to define the order in
+which the patches must be applied.
+
+.. note:: Patches can be compressed.
+
+Patches are looked for at several locations:
+
+1.  the ``patches/`` folder in your BSP (``${PTXDIST_WORKSPACE}/patches``)
+
+2.  the folder ``patches/`` folder relative to your selected platformconfig
+    file (``${PTXDIST_PLATFORMCONFIGDIR}/patches``). If your platformconfig
+    file is at ``configs/|ptxdistPlatformConfigDir|/platformconfig``, this
+    patch folder will be ``configs/|ptxdistPlatformConfigDir|/patches/``.
+
+3.  the ``patches/`` folder in PTXdist's main installation directory
+    (``${PTXDIST_TOPDIR}/patches``)
+
+The list is tried from first to last.
+If no patches were found in one of the locations, the next location is tried.
+When all locations have been tried unsuccessfully, the package is not patched.
+
+This search order can be used to use specific patch series for specific
+cases.
+
+-  platform specific
+
+-  project specific
+
+-  common case
+
+-  bug fixing
+
+The *bug fixing* case is used in accordance to a replacement of a rule
+file. If this was done due to a backport, and the more recent PTXdist
+revision does not only exchange the rule file but also the patch series,
+this mechanism ensures that both relevant parts can be updated in the
+project.
+
+Runtime Configuration
+~~~~~~~~~~~~~~~~~~~~~
+
+Many packages are using run-time configuration files along with their
+executables and libraries. PTXdist provides default configuration files
+for the most common cases. These files can be found in the
+``projectroot/etc`` directory and they are using the same names as the ones
+at run-time (and their install directory on the target side will also be
+``/etc``).
+
+But some of these default configuration files are empty, due to the
+absence of a common case. The project must provide replacements of these
+files with a more useful content in every case where the (empty) default
+one does not meet the target’s requirements.
+
+PTXdist first searches in the local project directory for a specific
+configuration file and falls back to use the default one if none exists
+locally. Refer section :ref:`install_alternative` for further
+details in which order and locations PTXdist searches for these kind of files.
+
+A popular example is the configuration file ``/etc/fstab``. The default
+one coming with PTXdist works for the most common cases. But if our
+project requires a special setup, we can just copy the default one to
+the local ``./projectroot/etc/fstab``, modify it and we are done. The
+next time PTXdist builds the root filesystem it will use the local
+``fstab`` instead of the global (default) one.
diff --git a/doc/dev_layers_in_ptxdist.rst b/doc/dev_layers_in_ptxdist.rst
new file mode 100644
index 000000000..ec92c8c8a
--- /dev/null
+++ b/doc/dev_layers_in_ptxdist.rst
@@ -0,0 +1,111 @@
+.. _layers-in-ptxdist:
+
+Layers in PTXdist
+-----------------
+
+For better maintenance or other reasons, a PTXdist project can be split
+into multiple layers. Each layer has exactly the same directory hierarchy
+as described in :ref:`directory_hierarchy` and other chapters.
+
+All layers are explicitly stacked in the filesystem. The top layer is the
+workspace of the PTXdist project. Any ``selected_*`` links and the platform
+build directory are created here. The layer below is defined by the
+subdirectory or symlink named ``base/``. More can be stacked the same
+way, so ``base/base/`` is the third layer and so on.
+In many ways, PTXdist itself can be considered as the bottom layer. This is
+either implicit or explicit with one last ``base/`` symlink.
+
+A project can overwrite files provided by PTXdist in many different ways,
+e.g. rule files or files installed with :ref:`install_alternative` etc.
+This concept expands naturally to layers. Each layer can overwrite files
+provided by lower layers in the exact same way. Any files are always
+searched for in a strict layer by layer order.
+
+Writing Layer Aware Rules
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For the most part, package rules work just as expected when multiple layers
+are used. Any layer specific handling is done implicitly by PTXdist.
+However, there are a few things that need special handling.
+
+The variables :ref:`PTXDIST_WORKSPACE<ptxdist_workspace>` and
+:ref:`PTXDIST_PLATFORMCONFIGDIR`<ptxdist_platformconfigdir>` always refer
+to the directories in the top layer. These variables might be used in rules
+files like this:
+
+.. code-block:: make
+
+   MY_KERNEL_CONFIG := $(PTXDIST_PLATFORMCONFIGDIR)/kernelconfig.special
+
+If the referenced file is in any layer but the top one then it will not
+be found. To handle use-cases like this, the macros :ref:`in_path` and
+:ref:`in_platformconfigdir` can be used:
+
+.. code-block:: make
+
+   MY_KERNEL_CONFIG := $(call ptx/in-platformconfigdir, kernelconfig.special)
+
+This way, the layers are searched top to bottom until the config file is
+found.
+
+PTXdist Config Files with Multiple Layers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In many cases a layer may want to modify the **ptxconfig** by enabling or
+disabling some options. Any changes must be propagated through the whole
+layer stack.
+
+The features and workflow described here apply to the **ptxconfig**, the
+**platformconfig** and any **collectionconfig** used in the project.
+
+To do this, PTXdist stores a delta config to the layer below and a full
+config file in each layer. If the two files are missing then the config is
+unchanged. The bottom layer has only the config file and no delta.
+
+At runtime, PTXdist will always use the full config file in the top layer
+where the config exists. Before doing so, it will ensure that the config is
+consistent across all layers. This means that, for any layer that contains a
+delta config, the full config file of the layer below has not changed since
+the delta config was last updated. If any inconsistency is detected,
+PTXdist will abort.
+
+For any command that modifies the config file, except ``oldconfig``,
+PTXdist will use kconfig implicitly on all layers to check if the config
+for this layer is up to date. This is a stricter check than the consistency
+validation. For example, if a new package was added to a layer without
+updating the **ptxconfig** then this will be detected and PTXdist will
+abort. If all other layers are up to date, then PTXdist will use the delta
+config of the top layer, apply it to the full config of the layer below
+and execute the specified command with the resulting config file.
+
+.. note:: If the config file does not exist yet on the top layer, then it
+  will be created if changes to the config are made. Similarly the config
+  will be deleted if the delta is empty after the changes. In either case
+  it may be necessary to update any ``selected_*`` link to point to the
+  correct config.
+
+If PTXdist detects an inconsistency or an out of date config file then it
+must be updated before they can be used. This can be done by using the
+``oldconfig`` command. In this special case, PTXdist will iterate from the
+bottom to the top layer and run ``oldconfig`` for each of them. It will
+use the delta config applied to the full config of the layer below at each
+step. This means that it's possible to enable or disable a option in the
+bottom layer and ``oldconfig`` will propagate this change to all other
+layers.
+
+Packages with kconfig Based Config Files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For packages such as the Linux kernel that have kconfig based config files,
+a lot of the infrastructure to handle config files and deltas across
+multiple layers can be reused. Consistency validation is done implicitly
+and ``menuconfig`` and other kconfig commands will use config files and
+deltas as expected.
+
+It's not possible to implicitly run ``oldconfig`` on other layers (this may
+require a different source tree for the packages), so any inconsistencies
+must be resolved manually by running ``oldconfig`` explicitly on each
+layer.
+
+The make macros that provide these features are currently used by the
+barebox and kernel packages and templates.
diff --git a/doc/dev_manual.rst b/doc/dev_manual.rst
index 50827b6a9..47a77a9be 100644
--- a/doc/dev_manual.rst
+++ b/doc/dev_manual.rst
@@ -5,1758 +5,12 @@ PTXdist Developer’s Manual
 
 This chapter shows all (or most) of the details of how PTXdist works.
 
--  where are the files stored that PTXdist uses when building packages
-
--  how patching works
-
--  where is PTXdist fetching a package’s run-time configuration files
-   from
-
--  how to control a package’s build stages
-
--  how to add new packages
-
-.. _directory_hierarchy:
-
-PTXdist’s Directory Hierarchy
------------------------------
-
-.. note:: Referenced directories are meant relative to the PTXdist main
-  installation location (if not otherwise stated). If not configured
-  differently, this main path is ``/usr/local/lib/ptxdist-|ptxdistVendorVersion|``
-
-Rule Files
-~~~~~~~~~~
-
-When building a single package, PTXdist needs the information on how to
-handle the package, i.e. on how to get it from the source up to what the
-target needs at run-time. This information is provided by a rule file per
-package.
-
-PTXdist collects all rule files in its ``rules/`` directory. Whenever
-PTXdist builds something, all these rule files are scanned at once.
-These rule files are global rule files, valid for all projects. PTXdist
-uses a mechanism to be able to add or replace specific rule files on a
-per project base. If a ``rules/`` directory exists in the current
-project, its content is scanned too. These project local rule files are
-used in addition to the global rule files or – if they are using the
-same name as a global rule file – **replacing** the global rule file.
-
-The replacing mechanism can be used to extend or adapt packages for
-specific project requirements. Or it can be used for bug fixing by
-backporting rule files from more recent PTXdist revisions to projects
-that are stuck to an older PTXdist revision for maintenance only.
-
-Patch Series
-~~~~~~~~~~~~
-
-There are many packages in the wild that are not cross build aware. They
-fail compiling some files, use wrong include paths or try to link
-against host libraries. To be successful in the embedded world, these
-types of failures must be fixed. If required, PTXdist provides such
-fixes per package. They are organized in *patch series* and can be found
-in a ``patches/`` directory within a subdirectory using the same name
-as the package itself.
-
-PTXdist uses the utility ``patch`` or ``quilt`` (or ``git`` on demand) to apply
-an existing patch series after extracting the archive. So, every patch series
-contains a set of patches and one ``series`` file to define the order in
-which the patches must be applied.
-
-.. note:: Patches can be compressed.
-
-Patches are looked for at several locations:
-
-1.  the ``patches/`` folder in your BSP (``${PTXDIST_WORKSPACE}/patches``)
-
-2.  the folder ``patches/`` folder relative to your selected platformconfig
-    file (``${PTXDIST_PLATFORMCONFIGDIR}/patches``). If your platformconfig
-    file is at ``configs/|ptxdistPlatformConfigDir|/platformconfig``, this
-    patch folder will be ``configs/|ptxdistPlatformConfigDir|/patches/``.
-
-3.  the ``patches/`` folder in PTXdist's main installation directory
-    (``${PTXDIST_TOPDIR}/patches``)
-
-The list is tried from first to last.
-If no patches were found in one of the locations, the next location is tried.
-When all locations have been tried unsuccessfully, the package is not patched.
-
-This search order can be used to use specific patch series for specific
-cases.
-
--  platform specific
-
--  project specific
-
--  common case
-
--  bug fixing
-
-The *bug fixing* case is used in accordance to a replacement of a rule
-file. If this was done due to a backport, and the more recent PTXdist
-revision does not only exchange the rule file but also the patch series,
-this mechanism ensures that both relevant parts can be updated in the
-project.
-
-Runtime Configuration
-~~~~~~~~~~~~~~~~~~~~~
-
-Many packages are using run-time configuration files along with their
-executables and libraries. PTXdist provides default configuration files
-for the most common cases. These files can be found in the
-``projectroot/etc`` directory and they are using the same names as the ones
-at run-time (and their install directory on the target side will also be
-``/etc``).
-
-But some of these default configuration files are empty, due to the
-absence of a common case. The project must provide replacements of these
-files with a more useful content in every case where the (empty) default
-one does not meet the target’s requirements.
-
-PTXdist first searches in the local project directory for a specific
-configuration file and falls back to use the default one if none exists
-locally. Refer section :ref:`install_alternative` for further
-details in which order and locations PTXdist searches for these kind of files.
-
-A popular example is the configuration file ``/etc/fstab``. The default
-one coming with PTXdist works for the most common cases. But if our
-project requires a special setup, we can just copy the default one to
-the local ``./projectroot/etc/fstab``, modify it and we are done. The
-next time PTXdist builds the root filesystem it will use the local
-``fstab`` instead of the global (default) one.
-
-.. _adding_new_packages:
-
-Adding New Packages
--------------------
-
-PTXdist provides a huge amount of applications sufficient for the most
-embedded use cases. But there is still need for some fancy new packages.
-This section describes the steps and the background on how to integrate
-new packages into the project.
-
-At first a summary about possible application types which PTXdist can
-handle:
-
--  **host type**: This kind of package is built to run on the build
-   host. Most of the time such a package is needed if another
-   target-relevant package needs to generate some data. For example the
-   *glib* package depends on its own to create some data. But if it is
-   compiled for the target, it can’t do so. That’s why a host glib
-   package is required to provide these utilities runnable on the build
-   host. It sounds strange to build a host package, even if on the build
-   host such utilities are already installed. But this way ensures that
-   there are no dependencies regarding the build host system.
-
--  **target type**: This kind of package is built for the target.
-
--  **cross type**: This kind of package is built for the build host, but
-   creates architecture specific data for the target.
-
--  **src-autoconf-prog**: This kind of package is built for the target.
-   It is intended for development, as it does not handle a released
-   archive but a plain source project instead. Creating such a package
-   will also create a small autotools based source template project on
-   demand to give the developer an easy point to start. This template is
-   prepared to build a single executable program. For further details refer
-   section :ref:`adding_src_autoconf_exec`.
-
--  **src-autoconf-lib**: This kind of package is built for the target.
-   It is intended for development, as it does not handle a released
-   archive but a plain source project instead. Creating such a package
-   will also create a small autotools/libtool based source template
-   project on demand to give the developer an easy point to start. This
-   template is prepared to build a single shared library. For further
-   details refer section :ref:`adding_src_autoconf_lib`.
-
--  **src-autoconf-proglib**: This kind of package is built for the
-   target. It is intended for development, as it does not handle a
-   released archive but a plain source project instead. Creating such a
-   package will also create a small autotools/libtool based template
-   project on demand to give the developer an easy point to start. This
-   template is prepared to build a single shared library and a single
-   executable program. The program will be linked against the shared
-   library. For further details refer section :ref:`adding_src_autoconf_exec_lib`.
-
--  **file**: This kind of package is intended to add a few simple files
-   into the build process. We assume these files do not need any
-   processing, they are ready to use and must only be present in the
-   build process or at run-time (HTML files for example). Refer to the
-   section :ref:`adding_files` for further details on how to use
-   it.
-
--  **src-make-prog**: This kind of package is built for the target. It’s
-   intended for development, as it does not handle a released archive
-   but a plain source project instead. Creating such a package will also
-   create a simple makefile-based template project the developer can use
-   as a starting point for development.
-
--  **src-cmake-prog**: This kind of package is built for the target.
-   It’s intended for developments based on the *cmake* buildsystem.
-   Various projects are using *cmake* instead of *make* and can be built
-   with this package type. PTXdist will prepare it to compile sources in
-   accordance to the target libraries and their settings. Creating such
-   a package will also create a simple template project to be used as a
-   starting point for development.
-
--  **src-qmake-prog**: This kind of package is built for the target.
-   It’s intended for developments based on the *qmake* buildsystem. If
-   the developer is going to develop a QT based application, this rule
-   is prepared to compile sources in accordance to the target libraries
-   and their settings. Creating such a package will also create a simple
-   template project to be used as a starting point for development.
-
--  **src-meson-prog**: This kind of package is built for the target.
-   It’s intended for developments based on the *meson* buildsystem.
-   Various projects are using *meson* today and can be built
-   with this package type. PTXdist will prepare it to compile sources in
-   accordance to the target libraries and their settings. Creating such
-   a package will also create a simple template project to be used as a
-   starting point for development.
-
--  **font**: This package is a helper to add X font files to the root
-   filesystem. This package does not create an additional IPKG, instead
-   it adds the font to the existing font IPKG. This includes the
-   generation of the directory index files, required by the Xorg
-   framework to recognize the font file.
-
--  **src-linux-driver**: This kind of package builds an out of tree
-   kernel driver. It also creates a driver template to give the
-   developer an easy point to start.
-
--  **kernel**: PTXdist comes with the ability to handle one kernel in its
-   platform. This type of package enables us to handle more than one kernel in
-   the project.
-
--  **barebox**: PTXdist comes with the ability to handle one bootloader in its
-   platform. This type of package enables us to handle more than one bootloader
-   in the project.
-
--  **image-tgz**: This kind of package creates a tar ball from a list of
-   packages. It is often uses as an input for other image packages.
-
--  **image-genimage**: This kind of package can handle all kind of image
-   generation for almost every target independent of its complexity.
-
--  **blspec-entry**: PTXdist comes with the ability to handle one bootspec in its
-   platform. This type of package enables us to handle more than one bootspec
-   in the project.
-
-.. _foo_example:
-
-Rule File Creation
-~~~~~~~~~~~~~~~~~~
-
-To create such a new package, we create a project local ``rules/``
-directory first. Then we run
-
-.. code-block:: text
-
-    $ ptxdist newpackage <package type>
-
-If we omit the <``package type``\ >, PTXdist will list all available
-package types.
-
-In our first example, we want to add a new target type archive package.
-When running the
-
-.. code-block:: text
-
-    $ ptxdist newpackage target
-
-command, PTXdist asks a few questions about this package. This
-information is the basic data PTXdist must know about the package.
-
-.. code-block:: text
-
-    ptxdist: creating a new 'target' package:
-
-    ptxdist: enter package name.......: foo
-    ptxdist: enter version number.....: 1.1.0
-    ptxdist: enter URL of basedir.....: http://www.foo.com/download/src
-    ptxdist: enter suffix.............: tar.gz
-    ptxdist: enter package author.....: My Name <me@my-org.com>
-    ptxdist: enter package section....: project_specific
-
-What we have to answer:
-
--  **package name**: As this kind of package handles a source archive,
-   the correct answer here is the basename of the archive’s file name.
-   If its full name is ``foo-1.1.0.tar.gz``, then ``foo`` is the
-   basename to enter here.
-
--  **version number**: Most source archives are using a release or
-   version number in their file name. If its full name is
-   ``foo-1.1.0.tar.gz``, then ``1.1.0`` is the version number to enter
-   here.
-
--  **URL of basedir**: This URL tells PTXdist where to download the
-   source archive from the web (if not already done). If the full URL to
-   download the archive is
-   ``http://www.foo.com/download/src/foo-1.1.0.tar.gz``, the basedir
-   part ``http://www.foo.com/download/src`` is to be entered here.
-
--  **suffix**: Archives are using various formats for distribution.
-   PTXdist uses the *suffix* entry to select the matching extraction
-   tool. If the archive’s full name is ``foo-1.1.0.tar.gz``, then
-   ``tar.gz`` is the suffix to enter here.
-
--  **package author**: If we intend to contribute this new package to
-   PTXdist mainline, we should add our name here. This name will be used
-   in the copyright note of the rule file and will also be added to the
-   generated ipkg. When you run ``ptxdist setup`` prior to this call,
-   you can enter your name and your email address, so PTXdist will use
-   it as the default (very handy if you intend to add many new
-   packages).
-
--  **package section**: We can enter here the menu section name where
-   our new package menu entry should be listed. In the first step we can
-   leave the default name unchanged. It’s a string in the menu file
-   only, so changing it later on is still possible.
-
-Make it Work
-~~~~~~~~~~~~
-
-Generating the rule file is only one of the required steps to get a new
-package. The next steps to make it work are to check if all stages are
-working as expected and to select the required parts to get them
-installed in the target root filesystem. Also we must find a reasonable
-location where to add our new menu entry to configure the package.
-
-The generated skeleton starts to add the new menu entry in the main
-configure menu (if we left the section name unchanged). Running
-``ptxdist menuconfig`` will show it on top of all other menus entries.
-
-.. important:: 
-  To be able to implement and test all the other required steps for adding
-  a new package, we first must enable the package for building. (Fine
-  tuning the menu can happen later on.)
-
-
-The rule file skeleton still lacks some important information. Let’s
-take a look into some of the top lines of the generated rule file
-``./rules/foo.make``:
-
-.. code-block:: make
-
-    FOO_VERSION	:= 1.1.0
-    FOO_MD5	:=
-    FOO		:= foo-$(FOO_VERSION)
-    FOO_SUFFIX	:= tar.gz
-    FOO_URL	:= http://www.foo.com/download/src/$(FOO).$(FOO_SUFFIX)
-    FOO_SOURCE	:= $(SRCDIR)/$(FOO).$(FOO_SUFFIX)
-    FOO_DIR	:= $(BUILDDIR)/$(FOO)
-    FOO_LICENSE	:= unknown
-
-We can find these lines with different content in most or all of the
-other rule files PTXdist comes with. Up to the underline character is
-always the package name and after the underline character is always
-PTXdist specific. What does it mean:
-
--  ``*_VERSION`` brings in the version number of the release and is used
-   for the download and IPKG/OPKG package generation.
-
--  ``*_MD5`` to be sure the correct package has been downloaded, PTXdist
-   checks the given MD5 sum against the archive content. If both sums do
-   not match, PTXdist rejects the archive and fails the currently
-   running build.
-
--  ``*_SUFFIX`` defines the archive type, to make PTXdist choosing the
-   correct extracting tool.
-
--  ``*_URL`` defines the full qualified URL into the web for download. If
-   alternative download locations are known, they can be listed in this
-   variable, delimiter character is the space.
-
--  ``*_SOURCE`` tells PTXdist where to store the downloaded package.
-
--  ``*_DIR`` points to the directory this package will be built later on
-   by PTXdist.
-
--  ``*_LICENSE`` enables the user to get a list of licenses she/he is
-   using in her/his project (licenses of the enabled packages).
-
-After enabling the menu entry, we can start to check the *get* and
-*extract* stages, calling them manually one after another.
-
-.. note:: The shown commands below expect that PTXdist downloads the
-  archives to a global directory named ``global_src``. This is not the
-  default setting, but we recommend to use a global directory to share all
-  archives between PTXdist based projects. Advantage is every download
-  happens only once. Refer to the ``setup`` command PTXdist provides.
-
-.. code-block:: text
-
-    $ ptxdist get foo
-
-    ---------------------------
-    target: foo-1.1.0.tar.gz
-    ---------------------------
-
-    --2009-12-21 10:54:45--  http://www.foo.com/download/src/foo-1.1.0.tar.gz
-    Length: 291190 (284K) [application/x-gzip]
-    Saving to: `/global_src/foo-1.1.0.tar.gz.XXXXOGncZA'
-
-    100%[======================================>] 291,190      170K/s   in 1.7s
-
-    2009-12-21 10:54:48 (170 KB/s) - `/global_src/foo-1.1.0.tar.gz' saved [291190/291190]
-
-This command should start to download the source archive. If it fails,
-we should check our network connection, proxy setup or if the given URL
-in use is correct.
-
-.. note:: Sometimes we do not know the content of all the other variables in
-  the rule file. To get an idea what content a variable has, we can ask
-  PTXdist about it:
-
-.. code-block:: text
-
-    $ ptxdist print FOO_URL
-    http://www.foo.com/download/src/foo-1.1.0.tar.gz
-
-The next step would be to extract the archive. But as PTXdist checks the
-MD5 sum in this case, this step will fail, because the ``FOO_MD5``
-variable is still empty. Let’s fill it:
-
-.. code-block:: text
-
-    $ md5sum /global_src/foo-1.1.0.tar.gz
-    9a09840ab775a139ebb00f57a587b447
-
-This string must be assigned to the FOO_MD5 in our new ``foo.make``
-rule file:
-
-.. code-block:: text
-
-    FOO_MD5		:= 9a09840ab775a139ebb00f57a587b447
-
-We are now prepared for the next step:
-
-.. code-block:: text
-
-    $ ptxdist extract foo
-
-    -----------------------
-    target: foo.extract
-    -----------------------
-
-    extract: archive=/global_src/foo-1.1.0.tar.gz
-    extract: dest=/home/jbe/my_new_prj/build-target
-    PATCHIN: packet=foo-1.1.0
-    PATCHIN: dir=/home/jbe/my_new_prj/build-target/foo-1.1.0
-    PATCHIN: no patches for foo-1.1.0 available
-    Fixing up /home/jbe/my_new_prj/build-target/foo-1.1.0/configure
-    finished target foo.extract
-
-In this example we expect an autotoolized source package. E.g. to
-prepare the build, the archive comes with a ``configure`` script. This
-is the default case for PTXdist. So, there is no need to modify the rule
-file and we can simply run:
-
-.. code-block:: text
-
-    $ ptxdist prepare foo
-
-    -----------------------
-    target: foo.prepare
-    -----------------------
-
-    [...]
-
-    checking build system type... i686-host-linux-gnu
-    checking host system type... |ptxdistCompilerName|
-    checking whether to enable maintainer-specific portions of Makefiles... no
-    checking for a BSD-compatible install... /usr/bin/install -c
-    checking whether build environment is sane... yes
-    checking for a thread-safe mkdir -p... /bin/mkdir -p
-    checking for gawk... gawk
-    checking whether make sets $(MAKE)... yes
-    checking for |ptxdistCompilerName|-strip... |ptxdistCompilerName|-strip
-    checking for |ptxdistCompilerName|-gcc... |ptxdistCompilerName|-gcc
-    checking for C compiler default output file name... a.out
-
-    [...]
-
-    configure: creating ./config.status
-    config.status: creating Makefile
-    config.status: creating ppa_protocol/Makefile
-    config.status: creating config.h
-    config.status: executing depfiles commands
-    finished target foo.prepare
-
-At this stage things can fail:
-
--  A wrong or no MD5 sum was given
-
--  The ``configure`` script is not cross compile aware
-
--  The package depends on external components (libraries for example)
-
-If the ``configure`` script is not cross compile aware, we are out of
-luck. We must patch the source archive in this case to make it work.
-Refer to the section :ref:`configure_rebuild` on how to use
-PTXdist’s features to simplify this task.
-If the package depends on external components, these components might
-be already part of PTXdist. In this case we just have to add this
-dependency into the menu file and we are done. But if PTXdist cannot
-fulfill this dependency, we also must add it as a separate package
-first.
-
-If the *prepare* stage has finished successfully, the next step is to
-compile the package.
-
-.. code-block:: text
-
-    $ ptxdist compile foo
-
-    -----------------------
-    target: foo.compile
-    -----------------------
-
-    make[1]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-    make  all-recursive
-    make[2]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-    make[3]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-
-    [...]
-
-    make[3]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-    make[2]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-    make[1]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-    finished target foo.compile
-
-At this stage things can fail:
-
--  The build system is not cross compile aware (it tries to execute just
-   created target binaries for example)
-
--  The package depends on external components (libraries for example)
-   not detected by ``configure``
-
--  Sources are ignoring the endianness of some architectures or using
-   header files from the build host system (from ``/usr/include`` for
-   example)
-
--  The linker uses libraries from the build host system (from
-   ``/usr/lib`` for example) by accident
-
-In all of these cases we must patch the sources to make them work. Refer
-to section :ref:`patching_packages` on how to use PTXdist’s
-features to simplify this task.
-
-In this example we expect the best case: everything went fine, even for
-cross compiling. So, we can continue with the next stage: *install*
-
-.. code-block:: text
-
-    $ ptxdist install foo
-
-    -----------------------
-    target: foo.install
-    -----------------------
-
-    make[1]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-    make[2]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-    make[3]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-    test -z "/usr/bin" || /bin/mkdir -p "/home/jbe/my_new_prj/build-target/foo-1.1.0/usr/bin"
-      /usr/bin/install -c 'foo' '/home/jbe/my_new_prj/build-target/foo-1.1.0/usr/bin/foo'
-    make[3]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-    make[2]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-    make[1]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0'
-    finished target foo.install
-
-    ----------------------------
-    target: foo.install.post
-    ----------------------------
-
-    finished target foo.install.post
-
-This *install* stage does not install anything to the target root
-filesystem. It is mostly intended to install libraries and header files
-other programs should link against later on.
-
-The last stage – *targetinstall* – is the one that defines the package’s
-components to be forwarded to the target’s root filesystem. Due to the
-absence of a generic way, this is the task of the developer. So, at this
-point of time we must run our favourite editor again and modify our new
-rule file ``./rules/foo.make``.
-
-The skeleton for the *targetinstall* stage looks like this:
-
-.. code-block:: make
-
-    # ----------------------------------------------------------------------------
-    # Target-Install
-    # ----------------------------------------------------------------------------
-
-    $(STATEDIR)/foo.targetinstall:
-    	@$(call targetinfo)
-
-    	@$(call install_init,  foo)
-    	@$(call install_fixup, foo,PACKAGE,foo)
-    	@$(call install_fixup, foo,PRIORITY,optional)
-    	@$(call install_fixup, foo,VERSION,$(FOO_VERSION))
-    	@$(call install_fixup, foo,SECTION,base)
-    	@$(call install_fixup, foo,AUTHOR,"My Name <me@my-org.com>")
-    	@$(call install_fixup, foo,DEPENDS,)
-    	@$(call install_fixup, foo,DESCRIPTION,missing)
-
-    	@$(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/foobar, /dev/null)
-
-    	@$(call install_finish, foo)
-    	@$(call touch)
-
-The “header” of this stage defines some information IPKG needs. The
-important part that we must modify is the call to the ``install_copy``
-macro (refer to section :ref:`reference_macros` for more details
-about this kind of macros). This call instructs PTXdist to include the
-given file (with UID, GID and permissions) into the IPKG, which means to
-install this file to the target’s root filesystem.
-
-From the previous *install* stage we know this package installs an
-executable called ``foo`` to location ``/usr/bin``. We can do the same
-for our target by changing the *install\_copy* line to:
-
-.. code-block:: none
-
-    @$(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/foo, /usr/bin/foo)
-
-To check it, we just run:
-
-.. code-block:: text
-
-    $ ptxdist targetinstall foo
-
-    -----------------------------
-    target: foo.targetinstall
-    -----------------------------
-
-    install_init:   preparing for image creation...
-    install_init:   @ARCH@ -> i386 ... done
-    install_init:   preinst not available
-    install_init:   postinst not available
-    install_init:   prerm not available
-    install_init:   postrm not available
-    install_fixup:  @PACKAGE@ -> foo ... done.
-    install_fixup:  @PRIORITY@ -> optional ... done.
-    install_fixup:  @VERSION@ -> 1.1.0 ... done.
-    install_fixup:  @SECTION@ -> base ... done.
-    install_fixup:  @AUTHOR@ -> "My Name <me\@my-org.com>" ... done.
-    install_fixup:  @DESCRIPTION@ -> missing ... done.
-    install_copy:
-      src=/home/jbe/my_new_prj/build-target/foo-1.1.0/foo
-      dst=/usr/bin/foo
-      owner=0
-      group=0
-      permissions=0755
-    xpkg_finish:    collecting license (unknown) ... done.
-    xpkg_finish:    creating ipkg package ... done.
-    finished target foo.targetinstall
-
-    ----------------------------------
-    target: foo.targetinstall.post
-    ----------------------------------
-
-    finished target foo.targetinstall.post
-
-After this command, the target’s root filesystem contains a file called
-``/usr/bin/foo`` owned by root, its group is also root and everyone has
-execution permissions, but only the user root has write permissions.
-
-One last task of this port is still open: A reasonable location for
-the new menu entry in PTXdist’s menu hierarchy. PTXdist arranges its
-menus on the meaning of each package. Is it a network related tool? Or
-a scripting language? Or a graphical application?
-Each of these global meanings has its own submenu, where we can add
-our new entry to. We just have to edit the head of our new menu file
-``./rules/foo.in`` to add it to a specific global menu. If our new
-package is a network related tool, the head of the menu file should
-look like:
-
-.. code-block:: kconfig
-
-    ## SECTION=networking
-
-We can grep through the other menu files from the PTXdist main
-installation ``rules/`` directory to get an idea what section names are
-available:
-
-.. code-block:: text
-
-    rules/ $ find . -name \*.in | xargs grep "## SECTION"
-    ./acpid.in:## SECTION=shell_and_console
-    ./alsa-lib.in:## SECTION=system_libraries
-    ./alsa-utils.in:## SECTION=multimedia_sound
-    ./apache2.in:## SECTION=networking
-    ./apache2_mod_python.in:## SECTION=networking
-    [...]
-    ./xkeyboard-config.in:## SECTION=multimedia_xorg_data
-    ./xorg-app-xev.in:## SECTION=multimedia_xorg_app
-    ./xorg-app-xrandr.in:## SECTION=multimedia_xorg_app
-    ./host-eggdbus.in:## SECTION=hosttools_noprompt
-    ./libssh2.in:## SECTION=networking
-
-Porting a new package to PTXdist is (almost) finished now.
-
-To check it right away, we simply run these two commands:
-
-.. code-block:: text
-
-    $ ptxdist clean foo
-    rm -rf /home/jbe/my_new_prj/state/foo.*
-    rm -rf /home/jbe/my_new_prj/packages/foo_*
-    rm -rf /home/jbe/my_new_prj/build-target/foo-1.1.0
-    $ ptxdist targetinstall foo
-
-    [...]
-
-.. important:: Discover somehow hidden dependencies with one more last check!
-
-Up to this point all the development of the new package was done in an already
-built BSP. Doing so sometimes somehow hidden dependencies cannot be seen:
-everything seems fine, the new package builds always successfully and the
-results are working on the target.
-
-So to check for this kind of dependencies there is still one more final check
-to do (even if its boring and takes time):
-
-.. code-block:: text
-
-    $ ptxdist clean
-    [...]
-    $ ptxdist targetinstall foo
-    [...]
-
-This will re-start with a **clean** BSP and builds exactly the new package and
-its (known) dependencies. If this builds successfully as well we are really done
-with the new package.
-
-Some Notes about Licenses
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The already mentioned rule variable ``*_LICENSE`` (e.g. ``FOO_LICENSE`` in our
-example) is very important and must be filled by the developer of the package.
-Many licenses bring in obligations using the corresponding package (*attribution*
-for example). To make life easier for everybody the license for a package must
-be provided. *SPDX* license identifiers unify the license names and are used
-in PTXdist to identify license types and obligations.
-
-If a package comes with more than one license, all of their SPDX identifiers
-must be listed and connected with the keyword ``AND``. If your package comes
-with GPL-2.0 and LGPL-2.1 licenses, the definition should look like this:
-
-.. code-block:: make
-
-   FOO_LICENSE := GPL-2.0 AND LGPL-2.1
-
-One specific obligation cannot be detected examining the SPDX license identifiers
-by PTXdist: *the license choice*. In this case all licenses of choice must be
-listed and connected by the keyword ``OR``.
-
-If, for example, your obligation is to select one of the licenses *GPL-2.0* **or**
-*GPL-3.0*, the ``*_LICENSE`` variable should look like this:
-
-.. code-block:: make
-
-   FOO_LICENSE := GPL-2.0 OR GPL-3.0
-
-SPDX License Identifiers
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-A list of SPDX license identifiers can be found here:
-
-   https://spdx.org/licenses/
-
-Help to Detect the Correct License
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-License identification isn't trivial. A help in doing so can be the following
-repository and its content. It contains a list of known licenses based on their
-SPDX identifier. The content is without formatting to simplify text search.
-
-   https://github.com/spdx/license-list-data/tree/master/text
-
-Advanced Rule Files
-~~~~~~~~~~~~~~~~~~~
-
-The previous example on how to create a rule file sometimes works as
-shown above. But most of the time source archives are not that simple.
-In this section we want to give the user a more detailed selection how
-the package will be built.
-
-Adding Static Configure Parameters
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The ``configure`` scripts of various source archives provide additional
-parameters to enable or disable features, or to configure them in a
-specific way.
-
-We assume the ``configure`` script of our ``foo`` example (refer to
-section :ref:`foo_example`) supports two additional parameters:
-
--  **--enable-debug**: Make the program more noisy. It’s disabled by
-   default.
-
--  **--with-bar**: Also build the special executable **bar**. Building
-   this executable is also disabled by default.
-
-We now want to forward these options to the ``configure`` script when it
-runs in the *prepare* stage. To do so, we must again open the rule file
-with our favourite editor and navigate to the *prepare* stage entry.
-
-PTXdist uses the variable ``FOO_CONF_OPT`` as the list of parameters to
-be given to ``configure``.
-
-Currently this variable is commented out and defined to:
-
-.. code-block:: make
-
-    # FOO_CONF_OPT := $(CROSS_AUTOCONF_USR)
-
-The variable ``CROSS_AUTOCONF_USR`` is predefined by PTXdist and
-contains all basic parameters to instruct ``configure`` to prepare for a
-**cross** compile environment.
-
-To use the two additional mentioned ``configure`` parameters, we comment
-in this line and supplement this expression as follows:
-
-.. code-block:: make
-
-    FOO_CONF_OPT := \
-        $(CROSS_AUTOCONF_USR) \
-        --enable-debug \
-        --with-bar
-
-.. note:: We recommend to use this format with each parameter on a line of
- its own. This format is easier to read and a diff shows more exactly any
- change.
-
-To do a fast check if this addition was successful, we run:
-
-.. code-block:: text
-
-    $ ptxdist print FOO_CONF_OPT
-    --prefix=/usr --sysconfdir=/etc --host=|ptxdistCompilerName| --build=i686-host-linux-gnu --enable-debug --with-bar
-
-.. note:: It depends on the currently selected platform and its architecture
- what content this variable will have. The content shown above is an
- example for a target.
-
-Or re-build the package with the new settings:
-
-.. code-block:: text
-
-    $ ptxdist drop foo prepare
-    $ ptxdist targetinstall foo
-
-Adding Dynamic Configure Parameters
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Sometimes it makes sense to add this kind of parameters on demand only;
-especially a parameter like ``--enable-debug``. To let the user decide
-if this parameter is to be used or not, we must add a menu entry. So,
-let’s expand our menu. Here is its current content:
-
-.. code-block:: kconfig
-
-    ## SECTION=project_specific
-
-    config FOO
-            tristate
-            prompt "foo"
-            help
-              FIXME
-
-We’ll add two menu entries, one for each optional parameter we want to
-add on demand to the ``configure`` parameters:
-
-.. code-block:: kconfig
-
-    ## SECTION=project_specific
-
-    config FOO
-           tristate
-           prompt "foo"
-           help
-             FIXME
-
-    if FOO
-    config FOO_DEBUG
-           bool
-           prompt "add debug noise"
-
-    config FOO_BAR
-           bool
-           prompt "build bar"
-
-    endif
-
-.. important:: Always follow the rule to extend the base name by a suboption
-  name as the trailing part of the variable name. This gives PTXdist the ability
-  to detect a change in the package’s settings (via menuconfig) to force its
-  rebuild on demand.
-
-To make usage of the new menu entries, we must check them in the rule
-file and add the correct parameters:
-
-.. code-block:: make
-
-    #
-    # autoconf
-    #
-    FOO_CONF_OPT := \
-        $(CROSS_AUTOCONF_USR) \
-        --$(call ptx/endis, PTXCONF_FOO_DEBUG)-debug \
-        --$(call ptx/wwo, PTXCONF_FOO_BAR)-bar
-
-.. important:: Please note the leading ``PTXCONF_`` for each define. While Kconfig is
-  using ``FOO_BAR``, the rule file must use ``PTXCONF_FOO_BAR`` instead.
-
-.. note:: Refer :ref:`Rule File Macro Reference <param_macros>` for further
-   details about these special kind of option macros (e.g. ``ptx/...``).
-
-It is a good practice to always add both settings, e.g. ``--disable-debug``
-even if this is the default case. Sometimes ``configure`` tries to guess
-something and the binary result might differ depending on the build
-order. For example some kind of package would also build some X related
-tools, if X libraries are found. In this case it depends on the build
-order, if the X related tools are built or not. All the autocheck
-features are problematic here. So, if we do not want ``configure`` to
-guess its settings we **must disable everything we do not want**.
-
-To support this process, PTXdist supplies a helper script, located at
-``/path/to/ptxdist/scripts/configure-helper.py`` that compares the configure
-output with the settings from ``FOO_CONF_OPT``:
-
-.. code-block:: text
-
-    $ /opt/ptxdist-2017.06.0/scripts/configure-helper.py -p libsigrok
-    --- rules/libsigrok.make
-    +++ libsigrok-0.5.0
-    @@ -4,3 +4,74 @@
-     	--libdir=/usr/lib
-     	--build=x86_64-host-linux-gnu
-     	--host=arm-v7a-linux-gnueabihf
-    +	--enable-warnings=min|max|fatal|no
-    +	--disable-largefile
-    +	--enable-all-drivers
-    +	--enable-agilent-dmm
-    [...]
-    +	--enable-ruby
-    +	--enable-java
-    +	--without-libserialport
-    +	--without-libftdi
-    +	--without-libusb
-    +	--without-librevisa
-    +	--without-libgpib
-    +	--without-libieee1284
-    +	--with-jni-include-path=DIR-LIST
-
-In this example, many configure options from libsigrok (marked with ``+``)
-are not yet present in ``LIBSIGROK_CONF_OPT`` and must be added, possibly also
-by providing more dynamic options in the package definition.
-
-If some parts of a package are built on demand only, they must also be
-installed on demand only. Besides the *prepare* stage, we also must
-modify our *targetinstall* stage:
-
-.. code-block:: make
-
-    	@$(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/foo, /usr/bin/foo)
-
-    ifdef PTXCONF_FOO_BAR
-    	@$(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/bar, /usr/bin/bar)
-    endif
-
-    	@$(call install_finish, foo)
-    	@$(call touch)
-
-Now we can play with our new menu entries and check if they are working
-as expected:
-
-.. code-block:: text
-
-    $ ptxdist menuconfig
-    $ ptxdist targetinstall foo
-
-Whenever we change a *FOO* related menu entry, PTXdist should detect it
-and re-build the package when a new build is started.
-
-.. _external_dependencies:
-
-Managing External Compile Time Dependencies
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-While running the prepare stage, it could happen that it fails due to a
-missing external dependency.
-
-For example:
-
-.. code-block:: text
-
-    checking whether zlib exists....failed
-
-In this example, our new package depends on the compression library
-*zlib*. PTXdist comes with a target *zlib*. All we need to do in this
-case is to declare that our new package *foo* depends on *zlib*. This
-kind of dependency is managed in the menu file of our new package by
-simply adding the ``select ZLIB`` line. After this addition our menu
-file looks like:
-
-.. code-block:: kconfig
-
-    ## SECTION=project_specific
-
-    config FOO
-           tristate
-           select ZLIB
-           prompt "foo"
-           help
-             FIXME
-
-    if FOO
-    config FOO_DEBUG
-           bool
-           prompt "add debug noise"
-
-    config FOO_BAR
-           bool
-           prompt "build bar"
-
-    endif
-
-PTXdist now builds the *zlib* first and our new package thereafter.
-
-Refer :ref:`external_dependencies_variants` for more specific dependency
-description.
-
-Managing External Compile Time Dependencies on Demand
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-It is good practice to add only those dependencies that are really
-required for the current configuration of the package. If the package
-provides the features *foo* and *bar* and its ``configure`` provides
-switches to enable/disable them independently, we can also add
-dependencies on demand. Let’s assume feature *foo* needs the compression
-library *libz* and *bar* needs the XML2 library *libxml2*. These
-libraries are only required at run-time if the corresponding feature is
-enabled. To add these dependencies on demand, the menu file looks like:
-
-.. code-block:: kconfig
-
-    ## SECTION=project_specific
-
-    config FOO
-           tristate
-           select ZLIB if FOO_FOO
-           select LIBXML2 if FOO_BAR
-           prompt "foo"
-           help
-             FIXME
-
-    if FOO
-    config FOO_DEBUG
-           bool
-           prompt "add debug noise"
-
-    config FOO_FOO
-           bool
-           prompt "build foo"
-
-    config FOO_BAR
-           bool
-           prompt "build bar"
-
-    endif
-
-.. important:: Do not add these ``select`` statements to the corresponding menu entry.
-  They must belong to the main menu entry of the package to ensure that
-  the calculation of the dependencies between the packages is done in a
-  correct manner.
-
-Managing External Runtime Dependencies
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Some packages are building all of their components and also installing
-them into the target’s sysroot. But only their *targetinstall* stage
-decides which parts are copied to the root filesystem. So, compiling and
-linking of our package will work, because everything required is found
-in the target’s sysroot.
-
-In our example there is a hidden dependency to the math library
-``libm``. Our new package was built successfully, because the linker was
-able to link our binaries against the ``libm`` from the toolchain. But
-in this case the ``libm`` must also be available in the target’s root
-filesystem to fulfil the run-time dependency: We have to force PTXdist to
-install ``libm``. ``libm`` is part of the *glibc* package, but is not
-installed by default (to keep the root filesystem small). So, it **does
-not** help to select the ``GLIBC`` symbol, to get a ``libm`` at run-time.
-
-The correct solution here is to add a ``select LIBC_M`` to our menu
-file. With all the additions above it now looks like:
-
-.. code-block:: kconfig
-
-    ## SECTION=project_specific
-
-    config FOO
-           tristate
-           select ZLIB if FOO_FOO
-           select LIBXML2 if FOO_BAR
-           select LIBC_M
-           prompt "foo"
-           help
-             FIXME
-
-    if FOO
-    config FOO_DEBUG
-           bool
-           prompt "add debug noise"
-
-    config FOO_FOO
-           bool
-           prompt "build foo"
-
-    config FOO_BAR
-           bool
-           prompt "build bar"
-
-    endif
-
-.. note:: There are other packages around, that do not install everything by
-  default. If our new package needs something special, we must take a look
-  into the menu of the other package how to force the required components
-  to be installed and add the corresponding ``selects`` to our own menu
-  file. In this case it does not help to enable the required parts in our
-  project configuration, because this has no effect on the build order!
-
-Managing Plain Makefile Packages
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Many packages are still coming with a plain ``Makefile``. The user has
-to adapt it to make it work in a cross compile environment as well.
-PTXdist can also handle this kind of packages. We only have to specify
-a special *prepare* and *compile* stage.
-
-Such packages often have no special need for any kind of preparation. In
-this we must instruct PTXdist to do nothing in the *prepare* stage:
-
-.. code-block:: make
-
-    FOO_CONF_TOOL := NO
-
-To compile the package, we can use ``make``\ ’s feature to overwrite
-variables used in the ``Makefile``. With this feature we can still use
-the original ``Makefile`` but with our own (cross compile) settings.
-
-Most of the time the generic compile rule can be used, only a few
-settings are required. For a well defined ``Makefile`` it is sufficient to
-set up the correct cross compile environment for the *compile* stage:
-
-.. code-block:: make
-
-    FOO_MAKE_ENV := $(CROSS_ENV)
-
-``make`` will be called in this case with:
-
-``$(FOO_MAKE_ENV) $(MAKE) -C $(FOO_DIR) $(FOO_MAKE_OPT)``
-
-So, in the rule file only the two variables ``FOO_MAKE_ENV`` and
-``FOO_MAKE_OPT`` must be set, to forward the required settings to the
-package’s buildsystem. If the package cannot be built in parallel, we
-can also add the ``FOO_MAKE_PAR := NO``. ``YES`` is the default.
-
-Managing CMake/QMake/Meson Packages
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Building packages that use ``cmake``, ``qmake`` or ``meson`` is much like
-building packages with an autotools based buildsystem. We need to specify
-the configuration tool:
-
-.. code-block:: make
-
-    FOO_CONF_TOOL := cmake
-
-or
-
-.. code-block:: make
-
-    FOO_CONF_TOOL := qmake
-
-or respectively
-
-.. code-block:: make
-
-    FOO_CONF_TOOL := meson
-
-And provide the correct configuration options. The syntax is different so
-PTXdist provides additional macros to simplify configurable features.
-For ``cmake`` the configuration options typically look like this:
-
-.. code-block:: make
-
-    FOO_CONF_OPT := \
-    	$(CROSS_CMAKE_USR) \
-    	-DBUILD_TESTS:BOOL=OFF \
-    	-DENABLE_BAR:BOOL=$(call ptx/onoff, PTXCONF_FOO_BAR)
-
-For ``qmake`` the configuration options typically look like this:
-
-.. code-block:: make
-
-    FOO_CONF_OPT := \
-    	$(CROSS_QMAKE_OPT) \
-    	PREFIX=/usr
-
-And for ``meson`` the configuration options typically look like this:
-
-.. code-block:: make
-
-    FOO_CONF_OPT := \
-    	$(CROSS_MESON_USR) \
-    	-Dbar=$(call ptx/truefalse,PTXCONF_FOO_BAR)
-
-Please note that currently only host and target ``cmake``\/``meson`` packages
-and only target ``qmake`` packages are supported.
-
-Managing Python Packages
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-As with any other package, the correct configuration tool must be selected
-for Python packages:
-
-.. code-block:: make
-
-    FOO_CONF_TOOL := python
-
-.. note:: For Python3 packages the value must be ``python3``.
-
-No Makefiles are used when building Python packages so the usual ``make``
-and ``make install`` for the *compile* and *install* stages cannot be used.
-PTXdist will call ``python setup.py build`` and ``python setup.py install``
-instead.
-
-.. note:: *FOO* is still the name of our example package. It must be
-  replaced by the real package name.
-
-
-.. _patching_packages:
-
-Patching Packages
-~~~~~~~~~~~~~~~~~
-
-There can be various reasons why a package must be patched:
-
--  Package is broken for cross compile environments
-
--  Package is broken within a specific feature
-
--  Package is vulnerable and needs some fixes
-
--  or anything else (this case is the most common one)
-
-Ideally, those problems should be addressed in the original project,
-so any patches you add to your BSP or to PTXdist should also be submitted upstream.
-The upstream project can often provide better feedback, they can integrate your
-patch into a new release, and also maintain your changes as part of the project.
-This way we make sure that all advantages of the open source idea work for us;
-and your patch can be removed again later when a new release of the project is
-integrated into your BSP or into PTXdist.
-
-PTXdist handles patching automatically.
-After extracting the archive of a package, PTXdist checks for the existence of
-a patch directory named like its ``<PKG>_PATCHES`` variable, or, if this variable
-is not set, like its ``<PKG>`` variable.
-The patch directory is then searched in all locations listed by the
-``PTXDIST_PATH_PATCHES`` variable, and the first one found is used.
-Take an exemplary package ``foo`` with version ``1.1.0``:
-The variable ``FOO`` will have the value ``foo-1.1.0``, so PTXdist will look for
-a patch directory named ``foo-1.1.0`` in the following locations:
-
-#. the current layer:
-
-   a. project (``./patches/foo-1.1.0``)
-   b. platform (``./configs/|ptxdistPlatformConfigDir|/patches/foo-1.1.0``)
-
-#. any :ref:`base layers <layers-in-ptxdist>`,
-   applying the same search order as above for each layer recursively
-
-#. ptxdist (``<ptxdist/installation/path>/patches/foo-1.1.0``)
-
-The patches from the first location found are used. Note: Due to this
-search order, a PTXdist project can replace global patches from the
-PTXdist installation. This can be useful if a project sticks to a
-specific PTXdist revision but fixes from a more recent revision of
-PTXdist should be used.
-
-PTXdist uses the utilities *git*, *patch* or *quilt* to work with
-patches or patch series. We recommend *git*, as it can manage patch
-series in a very easy way.
-
-Creating a Patch Series for a Package
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-To create a patch series for the first time, we can run the following
-steps. We are still using our *foo-1.1.0* example package here:
-
-Using Quilt
-"""""""""""
-
-We create a special directory for the patch series in the local project
-directory:
-
-.. code-block:: text
-
-    $ mkdir -p patches/foo-1.1.0
-
-PTXdist expects a ``series`` file in the patch directory and at least
-one patch. Otherwise it fails. Due to the fact that we do not have any
-patch content yet, we’ll start with a dummy entry in the ``series`` file
-and an empty ``patch`` file.
-
-.. code-block:: text
-
-    $ touch patches/foo-1.1.0/dummy
-    $ echo dummy > patches/foo-1.1.0/series
-
-Next is to extract the package (if already done, we must remove it
-first):
-
-.. code-block:: text
-
-    $ ptxdist extract foo
-
-This will extract the archive and create a symbolic link in the build
-directory pointing to our local patch directory. Working this way will
-ensure that we do not lose our created patches if we enter
-``ptxdist clean foo`` by accident. In our case the patches are still
-present in ``patches/foo-1.1.0`` and can be used the next time we
-extract the package again.
-
-All we have to do now is to do the modification we need to make the
-package work. We change into the build directory and use quilt_ to
-create new patches, add files to respective patches, modify these files
-and refresh the patches to save our changes.
-See the *quilt* documentation (``man 1 quilt``) for more information.
-
-.. note:: For patches that are intended for PTXdist upstream use the git
-  workflow described below to get proper patch headers.
-
-.. _quilt: http://savannah.nongnu.org/projects/quilt
-
-Using Git
-"""""""""
-
-Create the patch directory like above for *quilt*,
-but only add an empty series file:
-
-.. code-block:: text
-
-    $ mkdir -p patches/foo-1.1.0
-    $ touch patches/foo-1.1.0/series
-
-Then extract the package with an additional command line switch:
-
-.. code-block:: text
-
-    $ ptxdist --git extract foo
-
-The empty series file makes PTXdist create a Git repository in the
-respective package build directory,
-and import the package source as the first commit.
-
-.. note:: Optionally, you can enable the setting *Developer Options →
-  use git to apply patches* in `ptxdist setup` to get this behaviour
-  as a default for every package.
-  However, note that this setting is meant for development only, and can lead
-  to failures – some packages try to determine if they are being compiled from
-  a Git source tree, and behave differently in that case.
-
-Then you can change into the package build directory
-(``platform-<name>/build-target/foo-1.1.0``),
-patch the required source files,
-and make Git commits on the way.
-The Git history should now look something like this:
-
-.. code-block:: text
-
-    $ git log --oneline --decorate
-    * df343e821851 (HEAD -> master) Makefile: don't build the tests
-    * 65a360c2bd60 strfry.c: frobnicate the excusator
-    * fdc315f6844c (tag: foobar-1.1.0, tag: base) initial commit
-
-Finally, call ``git ptx-patches`` to transform those Git commits into the patch
-series in the ``patches/foo-1.1.0`` folder.
-This way they don't get lost when cleaning the package.
-
-.. note:: PTXdist will only create a Git repository for packages with
-  patches.  To use Git to generate the first patch, create an empty series
-  file ``patches/foobar-1.1.0/series`` before extracting the packages. This
-  will tell PTXdist to use Git anyways and ``git ptx-patches`` will put the
-  patches there.
-
-Both approaches (Git and quilt) are not suitable for modifying files
-that are autogenerated in autotools-based buildsystems.
-Refer to the section :ref:`configure_rebuild` on how PTXdist can
-handle this special task.
-
-Adding More Patches to a Package
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-If we want to add more patches to an already patched package, we can use
-nearly the same way as creating patches for the first time. But if the
-patch series comes from the PTXdist main installation, we do not have
-write permissions to these directories (do NEVER work on the main
-installation directories, NEVER, NEVER, NEVER). Due to the search order
-in which PTXdist searches for patches for a specific package, we can
-copy the global patch series to our local project directory. Now we have
-the permissions to add more patches or modify the existing ones. Also
-*quilt* and *git* are our friends here to manage the patch series.
-
-If we think that our new patches are valuable also for others, or they
-fix an error, it could be a good idea to send these patches to PTXdist
-mainline, and to the upstream project too.
-
-
-.. _configure_rebuild:
-
-Modifying Autotoolized Packages
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Autotoolized packages are very picky when automatically generated files
-get patched. The patch order is very important in this case and
-sometimes it even fails and nobody knows why.
-
-To improve a package’s autotools-based build system, PTXdist comes with
-its own project local autotools to regenerate the autotools template
-files, instead of patching them. With this feature, only the template
-files must be patched, the required ``configure`` script and the
-``Makefile.in`` files are regenerated in the final stages of the
-*prepare* step.
-
-This feature works like the regular patching mechanism. The only
-difference is the additional ``autogen.sh`` file in the patch directory.
-If it exists and has execution permissions, it will be called after the
-package was patched (while the *extract* stage is running).
-
-Its content depends on developer needs; for the most simple case the
-content can be:
-
-.. code-block:: bash
-
-    #!/bin/bash
-
-    aclocal $ACLOCAL_FLAGS
-
-    libtoolize \
-            --force \
-            --copy
-
-    autoreconf \
-            --force \
-            --install \
-            --warnings=cross \
-            --warnings=syntax \
-            --warnings=obsolete \
-            --warnings=unsupported
-
-.. note:: In this way not yet autotoolized package can be autotoolized. We
-  just have to add the common autotool template files (``configure.ac``
-  and ``Makefile.am`` for example) via a patch series to the package
-  source and the ``autogen.sh`` to the patch directory.
-
-.. _adding_files:
-
-Adding Binary Only Files
-------------------------
-
-Sometimes a few binary files have to be added into the root filesystem.
-Or - to be more precise - some files, that do not need to be built in
-any way.
-
-On the other hand, sometimes files should be included that are not
-covered by any open source license and so, should not be shipped in the
-source code format.
-
-Add Binary Files File by File
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Doing to on a file by file base can happen by just using the ``install_copy``
-macro in the *targetinstall* stage in our own customized rules file.
-
-.. code-block:: none
-
-    @$(call install_copy, binary_example, 0, 0, 0644, \
-       </path/to/some/file/>ptx_logo.png, \
-       /example/ptx_logo.png)
-
-It copies the file ``ptx_logo.png`` from some location to target’s root
-filesystem. Refer :ref:`install_copy` for further information about using the
-``install_copy`` macro.
-
-The disadvantage of this method is: if we want to install more than one
-file, we need one call to the ``install_copy`` macro per file. This is
-even harder if not only a set of files is to be installed, but a whole
-directory tree with files instead.
-
-Add Binary Files via an Archive
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-If a whole tree of files is to be installed, working with a *tar* based
-archive could make life easier. In this case the archive itself provides
-all the required information the files are needing to be installed in a
-correct manner:
-
--  the file itself and its name
-
--  the directory structure and the final location of every file in this
-   structure
-
--  user and group ID on a per file base
-
-.. code-block:: none
-
-    @$(call install_archive, binary_example, -, -, \
-       </path/to/an/>archive.tgz, /)
-
-Refer :ref:`install_archive` for further information about using the
-``install_archive`` macro.
-
-Using an archive can be useful to install parts of the root filesystem
-that are not covered by any open source license. Its possible to ship
-the binaries within the regular BSP, without the need for their sources.
-However it is possible for the customer to re-create everything required
-from the BSP to get their target up and running again.
-
-Another use case for the archive method could be the support for
-different development teams. One team provides a software component in
-the archive format, the other team does not need to build it but can use
-it in the same way than every other software component.
-
-Creating a Rules File
-~~~~~~~~~~~~~~~~~~~~~
-
-Let PTXdist create one for us.
-
-.. code-block:: text
-
-    $ ptxdist newpackage file
-
-    ptxdist: creating a new 'file' package:
-
-    ptxdist: enter package name.......: my_binfiles
-    ptxdist: enter version number.....: 1
-    ptxdist: enter package author.....: My Name <me@my-org.com>
-    ptxdist: enter package section....: rootfs
-
-Now two new files are present in the BSP:
-
-#. ``rules/my_binfiles.in`` The template for the menu
-
-#. ``rules/my_binfiles.make`` The rules template
-
-Both files now must be customized to meet our requirements. Due to the
-answer *rootfs* to the “``enter package section``” question, we will
-find the new menu entry in:
-
-.. code-block:: text
-
-    Root Filesystem --->
-    	< > my_binfiles (NEW)
-
-Enabling this new entry will also run our stages in
-``rules/my_binfiles.make`` the next time we enter:
-
-.. code-block:: text
-
-    $ ptxdist go
-
-Creating New Package Templates
-------------------------------
-
-For larger projects it can be convenient to have project specific package
-templates. This can be achieved by either modifying existing templates or
-by creating completely new templates.
-
-Modifying a Template
-~~~~~~~~~~~~~~~~~~~~
-
-A template can be modified by providing new input files. This is easier
-than creating a new template but does not allow to specify new variables to
-substitute in the input files.
-
-PTXdist looks for template files the same way it looks for rules files. The
-only difference is, that it searches in the ``templates/`` subdirectory.
-So a modified ``./rules/templates/template-target-make`` can be used to
-tweak the ``target`` template.
-
-Creating a New Template
-~~~~~~~~~~~~~~~~~~~~~~~
-
-For a completely new template, some bash scripting is required. All shell
-code must be placed in a file named like this:
-``./scripts/lib/ptxd_lib_*.sh``.
-
-The minimum requirement for a new template is:
--  a shell function that creates the new package
--  registering the new template
-
-.. code-block:: sh
-
-    ptxd_template_new_mypkg() {
-        # create the package here
-    }
-    export -f ptxd_template_new_mypkg
-    ptxd_template_help_list[${#ptxd_template_help_list[@]}]="mypkg"
-    ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create awesome mypkg package"
-
-PTXdist provides several helper functions to simplify the template.
-Using those functions, the package creation process is split into two
-parts:
-
--  query the user for input and export variables.
--  create the new package files from the template source files by
-   substituting all instances of ``@<variable>@`` with the value of the
-   corresponding variable.
-
-A simple template function could look like this:
-
-.. code-block:: sh
-
-    ptxd_template_new_mypkg() {
-        ptxd_template_read_basic &&
-        ptxd_template_read "enter download section" DL_SECTION "foobar"
-        ptxd_template_read_author &&
-        export section="local_${dlsection}" &&
-        ptxd_template_write_rules
-    }
-
-This template requires ``rules/templates/template-mypkg-make`` and
-``rules/templates/template-mypkg-in`` as source files. They could be
-derived from the ``target`` template with a simple modification:
-
-.. code-block:: make
-
-    @PACKAGE@_SUFFIX	:= tar.xz
-    @PACKAGE@_URL	:= http://dl.my-company.local/downloads/@DL_SECTION@/$(@PACKAGE@).$(@PACKAGE@_SUFFIX)
-
-The helper functions that are used in the example above are defined in
-``scripts/lib/ptxd_lib_template.sh`` in the PTXdist source tree.
-
-The template is a normal shell function. Arbitrary things can be done here
-to create the new package. The helper functions are just the most
-convenient way to crate simple templates. It is also possible to create
-more files. For examples, the builtin ``genimage`` template creates a extra
-config file for the new package.
-
-.. _layers-in-ptxdist:
-
-Layers in PTXdist
------------------
-
-For better maintenance or other reasons, a PTXdist project can be split
-into multiple layers. Each layer has exactly the same directory hierarchy
-as described in :ref:`directory_hierarchy` and other chapters.
-
-All layers are explicitly stacked in the filesystem. The top layer is the
-workspace of the PTXdist project. Any ``selected_*`` links and the platform
-build directory are created here. The layer below is defined by the
-subdirectory or symlink named ``base/``. More can be stacked the same
-way, so ``base/base/`` is the third layer and so on.
-In many ways, PTXdist itself can be considered as the bottom layer. This is
-either implicit or explicit with one last ``base/`` symlink.
-
-A project can overwrite files provided by PTXdist in many different ways,
-e.g. rule files or files installed with :ref:`install_alternative` etc.
-This concept expands naturally to layers. Each layer can overwrite files
-provided by lower layers in the exact same way. Any files are always
-searched for in a strict layer by layer order.
-
-Writing Layer Aware Rules
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-For the most part, package rules work just as expected when multiple layers
-are used. Any layer specific handling is done implicitly by PTXdist.
-However, there are a few things that need special handling.
-
-The variables :ref:`PTXDIST_WORKSPACE<ptxdist_workspace>` and
-:ref:`PTXDIST_PLATFORMCONFIGDIR`<ptxdist_platformconfigdir>` always refer
-to the directories in the top layer. These variables might be used in rules
-files like this:
-
-.. code-block:: make
-
-   MY_KERNEL_CONFIG := $(PTXDIST_PLATFORMCONFIGDIR)/kernelconfig.special
-
-If the referenced file is in any layer but the top one then it will not
-be found. To handle use-cases like this, the macros :ref:`in_path` and
-:ref:`in_platformconfigdir` can be used:
-
-.. code-block:: make
-
-   MY_KERNEL_CONFIG := $(call ptx/in-platformconfigdir, kernelconfig.special)
-
-This way, the layers are searched top to bottom until the config file is
-found.
-
-PTXdist Config Files with Multiple Layers
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-In many cases a layer may want to modify the **ptxconfig** by enabling or
-disabling some options. Any changes must be propagated through the whole
-layer stack.
-
-The features and workflow described here apply to the **ptxconfig**, the
-**platformconfig** and any **collectionconfig** used in the project.
-
-To do this, PTXdist stores a delta config to the layer below and a full
-config file in each layer. If the two files are missing then the config is
-unchanged. The bottom layer has only the config file and no delta.
-
-At runtime, PTXdist will always use the full config file in the top layer
-where the config exists. Before doing so, it will ensure that the config is
-consistent across all layers. This means that, for any layer that contains a
-delta config, the full config file of the layer below has not changed since
-the delta config was last updated. If any inconsistency is detected,
-PTXdist will abort.
-
-For any command that modifies the config file, except ``oldconfig``,
-PTXdist will use kconfig implicitly on all layers to check if the config
-for this layer is up to date. This is a stricter check than the consistency
-validation. For example, if a new package was added to a layer without
-updating the **ptxconfig** then this will be detected and PTXdist will
-abort. If all other layers are up to date, then PTXdist will use the delta
-config of the top layer, apply it to the full config of the layer below
-and execute the specified command with the resulting config file.
-
-.. note:: If the config file does not exist yet on the top layer, then it
-  will be created if changes to the config are made. Similarly the config
-  will be deleted if the delta is empty after the changes. In either case
-  it may be necessary to update any ``selected_*`` link to point to the
-  correct config.
-
-If PTXdist detects an inconsistency or an out of date config file then it
-must be updated before they can be used. This can be done by using the
-``oldconfig`` command. In this special case, PTXdist will iterate from the
-bottom to the top layer and run ``oldconfig`` for each of them. It will
-use the delta config applied to the full config of the layer below at each
-step. This means that it's possible to enable or disable a option in the
-bottom layer and ``oldconfig`` will propagate this change to all other
-layers.
-
-Packages with kconfig Based Config Files
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-For packages such as the Linux kernel that have kconfig based config files,
-a lot of the infrastructure to handle config files and deltas across
-multiple layers can be reused. Consistency validation is done implicitly
-and ``menuconfig`` and other kconfig commands will use config files and
-deltas as expected.
-
-It's not possible to implicitly run ``oldconfig`` on other layers (this may
-require a different source tree for the packages), so any inconsistencies
-must be resolved manually by running ``oldconfig`` explicitly on each
-layer.
-
-The make macros that provide these features are currently used by the
-barebox and kernel packages and templates.
+.. toctree::
+   :glob:
+   :maxdepth: 2
+
+   dev_dir_hierarchy
+   dev_add_new_pkgs
+   dev_add_bin_only_files
+   dev_create_new_pkg_templates
+   dev_layers_in_ptxdist
-- 
2.27.0


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

[ptxdist] [PATCH 3/5] doc: move code signing docs from scripts/ into doc/

[Bastian Krause]

Signed-off-by: Bastian Krause <bst@pengutronix.de>
---
 doc/dev_code_signing.rst             | 36 ++++++++++++++++++++++++++++
 doc/dev_manual.rst                   |  1 +
 scripts/lib/ptxd_lib_code_signing.sh | 32 ++-----------------------
 3 files changed, 39 insertions(+), 30 deletions(-)
 create mode 100644 doc/dev_code_signing.rst

diff --git a/doc/dev_code_signing.rst b/doc/dev_code_signing.rst
new file mode 100644
index 000000000..de0087f8b
--- /dev/null
+++ b/doc/dev_code_signing.rst
@@ -0,0 +1,36 @@
+.. _code_signing:
+
+Code Signing
+------------
+
+This is an overview over the ptxdist signing infrastructure.
+ptxdist uses PKCS#11 internally for providing access to keys and certificates.
+Packages that wish to sign something should implement a PKCS#11 interface.
+
+As PKCS#11 URIs usually differ between different usecases (release vs.
+development) the URIs normally are not hardcoded in the package configuration.
+Instead, ptxdist has the idea of "roles" which are string identifiers used to
+access a single private/public key pair and a certificate.
+
+ptxdist supports Hardware Security Modules (HSM).
+In case a HSM is not present or shall not be used SoftHSM is used internally to
+transparently provide the same API internally.
+
+For each role a PKCS#11 URI must be known by ptxdist.
+In case of a HSM the keys and certificates are stored in the HSM, but ptxdist
+needs to know the PKCS#11 URI to access the keys.
+This is done in ptxdist rule files calling cs_set_uri <role> <uri>.
+For SoftHSM the URI is generated internally by ptxdist, but instead the
+keys/certificates for each role have have to be imported.
+This is done with the cs_import_* functions below.
+
+During each invocation of ptxdist exactly one key provider is active.
+The code signing provider can be chosen with the PTXCONF_CODE_SIGNING_PROVIDER
+variable.
+A code signing provider is a package resposible for providing the role <->
+PKCS#11 URI relationships in case a HSM is used or for providing the key
+material in case SoftHSM is used.
+
+A package which wants to sign something or which needs access to keys has to
+select CODE_SIGNING.
+This makes sure the keys are ready when the package is being built.
diff --git a/doc/dev_manual.rst b/doc/dev_manual.rst
index 47a77a9be..03a05a661 100644
--- a/doc/dev_manual.rst
+++ b/doc/dev_manual.rst
@@ -14,3 +14,4 @@ This chapter shows all (or most) of the details of how PTXdist works.
    dev_add_bin_only_files
    dev_create_new_pkg_templates
    dev_layers_in_ptxdist
+   dev_code_signing
diff --git a/scripts/lib/ptxd_lib_code_signing.sh b/scripts/lib/ptxd_lib_code_signing.sh
index a7779f821..34d7251fd 100644
--- a/scripts/lib/ptxd_lib_code_signing.sh
+++ b/scripts/lib/ptxd_lib_code_signing.sh
@@ -7,36 +7,8 @@
 #
 
 #
-# This is an overview over the ptxdist signing infrastructure. ptxdist
-# uses PKCS#11 internally for providing access to keys and certificates.
-# Packages that wish to sign something should implement a PKCS#11 interface.
-#
-# As PKCS#11 URIs usually differ between different usecases (release vs.
-# development) the URIs normally are not hardcoded in the package
-# configuration. Instead, ptxdist has the idea of "roles" which are string
-# identifiers used to access a single private/public key pair and a
-# certificate.
-#
-# ptxdist supports Hardware Security Modules (HSM). In case a HSM is not
-# present or shall not be used SoftHSM is used internally to transparently
-# provide the same API internally.
-#
-# For each role a PKCS#11 URI must be known by ptxdist. In case of a HSM
-# the keys and certificates are stored in the HSM, but ptxdist needs to know
-# the PKCS#11 URI to access the keys. This is done in ptxdist rule files
-# calling cs_set_uri <role> <uri>. For SoftHSM the URI is generated internally
-# by ptxdist, but instead the keys/certificates for each role have have to
-# be imported. This is done with the cs_import_* functions below.
-#
-# During each invocation of ptxdist exactly one key provider is active. The
-# code signing provider can be chosen with the PTXCONF_CODE_SIGNING_PROVIDER
-# variable. A code signing provider is a package resposible for providing
-# the role <-> PKCS#11 URI relationships in case a HSM is used or for providing
-# the key material in case SoftHSM is used.
-#
-# A package which wants to sign something or which needs access to keys has
-# to select CODE_SIGNING. This makes sure the keys are ready when the package
-# is being built.
+# See doc/dev_code_signing.inc for documentation about ptxdist's code signing
+# infrastructure.
 #
 
 cs_check_env() {
-- 
2.27.0


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

[ptxdist] [PATCH 4/5] doc: dev_code_signing: rework and extend code signing section

[Bastian Krause]

Signed-off-by: Bastian Krause <bst@pengutronix.de>
---
 doc/dev_code_signing.rst | 139 ++++++++++++++++++++++++++++++++-------
 1 file changed, 116 insertions(+), 23 deletions(-)

diff --git a/doc/dev_code_signing.rst b/doc/dev_code_signing.rst
index de0087f8b..c5b5786f9 100644
--- a/doc/dev_code_signing.rst
+++ b/doc/dev_code_signing.rst
@@ -3,34 +3,127 @@
 Code Signing
 ------------
 
-This is an overview over the ptxdist signing infrastructure.
-ptxdist uses PKCS#11 internally for providing access to keys and certificates.
-Packages that wish to sign something should implement a PKCS#11 interface.
+In order to make sure an artifact was created by a known authority and was not
+altered later, digital signatures play a key role when building firmware
+images.
+This is also essential when a verified boot chain is established, e.g. via
+*High Assurance Boot* (HAB), signed FIT images, and a verified root file
+system.
+
+PTXdist uses `PKCS#11 <pkcs11-doc_>`_ internally to provide access to keys and
+certificates, therefore code signing consumers should implement a PKCS#11
+interface to make use of PTXdist's code signing infrastructure.
 
 As PKCS#11 URIs usually differ between different usecases (release vs.
-development) the URIs normally are not hardcoded in the package configuration.
-Instead, ptxdist has the idea of "roles" which are string identifiers used to
+development) the URIs are usually not hardcoded in the package configuration.
+Instead, PTXdist has the idea of **roles** which are string identifiers used to
 access a single private/public key pair and a certificate.
 
-ptxdist supports Hardware Security Modules (HSM).
-In case a HSM is not present or shall not be used SoftHSM is used internally to
-transparently provide the same API internally.
+Finally, one or several **code signing providers** supply the mapping from
+roles to the respective key material or even provide it themselves for
+development.
+
+PTXdist supports *Hardware Security Modules* (HSM).
+In case an HSM is not present or shall not be used, PTXdist can emulate it
+using `SoftHSM <softhsm_>`_, while still transparently providing the same API
+to code signing consumers.
+
+.. _pkcs11-doc: https://www.cryptsoft.com/pkcs11doc/
+.. _softhsm: https://www.opendnssec.org/softhsm/
 
-For each role a PKCS#11 URI must be known by ptxdist.
-In case of a HSM the keys and certificates are stored in the HSM, but ptxdist
+.. _code_signing_providers:
+
+Code Signing Providers
+~~~~~~~~~~~~~~~~~~~~~~
+
+For each role a PKCS#11 URI must be known by PTXdist.
+In case of an HSM the keys and certificates are stored in the HSM, but PTXdist
 needs to know the PKCS#11 URI to access the keys.
-This is done in ptxdist rule files calling cs_set_uri <role> <uri>.
-For SoftHSM the URI is generated internally by ptxdist, but instead the
-keys/certificates for each role have have to be imported.
-This is done with the cs_import_* functions below.
-
-During each invocation of ptxdist exactly one key provider is active.
-The code signing provider can be chosen with the PTXCONF_CODE_SIGNING_PROVIDER
-variable.
-A code signing provider is a package resposible for providing the role <->
-PKCS#11 URI relationships in case a HSM is used or for providing the key
+For SoftHSM the URI is generated internally by PTXdist, but instead the
+keys/certificates for each role have to be imported by the code signing
+provider into the SoftHSM.
+
+A code signing provider is a package responsible for providing the role ↔
+PKCS#11 URI relationships in case an HSM is used, or for providing the key
 material in case SoftHSM is used.
 
-A package which wants to sign something or which needs access to keys has to
-select CODE_SIGNING.
-This makes sure the keys are ready when the package is being built.
+When ``PTXCONF_CODE_SIGNING`` is enabled exactly one code signing provider is
+active during each invocation of PTXdist.
+
+PTXdist comes equipped with a development code signing provider "devel"
+implemented via the package ``host-ptx-code-signing-dev``.
+It imports publicly available development keys for each role into the SoftHSM.
+
+.. important:: The ``host-ptx-code-signing-dev`` code signing provider can be
+  used to sign artifacts for development purposes, but **must not** be used for
+  production.
+
+Creating Custom Code Signing Providers
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When a set of release keys or project-specific development keys should be
+used (e.g. to achieve backward compatibility) a new code signing provider
+must be introduced.
+
+Use ``ptxdist newpackage code-signing-provider`` to generate such a new code
+signing provider.
+The generated files must now be adjusted to the use case, depending on whether
+a specific HSM or SoftHSM should be used.
+The generated shell script in ``local_src/<name>-code-signing/`` contains
+examples for both use cases.
+See :ref:`code_signing_helper_functions` for an explanation of the available
+code signing helpers.
+It is the code signing provider's responsibility to select the host
+tools required by the code signing helper functions it uses.
+In case of SoftHSM use cases the keys should also be placed inside
+``local_src/<name>-code-signing/``
+
+In case an HSM is used it is required to extend the ``CODE_SIGNING_ENV`` with
+additional environment variables via a pre rule in
+``$(PTXDIST_PLATFORMCONFIGDIR)/rules/pre/``.
+For example, for Nitrokey HSMs which use *OpenSC* the pre rule could look like
+this:
+
+.. code-block:: make
+
+    ifdef PTXCONF_CODE_SIGNING_PROVIDER_<NAME>
+    CODE_SIGNING_ENV += \
+    	PKCS11_MODULE_PATH="${PTXDIST_SYSROOT_HOST}/lib/pkcs11/opensc-pkcs11.so"
+    endif
+
+Note that the module is built in the BSP in this case (via
+``select HOST_OPENSC_PCSC`` in the code signing provider's menu file).
+This is not strictly required, it is also possible to use an otherwise
+distributed module, e.g. by the HSM manufacturer.
+
+Switching the code signing provider is now possible with
+``ptxdist platformconfig``, then navigate to *Code signing* → *Code signing
+provider*.
+
+Code Signing Consumers
+~~~~~~~~~~~~~~~~~~~~~~
+
+A package has to select ``CODE_SIGNING`` if it wants to sign something, or if
+it needs access to keys and/or certificates.
+The config symbol is available in ptxconfig as well as in platformconfig.
+Selecting this symbol makes sure the keys and certificates are ready when the
+package is being built.
+
+By adding ``CODE_SIGNING_ENV`` to the package's make/conf/image environment a
+tool implementing a PKCS#11 interface can access the HSM or SoftHSM.
+The PKCS#11 URI can be retrieved via :ref:`cs_get_uri` and passed on, usually
+also via an environment variable.
+
+:ref:`cs_get_ca` can be used to install a keyring to the root file system, e.g.:
+
+.. code-block:: none
+
+    $(call install_copy, rauc, 0, 0, 0644, \
+      $(shell cs_get_ca update), \
+      /etc/rauc/ca.cert.pem)
+
+.. note:: When code signing helper functions are used in make variables (e.g.
+  for environment definitions) recursively expanded variables must be used
+  (``=``, not ``:=``).
+  Otherwise the variable is expanded before a code signing provider can perform
+  its setup.
-- 
2.27.0


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

[ptxdist] [PATCH 5/5] doc: introduce ref_code_signing_helpers

[Bastian Krause]

Signed-off-by: Bastian Krause <bst@pengutronix.de>
---
 doc/ref_code_signing_helpers.inc | 246 +++++++++++++++++++++++++++++++
 doc/ref_manual.rst               |   1 +
 2 files changed, 247 insertions(+)
 create mode 100644 doc/ref_code_signing_helpers.inc

diff --git a/doc/ref_code_signing_helpers.inc b/doc/ref_code_signing_helpers.inc
new file mode 100644
index 000000000..cec5c9558
--- /dev/null
+++ b/doc/ref_code_signing_helpers.inc
@@ -0,0 +1,246 @@
+.. _code_signing_helper_functions:
+
+Code Signing Helper Functions
+-----------------------------
+
+PTXdist provides various bash helper functions to be used in :ref:`code signing
+providers <code_signing_providers>`.
+These helpers set up SoftHSM, define roles, perform PKCS#11 URI handling as
+well as key and certificate authority (CA) handling.
+
+PTXdist stores URIs and CAs using these helpers in
+``$(PTXDIST_SYSROOT_HOST)/var/lib/keys/<signing-provider>/<role>/{uri,ca.pem}``.
+
+.. _cs_init_softhsm:
+
+cs_init_softhsm
+~~~~~~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_init_softhsm
+
+Initialize SoftHSM, and set the initial pins.
+
+Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``
+
+.. _cs_define_role:
+
+cs_define_role
+~~~~~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_define_role <role>
+
+Define new key role.
+
+A default PKCS#11 URI is set implicitly as convenience for SoftHSM use cases.
+
+.. _cs_set_uri:
+
+cs_set_uri
+~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_set_uri <role> <URI>
+
+Set given PKCS#11 URI for role.
+
+Preconditions:
+
+- the role must have been defined (see :ref:`cs_define_role`)
+
+.. _cs_get_uri:
+
+cs_get_uri
+~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_get_uri <role>
+
+Get PKCS#11 URI for role.
+
+Preconditions:
+
+- the URI must have been set (see :ref:`cs_set_uri`)
+
+cs_import_cert_from_der
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_import_cert_from_der <role> <DER>
+
+Import certificate from a given DER file for role.
+To be used with SoftHSM only.
+
+Preconditions:
+
+- the role must have been defined (see :ref:`cs_define_role`)
+- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
+
+Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``
+
+cs_import_cert_from_pem
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_import_cert_from_pem <role> <PEM>
+
+Import certificate from a given PEM file for role.
+To be used with SoftHSM only.
+
+Preconditions:
+
+- the role must have been defined (see :ref:`cs_define_role`)
+- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
+
+Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
+
+cs_import_pubkey_from_pem
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_import_pubkey_from_pem <role> <PEM>
+
+Import public key from a given PEM file for role.
+To be used with SoftHSM only.
+
+Preconditions:
+
+- the role must have been defined (see :ref:`cs_define_role`)
+- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
+
+Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
+
+cs_import_privkey_from_pem
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_import_privkey_from_pem <role> <PEM>
+
+Import private key from a given PEM file for role.
+To be used with SoftHSM only.
+
+Preconditions:
+
+- the role must have been defined (see :ref:`cs_define_role`)
+- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
+
+Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
+
+cs_import_key_from_pem
+~~~~~~~~~~~~~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_import_key_from_pem <role> <PEM>
+
+Import private/public key pair from a given PEM file for role.
+To be used with SoftHSM only.
+
+Preconditions:
+
+- the role must have been defined (see :ref:`cs_define_role`)
+- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
+
+Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
+
+.. _cs_get_ca:
+
+cs_get_ca
+~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_get_ca <role>
+
+Get path to the CA in PEM format for role.
+
+Preconditions:
+
+- a certificate must have been appended to the CA
+  (see :ref:`cs_append_ca_from_pem`, :ref:`cs_append_ca_from_der`,
+  :ref:`cs_append_ca_from_uri`)
+
+.. _cs_append_ca_from_pem:
+
+cs_append_ca_from_pem
+~~~~~~~~~~~~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_append_ca_from_pem <role> <PEM>
+
+Append certificate from a given PEM file for role.
+
+Preconditions:
+
+- the role must have been defined (see :ref:`cs_define_role`)
+
+.. _cs_append_ca_from_der:
+
+cs_append_ca_from_der
+~~~~~~~~~~~~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_append_ca_from_der <role> <DER>
+
+Append certificate from a given DER file for role.
+
+Preconditions:
+
+- the role must have been defined (see :ref:`cs_define_role`)
+
+Necessary package dependencies for code signing provider: ``HOST_OPENSSL``
+
+.. _cs_append_ca_from_uri:
+
+cs_append_ca_from_uri
+~~~~~~~~~~~~~~~~~~~~~
+
+Usage:
+
+.. code-block:: bash
+
+    cs_append_ca_from_uri <role> [<URI>]
+
+Append certificate from a given PKCS#11 URI for role.
+If URI is omitted the already set URI for role is used.
+
+Preconditions:
+
+- the role must have been defined (see :ref:`cs_define_role`)
+
+Necessary package dependencies for code signing provider: ``HOST_OPENSSL``, ``HOST_EXTRACT_CERT``
diff --git a/doc/ref_manual.rst b/doc/ref_manual.rst
index 6541da3ea..e605e52a6 100644
--- a/doc/ref_manual.rst
+++ b/doc/ref_manual.rst
@@ -4,6 +4,7 @@ PTXdist Reference
 .. include:: ref_make_variables.inc
 .. include:: ref_make_macros.inc
 .. include:: ref_rule_file_layout.inc
+.. include:: ref_code_signing_helpers.inc
 
 .. _ptxdist_parameter_reference:
 
-- 
2.27.0


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 4/5] doc: dev_code_signing: rework and extend code signing section

[Roland Hieber]

On Mon, Jun 08, 2020 at 10:53:04AM +0200, Bastian Krause wrote:
> Signed-off-by: Bastian Krause <bst@pengutronix.de>
> ---
>  doc/dev_code_signing.rst | 139 ++++++++++++++++++++++++++++++++-------
>  1 file changed, 116 insertions(+), 23 deletions(-)
> 
> diff --git a/doc/dev_code_signing.rst b/doc/dev_code_signing.rst
> index de0087f8b..c5b5786f9 100644
> --- a/doc/dev_code_signing.rst
> +++ b/doc/dev_code_signing.rst
> @@ -3,34 +3,127 @@
>  Code Signing
>  ------------
>  
> -This is an overview over the ptxdist signing infrastructure.
> -ptxdist uses PKCS#11 internally for providing access to keys and certificates.
> -Packages that wish to sign something should implement a PKCS#11 interface.
> +In order to make sure an artifact was created by a known authority and was not
> +altered later, digital signatures play a key role when building firmware
> +images.
> +This is also essential when a verified boot chain is established, e.g. via
> +*High Assurance Boot* (HAB), signed FIT images, and a verified root file
> +system.
> +
> +PTXdist uses `PKCS#11 <pkcs11-doc_>`_ internally to provide access to keys and
> +certificates, therefore code signing consumers should implement a PKCS#11
> +interface to make use of PTXdist's code signing infrastructure.
>  
>  As PKCS#11 URIs usually differ between different usecases (release vs.
> -development) the URIs normally are not hardcoded in the package configuration.
> -Instead, ptxdist has the idea of "roles" which are string identifiers used to
> +development) the URIs are usually not hardcoded in the package configuration.
> +Instead, PTXdist has the idea of **roles** which are string identifiers used to
>  access a single private/public key pair and a certificate.
>  
> -ptxdist supports Hardware Security Modules (HSM).
> -In case a HSM is not present or shall not be used SoftHSM is used internally to
> -transparently provide the same API internally.
> +Finally, one or several **code signing providers** supply the mapping from
> +roles to the respective key material or even provide it themselves for
> +development.
> +
> +PTXdist supports *Hardware Security Modules* (HSM).
> +In case an HSM is not present or shall not be used, PTXdist can emulate it
> +using `SoftHSM <softhsm_>`_, while still transparently providing the same API
> +to code signing consumers.
> +
> +.. _pkcs11-doc: https://www.cryptsoft.com/pkcs11doc/
> +.. _softhsm: https://www.opendnssec.org/softhsm/
>  
> -For each role a PKCS#11 URI must be known by ptxdist.
> -In case of a HSM the keys and certificates are stored in the HSM, but ptxdist
> +.. _code_signing_providers:
> +
> +Code Signing Providers
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +For each role a PKCS#11 URI must be known by PTXdist.

"known to PTXdist" sounds more natural to me here, but that's only a
small nitpick in case someone finds any more issues that justify a full
v2 ;)

 - Roland

> +In case of an HSM the keys and certificates are stored in the HSM, but PTXdist
>  needs to know the PKCS#11 URI to access the keys.
> -This is done in ptxdist rule files calling cs_set_uri <role> <uri>.
> -For SoftHSM the URI is generated internally by ptxdist, but instead the
> -keys/certificates for each role have have to be imported.
> -This is done with the cs_import_* functions below.
> -
> -During each invocation of ptxdist exactly one key provider is active.
> -The code signing provider can be chosen with the PTXCONF_CODE_SIGNING_PROVIDER
> -variable.
> -A code signing provider is a package resposible for providing the role <->
> -PKCS#11 URI relationships in case a HSM is used or for providing the key
> +For SoftHSM the URI is generated internally by PTXdist, but instead the
> +keys/certificates for each role have to be imported by the code signing
> +provider into the SoftHSM.
> +
> +A code signing provider is a package responsible for providing the role ↔
> +PKCS#11 URI relationships in case an HSM is used, or for providing the key
>  material in case SoftHSM is used.
>  
> -A package which wants to sign something or which needs access to keys has to
> -select CODE_SIGNING.
> -This makes sure the keys are ready when the package is being built.
> +When ``PTXCONF_CODE_SIGNING`` is enabled exactly one code signing provider is
> +active during each invocation of PTXdist.
> +
> +PTXdist comes equipped with a development code signing provider "devel"
> +implemented via the package ``host-ptx-code-signing-dev``.
> +It imports publicly available development keys for each role into the SoftHSM.
> +
> +.. important:: The ``host-ptx-code-signing-dev`` code signing provider can be
> +  used to sign artifacts for development purposes, but **must not** be used for
> +  production.
> +
> +Creating Custom Code Signing Providers
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +When a set of release keys or project-specific development keys should be
> +used (e.g. to achieve backward compatibility) a new code signing provider
> +must be introduced.
> +
> +Use ``ptxdist newpackage code-signing-provider`` to generate such a new code
> +signing provider.
> +The generated files must now be adjusted to the use case, depending on whether
> +a specific HSM or SoftHSM should be used.
> +The generated shell script in ``local_src/<name>-code-signing/`` contains
> +examples for both use cases.
> +See :ref:`code_signing_helper_functions` for an explanation of the available
> +code signing helpers.
> +It is the code signing provider's responsibility to select the host
> +tools required by the code signing helper functions it uses.
> +In case of SoftHSM use cases the keys should also be placed inside
> +``local_src/<name>-code-signing/``
> +
> +In case an HSM is used it is required to extend the ``CODE_SIGNING_ENV`` with
> +additional environment variables via a pre rule in
> +``$(PTXDIST_PLATFORMCONFIGDIR)/rules/pre/``.
> +For example, for Nitrokey HSMs which use *OpenSC* the pre rule could look like
> +this:
> +
> +.. code-block:: make
> +
> +    ifdef PTXCONF_CODE_SIGNING_PROVIDER_<NAME>
> +    CODE_SIGNING_ENV += \
> +    	PKCS11_MODULE_PATH="${PTXDIST_SYSROOT_HOST}/lib/pkcs11/opensc-pkcs11.so"
> +    endif
> +
> +Note that the module is built in the BSP in this case (via
> +``select HOST_OPENSC_PCSC`` in the code signing provider's menu file).
> +This is not strictly required, it is also possible to use an otherwise
> +distributed module, e.g. by the HSM manufacturer.
> +
> +Switching the code signing provider is now possible with
> +``ptxdist platformconfig``, then navigate to *Code signing* → *Code signing
> +provider*.
> +
> +Code Signing Consumers
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +A package has to select ``CODE_SIGNING`` if it wants to sign something, or if
> +it needs access to keys and/or certificates.
> +The config symbol is available in ptxconfig as well as in platformconfig.
> +Selecting this symbol makes sure the keys and certificates are ready when the
> +package is being built.
> +
> +By adding ``CODE_SIGNING_ENV`` to the package's make/conf/image environment a
> +tool implementing a PKCS#11 interface can access the HSM or SoftHSM.
> +The PKCS#11 URI can be retrieved via :ref:`cs_get_uri` and passed on, usually
> +also via an environment variable.
> +
> +:ref:`cs_get_ca` can be used to install a keyring to the root file system, e.g.:
> +
> +.. code-block:: none
> +
> +    $(call install_copy, rauc, 0, 0, 0644, \
> +      $(shell cs_get_ca update), \
> +      /etc/rauc/ca.cert.pem)
> +
> +.. note:: When code signing helper functions are used in make variables (e.g.
> +  for environment definitions) recursively expanded variables must be used
> +  (``=``, not ``:=``).
> +  Otherwise the variable is expanded before a code signing provider can perform
> +  its setup.
> -- 
> 2.27.0
> 
> 

-- 
Roland Hieber, Pengutronix e.K.          | r.hieber@pengutronix.de     |
Steuerwalder Str. 21                     | https://www.pengutronix.de/ |
31137 Hildesheim, Germany                | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686         | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 0/5] Add code-signing-provider template, add code signing docs

[Roland Hieber]

On Mon, Jun 08, 2020 at 10:53:00AM +0200, Bastian Krause wrote:
> The code signing infrastructure is available since 2019 in PTXdist. Now
> it's time to document it, especially since RAUC also uses it.
> Unfortunately this did not make it into ptxdist-2020.06.0.
> 
> To make things easier introduce a code-signing-provider template.
> 
> Bastian Krause (5):
>   package templates: add code-signing-provider template
>   doc: dev_manual: split up into multiple files
>   doc: move code signing docs from scripts/ into doc/
>   doc: dev_code_signing: rework and extend code signing section
>   doc: introduce ref_code_signing_helpers

For the whole series:

Reviewed-by: Roland Hieber <rhi@pengutronix.de>


> 
>  doc/dev_add_bin_only_files.rst                |  105 +
>  doc/dev_add_new_pkgs.rst                      | 1339 +++++++++++++
>  doc/dev_code_signing.rst                      |  129 ++
>  doc/dev_create_new_pkg_templates.rst          |   77 +
>  doc/dev_dir_hierarchy.rst                     |  108 +
>  doc/dev_layers_in_ptxdist.rst                 |  111 ++
>  doc/dev_manual.rst                            | 1765 +----------------
>  doc/ref_code_signing_helpers.inc              |  246 +++
>  doc/ref_manual.rst                            |    1 +
>  .../code-signing-provider/ptxdist-set-keys.sh |   96 +
>  .../template-code-signing-provider-choice-in  |    5 +
>  .../template-code-signing-provider-in         |   16 +
>  .../template-code-signing-provider-make       |   41 +
>  scripts/lib/ptxd_lib_code_signing.sh          |   32 +-
>  scripts/lib/ptxd_lib_template.sh              |   16 +
>  15 files changed, 2302 insertions(+), 1785 deletions(-)
>  create mode 100644 doc/dev_add_bin_only_files.rst
>  create mode 100644 doc/dev_add_new_pkgs.rst
>  create mode 100644 doc/dev_code_signing.rst
>  create mode 100644 doc/dev_create_new_pkg_templates.rst
>  create mode 100644 doc/dev_dir_hierarchy.rst
>  create mode 100644 doc/dev_layers_in_ptxdist.rst
>  create mode 100644 doc/ref_code_signing_helpers.inc
>  create mode 100755 rules/templates/code-signing-provider/ptxdist-set-keys.sh
>  create mode 100644 rules/templates/template-code-signing-provider-choice-in
>  create mode 100644 rules/templates/template-code-signing-provider-in
>  create mode 100644 rules/templates/template-code-signing-provider-make
> 
> -- 
> 2.27.0
> 
> 

-- 
Roland Hieber, Pengutronix e.K.          | r.hieber@pengutronix.de     |
Steuerwalder Str. 21                     | https://www.pengutronix.de/ |
31137 Hildesheim, Germany                | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686         | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 0/5] Add code-signing-provider template, add code signing docs

[Ladislav Michl]

On Mon, Jun 08, 2020 at 12:17:03PM +0200, Roland Hieber wrote:
> On Mon, Jun 08, 2020 at 10:53:00AM +0200, Bastian Krause wrote:
> > The code signing infrastructure is available since 2019 in PTXdist. Now
> > it's time to document it, especially since RAUC also uses it.
> > Unfortunately this did not make it into ptxdist-2020.06.0.
> > 
> > To make things easier introduce a code-signing-provider template.
> > 
> > Bastian Krause (5):
> >   package templates: add code-signing-provider template
> >   doc: dev_manual: split up into multiple files
> >   doc: move code signing docs from scripts/ into doc/
> >   doc: dev_code_signing: rework and extend code signing section
> >   doc: introduce ref_code_signing_helpers
> 
> For the whole series:
> 
> Reviewed-by: Roland Hieber <rhi@pengutronix.de>

...and as I successfully migrated my BSP to code-signing-provider using
this documentation (and template):

Tested-by: Ladislav Michl <ladis@linux-mips.org>

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 4/5] doc: dev_code_signing: rework and extend code signing section

[Bastian Krause]

On 6/8/20 12:16 PM, Roland Hieber wrote:
> On Mon, Jun 08, 2020 at 10:53:04AM +0200, Bastian Krause wrote:
>> Signed-off-by: Bastian Krause <bst@pengutronix.de>
>> ---
>>  doc/dev_code_signing.rst | 139 ++++++++++++++++++++++++++++++++-------
>>  1 file changed, 116 insertions(+), 23 deletions(-)
>>
>> diff --git a/doc/dev_code_signing.rst b/doc/dev_code_signing.rst
>> index de0087f8b..c5b5786f9 100644
>> --- a/doc/dev_code_signing.rst
>> +++ b/doc/dev_code_signing.rst
>> @@ -3,34 +3,127 @@
>>  Code Signing
>>  ------------
>>  
>> -This is an overview over the ptxdist signing infrastructure.
>> -ptxdist uses PKCS#11 internally for providing access to keys and certificates.
>> -Packages that wish to sign something should implement a PKCS#11 interface.
>> +In order to make sure an artifact was created by a known authority and was not
>> +altered later, digital signatures play a key role when building firmware
>> +images.
>> +This is also essential when a verified boot chain is established, e.g. via
>> +*High Assurance Boot* (HAB), signed FIT images, and a verified root file
>> +system.
>> +
>> +PTXdist uses `PKCS#11 <pkcs11-doc_>`_ internally to provide access to keys and
>> +certificates, therefore code signing consumers should implement a PKCS#11
>> +interface to make use of PTXdist's code signing infrastructure.
>>  
>>  As PKCS#11 URIs usually differ between different usecases (release vs.
>> -development) the URIs normally are not hardcoded in the package configuration.
>> -Instead, ptxdist has the idea of "roles" which are string identifiers used to
>> +development) the URIs are usually not hardcoded in the package configuration.
>> +Instead, PTXdist has the idea of **roles** which are string identifiers used to
>>  access a single private/public key pair and a certificate.
>>  
>> -ptxdist supports Hardware Security Modules (HSM).
>> -In case a HSM is not present or shall not be used SoftHSM is used internally to
>> -transparently provide the same API internally.
>> +Finally, one or several **code signing providers** supply the mapping from
>> +roles to the respective key material or even provide it themselves for
>> +development.
>> +
>> +PTXdist supports *Hardware Security Modules* (HSM).
>> +In case an HSM is not present or shall not be used, PTXdist can emulate it
>> +using `SoftHSM <softhsm_>`_, while still transparently providing the same API
>> +to code signing consumers.
>> +
>> +.. _pkcs11-doc: https://www.cryptsoft.com/pkcs11doc/
>> +.. _softhsm: https://www.opendnssec.org/softhsm/
>>  
>> -For each role a PKCS#11 URI must be known by ptxdist.
>> -In case of a HSM the keys and certificates are stored in the HSM, but ptxdist
>> +.. _code_signing_providers:
>> +
>> +Code Signing Providers
>> +~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +For each role a PKCS#11 URI must be known by PTXdist.
> 
> "known to PTXdist" sounds more natural to me here, but that's only a
> small nitpick in case someone finds any more issues that justify a full
> v2 ;)

"For each role a PKCS#11 URI must be known by PTXdist." is a perfectly
valid sentence. "known to" as well as "known by" are applicable. I would
leave it this way, unless some grammar guru explains the difference in
this very sentence and argues Roland's variant fits better.

Regards,
Bastian

> 
>  - Roland
> 
>> +In case of an HSM the keys and certificates are stored in the HSM, but PTXdist
>>  needs to know the PKCS#11 URI to access the keys.
>> -This is done in ptxdist rule files calling cs_set_uri <role> <uri>.
>> -For SoftHSM the URI is generated internally by ptxdist, but instead the
>> -keys/certificates for each role have have to be imported.
>> -This is done with the cs_import_* functions below.
>> -
>> -During each invocation of ptxdist exactly one key provider is active.
>> -The code signing provider can be chosen with the PTXCONF_CODE_SIGNING_PROVIDER
>> -variable.
>> -A code signing provider is a package resposible for providing the role <->
>> -PKCS#11 URI relationships in case a HSM is used or for providing the key
>> +For SoftHSM the URI is generated internally by PTXdist, but instead the
>> +keys/certificates for each role have to be imported by the code signing
>> +provider into the SoftHSM.
>> +
>> +A code signing provider is a package responsible for providing the role ↔
>> +PKCS#11 URI relationships in case an HSM is used, or for providing the key
>>  material in case SoftHSM is used.
>>  
>> -A package which wants to sign something or which needs access to keys has to
>> -select CODE_SIGNING.
>> -This makes sure the keys are ready when the package is being built.
>> +When ``PTXCONF_CODE_SIGNING`` is enabled exactly one code signing provider is
>> +active during each invocation of PTXdist.
>> +
>> +PTXdist comes equipped with a development code signing provider "devel"
>> +implemented via the package ``host-ptx-code-signing-dev``.
>> +It imports publicly available development keys for each role into the SoftHSM.
>> +
>> +.. important:: The ``host-ptx-code-signing-dev`` code signing provider can be
>> +  used to sign artifacts for development purposes, but **must not** be used for
>> +  production.
>> +
>> +Creating Custom Code Signing Providers
>> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> +
>> +When a set of release keys or project-specific development keys should be
>> +used (e.g. to achieve backward compatibility) a new code signing provider
>> +must be introduced.
>> +
>> +Use ``ptxdist newpackage code-signing-provider`` to generate such a new code
>> +signing provider.
>> +The generated files must now be adjusted to the use case, depending on whether
>> +a specific HSM or SoftHSM should be used.
>> +The generated shell script in ``local_src/<name>-code-signing/`` contains
>> +examples for both use cases.
>> +See :ref:`code_signing_helper_functions` for an explanation of the available
>> +code signing helpers.
>> +It is the code signing provider's responsibility to select the host
>> +tools required by the code signing helper functions it uses.
>> +In case of SoftHSM use cases the keys should also be placed inside
>> +``local_src/<name>-code-signing/``
>> +
>> +In case an HSM is used it is required to extend the ``CODE_SIGNING_ENV`` with
>> +additional environment variables via a pre rule in
>> +``$(PTXDIST_PLATFORMCONFIGDIR)/rules/pre/``.
>> +For example, for Nitrokey HSMs which use *OpenSC* the pre rule could look like
>> +this:
>> +
>> +.. code-block:: make
>> +
>> +    ifdef PTXCONF_CODE_SIGNING_PROVIDER_<NAME>
>> +    CODE_SIGNING_ENV += \
>> +    	PKCS11_MODULE_PATH="${PTXDIST_SYSROOT_HOST}/lib/pkcs11/opensc-pkcs11.so"
>> +    endif
>> +
>> +Note that the module is built in the BSP in this case (via
>> +``select HOST_OPENSC_PCSC`` in the code signing provider's menu file).
>> +This is not strictly required, it is also possible to use an otherwise
>> +distributed module, e.g. by the HSM manufacturer.
>> +
>> +Switching the code signing provider is now possible with
>> +``ptxdist platformconfig``, then navigate to *Code signing* → *Code signing
>> +provider*.
>> +
>> +Code Signing Consumers
>> +~~~~~~~~~~~~~~~~~~~~~~
>> +
>> +A package has to select ``CODE_SIGNING`` if it wants to sign something, or if
>> +it needs access to keys and/or certificates.
>> +The config symbol is available in ptxconfig as well as in platformconfig.
>> +Selecting this symbol makes sure the keys and certificates are ready when the
>> +package is being built.
>> +
>> +By adding ``CODE_SIGNING_ENV`` to the package's make/conf/image environment a
>> +tool implementing a PKCS#11 interface can access the HSM or SoftHSM.
>> +The PKCS#11 URI can be retrieved via :ref:`cs_get_uri` and passed on, usually
>> +also via an environment variable.
>> +
>> +:ref:`cs_get_ca` can be used to install a keyring to the root file system, e.g.:
>> +
>> +.. code-block:: none
>> +
>> +    $(call install_copy, rauc, 0, 0, 0644, \
>> +      $(shell cs_get_ca update), \
>> +      /etc/rauc/ca.cert.pem)
>> +
>> +.. note:: When code signing helper functions are used in make variables (e.g.
>> +  for environment definitions) recursively expanded variables must be used
>> +  (``=``, not ``:=``).
>> +  Otherwise the variable is expanded before a code signing provider can perform
>> +  its setup.
>> -- 
>> 2.27.0
>>
>>
> 


-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 1/5] package templates: add code-signing-provider template

[Michael Olbrich]

On Mon, Jun 08, 2020 at 10:53:01AM +0200, Bastian Krause wrote:
> A ptxdist code signing provider is a package which selects the required
> host tools needed for the code signing helpers to work. A shell script
> is needed to define roles, set PKCS#11 URIs and import keys if SoftHSM
> is used. In order to simplify its creation provide a template along with
> an example script.

I think we should query whether a HSM or SoftHSM will be used and install
an appropriate script and set the correct dependencies.

> Signed-off-by: Bastian Krause <bst@pengutronix.de>
> ---
>  .../code-signing-provider/ptxdist-set-keys.sh | 96 +++++++++++++++++++
>  .../template-code-signing-provider-choice-in  |  5 +
>  .../template-code-signing-provider-in         | 16 ++++
>  .../template-code-signing-provider-make       | 41 ++++++++
>  scripts/lib/ptxd_lib_template.sh              | 16 ++++
>  5 files changed, 174 insertions(+)
>  create mode 100755 rules/templates/code-signing-provider/ptxdist-set-keys.sh
>  create mode 100644 rules/templates/template-code-signing-provider-choice-in
>  create mode 100644 rules/templates/template-code-signing-provider-in
>  create mode 100644 rules/templates/template-code-signing-provider-make
> 
> diff --git a/rules/templates/code-signing-provider/ptxdist-set-keys.sh b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
> new file mode 100755
> index 000000000..040a61534
> --- /dev/null
> +++ b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
> @@ -0,0 +1,96 @@
> +#!/bin/bash
> +
> +set -e
> +
> +set_fit_keys() {
> +	local r="image-kernel-fit"
> +	cs_define_role "${r}"
> +
> +	# HSM use case
> +	cs_set_uri "${r}" "pkcs11:token=foo;object=kernel-fit"
> +}
> +
> +import_fit_keys() {
> +	local fit_cert_dir=fit
> +	local r="image-kernel-fit"
> +	cs_define_role "${r}"
> +
> +	cs_import_cert_from_der "${r}" "${fit_cert_dir}/fit-4096-development.crt"
> +	cs_import_pubkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
> +	cs_import_privkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
> +}
> +
> +set_rauc_keys() {
> +	local r="update"
> +	cs_define_role "${r}"
> +	cs_set_uri "${r}" "pkcs11:token=foo;object=rauc"
> +	cs_append_ca_from_uri "${r}"
> +}
> +
> +import_rauc_keys() {
> +	local rauc_cert_dir=rauc
> +	local r="update"
> +	cs_define_role "${r}"
> +
> +	# SoftHSM use case
> +	cs_import_cert_from_pem "${r}" "${rauc_cert_dir}/rauc.cert.pem"
> +	cs_import_pubkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
> +	cs_import_privkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
> +
> +	cs_append_ca_from_uri "${r}"
> +}
> +
> +set_imx_habv4_keys() {
> +	# HSM use case, assuming it contains only 1st CSF/IMG key
> +	for i in 1 2 3 4; do
> +		r="imx-habv4-srk${i}"
> +		cs_define_role "${r}"
> +		cs_set_uri "${r}" "pkcs11:token=foo;object=srk-release${i}"
> +		cs_append_ca_from_uri "${r}"
> +	done
> +
> +	r="imx-habv4-csf1"
> +	cs_define_role ${r}
> +	cs_set_uri "${r}" "pkcs11:token=foo;object=csf1"
> +
> +	r="imx-habv4-img1"
> +	cs_define_role ${r}
> +	cs_set_uri "${r}" "pkcs11:token=foo;object=img1"
> +}
> +
> +import_imx_habv4_keys() {
> +	local imx_habv4_key_dir="habv4"
> +	local crts="${imx_habv4_key_dir}/crts"
> +	local keys="${imx_habv4_key_dir}/keys"
> +	local OPENSSL_KEYPASS="${imx_habv4_key_dir}/keys/key_pass.txt"
> +
> +	for i in 1 2 3 4; do
> +		r="imx-habv4-srk${i}"
> +		cs_define_role "${r}"
> +		cs_import_cert_from_der "${r}" "${crts}/SRK${i}_sha256_4096_65537_v3_ca_crt.der"
> +		cs_import_key_from_pem "${r}" "${keys}/SRK${i}_sha256_4096_65537_v3_ca_key.pem"
> +		cs_append_ca_from_uri "${r}"
> +
> +		r="imx-habv4-csf${i}"
> +		cs_define_role "${r}"
> +		cs_import_cert_from_der "${r}" "${crts}/CSF${i}_1_sha256_4096_65537_v3_usr_crt.der"
> +		cs_import_key_from_pem "${r}" "${keys}/CSF${i}_1_sha256_4096_65537_v3_usr_key.pem"
> +
> +		r="imx-habv4-img${i}"
> +		cs_define_role "${r}"
> +		cs_import_cert_from_der "${r}" "${crts}/IMG${i}_1_sha256_4096_65537_v3_usr_crt.der"
> +		cs_import_key_from_pem "${r}" "${keys}/IMG${i}_1_sha256_4096_65537_v3_usr_key.pem"
> +	done
> +}
> +
> +
> +# HSM use case
> +#set_fit_keys
> +#set_rauc_keys
> +#set_imx_habv4_keys
> +
> +# or: SoftHSM use case
> +#cs_init_softhsm
> +#import_fit_keys
> +#import_rauc_keys
> +#import_imx_habv4_keys

Split this into two scripts that work for the correct use-case.
And use the wizard.sh to delete one and rename the other.

> diff --git a/rules/templates/template-code-signing-provider-choice-in b/rules/templates/template-code-signing-provider-choice-in
> new file mode 100644
> index 000000000..e2108f870
> --- /dev/null
> +++ b/rules/templates/template-code-signing-provider-choice-in
> @@ -0,0 +1,5 @@
> +## SECTION=code_signing_provider
> +
> +config CODE_SIGNING_PROVIDER_@PACKAGE@
> +	bool
> +	prompt "@package@"
> diff --git a/rules/templates/template-code-signing-provider-in b/rules/templates/template-code-signing-provider-in
> new file mode 100644
> index 000000000..a0c61e6ef
> --- /dev/null
> +++ b/rules/templates/template-code-signing-provider-in
> @@ -0,0 +1,16 @@
> +## SECTION=code_signing
> +
> +config CODE_SIGNING
> +	select HOST_@PACKAGE@_CODE_SIGNING if CODE_SIGNING_PROVIDER_@PACKAGE@
> +
> +config CODE_SIGNING_PROVIDER
> +	default "@package@" if CODE_SIGNING_PROVIDER_@PACKAGE@
> +
> +config HOST_@PACKAGE@_CODE_SIGNING
> +	bool
> +	select HOST_OPENSC
> +	select HOST_LIBP11
> +	select HOST_OPENSSL
> +	#select HOST_SOFTHSM
> +	#select HOST_OPENSC_PCSC
> +	#select HOST_EXTRACT_CERT

We can substitute multi-line values here. So just

@DEPENDENCIES@

and set that to the correct full list of dependencies in the script.


> diff --git a/rules/templates/template-code-signing-provider-make b/rules/templates/template-code-signing-provider-make
> new file mode 100644
> index 000000000..94830d92e
> --- /dev/null
> +++ b/rules/templates/template-code-signing-provider-make
> @@ -0,0 +1,41 @@
> +# -*-makefile-*-
> +#
> +# Copyright (C) @YEAR@ by @AUTHOR@
> +#
> +# For further information about the PTXdist project and license conditions
> +# see the README file.
> +#
> +
> +#
> +# We provide this package
> +#
> +HOST_PACKAGES-$(PTXCONF_HOST_@PACKAGE@_CODE_SIGNING) += host-@package@-code-signing
> +
> +#
> +# Paths and names
> +#
> +HOST_@PACKAGE@_CODE_SIGNING_VERSION	:= @VERSION@
> +HOST_@PACKAGE@_CODE_SIGNING		:= @package@-code-signing-$(HOST_@PACKAGE@_CODE_SIGNING_VERSION)
> +HOST_@PACKAGE@_CODE_SIGNING_URL		:= file://local_src/@package@-code-signing
> +HOST_@PACKAGE@_CODE_SIGNING_DIR		:= $(HOST_BUILDDIR)/$(HOST_@PACKAGE@_CODE_SIGNING)
> +
> +HOST_@PACKAGE@_CODE_SIGNING_CONF_TOOL	:= NO
> +
> +# ----------------------------------------------------------------------------
> +# Compile
> +# ----------------------------------------------------------------------------
> +
> +HOST_@PACKAGE@_CODE_SIGNING_MAKE_ENV	:= \
> +	$(CODE_SIGNING_ENV)
> +
> +$(STATEDIR)/host-@package@-code-signing.compile:
> +	@$(call targetinfo)
> +	@$(call world/execute, HOST_@PACKAGE@_CODE_SIGNING, \
> +		./ptxdist-set-keys.sh)
> +	@$(call touch)
> +
> +$(STATEDIR)/host-@package@-code-signing.install:
> +	@$(call targetinfo)
> +	@$(call touch)
> +
> +# vim: syntax=make
> diff --git a/scripts/lib/ptxd_lib_template.sh b/scripts/lib/ptxd_lib_template.sh
> index f39e6e033..b89981f45 100644
> --- a/scripts/lib/ptxd_lib_template.sh
> +++ b/scripts/lib/ptxd_lib_template.sh
> @@ -460,3 +460,19 @@ ptxd_template_new_blspec_entry() {
>  export -f ptxd_template_new_blspec_entry
>  ptxd_template_help_list[${#ptxd_template_help_list[@]}]="blspec-entry"
>  ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create package for a bootloader spec entry"
> +
> +ptxd_template_new_code_signing_provider() {
> +    export class="host-"
> +    ptxd_template_read_basic &&
> +    ptxd_template_read_author &&

The question for the type should be here. Maybe provide an list and the
user must input the index number or something like that.

Michael

> +    package_filename="${package_filename}-code-signing"
> +    ptxd_template_write_platform_rules
> +    local template_file="$(ptxd_template_file "${template}-choice-in")"
> +    local filename="${PTXDIST_PLATFORMCONFIGDIR}/platforms/${class}${package_filename}-choice.in"
> +    ptxd_template_filter "${template_file}" "${filename}"
> +    package="${package}-code-signing"
> +    ptxd_template_write_src
> +}
> +export -f ptxd_template_new_code_signing_provider
> +ptxd_template_help_list[${#ptxd_template_help_list[@]}]="code-signing-provider"
> +ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create package for a code signing provider"
> -- 
> 2.27.0
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 5/5] doc: introduce ref_code_signing_helpers

[Michael Olbrich]

On Mon, Jun 08, 2020 at 10:53:05AM +0200, Bastian Krause wrote:
> Signed-off-by: Bastian Krause <bst@pengutronix.de>

This does not apply on master. The sections in doc/ref_manual.rst are now
separate pages. Can you please rebase.

Michael

> ---
>  doc/ref_code_signing_helpers.inc | 246 +++++++++++++++++++++++++++++++
>  doc/ref_manual.rst               |   1 +
>  2 files changed, 247 insertions(+)
>  create mode 100644 doc/ref_code_signing_helpers.inc
> 
> diff --git a/doc/ref_code_signing_helpers.inc b/doc/ref_code_signing_helpers.inc
> new file mode 100644
> index 000000000..cec5c9558
> --- /dev/null
> +++ b/doc/ref_code_signing_helpers.inc
> @@ -0,0 +1,246 @@
> +.. _code_signing_helper_functions:
> +
> +Code Signing Helper Functions
> +-----------------------------
> +
> +PTXdist provides various bash helper functions to be used in :ref:`code signing
> +providers <code_signing_providers>`.
> +These helpers set up SoftHSM, define roles, perform PKCS#11 URI handling as
> +well as key and certificate authority (CA) handling.
> +
> +PTXdist stores URIs and CAs using these helpers in
> +``$(PTXDIST_SYSROOT_HOST)/var/lib/keys/<signing-provider>/<role>/{uri,ca.pem}``.
> +
> +.. _cs_init_softhsm:
> +
> +cs_init_softhsm
> +~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_init_softhsm
> +
> +Initialize SoftHSM, and set the initial pins.
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``
> +
> +.. _cs_define_role:
> +
> +cs_define_role
> +~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_define_role <role>
> +
> +Define new key role.
> +
> +A default PKCS#11 URI is set implicitly as convenience for SoftHSM use cases.
> +
> +.. _cs_set_uri:
> +
> +cs_set_uri
> +~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_set_uri <role> <URI>
> +
> +Set given PKCS#11 URI for role.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +
> +.. _cs_get_uri:
> +
> +cs_get_uri
> +~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_get_uri <role>
> +
> +Get PKCS#11 URI for role.
> +
> +Preconditions:
> +
> +- the URI must have been set (see :ref:`cs_set_uri`)
> +
> +cs_import_cert_from_der
> +~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_import_cert_from_der <role> <DER>
> +
> +Import certificate from a given DER file for role.
> +To be used with SoftHSM only.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``
> +
> +cs_import_cert_from_pem
> +~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_import_cert_from_pem <role> <PEM>
> +
> +Import certificate from a given PEM file for role.
> +To be used with SoftHSM only.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
> +
> +cs_import_pubkey_from_pem
> +~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_import_pubkey_from_pem <role> <PEM>
> +
> +Import public key from a given PEM file for role.
> +To be used with SoftHSM only.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
> +
> +cs_import_privkey_from_pem
> +~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_import_privkey_from_pem <role> <PEM>
> +
> +Import private key from a given PEM file for role.
> +To be used with SoftHSM only.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
> +
> +cs_import_key_from_pem
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_import_key_from_pem <role> <PEM>
> +
> +Import private/public key pair from a given PEM file for role.
> +To be used with SoftHSM only.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
> +
> +.. _cs_get_ca:
> +
> +cs_get_ca
> +~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_get_ca <role>
> +
> +Get path to the CA in PEM format for role.
> +
> +Preconditions:
> +
> +- a certificate must have been appended to the CA
> +  (see :ref:`cs_append_ca_from_pem`, :ref:`cs_append_ca_from_der`,
> +  :ref:`cs_append_ca_from_uri`)
> +
> +.. _cs_append_ca_from_pem:
> +
> +cs_append_ca_from_pem
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_append_ca_from_pem <role> <PEM>
> +
> +Append certificate from a given PEM file for role.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +
> +.. _cs_append_ca_from_der:
> +
> +cs_append_ca_from_der
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_append_ca_from_der <role> <DER>
> +
> +Append certificate from a given DER file for role.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_OPENSSL``
> +
> +.. _cs_append_ca_from_uri:
> +
> +cs_append_ca_from_uri
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_append_ca_from_uri <role> [<URI>]
> +
> +Append certificate from a given PKCS#11 URI for role.
> +If URI is omitted the already set URI for role is used.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_OPENSSL``, ``HOST_EXTRACT_CERT``
> diff --git a/doc/ref_manual.rst b/doc/ref_manual.rst
> index 6541da3ea..e605e52a6 100644
> --- a/doc/ref_manual.rst
> +++ b/doc/ref_manual.rst
> @@ -4,6 +4,7 @@ PTXdist Reference
>  .. include:: ref_make_variables.inc
>  .. include:: ref_make_macros.inc
>  .. include:: ref_rule_file_layout.inc
> +.. include:: ref_code_signing_helpers.inc
>  
>  .. _ptxdist_parameter_reference:
>  
> -- 
> 2.27.0
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 5/5] doc: introduce ref_code_signing_helpers

[Michael Olbrich]

On Mon, Jun 08, 2020 at 10:53:05AM +0200, Bastian Krause wrote:
> Signed-off-by: Bastian Krause <bst@pengutronix.de>
> ---
>  doc/ref_code_signing_helpers.inc | 246 +++++++++++++++++++++++++++++++
>  doc/ref_manual.rst               |   1 +
>  2 files changed, 247 insertions(+)
>  create mode 100644 doc/ref_code_signing_helpers.inc
> 
> diff --git a/doc/ref_code_signing_helpers.inc b/doc/ref_code_signing_helpers.inc
> new file mode 100644
> index 000000000..cec5c9558
> --- /dev/null
> +++ b/doc/ref_code_signing_helpers.inc
> @@ -0,0 +1,246 @@
> +.. _code_signing_helper_functions:
> +
> +Code Signing Helper Functions
> +-----------------------------
> +
> +PTXdist provides various bash helper functions to be used in :ref:`code signing
> +providers <code_signing_providers>`.
> +These helpers set up SoftHSM, define roles, perform PKCS#11 URI handling as
> +well as key and certificate authority (CA) handling.

Maybe split/sort the helpers into groups. For provider (hsm/softhsm) and
consumer.

Michael

> +PTXdist stores URIs and CAs using these helpers in
> +``$(PTXDIST_SYSROOT_HOST)/var/lib/keys/<signing-provider>/<role>/{uri,ca.pem}``.
> +
> +.. _cs_init_softhsm:
> +
> +cs_init_softhsm
> +~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_init_softhsm
> +
> +Initialize SoftHSM, and set the initial pins.
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``
> +
> +.. _cs_define_role:
> +
> +cs_define_role
> +~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_define_role <role>
> +
> +Define new key role.
> +
> +A default PKCS#11 URI is set implicitly as convenience for SoftHSM use cases.
> +
> +.. _cs_set_uri:
> +
> +cs_set_uri
> +~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_set_uri <role> <URI>
> +
> +Set given PKCS#11 URI for role.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +
> +.. _cs_get_uri:
> +
> +cs_get_uri
> +~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_get_uri <role>
> +
> +Get PKCS#11 URI for role.
> +
> +Preconditions:
> +
> +- the URI must have been set (see :ref:`cs_set_uri`)
> +
> +cs_import_cert_from_der
> +~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_import_cert_from_der <role> <DER>
> +
> +Import certificate from a given DER file for role.
> +To be used with SoftHSM only.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``
> +
> +cs_import_cert_from_pem
> +~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_import_cert_from_pem <role> <PEM>
> +
> +Import certificate from a given PEM file for role.
> +To be used with SoftHSM only.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
> +
> +cs_import_pubkey_from_pem
> +~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_import_pubkey_from_pem <role> <PEM>
> +
> +Import public key from a given PEM file for role.
> +To be used with SoftHSM only.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
> +
> +cs_import_privkey_from_pem
> +~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_import_privkey_from_pem <role> <PEM>
> +
> +Import private key from a given PEM file for role.
> +To be used with SoftHSM only.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
> +
> +cs_import_key_from_pem
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_import_key_from_pem <role> <PEM>
> +
> +Import private/public key pair from a given PEM file for role.
> +To be used with SoftHSM only.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +- SoftHSM must have been initialized (see :ref:`cs_init_softhsm`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_SOFTHSM``, ``HOST_OPENSSL``
> +
> +.. _cs_get_ca:
> +
> +cs_get_ca
> +~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_get_ca <role>
> +
> +Get path to the CA in PEM format for role.
> +
> +Preconditions:
> +
> +- a certificate must have been appended to the CA
> +  (see :ref:`cs_append_ca_from_pem`, :ref:`cs_append_ca_from_der`,
> +  :ref:`cs_append_ca_from_uri`)
> +
> +.. _cs_append_ca_from_pem:
> +
> +cs_append_ca_from_pem
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_append_ca_from_pem <role> <PEM>
> +
> +Append certificate from a given PEM file for role.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +
> +.. _cs_append_ca_from_der:
> +
> +cs_append_ca_from_der
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_append_ca_from_der <role> <DER>
> +
> +Append certificate from a given DER file for role.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_OPENSSL``
> +
> +.. _cs_append_ca_from_uri:
> +
> +cs_append_ca_from_uri
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +Usage:
> +
> +.. code-block:: bash
> +
> +    cs_append_ca_from_uri <role> [<URI>]
> +
> +Append certificate from a given PKCS#11 URI for role.
> +If URI is omitted the already set URI for role is used.
> +
> +Preconditions:
> +
> +- the role must have been defined (see :ref:`cs_define_role`)
> +
> +Necessary package dependencies for code signing provider: ``HOST_OPENSSL``, ``HOST_EXTRACT_CERT``
> diff --git a/doc/ref_manual.rst b/doc/ref_manual.rst
> index 6541da3ea..e605e52a6 100644
> --- a/doc/ref_manual.rst
> +++ b/doc/ref_manual.rst
> @@ -4,6 +4,7 @@ PTXdist Reference
>  .. include:: ref_make_variables.inc
>  .. include:: ref_make_macros.inc
>  .. include:: ref_rule_file_layout.inc
> +.. include:: ref_code_signing_helpers.inc
>  
>  .. _ptxdist_parameter_reference:
>  
> -- 
> 2.27.0
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 1/5] package templates: add code-signing-provider template

[Michael Olbrich]

On Fri, Jun 12, 2020 at 11:18:25AM +0200, Michael Olbrich wrote:
> On Mon, Jun 08, 2020 at 10:53:01AM +0200, Bastian Krause wrote:
> > A ptxdist code signing provider is a package which selects the required
> > host tools needed for the code signing helpers to work. A shell script
> > is needed to define roles, set PKCS#11 URIs and import keys if SoftHSM
> > is used. In order to simplify its creation provide a template along with
> > an example script.
> 
> I think we should query whether a HSM or SoftHSM will be used and install
> an appropriate script and set the correct dependencies.
> 
> > Signed-off-by: Bastian Krause <bst@pengutronix.de>
> > ---
> >  .../code-signing-provider/ptxdist-set-keys.sh | 96 +++++++++++++++++++
> >  .../template-code-signing-provider-choice-in  |  5 +
> >  .../template-code-signing-provider-in         | 16 ++++
> >  .../template-code-signing-provider-make       | 41 ++++++++
> >  scripts/lib/ptxd_lib_template.sh              | 16 ++++
> >  5 files changed, 174 insertions(+)
> >  create mode 100755 rules/templates/code-signing-provider/ptxdist-set-keys.sh
> >  create mode 100644 rules/templates/template-code-signing-provider-choice-in
> >  create mode 100644 rules/templates/template-code-signing-provider-in
> >  create mode 100644 rules/templates/template-code-signing-provider-make
> > 
> > diff --git a/rules/templates/code-signing-provider/ptxdist-set-keys.sh b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
> > new file mode 100755
> > index 000000000..040a61534
> > --- /dev/null
> > +++ b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
> > @@ -0,0 +1,96 @@
> > +#!/bin/bash
> > +
> > +set -e
> > +
> > +set_fit_keys() {
> > +	local r="image-kernel-fit"
> > +	cs_define_role "${r}"
> > +
> > +	# HSM use case
> > +	cs_set_uri "${r}" "pkcs11:token=foo;object=kernel-fit"
> > +}
> > +
> > +import_fit_keys() {
> > +	local fit_cert_dir=fit
> > +	local r="image-kernel-fit"
> > +	cs_define_role "${r}"
> > +
> > +	cs_import_cert_from_der "${r}" "${fit_cert_dir}/fit-4096-development.crt"
> > +	cs_import_pubkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
> > +	cs_import_privkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
> > +}
> > +
> > +set_rauc_keys() {
> > +	local r="update"
> > +	cs_define_role "${r}"
> > +	cs_set_uri "${r}" "pkcs11:token=foo;object=rauc"
> > +	cs_append_ca_from_uri "${r}"
> > +}
> > +
> > +import_rauc_keys() {
> > +	local rauc_cert_dir=rauc
> > +	local r="update"
> > +	cs_define_role "${r}"
> > +
> > +	# SoftHSM use case
> > +	cs_import_cert_from_pem "${r}" "${rauc_cert_dir}/rauc.cert.pem"
> > +	cs_import_pubkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
> > +	cs_import_privkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
> > +
> > +	cs_append_ca_from_uri "${r}"
> > +}
> > +
> > +set_imx_habv4_keys() {
> > +	# HSM use case, assuming it contains only 1st CSF/IMG key
> > +	for i in 1 2 3 4; do
> > +		r="imx-habv4-srk${i}"
> > +		cs_define_role "${r}"
> > +		cs_set_uri "${r}" "pkcs11:token=foo;object=srk-release${i}"
> > +		cs_append_ca_from_uri "${r}"
> > +	done
> > +
> > +	r="imx-habv4-csf1"
> > +	cs_define_role ${r}
> > +	cs_set_uri "${r}" "pkcs11:token=foo;object=csf1"
> > +
> > +	r="imx-habv4-img1"
> > +	cs_define_role ${r}
> > +	cs_set_uri "${r}" "pkcs11:token=foo;object=img1"
> > +}
> > +
> > +import_imx_habv4_keys() {
> > +	local imx_habv4_key_dir="habv4"
> > +	local crts="${imx_habv4_key_dir}/crts"
> > +	local keys="${imx_habv4_key_dir}/keys"
> > +	local OPENSSL_KEYPASS="${imx_habv4_key_dir}/keys/key_pass.txt"
> > +
> > +	for i in 1 2 3 4; do
> > +		r="imx-habv4-srk${i}"
> > +		cs_define_role "${r}"
> > +		cs_import_cert_from_der "${r}" "${crts}/SRK${i}_sha256_4096_65537_v3_ca_crt.der"
> > +		cs_import_key_from_pem "${r}" "${keys}/SRK${i}_sha256_4096_65537_v3_ca_key.pem"
> > +		cs_append_ca_from_uri "${r}"
> > +
> > +		r="imx-habv4-csf${i}"
> > +		cs_define_role "${r}"
> > +		cs_import_cert_from_der "${r}" "${crts}/CSF${i}_1_sha256_4096_65537_v3_usr_crt.der"
> > +		cs_import_key_from_pem "${r}" "${keys}/CSF${i}_1_sha256_4096_65537_v3_usr_key.pem"
> > +
> > +		r="imx-habv4-img${i}"
> > +		cs_define_role "${r}"
> > +		cs_import_cert_from_der "${r}" "${crts}/IMG${i}_1_sha256_4096_65537_v3_usr_crt.der"
> > +		cs_import_key_from_pem "${r}" "${keys}/IMG${i}_1_sha256_4096_65537_v3_usr_key.pem"
> > +	done
> > +}
> > +
> > +
> > +# HSM use case
> > +#set_fit_keys
> > +#set_rauc_keys
> > +#set_imx_habv4_keys
> > +
> > +# or: SoftHSM use case
> > +#cs_init_softhsm
> > +#import_fit_keys
> > +#import_rauc_keys
> > +#import_imx_habv4_keys
> 
> Split this into two scripts that work for the correct use-case.
> And use the wizard.sh to delete one and rename the other.
> 
> > diff --git a/rules/templates/template-code-signing-provider-choice-in b/rules/templates/template-code-signing-provider-choice-in
> > new file mode 100644
> > index 000000000..e2108f870
> > --- /dev/null
> > +++ b/rules/templates/template-code-signing-provider-choice-in
> > @@ -0,0 +1,5 @@
> > +## SECTION=code_signing_provider
> > +
> > +config CODE_SIGNING_PROVIDER_@PACKAGE@
> > +	bool
> > +	prompt "@package@"
> > diff --git a/rules/templates/template-code-signing-provider-in b/rules/templates/template-code-signing-provider-in
> > new file mode 100644
> > index 000000000..a0c61e6ef
> > --- /dev/null
> > +++ b/rules/templates/template-code-signing-provider-in
> > @@ -0,0 +1,16 @@
> > +## SECTION=code_signing
> > +
> > +config CODE_SIGNING
> > +	select HOST_@PACKAGE@_CODE_SIGNING if CODE_SIGNING_PROVIDER_@PACKAGE@
> > +
> > +config CODE_SIGNING_PROVIDER
> > +	default "@package@" if CODE_SIGNING_PROVIDER_@PACKAGE@
> > +
> > +config HOST_@PACKAGE@_CODE_SIGNING
> > +	bool
> > +	select HOST_OPENSC
> > +	select HOST_LIBP11
> > +	select HOST_OPENSSL
> > +	#select HOST_SOFTHSM
> > +	#select HOST_OPENSC_PCSC
> > +	#select HOST_EXTRACT_CERT
> 
> We can substitute multi-line values here. So just
> 
> @DEPENDENCIES@
> 
> and set that to the correct full list of dependencies in the script.
> 
> 
> > diff --git a/rules/templates/template-code-signing-provider-make b/rules/templates/template-code-signing-provider-make
> > new file mode 100644
> > index 000000000..94830d92e
> > --- /dev/null
> > +++ b/rules/templates/template-code-signing-provider-make
> > @@ -0,0 +1,41 @@
> > +# -*-makefile-*-
> > +#
> > +# Copyright (C) @YEAR@ by @AUTHOR@
> > +#
> > +# For further information about the PTXdist project and license conditions
> > +# see the README file.
> > +#
> > +
> > +#
> > +# We provide this package
> > +#
> > +HOST_PACKAGES-$(PTXCONF_HOST_@PACKAGE@_CODE_SIGNING) += host-@package@-code-signing
> > +
> > +#
> > +# Paths and names
> > +#
> > +HOST_@PACKAGE@_CODE_SIGNING_VERSION	:= @VERSION@
> > +HOST_@PACKAGE@_CODE_SIGNING		:= @package@-code-signing-$(HOST_@PACKAGE@_CODE_SIGNING_VERSION)
> > +HOST_@PACKAGE@_CODE_SIGNING_URL		:= file://local_src/@package@-code-signing
> > +HOST_@PACKAGE@_CODE_SIGNING_DIR		:= $(HOST_BUILDDIR)/$(HOST_@PACKAGE@_CODE_SIGNING)
> > +
> > +HOST_@PACKAGE@_CODE_SIGNING_CONF_TOOL	:= NO
> > +
> > +# ----------------------------------------------------------------------------
> > +# Compile
> > +# ----------------------------------------------------------------------------
> > +
> > +HOST_@PACKAGE@_CODE_SIGNING_MAKE_ENV	:= \
> > +	$(CODE_SIGNING_ENV)
> > +
> > +$(STATEDIR)/host-@package@-code-signing.compile:
> > +	@$(call targetinfo)
> > +	@$(call world/execute, HOST_@PACKAGE@_CODE_SIGNING, \
> > +		./ptxdist-set-keys.sh)
> > +	@$(call touch)
> > +
> > +$(STATEDIR)/host-@package@-code-signing.install:
> > +	@$(call targetinfo)
> > +	@$(call touch)
> > +
> > +# vim: syntax=make
> > diff --git a/scripts/lib/ptxd_lib_template.sh b/scripts/lib/ptxd_lib_template.sh
> > index f39e6e033..b89981f45 100644
> > --- a/scripts/lib/ptxd_lib_template.sh
> > +++ b/scripts/lib/ptxd_lib_template.sh
> > @@ -460,3 +460,19 @@ ptxd_template_new_blspec_entry() {
> >  export -f ptxd_template_new_blspec_entry
> >  ptxd_template_help_list[${#ptxd_template_help_list[@]}]="blspec-entry"
> >  ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create package for a bootloader spec entry"
> > +
> > +ptxd_template_new_code_signing_provider() {
> > +    export class="host-"
> > +    ptxd_template_read_basic &&
> > +    ptxd_template_read_author &&
> 
> The question for the type should be here. Maybe provide an list and the
> user must input the index number or something like that.

So, after reading the docs, I think there should be 3 options here:

1) SoftHSM
2) HSM (with OpenSC)
3) HSM (custom)

And for the HSM cases, the template should also provide the rules/pre/...
file for CODE_SIGNING_ENV. Maybe with the module name as a variable?
Substitute 'opensc-pkcs11' for OpenSC and 'fixme' otherwise.

The documentation should still contain the same information but add a note
what the template generates.

Michael

> Michael
> 
> > +    package_filename="${package_filename}-code-signing"
> > +    ptxd_template_write_platform_rules
> > +    local template_file="$(ptxd_template_file "${template}-choice-in")"
> > +    local filename="${PTXDIST_PLATFORMCONFIGDIR}/platforms/${class}${package_filename}-choice.in"
> > +    ptxd_template_filter "${template_file}" "${filename}"
> > +    package="${package}-code-signing"
> > +    ptxd_template_write_src
> > +}
> > +export -f ptxd_template_new_code_signing_provider
> > +ptxd_template_help_list[${#ptxd_template_help_list[@]}]="code-signing-provider"
> > +ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create package for a code signing provider"
> > -- 
> > 2.27.0
> > 
> > 
> > _______________________________________________
> > ptxdist mailing list
> > ptxdist@pengutronix.de
> > To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> > 
> 
> -- 
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 0/5] Add code-signing-provider template, add code signing docs

[Michael Olbrich]

On Mon, Jun 08, 2020 at 10:53:00AM +0200, Bastian Krause wrote:
> The code signing infrastructure is available since 2019 in PTXdist. Now
> it's time to document it, especially since RAUC also uses it.
> Unfortunately this did not make it into ptxdist-2020.06.0.
> 
> To make things easier introduce a code-signing-provider template.

This looks already quite nice. Thank you. Other than the comments I've made
for some of the patches, this is good to merge.

Michael

> Bastian Krause (5):
>   package templates: add code-signing-provider template
>   doc: dev_manual: split up into multiple files
>   doc: move code signing docs from scripts/ into doc/
>   doc: dev_code_signing: rework and extend code signing section
>   doc: introduce ref_code_signing_helpers
> 
>  doc/dev_add_bin_only_files.rst                |  105 +
>  doc/dev_add_new_pkgs.rst                      | 1339 +++++++++++++
>  doc/dev_code_signing.rst                      |  129 ++
>  doc/dev_create_new_pkg_templates.rst          |   77 +
>  doc/dev_dir_hierarchy.rst                     |  108 +
>  doc/dev_layers_in_ptxdist.rst                 |  111 ++
>  doc/dev_manual.rst                            | 1765 +----------------
>  doc/ref_code_signing_helpers.inc              |  246 +++
>  doc/ref_manual.rst                            |    1 +
>  .../code-signing-provider/ptxdist-set-keys.sh |   96 +
>  .../template-code-signing-provider-choice-in  |    5 +
>  .../template-code-signing-provider-in         |   16 +
>  .../template-code-signing-provider-make       |   41 +
>  scripts/lib/ptxd_lib_code_signing.sh          |   32 +-
>  scripts/lib/ptxd_lib_template.sh              |   16 +
>  15 files changed, 2302 insertions(+), 1785 deletions(-)
>  create mode 100644 doc/dev_add_bin_only_files.rst
>  create mode 100644 doc/dev_add_new_pkgs.rst
>  create mode 100644 doc/dev_code_signing.rst
>  create mode 100644 doc/dev_create_new_pkg_templates.rst
>  create mode 100644 doc/dev_dir_hierarchy.rst
>  create mode 100644 doc/dev_layers_in_ptxdist.rst
>  create mode 100644 doc/ref_code_signing_helpers.inc
>  create mode 100755 rules/templates/code-signing-provider/ptxdist-set-keys.sh
>  create mode 100644 rules/templates/template-code-signing-provider-choice-in
>  create mode 100644 rules/templates/template-code-signing-provider-in
>  create mode 100644 rules/templates/template-code-signing-provider-make
> 
> -- 
> 2.27.0
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 1/5] package templates: add code-signing-provider template

[Michael Olbrich]

On Fri, Jun 12, 2020 at 11:54:39AM +0200, Michael Olbrich wrote:
> On Fri, Jun 12, 2020 at 11:18:25AM +0200, Michael Olbrich wrote:
> > On Mon, Jun 08, 2020 at 10:53:01AM +0200, Bastian Krause wrote:
> > > A ptxdist code signing provider is a package which selects the required
> > > host tools needed for the code signing helpers to work. A shell script
> > > is needed to define roles, set PKCS#11 URIs and import keys if SoftHSM
> > > is used. In order to simplify its creation provide a template along with
> > > an example script.
> > 
> > I think we should query whether a HSM or SoftHSM will be used and install
> > an appropriate script and set the correct dependencies.
> > 
> > > Signed-off-by: Bastian Krause <bst@pengutronix.de>
> > > ---
> > >  .../code-signing-provider/ptxdist-set-keys.sh | 96 +++++++++++++++++++
> > >  .../template-code-signing-provider-choice-in  |  5 +
> > >  .../template-code-signing-provider-in         | 16 ++++
> > >  .../template-code-signing-provider-make       | 41 ++++++++
> > >  scripts/lib/ptxd_lib_template.sh              | 16 ++++
> > >  5 files changed, 174 insertions(+)
> > >  create mode 100755 rules/templates/code-signing-provider/ptxdist-set-keys.sh
> > >  create mode 100644 rules/templates/template-code-signing-provider-choice-in
> > >  create mode 100644 rules/templates/template-code-signing-provider-in
> > >  create mode 100644 rules/templates/template-code-signing-provider-make
> > > 
> > > diff --git a/rules/templates/code-signing-provider/ptxdist-set-keys.sh b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
> > > new file mode 100755
> > > index 000000000..040a61534
> > > --- /dev/null
> > > +++ b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
> > > @@ -0,0 +1,96 @@
> > > +#!/bin/bash
> > > +
> > > +set -e
> > > +
> > > +set_fit_keys() {
> > > +	local r="image-kernel-fit"
> > > +	cs_define_role "${r}"
> > > +
> > > +	# HSM use case
> > > +	cs_set_uri "${r}" "pkcs11:token=foo;object=kernel-fit"
> > > +}
> > > +
> > > +import_fit_keys() {
> > > +	local fit_cert_dir=fit
> > > +	local r="image-kernel-fit"
> > > +	cs_define_role "${r}"
> > > +
> > > +	cs_import_cert_from_der "${r}" "${fit_cert_dir}/fit-4096-development.crt"
> > > +	cs_import_pubkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
> > > +	cs_import_privkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
> > > +}
> > > +
> > > +set_rauc_keys() {
> > > +	local r="update"
> > > +	cs_define_role "${r}"
> > > +	cs_set_uri "${r}" "pkcs11:token=foo;object=rauc"
> > > +	cs_append_ca_from_uri "${r}"
> > > +}
> > > +
> > > +import_rauc_keys() {
> > > +	local rauc_cert_dir=rauc
> > > +	local r="update"
> > > +	cs_define_role "${r}"
> > > +
> > > +	# SoftHSM use case
> > > +	cs_import_cert_from_pem "${r}" "${rauc_cert_dir}/rauc.cert.pem"
> > > +	cs_import_pubkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
> > > +	cs_import_privkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
> > > +
> > > +	cs_append_ca_from_uri "${r}"
> > > +}
> > > +
> > > +set_imx_habv4_keys() {
> > > +	# HSM use case, assuming it contains only 1st CSF/IMG key
> > > +	for i in 1 2 3 4; do
> > > +		r="imx-habv4-srk${i}"
> > > +		cs_define_role "${r}"
> > > +		cs_set_uri "${r}" "pkcs11:token=foo;object=srk-release${i}"
> > > +		cs_append_ca_from_uri "${r}"
> > > +	done
> > > +
> > > +	r="imx-habv4-csf1"
> > > +	cs_define_role ${r}
> > > +	cs_set_uri "${r}" "pkcs11:token=foo;object=csf1"
> > > +
> > > +	r="imx-habv4-img1"
> > > +	cs_define_role ${r}
> > > +	cs_set_uri "${r}" "pkcs11:token=foo;object=img1"
> > > +}
> > > +
> > > +import_imx_habv4_keys() {
> > > +	local imx_habv4_key_dir="habv4"
> > > +	local crts="${imx_habv4_key_dir}/crts"
> > > +	local keys="${imx_habv4_key_dir}/keys"
> > > +	local OPENSSL_KEYPASS="${imx_habv4_key_dir}/keys/key_pass.txt"
> > > +
> > > +	for i in 1 2 3 4; do
> > > +		r="imx-habv4-srk${i}"
> > > +		cs_define_role "${r}"
> > > +		cs_import_cert_from_der "${r}" "${crts}/SRK${i}_sha256_4096_65537_v3_ca_crt.der"
> > > +		cs_import_key_from_pem "${r}" "${keys}/SRK${i}_sha256_4096_65537_v3_ca_key.pem"
> > > +		cs_append_ca_from_uri "${r}"
> > > +
> > > +		r="imx-habv4-csf${i}"
> > > +		cs_define_role "${r}"
> > > +		cs_import_cert_from_der "${r}" "${crts}/CSF${i}_1_sha256_4096_65537_v3_usr_crt.der"
> > > +		cs_import_key_from_pem "${r}" "${keys}/CSF${i}_1_sha256_4096_65537_v3_usr_key.pem"
> > > +
> > > +		r="imx-habv4-img${i}"
> > > +		cs_define_role "${r}"
> > > +		cs_import_cert_from_der "${r}" "${crts}/IMG${i}_1_sha256_4096_65537_v3_usr_crt.der"
> > > +		cs_import_key_from_pem "${r}" "${keys}/IMG${i}_1_sha256_4096_65537_v3_usr_key.pem"
> > > +	done
> > > +}
> > > +
> > > +
> > > +# HSM use case
> > > +#set_fit_keys
> > > +#set_rauc_keys
> > > +#set_imx_habv4_keys
> > > +
> > > +# or: SoftHSM use case
> > > +#cs_init_softhsm
> > > +#import_fit_keys
> > > +#import_rauc_keys
> > > +#import_imx_habv4_keys
> > 
> > Split this into two scripts that work for the correct use-case.
> > And use the wizard.sh to delete one and rename the other.
> > 
> > > diff --git a/rules/templates/template-code-signing-provider-choice-in b/rules/templates/template-code-signing-provider-choice-in
> > > new file mode 100644
> > > index 000000000..e2108f870
> > > --- /dev/null
> > > +++ b/rules/templates/template-code-signing-provider-choice-in
> > > @@ -0,0 +1,5 @@
> > > +## SECTION=code_signing_provider
> > > +
> > > +config CODE_SIGNING_PROVIDER_@PACKAGE@
> > > +	bool
> > > +	prompt "@package@"
> > > diff --git a/rules/templates/template-code-signing-provider-in b/rules/templates/template-code-signing-provider-in
> > > new file mode 100644
> > > index 000000000..a0c61e6ef
> > > --- /dev/null
> > > +++ b/rules/templates/template-code-signing-provider-in
> > > @@ -0,0 +1,16 @@
> > > +## SECTION=code_signing
> > > +
> > > +config CODE_SIGNING
> > > +	select HOST_@PACKAGE@_CODE_SIGNING if CODE_SIGNING_PROVIDER_@PACKAGE@
> > > +
> > > +config CODE_SIGNING_PROVIDER
> > > +	default "@package@" if CODE_SIGNING_PROVIDER_@PACKAGE@
> > > +
> > > +config HOST_@PACKAGE@_CODE_SIGNING
> > > +	bool
> > > +	select HOST_OPENSC
> > > +	select HOST_LIBP11
> > > +	select HOST_OPENSSL
> > > +	#select HOST_SOFTHSM
> > > +	#select HOST_OPENSC_PCSC
> > > +	#select HOST_EXTRACT_CERT
> > 
> > We can substitute multi-line values here. So just
> > 
> > @DEPENDENCIES@
> > 
> > and set that to the correct full list of dependencies in the script.
> > 
> > 
> > > diff --git a/rules/templates/template-code-signing-provider-make b/rules/templates/template-code-signing-provider-make
> > > new file mode 100644
> > > index 000000000..94830d92e
> > > --- /dev/null
> > > +++ b/rules/templates/template-code-signing-provider-make
> > > @@ -0,0 +1,41 @@
> > > +# -*-makefile-*-
> > > +#
> > > +# Copyright (C) @YEAR@ by @AUTHOR@
> > > +#
> > > +# For further information about the PTXdist project and license conditions
> > > +# see the README file.
> > > +#
> > > +
> > > +#
> > > +# We provide this package
> > > +#
> > > +HOST_PACKAGES-$(PTXCONF_HOST_@PACKAGE@_CODE_SIGNING) += host-@package@-code-signing
> > > +
> > > +#
> > > +# Paths and names
> > > +#
> > > +HOST_@PACKAGE@_CODE_SIGNING_VERSION	:= @VERSION@
> > > +HOST_@PACKAGE@_CODE_SIGNING		:= @package@-code-signing-$(HOST_@PACKAGE@_CODE_SIGNING_VERSION)
> > > +HOST_@PACKAGE@_CODE_SIGNING_URL		:= file://local_src/@package@-code-signing
> > > +HOST_@PACKAGE@_CODE_SIGNING_DIR		:= $(HOST_BUILDDIR)/$(HOST_@PACKAGE@_CODE_SIGNING)
> > > +
> > > +HOST_@PACKAGE@_CODE_SIGNING_CONF_TOOL	:= NO
> > > +
> > > +# ----------------------------------------------------------------------------
> > > +# Compile
> > > +# ----------------------------------------------------------------------------
> > > +
> > > +HOST_@PACKAGE@_CODE_SIGNING_MAKE_ENV	:= \
> > > +	$(CODE_SIGNING_ENV)
> > > +
> > > +$(STATEDIR)/host-@package@-code-signing.compile:
> > > +	@$(call targetinfo)
> > > +	@$(call world/execute, HOST_@PACKAGE@_CODE_SIGNING, \
> > > +		./ptxdist-set-keys.sh)
> > > +	@$(call touch)
> > > +
> > > +$(STATEDIR)/host-@package@-code-signing.install:
> > > +	@$(call targetinfo)
> > > +	@$(call touch)
> > > +
> > > +# vim: syntax=make
> > > diff --git a/scripts/lib/ptxd_lib_template.sh b/scripts/lib/ptxd_lib_template.sh
> > > index f39e6e033..b89981f45 100644
> > > --- a/scripts/lib/ptxd_lib_template.sh
> > > +++ b/scripts/lib/ptxd_lib_template.sh
> > > @@ -460,3 +460,19 @@ ptxd_template_new_blspec_entry() {
> > >  export -f ptxd_template_new_blspec_entry
> > >  ptxd_template_help_list[${#ptxd_template_help_list[@]}]="blspec-entry"
> > >  ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create package for a bootloader spec entry"
> > > +
> > > +ptxd_template_new_code_signing_provider() {
> > > +    export class="host-"
> > > +    ptxd_template_read_basic &&
> > > +    ptxd_template_read_author &&
> > 
> > The question for the type should be here. Maybe provide an list and the
> > user must input the index number or something like that.
> 
> So, after reading the docs, I think there should be 3 options here:
> 
> 1) SoftHSM
> 2) HSM (with OpenSC)
> 3) HSM (custom)
> 
> And for the HSM cases, the template should also provide the rules/pre/...
> file for CODE_SIGNING_ENV. Maybe with the module name as a variable?
> Substitute 'opensc-pkcs11' for OpenSC and 'fixme' otherwise.

Or ask for the module name? I don't have a good understanding what is
usually necessary for other HSMs beyond the PKCS11_MODULE_PATH, so I don't
know if that makes sense.

> The documentation should still contain the same information but add a note
> what the template generates.
> 
> Michael
> 
> > Michael
> > 
> > > +    package_filename="${package_filename}-code-signing"
> > > +    ptxd_template_write_platform_rules
> > > +    local template_file="$(ptxd_template_file "${template}-choice-in")"
> > > +    local filename="${PTXDIST_PLATFORMCONFIGDIR}/platforms/${class}${package_filename}-choice.in"
> > > +    ptxd_template_filter "${template_file}" "${filename}"
> > > +    package="${package}-code-signing"
> > > +    ptxd_template_write_src
> > > +}
> > > +export -f ptxd_template_new_code_signing_provider
> > > +ptxd_template_help_list[${#ptxd_template_help_list[@]}]="code-signing-provider"
> > > +ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create package for a code signing provider"
> > > -- 
> > > 2.27.0
> > > 
> > > 
> > > _______________________________________________
> > > ptxdist mailing list
> > > ptxdist@pengutronix.de
> > > To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> > > 
> > 
> > -- 
> > Pengutronix e.K.                           |                             |
> > Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> > 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> > Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> > 
> > _______________________________________________
> > ptxdist mailing list
> > ptxdist@pengutronix.de
> > To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> > 
> 
> -- 
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 1/5] package templates: add code-signing-provider template

[Bastian Krause]

On 6/12/20 12:05 PM, Michael Olbrich wrote:
> On Fri, Jun 12, 2020 at 11:54:39AM +0200, Michael Olbrich wrote:
>> On Fri, Jun 12, 2020 at 11:18:25AM +0200, Michael Olbrich wrote:
>>> On Mon, Jun 08, 2020 at 10:53:01AM +0200, Bastian Krause wrote:
>>>> A ptxdist code signing provider is a package which selects the required
>>>> host tools needed for the code signing helpers to work. A shell script
>>>> is needed to define roles, set PKCS#11 URIs and import keys if SoftHSM
>>>> is used. In order to simplify its creation provide a template along with
>>>> an example script.
>>>
>>> I think we should query whether a HSM or SoftHSM will be used and install
>>> an appropriate script and set the correct dependencies.
>>>
>>>> Signed-off-by: Bastian Krause <bst@pengutronix.de>
>>>> ---
>>>>  .../code-signing-provider/ptxdist-set-keys.sh | 96 +++++++++++++++++++
>>>>  .../template-code-signing-provider-choice-in  |  5 +
>>>>  .../template-code-signing-provider-in         | 16 ++++
>>>>  .../template-code-signing-provider-make       | 41 ++++++++
>>>>  scripts/lib/ptxd_lib_template.sh              | 16 ++++
>>>>  5 files changed, 174 insertions(+)
>>>>  create mode 100755 rules/templates/code-signing-provider/ptxdist-set-keys.sh
>>>>  create mode 100644 rules/templates/template-code-signing-provider-choice-in
>>>>  create mode 100644 rules/templates/template-code-signing-provider-in
>>>>  create mode 100644 rules/templates/template-code-signing-provider-make
>>>>
>>>> diff --git a/rules/templates/code-signing-provider/ptxdist-set-keys.sh b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
>>>> new file mode 100755
>>>> index 000000000..040a61534
>>>> --- /dev/null
>>>> +++ b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
>>>> @@ -0,0 +1,96 @@
>>>> +#!/bin/bash
>>>> +
>>>> +set -e
>>>> +
>>>> +set_fit_keys() {
>>>> +	local r="image-kernel-fit"
>>>> +	cs_define_role "${r}"
>>>> +
>>>> +	# HSM use case
>>>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=kernel-fit"
>>>> +}
>>>> +
>>>> +import_fit_keys() {
>>>> +	local fit_cert_dir=fit
>>>> +	local r="image-kernel-fit"
>>>> +	cs_define_role "${r}"
>>>> +
>>>> +	cs_import_cert_from_der "${r}" "${fit_cert_dir}/fit-4096-development.crt"
>>>> +	cs_import_pubkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
>>>> +	cs_import_privkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
>>>> +}
>>>> +
>>>> +set_rauc_keys() {
>>>> +	local r="update"
>>>> +	cs_define_role "${r}"
>>>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=rauc"
>>>> +	cs_append_ca_from_uri "${r}"
>>>> +}
>>>> +
>>>> +import_rauc_keys() {
>>>> +	local rauc_cert_dir=rauc
>>>> +	local r="update"
>>>> +	cs_define_role "${r}"
>>>> +
>>>> +	# SoftHSM use case
>>>> +	cs_import_cert_from_pem "${r}" "${rauc_cert_dir}/rauc.cert.pem"
>>>> +	cs_import_pubkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
>>>> +	cs_import_privkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
>>>> +
>>>> +	cs_append_ca_from_uri "${r}"
>>>> +}
>>>> +
>>>> +set_imx_habv4_keys() {
>>>> +	# HSM use case, assuming it contains only 1st CSF/IMG key
>>>> +	for i in 1 2 3 4; do
>>>> +		r="imx-habv4-srk${i}"
>>>> +		cs_define_role "${r}"
>>>> +		cs_set_uri "${r}" "pkcs11:token=foo;object=srk-release${i}"
>>>> +		cs_append_ca_from_uri "${r}"
>>>> +	done
>>>> +
>>>> +	r="imx-habv4-csf1"
>>>> +	cs_define_role ${r}
>>>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=csf1"
>>>> +
>>>> +	r="imx-habv4-img1"
>>>> +	cs_define_role ${r}
>>>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=img1"
>>>> +}
>>>> +
>>>> +import_imx_habv4_keys() {
>>>> +	local imx_habv4_key_dir="habv4"
>>>> +	local crts="${imx_habv4_key_dir}/crts"
>>>> +	local keys="${imx_habv4_key_dir}/keys"
>>>> +	local OPENSSL_KEYPASS="${imx_habv4_key_dir}/keys/key_pass.txt"
>>>> +
>>>> +	for i in 1 2 3 4; do
>>>> +		r="imx-habv4-srk${i}"
>>>> +		cs_define_role "${r}"
>>>> +		cs_import_cert_from_der "${r}" "${crts}/SRK${i}_sha256_4096_65537_v3_ca_crt.der"
>>>> +		cs_import_key_from_pem "${r}" "${keys}/SRK${i}_sha256_4096_65537_v3_ca_key.pem"
>>>> +		cs_append_ca_from_uri "${r}"
>>>> +
>>>> +		r="imx-habv4-csf${i}"
>>>> +		cs_define_role "${r}"
>>>> +		cs_import_cert_from_der "${r}" "${crts}/CSF${i}_1_sha256_4096_65537_v3_usr_crt.der"
>>>> +		cs_import_key_from_pem "${r}" "${keys}/CSF${i}_1_sha256_4096_65537_v3_usr_key.pem"
>>>> +
>>>> +		r="imx-habv4-img${i}"
>>>> +		cs_define_role "${r}"
>>>> +		cs_import_cert_from_der "${r}" "${crts}/IMG${i}_1_sha256_4096_65537_v3_usr_crt.der"
>>>> +		cs_import_key_from_pem "${r}" "${keys}/IMG${i}_1_sha256_4096_65537_v3_usr_key.pem"
>>>> +	done
>>>> +}
>>>> +
>>>> +
>>>> +# HSM use case
>>>> +#set_fit_keys
>>>> +#set_rauc_keys
>>>> +#set_imx_habv4_keys
>>>> +
>>>> +# or: SoftHSM use case
>>>> +#cs_init_softhsm
>>>> +#import_fit_keys
>>>> +#import_rauc_keys
>>>> +#import_imx_habv4_keys
>>>
>>> Split this into two scripts that work for the correct use-case.
>>> And use the wizard.sh to delete one and rename the other.
>>>
>>>> diff --git a/rules/templates/template-code-signing-provider-choice-in b/rules/templates/template-code-signing-provider-choice-in
>>>> new file mode 100644
>>>> index 000000000..e2108f870
>>>> --- /dev/null
>>>> +++ b/rules/templates/template-code-signing-provider-choice-in
>>>> @@ -0,0 +1,5 @@
>>>> +## SECTION=code_signing_provider
>>>> +
>>>> +config CODE_SIGNING_PROVIDER_@PACKAGE@
>>>> +	bool
>>>> +	prompt "@package@"
>>>> diff --git a/rules/templates/template-code-signing-provider-in b/rules/templates/template-code-signing-provider-in
>>>> new file mode 100644
>>>> index 000000000..a0c61e6ef
>>>> --- /dev/null
>>>> +++ b/rules/templates/template-code-signing-provider-in
>>>> @@ -0,0 +1,16 @@
>>>> +## SECTION=code_signing
>>>> +
>>>> +config CODE_SIGNING
>>>> +	select HOST_@PACKAGE@_CODE_SIGNING if CODE_SIGNING_PROVIDER_@PACKAGE@
>>>> +
>>>> +config CODE_SIGNING_PROVIDER
>>>> +	default "@package@" if CODE_SIGNING_PROVIDER_@PACKAGE@
>>>> +
>>>> +config HOST_@PACKAGE@_CODE_SIGNING
>>>> +	bool
>>>> +	select HOST_OPENSC
>>>> +	select HOST_LIBP11
>>>> +	select HOST_OPENSSL
>>>> +	#select HOST_SOFTHSM
>>>> +	#select HOST_OPENSC_PCSC
>>>> +	#select HOST_EXTRACT_CERT
>>>
>>> We can substitute multi-line values here. So just
>>>
>>> @DEPENDENCIES@
>>>
>>> and set that to the correct full list of dependencies in the script.
>>>
>>>
>>>> diff --git a/rules/templates/template-code-signing-provider-make b/rules/templates/template-code-signing-provider-make
>>>> new file mode 100644
>>>> index 000000000..94830d92e
>>>> --- /dev/null
>>>> +++ b/rules/templates/template-code-signing-provider-make
>>>> @@ -0,0 +1,41 @@
>>>> +# -*-makefile-*-
>>>> +#
>>>> +# Copyright (C) @YEAR@ by @AUTHOR@
>>>> +#
>>>> +# For further information about the PTXdist project and license conditions
>>>> +# see the README file.
>>>> +#
>>>> +
>>>> +#
>>>> +# We provide this package
>>>> +#
>>>> +HOST_PACKAGES-$(PTXCONF_HOST_@PACKAGE@_CODE_SIGNING) += host-@package@-code-signing
>>>> +
>>>> +#
>>>> +# Paths and names
>>>> +#
>>>> +HOST_@PACKAGE@_CODE_SIGNING_VERSION	:= @VERSION@
>>>> +HOST_@PACKAGE@_CODE_SIGNING		:= @package@-code-signing-$(HOST_@PACKAGE@_CODE_SIGNING_VERSION)
>>>> +HOST_@PACKAGE@_CODE_SIGNING_URL		:= file://local_src/@package@-code-signing
>>>> +HOST_@PACKAGE@_CODE_SIGNING_DIR		:= $(HOST_BUILDDIR)/$(HOST_@PACKAGE@_CODE_SIGNING)
>>>> +
>>>> +HOST_@PACKAGE@_CODE_SIGNING_CONF_TOOL	:= NO
>>>> +
>>>> +# ----------------------------------------------------------------------------
>>>> +# Compile
>>>> +# ----------------------------------------------------------------------------
>>>> +
>>>> +HOST_@PACKAGE@_CODE_SIGNING_MAKE_ENV	:= \
>>>> +	$(CODE_SIGNING_ENV)
>>>> +
>>>> +$(STATEDIR)/host-@package@-code-signing.compile:
>>>> +	@$(call targetinfo)
>>>> +	@$(call world/execute, HOST_@PACKAGE@_CODE_SIGNING, \
>>>> +		./ptxdist-set-keys.sh)
>>>> +	@$(call touch)
>>>> +
>>>> +$(STATEDIR)/host-@package@-code-signing.install:
>>>> +	@$(call targetinfo)
>>>> +	@$(call touch)
>>>> +
>>>> +# vim: syntax=make
>>>> diff --git a/scripts/lib/ptxd_lib_template.sh b/scripts/lib/ptxd_lib_template.sh
>>>> index f39e6e033..b89981f45 100644
>>>> --- a/scripts/lib/ptxd_lib_template.sh
>>>> +++ b/scripts/lib/ptxd_lib_template.sh
>>>> @@ -460,3 +460,19 @@ ptxd_template_new_blspec_entry() {
>>>>  export -f ptxd_template_new_blspec_entry
>>>>  ptxd_template_help_list[${#ptxd_template_help_list[@]}]="blspec-entry"
>>>>  ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create package for a bootloader spec entry"
>>>> +
>>>> +ptxd_template_new_code_signing_provider() {
>>>> +    export class="host-"
>>>> +    ptxd_template_read_basic &&
>>>> +    ptxd_template_read_author &&
>>>
>>> The question for the type should be here. Maybe provide an list and the
>>> user must input the index number or something like that.
>>
>> So, after reading the docs, I think there should be 3 options here:
>>
>> 1) SoftHSM
>> 2) HSM (with OpenSC)
>> 3) HSM (custom)
>>
>> And for the HSM cases, the template should also provide the rules/pre/...
>> file for CODE_SIGNING_ENV. Maybe with the module name as a variable?
>> Substitute 'opensc-pkcs11' for OpenSC and 'fixme' otherwise.
> 
> Or ask for the module name? I don't have a good understanding what is
> usually necessary for other HSMs beyond the PKCS11_MODULE_PATH, so I don't
> know if that makes sense.

I've disucssed this with Marc some time ago. We came up with the idea of
setting the HSM specifics in CODE_SIGNING_ENV via another code signing
provider helper, e.g. "cs_append_env". In our opinion the code signing
provider should know (and set) such extra environment variables.

Regards,
Bastian

> 
>> The documentation should still contain the same information but add a note
>> what the template generates.
>>
>> Michael
>>
>>> Michael
>>>
>>>> +    package_filename="${package_filename}-code-signing"
>>>> +    ptxd_template_write_platform_rules
>>>> +    local template_file="$(ptxd_template_file "${template}-choice-in")"
>>>> +    local filename="${PTXDIST_PLATFORMCONFIGDIR}/platforms/${class}${package_filename}-choice.in"
>>>> +    ptxd_template_filter "${template_file}" "${filename}"
>>>> +    package="${package}-code-signing"
>>>> +    ptxd_template_write_src
>>>> +}
>>>> +export -f ptxd_template_new_code_signing_provider
>>>> +ptxd_template_help_list[${#ptxd_template_help_list[@]}]="code-signing-provider"
>>>> +ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create package for a code signing provider"
>>>> -- 
>>>> 2.27.0
>>>>
>>>>
>>>> _______________________________________________
>>>> ptxdist mailing list
>>>> ptxdist@pengutronix.de
>>>> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
>>>>
>>>
>>> -- 
>>> Pengutronix e.K.                           |                             |
>>> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
>>> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
>>> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
>>>
>>> _______________________________________________
>>> ptxdist mailing list
>>> ptxdist@pengutronix.de
>>> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
>>>
>>
>> -- 
>> Pengutronix e.K.                           |                             |
>> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
>> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
>> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
>>
>> _______________________________________________
>> ptxdist mailing list
>> ptxdist@pengutronix.de
>> To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
>>
> 


-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 1/5] package templates: add code-signing-provider template

[Bastian Krause]

On 6/12/20 11:18 AM, Michael Olbrich wrote:
> On Mon, Jun 08, 2020 at 10:53:01AM +0200, Bastian Krause wrote:
>> A ptxdist code signing provider is a package which selects the required
>> host tools needed for the code signing helpers to work. A shell script
>> is needed to define roles, set PKCS#11 URIs and import keys if SoftHSM
>> is used. In order to simplify its creation provide a template along with
>> an example script.
> 
> I think we should query whether a HSM or SoftHSM will be used and install
> an appropriate script and set the correct dependencies.
> 
>> Signed-off-by: Bastian Krause <bst@pengutronix.de>
>> ---
>>  .../code-signing-provider/ptxdist-set-keys.sh | 96 +++++++++++++++++++
>>  .../template-code-signing-provider-choice-in  |  5 +
>>  .../template-code-signing-provider-in         | 16 ++++
>>  .../template-code-signing-provider-make       | 41 ++++++++
>>  scripts/lib/ptxd_lib_template.sh              | 16 ++++
>>  5 files changed, 174 insertions(+)
>>  create mode 100755 rules/templates/code-signing-provider/ptxdist-set-keys.sh
>>  create mode 100644 rules/templates/template-code-signing-provider-choice-in
>>  create mode 100644 rules/templates/template-code-signing-provider-in
>>  create mode 100644 rules/templates/template-code-signing-provider-make
>>
>> diff --git a/rules/templates/code-signing-provider/ptxdist-set-keys.sh b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
>> new file mode 100755
>> index 000000000..040a61534
>> --- /dev/null
>> +++ b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
>> @@ -0,0 +1,96 @@
>> +#!/bin/bash
>> +
>> +set -e
>> +
>> +set_fit_keys() {
>> +	local r="image-kernel-fit"
>> +	cs_define_role "${r}"
>> +
>> +	# HSM use case
>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=kernel-fit"
>> +}
>> +
>> +import_fit_keys() {
>> +	local fit_cert_dir=fit
>> +	local r="image-kernel-fit"
>> +	cs_define_role "${r}"
>> +
>> +	cs_import_cert_from_der "${r}" "${fit_cert_dir}/fit-4096-development.crt"
>> +	cs_import_pubkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
>> +	cs_import_privkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
>> +}
>> +
>> +set_rauc_keys() {
>> +	local r="update"
>> +	cs_define_role "${r}"
>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=rauc"
>> +	cs_append_ca_from_uri "${r}"
>> +}
>> +
>> +import_rauc_keys() {
>> +	local rauc_cert_dir=rauc
>> +	local r="update"
>> +	cs_define_role "${r}"
>> +
>> +	# SoftHSM use case
>> +	cs_import_cert_from_pem "${r}" "${rauc_cert_dir}/rauc.cert.pem"
>> +	cs_import_pubkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
>> +	cs_import_privkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
>> +
>> +	cs_append_ca_from_uri "${r}"
>> +}
>> +
>> +set_imx_habv4_keys() {
>> +	# HSM use case, assuming it contains only 1st CSF/IMG key
>> +	for i in 1 2 3 4; do
>> +		r="imx-habv4-srk${i}"
>> +		cs_define_role "${r}"
>> +		cs_set_uri "${r}" "pkcs11:token=foo;object=srk-release${i}"
>> +		cs_append_ca_from_uri "${r}"
>> +	done
>> +
>> +	r="imx-habv4-csf1"
>> +	cs_define_role ${r}
>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=csf1"
>> +
>> +	r="imx-habv4-img1"
>> +	cs_define_role ${r}
>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=img1"
>> +}
>> +
>> +import_imx_habv4_keys() {
>> +	local imx_habv4_key_dir="habv4"
>> +	local crts="${imx_habv4_key_dir}/crts"
>> +	local keys="${imx_habv4_key_dir}/keys"
>> +	local OPENSSL_KEYPASS="${imx_habv4_key_dir}/keys/key_pass.txt"
>> +
>> +	for i in 1 2 3 4; do
>> +		r="imx-habv4-srk${i}"
>> +		cs_define_role "${r}"
>> +		cs_import_cert_from_der "${r}" "${crts}/SRK${i}_sha256_4096_65537_v3_ca_crt.der"
>> +		cs_import_key_from_pem "${r}" "${keys}/SRK${i}_sha256_4096_65537_v3_ca_key.pem"
>> +		cs_append_ca_from_uri "${r}"
>> +
>> +		r="imx-habv4-csf${i}"
>> +		cs_define_role "${r}"
>> +		cs_import_cert_from_der "${r}" "${crts}/CSF${i}_1_sha256_4096_65537_v3_usr_crt.der"
>> +		cs_import_key_from_pem "${r}" "${keys}/CSF${i}_1_sha256_4096_65537_v3_usr_key.pem"
>> +
>> +		r="imx-habv4-img${i}"
>> +		cs_define_role "${r}"
>> +		cs_import_cert_from_der "${r}" "${crts}/IMG${i}_1_sha256_4096_65537_v3_usr_crt.der"
>> +		cs_import_key_from_pem "${r}" "${keys}/IMG${i}_1_sha256_4096_65537_v3_usr_key.pem"
>> +	done
>> +}
>> +
>> +
>> +# HSM use case
>> +#set_fit_keys
>> +#set_rauc_keys
>> +#set_imx_habv4_keys
>> +
>> +# or: SoftHSM use case
>> +#cs_init_softhsm
>> +#import_fit_keys
>> +#import_rauc_keys
>> +#import_imx_habv4_keys
> 
> Split this into two scripts that work for the correct use-case.
> And use the wizard.sh to delete one and rename the other.
> 
>> diff --git a/rules/templates/template-code-signing-provider-choice-in b/rules/templates/template-code-signing-provider-choice-in
>> new file mode 100644
>> index 000000000..e2108f870
>> --- /dev/null
>> +++ b/rules/templates/template-code-signing-provider-choice-in
>> @@ -0,0 +1,5 @@
>> +## SECTION=code_signing_provider
>> +
>> +config CODE_SIGNING_PROVIDER_@PACKAGE@
>> +	bool
>> +	prompt "@package@"
>> diff --git a/rules/templates/template-code-signing-provider-in b/rules/templates/template-code-signing-provider-in
>> new file mode 100644
>> index 000000000..a0c61e6ef
>> --- /dev/null
>> +++ b/rules/templates/template-code-signing-provider-in
>> @@ -0,0 +1,16 @@
>> +## SECTION=code_signing
>> +
>> +config CODE_SIGNING
>> +	select HOST_@PACKAGE@_CODE_SIGNING if CODE_SIGNING_PROVIDER_@PACKAGE@
>> +
>> +config CODE_SIGNING_PROVIDER
>> +	default "@package@" if CODE_SIGNING_PROVIDER_@PACKAGE@
>> +
>> +config HOST_@PACKAGE@_CODE_SIGNING
>> +	bool
>> +	select HOST_OPENSC
>> +	select HOST_LIBP11
>> +	select HOST_OPENSSL
>> +	#select HOST_SOFTHSM
>> +	#select HOST_OPENSC_PCSC
>> +	#select HOST_EXTRACT_CERT
> 
> We can substitute multi-line values here. So just
> 
> @DEPENDENCIES@
> 
> and set that to the correct full list of dependencies in the script.

With "the script" you mean wizard.sh, right?

Note that wizard.sh is not called if local_src/<name> already exist and
the user answers "y" when asked if the files should be overwritten [1].
The .in file will contain @PLACEHHOLDER@s in that case.

(How) should we handle that?

Regards,
Bastian

[1]
https://git.pengutronix.de/cgit/ptxdist/tree/scripts/lib/ptxd_lib_template.sh?id=560f69b173794383a9dbfb6bc5be38a81c3e3b05#n191

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 1/5] package templates: add code-signing-provider template

[Michael Olbrich]

On Tue, Jun 16, 2020 at 03:56:48PM +0200, Bastian Krause wrote:
> 
> On 6/12/20 11:18 AM, Michael Olbrich wrote:
> > On Mon, Jun 08, 2020 at 10:53:01AM +0200, Bastian Krause wrote:
> >> A ptxdist code signing provider is a package which selects the required
> >> host tools needed for the code signing helpers to work. A shell script
> >> is needed to define roles, set PKCS#11 URIs and import keys if SoftHSM
> >> is used. In order to simplify its creation provide a template along with
> >> an example script.
> > 
> > I think we should query whether a HSM or SoftHSM will be used and install
> > an appropriate script and set the correct dependencies.
> > 
> >> Signed-off-by: Bastian Krause <bst@pengutronix.de>
> >> ---
> >>  .../code-signing-provider/ptxdist-set-keys.sh | 96 +++++++++++++++++++
> >>  .../template-code-signing-provider-choice-in  |  5 +
> >>  .../template-code-signing-provider-in         | 16 ++++
> >>  .../template-code-signing-provider-make       | 41 ++++++++
> >>  scripts/lib/ptxd_lib_template.sh              | 16 ++++
> >>  5 files changed, 174 insertions(+)
> >>  create mode 100755 rules/templates/code-signing-provider/ptxdist-set-keys.sh
> >>  create mode 100644 rules/templates/template-code-signing-provider-choice-in
> >>  create mode 100644 rules/templates/template-code-signing-provider-in
> >>  create mode 100644 rules/templates/template-code-signing-provider-make
> >>
> >> diff --git a/rules/templates/code-signing-provider/ptxdist-set-keys.sh b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
> >> new file mode 100755
> >> index 000000000..040a61534
> >> --- /dev/null
> >> +++ b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
> >> @@ -0,0 +1,96 @@
> >> +#!/bin/bash
> >> +
> >> +set -e
> >> +
> >> +set_fit_keys() {
> >> +	local r="image-kernel-fit"
> >> +	cs_define_role "${r}"
> >> +
> >> +	# HSM use case
> >> +	cs_set_uri "${r}" "pkcs11:token=foo;object=kernel-fit"
> >> +}
> >> +
> >> +import_fit_keys() {
> >> +	local fit_cert_dir=fit
> >> +	local r="image-kernel-fit"
> >> +	cs_define_role "${r}"
> >> +
> >> +	cs_import_cert_from_der "${r}" "${fit_cert_dir}/fit-4096-development.crt"
> >> +	cs_import_pubkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
> >> +	cs_import_privkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
> >> +}
> >> +
> >> +set_rauc_keys() {
> >> +	local r="update"
> >> +	cs_define_role "${r}"
> >> +	cs_set_uri "${r}" "pkcs11:token=foo;object=rauc"
> >> +	cs_append_ca_from_uri "${r}"
> >> +}
> >> +
> >> +import_rauc_keys() {
> >> +	local rauc_cert_dir=rauc
> >> +	local r="update"
> >> +	cs_define_role "${r}"
> >> +
> >> +	# SoftHSM use case
> >> +	cs_import_cert_from_pem "${r}" "${rauc_cert_dir}/rauc.cert.pem"
> >> +	cs_import_pubkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
> >> +	cs_import_privkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
> >> +
> >> +	cs_append_ca_from_uri "${r}"
> >> +}
> >> +
> >> +set_imx_habv4_keys() {
> >> +	# HSM use case, assuming it contains only 1st CSF/IMG key
> >> +	for i in 1 2 3 4; do
> >> +		r="imx-habv4-srk${i}"
> >> +		cs_define_role "${r}"
> >> +		cs_set_uri "${r}" "pkcs11:token=foo;object=srk-release${i}"
> >> +		cs_append_ca_from_uri "${r}"
> >> +	done
> >> +
> >> +	r="imx-habv4-csf1"
> >> +	cs_define_role ${r}
> >> +	cs_set_uri "${r}" "pkcs11:token=foo;object=csf1"
> >> +
> >> +	r="imx-habv4-img1"
> >> +	cs_define_role ${r}
> >> +	cs_set_uri "${r}" "pkcs11:token=foo;object=img1"
> >> +}
> >> +
> >> +import_imx_habv4_keys() {
> >> +	local imx_habv4_key_dir="habv4"
> >> +	local crts="${imx_habv4_key_dir}/crts"
> >> +	local keys="${imx_habv4_key_dir}/keys"
> >> +	local OPENSSL_KEYPASS="${imx_habv4_key_dir}/keys/key_pass.txt"
> >> +
> >> +	for i in 1 2 3 4; do
> >> +		r="imx-habv4-srk${i}"
> >> +		cs_define_role "${r}"
> >> +		cs_import_cert_from_der "${r}" "${crts}/SRK${i}_sha256_4096_65537_v3_ca_crt.der"
> >> +		cs_import_key_from_pem "${r}" "${keys}/SRK${i}_sha256_4096_65537_v3_ca_key.pem"
> >> +		cs_append_ca_from_uri "${r}"
> >> +
> >> +		r="imx-habv4-csf${i}"
> >> +		cs_define_role "${r}"
> >> +		cs_import_cert_from_der "${r}" "${crts}/CSF${i}_1_sha256_4096_65537_v3_usr_crt.der"
> >> +		cs_import_key_from_pem "${r}" "${keys}/CSF${i}_1_sha256_4096_65537_v3_usr_key.pem"
> >> +
> >> +		r="imx-habv4-img${i}"
> >> +		cs_define_role "${r}"
> >> +		cs_import_cert_from_der "${r}" "${crts}/IMG${i}_1_sha256_4096_65537_v3_usr_crt.der"
> >> +		cs_import_key_from_pem "${r}" "${keys}/IMG${i}_1_sha256_4096_65537_v3_usr_key.pem"
> >> +	done
> >> +}
> >> +
> >> +
> >> +# HSM use case
> >> +#set_fit_keys
> >> +#set_rauc_keys
> >> +#set_imx_habv4_keys
> >> +
> >> +# or: SoftHSM use case
> >> +#cs_init_softhsm
> >> +#import_fit_keys
> >> +#import_rauc_keys
> >> +#import_imx_habv4_keys
> > 
> > Split this into two scripts that work for the correct use-case.
> > And use the wizard.sh to delete one and rename the other.
> > 
> >> diff --git a/rules/templates/template-code-signing-provider-choice-in b/rules/templates/template-code-signing-provider-choice-in
> >> new file mode 100644
> >> index 000000000..e2108f870
> >> --- /dev/null
> >> +++ b/rules/templates/template-code-signing-provider-choice-in
> >> @@ -0,0 +1,5 @@
> >> +## SECTION=code_signing_provider
> >> +
> >> +config CODE_SIGNING_PROVIDER_@PACKAGE@
> >> +	bool
> >> +	prompt "@package@"
> >> diff --git a/rules/templates/template-code-signing-provider-in b/rules/templates/template-code-signing-provider-in
> >> new file mode 100644
> >> index 000000000..a0c61e6ef
> >> --- /dev/null
> >> +++ b/rules/templates/template-code-signing-provider-in
> >> @@ -0,0 +1,16 @@
> >> +## SECTION=code_signing
> >> +
> >> +config CODE_SIGNING
> >> +	select HOST_@PACKAGE@_CODE_SIGNING if CODE_SIGNING_PROVIDER_@PACKAGE@
> >> +
> >> +config CODE_SIGNING_PROVIDER
> >> +	default "@package@" if CODE_SIGNING_PROVIDER_@PACKAGE@
> >> +
> >> +config HOST_@PACKAGE@_CODE_SIGNING
> >> +	bool
> >> +	select HOST_OPENSC
> >> +	select HOST_LIBP11
> >> +	select HOST_OPENSSL
> >> +	#select HOST_SOFTHSM
> >> +	#select HOST_OPENSC_PCSC
> >> +	#select HOST_EXTRACT_CERT
> > 
> > We can substitute multi-line values here. So just
> > 
> > @DEPENDENCIES@
> > 
> > and set that to the correct full list of dependencies in the script.
> 
> With "the script" you mean wizard.sh, right?
> 
> Note that wizard.sh is not called if local_src/<name> already exist and
> the user answers "y" when asked if the files should be overwritten [1].
> The .in file will contain @PLACEHHOLDER@s in that case.

Not the wizard.sh. Any variable that you export in
ptxd_template_new_code_signing_provider() will be substituted when
ptxd_template_write_platform_rules() creates the .in file.

Michael

> (How) should we handle that?
> 
> Regards,
> Bastian
> 
> [1]
> https://git.pengutronix.de/cgit/ptxdist/tree/scripts/lib/ptxd_lib_template.sh?id=560f69b173794383a9dbfb6bc5be38a81c3e3b05#n191
> 
> -- 
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] [PATCH 1/5] package templates: add code-signing-provider template

[Bastian Krause]

On 6/12/20 1:05 PM, Bastian Krause wrote:
> 
> On 6/12/20 12:05 PM, Michael Olbrich wrote:
>> On Fri, Jun 12, 2020 at 11:54:39AM +0200, Michael Olbrich wrote:
>>> On Fri, Jun 12, 2020 at 11:18:25AM +0200, Michael Olbrich wrote:
>>>> On Mon, Jun 08, 2020 at 10:53:01AM +0200, Bastian Krause wrote:
>>>>> A ptxdist code signing provider is a package which selects the required
>>>>> host tools needed for the code signing helpers to work. A shell script
>>>>> is needed to define roles, set PKCS#11 URIs and import keys if SoftHSM
>>>>> is used. In order to simplify its creation provide a template along with
>>>>> an example script.
>>>>
>>>> I think we should query whether a HSM or SoftHSM will be used and install
>>>> an appropriate script and set the correct dependencies.
>>>>
>>>>> Signed-off-by: Bastian Krause <bst@pengutronix.de>
>>>>> ---
>>>>>  .../code-signing-provider/ptxdist-set-keys.sh | 96 +++++++++++++++++++
>>>>>  .../template-code-signing-provider-choice-in  |  5 +
>>>>>  .../template-code-signing-provider-in         | 16 ++++
>>>>>  .../template-code-signing-provider-make       | 41 ++++++++
>>>>>  scripts/lib/ptxd_lib_template.sh              | 16 ++++
>>>>>  5 files changed, 174 insertions(+)
>>>>>  create mode 100755 rules/templates/code-signing-provider/ptxdist-set-keys.sh
>>>>>  create mode 100644 rules/templates/template-code-signing-provider-choice-in
>>>>>  create mode 100644 rules/templates/template-code-signing-provider-in
>>>>>  create mode 100644 rules/templates/template-code-signing-provider-make
>>>>>
>>>>> diff --git a/rules/templates/code-signing-provider/ptxdist-set-keys.sh b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
>>>>> new file mode 100755
>>>>> index 000000000..040a61534
>>>>> --- /dev/null
>>>>> +++ b/rules/templates/code-signing-provider/ptxdist-set-keys.sh
>>>>> @@ -0,0 +1,96 @@
>>>>> +#!/bin/bash
>>>>> +
>>>>> +set -e
>>>>> +
>>>>> +set_fit_keys() {
>>>>> +	local r="image-kernel-fit"
>>>>> +	cs_define_role "${r}"
>>>>> +
>>>>> +	# HSM use case
>>>>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=kernel-fit"
>>>>> +}
>>>>> +
>>>>> +import_fit_keys() {
>>>>> +	local fit_cert_dir=fit
>>>>> +	local r="image-kernel-fit"
>>>>> +	cs_define_role "${r}"
>>>>> +
>>>>> +	cs_import_cert_from_der "${r}" "${fit_cert_dir}/fit-4096-development.crt"
>>>>> +	cs_import_pubkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
>>>>> +	cs_import_privkey_from_pem "${r}" "${fit_cert_dir}/fit-4096-development.key"
>>>>> +}
>>>>> +
>>>>> +set_rauc_keys() {
>>>>> +	local r="update"
>>>>> +	cs_define_role "${r}"
>>>>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=rauc"
>>>>> +	cs_append_ca_from_uri "${r}"
>>>>> +}
>>>>> +
>>>>> +import_rauc_keys() {
>>>>> +	local rauc_cert_dir=rauc
>>>>> +	local r="update"
>>>>> +	cs_define_role "${r}"
>>>>> +
>>>>> +	# SoftHSM use case
>>>>> +	cs_import_cert_from_pem "${r}" "${rauc_cert_dir}/rauc.cert.pem"
>>>>> +	cs_import_pubkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
>>>>> +	cs_import_privkey_from_pem "${r}" "${rauc_cert_dir}/rauc.key.pem"
>>>>> +
>>>>> +	cs_append_ca_from_uri "${r}"
>>>>> +}
>>>>> +
>>>>> +set_imx_habv4_keys() {
>>>>> +	# HSM use case, assuming it contains only 1st CSF/IMG key
>>>>> +	for i in 1 2 3 4; do
>>>>> +		r="imx-habv4-srk${i}"
>>>>> +		cs_define_role "${r}"
>>>>> +		cs_set_uri "${r}" "pkcs11:token=foo;object=srk-release${i}"
>>>>> +		cs_append_ca_from_uri "${r}"
>>>>> +	done
>>>>> +
>>>>> +	r="imx-habv4-csf1"
>>>>> +	cs_define_role ${r}
>>>>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=csf1"
>>>>> +
>>>>> +	r="imx-habv4-img1"
>>>>> +	cs_define_role ${r}
>>>>> +	cs_set_uri "${r}" "pkcs11:token=foo;object=img1"
>>>>> +}
>>>>> +
>>>>> +import_imx_habv4_keys() {
>>>>> +	local imx_habv4_key_dir="habv4"
>>>>> +	local crts="${imx_habv4_key_dir}/crts"
>>>>> +	local keys="${imx_habv4_key_dir}/keys"
>>>>> +	local OPENSSL_KEYPASS="${imx_habv4_key_dir}/keys/key_pass.txt"
>>>>> +
>>>>> +	for i in 1 2 3 4; do
>>>>> +		r="imx-habv4-srk${i}"
>>>>> +		cs_define_role "${r}"
>>>>> +		cs_import_cert_from_der "${r}" "${crts}/SRK${i}_sha256_4096_65537_v3_ca_crt.der"
>>>>> +		cs_import_key_from_pem "${r}" "${keys}/SRK${i}_sha256_4096_65537_v3_ca_key.pem"
>>>>> +		cs_append_ca_from_uri "${r}"
>>>>> +
>>>>> +		r="imx-habv4-csf${i}"
>>>>> +		cs_define_role "${r}"
>>>>> +		cs_import_cert_from_der "${r}" "${crts}/CSF${i}_1_sha256_4096_65537_v3_usr_crt.der"
>>>>> +		cs_import_key_from_pem "${r}" "${keys}/CSF${i}_1_sha256_4096_65537_v3_usr_key.pem"
>>>>> +
>>>>> +		r="imx-habv4-img${i}"
>>>>> +		cs_define_role "${r}"
>>>>> +		cs_import_cert_from_der "${r}" "${crts}/IMG${i}_1_sha256_4096_65537_v3_usr_crt.der"
>>>>> +		cs_import_key_from_pem "${r}" "${keys}/IMG${i}_1_sha256_4096_65537_v3_usr_key.pem"
>>>>> +	done
>>>>> +}
>>>>> +
>>>>> +
>>>>> +# HSM use case
>>>>> +#set_fit_keys
>>>>> +#set_rauc_keys
>>>>> +#set_imx_habv4_keys
>>>>> +
>>>>> +# or: SoftHSM use case
>>>>> +#cs_init_softhsm
>>>>> +#import_fit_keys
>>>>> +#import_rauc_keys
>>>>> +#import_imx_habv4_keys
>>>>
>>>> Split this into two scripts that work for the correct use-case.
>>>> And use the wizard.sh to delete one and rename the other.
>>>>
>>>>> diff --git a/rules/templates/template-code-signing-provider-choice-in b/rules/templates/template-code-signing-provider-choice-in
>>>>> new file mode 100644
>>>>> index 000000000..e2108f870
>>>>> --- /dev/null
>>>>> +++ b/rules/templates/template-code-signing-provider-choice-in
>>>>> @@ -0,0 +1,5 @@
>>>>> +## SECTION=code_signing_provider
>>>>> +
>>>>> +config CODE_SIGNING_PROVIDER_@PACKAGE@
>>>>> +	bool
>>>>> +	prompt "@package@"
>>>>> diff --git a/rules/templates/template-code-signing-provider-in b/rules/templates/template-code-signing-provider-in
>>>>> new file mode 100644
>>>>> index 000000000..a0c61e6ef
>>>>> --- /dev/null
>>>>> +++ b/rules/templates/template-code-signing-provider-in
>>>>> @@ -0,0 +1,16 @@
>>>>> +## SECTION=code_signing
>>>>> +
>>>>> +config CODE_SIGNING
>>>>> +	select HOST_@PACKAGE@_CODE_SIGNING if CODE_SIGNING_PROVIDER_@PACKAGE@
>>>>> +
>>>>> +config CODE_SIGNING_PROVIDER
>>>>> +	default "@package@" if CODE_SIGNING_PROVIDER_@PACKAGE@
>>>>> +
>>>>> +config HOST_@PACKAGE@_CODE_SIGNING
>>>>> +	bool
>>>>> +	select HOST_OPENSC
>>>>> +	select HOST_LIBP11
>>>>> +	select HOST_OPENSSL
>>>>> +	#select HOST_SOFTHSM
>>>>> +	#select HOST_OPENSC_PCSC
>>>>> +	#select HOST_EXTRACT_CERT
>>>>
>>>> We can substitute multi-line values here. So just
>>>>
>>>> @DEPENDENCIES@
>>>>
>>>> and set that to the correct full list of dependencies in the script.
>>>>
>>>>
>>>>> diff --git a/rules/templates/template-code-signing-provider-make b/rules/templates/template-code-signing-provider-make
>>>>> new file mode 100644
>>>>> index 000000000..94830d92e
>>>>> --- /dev/null
>>>>> +++ b/rules/templates/template-code-signing-provider-make
>>>>> @@ -0,0 +1,41 @@
>>>>> +# -*-makefile-*-
>>>>> +#
>>>>> +# Copyright (C) @YEAR@ by @AUTHOR@
>>>>> +#
>>>>> +# For further information about the PTXdist project and license conditions
>>>>> +# see the README file.
>>>>> +#
>>>>> +
>>>>> +#
>>>>> +# We provide this package
>>>>> +#
>>>>> +HOST_PACKAGES-$(PTXCONF_HOST_@PACKAGE@_CODE_SIGNING) += host-@package@-code-signing
>>>>> +
>>>>> +#
>>>>> +# Paths and names
>>>>> +#
>>>>> +HOST_@PACKAGE@_CODE_SIGNING_VERSION	:= @VERSION@
>>>>> +HOST_@PACKAGE@_CODE_SIGNING		:= @package@-code-signing-$(HOST_@PACKAGE@_CODE_SIGNING_VERSION)
>>>>> +HOST_@PACKAGE@_CODE_SIGNING_URL		:= file://local_src/@package@-code-signing
>>>>> +HOST_@PACKAGE@_CODE_SIGNING_DIR		:= $(HOST_BUILDDIR)/$(HOST_@PACKAGE@_CODE_SIGNING)
>>>>> +
>>>>> +HOST_@PACKAGE@_CODE_SIGNING_CONF_TOOL	:= NO
>>>>> +
>>>>> +# ----------------------------------------------------------------------------
>>>>> +# Compile
>>>>> +# ----------------------------------------------------------------------------
>>>>> +
>>>>> +HOST_@PACKAGE@_CODE_SIGNING_MAKE_ENV	:= \
>>>>> +	$(CODE_SIGNING_ENV)
>>>>> +
>>>>> +$(STATEDIR)/host-@package@-code-signing.compile:
>>>>> +	@$(call targetinfo)
>>>>> +	@$(call world/execute, HOST_@PACKAGE@_CODE_SIGNING, \
>>>>> +		./ptxdist-set-keys.sh)
>>>>> +	@$(call touch)
>>>>> +
>>>>> +$(STATEDIR)/host-@package@-code-signing.install:
>>>>> +	@$(call targetinfo)
>>>>> +	@$(call touch)
>>>>> +
>>>>> +# vim: syntax=make
>>>>> diff --git a/scripts/lib/ptxd_lib_template.sh b/scripts/lib/ptxd_lib_template.sh
>>>>> index f39e6e033..b89981f45 100644
>>>>> --- a/scripts/lib/ptxd_lib_template.sh
>>>>> +++ b/scripts/lib/ptxd_lib_template.sh
>>>>> @@ -460,3 +460,19 @@ ptxd_template_new_blspec_entry() {
>>>>>  export -f ptxd_template_new_blspec_entry
>>>>>  ptxd_template_help_list[${#ptxd_template_help_list[@]}]="blspec-entry"
>>>>>  ptxd_template_help_list[${#ptxd_template_help_list[@]}]="create package for a bootloader spec entry"
>>>>> +
>>>>> +ptxd_template_new_code_signing_provider() {
>>>>> +    export class="host-"
>>>>> +    ptxd_template_read_basic &&
>>>>> +    ptxd_template_read_author &&
>>>>
>>>> The question for the type should be here. Maybe provide an list and the
>>>> user must input the index number or something like that.
>>>
>>> So, after reading the docs, I think there should be 3 options here:
>>>
>>> 1) SoftHSM
>>> 2) HSM (with OpenSC)
>>> 3) HSM (custom)
>>>
>>> And for the HSM cases, the template should also provide the rules/pre/...
>>> file for CODE_SIGNING_ENV. Maybe with the module name as a variable?
>>> Substitute 'opensc-pkcs11' for OpenSC and 'fixme' otherwise.
>>
>> Or ask for the module name? I don't have a good understanding what is
>> usually necessary for other HSMs beyond the PKCS11_MODULE_PATH, so I don't
>> know if that makes sense.
> 
> I've disucssed this with Marc some time ago. We came up with the idea of
> setting the HSM specifics in CODE_SIGNING_ENV via another code signing
> provider helper, e.g. "cs_append_env". In our opinion the code signing
> provider should know (and set) such extra environment variables.

Note: this implementation idea was not addressed in v3 of the patch
series. v3 now uses generated pre make files to extend CODE_SIGNING_ENV
for the selected HSM use case. It would still be nice to have something
like "cs_append_env" as a substitute for the pre rules in the future.
Leaving it open to someone else :)

Regards,
Bastian

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de