Re: [ptxdist] [PATCH 2/2] doc: working with licensing information in packages

[Roland Hieber <rhi@pengutronix.de>]

Any comments here?

 - Roland

On Mon, May 11, 2020 at 12:03:06PM +0200, Roland Hieber wrote:
> Co-authored-by: Felicitas Jung <f.jung@pengutronix.de>
> Signed-off-by: Felicitas Jung <f.jung@pengutronix.de>
> Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> ---
>  doc/contributing.rst        |   5 +
>  doc/daily_work.inc          |   2 +
>  doc/daily_work_licenses.inc | 208 ++++++++++++++++++++++++++++++++++++
>  doc/ref_make_variables.inc  |   4 +
>  4 files changed, 219 insertions(+)
>  create mode 100644 doc/daily_work_licenses.inc
> 
> diff --git a/doc/contributing.rst b/doc/contributing.rst
> index 705f01377d32..7352b46dfcf0 100644
> --- a/doc/contributing.rst
> +++ b/doc/contributing.rst
> @@ -90,6 +90,11 @@ For new packages, the generated templates contain commented-out default
>  sections. These are meant as a helper to simplify creating custom stages.
>  Any remaining default stages must be removed.
>  
> +New packages should also have licensing information in the ``<PKG>_LICENSE``
> +and ``<PKG>_LICENSE_FILES`` variables.
> +Refer to the section :ref:`licensing_in_packages` for more information.
> +
> +
>  Helper Scripts
>  --------------
>  
> diff --git a/doc/daily_work.inc b/doc/daily_work.inc
> index a37aac4c3339..f68d25bf7cb5 100644
> --- a/doc/daily_work.inc
> +++ b/doc/daily_work.inc
> @@ -1472,3 +1472,5 @@ be enabled. A used mount option of the overlayfs in the default
>  newer.
>  If your kernel does not meet this requirement you can provide your own local
>  and adapted variant of the mentioned mount unit.
> +
> +.. include:: daily_work_licenses.inc
> diff --git a/doc/daily_work_licenses.inc b/doc/daily_work_licenses.inc
> new file mode 100644
> index 000000000000..7e90b7ba541d
> --- /dev/null
> +++ b/doc/daily_work_licenses.inc
> @@ -0,0 +1,208 @@
> +.. _licensing_in_packages:
> +
> +Tracking licensing information in packages
> +------------------------------------------
> +
> +PTXdist aims to track licensing information for every package.
> +This includes the license(s) under which a package can be distributed,
> +as well as the respective files in the package's source tree that state those terms.
> +Sadly there is no widely adopted standard for machine-readable licensing
> +information in source code (`yet <https://reuse.software>`_),
> +so here are a few hints where to look.
> +
> +There are many older package rules in PTXdist which don't specify licensing information.
> +If you want to help complete the database,
> +you can use ``grep -L _LICENSE_FILES rules/*.make`` (in the PTXdist tree) to find those rules.
> +Note however that this cannot find wrong or incomplete licensing information.
> +
> +Finding licensing information
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +You should first select and extract the package in question, and then have a
> +look at in the extracted package sources (usually something like
> +``platform-nnn/build-target/mypackage-1.0`` in your BSP, if in doubt see
> +``ptxdist package-info mypackage``).
> +
> +* Check for files named ``COPYING``, ``COPYRIGHT``,  or ``LICENSE``.
> +  These often only contain the license text and, in case of GPL, no information
> +  if the code is available under the *-only* or *-or-later* variant.
> +  Sometimes these files are in a folder ``/doc`` or ``/legal``.
> +
> +* Check the ``README``, if there is any.
> +  Often there is important information there, e.g. in case of GPL if the
> +  software is *GPL-x.x-or-later* or *GPL-x.x-only*.
> +
> +* Check some relevant-sounding files, like ``main.c`` for license headers.
> +  Often additional information can be found here.
> +
> +* If you want to be extra sure, use a license compliance toolchain (e.g.
> +  `FOSSology <https://www.fossology.org/>`__) on the project.
> +
> +On the other hand, there are some things that can be ignored for our purposes:
> +
> +* Everything that is auto-generated, either by a script in the project source,
> +  or by the build system previous to packaging.
> +  The generator itself cannot hold copyright, although the authors of the
> +  templates used for the generation or the authors of the generator can.
> +
> +* Most files belonging to the build system don't make it into the compiled code
> +  and can therefore be ignored (e.g. configure scripts, Makefiles).
> +  These cases sometimes can be hard to detect – if unsure, include the file in
> +  your research.
> +
> +Distillation down to license identifiers
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +We use the `SPDX license identifiers <https://spdx.org/licenses/>`_.
> +
> +Either the license is clear, e.g. because it says "GPL 2.0" (roughly check the
> +license content to be sure), or you can use tools like
> +`FOSSology <https://www.fossology.org>`__,
> +`licensecheck <https://wiki.debian.org/CopyrightReviewTools#Command-line_tools_in_Debian>`_,
> +or `spdx-license-match <https://github.com/rohieb/spdx-license-match>`_
> +to detect license material in the project.
> +
> +License texts don't have to match exactly, you should apply the
> +`SPDX Matching Guidelines <https://spdx.org/spdx-license-list/matching-guidelines>`_
> +accordingly.
> +The important part here is that the project's license and the SPDX identifier
> +describe the same licensing terms.
> +"Rather close" or "mostly similar" statements are not enough for a match,
> +but simple unimportant changes like replacing *"The Author"* with the project's
> +maintainer's name, or a change in e-mail adresses, are usually okay.
> +
> +For software that is not open-source according to the `OSI definition
> +<https://opensource.org/osd>`_, use the identifier ``proprietary``.
> +
> +If no license identifier matches, use ``unknown``.
> +If the project is considered open source or free software, you can
> +`report its license to be added to the SPDX license list
> +<https://github.com/spdx/license-list-XML/blob/master/CONTRIBUTING.md#request-a-new-license-or-exception-be-added-to-the-spdx-license-list>`_.
> +
> +Conflicting statements
> +^^^^^^^^^^^^^^^^^^^^^^
> +
> +Human interpretation is needed when statements inside the project conflict with
> +each other.
> +Some clues that can help you decide:
> +
> +Detailedness:
> +  If the header in COPYING or the README says *"GNU General Public License"*,
> +  but the license text is in fact a BSD license, the correct license is the BSD
> +  license.
> +
> +Author Intent:
> +  If the README says *"this is LGPL 2.1"*, but COPYING contains a GPL boilerplate
> +  license text, the correct licensing information is probably *"LGPL 2.1"* –
> +  the README written by the author prevails over the boilerplate text.
> +
> +Recency:
> +  If README and COPYING are both clearly written by the author themselves, and
> +  the README says *"don't do $thing*" and COPYING says *"do $thing*", the more
> +  recent file prevails.
> +
> +  .. note::
> +
> +   Any of such cases is considered a bug and should be reported to the upstream maintainer!
> +
> +License versions, and GPL-vv-only or GPL-vv-or-later?
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +If the ``COPYING`` file is a GPL text, it is still uncertain if the correct
> +license identifier is *GPL-vv-only* or *GPL-vv-or-later*.
> +The GPL text itself does not give information on that in its terms and
> +conditions.
> +Sometimes there is a notice at the top of the COPYING or the README file stating
> +whether *"-only"* or *"-or-later"* applies – this is the easy case.
> +Otherwise: check headers in relevant files.
> +
> +If no license information can be found, but one file mentions e.g. *"GPL-vv or
> +later"*, use that information for the whole project.
> +E.g.: no license information can be found except a ``COPYING`` which contains
> +a GPL-2.0 text → the license is GPL-2.0-only.
> +
> +Sometimes the best information available is statements like
> +*"this code is under GPL"* without any version information.
> +Such cases should be interpreted as the most liberal reading,
> +i.e. *GPL-1.0-or-later* (any possible GPL version).
> +
> +If multiple versions and variants can be found in the project, combine them with
> +``AND``, e.g.: ``GPL-2.0-only AND GPL-2.0-or-later`` in the license identifier.
> +
> +Public domain software
> +^^^^^^^^^^^^^^^^^^^^^^
> +
> +For `good reasons <https://wiki.spdx.org/view/Legal_Team/Decisions/Dealing_with_Public_Domain_within_SPDX_Files>`_,
> +SPDX doesn't supply a license identifier for "Public Domain".
> +Nevertheless, some PTXdist package rules specify ``public_domain`` as their
> +respective license identifier.
> +When this is done, it is purely for historical reasons, and ``public_domain``
> +should normally not be used for new packages.
> +Some of those "Public Domain" dedications in packages have since been accepted
> +in SPDX, e.g. `libselinux <https://spdx.org/licenses/libselinux-1.0.html>`_ or
> +`SQLite <https://spdx.org/licenses/blessing.html>`_.
> +
> +No license information at all
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +No license - no usage rights!
> +
> +Definitely report this bug to the upstream maintainer.
> +Maybe even point them in the direction of `machine-readablity <https://reuse.software/>`_ :)
> +
> +Adding license files to PTXdist package rules
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The SPDX license identifier of the package goes into the ``<PKG>_LICENSE``
> +variable in the respective package rule file.
> +All relevant files identified in the steps above are then added to the variable ``<PKG>_LICENSE``,
> +including a checksum so that PTXdist complains when they change.
> +
> +Example:
> +
> +.. code-block:: make
> +   :caption: ddrescue.make
> +
> +   DDRESCUE_LICENSE	:= GPL-2.0-or-later AND BSD-2-Clause
> +   DDRESCUE_LICENSE_FILES	:= \
> +           file://COPYING;md5=76d6e300ffd8fb9d18bd9b136a9bba13 \
> +           file://main.cc;startline=1;endline=16;md5=a01d61d3293ce28b883d8ba0c497e968 \
> +           file://arg_parser.cc;startline=1;endline=18;md5=41d1341d0d733a5d24b26dc3cbc1ac42
> +
> +See the section :ref:`package_specific_variables` for more information about
> +the syntax of those two variables.
> +
> +The MD5 sum for a block of lines can be generated with sed's ``p`` (print)
> +command applied to a range of lines.
> +For the example above, lines 1 to 16 of main.cc would be:
> +
> +.. code-block:: terminal
> +
> +   $ sed -n 1,16p main.cc | md5sum -
> +   a01d61d3293ce28b883d8ba0c497e968
> +
> +If the copyright statement contains a string of years, leave those lines out for
> +the calculation of the checksum, as an added year does not change the license
> +(in fact, not even a single year is needed for the license to be valid),
> +but only makes package version updates more cumbersome.
> +
> +If additional information is in the ``README`` or license headers in source
> +files are used, also include these files (for source code: one of each is enough),
> +but use md5sum only on the relevant lines, so changes in the rest of the file do
> +not appear as license changes.
> +
> +For rather chaotic directories with lots of license files, definetly include at
> +least one relevant source file with license headers (if there are any), as some
> +developers tend to accumulate license files without adjusting it to license
> +changes in their source.
> +
> +As in the example above, sometimes more than one license applies.
> +If different files in the package are under different licenses, use ``AND`` (e.g.
> +``GPL-2.0-only AND LGPL-2.1``).
> +If it leaves the choice to modify/redistribute under one or the other
> +license, use ``OR``.
> +
> +.. note::
> +
> +   For each single license in the compound statement, include at least one file
> +   with checksum in the ``<PKG>_LICENSE_FILES`` variable.
> diff --git a/doc/ref_make_variables.inc b/doc/ref_make_variables.inc
> index 56912bb2e364..701c029591d8 100644
> --- a/doc/ref_make_variables.inc
> +++ b/doc/ref_make_variables.inc
> @@ -127,6 +127,8 @@ Other useful variables:
>    that are built and installed during the PTXdist build run.
>    There are analogous ``-y`` and ``-m`` variants of those variables too.
>  
> +.. _package_specific_variables:
> +
>  Package Specific Variables
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~
>  
> @@ -228,6 +230,7 @@ Package Definition
>    here. Use ``proprietary`` for proprietary packages and ``ignore`` for
>    packages without their own license, e.g. meta packages or packages that
>    only install files from ``projectroot/``.
> +  See the section :ref:`licensing_in_packages` for more information.
>  
>  ``<PKG>_LICENSE_FILES``
>    A space separated list of URLs of license text files. The URLs must be
> @@ -239,6 +242,7 @@ Package Definition
>    used in case the specified file contains more than just the license text,
>    e.g. if the license is in the header of a source file. For non ASCII or
>    UTF-8 files the encoding can be specified with ``encoding=<enc>``.
> +  See the section :ref:`licensing_in_packages` for more information.
>  
>  For most packages the variables described above are undefined by default.
>  However, for cross and host packages these variables default to the value
> -- 
> 2.20.1
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de

-- 
Roland Hieber, Pengutronix e.K.          | r.hieber@pengutronix.de     |
Steuerwalder Str. 21                     | https://www.pengutronix.de/ |
31137 Hildesheim, Germany                | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686         | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de