Re: [ptxdist] [PATCH 2/2] doc: working with licensing information in packages

[Alexander Dahl <ada@thorsis.com>]

Hei hei,

see below …

Am Montag, 11. Mai 2020, 12:03:06 CEST schrieb Roland Hieber:
> Co-authored-by: Felicitas Jung <f.jung@pengutronix.de>
> Signed-off-by: Felicitas Jung <f.jung@pengutronix.de>
> Signed-off-by: Roland Hieber <rhi@pengutronix.de>
> ---
>  doc/contributing.rst        |   5 +
>  doc/daily_work.inc          |   2 +
>  doc/daily_work_licenses.inc | 208 ++++++++++++++++++++++++++++++++++++
>  doc/ref_make_variables.inc  |   4 +
>  4 files changed, 219 insertions(+)
>  create mode 100644 doc/daily_work_licenses.inc
> 
> diff --git a/doc/contributing.rst b/doc/contributing.rst
> index 705f01377d32..7352b46dfcf0 100644
> --- a/doc/contributing.rst
> +++ b/doc/contributing.rst
> @@ -90,6 +90,11 @@ For new packages, the generated templates contain
> commented-out default sections. These are meant as a helper to simplify
> creating custom stages. Any remaining default stages must be removed.
> 
> +New packages should also have licensing information in the
> ``<PKG>_LICENSE`` +and ``<PKG>_LICENSE_FILES`` variables.
> +Refer to the section :ref:`licensing_in_packages` for more information.
> +
> +
>  Helper Scripts
>  --------------
> 
> diff --git a/doc/daily_work.inc b/doc/daily_work.inc
> index a37aac4c3339..f68d25bf7cb5 100644
> --- a/doc/daily_work.inc
> +++ b/doc/daily_work.inc
> @@ -1472,3 +1472,5 @@ be enabled. A used mount option of the overlayfs in
> the default newer.
>  If your kernel does not meet this requirement you can provide your own
> local and adapted variant of the mentioned mount unit.
> +
> +.. include:: daily_work_licenses.inc
> diff --git a/doc/daily_work_licenses.inc b/doc/daily_work_licenses.inc
> new file mode 100644
> index 000000000000..7e90b7ba541d
> --- /dev/null
> +++ b/doc/daily_work_licenses.inc
> @@ -0,0 +1,208 @@
> +.. _licensing_in_packages:
> +
> +Tracking licensing information in packages
> +------------------------------------------
> +
> +PTXdist aims to track licensing information for every package.
> +This includes the license(s) under which a package can be distributed,
> +as well as the respective files in the package's source tree that state
> those terms. +Sadly there is no widely adopted standard for
> machine-readable licensing +information in source code (`yet
> <https://reuse.software>`_),
> +so here are a few hints where to look.
> +
> +There are many older package rules in PTXdist which don't specify licensing
> information. +If you want to help complete the database,
> +you can use ``grep -L _LICENSE_FILES rules/*.make`` (in the PTXdist tree)
> to find those rules. +Note however that this cannot find wrong or
> incomplete licensing information. +
> +Finding licensing information
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +You should first select and extract the package in question, and then have
> a +look at in the extracted package sources (usually something like
> +``platform-nnn/build-target/mypackage-1.0`` in your BSP, if in doubt see
> +``ptxdist package-info mypackage``).
> +
> +* Check for files named ``COPYING``, ``COPYRIGHT``,  or ``LICENSE``.
> +  These often only contain the license text and, in case of GPL, no
> information +  if the code is available under the *-only* or *-or-later*
> variant. +  Sometimes these files are in a folder ``/doc`` or ``/legal``.
> +
> +* Check the ``README``, if there is any.
> +  Often there is important information there, e.g. in case of GPL if the
> +  software is *GPL-x.x-or-later* or *GPL-x.x-only*.
> +
> +* Check some relevant-sounding files, like ``main.c`` for license headers.
> +  Often additional information can be found here.
> +
> +* If you want to be extra sure, use a license compliance toolchain (e.g.
> +  `FOSSology <https://www.fossology.org/>`__) on the project.
> +
> +On the other hand, there are some things that can be ignored for our
> purposes: +
> +* Everything that is auto-generated, either by a script in the project
> source, +  or by the build system previous to packaging.
> +  The generator itself cannot hold copyright, although the authors of the
> +  templates used for the generation or the authors of the generator can.
> +
> +* Most files belonging to the build system don't make it into the compiled
> code +  and can therefore be ignored (e.g. configure scripts, Makefiles). +
>  These cases sometimes can be hard to detect – if unsure, include the file
> in +  your research.
> +
> +Distillation down to license identifiers
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +We use the `SPDX license identifiers <https://spdx.org/licenses/>`_.
> +
> +Either the license is clear, e.g. because it says "GPL 2.0" (roughly check
> the +license content to be sure), or you can use tools like
> +`FOSSology <https://www.fossology.org>`__,
> +`licensecheck
> <https://wiki.debian.org/CopyrightReviewTools#Command-line_tools_in_Debian>
> `_, +or `spdx-license-match <https://github.com/rohieb/spdx-license-match>`_
> +to detect license material in the project.
> +
> +License texts don't have to match exactly, you should apply the
> +`SPDX Matching Guidelines
> <https://spdx.org/spdx-license-list/matching-guidelines>`_ +accordingly.
> +The important part here is that the project's license and the SPDX
> identifier +describe the same licensing terms.
> +"Rather close" or "mostly similar" statements are not enough for a match,
> +but simple unimportant changes like replacing *"The Author"* with the
> project's +maintainer's name, or a change in e-mail adresses, are usually
> okay. +
> +For software that is not open-source according to the `OSI definition
> +<https://opensource.org/osd>`_, use the identifier ``proprietary``.
> +
> +If no license identifier matches, use ``unknown``.
> +If the project is considered open source or free software, you can
> +`report its license to be added to the SPDX license list
> +<https://github.com/spdx/license-list-XML/blob/master/CONTRIBUTING.md#reque
> st-a-new-license-or-exception-be-added-to-the-spdx-license-list>`_. +
> +Conflicting statements
> +^^^^^^^^^^^^^^^^^^^^^^
> +
> +Human interpretation is needed when statements inside the project conflict
> with +each other.
> +Some clues that can help you decide:
> +
> +Detailedness:
> +  If the header in COPYING or the README says *"GNU General Public
> License"*, +  but the license text is in fact a BSD license, the correct
> license is the BSD +  license.
> +
> +Author Intent:
> +  If the README says *"this is LGPL 2.1"*, but COPYING contains a GPL
> boilerplate +  license text, the correct licensing information is probably
> *"LGPL 2.1"* – +  the README written by the author prevails over the
> boilerplate text. +
> +Recency:
> +  If README and COPYING are both clearly written by the author themselves,
> and +  the README says *"don't do $thing*" and COPYING says *"do $thing*",
> the more +  recent file prevails.
> +
> +  .. note::
> +
> +   Any of such cases is considered a bug and should be reported to the
> upstream maintainer! +
> +License versions, and GPL-vv-only or GPL-vv-or-later?
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +If the ``COPYING`` file is a GPL text, it is still uncertain if the correct
> +license identifier is *GPL-vv-only* or *GPL-vv-or-later*.
> +The GPL text itself does not give information on that in its terms and
> +conditions.
> +Sometimes there is a notice at the top of the COPYING or the README file
> stating +whether *"-only"* or *"-or-later"* applies – this is the easy
> case. +Otherwise: check headers in relevant files.
> +
> +If no license information can be found, but one file mentions e.g. *"GPL-vv
> or +later"*, use that information for the whole project.
> +E.g.: no license information can be found except a ``COPYING`` which
> contains +a GPL-2.0 text → the license is GPL-2.0-only.
> +
> +Sometimes the best information available is statements like
> +*"this code is under GPL"* without any version information.
> +Such cases should be interpreted as the most liberal reading,
> +i.e. *GPL-1.0-or-later* (any possible GPL version).
> +
> +If multiple versions and variants can be found in the project, combine them
> with +``AND``, e.g.: ``GPL-2.0-only AND GPL-2.0-or-later`` in the license
> identifier. +
> +Public domain software
> +^^^^^^^^^^^^^^^^^^^^^^
> +
> +For `good reasons
> <https://wiki.spdx.org/view/Legal_Team/Decisions/Dealing_with_Public_Domain
> _within_SPDX_Files>`_, +SPDX doesn't supply a license identifier for "Public
> Domain".
> +Nevertheless, some PTXdist package rules specify ``public_domain`` as their
> +respective license identifier.
> +When this is done, it is purely for historical reasons, and
> ``public_domain`` +should normally not be used for new packages.
> +Some of those "Public Domain" dedications in packages have since been
> accepted +in SPDX, e.g. `libselinux
> <https://spdx.org/licenses/libselinux-1.0.html>`_ or +`SQLite
> <https://spdx.org/licenses/blessing.html>`_.
> +
> +No license information at all
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +No license - no usage rights!
> +
> +Definitely report this bug to the upstream maintainer.
> +Maybe even point them in the direction of `machine-readablity
> <https://reuse.software/>`_ :) +
> +Adding license files to PTXdist package rules
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The SPDX license identifier of the package goes into the ``<PKG>_LICENSE``
> +variable in the respective package rule file.
> +All relevant files identified in the steps above are then added to the
> variable ``<PKG>_LICENSE``, +including a checksum so that PTXdist complains
> when they change.
> +
> +Example:
> +
> +.. code-block:: make
> +   :caption: ddrescue.make
> +
> +   DDRESCUE_LICENSE	:= GPL-2.0-or-later AND BSD-2-Clause
> +   DDRESCUE_LICENSE_FILES	:= \
> +           file://COPYING;md5=76d6e300ffd8fb9d18bd9b136a9bba13 \
> +          
> file://main.cc;startline=1;endline=16;md5=a01d61d3293ce28b883d8ba0c497e968
> \ +          
> file://arg_parser.cc;startline=1;endline=18;md5=41d1341d0d733a5d24b26dc3cbc
> 1ac42 +
> +See the section :ref:`package_specific_variables` for more information
> about +the syntax of those two variables.
> +
> +The MD5 sum for a block of lines can be generated with sed's ``p`` (print)
> +command applied to a range of lines.
> +For the example above, lines 1 to 16 of main.cc would be:
> +
> +.. code-block:: terminal
> +
> +   $ sed -n 1,16p main.cc | md5sum -
> +   a01d61d3293ce28b883d8ba0c497e968
> +
> +If the copyright statement contains a string of years, leave those lines
> out for +the calculation of the checksum, as an added year does not change
> the license +(in fact, not even a single year is needed for the license to
> be valid), +but only makes package version updates more cumbersome.
> +
> +If additional information is in the ``README`` or license headers in source
> +files are used, also include these files (for source code: one of each is
> enough), +but use md5sum only on the relevant lines, so changes in the rest
> of the file do +not appear as license changes.
> +
> +For rather chaotic directories with lots of license files, definetly
> include at +least one relevant source file with license headers (if there
> are any), as some +developers tend to accumulate license files without
> adjusting it to license +changes in their source.
> +
> +As in the example above, sometimes more than one license applies.
> +If different files in the package are under different licenses, use ``AND``
> (e.g. +``GPL-2.0-only AND LGPL-2.1``).
> +If it leaves the choice to modify/redistribute under one or the other
> +license, use ``OR``.
> +
> +.. note::
> +
> +   For each single license in the compound statement, include at least one
> file +   with checksum in the ``<PKG>_LICENSE_FILES`` variable.
> diff --git a/doc/ref_make_variables.inc b/doc/ref_make_variables.inc
> index 56912bb2e364..701c029591d8 100644
> --- a/doc/ref_make_variables.inc
> +++ b/doc/ref_make_variables.inc
> @@ -127,6 +127,8 @@ Other useful variables:
>    that are built and installed during the PTXdist build run.
>    There are analogous ``-y`` and ``-m`` variants of those variables too.
> 
> +.. _package_specific_variables:
> +
>  Package Specific Variables
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> @@ -228,6 +230,7 @@ Package Definition
>    here. Use ``proprietary`` for proprietary packages and ``ignore`` for
>    packages without their own license, e.g. meta packages or packages that
>    only install files from ``projectroot/``.
> +  See the section :ref:`licensing_in_packages` for more information.
> 
>  ``<PKG>_LICENSE_FILES``
>    A space separated list of URLs of license text files. The URLs must be
> @@ -239,6 +242,7 @@ Package Definition
>    used in case the specified file contains more than just the license text,
> e.g. if the license is in the header of a source file. For non ASCII or
> UTF-8 files the encoding can be specified with ``encoding=<enc>``. +  See
> the section :ref:`licensing_in_packages` for more information.
> 
>  For most packages the variables described above are undefined by default.
>  However, for cross and host packages these variables default to the value

I read that whole patch text, and nothing really catched my eye. Sounds quite 
complete and is a better explanation on licensing issues than most bits and 
pieces on the web. I did not check thoroughly for spelling mistakes or even 
doc build errors, so:

Acked-by: Alexander Dahl <ada@thorsis.com>

Greets
Alex




_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de