Re: [ptxdist] license-report: extend to generate textual list of package + licenses

[Michael Olbrich <m.olbrich@pengutronix.de>]

On Tue, Oct 04, 2016 at 07:15:07PM +0200, Andreas Pretzsch wrote:
> On Di, 2016-10-04 at 15:02 +0200, Michael Olbrich wrote:
> > > > [ptxdist make license-report]
> > > 
> > > First, this one is wonderful, and works like a charm. Thanks a lot.
> > > 
> > > What I am missing is an overview.
> > > Mainly as an external list, but maybe also in the reports.
> > > 
> > > I'm talking about e.g. a simple CSV, listing all the included packages,
> > > like in "pkgname | pkgversion | licenses". So something like
> > > [...]
> > > busybox|1.24.2|GPL-2.0
> > > [...]
> > > Sensible escaping would be something to look at, but anyway.
> > > [...]
> > > 
> > > Has anyone already (partly) implemented this ?
> > > Would be happy to pick up the pieces and also mainline it. Just asking,
> > > before I start hacking things twice...
> > 
> > I don't think there are any implementations like this.
> > 
> > If we do something like this, then I'd like to take this a step further. I
> > think there are more use-cases for data like this. And there a probably
> > conflicting requirements for the different files and output formats.
> 
> For sure. Both.
> Just, I'd like to start with a simple one, to have something. Or to be
> more precise, I have to, due to customer requirement. So in any case,
> I'll hack something the next few days. In case I'll hook it into the
> ptxdist scripts, I'll CC the list, as a RFC.
> Of course, I'm happy about any pointers where to put my fingers on in
> the scripts ;-)

You will probably need to overwrite ptxd_make_world_license() and expand it
to generate the needed per package data.
Then you can use ptxd_make_license_report() as a template to generate the
final file.

> Out of my experience with these topics (license compliance, due
> diligence by laywer), the todays report is wonderful !
> And sufficient here. Maybe it could be condensed in some points (like
> referencing fixed texts as the GPL instead of copying each time), but
> even this is not necessary. And/or converted to some other format (text,
> html), to be included in a product delivery itself (menu item, web
> interface, whatever).
> 
> But to ease everyones task, and esp. have both developers and lawyers
> easily check what changed, we need a simple list. In whatever format,
> and included in the report or not.

The current latex/pdf result was always just a first step for me.

> > The main issue with PTXdist is that we cannot just iterate over all
> > packages is one script. So what I'd like to see is something like the
> > current report stage to aggregate all possibly interesting data about the
> > packages and later convert an filter that into the required output.
> > What would be a good intermediate format for this?
> 
> Sounds like the best way to do it. About an intermediate format, hmmm,
> good question. Not sure if there is one per se. Or from an pragmatic
> point of view, we already have one, the information in the report
> generation scripts.

I meant a file format that's easy to parse later.

One of the problems here is, that the data structures get rather complex
and are not something that can really be handled in shell scripts.

> >  I'd like to avoid escaping problems if possible.
> 
> Me too :-)
> CSV is always fun...
> 
> Looking at the minimal information (package name, version, license tags,
> maybe flags), I think we will not run into encoding and escaping issues
> here. As long as we leave out the URL. And stick with western package
> names.
> Just take everything as a string, and use | as separator. Should be good
> for now. At least, this would be my first try.
> And even if the format changes some day, this could be converted
> externally with minimal effort.

Two points here:
1. just CSV or similar makes it hard to add or remove fields.
2. Depending on what you use to parse the files, 0x1F (ascii unit
separator) or something like that, might be a good alternative. We use this
in the permission files to avoid characters that might appear in file
names.

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de