Re: [ptxdist] SBOM support

[Christian Melki <christian.melki@t2data.com>]

Shameless plug here.

We (t2data) have been developing devsecops tools for quite some time.
MAIA, our internally developed tool is capable of a lot of things.
Among them is various classifications of software with licenses, CPE,
CVE, CWE etc. Including availability of new versions etc.
Built in is also various other aspects of the development process,
beside SBOM handling.
I use MAIA to help me track my projects in ptxdist and managing change
notifications so I easier can maintain ptxdist stuff.

https://t2data.com/maia-software/

Regards,
Christian

On 9/11/23 15:11, Gavin Schenk wrote:
> Hi,
> 
>> On Thu, Sep 07, 2023 at 03:03:47PM +0000, Simon Falsig wrote:
>>> I saw a post from 2021 to the mailing list on generating SBOMs from ptxdist.
>>> Has there been any further work on this?
>>
>> I've not worked on this and I'm not aware of any other efforts in this
>> direction.
>>
>>> We've been looking at implementing this internally - plan would be to generate
>>> the SBOM in CycloneDX format, and consume it with Dependency-Track 
>>> (https://dependencytrack.org) for automatic vulnerability and license monitoring.
>>>
>>> Looks like we're quite close to having a working setup, but it'd make a lot more
>>> sense to have it upstreamed rather than as local patches, so would like to get a
>>> bit of input on the approach, and see if we can make that happen :)
> 
> we have a similar task. There is a rudimentary prototype running on our
> site. It uses licencse-complicance.yaml and a CPE-Dictionary from
> https://nvd.nist.gov/products/cpe to generate a bom.json.  This bom.json
> can than be feed into dependency tracker (We are using docker-compose
> for demonstration).
> 
> <snip bom>
> {
>     "bomFormat": "CycloneDX",
>     "specVersion": "1.4",
>     "serialNumber": "urn:uuid:f20af5d1-0480-477d-8bea-8cbcf6d9268c",
>     "components": [
>         {
>             "name": "attr",
>             "version": "2.4.47",
>             "cpe": "cpe:2.3:a:attr_project:attr:2.4.47:*:*:*:*:*:*:*",
>             "licenses": [
>                 {
>                     "license": {
>                         "id": "GPL-2.0-only"
>                     }
>                 },
>                 {
>                     "license": {
>                         "id": "LGPL-2.0-only"
>                     }
>                 }
>             ],
>             "hashes": [
>                 {
>                     "alg": "MD5",
>                     "content": "84f58dec00b60f2dc8fd1c9709291cc7"
>                 }
>             ]
>         },
>         ...
> </snip>
> 
> 
>> I know very little about this stuff, but I'm open to add support for this.
>> So please sent patches once they are ready.
>>
>>> We've identified two main steps:
>>> 1. Generate the SBOM itself. A minimal version of this can be created from the
>>>    output of the existing fast-bsp-report in 40 lines of Python, using the
>>>    CycloneDX library.
>>>    I'd assume that such a script would just go into the scripts folder in ptxdist?
>>
>> Yes, just put it in the scripts/ folder.
>>
> 
> Our script has 110 lines python. It patches some package names
> that does not match with the db e.g.
> 
> expat -> libexpat
> gdbserver -> gdb
> kernel -> linux_kernel
> libmod -> kmod
> ...
> 
> I am unsure, if this is too error prone. Adding CPE vars to the recepies,
> where only the VERSION is variable, should be more robust.
> 
>>>    Is there a common way of tracking / documenting dependencies of such scripts?
>>
>> You mean Python packages used by the script? Not directly.
> 
> ptxdist-cyclonedx-bom master 
>  ❯ cat requirements.txt 
> pyyaml==6.0.0
> lxml==4.9.2
> 
>> Is the SBOM something that should be created for an image or for the
>> BSP as a whole?
> 
> It should be per image imho. Because depending on configuration, layers,
> collectionconfig, content of images differs a lot. Starting from the
> image, should generate data that is 100% consistent. Not much of a
> difference to licence reports, I think.
> 
>> For the license stuff I have plans to created documents for images. Because
>> those are delivered to customers, so license compliance really needs to be
>> done for each image.
> 
> And this is the same for SBOMi, as mentioned before.
> 
>> I'm not sure what makes sense. So you could either create a global option
>> and then generate the SBOM with each image. And then select
>> HOST_SYSTEM_PYTHON3_* in that option.
>> Or create an SBOM "image" for the whole BSP and select the options there.
> 
> One thing to keep in mind is that cyclonedx is one format, but not the
> only one. Today it is one of the most common used. But there are
> alternatives. So there might be other users who might need the sbom in
> different formats. I wonder if it is easier to adapt to other formats
> using the one, or other approach?
> 
> 
>>> 2. To track vulnerabilities, it's necessary to track the Common Platform
>>>    Enumeration (CPE) name of each package (from https://nvd.nist.gov/products/cpe).
>>>    This will allow matching packages to CVEs.
>>>    My suggestion would be to add a _CPE variable to each package (built from
>>>    whatever other variables make sense, typically _VERSION).
>>
>> Makes sense to me. From what I understand, the CPE is machine readable and
>> can be split into its components if needed, right?
> 
> I think it is. Here CPE number example for package expat:
> "cpe:2.3:a:attr_project:attr:2.4.47:*:*:*:*:*:*:*"
>                                 ^- PTXCONF_EXPAT_VERSION
> 
>>>    I managed to add this
>>>    for the fast report (extracting it to pkg_cpe in rules/post/ptxd_make_world_common.make,
>>>    and adding it to the report in scripts/lib/ptxd_make_world_report.sh), but I
>>>    wouldn't be surprised if there are other places/report that need to track this
>>>    also for consistency?
>>
>> Just the fast and full reports are enough. Anything else in that direction
>> is legacy anyways and should be replaced with stuff based on these reports
>> anyways.
>>
>>>    Packages that specify _CPE would then have this included in their report, and
>>>    there'd be no change for the packages that don't specify it.
>>>
>>>
>>> I'd be happy to get a bit of initial feedback on the approach. I'll have a look
>>> at putting up some initial patches in the coming days too.
> 
> I like it a lot. Need reviewers, testers, 1 mio Euro? We are willing to
> contribute.
> 
> Regards
> Gavin
>