Re: [ptxdist] [PATCH] RFC: sbom_report: Add support

[Bruno Thomsen <bruno.thomsen@gmail.com>]

Den man. 18. sep. 2023 kl. 16.34 skrev Simon Falsig <sfalsig@verity.net>:
>
> From: Simon Falsig <sfalsig@verity.ch>
>
> This provides support for building SBOMs in CycloneDX format.
>
> A target is added alongside the other reports, that (based on the
> fast-bsp-report) extracts name, version, cpe and license of each target
> package, and puts these into a final sbom-report in CycloneDX/JSON
> format.
>
> This requires a working Python3 setup with the cyclonedx-bom package
> installed.

Hi Simon,

I have tested this together with GitLab Dependency Scanning in Ultimate SaaS,
and it seems to be working well.

Tested-by: Bruno Thomsen <bruno.thomsen@gmail.com>

.gitlab-ci.yml example snippet:

-------------8<-------------

ptxdist sbom:
  stage: build
  script:
    - cd ptxdist
    - ./p sbom-report
  artifacts:
    reports:
      dependency_scanning: <platform-dir>/release/sbom-report.json

-------------8<-------------

Thanks for working on this.

/Bruno

> ---
>  bin/ptxdist                          |  3 +-
>  rules/post/ptxd_make_report.make     | 15 ++++++--
>  scripts/lib/ptxd_make_report.sh      | 16 +++++++++
>  scripts/lib/ptxd_make_sbom_report.py | 54 ++++++++++++++++++++++++++++
>  4 files changed, 85 insertions(+), 3 deletions(-)
>  create mode 100644 scripts/lib/ptxd_make_sbom_report.py
>
> diff --git a/bin/ptxdist b/bin/ptxdist
> index dfb619cbd..15be851f5 100755
> --- a/bin/ptxdist
> +++ b/bin/ptxdist
> @@ -780,6 +780,7 @@ Misc:
>    full-bsp-report              generate a yaml file that describes the BSP and
>                                 all packages. More data but will build all
>                                 packages if necessary.
> +  sbom-report                  generate a CycloneDX json SBOM
>    print <var>                  print the contents of a variable, in the way
>                                 it is known by "make"
>    printnext <var>              assumes that the contents of <var> is another
> @@ -1807,7 +1808,7 @@ EOF
>                         ptxd_make_log export_src EXPORTDIR="${1}"
>                         exit
>                         ;;
> -               fast-bsp-report|full-bsp-report)
> +               fast-bsp-report|full-bsp-report|sbom-report)
>                         check_premake_compiler &&
>                         ptxd_make_log "${cmd}"
>                         exit
> diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make
> index eecd2a577..ffa398c95 100644
> --- a/rules/post/ptxd_make_report.make
> +++ b/rules/post/ptxd_make_report.make
> @@ -10,7 +10,9 @@ ptx/report-env = \
>         $(image/env) \
>         ptx_report_target="$(strip $(1))" \
>         ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \
> -       ptx_image_packages="$(IMAGE_PACKAGES)"
> +       ptx_image_packages="$(IMAGE_PACKAGES)" \
> +       ptx_target_packages="$(PACKAGES)"
> +
>
>  PHONY += full-bsp-report
>  full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml
> @@ -26,13 +28,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \
>         @$(call ptx/report-env, $@) ptxd_make_full_bsp_report
>         @$(call finish)
>
> +
>  PHONY += fast-bsp-report
>  fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml
>
> -
>  $(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED)))
>         @$(call targetinfo)
>         @$(call ptx/report-env, $@) ptxd_make_fast_bsp_report
>         @$(call finish)
>
> +
> +PHONY += sbom-report
> +sbom-report: $(RELEASEDIR)/sbom-report.json
> +
> +$(RELEASEDIR)/sbom-report.json: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES)))
> +       @$(call targetinfo)
> +       @$(call ptx/report-env, $@) ptxd_make_sbom_report
> +       @$(call finish)
> +
>  # vim: syntax=make
> diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh
> index a363ca5b3..e2da4c05f 100644
> --- a/scripts/lib/ptxd_make_report.sh
> +++ b/scripts/lib/ptxd_make_report.sh
> @@ -144,3 +144,19 @@ ptxd_make_fast_bsp_report() {
>  }
>  export -f ptxd_make_fast_bsp_report
>
> +ptxd_make_sbom_report() {
> +    local -a ptxd_reply
> +    local pkg_lic pkg
> +
> +    ptxd_make_layer_init || return
> +
> +    echo "Generating $(ptxd_print_path "${ptx_report_target}") ..."
> +    echo
> +
> +    mkdir -p "$(dirname "${ptx_report_target}")" &&
> +    python3 ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report &&
> +    mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" ||
> +    ptxd_bailout "failed to create SBOM report"
> +}
> +export -f ptxd_make_sbom_report
> +
> diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py
> new file mode 100644
> index 000000000..cc6a6f703
> --- /dev/null
> +++ b/scripts/lib/ptxd_make_sbom_report.py
> @@ -0,0 +1,54 @@
> +from cyclonedx.factory.license import LicenseFactory
> +from cyclonedx.factory.license import LicenseChoiceFactory
> +from cyclonedx.model.bom import Bom
> +from cyclonedx.model.component import Component
> +from cyclonedx.output.json import JsonV1Dot4
> +import sys
> +import re
> +
> +lFac = LicenseFactory()
> +lcFac = LicenseChoiceFactory(license_factory=lFac)
> +bom = Bom()
> +
> +for i in range(2, len(sys.argv)):
> +    pkg_report = sys.argv[1] + sys.argv[i] + ".yaml"
> +    with open(pkg_report, 'r') as file:
> +        content = file.read()
> +        name_ = re.search("name: \'(.+)\'", content).group(1)
> +        version_ = re.search("version: \'(.+)\'", content).group(1)
> +
> +        # First see if we have a full CPE specified, then use that
> +        cpe_match = re.search("cpe: \'(.+)\'", content)
> +        cpe_ = None
> +        if cpe_match is not None:
> +            cpe_ = cpe_match.group(1)
> +        else:
> +            # See if we have the individual components
> +            cpe_vendor_match = re.search("cpe_vendor: \'(.+)\'", content)
> +            cpe_product_match = re.search("cpe_product: \'(.+)\'", content)
> +            cpe_version_match = re.search("cpe_version: \'(.+)\'", content)
> +            if cpe_vendor_match is not None and cpe_product_match is not None and cpe_version_match is not None:
> +                cpe_ = "cpe:2.3:a:{vendor}:{product}:{version}:*:*:*:*:*:*:*".format(vendor=cpe_vendor_match.group(1),
> +                                                                                     product=cpe_product_match.group(1),
> +                                                                                     version=cpe_version_match.group(1))
> +
> +        if cpe_ is not None:
> +            # We have a CPE, let's validate it. Regex from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd
> +            if not re.fullmatch("cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;"
> +                                "<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)"
> +                                "|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;<=>@\[\]\^`\{\|}~]"
> +                                "))+(\?*|\*?))|[\*\-])){4}", cpe_):
> +                raise ValueError("Constructed CPE is not valid: {cpe}".format(cpe=cpe_))
> +
> +        licenses_ = re.search("licenses: \'(.+)\'", content).group(1)
> +        comp = Component(
> +            name=name_,
> +            version=version_,
> +            cpe=cpe_,
> +            licenses=[lcFac.make_with_license(licenses_)],
> +            bom_ref=name_ + "@" + version_
> +        )
> +        bom.components.add(comp)
> +
> +serializedJSON = JsonV1Dot4(bom).output_as_string()
> +print(serializedJSON)
> --
> 2.25.1
>
>