From mboxrd@z Thu Jan 1 00:00:00 1970 Delivery-date: Sat, 21 Oct 2023 15:53:15 +0200 Received: from metis.whiteo.stw.pengutronix.de ([2a0a:edc0:2:b01:1d::104]) by lore.white.stw.pengutronix.de with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1quCPj-00Ge0T-Fa for lore@lore.pengutronix.de; Sat, 21 Oct 2023 15:53:15 +0200 Received: from localhost ([127.0.0.1] helo=metis.whiteo.stw.pengutronix.de) by metis.whiteo.stw.pengutronix.de with esmtp (Exim 4.92) (envelope-from ) id 1quCPi-0003Gp-7D; Sat, 21 Oct 2023 15:53:14 +0200 Received: from mail-yw1-x1129.google.com ([2607:f8b0:4864:20::1129]) by metis.whiteo.stw.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1quCPG-0003GA-V5 for ptxdist@pengutronix.de; Sat, 21 Oct 2023 15:52:51 +0200 Received: by mail-yw1-x1129.google.com with SMTP id 00721157ae682-5a7af20c488so18999187b3.1 for ; Sat, 21 Oct 2023 06:52:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697896365; x=1698501165; darn=pengutronix.de; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Myl6fbIjiO3ox5e/OmZfzkpZNh9+QEW4M74EpZFuX+s=; b=XC9y+39YDCktVC8dZTS/TMioCHtrYcvG7HQuw+yUFDkrRoNQ3umx/T7+jwy2frvQPa G2Cg9+Yvx/bdwS0HU45uaIawQKgP/wUKDTkYGVswZCrlaP3Z1zX+jPzHJB1UlV845rf5 m/1LBIkTNnxQP7ta7G40yCw62HCrFUkOKYZCedKWdj8yDqVJg58H84dTTkSKh6ogEIk2 wuxb6HtdJmMO20LZOng5Un2H6Uoq+17THTCUSlOfUUy6kRNf+OUDcMMgOVM7wihE6pxS 82WdCNoTSrlNVhPfi94wweA4KM+DyqZYfpPpwuZ3gOJ7cHysenFiGDQCHh1pZM22yLRY HPQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697896365; x=1698501165; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Myl6fbIjiO3ox5e/OmZfzkpZNh9+QEW4M74EpZFuX+s=; b=WPr1g0sb3vTYDN2/TRnteBHX6se9DtmcBElKhaVYi/mdwd6KLQ3VcRQ7eZMXrQkkS9 BHQVhM6ekoc0vhJc1oeXjXYwCTDO9XQ7bvPE1G9sOGum2jvCnErhAKikhiH+N2PxWLJ9 4MTx1L7lQHSpY06MhyVhey/Pas45N0WNsiAWSAX6tD3l7JQsyoIZZSwuWQ8EZeHIKy+z qFgAQYnq7rUtB1QwK4qTgzRlRJOAGhYr0Fv1P6q6FrcTksBj//yvErCG7//JzpK8nMLk w5gPF5QLIHkGlJkpl4ZfQ5B6a7q8xhzG14DxhkknLgEF7KH+h6Cy2OXzwFR8TabELW/y Nlug== X-Gm-Message-State: AOJu0YzeuWKVA803QHSxamYL7nornftCOWMuRyrrkWPOXfPmFt0Rczh6 rOkE27MWt4VSIX5w5h3XD53u0C2RvU2b2PiCylEWquzrj8c= X-Google-Smtp-Source: AGHT+IEMocz3Ytt21trF+pk2I4DKNToocGjmJVth4enRMBxV8BQl0QG4pPM8mqtknLknnEF5eg8kB0It+gr5/MA9W84= X-Received: by 2002:a81:480c:0:b0:5a9:117d:ddbc with SMTP id v12-20020a81480c000000b005a9117dddbcmr1897908ywa.15.1697896365362; Sat, 21 Oct 2023 06:52:45 -0700 (PDT) MIME-Version: 1.0 References: <20230913160546.71046-3-sfalsig@verity.net> <20230918143339.4126-1-sfalsig@verity.net> In-Reply-To: <20230918143339.4126-1-sfalsig@verity.net> From: Bruno Thomsen Date: Sat, 21 Oct 2023 15:52:29 +0200 Message-ID: To: ptxdist@pengutronix.de Content-Type: text/plain; charset="UTF-8" X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on metis.whiteo.stw.pengutronix.de X-Spam-Level: X-Spam-Status: No, score=-2.7 required=4.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 Subject: Re: [ptxdist] [PATCH] RFC: sbom_report: Add support X-BeenThere: ptxdist@pengutronix.de X-Mailman-Version: 2.1.29 Precedence: list List-Id: PTXdist Development Mailing List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: ptxdist@pengutronix.de Cc: Simon Falsig Sender: "ptxdist" X-SA-Exim-Connect-IP: 127.0.0.1 X-SA-Exim-Mail-From: ptxdist-bounces@pengutronix.de X-SA-Exim-Scanned: No (on metis.whiteo.stw.pengutronix.de); SAEximRunCond expanded to false Den man. 18. sep. 2023 kl. 16.34 skrev Simon Falsig : > > From: Simon Falsig > > This provides support for building SBOMs in CycloneDX format. > > A target is added alongside the other reports, that (based on the > fast-bsp-report) extracts name, version, cpe and license of each target > package, and puts these into a final sbom-report in CycloneDX/JSON > format. > > This requires a working Python3 setup with the cyclonedx-bom package > installed. Hi Simon, I have tested this together with GitLab Dependency Scanning in Ultimate SaaS, and it seems to be working well. Tested-by: Bruno Thomsen .gitlab-ci.yml example snippet: -------------8<------------- ptxdist sbom: stage: build script: - cd ptxdist - ./p sbom-report artifacts: reports: dependency_scanning: /release/sbom-report.json -------------8<------------- Thanks for working on this. /Bruno > --- > bin/ptxdist | 3 +- > rules/post/ptxd_make_report.make | 15 ++++++-- > scripts/lib/ptxd_make_report.sh | 16 +++++++++ > scripts/lib/ptxd_make_sbom_report.py | 54 ++++++++++++++++++++++++++++ > 4 files changed, 85 insertions(+), 3 deletions(-) > create mode 100644 scripts/lib/ptxd_make_sbom_report.py > > diff --git a/bin/ptxdist b/bin/ptxdist > index dfb619cbd..15be851f5 100755 > --- a/bin/ptxdist > +++ b/bin/ptxdist > @@ -780,6 +780,7 @@ Misc: > full-bsp-report generate a yaml file that describes the BSP and > all packages. More data but will build all > packages if necessary. > + sbom-report generate a CycloneDX json SBOM > print print the contents of a variable, in the way > it is known by "make" > printnext assumes that the contents of is another > @@ -1807,7 +1808,7 @@ EOF > ptxd_make_log export_src EXPORTDIR="${1}" > exit > ;; > - fast-bsp-report|full-bsp-report) > + fast-bsp-report|full-bsp-report|sbom-report) > check_premake_compiler && > ptxd_make_log "${cmd}" > exit > diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make > index eecd2a577..ffa398c95 100644 > --- a/rules/post/ptxd_make_report.make > +++ b/rules/post/ptxd_make_report.make > @@ -10,7 +10,9 @@ ptx/report-env = \ > $(image/env) \ > ptx_report_target="$(strip $(1))" \ > ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \ > - ptx_image_packages="$(IMAGE_PACKAGES)" > + ptx_image_packages="$(IMAGE_PACKAGES)" \ > + ptx_target_packages="$(PACKAGES)" > + > > PHONY += full-bsp-report > full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml > @@ -26,13 +28,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \ > @$(call ptx/report-env, $@) ptxd_make_full_bsp_report > @$(call finish) > > + > PHONY += fast-bsp-report > fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml > > - > $(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED))) > @$(call targetinfo) > @$(call ptx/report-env, $@) ptxd_make_fast_bsp_report > @$(call finish) > > + > +PHONY += sbom-report > +sbom-report: $(RELEASEDIR)/sbom-report.json > + > +$(RELEASEDIR)/sbom-report.json: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES))) > + @$(call targetinfo) > + @$(call ptx/report-env, $@) ptxd_make_sbom_report > + @$(call finish) > + > # vim: syntax=make > diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh > index a363ca5b3..e2da4c05f 100644 > --- a/scripts/lib/ptxd_make_report.sh > +++ b/scripts/lib/ptxd_make_report.sh > @@ -144,3 +144,19 @@ ptxd_make_fast_bsp_report() { > } > export -f ptxd_make_fast_bsp_report > > +ptxd_make_sbom_report() { > + local -a ptxd_reply > + local pkg_lic pkg > + > + ptxd_make_layer_init || return > + > + echo "Generating $(ptxd_print_path "${ptx_report_target}") ..." > + echo > + > + mkdir -p "$(dirname "${ptx_report_target}")" && > + python3 ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report && > + mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" || > + ptxd_bailout "failed to create SBOM report" > +} > +export -f ptxd_make_sbom_report > + > diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py > new file mode 100644 > index 000000000..cc6a6f703 > --- /dev/null > +++ b/scripts/lib/ptxd_make_sbom_report.py > @@ -0,0 +1,54 @@ > +from cyclonedx.factory.license import LicenseFactory > +from cyclonedx.factory.license import LicenseChoiceFactory > +from cyclonedx.model.bom import Bom > +from cyclonedx.model.component import Component > +from cyclonedx.output.json import JsonV1Dot4 > +import sys > +import re > + > +lFac = LicenseFactory() > +lcFac = LicenseChoiceFactory(license_factory=lFac) > +bom = Bom() > + > +for i in range(2, len(sys.argv)): > + pkg_report = sys.argv[1] + sys.argv[i] + ".yaml" > + with open(pkg_report, 'r') as file: > + content = file.read() > + name_ = re.search("name: \'(.+)\'", content).group(1) > + version_ = re.search("version: \'(.+)\'", content).group(1) > + > + # First see if we have a full CPE specified, then use that > + cpe_match = re.search("cpe: \'(.+)\'", content) > + cpe_ = None > + if cpe_match is not None: > + cpe_ = cpe_match.group(1) > + else: > + # See if we have the individual components > + cpe_vendor_match = re.search("cpe_vendor: \'(.+)\'", content) > + cpe_product_match = re.search("cpe_product: \'(.+)\'", content) > + cpe_version_match = re.search("cpe_version: \'(.+)\'", content) > + if cpe_vendor_match is not None and cpe_product_match is not None and cpe_version_match is not None: > + cpe_ = "cpe:2.3:a:{vendor}:{product}:{version}:*:*:*:*:*:*:*".format(vendor=cpe_vendor_match.group(1), > + product=cpe_product_match.group(1), > + version=cpe_version_match.group(1)) > + > + if cpe_ is not None: > + # We have a CPE, let's validate it. Regex from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd > + if not re.fullmatch("cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;" > + "<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)" > + "|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;<=>@\[\]\^`\{\|}~]" > + "))+(\?*|\*?))|[\*\-])){4}", cpe_): > + raise ValueError("Constructed CPE is not valid: {cpe}".format(cpe=cpe_)) > + > + licenses_ = re.search("licenses: \'(.+)\'", content).group(1) > + comp = Component( > + name=name_, > + version=version_, > + cpe=cpe_, > + licenses=[lcFac.make_with_license(licenses_)], > + bom_ref=name_ + "@" + version_ > + ) > + bom.components.add(comp) > + > +serializedJSON = JsonV1Dot4(bom).output_as_string() > +print(serializedJSON) > -- > 2.25.1 > >