[ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages

[ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages

[Simon Falsig]

From: Simon Falsig <sfalsig@verity.ch>

If a package specifies a CPE, this is extracted into the fast report for
that package. If no CPE is specified, then no value is added.

The CPE (Common Platform Enumerator) allows matching CVEs to specific
packages, and see if these apply to a specific deployment.
---
 rules/post/ptxd_make_world_common.make | 1 +
 scripts/lib/ptxd_make_world_report.sh  | 1 +
 2 files changed, 2 insertions(+)

diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make
index 08120607a..6c646fb16 100644
--- a/rules/post/ptxd_make_world_common.make
+++ b/rules/post/ptxd_make_world_common.make
@@ -78,6 +78,7 @@ world/env/impl = \
 	pkg_PKG="$(call ptx/escape,$(1))"					\
 	pkg_pkg="$(call ptx/escape,$($(1)))"					\
 	pkg_version="$(call ptx/escape,$($(1)_VERSION))"			\
+	pkg_cpe="$(call ptx/escape,$($(1)_CPE))"			\
 	pkg_config="$(call ptx/escape,$($(1)_CONFIG))"				\
 	pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))"			\
 	pkg_path="$(call ptx/escape,$($(1)_PATH))"				\
diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh
index dbdae5736..2bfe4c201 100644
--- a/scripts/lib/ptxd_make_world_report.sh
+++ b/scripts/lib/ptxd_make_world_report.sh
@@ -39,6 +39,7 @@ ptxd_make_world_report_yaml() {
     do_list "rundeps:" "${pkg_run_deps}"
     do_echo "config:" "${pkg_config}"
     do_echo "version:" "${pkg_version}"
+    do_echo "cpe:" "${pkg_cpe}"
     do_list "url:" "${pkg_url}"
     do_echo "md5:" "${pkg_md5}"
     do_echo "source:" "${pkg_src}"
-- 
2.25.1

[ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages

[Simon Falsig]

From: Simon Falsig <sfalsig@verity.ch>

Just to see how this could look for a handful of packages. Note that all
of these have a different way of specifying the vendor ID (one is
$PACKAGE_project, one is just $PACKAGE, one is something completely
different).
---
 rules/acl.make            | 1 +
 rules/busybox.make        | 1 +
 rules/wpa_supplicant.make | 1 +
 3 files changed, 3 insertions(+)

diff --git a/rules/acl.make b/rules/acl.make
index daa5837c9..10769681b 100644
--- a/rules/acl.make
+++ b/rules/acl.make
@@ -21,6 +21,7 @@ ACL_SUFFIX	:= tar.gz
 ACL_URL		:= http://download.savannah.gnu.org/releases/acl/$(ACL).$(ACL_SUFFIX)
 ACL_SOURCE	:= $(SRCDIR)/$(ACL).$(ACL_SUFFIX)
 ACL_DIR		:= $(BUILDDIR)/$(ACL)
+ACL_CPE		:= cpe:2.3:a:acl_project:acl:$(ACL_VERSION):*:*:*:*:*:*:*
 ACL_LICENSE	:= LGPL-2.1-or-later
 ACL_LICENSE_FILES:= file://doc/COPYING.LGPL;md5=9e9a206917f8af112da634ce3ab41764
 ifdef PTXCONF_ACL_TOOLS
diff --git a/rules/busybox.make b/rules/busybox.make
index 413ad735c..eb8721229 100644
--- a/rules/busybox.make
+++ b/rules/busybox.make
@@ -22,6 +22,7 @@ BUSYBOX_URL	:= https://www.busybox.net/downloads/$(BUSYBOX).$(BUSYBOX_SUFFIX)
 BUSYBOX_SOURCE	:= $(SRCDIR)/$(BUSYBOX).$(BUSYBOX_SUFFIX)
 BUSYBOX_DIR	:= $(BUILDDIR)/$(BUSYBOX)
 BUSYBOX_KCONFIG	:= $(BUSYBOX_DIR)/Config.in
+BUSYBOX_CPE	:= cpe:2.3:a:busybox:busybox:$(BUSYBOX_VERSION):*:*:*:*:*:*:*
 BUSYBOX_LICENSE	:= GPL-2.0-only
 BUSYBOX_LICENSE_FILES := file://LICENSE;md5=de10de48642ab74318e893a61105afbb
 
diff --git a/rules/wpa_supplicant.make b/rules/wpa_supplicant.make
index 55d43b9b2..c0bf54f25 100644
--- a/rules/wpa_supplicant.make
+++ b/rules/wpa_supplicant.make
@@ -26,6 +26,7 @@ WPA_SUPPLICANT_SUBDIR	:= $(WPA_SUPPLICANT_NAME)
 # Use '=' to delay $(shell ...) calls until this is needed
 WPA_SUPPLICANT_CONFIG	 = $(call ptx/get-alternative, config/wpasupplicant, defconfig)
 WPA_SUPPLICANT_DOTCONFIG:= $(BUILDDIR)/$(WPA_SUPPLICANT)/$(WPA_SUPPLICANT_SUBDIR)/.config
+WPA_SUPPLICANT_CPE	:= cpe:2.3:a:w1.fi:wpa_supplicant:$(WPA_SUPPLICANT_VERSION):*:*:*:*:*:*:*
 WPA_SUPPLICANT_LICENSE	:= GPL-2.0-only
 
 # ----------------------------------------------------------------------------
-- 
2.25.1

[ptxdist] [PATCH] RFC: Add CPE for a few packages

[Simon Falsig]

From: Simon Falsig <sfalsig@verity.ch>

Just to see how this could look for a handful of packages. Note that all
of these have a different way of specifying the vendor ID (one is
$PACKAGE_project, one is just $PACKAGE, one is something completely
different).
---
 rules/acl.make            | 1 +
 rules/busybox.make        | 2 ++
 rules/wpa_supplicant.make | 3 +++
 3 files changed, 6 insertions(+)

diff --git a/rules/acl.make b/rules/acl.make
index daa5837c9..10769681b 100644
--- a/rules/acl.make
+++ b/rules/acl.make
@@ -21,6 +21,7 @@ ACL_SUFFIX	:= tar.gz
 ACL_URL		:= http://download.savannah.gnu.org/releases/acl/$(ACL).$(ACL_SUFFIX)
 ACL_SOURCE	:= $(SRCDIR)/$(ACL).$(ACL_SUFFIX)
 ACL_DIR		:= $(BUILDDIR)/$(ACL)
+ACL_CPE		:= cpe:2.3:a:acl_project:acl:$(ACL_VERSION):*:*:*:*:*:*:*
 ACL_LICENSE	:= LGPL-2.1-or-later
 ACL_LICENSE_FILES:= file://doc/COPYING.LGPL;md5=9e9a206917f8af112da634ce3ab41764
 ifdef PTXCONF_ACL_TOOLS
diff --git a/rules/busybox.make b/rules/busybox.make
index 413ad735c..b3ed2ff7a 100644
--- a/rules/busybox.make
+++ b/rules/busybox.make
@@ -22,6 +22,8 @@ BUSYBOX_URL	:= https://www.busybox.net/downloads/$(BUSYBOX).$(BUSYBOX_SUFFIX)
 BUSYBOX_SOURCE	:= $(SRCDIR)/$(BUSYBOX).$(BUSYBOX_SUFFIX)
 BUSYBOX_DIR	:= $(BUILDDIR)/$(BUSYBOX)
 BUSYBOX_KCONFIG	:= $(BUSYBOX_DIR)/Config.in
+BUSYBOX_CPE_VENDOR	:= busybox
+BUSYBOX_CPE_PRODUCT	:= busybox
 BUSYBOX_LICENSE	:= GPL-2.0-only
 BUSYBOX_LICENSE_FILES := file://LICENSE;md5=de10de48642ab74318e893a61105afbb
 
diff --git a/rules/wpa_supplicant.make b/rules/wpa_supplicant.make
index 55d43b9b2..2f4d9c607 100644
--- a/rules/wpa_supplicant.make
+++ b/rules/wpa_supplicant.make
@@ -26,6 +26,9 @@ WPA_SUPPLICANT_SUBDIR	:= $(WPA_SUPPLICANT_NAME)
 # Use '=' to delay $(shell ...) calls until this is needed
 WPA_SUPPLICANT_CONFIG	 = $(call ptx/get-alternative, config/wpasupplicant, defconfig)
 WPA_SUPPLICANT_DOTCONFIG:= $(BUILDDIR)/$(WPA_SUPPLICANT)/$(WPA_SUPPLICANT_SUBDIR)/.config
+WPA_SUPPLICANT_CPE_VENDOR := w1.fi
+WPA_SUPPLICANT_CPE_PRODUCT := wpa_supplicant
+WPA_SUPPLICANT_CPE_VERSION := $(WPA_SUPPLICANT_VERSION)
 WPA_SUPPLICANT_LICENSE	:= GPL-2.0-only
 
 # ----------------------------------------------------------------------------
-- 
2.25.1

[ptxdist] [PATCH 3/3] RFC: sbom_report: Add support

[Simon Falsig]

From: Simon Falsig <sfalsig@verity.ch>

This provides support for building SBOMs in CycloneDX format.

A target is added alongside the other reports, that (based on the
fast-bsp-report) extracts name, version, cpe and license of each target
package, and puts these into a final sbom-report in CycloneDX/JSON
format.

This requires a working Python3 setup with the cyclonedx-bom package
installed.
---
 bin/ptxdist                          |  3 ++-
 rules/post/ptxd_make_report.make     | 15 ++++++++++--
 scripts/lib/ptxd_make_report.sh      | 16 +++++++++++++
 scripts/lib/ptxd_make_sbom_report.py | 35 ++++++++++++++++++++++++++++
 4 files changed, 66 insertions(+), 3 deletions(-)
 create mode 100644 scripts/lib/ptxd_make_sbom_report.py

diff --git a/bin/ptxdist b/bin/ptxdist
index dfb619cbd..15be851f5 100755
--- a/bin/ptxdist
+++ b/bin/ptxdist
@@ -780,6 +780,7 @@ Misc:
   full-bsp-report		generate a yaml file that describes the BSP and
 				all packages. More data but will build all
 				packages if necessary.
+  sbom-report			generate a CycloneDX json SBOM
   print <var>			print the contents of a variable, in the way
 				it is known by "make"
   printnext <var>		assumes that the contents of <var> is another
@@ -1807,7 +1808,7 @@ EOF
 			ptxd_make_log export_src EXPORTDIR="${1}"
 			exit
 			;;
-		fast-bsp-report|full-bsp-report)
+		fast-bsp-report|full-bsp-report|sbom-report)
 			check_premake_compiler &&
 			ptxd_make_log "${cmd}"
 			exit
diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make
index eecd2a577..ffa398c95 100644
--- a/rules/post/ptxd_make_report.make
+++ b/rules/post/ptxd_make_report.make
@@ -10,7 +10,9 @@ ptx/report-env = \
 	$(image/env) \
 	ptx_report_target="$(strip $(1))" \
 	ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \
-	ptx_image_packages="$(IMAGE_PACKAGES)"
+	ptx_image_packages="$(IMAGE_PACKAGES)" \
+	ptx_target_packages="$(PACKAGES)"
+
 
 PHONY += full-bsp-report
 full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml
@@ -26,13 +28,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \
 	@$(call ptx/report-env, $@) ptxd_make_full_bsp_report
 	@$(call finish)
 
+
 PHONY += fast-bsp-report
 fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml
 
-
 $(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED)))
 	@$(call targetinfo)
 	@$(call ptx/report-env, $@) ptxd_make_fast_bsp_report
 	@$(call finish)
 
+
+PHONY += sbom-report
+sbom-report: $(RELEASEDIR)/sbom-report.json
+
+$(RELEASEDIR)/sbom-report.json: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES)))
+	@$(call targetinfo)
+	@$(call ptx/report-env, $@) ptxd_make_sbom_report
+	@$(call finish)
+
 # vim: syntax=make
diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh
index a363ca5b3..e2da4c05f 100644
--- a/scripts/lib/ptxd_make_report.sh
+++ b/scripts/lib/ptxd_make_report.sh
@@ -144,3 +144,19 @@ ptxd_make_fast_bsp_report() {
 }
 export -f ptxd_make_fast_bsp_report
 
+ptxd_make_sbom_report() {
+    local -a ptxd_reply
+    local pkg_lic pkg
+
+    ptxd_make_layer_init || return
+
+    echo "Generating $(ptxd_print_path "${ptx_report_target}") ..."
+    echo
+
+    mkdir -p "$(dirname "${ptx_report_target}")" &&
+    python3 ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report &&
+    mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" ||
+    ptxd_bailout "failed to create SBOM report"
+}
+export -f ptxd_make_sbom_report
+
diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py
new file mode 100644
index 000000000..aecb4fae5
--- /dev/null
+++ b/scripts/lib/ptxd_make_sbom_report.py
@@ -0,0 +1,35 @@
+from cyclonedx.factory.license import LicenseFactory
+from cyclonedx.factory.license import LicenseChoiceFactory
+from cyclonedx.model.bom import Bom
+from cyclonedx.model.component import Component
+from cyclonedx.output.json import JsonV1Dot4
+import sys
+import re
+
+lFac  = LicenseFactory()
+lcFac = LicenseChoiceFactory(license_factory=lFac)
+bom   = Bom()
+
+for i in range(2,len(sys.argv)):
+    pkg_report = sys.argv[1] + sys.argv[i] + ".yaml"
+    with open(pkg_report, 'r') as file:
+        content   = file.read()
+        name_     = re.search("name: \'(.+)\'", content).group(1)
+        version_  = re.search("version: \'(.+)\'", content).group(1)
+        cpeMatch = re.search("cpe: \'(.+)\'", content)
+        cpe_ = None
+        if cpeMatch is not None:
+            cpe_ = cpeMatch.group(1)
+        licenses_ = re.search("licenses: \'(.+)\'", content).group(1)
+        comp = Component(
+            name     = name_,
+            version  = version_,
+            cpe      = cpe_,
+            licenses = [lcFac.make_with_license(licenses_)],
+            bom_ref  = name_ + "@" + version_
+        )
+        bom.components.add(comp)
+
+serializedJSON = JsonV1Dot4(bom).output_as_string()
+print(serializedJSON)
+
-- 
2.25.1

[ptxdist] [PATCH] RFC: sbom_report: Add support

[Simon Falsig]

From: Simon Falsig <sfalsig@verity.ch>

This provides support for building SBOMs in CycloneDX format.

A target is added alongside the other reports, that (based on the
fast-bsp-report) extracts name, version, cpe and license of each target
package, and puts these into a final sbom-report in CycloneDX/JSON
format.

This requires a working Python3 setup with the cyclonedx-bom package
installed.
---
 bin/ptxdist                          |  3 +-
 rules/post/ptxd_make_report.make     | 15 ++++++--
 scripts/lib/ptxd_make_report.sh      | 16 +++++++++
 scripts/lib/ptxd_make_sbom_report.py | 54 ++++++++++++++++++++++++++++
 4 files changed, 85 insertions(+), 3 deletions(-)
 create mode 100644 scripts/lib/ptxd_make_sbom_report.py

diff --git a/bin/ptxdist b/bin/ptxdist
index dfb619cbd..15be851f5 100755
--- a/bin/ptxdist
+++ b/bin/ptxdist
@@ -780,6 +780,7 @@ Misc:
   full-bsp-report		generate a yaml file that describes the BSP and
 				all packages. More data but will build all
 				packages if necessary.
+  sbom-report			generate a CycloneDX json SBOM
   print <var>			print the contents of a variable, in the way
 				it is known by "make"
   printnext <var>		assumes that the contents of <var> is another
@@ -1807,7 +1808,7 @@ EOF
 			ptxd_make_log export_src EXPORTDIR="${1}"
 			exit
 			;;
-		fast-bsp-report|full-bsp-report)
+		fast-bsp-report|full-bsp-report|sbom-report)
 			check_premake_compiler &&
 			ptxd_make_log "${cmd}"
 			exit
diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make
index eecd2a577..ffa398c95 100644
--- a/rules/post/ptxd_make_report.make
+++ b/rules/post/ptxd_make_report.make
@@ -10,7 +10,9 @@ ptx/report-env = \
 	$(image/env) \
 	ptx_report_target="$(strip $(1))" \
 	ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \
-	ptx_image_packages="$(IMAGE_PACKAGES)"
+	ptx_image_packages="$(IMAGE_PACKAGES)" \
+	ptx_target_packages="$(PACKAGES)"
+
 
 PHONY += full-bsp-report
 full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml
@@ -26,13 +28,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \
 	@$(call ptx/report-env, $@) ptxd_make_full_bsp_report
 	@$(call finish)
 
+
 PHONY += fast-bsp-report
 fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml
 
-
 $(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED)))
 	@$(call targetinfo)
 	@$(call ptx/report-env, $@) ptxd_make_fast_bsp_report
 	@$(call finish)
 
+
+PHONY += sbom-report
+sbom-report: $(RELEASEDIR)/sbom-report.json
+
+$(RELEASEDIR)/sbom-report.json: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES)))
+	@$(call targetinfo)
+	@$(call ptx/report-env, $@) ptxd_make_sbom_report
+	@$(call finish)
+
 # vim: syntax=make
diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh
index a363ca5b3..e2da4c05f 100644
--- a/scripts/lib/ptxd_make_report.sh
+++ b/scripts/lib/ptxd_make_report.sh
@@ -144,3 +144,19 @@ ptxd_make_fast_bsp_report() {
 }
 export -f ptxd_make_fast_bsp_report
 
+ptxd_make_sbom_report() {
+    local -a ptxd_reply
+    local pkg_lic pkg
+
+    ptxd_make_layer_init || return
+
+    echo "Generating $(ptxd_print_path "${ptx_report_target}") ..."
+    echo
+
+    mkdir -p "$(dirname "${ptx_report_target}")" &&
+    python3 ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report &&
+    mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" ||
+    ptxd_bailout "failed to create SBOM report"
+}
+export -f ptxd_make_sbom_report
+
diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py
new file mode 100644
index 000000000..cc6a6f703
--- /dev/null
+++ b/scripts/lib/ptxd_make_sbom_report.py
@@ -0,0 +1,54 @@
+from cyclonedx.factory.license import LicenseFactory
+from cyclonedx.factory.license import LicenseChoiceFactory
+from cyclonedx.model.bom import Bom
+from cyclonedx.model.component import Component
+from cyclonedx.output.json import JsonV1Dot4
+import sys
+import re
+
+lFac = LicenseFactory()
+lcFac = LicenseChoiceFactory(license_factory=lFac)
+bom = Bom()
+
+for i in range(2, len(sys.argv)):
+    pkg_report = sys.argv[1] + sys.argv[i] + ".yaml"
+    with open(pkg_report, 'r') as file:
+        content = file.read()
+        name_ = re.search("name: \'(.+)\'", content).group(1)
+        version_ = re.search("version: \'(.+)\'", content).group(1)
+
+        # First see if we have a full CPE specified, then use that
+        cpe_match = re.search("cpe: \'(.+)\'", content)
+        cpe_ = None
+        if cpe_match is not None:
+            cpe_ = cpe_match.group(1)
+        else:
+            # See if we have the individual components
+            cpe_vendor_match = re.search("cpe_vendor: \'(.+)\'", content)
+            cpe_product_match = re.search("cpe_product: \'(.+)\'", content)
+            cpe_version_match = re.search("cpe_version: \'(.+)\'", content)
+            if cpe_vendor_match is not None and cpe_product_match is not None and cpe_version_match is not None:
+                cpe_ = "cpe:2.3:a:{vendor}:{product}:{version}:*:*:*:*:*:*:*".format(vendor=cpe_vendor_match.group(1),
+                                                                                     product=cpe_product_match.group(1),
+                                                                                     version=cpe_version_match.group(1))
+
+        if cpe_ is not None:
+            # We have a CPE, let's validate it. Regex from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd
+            if not re.fullmatch("cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;"
+                                "<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)"
+                                "|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;<=>@\[\]\^`\{\|}~]"
+                                "))+(\?*|\*?))|[\*\-])){4}", cpe_):
+                raise ValueError("Constructed CPE is not valid: {cpe}".format(cpe=cpe_))
+
+        licenses_ = re.search("licenses: \'(.+)\'", content).group(1)
+        comp = Component(
+            name=name_,
+            version=version_,
+            cpe=cpe_,
+            licenses=[lcFac.make_with_license(licenses_)],
+            bom_ref=name_ + "@" + version_
+        )
+        bom.components.add(comp)
+
+serializedJSON = JsonV1Dot4(bom).output_as_string()
+print(serializedJSON)
-- 
2.25.1

Re: [ptxdist] [PATCH] RFC: sbom_report: Add support

[Bruno Thomsen]

Den man. 18. sep. 2023 kl. 16.34 skrev Simon Falsig <sfalsig@verity.net>:
>
> From: Simon Falsig <sfalsig@verity.ch>
>
> This provides support for building SBOMs in CycloneDX format.
>
> A target is added alongside the other reports, that (based on the
> fast-bsp-report) extracts name, version, cpe and license of each target
> package, and puts these into a final sbom-report in CycloneDX/JSON
> format.
>
> This requires a working Python3 setup with the cyclonedx-bom package
> installed.

Hi Simon,

I have tested this together with GitLab Dependency Scanning in Ultimate SaaS,
and it seems to be working well.

Tested-by: Bruno Thomsen <bruno.thomsen@gmail.com>

.gitlab-ci.yml example snippet:

-------------8<-------------

ptxdist sbom:
  stage: build
  script:
    - cd ptxdist
    - ./p sbom-report
  artifacts:
    reports:
      dependency_scanning: <platform-dir>/release/sbom-report.json

-------------8<-------------

Thanks for working on this.

/Bruno

> ---
>  bin/ptxdist                          |  3 +-
>  rules/post/ptxd_make_report.make     | 15 ++++++--
>  scripts/lib/ptxd_make_report.sh      | 16 +++++++++
>  scripts/lib/ptxd_make_sbom_report.py | 54 ++++++++++++++++++++++++++++
>  4 files changed, 85 insertions(+), 3 deletions(-)
>  create mode 100644 scripts/lib/ptxd_make_sbom_report.py
>
> diff --git a/bin/ptxdist b/bin/ptxdist
> index dfb619cbd..15be851f5 100755
> --- a/bin/ptxdist
> +++ b/bin/ptxdist
> @@ -780,6 +780,7 @@ Misc:
>    full-bsp-report              generate a yaml file that describes the BSP and
>                                 all packages. More data but will build all
>                                 packages if necessary.
> +  sbom-report                  generate a CycloneDX json SBOM
>    print <var>                  print the contents of a variable, in the way
>                                 it is known by "make"
>    printnext <var>              assumes that the contents of <var> is another
> @@ -1807,7 +1808,7 @@ EOF
>                         ptxd_make_log export_src EXPORTDIR="${1}"
>                         exit
>                         ;;
> -               fast-bsp-report|full-bsp-report)
> +               fast-bsp-report|full-bsp-report|sbom-report)
>                         check_premake_compiler &&
>                         ptxd_make_log "${cmd}"
>                         exit
> diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make
> index eecd2a577..ffa398c95 100644
> --- a/rules/post/ptxd_make_report.make
> +++ b/rules/post/ptxd_make_report.make
> @@ -10,7 +10,9 @@ ptx/report-env = \
>         $(image/env) \
>         ptx_report_target="$(strip $(1))" \
>         ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \
> -       ptx_image_packages="$(IMAGE_PACKAGES)"
> +       ptx_image_packages="$(IMAGE_PACKAGES)" \
> +       ptx_target_packages="$(PACKAGES)"
> +
>
>  PHONY += full-bsp-report
>  full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml
> @@ -26,13 +28,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \
>         @$(call ptx/report-env, $@) ptxd_make_full_bsp_report
>         @$(call finish)
>
> +
>  PHONY += fast-bsp-report
>  fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml
>
> -
>  $(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED)))
>         @$(call targetinfo)
>         @$(call ptx/report-env, $@) ptxd_make_fast_bsp_report
>         @$(call finish)
>
> +
> +PHONY += sbom-report
> +sbom-report: $(RELEASEDIR)/sbom-report.json
> +
> +$(RELEASEDIR)/sbom-report.json: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES)))
> +       @$(call targetinfo)
> +       @$(call ptx/report-env, $@) ptxd_make_sbom_report
> +       @$(call finish)
> +
>  # vim: syntax=make
> diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh
> index a363ca5b3..e2da4c05f 100644
> --- a/scripts/lib/ptxd_make_report.sh
> +++ b/scripts/lib/ptxd_make_report.sh
> @@ -144,3 +144,19 @@ ptxd_make_fast_bsp_report() {
>  }
>  export -f ptxd_make_fast_bsp_report
>
> +ptxd_make_sbom_report() {
> +    local -a ptxd_reply
> +    local pkg_lic pkg
> +
> +    ptxd_make_layer_init || return
> +
> +    echo "Generating $(ptxd_print_path "${ptx_report_target}") ..."
> +    echo
> +
> +    mkdir -p "$(dirname "${ptx_report_target}")" &&
> +    python3 ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report &&
> +    mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" ||
> +    ptxd_bailout "failed to create SBOM report"
> +}
> +export -f ptxd_make_sbom_report
> +
> diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py
> new file mode 100644
> index 000000000..cc6a6f703
> --- /dev/null
> +++ b/scripts/lib/ptxd_make_sbom_report.py
> @@ -0,0 +1,54 @@
> +from cyclonedx.factory.license import LicenseFactory
> +from cyclonedx.factory.license import LicenseChoiceFactory
> +from cyclonedx.model.bom import Bom
> +from cyclonedx.model.component import Component
> +from cyclonedx.output.json import JsonV1Dot4
> +import sys
> +import re
> +
> +lFac = LicenseFactory()
> +lcFac = LicenseChoiceFactory(license_factory=lFac)
> +bom = Bom()
> +
> +for i in range(2, len(sys.argv)):
> +    pkg_report = sys.argv[1] + sys.argv[i] + ".yaml"
> +    with open(pkg_report, 'r') as file:
> +        content = file.read()
> +        name_ = re.search("name: \'(.+)\'", content).group(1)
> +        version_ = re.search("version: \'(.+)\'", content).group(1)
> +
> +        # First see if we have a full CPE specified, then use that
> +        cpe_match = re.search("cpe: \'(.+)\'", content)
> +        cpe_ = None
> +        if cpe_match is not None:
> +            cpe_ = cpe_match.group(1)
> +        else:
> +            # See if we have the individual components
> +            cpe_vendor_match = re.search("cpe_vendor: \'(.+)\'", content)
> +            cpe_product_match = re.search("cpe_product: \'(.+)\'", content)
> +            cpe_version_match = re.search("cpe_version: \'(.+)\'", content)
> +            if cpe_vendor_match is not None and cpe_product_match is not None and cpe_version_match is not None:
> +                cpe_ = "cpe:2.3:a:{vendor}:{product}:{version}:*:*:*:*:*:*:*".format(vendor=cpe_vendor_match.group(1),
> +                                                                                     product=cpe_product_match.group(1),
> +                                                                                     version=cpe_version_match.group(1))
> +
> +        if cpe_ is not None:
> +            # We have a CPE, let's validate it. Regex from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd
> +            if not re.fullmatch("cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;"
> +                                "<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)"
> +                                "|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;<=>@\[\]\^`\{\|}~]"
> +                                "))+(\?*|\*?))|[\*\-])){4}", cpe_):
> +                raise ValueError("Constructed CPE is not valid: {cpe}".format(cpe=cpe_))
> +
> +        licenses_ = re.search("licenses: \'(.+)\'", content).group(1)
> +        comp = Component(
> +            name=name_,
> +            version=version_,
> +            cpe=cpe_,
> +            licenses=[lcFac.make_with_license(licenses_)],
> +            bom_ref=name_ + "@" + version_
> +        )
> +        bom.components.add(comp)
> +
> +serializedJSON = JsonV1Dot4(bom).output_as_string()
> +print(serializedJSON)
> --
> 2.25.1
>
>

Re: [ptxdist] [PATCH] RFC: sbom_report: Add support

[Simon Falsig]

> -----Original Message-----
> From: Bruno Thomsen <bruno.thomsen@gmail.com>
> Sent: Saturday, October 21, 2023 15:52
> Subject: Re: [ptxdist] [PATCH] RFC: sbom_report: Add support
> 
> Den man. 18. sep. 2023 kl. 16.34 skrev Simon Falsig <sfalsig@verity.net>:
> >
> > From: Simon Falsig <sfalsig@verity.ch>
> >
> > This provides support for building SBOMs in CycloneDX format.
> >
> > A target is added alongside the other reports, that (based on the
> > fast-bsp-report) extracts name, version, cpe and license of each
> > target package, and puts these into a final sbom-report in
> > CycloneDX/JSON format.
> >
> > This requires a working Python3 setup with the cyclonedx-bom package
> > installed.
> 
> Hi Simon,
> 
> I have tested this together with GitLab Dependency Scanning in Ultimate
> SaaS, and it seems to be working well.
> 
> Tested-by: Bruno Thomsen <bruno.thomsen@gmail.com>
> 
> .gitlab-ci.yml example snippet:
> 
> -------------8<-------------
> 
> ptxdist sbom:
>   stage: build
>   script:
>     - cd ptxdist
>     - ./p sbom-report
>   artifacts:
>     reports:
>       dependency_scanning: <platform-dir>/release/sbom-report.json
> 
> -------------8<-------------
> 
> Thanks for working on this.
> 
> /Bruno
> 

Thanks, Bruno!

I've sadly not had time to work more on this lately, but am hoping to pick it up again later in November.

A bit stuck on how to properly provide / ensure that the needed Python cyclonedx libraries are available on the system. If anyone has any suggestions, I'd be happy to hear them :)

Best regards,
Simon

Re: [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages

[Christian Melki]

On 9/13/23 18:05, Simon Falsig wrote:
> From: Simon Falsig <sfalsig@verity.ch>
> 
> If a package specifies a CPE, this is extracted into the fast report for
> that package. If no CPE is specified, then no value is added.
> 
> The CPE (Common Platform Enumerator) allows matching CVEs to specific
> packages, and see if these apply to a specific deployment.

Hi Simon.

I think this is a good thing going forward, but some minor nag.
My objection would be that sticking full versioned CPE strings straight
into the .make as an only-source just creates clutter.
As an full CPE override, absolutely though.

I suggest that some basic CPE modelling should be done by ptxdist, with
possibly trivial hinting or nameing in the .make, with complete
overrides as a last resort. That way ptxdist could start by filling most
stuff and people could override on demand.

I'd primarily poke the vendor:product tuple. Maybe ptxdist could do
packagename:packagename as default. If you specify the smaller override
it could be something like APPL_CPE_VENDOR and APPL_CPE_PRODUCT. Here
you could use * or other strings. Overriding any of them or both.
APPL_CPE would serve as the full override.

That could help in hiding CPE format or other usages (subject to
changes) in a lot of places. Hopefully, most packages won't require
extra information to match.

Regards,
Christian

> ---
>  rules/post/ptxd_make_world_common.make | 1 +
>  scripts/lib/ptxd_make_world_report.sh  | 1 +
>  2 files changed, 2 insertions(+)
> 
> diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make
> index 08120607a..6c646fb16 100644
> --- a/rules/post/ptxd_make_world_common.make
> +++ b/rules/post/ptxd_make_world_common.make
> @@ -78,6 +78,7 @@ world/env/impl = \
>  	pkg_PKG="$(call ptx/escape,$(1))"					\
>  	pkg_pkg="$(call ptx/escape,$($(1)))"					\
>  	pkg_version="$(call ptx/escape,$($(1)_VERSION))"			\
> +	pkg_cpe="$(call ptx/escape,$($(1)_CPE))"			\
>  	pkg_config="$(call ptx/escape,$($(1)_CONFIG))"				\
>  	pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))"			\
>  	pkg_path="$(call ptx/escape,$($(1)_PATH))"				\
> diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh
> index dbdae5736..2bfe4c201 100644
> --- a/scripts/lib/ptxd_make_world_report.sh
> +++ b/scripts/lib/ptxd_make_world_report.sh
> @@ -39,6 +39,7 @@ ptxd_make_world_report_yaml() {
>      do_list "rundeps:" "${pkg_run_deps}"
>      do_echo "config:" "${pkg_config}"
>      do_echo "version:" "${pkg_version}"
> +    do_echo "cpe:" "${pkg_cpe}"
>      do_list "url:" "${pkg_url}"
>      do_echo "md5:" "${pkg_md5}"
>      do_echo "source:" "${pkg_src}"

Re: [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages

[Simon Falsig]

Hi Christian,

> From: Christian Melki <christian.melki@t2data.com>
> Sent: Wednesday, September 13, 2023 23:17
> 
> On 9/13/23 18:05, Simon Falsig wrote:
> > From: Simon Falsig <sfalsig@verity.ch>
> >
> > If a package specifies a CPE, this is extracted into the fast report
> > for that package. If no CPE is specified, then no value is added.
> >
> > The CPE (Common Platform Enumerator) allows matching CVEs to specific
> > packages, and see if these apply to a specific deployment.
> 
> Hi Simon.
> 
> I think this is a good thing going forward, but some minor nag.
> My objection would be that sticking full versioned CPE strings straight
> into the .make as an only-source just creates clutter.
> As an full CPE override, absolutely though.
> 
> I suggest that some basic CPE modelling should be done by ptxdist, with
> possibly trivial hinting or nameing in the .make, with complete overrides
> as a last resort. That way ptxdist could start by filling most stuff and
> people could override on demand.
> 
> I'd primarily poke the vendor:product tuple. Maybe ptxdist could do
> packagename:packagename as default. If you specify the smaller override it
> could be something like APPL_CPE_VENDOR and APPL_CPE_PRODUCT. Here you
> could use * or other strings. Overriding any of them or both.
> APPL_CPE would serve as the full override.
> 
> That could help in hiding CPE format or other usages (subject to
> changes) in a lot of places. Hopefully, most packages won't require extra
> information to match.
> 
> Regards,
> Christian
> 

I fully agree that the complete CPE string in the makefile is not desirable.
When I was preparing the patch I was considering other ways, but in the end
decided to just start with the simplest, to avoid implementing too complex
logic before the first review...

Also, I was struggling to find the right tradeoff between granularity and
user-friendliness. I guess requiring either none or both of _CPE_VENDOR and
_CPE_PRODUCT could work (to reduce the risk of wrong CPEs due to incorrect
generic defaults - so only generate a CPE if both exist) - and then allowing
a full override if _CPE is specified.
In most cases that should be enough (for now at least), I think. I'll have
a look at an implementation one of the next days.

Thanks for the input!
- Simon

[ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages

[Simon Falsig]

From: Simon Falsig <sfalsig@verity.ch>

If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is
extracted into the fast report for that package. If no CPE is
specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is
added.

By default, the existing VERSION is used, but can be overridden with
CPE_VERSION.

Constructed CPEs are validated against the official CPE regex.

The CPE (Common Platform Enumerator) allows matching CVEs to specific
packages, and see if these apply to a specific deployment.
---
 rules/post/ptxd_make_world_common.make |  4 ++++
 scripts/lib/ptxd_make_world_report.sh  | 29 ++++++++++++++++++++++++++
 2 files changed, 33 insertions(+)

diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make
index 08120607a..0804f0b81 100644
--- a/rules/post/ptxd_make_world_common.make
+++ b/rules/post/ptxd_make_world_common.make
@@ -78,6 +78,10 @@ world/env/impl = \
 	pkg_PKG="$(call ptx/escape,$(1))"					\
 	pkg_pkg="$(call ptx/escape,$($(1)))"					\
 	pkg_version="$(call ptx/escape,$($(1)_VERSION))"			\
+	pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))"			\
+	pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))"		\
+	pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))"		\
+	pkg_cpe="$(call ptx/escape,$($(1)_CPE))"				\
 	pkg_config="$(call ptx/escape,$($(1)_CONFIG))"				\
 	pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))"			\
 	pkg_path="$(call ptx/escape,$($(1)_PATH))"				\
diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh
index dbdae5736..11f17b405 100644
--- a/scripts/lib/ptxd_make_world_report.sh
+++ b/scripts/lib/ptxd_make_world_report.sh
@@ -31,6 +31,30 @@ ptxd_make_world_report_yaml() {
 	    awk "BEGIN { RS=\" \" } { if (\$1) print \"- '\" \$1 \"'\" }" <<<"${2}"
 	fi
     }
+    do_build_cpe() {
+        prefix="${1}"
+        cpe="${2}"
+        vendor="${3}"
+        product="${4}"
+        version="${5}"
+        if [ -n "${cpe}" ]; then
+            # If a cpe is fully specified, then use that
+            :
+        elif [ -n "${vendor}" -a -n "${product}" -a -n "${version}" ]; then
+            # Otherwise, if we have vendor, product and version, then build a CPE2.3 string from it
+            cpe="cpe:2.3:a:${vendor}:${product}:${version}:*:*:*:*:*:*:*"
+        fi
+        if [ -n "$cpe" ]; then
+            # Validate the resulting CPE string
+            # Regex taken from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd
+            if echo "$cpe" | grep -Eq 'cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&'\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&''\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){4}'; then
+                echo "${prefix} ${cpe}"
+            else
+                >&2 echo "Error! $cpe is not valid CPE format string"
+                return 1
+            fi
+        fi
+    }
     do_echo "name:" "${pkg_label}"
     do_echo "rulefile:" "${pkg_makefile}"
     do_list "extra-rulefiles:" "${pkg_extra_makefiles}"
@@ -39,6 +63,11 @@ ptxd_make_world_report_yaml() {
     do_list "rundeps:" "${pkg_run_deps}"
     do_echo "config:" "${pkg_config}"
     do_echo "version:" "${pkg_version}"
+    if [ ! -n "${pkg_cpe_version}" ]; then
+    # Default to using pkg_version for the CPE string, unless _CPE_VERSION is explicitly specified
+    pkg_cpe_version="${pkg_version}";
+    fi
+    do_build_cpe "cpe:" "${pkg_cpe}" "${pkg_cpe_vendor}" "${pkg_cpe_product}" "${pkg_cpe_version}"
     do_list "url:" "${pkg_url}"
     do_echo "md5:" "${pkg_md5}"
     do_echo "source:" "${pkg_src}"
-- 
2.25.1

Re: [ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages

[Michael Olbrich]

On Fri, Sep 15, 2023 at 12:14:30PM +0200, Simon Falsig wrote:
> From: Simon Falsig <sfalsig@verity.ch>
> 
> If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is
> extracted into the fast report for that package. If no CPE is
> specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is
> added.
> 
> By default, the existing VERSION is used, but can be overridden with
> CPE_VERSION.
> 
> Constructed CPEs are validated against the official CPE regex.
> 
> The CPE (Common Platform Enumerator) allows matching CVEs to specific
> packages, and see if these apply to a specific deployment.
> ---
>  rules/post/ptxd_make_world_common.make |  4 ++++
>  scripts/lib/ptxd_make_world_report.sh  | 29 ++++++++++++++++++++++++++
>  2 files changed, 33 insertions(+)
> 
> diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make
> index 08120607a..0804f0b81 100644
> --- a/rules/post/ptxd_make_world_common.make
> +++ b/rules/post/ptxd_make_world_common.make
> @@ -78,6 +78,10 @@ world/env/impl = \
>  	pkg_PKG="$(call ptx/escape,$(1))"					\
>  	pkg_pkg="$(call ptx/escape,$($(1)))"					\
>  	pkg_version="$(call ptx/escape,$($(1)_VERSION))"			\
> +	pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))"			\
> +	pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))"		\
> +	pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))"		\
> +	pkg_cpe="$(call ptx/escape,$($(1)_CPE))"				\
>  	pkg_config="$(call ptx/escape,$($(1)_CONFIG))"				\
>  	pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))"			\
>  	pkg_path="$(call ptx/escape,$($(1)_PATH))"				\
> diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh
> index dbdae5736..11f17b405 100644
> --- a/scripts/lib/ptxd_make_world_report.sh
> +++ b/scripts/lib/ptxd_make_world_report.sh
> @@ -31,6 +31,30 @@ ptxd_make_world_report_yaml() {
>  	    awk "BEGIN { RS=\" \" } { if (\$1) print \"- '\" \$1 \"'\" }" <<<"${2}"
>  	fi
>      }
> +    do_build_cpe() {
> +        prefix="${1}"
> +        cpe="${2}"
> +        vendor="${3}"
> +        product="${4}"
> +        version="${5}"
> +        if [ -n "${cpe}" ]; then
> +            # If a cpe is fully specified, then use that
> +            :
> +        elif [ -n "${vendor}" -a -n "${product}" -a -n "${version}" ]; then
> +            # Otherwise, if we have vendor, product and version, then build a CPE2.3 string from it
> +            cpe="cpe:2.3:a:${vendor}:${product}:${version}:*:*:*:*:*:*:*"
> +        fi

Hmmm, I think we should preserve the original data in the report. Building
the cpe string should happen in the SBOM script. So:

  cpe: ....

or:

  cpe-vendor: ...
  cpe-product: ...

and maybe:

  cpe-version: ...

> +        if [ -n "$cpe" ]; then
> +            # Validate the resulting CPE string
> +            # Regex taken from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd
> +            if echo "$cpe" | grep -Eq 'cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&'\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&''\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){4}'; then
> +                echo "${prefix} ${cpe}"
> +            else
> +                >&2 echo "Error! $cpe is not valid CPE format string"
> +                return 1
> +            fi

Hmmm, I'm not sure where the validation should take place. Here or the SBOM
script. I don't mind either way.

FYI, your indention is wrong. Please check the rest of the script. I know
the style is a bit strange, but lets keep things consistent.

Michael

> +        fi
> +    }
>      do_echo "name:" "${pkg_label}"
>      do_echo "rulefile:" "${pkg_makefile}"
>      do_list "extra-rulefiles:" "${pkg_extra_makefiles}"
> @@ -39,6 +63,11 @@ ptxd_make_world_report_yaml() {
>      do_list "rundeps:" "${pkg_run_deps}"
>      do_echo "config:" "${pkg_config}"
>      do_echo "version:" "${pkg_version}"
> +    if [ ! -n "${pkg_cpe_version}" ]; then
> +    # Default to using pkg_version for the CPE string, unless _CPE_VERSION is explicitly specified
> +    pkg_cpe_version="${pkg_version}";
> +    fi
> +    do_build_cpe "cpe:" "${pkg_cpe}" "${pkg_cpe_vendor}" "${pkg_cpe_product}" "${pkg_cpe_version}"
>      do_list "url:" "${pkg_url}"
>      do_echo "md5:" "${pkg_md5}"
>      do_echo "source:" "${pkg_src}"
> -- 
> 2.25.1
> 
> 
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

[ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages

[Simon Falsig]

From: Simon Falsig <sfalsig@verity.ch>

If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is
extracted into the fast report for that package. If no CPE is
specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is
added.

By default, the existing VERSION is used, but can be overridden with
CPE_VERSION.

Constructed CPEs are validated against the official CPE regex.

The CPE (Common Platform Enumerator) allows matching CVEs to specific
packages, and see if these apply to a specific deployment.
---
 rules/post/ptxd_make_world_common.make | 4 ++++
 scripts/lib/ptxd_make_world_report.sh  | 9 +++++++++
 2 files changed, 13 insertions(+)

diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make
index 08120607a..0804f0b81 100644
--- a/rules/post/ptxd_make_world_common.make
+++ b/rules/post/ptxd_make_world_common.make
@@ -78,6 +78,10 @@ world/env/impl = \
 	pkg_PKG="$(call ptx/escape,$(1))"					\
 	pkg_pkg="$(call ptx/escape,$($(1)))"					\
 	pkg_version="$(call ptx/escape,$($(1)_VERSION))"			\
+	pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))"			\
+	pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))"		\
+	pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))"		\
+	pkg_cpe="$(call ptx/escape,$($(1)_CPE))"				\
 	pkg_config="$(call ptx/escape,$($(1)_CONFIG))"				\
 	pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))"			\
 	pkg_path="$(call ptx/escape,$($(1)_PATH))"				\
diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh
index dbdae5736..dea25635b 100644
--- a/scripts/lib/ptxd_make_world_report.sh
+++ b/scripts/lib/ptxd_make_world_report.sh
@@ -39,6 +39,15 @@ ptxd_make_world_report_yaml() {
     do_list "rundeps:" "${pkg_run_deps}"
     do_echo "config:" "${pkg_config}"
     do_echo "version:" "${pkg_version}"
+    if [ ! -n "${pkg_cpe_version}" -a ! -n "${pkg_cpe}" ]; then
+	# Default to using pkg_version for the CPE string, unless _CPE_VERSION or _CPE are explicitly
+	# specified. In the case of the latter, there's no need to keep track of the version separately.
+	pkg_cpe_version="${pkg_version}"
+    fi
+    do_echo "cpe:" "${pkg_cpe}"
+    do_echo "cpe_vendor:" "${pkg_cpe_vendor}"
+    do_echo "cpe_product:" "${pkg_cpe_product}"
+    do_echo "cpe_version:" "${pkg_cpe_version}"
     do_list "url:" "${pkg_url}"
     do_echo "md5:" "${pkg_md5}"
     do_echo "source:" "${pkg_src}"
-- 
2.25.1

Re: [ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages

[Simon Falsig]

Hi Michael,

> From: ptxdist <ptxdist-bounces@pengutronix.de> On Behalf Of Michael
> Olbrich
> Sent: Friday, September 15, 2023 12:39
>
> On Fri, Sep 15, 2023 at 12:14:30PM +0200, Simon Falsig wrote:
> > From: Simon Falsig <sfalsig@verity.ch>
> >
> > If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is
> > extracted into the fast report for that package. If no CPE is
> > specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is
> > added.
> >
> > By default, the existing VERSION is used, but can be overridden with
> > CPE_VERSION.
> >
> > Constructed CPEs are validated against the official CPE regex.
> >
> > The CPE (Common Platform Enumerator) allows matching CVEs to specific
> > packages, and see if these apply to a specific deployment.
> > ---
> >  rules/post/ptxd_make_world_common.make |  4 ++++
> > scripts/lib/ptxd_make_world_report.sh  | 29 ++++++++++++++++++++++++++
> >  2 files changed, 33 insertions(+)
> >
> > diff --git a/rules/post/ptxd_make_world_common.make
> > b/rules/post/ptxd_make_world_common.make
> > index 08120607a..0804f0b81 100644
> > --- a/rules/post/ptxd_make_world_common.make
> > +++ b/rules/post/ptxd_make_world_common.make
> > @@ -78,6 +78,10 @@ world/env/impl = \
> >     pkg_PKG="$(call ptx/escape,$(1))"                                       \
> >     pkg_pkg="$(call ptx/escape,$($(1)))"                                    \
> >     pkg_version="$(call ptx/escape,$($(1)_VERSION))"                        \
> > +   pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))"
>       \
> > +   pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))"                \
> > +   pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))"                \
> > +   pkg_cpe="$(call ptx/escape,$($(1)_CPE))"                                \
> >     pkg_config="$(call ptx/escape,$($(1)_CONFIG))"                          \
> >     pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))"
>       \
> >     pkg_path="$(call ptx/escape,$($(1)_PATH))"                              \
> > diff --git a/scripts/lib/ptxd_make_world_report.sh
> > b/scripts/lib/ptxd_make_world_report.sh
> > index dbdae5736..11f17b405 100644
> > --- a/scripts/lib/ptxd_make_world_report.sh
> > +++ b/scripts/lib/ptxd_make_world_report.sh
> > @@ -31,6 +31,30 @@ ptxd_make_world_report_yaml() {
> >         awk "BEGIN { RS=\" \" } { if (\$1) print \"- '\" \$1 \"'\" }"
> <<<"${2}"
> >     fi
> >      }
> > +    do_build_cpe() {
> > +        prefix="${1}"
> > +        cpe="${2}"
> > +        vendor="${3}"
> > +        product="${4}"
> > +        version="${5}"
> > +        if [ -n "${cpe}" ]; then
> > +            # If a cpe is fully specified, then use that
> > +            :
> > +        elif [ -n "${vendor}" -a -n "${product}" -a -n "${version}" ];
> then
> > +            # Otherwise, if we have vendor, product and version, then
> build a CPE2.3 string from it
> > +
> cpe="cpe:2.3:a:${vendor}:${product}:${version}:*:*:*:*:*:*:*"
> > +        fi
>
> Hmmm, I think we should preserve the original data in the report. Building
> the cpe string should happen in the SBOM script. So:
>
>   cpe: ....
>
> or:
>
>   cpe-vendor: ...
>   cpe-product: ...
>
> and maybe:
>
>   cpe-version: ...
>

Makes sense - changed.

> > +        if [ -n "$cpe" ]; then
> > +            # Validate the resulting CPE string
> > +            # Regex taken from:
> https://csrc.nis/
> t.gov%2Fschema%2Fcpe%2F2.3%2Fcpe-
> naming_2.3.xsd&data=05%7C01%7Csfalsig%40verity.net%7Cc9dea8f344e64c0f2a570
> 8dbb5daf98c%7C06487c727d884632bf56071603defa0a%7C1%7C0%7C63830372431499528
> 3%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h
> aWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RQxWOHS3iGwu%2BUXaR%2Bc1FZRzo4rHk
> XX8U4fjSmWtalQ%3D&reserved=0
> > +            if echo "$cpe" | grep -Eq 'cpe:2\.3:[aho\*\-
> ](:(((\?*|\*?)([a-zA-Z0-9\-
> \._]|(\\[\\\*\?!"#$$%&'\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-
> ])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[\*\-
> ]))(:(((\?*|\*?)([a-zA-Z0-9\-
> \._]|(\\[\\\*\?!"#$$%&''\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-
> ])){4}'; then
> > +                echo "${prefix} ${cpe}"
> > +            else
> > +                >&2 echo "Error! $cpe is not valid CPE format string"
> > +                return 1
> > +            fi
>
> Hmmm, I'm not sure where the validation should take place. Here or the
> SBOM script. I don't mind either way.
>

I've switched to do the validation in the SBOM script. Matches better with
having the original cpe_... values in the individual reports, then building
and validating the complete CPE when doing the full report.

> FYI, your indention is wrong. Please check the rest of the script. I know
> the style is a bit strange, but lets keep things consistent.

Argh, sorry. I've tried fixing things up now - hope it's good!

>
> Michael

Thanks for the input!
 - Simon