* [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages @ 2023-09-13 16:05 Simon Falsig 2023-09-13 16:05 ` [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages Simon Falsig ` (2 more replies) 0 siblings, 3 replies; 13+ messages in thread From: Simon Falsig @ 2023-09-13 16:05 UTC (permalink / raw) To: ptxdist; +Cc: Simon Falsig From: Simon Falsig <sfalsig@verity.ch> If a package specifies a CPE, this is extracted into the fast report for that package. If no CPE is specified, then no value is added. The CPE (Common Platform Enumerator) allows matching CVEs to specific packages, and see if these apply to a specific deployment. --- rules/post/ptxd_make_world_common.make | 1 + scripts/lib/ptxd_make_world_report.sh | 1 + 2 files changed, 2 insertions(+) diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make index 08120607a..6c646fb16 100644 --- a/rules/post/ptxd_make_world_common.make +++ b/rules/post/ptxd_make_world_common.make @@ -78,6 +78,7 @@ world/env/impl = \ pkg_PKG="$(call ptx/escape,$(1))" \ pkg_pkg="$(call ptx/escape,$($(1)))" \ pkg_version="$(call ptx/escape,$($(1)_VERSION))" \ + pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \ pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \ pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))" \ pkg_path="$(call ptx/escape,$($(1)_PATH))" \ diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh index dbdae5736..2bfe4c201 100644 --- a/scripts/lib/ptxd_make_world_report.sh +++ b/scripts/lib/ptxd_make_world_report.sh @@ -39,6 +39,7 @@ ptxd_make_world_report_yaml() { do_list "rundeps:" "${pkg_run_deps}" do_echo "config:" "${pkg_config}" do_echo "version:" "${pkg_version}" + do_echo "cpe:" "${pkg_cpe}" do_list "url:" "${pkg_url}" do_echo "md5:" "${pkg_md5}" do_echo "source:" "${pkg_src}" -- 2.25.1 ^ permalink raw reply [flat|nested] 13+ messages in thread
* [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages 2023-09-13 16:05 [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Simon Falsig @ 2023-09-13 16:05 ` Simon Falsig 2023-09-15 10:15 ` [ptxdist] [PATCH] " Simon Falsig 2023-09-13 16:05 ` [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support Simon Falsig 2023-09-13 21:16 ` [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Christian Melki 2 siblings, 1 reply; 13+ messages in thread From: Simon Falsig @ 2023-09-13 16:05 UTC (permalink / raw) To: ptxdist; +Cc: Simon Falsig From: Simon Falsig <sfalsig@verity.ch> Just to see how this could look for a handful of packages. Note that all of these have a different way of specifying the vendor ID (one is $PACKAGE_project, one is just $PACKAGE, one is something completely different). --- rules/acl.make | 1 + rules/busybox.make | 1 + rules/wpa_supplicant.make | 1 + 3 files changed, 3 insertions(+) diff --git a/rules/acl.make b/rules/acl.make index daa5837c9..10769681b 100644 --- a/rules/acl.make +++ b/rules/acl.make @@ -21,6 +21,7 @@ ACL_SUFFIX := tar.gz ACL_URL := http://download.savannah.gnu.org/releases/acl/$(ACL).$(ACL_SUFFIX) ACL_SOURCE := $(SRCDIR)/$(ACL).$(ACL_SUFFIX) ACL_DIR := $(BUILDDIR)/$(ACL) +ACL_CPE := cpe:2.3:a:acl_project:acl:$(ACL_VERSION):*:*:*:*:*:*:* ACL_LICENSE := LGPL-2.1-or-later ACL_LICENSE_FILES:= file://doc/COPYING.LGPL;md5=9e9a206917f8af112da634ce3ab41764 ifdef PTXCONF_ACL_TOOLS diff --git a/rules/busybox.make b/rules/busybox.make index 413ad735c..eb8721229 100644 --- a/rules/busybox.make +++ b/rules/busybox.make @@ -22,6 +22,7 @@ BUSYBOX_URL := https://www.busybox.net/downloads/$(BUSYBOX).$(BUSYBOX_SUFFIX) BUSYBOX_SOURCE := $(SRCDIR)/$(BUSYBOX).$(BUSYBOX_SUFFIX) BUSYBOX_DIR := $(BUILDDIR)/$(BUSYBOX) BUSYBOX_KCONFIG := $(BUSYBOX_DIR)/Config.in +BUSYBOX_CPE := cpe:2.3:a:busybox:busybox:$(BUSYBOX_VERSION):*:*:*:*:*:*:* BUSYBOX_LICENSE := GPL-2.0-only BUSYBOX_LICENSE_FILES := file://LICENSE;md5=de10de48642ab74318e893a61105afbb diff --git a/rules/wpa_supplicant.make b/rules/wpa_supplicant.make index 55d43b9b2..c0bf54f25 100644 --- a/rules/wpa_supplicant.make +++ b/rules/wpa_supplicant.make @@ -26,6 +26,7 @@ WPA_SUPPLICANT_SUBDIR := $(WPA_SUPPLICANT_NAME) # Use '=' to delay $(shell ...) calls until this is needed WPA_SUPPLICANT_CONFIG = $(call ptx/get-alternative, config/wpasupplicant, defconfig) WPA_SUPPLICANT_DOTCONFIG:= $(BUILDDIR)/$(WPA_SUPPLICANT)/$(WPA_SUPPLICANT_SUBDIR)/.config +WPA_SUPPLICANT_CPE := cpe:2.3:a:w1.fi:wpa_supplicant:$(WPA_SUPPLICANT_VERSION):*:*:*:*:*:*:* WPA_SUPPLICANT_LICENSE := GPL-2.0-only # ---------------------------------------------------------------------------- -- 2.25.1 ^ permalink raw reply [flat|nested] 13+ messages in thread
* [ptxdist] [PATCH] RFC: Add CPE for a few packages 2023-09-13 16:05 ` [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages Simon Falsig @ 2023-09-15 10:15 ` Simon Falsig 0 siblings, 0 replies; 13+ messages in thread From: Simon Falsig @ 2023-09-15 10:15 UTC (permalink / raw) To: ptxdist; +Cc: Simon Falsig From: Simon Falsig <sfalsig@verity.ch> Just to see how this could look for a handful of packages. Note that all of these have a different way of specifying the vendor ID (one is $PACKAGE_project, one is just $PACKAGE, one is something completely different). --- rules/acl.make | 1 + rules/busybox.make | 2 ++ rules/wpa_supplicant.make | 3 +++ 3 files changed, 6 insertions(+) diff --git a/rules/acl.make b/rules/acl.make index daa5837c9..10769681b 100644 --- a/rules/acl.make +++ b/rules/acl.make @@ -21,6 +21,7 @@ ACL_SUFFIX := tar.gz ACL_URL := http://download.savannah.gnu.org/releases/acl/$(ACL).$(ACL_SUFFIX) ACL_SOURCE := $(SRCDIR)/$(ACL).$(ACL_SUFFIX) ACL_DIR := $(BUILDDIR)/$(ACL) +ACL_CPE := cpe:2.3:a:acl_project:acl:$(ACL_VERSION):*:*:*:*:*:*:* ACL_LICENSE := LGPL-2.1-or-later ACL_LICENSE_FILES:= file://doc/COPYING.LGPL;md5=9e9a206917f8af112da634ce3ab41764 ifdef PTXCONF_ACL_TOOLS diff --git a/rules/busybox.make b/rules/busybox.make index 413ad735c..b3ed2ff7a 100644 --- a/rules/busybox.make +++ b/rules/busybox.make @@ -22,6 +22,8 @@ BUSYBOX_URL := https://www.busybox.net/downloads/$(BUSYBOX).$(BUSYBOX_SUFFIX) BUSYBOX_SOURCE := $(SRCDIR)/$(BUSYBOX).$(BUSYBOX_SUFFIX) BUSYBOX_DIR := $(BUILDDIR)/$(BUSYBOX) BUSYBOX_KCONFIG := $(BUSYBOX_DIR)/Config.in +BUSYBOX_CPE_VENDOR := busybox +BUSYBOX_CPE_PRODUCT := busybox BUSYBOX_LICENSE := GPL-2.0-only BUSYBOX_LICENSE_FILES := file://LICENSE;md5=de10de48642ab74318e893a61105afbb diff --git a/rules/wpa_supplicant.make b/rules/wpa_supplicant.make index 55d43b9b2..2f4d9c607 100644 --- a/rules/wpa_supplicant.make +++ b/rules/wpa_supplicant.make @@ -26,6 +26,9 @@ WPA_SUPPLICANT_SUBDIR := $(WPA_SUPPLICANT_NAME) # Use '=' to delay $(shell ...) calls until this is needed WPA_SUPPLICANT_CONFIG = $(call ptx/get-alternative, config/wpasupplicant, defconfig) WPA_SUPPLICANT_DOTCONFIG:= $(BUILDDIR)/$(WPA_SUPPLICANT)/$(WPA_SUPPLICANT_SUBDIR)/.config +WPA_SUPPLICANT_CPE_VENDOR := w1.fi +WPA_SUPPLICANT_CPE_PRODUCT := wpa_supplicant +WPA_SUPPLICANT_CPE_VERSION := $(WPA_SUPPLICANT_VERSION) WPA_SUPPLICANT_LICENSE := GPL-2.0-only # ---------------------------------------------------------------------------- -- 2.25.1 ^ permalink raw reply [flat|nested] 13+ messages in thread
* [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support 2023-09-13 16:05 [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Simon Falsig 2023-09-13 16:05 ` [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages Simon Falsig @ 2023-09-13 16:05 ` Simon Falsig 2023-09-18 14:33 ` [ptxdist] [PATCH] " Simon Falsig 2023-09-13 21:16 ` [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Christian Melki 2 siblings, 1 reply; 13+ messages in thread From: Simon Falsig @ 2023-09-13 16:05 UTC (permalink / raw) To: ptxdist; +Cc: Simon Falsig From: Simon Falsig <sfalsig@verity.ch> This provides support for building SBOMs in CycloneDX format. A target is added alongside the other reports, that (based on the fast-bsp-report) extracts name, version, cpe and license of each target package, and puts these into a final sbom-report in CycloneDX/JSON format. This requires a working Python3 setup with the cyclonedx-bom package installed. --- bin/ptxdist | 3 ++- rules/post/ptxd_make_report.make | 15 ++++++++++-- scripts/lib/ptxd_make_report.sh | 16 +++++++++++++ scripts/lib/ptxd_make_sbom_report.py | 35 ++++++++++++++++++++++++++++ 4 files changed, 66 insertions(+), 3 deletions(-) create mode 100644 scripts/lib/ptxd_make_sbom_report.py diff --git a/bin/ptxdist b/bin/ptxdist index dfb619cbd..15be851f5 100755 --- a/bin/ptxdist +++ b/bin/ptxdist @@ -780,6 +780,7 @@ Misc: full-bsp-report generate a yaml file that describes the BSP and all packages. More data but will build all packages if necessary. + sbom-report generate a CycloneDX json SBOM print <var> print the contents of a variable, in the way it is known by "make" printnext <var> assumes that the contents of <var> is another @@ -1807,7 +1808,7 @@ EOF ptxd_make_log export_src EXPORTDIR="${1}" exit ;; - fast-bsp-report|full-bsp-report) + fast-bsp-report|full-bsp-report|sbom-report) check_premake_compiler && ptxd_make_log "${cmd}" exit diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make index eecd2a577..ffa398c95 100644 --- a/rules/post/ptxd_make_report.make +++ b/rules/post/ptxd_make_report.make @@ -10,7 +10,9 @@ ptx/report-env = \ $(image/env) \ ptx_report_target="$(strip $(1))" \ ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \ - ptx_image_packages="$(IMAGE_PACKAGES)" + ptx_image_packages="$(IMAGE_PACKAGES)" \ + ptx_target_packages="$(PACKAGES)" + PHONY += full-bsp-report full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml @@ -26,13 +28,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \ @$(call ptx/report-env, $@) ptxd_make_full_bsp_report @$(call finish) + PHONY += fast-bsp-report fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml - $(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED))) @$(call targetinfo) @$(call ptx/report-env, $@) ptxd_make_fast_bsp_report @$(call finish) + +PHONY += sbom-report +sbom-report: $(RELEASEDIR)/sbom-report.json + +$(RELEASEDIR)/sbom-report.json: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES))) + @$(call targetinfo) + @$(call ptx/report-env, $@) ptxd_make_sbom_report + @$(call finish) + # vim: syntax=make diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh index a363ca5b3..e2da4c05f 100644 --- a/scripts/lib/ptxd_make_report.sh +++ b/scripts/lib/ptxd_make_report.sh @@ -144,3 +144,19 @@ ptxd_make_fast_bsp_report() { } export -f ptxd_make_fast_bsp_report +ptxd_make_sbom_report() { + local -a ptxd_reply + local pkg_lic pkg + + ptxd_make_layer_init || return + + echo "Generating $(ptxd_print_path "${ptx_report_target}") ..." + echo + + mkdir -p "$(dirname "${ptx_report_target}")" && + python3 ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report && + mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" || + ptxd_bailout "failed to create SBOM report" +} +export -f ptxd_make_sbom_report + diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py new file mode 100644 index 000000000..aecb4fae5 --- /dev/null +++ b/scripts/lib/ptxd_make_sbom_report.py @@ -0,0 +1,35 @@ +from cyclonedx.factory.license import LicenseFactory +from cyclonedx.factory.license import LicenseChoiceFactory +from cyclonedx.model.bom import Bom +from cyclonedx.model.component import Component +from cyclonedx.output.json import JsonV1Dot4 +import sys +import re + +lFac = LicenseFactory() +lcFac = LicenseChoiceFactory(license_factory=lFac) +bom = Bom() + +for i in range(2,len(sys.argv)): + pkg_report = sys.argv[1] + sys.argv[i] + ".yaml" + with open(pkg_report, 'r') as file: + content = file.read() + name_ = re.search("name: \'(.+)\'", content).group(1) + version_ = re.search("version: \'(.+)\'", content).group(1) + cpeMatch = re.search("cpe: \'(.+)\'", content) + cpe_ = None + if cpeMatch is not None: + cpe_ = cpeMatch.group(1) + licenses_ = re.search("licenses: \'(.+)\'", content).group(1) + comp = Component( + name = name_, + version = version_, + cpe = cpe_, + licenses = [lcFac.make_with_license(licenses_)], + bom_ref = name_ + "@" + version_ + ) + bom.components.add(comp) + +serializedJSON = JsonV1Dot4(bom).output_as_string() +print(serializedJSON) + -- 2.25.1 ^ permalink raw reply [flat|nested] 13+ messages in thread
* [ptxdist] [PATCH] RFC: sbom_report: Add support 2023-09-13 16:05 ` [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support Simon Falsig @ 2023-09-18 14:33 ` Simon Falsig 2023-10-21 13:52 ` Bruno Thomsen 0 siblings, 1 reply; 13+ messages in thread From: Simon Falsig @ 2023-09-18 14:33 UTC (permalink / raw) To: ptxdist; +Cc: Simon Falsig From: Simon Falsig <sfalsig@verity.ch> This provides support for building SBOMs in CycloneDX format. A target is added alongside the other reports, that (based on the fast-bsp-report) extracts name, version, cpe and license of each target package, and puts these into a final sbom-report in CycloneDX/JSON format. This requires a working Python3 setup with the cyclonedx-bom package installed. --- bin/ptxdist | 3 +- rules/post/ptxd_make_report.make | 15 ++++++-- scripts/lib/ptxd_make_report.sh | 16 +++++++++ scripts/lib/ptxd_make_sbom_report.py | 54 ++++++++++++++++++++++++++++ 4 files changed, 85 insertions(+), 3 deletions(-) create mode 100644 scripts/lib/ptxd_make_sbom_report.py diff --git a/bin/ptxdist b/bin/ptxdist index dfb619cbd..15be851f5 100755 --- a/bin/ptxdist +++ b/bin/ptxdist @@ -780,6 +780,7 @@ Misc: full-bsp-report generate a yaml file that describes the BSP and all packages. More data but will build all packages if necessary. + sbom-report generate a CycloneDX json SBOM print <var> print the contents of a variable, in the way it is known by "make" printnext <var> assumes that the contents of <var> is another @@ -1807,7 +1808,7 @@ EOF ptxd_make_log export_src EXPORTDIR="${1}" exit ;; - fast-bsp-report|full-bsp-report) + fast-bsp-report|full-bsp-report|sbom-report) check_premake_compiler && ptxd_make_log "${cmd}" exit diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make index eecd2a577..ffa398c95 100644 --- a/rules/post/ptxd_make_report.make +++ b/rules/post/ptxd_make_report.make @@ -10,7 +10,9 @@ ptx/report-env = \ $(image/env) \ ptx_report_target="$(strip $(1))" \ ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \ - ptx_image_packages="$(IMAGE_PACKAGES)" + ptx_image_packages="$(IMAGE_PACKAGES)" \ + ptx_target_packages="$(PACKAGES)" + PHONY += full-bsp-report full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml @@ -26,13 +28,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \ @$(call ptx/report-env, $@) ptxd_make_full_bsp_report @$(call finish) + PHONY += fast-bsp-report fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml - $(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED))) @$(call targetinfo) @$(call ptx/report-env, $@) ptxd_make_fast_bsp_report @$(call finish) + +PHONY += sbom-report +sbom-report: $(RELEASEDIR)/sbom-report.json + +$(RELEASEDIR)/sbom-report.json: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES))) + @$(call targetinfo) + @$(call ptx/report-env, $@) ptxd_make_sbom_report + @$(call finish) + # vim: syntax=make diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh index a363ca5b3..e2da4c05f 100644 --- a/scripts/lib/ptxd_make_report.sh +++ b/scripts/lib/ptxd_make_report.sh @@ -144,3 +144,19 @@ ptxd_make_fast_bsp_report() { } export -f ptxd_make_fast_bsp_report +ptxd_make_sbom_report() { + local -a ptxd_reply + local pkg_lic pkg + + ptxd_make_layer_init || return + + echo "Generating $(ptxd_print_path "${ptx_report_target}") ..." + echo + + mkdir -p "$(dirname "${ptx_report_target}")" && + python3 ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report && + mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" || + ptxd_bailout "failed to create SBOM report" +} +export -f ptxd_make_sbom_report + diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py new file mode 100644 index 000000000..cc6a6f703 --- /dev/null +++ b/scripts/lib/ptxd_make_sbom_report.py @@ -0,0 +1,54 @@ +from cyclonedx.factory.license import LicenseFactory +from cyclonedx.factory.license import LicenseChoiceFactory +from cyclonedx.model.bom import Bom +from cyclonedx.model.component import Component +from cyclonedx.output.json import JsonV1Dot4 +import sys +import re + +lFac = LicenseFactory() +lcFac = LicenseChoiceFactory(license_factory=lFac) +bom = Bom() + +for i in range(2, len(sys.argv)): + pkg_report = sys.argv[1] + sys.argv[i] + ".yaml" + with open(pkg_report, 'r') as file: + content = file.read() + name_ = re.search("name: \'(.+)\'", content).group(1) + version_ = re.search("version: \'(.+)\'", content).group(1) + + # First see if we have a full CPE specified, then use that + cpe_match = re.search("cpe: \'(.+)\'", content) + cpe_ = None + if cpe_match is not None: + cpe_ = cpe_match.group(1) + else: + # See if we have the individual components + cpe_vendor_match = re.search("cpe_vendor: \'(.+)\'", content) + cpe_product_match = re.search("cpe_product: \'(.+)\'", content) + cpe_version_match = re.search("cpe_version: \'(.+)\'", content) + if cpe_vendor_match is not None and cpe_product_match is not None and cpe_version_match is not None: + cpe_ = "cpe:2.3:a:{vendor}:{product}:{version}:*:*:*:*:*:*:*".format(vendor=cpe_vendor_match.group(1), + product=cpe_product_match.group(1), + version=cpe_version_match.group(1)) + + if cpe_ is not None: + # We have a CPE, let's validate it. Regex from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd + if not re.fullmatch("cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;" + "<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)" + "|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;<=>@\[\]\^`\{\|}~]" + "))+(\?*|\*?))|[\*\-])){4}", cpe_): + raise ValueError("Constructed CPE is not valid: {cpe}".format(cpe=cpe_)) + + licenses_ = re.search("licenses: \'(.+)\'", content).group(1) + comp = Component( + name=name_, + version=version_, + cpe=cpe_, + licenses=[lcFac.make_with_license(licenses_)], + bom_ref=name_ + "@" + version_ + ) + bom.components.add(comp) + +serializedJSON = JsonV1Dot4(bom).output_as_string() +print(serializedJSON) -- 2.25.1 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [ptxdist] [PATCH] RFC: sbom_report: Add support 2023-09-18 14:33 ` [ptxdist] [PATCH] " Simon Falsig @ 2023-10-21 13:52 ` Bruno Thomsen 2023-11-03 7:34 ` Simon Falsig 0 siblings, 1 reply; 13+ messages in thread From: Bruno Thomsen @ 2023-10-21 13:52 UTC (permalink / raw) To: ptxdist; +Cc: Simon Falsig Den man. 18. sep. 2023 kl. 16.34 skrev Simon Falsig <sfalsig@verity.net>: > > From: Simon Falsig <sfalsig@verity.ch> > > This provides support for building SBOMs in CycloneDX format. > > A target is added alongside the other reports, that (based on the > fast-bsp-report) extracts name, version, cpe and license of each target > package, and puts these into a final sbom-report in CycloneDX/JSON > format. > > This requires a working Python3 setup with the cyclonedx-bom package > installed. Hi Simon, I have tested this together with GitLab Dependency Scanning in Ultimate SaaS, and it seems to be working well. Tested-by: Bruno Thomsen <bruno.thomsen@gmail.com> .gitlab-ci.yml example snippet: -------------8<------------- ptxdist sbom: stage: build script: - cd ptxdist - ./p sbom-report artifacts: reports: dependency_scanning: <platform-dir>/release/sbom-report.json -------------8<------------- Thanks for working on this. /Bruno > --- > bin/ptxdist | 3 +- > rules/post/ptxd_make_report.make | 15 ++++++-- > scripts/lib/ptxd_make_report.sh | 16 +++++++++ > scripts/lib/ptxd_make_sbom_report.py | 54 ++++++++++++++++++++++++++++ > 4 files changed, 85 insertions(+), 3 deletions(-) > create mode 100644 scripts/lib/ptxd_make_sbom_report.py > > diff --git a/bin/ptxdist b/bin/ptxdist > index dfb619cbd..15be851f5 100755 > --- a/bin/ptxdist > +++ b/bin/ptxdist > @@ -780,6 +780,7 @@ Misc: > full-bsp-report generate a yaml file that describes the BSP and > all packages. More data but will build all > packages if necessary. > + sbom-report generate a CycloneDX json SBOM > print <var> print the contents of a variable, in the way > it is known by "make" > printnext <var> assumes that the contents of <var> is another > @@ -1807,7 +1808,7 @@ EOF > ptxd_make_log export_src EXPORTDIR="${1}" > exit > ;; > - fast-bsp-report|full-bsp-report) > + fast-bsp-report|full-bsp-report|sbom-report) > check_premake_compiler && > ptxd_make_log "${cmd}" > exit > diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make > index eecd2a577..ffa398c95 100644 > --- a/rules/post/ptxd_make_report.make > +++ b/rules/post/ptxd_make_report.make > @@ -10,7 +10,9 @@ ptx/report-env = \ > $(image/env) \ > ptx_report_target="$(strip $(1))" \ > ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \ > - ptx_image_packages="$(IMAGE_PACKAGES)" > + ptx_image_packages="$(IMAGE_PACKAGES)" \ > + ptx_target_packages="$(PACKAGES)" > + > > PHONY += full-bsp-report > full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml > @@ -26,13 +28,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \ > @$(call ptx/report-env, $@) ptxd_make_full_bsp_report > @$(call finish) > > + > PHONY += fast-bsp-report > fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml > > - > $(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED))) > @$(call targetinfo) > @$(call ptx/report-env, $@) ptxd_make_fast_bsp_report > @$(call finish) > > + > +PHONY += sbom-report > +sbom-report: $(RELEASEDIR)/sbom-report.json > + > +$(RELEASEDIR)/sbom-report.json: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES))) > + @$(call targetinfo) > + @$(call ptx/report-env, $@) ptxd_make_sbom_report > + @$(call finish) > + > # vim: syntax=make > diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh > index a363ca5b3..e2da4c05f 100644 > --- a/scripts/lib/ptxd_make_report.sh > +++ b/scripts/lib/ptxd_make_report.sh > @@ -144,3 +144,19 @@ ptxd_make_fast_bsp_report() { > } > export -f ptxd_make_fast_bsp_report > > +ptxd_make_sbom_report() { > + local -a ptxd_reply > + local pkg_lic pkg > + > + ptxd_make_layer_init || return > + > + echo "Generating $(ptxd_print_path "${ptx_report_target}") ..." > + echo > + > + mkdir -p "$(dirname "${ptx_report_target}")" && > + python3 ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report && > + mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" || > + ptxd_bailout "failed to create SBOM report" > +} > +export -f ptxd_make_sbom_report > + > diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py > new file mode 100644 > index 000000000..cc6a6f703 > --- /dev/null > +++ b/scripts/lib/ptxd_make_sbom_report.py > @@ -0,0 +1,54 @@ > +from cyclonedx.factory.license import LicenseFactory > +from cyclonedx.factory.license import LicenseChoiceFactory > +from cyclonedx.model.bom import Bom > +from cyclonedx.model.component import Component > +from cyclonedx.output.json import JsonV1Dot4 > +import sys > +import re > + > +lFac = LicenseFactory() > +lcFac = LicenseChoiceFactory(license_factory=lFac) > +bom = Bom() > + > +for i in range(2, len(sys.argv)): > + pkg_report = sys.argv[1] + sys.argv[i] + ".yaml" > + with open(pkg_report, 'r') as file: > + content = file.read() > + name_ = re.search("name: \'(.+)\'", content).group(1) > + version_ = re.search("version: \'(.+)\'", content).group(1) > + > + # First see if we have a full CPE specified, then use that > + cpe_match = re.search("cpe: \'(.+)\'", content) > + cpe_ = None > + if cpe_match is not None: > + cpe_ = cpe_match.group(1) > + else: > + # See if we have the individual components > + cpe_vendor_match = re.search("cpe_vendor: \'(.+)\'", content) > + cpe_product_match = re.search("cpe_product: \'(.+)\'", content) > + cpe_version_match = re.search("cpe_version: \'(.+)\'", content) > + if cpe_vendor_match is not None and cpe_product_match is not None and cpe_version_match is not None: > + cpe_ = "cpe:2.3:a:{vendor}:{product}:{version}:*:*:*:*:*:*:*".format(vendor=cpe_vendor_match.group(1), > + product=cpe_product_match.group(1), > + version=cpe_version_match.group(1)) > + > + if cpe_ is not None: > + # We have a CPE, let's validate it. Regex from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd > + if not re.fullmatch("cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;" > + "<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)" > + "|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;<=>@\[\]\^`\{\|}~]" > + "))+(\?*|\*?))|[\*\-])){4}", cpe_): > + raise ValueError("Constructed CPE is not valid: {cpe}".format(cpe=cpe_)) > + > + licenses_ = re.search("licenses: \'(.+)\'", content).group(1) > + comp = Component( > + name=name_, > + version=version_, > + cpe=cpe_, > + licenses=[lcFac.make_with_license(licenses_)], > + bom_ref=name_ + "@" + version_ > + ) > + bom.components.add(comp) > + > +serializedJSON = JsonV1Dot4(bom).output_as_string() > +print(serializedJSON) > -- > 2.25.1 > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [ptxdist] [PATCH] RFC: sbom_report: Add support 2023-10-21 13:52 ` Bruno Thomsen @ 2023-11-03 7:34 ` Simon Falsig 0 siblings, 0 replies; 13+ messages in thread From: Simon Falsig @ 2023-11-03 7:34 UTC (permalink / raw) To: Bruno Thomsen, ptxdist > -----Original Message----- > From: Bruno Thomsen <bruno.thomsen@gmail.com> > Sent: Saturday, October 21, 2023 15:52 > Subject: Re: [ptxdist] [PATCH] RFC: sbom_report: Add support > > Den man. 18. sep. 2023 kl. 16.34 skrev Simon Falsig <sfalsig@verity.net>: > > > > From: Simon Falsig <sfalsig@verity.ch> > > > > This provides support for building SBOMs in CycloneDX format. > > > > A target is added alongside the other reports, that (based on the > > fast-bsp-report) extracts name, version, cpe and license of each > > target package, and puts these into a final sbom-report in > > CycloneDX/JSON format. > > > > This requires a working Python3 setup with the cyclonedx-bom package > > installed. > > Hi Simon, > > I have tested this together with GitLab Dependency Scanning in Ultimate > SaaS, and it seems to be working well. > > Tested-by: Bruno Thomsen <bruno.thomsen@gmail.com> > > .gitlab-ci.yml example snippet: > > -------------8<------------- > > ptxdist sbom: > stage: build > script: > - cd ptxdist > - ./p sbom-report > artifacts: > reports: > dependency_scanning: <platform-dir>/release/sbom-report.json > > -------------8<------------- > > Thanks for working on this. > > /Bruno > Thanks, Bruno! I've sadly not had time to work more on this lately, but am hoping to pick it up again later in November. A bit stuck on how to properly provide / ensure that the needed Python cyclonedx libraries are available on the system. If anyone has any suggestions, I'd be happy to hear them :) Best regards, Simon ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages 2023-09-13 16:05 [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Simon Falsig 2023-09-13 16:05 ` [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages Simon Falsig 2023-09-13 16:05 ` [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support Simon Falsig @ 2023-09-13 21:16 ` Christian Melki 2023-09-14 6:46 ` Simon Falsig 2023-09-15 10:14 ` [ptxdist] [PATCH] " Simon Falsig 2 siblings, 2 replies; 13+ messages in thread From: Christian Melki @ 2023-09-13 21:16 UTC (permalink / raw) To: Simon Falsig; +Cc: ptxdist On 9/13/23 18:05, Simon Falsig wrote: > From: Simon Falsig <sfalsig@verity.ch> > > If a package specifies a CPE, this is extracted into the fast report for > that package. If no CPE is specified, then no value is added. > > The CPE (Common Platform Enumerator) allows matching CVEs to specific > packages, and see if these apply to a specific deployment. Hi Simon. I think this is a good thing going forward, but some minor nag. My objection would be that sticking full versioned CPE strings straight into the .make as an only-source just creates clutter. As an full CPE override, absolutely though. I suggest that some basic CPE modelling should be done by ptxdist, with possibly trivial hinting or nameing in the .make, with complete overrides as a last resort. That way ptxdist could start by filling most stuff and people could override on demand. I'd primarily poke the vendor:product tuple. Maybe ptxdist could do packagename:packagename as default. If you specify the smaller override it could be something like APPL_CPE_VENDOR and APPL_CPE_PRODUCT. Here you could use * or other strings. Overriding any of them or both. APPL_CPE would serve as the full override. That could help in hiding CPE format or other usages (subject to changes) in a lot of places. Hopefully, most packages won't require extra information to match. Regards, Christian > --- > rules/post/ptxd_make_world_common.make | 1 + > scripts/lib/ptxd_make_world_report.sh | 1 + > 2 files changed, 2 insertions(+) > > diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make > index 08120607a..6c646fb16 100644 > --- a/rules/post/ptxd_make_world_common.make > +++ b/rules/post/ptxd_make_world_common.make > @@ -78,6 +78,7 @@ world/env/impl = \ > pkg_PKG="$(call ptx/escape,$(1))" \ > pkg_pkg="$(call ptx/escape,$($(1)))" \ > pkg_version="$(call ptx/escape,$($(1)_VERSION))" \ > + pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \ > pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \ > pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))" \ > pkg_path="$(call ptx/escape,$($(1)_PATH))" \ > diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh > index dbdae5736..2bfe4c201 100644 > --- a/scripts/lib/ptxd_make_world_report.sh > +++ b/scripts/lib/ptxd_make_world_report.sh > @@ -39,6 +39,7 @@ ptxd_make_world_report_yaml() { > do_list "rundeps:" "${pkg_run_deps}" > do_echo "config:" "${pkg_config}" > do_echo "version:" "${pkg_version}" > + do_echo "cpe:" "${pkg_cpe}" > do_list "url:" "${pkg_url}" > do_echo "md5:" "${pkg_md5}" > do_echo "source:" "${pkg_src}" ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages 2023-09-13 21:16 ` [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Christian Melki @ 2023-09-14 6:46 ` Simon Falsig 2023-09-15 10:14 ` [ptxdist] [PATCH] " Simon Falsig 1 sibling, 0 replies; 13+ messages in thread From: Simon Falsig @ 2023-09-14 6:46 UTC (permalink / raw) To: christian.melki; +Cc: ptxdist Hi Christian, > From: Christian Melki <christian.melki@t2data.com> > Sent: Wednesday, September 13, 2023 23:17 > > On 9/13/23 18:05, Simon Falsig wrote: > > From: Simon Falsig <sfalsig@verity.ch> > > > > If a package specifies a CPE, this is extracted into the fast report > > for that package. If no CPE is specified, then no value is added. > > > > The CPE (Common Platform Enumerator) allows matching CVEs to specific > > packages, and see if these apply to a specific deployment. > > Hi Simon. > > I think this is a good thing going forward, but some minor nag. > My objection would be that sticking full versioned CPE strings straight > into the .make as an only-source just creates clutter. > As an full CPE override, absolutely though. > > I suggest that some basic CPE modelling should be done by ptxdist, with > possibly trivial hinting or nameing in the .make, with complete overrides > as a last resort. That way ptxdist could start by filling most stuff and > people could override on demand. > > I'd primarily poke the vendor:product tuple. Maybe ptxdist could do > packagename:packagename as default. If you specify the smaller override it > could be something like APPL_CPE_VENDOR and APPL_CPE_PRODUCT. Here you > could use * or other strings. Overriding any of them or both. > APPL_CPE would serve as the full override. > > That could help in hiding CPE format or other usages (subject to > changes) in a lot of places. Hopefully, most packages won't require extra > information to match. > > Regards, > Christian > I fully agree that the complete CPE string in the makefile is not desirable. When I was preparing the patch I was considering other ways, but in the end decided to just start with the simplest, to avoid implementing too complex logic before the first review... Also, I was struggling to find the right tradeoff between granularity and user-friendliness. I guess requiring either none or both of _CPE_VENDOR and _CPE_PRODUCT could work (to reduce the risk of wrong CPEs due to incorrect generic defaults - so only generate a CPE if both exist) - and then allowing a full override if _CPE is specified. In most cases that should be enough (for now at least), I think. I'll have a look at an implementation one of the next days. Thanks for the input! - Simon ^ permalink raw reply [flat|nested] 13+ messages in thread
* [ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages 2023-09-13 21:16 ` [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Christian Melki 2023-09-14 6:46 ` Simon Falsig @ 2023-09-15 10:14 ` Simon Falsig 2023-09-15 10:39 ` Michael Olbrich 1 sibling, 1 reply; 13+ messages in thread From: Simon Falsig @ 2023-09-15 10:14 UTC (permalink / raw) To: ptxdist; +Cc: Simon Falsig From: Simon Falsig <sfalsig@verity.ch> If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is extracted into the fast report for that package. If no CPE is specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is added. By default, the existing VERSION is used, but can be overridden with CPE_VERSION. Constructed CPEs are validated against the official CPE regex. The CPE (Common Platform Enumerator) allows matching CVEs to specific packages, and see if these apply to a specific deployment. --- rules/post/ptxd_make_world_common.make | 4 ++++ scripts/lib/ptxd_make_world_report.sh | 29 ++++++++++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make index 08120607a..0804f0b81 100644 --- a/rules/post/ptxd_make_world_common.make +++ b/rules/post/ptxd_make_world_common.make @@ -78,6 +78,10 @@ world/env/impl = \ pkg_PKG="$(call ptx/escape,$(1))" \ pkg_pkg="$(call ptx/escape,$($(1)))" \ pkg_version="$(call ptx/escape,$($(1)_VERSION))" \ + pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))" \ + pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))" \ + pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))" \ + pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \ pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \ pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))" \ pkg_path="$(call ptx/escape,$($(1)_PATH))" \ diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh index dbdae5736..11f17b405 100644 --- a/scripts/lib/ptxd_make_world_report.sh +++ b/scripts/lib/ptxd_make_world_report.sh @@ -31,6 +31,30 @@ ptxd_make_world_report_yaml() { awk "BEGIN { RS=\" \" } { if (\$1) print \"- '\" \$1 \"'\" }" <<<"${2}" fi } + do_build_cpe() { + prefix="${1}" + cpe="${2}" + vendor="${3}" + product="${4}" + version="${5}" + if [ -n "${cpe}" ]; then + # If a cpe is fully specified, then use that + : + elif [ -n "${vendor}" -a -n "${product}" -a -n "${version}" ]; then + # Otherwise, if we have vendor, product and version, then build a CPE2.3 string from it + cpe="cpe:2.3:a:${vendor}:${product}:${version}:*:*:*:*:*:*:*" + fi + if [ -n "$cpe" ]; then + # Validate the resulting CPE string + # Regex taken from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd + if echo "$cpe" | grep -Eq 'cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&'\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&''\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){4}'; then + echo "${prefix} ${cpe}" + else + >&2 echo "Error! $cpe is not valid CPE format string" + return 1 + fi + fi + } do_echo "name:" "${pkg_label}" do_echo "rulefile:" "${pkg_makefile}" do_list "extra-rulefiles:" "${pkg_extra_makefiles}" @@ -39,6 +63,11 @@ ptxd_make_world_report_yaml() { do_list "rundeps:" "${pkg_run_deps}" do_echo "config:" "${pkg_config}" do_echo "version:" "${pkg_version}" + if [ ! -n "${pkg_cpe_version}" ]; then + # Default to using pkg_version for the CPE string, unless _CPE_VERSION is explicitly specified + pkg_cpe_version="${pkg_version}"; + fi + do_build_cpe "cpe:" "${pkg_cpe}" "${pkg_cpe_vendor}" "${pkg_cpe_product}" "${pkg_cpe_version}" do_list "url:" "${pkg_url}" do_echo "md5:" "${pkg_md5}" do_echo "source:" "${pkg_src}" -- 2.25.1 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages 2023-09-15 10:14 ` [ptxdist] [PATCH] " Simon Falsig @ 2023-09-15 10:39 ` Michael Olbrich 2023-09-18 14:29 ` Simon Falsig 2023-09-18 14:37 ` Simon Falsig 0 siblings, 2 replies; 13+ messages in thread From: Michael Olbrich @ 2023-09-15 10:39 UTC (permalink / raw) To: Simon Falsig; +Cc: ptxdist, Simon Falsig On Fri, Sep 15, 2023 at 12:14:30PM +0200, Simon Falsig wrote: > From: Simon Falsig <sfalsig@verity.ch> > > If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is > extracted into the fast report for that package. If no CPE is > specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is > added. > > By default, the existing VERSION is used, but can be overridden with > CPE_VERSION. > > Constructed CPEs are validated against the official CPE regex. > > The CPE (Common Platform Enumerator) allows matching CVEs to specific > packages, and see if these apply to a specific deployment. > --- > rules/post/ptxd_make_world_common.make | 4 ++++ > scripts/lib/ptxd_make_world_report.sh | 29 ++++++++++++++++++++++++++ > 2 files changed, 33 insertions(+) > > diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make > index 08120607a..0804f0b81 100644 > --- a/rules/post/ptxd_make_world_common.make > +++ b/rules/post/ptxd_make_world_common.make > @@ -78,6 +78,10 @@ world/env/impl = \ > pkg_PKG="$(call ptx/escape,$(1))" \ > pkg_pkg="$(call ptx/escape,$($(1)))" \ > pkg_version="$(call ptx/escape,$($(1)_VERSION))" \ > + pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))" \ > + pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))" \ > + pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))" \ > + pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \ > pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \ > pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))" \ > pkg_path="$(call ptx/escape,$($(1)_PATH))" \ > diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh > index dbdae5736..11f17b405 100644 > --- a/scripts/lib/ptxd_make_world_report.sh > +++ b/scripts/lib/ptxd_make_world_report.sh > @@ -31,6 +31,30 @@ ptxd_make_world_report_yaml() { > awk "BEGIN { RS=\" \" } { if (\$1) print \"- '\" \$1 \"'\" }" <<<"${2}" > fi > } > + do_build_cpe() { > + prefix="${1}" > + cpe="${2}" > + vendor="${3}" > + product="${4}" > + version="${5}" > + if [ -n "${cpe}" ]; then > + # If a cpe is fully specified, then use that > + : > + elif [ -n "${vendor}" -a -n "${product}" -a -n "${version}" ]; then > + # Otherwise, if we have vendor, product and version, then build a CPE2.3 string from it > + cpe="cpe:2.3:a:${vendor}:${product}:${version}:*:*:*:*:*:*:*" > + fi Hmmm, I think we should preserve the original data in the report. Building the cpe string should happen in the SBOM script. So: cpe: .... or: cpe-vendor: ... cpe-product: ... and maybe: cpe-version: ... > + if [ -n "$cpe" ]; then > + # Validate the resulting CPE string > + # Regex taken from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd > + if echo "$cpe" | grep -Eq 'cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&'\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&''\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){4}'; then > + echo "${prefix} ${cpe}" > + else > + >&2 echo "Error! $cpe is not valid CPE format string" > + return 1 > + fi Hmmm, I'm not sure where the validation should take place. Here or the SBOM script. I don't mind either way. FYI, your indention is wrong. Please check the rest of the script. I know the style is a bit strange, but lets keep things consistent. Michael > + fi > + } > do_echo "name:" "${pkg_label}" > do_echo "rulefile:" "${pkg_makefile}" > do_list "extra-rulefiles:" "${pkg_extra_makefiles}" > @@ -39,6 +63,11 @@ ptxd_make_world_report_yaml() { > do_list "rundeps:" "${pkg_run_deps}" > do_echo "config:" "${pkg_config}" > do_echo "version:" "${pkg_version}" > + if [ ! -n "${pkg_cpe_version}" ]; then > + # Default to using pkg_version for the CPE string, unless _CPE_VERSION is explicitly specified > + pkg_cpe_version="${pkg_version}"; > + fi > + do_build_cpe "cpe:" "${pkg_cpe}" "${pkg_cpe_vendor}" "${pkg_cpe_product}" "${pkg_cpe_version}" > do_list "url:" "${pkg_url}" > do_echo "md5:" "${pkg_md5}" > do_echo "source:" "${pkg_src}" > -- > 2.25.1 > > > -- Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | ^ permalink raw reply [flat|nested] 13+ messages in thread
* [ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages 2023-09-15 10:39 ` Michael Olbrich @ 2023-09-18 14:29 ` Simon Falsig 2023-09-18 14:37 ` Simon Falsig 1 sibling, 0 replies; 13+ messages in thread From: Simon Falsig @ 2023-09-18 14:29 UTC (permalink / raw) To: ptxdist; +Cc: Simon Falsig From: Simon Falsig <sfalsig@verity.ch> If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is extracted into the fast report for that package. If no CPE is specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is added. By default, the existing VERSION is used, but can be overridden with CPE_VERSION. Constructed CPEs are validated against the official CPE regex. The CPE (Common Platform Enumerator) allows matching CVEs to specific packages, and see if these apply to a specific deployment. --- rules/post/ptxd_make_world_common.make | 4 ++++ scripts/lib/ptxd_make_world_report.sh | 9 +++++++++ 2 files changed, 13 insertions(+) diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make index 08120607a..0804f0b81 100644 --- a/rules/post/ptxd_make_world_common.make +++ b/rules/post/ptxd_make_world_common.make @@ -78,6 +78,10 @@ world/env/impl = \ pkg_PKG="$(call ptx/escape,$(1))" \ pkg_pkg="$(call ptx/escape,$($(1)))" \ pkg_version="$(call ptx/escape,$($(1)_VERSION))" \ + pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))" \ + pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))" \ + pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))" \ + pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \ pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \ pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))" \ pkg_path="$(call ptx/escape,$($(1)_PATH))" \ diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh index dbdae5736..dea25635b 100644 --- a/scripts/lib/ptxd_make_world_report.sh +++ b/scripts/lib/ptxd_make_world_report.sh @@ -39,6 +39,15 @@ ptxd_make_world_report_yaml() { do_list "rundeps:" "${pkg_run_deps}" do_echo "config:" "${pkg_config}" do_echo "version:" "${pkg_version}" + if [ ! -n "${pkg_cpe_version}" -a ! -n "${pkg_cpe}" ]; then + # Default to using pkg_version for the CPE string, unless _CPE_VERSION or _CPE are explicitly + # specified. In the case of the latter, there's no need to keep track of the version separately. + pkg_cpe_version="${pkg_version}" + fi + do_echo "cpe:" "${pkg_cpe}" + do_echo "cpe_vendor:" "${pkg_cpe_vendor}" + do_echo "cpe_product:" "${pkg_cpe_product}" + do_echo "cpe_version:" "${pkg_cpe_version}" do_list "url:" "${pkg_url}" do_echo "md5:" "${pkg_md5}" do_echo "source:" "${pkg_src}" -- 2.25.1 ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages 2023-09-15 10:39 ` Michael Olbrich 2023-09-18 14:29 ` Simon Falsig @ 2023-09-18 14:37 ` Simon Falsig 1 sibling, 0 replies; 13+ messages in thread From: Simon Falsig @ 2023-09-18 14:37 UTC (permalink / raw) To: ptxdist Hi Michael, > From: ptxdist <ptxdist-bounces@pengutronix.de> On Behalf Of Michael > Olbrich > Sent: Friday, September 15, 2023 12:39 > > On Fri, Sep 15, 2023 at 12:14:30PM +0200, Simon Falsig wrote: > > From: Simon Falsig <sfalsig@verity.ch> > > > > If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is > > extracted into the fast report for that package. If no CPE is > > specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is > > added. > > > > By default, the existing VERSION is used, but can be overridden with > > CPE_VERSION. > > > > Constructed CPEs are validated against the official CPE regex. > > > > The CPE (Common Platform Enumerator) allows matching CVEs to specific > > packages, and see if these apply to a specific deployment. > > --- > > rules/post/ptxd_make_world_common.make | 4 ++++ > > scripts/lib/ptxd_make_world_report.sh | 29 ++++++++++++++++++++++++++ > > 2 files changed, 33 insertions(+) > > > > diff --git a/rules/post/ptxd_make_world_common.make > > b/rules/post/ptxd_make_world_common.make > > index 08120607a..0804f0b81 100644 > > --- a/rules/post/ptxd_make_world_common.make > > +++ b/rules/post/ptxd_make_world_common.make > > @@ -78,6 +78,10 @@ world/env/impl = \ > > pkg_PKG="$(call ptx/escape,$(1))" \ > > pkg_pkg="$(call ptx/escape,$($(1)))" \ > > pkg_version="$(call ptx/escape,$($(1)_VERSION))" \ > > + pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))" > \ > > + pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))" \ > > + pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))" \ > > + pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \ > > pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \ > > pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))" > \ > > pkg_path="$(call ptx/escape,$($(1)_PATH))" \ > > diff --git a/scripts/lib/ptxd_make_world_report.sh > > b/scripts/lib/ptxd_make_world_report.sh > > index dbdae5736..11f17b405 100644 > > --- a/scripts/lib/ptxd_make_world_report.sh > > +++ b/scripts/lib/ptxd_make_world_report.sh > > @@ -31,6 +31,30 @@ ptxd_make_world_report_yaml() { > > awk "BEGIN { RS=\" \" } { if (\$1) print \"- '\" \$1 \"'\" }" > <<<"${2}" > > fi > > } > > + do_build_cpe() { > > + prefix="${1}" > > + cpe="${2}" > > + vendor="${3}" > > + product="${4}" > > + version="${5}" > > + if [ -n "${cpe}" ]; then > > + # If a cpe is fully specified, then use that > > + : > > + elif [ -n "${vendor}" -a -n "${product}" -a -n "${version}" ]; > then > > + # Otherwise, if we have vendor, product and version, then > build a CPE2.3 string from it > > + > cpe="cpe:2.3:a:${vendor}:${product}:${version}:*:*:*:*:*:*:*" > > + fi > > Hmmm, I think we should preserve the original data in the report. Building > the cpe string should happen in the SBOM script. So: > > cpe: .... > > or: > > cpe-vendor: ... > cpe-product: ... > > and maybe: > > cpe-version: ... > Makes sense - changed. > > + if [ -n "$cpe" ]; then > > + # Validate the resulting CPE string > > + # Regex taken from: > https://csrc.nis/ > t.gov%2Fschema%2Fcpe%2F2.3%2Fcpe- > naming_2.3.xsd&data=05%7C01%7Csfalsig%40verity.net%7Cc9dea8f344e64c0f2a570 > 8dbb5daf98c%7C06487c727d884632bf56071603defa0a%7C1%7C0%7C63830372431499528 > 3%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h > aWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RQxWOHS3iGwu%2BUXaR%2Bc1FZRzo4rHk > XX8U4fjSmWtalQ%3D&reserved=0 > > + if echo "$cpe" | grep -Eq 'cpe:2\.3:[aho\*\- > ](:(((\?*|\*?)([a-zA-Z0-9\- > \._]|(\\[\\\*\?!"#$$%&'\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\- > ])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[\*\- > ]))(:(((\?*|\*?)([a-zA-Z0-9\- > \._]|(\\[\\\*\?!"#$$%&''\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\- > ])){4}'; then > > + echo "${prefix} ${cpe}" > > + else > > + >&2 echo "Error! $cpe is not valid CPE format string" > > + return 1 > > + fi > > Hmmm, I'm not sure where the validation should take place. Here or the > SBOM script. I don't mind either way. > I've switched to do the validation in the SBOM script. Matches better with having the original cpe_... values in the individual reports, then building and validating the complete CPE when doing the full report. > FYI, your indention is wrong. Please check the rest of the script. I know > the style is a bit strange, but lets keep things consistent. Argh, sorry. I've tried fixing things up now - hope it's good! > > Michael Thanks for the input! - Simon ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2023-11-03 7:34 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-09-13 16:05 [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Simon Falsig 2023-09-13 16:05 ` [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages Simon Falsig 2023-09-15 10:15 ` [ptxdist] [PATCH] " Simon Falsig 2023-09-13 16:05 ` [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support Simon Falsig 2023-09-18 14:33 ` [ptxdist] [PATCH] " Simon Falsig 2023-10-21 13:52 ` Bruno Thomsen 2023-11-03 7:34 ` Simon Falsig 2023-09-13 21:16 ` [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Christian Melki 2023-09-14 6:46 ` Simon Falsig 2023-09-15 10:14 ` [ptxdist] [PATCH] " Simon Falsig 2023-09-15 10:39 ` Michael Olbrich 2023-09-18 14:29 ` Simon Falsig 2023-09-18 14:37 ` Simon Falsig
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox