* [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages
@ 2023-09-13 16:05 Simon Falsig
2023-09-13 16:05 ` [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages Simon Falsig
` (2 more replies)
0 siblings, 3 replies; 15+ messages in thread
From: Simon Falsig @ 2023-09-13 16:05 UTC (permalink / raw)
To: ptxdist; +Cc: Simon Falsig
From: Simon Falsig <sfalsig@verity.ch>
If a package specifies a CPE, this is extracted into the fast report for
that package. If no CPE is specified, then no value is added.
The CPE (Common Platform Enumerator) allows matching CVEs to specific
packages, and see if these apply to a specific deployment.
---
rules/post/ptxd_make_world_common.make | 1 +
scripts/lib/ptxd_make_world_report.sh | 1 +
2 files changed, 2 insertions(+)
diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make
index 08120607a..6c646fb16 100644
--- a/rules/post/ptxd_make_world_common.make
+++ b/rules/post/ptxd_make_world_common.make
@@ -78,6 +78,7 @@ world/env/impl = \
pkg_PKG="$(call ptx/escape,$(1))" \
pkg_pkg="$(call ptx/escape,$($(1)))" \
pkg_version="$(call ptx/escape,$($(1)_VERSION))" \
+ pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \
pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \
pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))" \
pkg_path="$(call ptx/escape,$($(1)_PATH))" \
diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh
index dbdae5736..2bfe4c201 100644
--- a/scripts/lib/ptxd_make_world_report.sh
+++ b/scripts/lib/ptxd_make_world_report.sh
@@ -39,6 +39,7 @@ ptxd_make_world_report_yaml() {
do_list "rundeps:" "${pkg_run_deps}"
do_echo "config:" "${pkg_config}"
do_echo "version:" "${pkg_version}"
+ do_echo "cpe:" "${pkg_cpe}"
do_list "url:" "${pkg_url}"
do_echo "md5:" "${pkg_md5}"
do_echo "source:" "${pkg_src}"
--
2.25.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages
2023-09-13 16:05 [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Simon Falsig
@ 2023-09-13 16:05 ` Simon Falsig
2023-09-15 10:15 ` [ptxdist] [PATCH] " Simon Falsig
2023-09-13 16:05 ` [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support Simon Falsig
2023-09-13 21:16 ` [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Christian Melki
2 siblings, 1 reply; 15+ messages in thread
From: Simon Falsig @ 2023-09-13 16:05 UTC (permalink / raw)
To: ptxdist; +Cc: Simon Falsig
From: Simon Falsig <sfalsig@verity.ch>
Just to see how this could look for a handful of packages. Note that all
of these have a different way of specifying the vendor ID (one is
$PACKAGE_project, one is just $PACKAGE, one is something completely
different).
---
rules/acl.make | 1 +
rules/busybox.make | 1 +
rules/wpa_supplicant.make | 1 +
3 files changed, 3 insertions(+)
diff --git a/rules/acl.make b/rules/acl.make
index daa5837c9..10769681b 100644
--- a/rules/acl.make
+++ b/rules/acl.make
@@ -21,6 +21,7 @@ ACL_SUFFIX := tar.gz
ACL_URL := http://download.savannah.gnu.org/releases/acl/$(ACL).$(ACL_SUFFIX)
ACL_SOURCE := $(SRCDIR)/$(ACL).$(ACL_SUFFIX)
ACL_DIR := $(BUILDDIR)/$(ACL)
+ACL_CPE := cpe:2.3:a:acl_project:acl:$(ACL_VERSION):*:*:*:*:*:*:*
ACL_LICENSE := LGPL-2.1-or-later
ACL_LICENSE_FILES:= file://doc/COPYING.LGPL;md5=9e9a206917f8af112da634ce3ab41764
ifdef PTXCONF_ACL_TOOLS
diff --git a/rules/busybox.make b/rules/busybox.make
index 413ad735c..eb8721229 100644
--- a/rules/busybox.make
+++ b/rules/busybox.make
@@ -22,6 +22,7 @@ BUSYBOX_URL := https://www.busybox.net/downloads/$(BUSYBOX).$(BUSYBOX_SUFFIX)
BUSYBOX_SOURCE := $(SRCDIR)/$(BUSYBOX).$(BUSYBOX_SUFFIX)
BUSYBOX_DIR := $(BUILDDIR)/$(BUSYBOX)
BUSYBOX_KCONFIG := $(BUSYBOX_DIR)/Config.in
+BUSYBOX_CPE := cpe:2.3:a:busybox:busybox:$(BUSYBOX_VERSION):*:*:*:*:*:*:*
BUSYBOX_LICENSE := GPL-2.0-only
BUSYBOX_LICENSE_FILES := file://LICENSE;md5=de10de48642ab74318e893a61105afbb
diff --git a/rules/wpa_supplicant.make b/rules/wpa_supplicant.make
index 55d43b9b2..c0bf54f25 100644
--- a/rules/wpa_supplicant.make
+++ b/rules/wpa_supplicant.make
@@ -26,6 +26,7 @@ WPA_SUPPLICANT_SUBDIR := $(WPA_SUPPLICANT_NAME)
# Use '=' to delay $(shell ...) calls until this is needed
WPA_SUPPLICANT_CONFIG = $(call ptx/get-alternative, config/wpasupplicant, defconfig)
WPA_SUPPLICANT_DOTCONFIG:= $(BUILDDIR)/$(WPA_SUPPLICANT)/$(WPA_SUPPLICANT_SUBDIR)/.config
+WPA_SUPPLICANT_CPE := cpe:2.3:a:w1.fi:wpa_supplicant:$(WPA_SUPPLICANT_VERSION):*:*:*:*:*:*:*
WPA_SUPPLICANT_LICENSE := GPL-2.0-only
# ----------------------------------------------------------------------------
--
2.25.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support
2023-09-13 16:05 [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Simon Falsig
2023-09-13 16:05 ` [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages Simon Falsig
@ 2023-09-13 16:05 ` Simon Falsig
2023-09-18 14:33 ` [ptxdist] [PATCH] " Simon Falsig
2023-09-13 21:16 ` [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Christian Melki
2 siblings, 1 reply; 15+ messages in thread
From: Simon Falsig @ 2023-09-13 16:05 UTC (permalink / raw)
To: ptxdist; +Cc: Simon Falsig
From: Simon Falsig <sfalsig@verity.ch>
This provides support for building SBOMs in CycloneDX format.
A target is added alongside the other reports, that (based on the
fast-bsp-report) extracts name, version, cpe and license of each target
package, and puts these into a final sbom-report in CycloneDX/JSON
format.
This requires a working Python3 setup with the cyclonedx-bom package
installed.
---
bin/ptxdist | 3 ++-
rules/post/ptxd_make_report.make | 15 ++++++++++--
scripts/lib/ptxd_make_report.sh | 16 +++++++++++++
scripts/lib/ptxd_make_sbom_report.py | 35 ++++++++++++++++++++++++++++
4 files changed, 66 insertions(+), 3 deletions(-)
create mode 100644 scripts/lib/ptxd_make_sbom_report.py
diff --git a/bin/ptxdist b/bin/ptxdist
index dfb619cbd..15be851f5 100755
--- a/bin/ptxdist
+++ b/bin/ptxdist
@@ -780,6 +780,7 @@ Misc:
full-bsp-report generate a yaml file that describes the BSP and
all packages. More data but will build all
packages if necessary.
+ sbom-report generate a CycloneDX json SBOM
print <var> print the contents of a variable, in the way
it is known by "make"
printnext <var> assumes that the contents of <var> is another
@@ -1807,7 +1808,7 @@ EOF
ptxd_make_log export_src EXPORTDIR="${1}"
exit
;;
- fast-bsp-report|full-bsp-report)
+ fast-bsp-report|full-bsp-report|sbom-report)
check_premake_compiler &&
ptxd_make_log "${cmd}"
exit
diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make
index eecd2a577..ffa398c95 100644
--- a/rules/post/ptxd_make_report.make
+++ b/rules/post/ptxd_make_report.make
@@ -10,7 +10,9 @@ ptx/report-env = \
$(image/env) \
ptx_report_target="$(strip $(1))" \
ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \
- ptx_image_packages="$(IMAGE_PACKAGES)"
+ ptx_image_packages="$(IMAGE_PACKAGES)" \
+ ptx_target_packages="$(PACKAGES)"
+
PHONY += full-bsp-report
full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml
@@ -26,13 +28,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \
@$(call ptx/report-env, $@) ptxd_make_full_bsp_report
@$(call finish)
+
PHONY += fast-bsp-report
fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml
-
$(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED)))
@$(call targetinfo)
@$(call ptx/report-env, $@) ptxd_make_fast_bsp_report
@$(call finish)
+
+PHONY += sbom-report
+sbom-report: $(RELEASEDIR)/sbom-report.json
+
+$(RELEASEDIR)/sbom-report.json: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES)))
+ @$(call targetinfo)
+ @$(call ptx/report-env, $@) ptxd_make_sbom_report
+ @$(call finish)
+
# vim: syntax=make
diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh
index a363ca5b3..e2da4c05f 100644
--- a/scripts/lib/ptxd_make_report.sh
+++ b/scripts/lib/ptxd_make_report.sh
@@ -144,3 +144,19 @@ ptxd_make_fast_bsp_report() {
}
export -f ptxd_make_fast_bsp_report
+ptxd_make_sbom_report() {
+ local -a ptxd_reply
+ local pkg_lic pkg
+
+ ptxd_make_layer_init || return
+
+ echo "Generating $(ptxd_print_path "${ptx_report_target}") ..."
+ echo
+
+ mkdir -p "$(dirname "${ptx_report_target}")" &&
+ python3 ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report &&
+ mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" ||
+ ptxd_bailout "failed to create SBOM report"
+}
+export -f ptxd_make_sbom_report
+
diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py
new file mode 100644
index 000000000..aecb4fae5
--- /dev/null
+++ b/scripts/lib/ptxd_make_sbom_report.py
@@ -0,0 +1,35 @@
+from cyclonedx.factory.license import LicenseFactory
+from cyclonedx.factory.license import LicenseChoiceFactory
+from cyclonedx.model.bom import Bom
+from cyclonedx.model.component import Component
+from cyclonedx.output.json import JsonV1Dot4
+import sys
+import re
+
+lFac = LicenseFactory()
+lcFac = LicenseChoiceFactory(license_factory=lFac)
+bom = Bom()
+
+for i in range(2,len(sys.argv)):
+ pkg_report = sys.argv[1] + sys.argv[i] + ".yaml"
+ with open(pkg_report, 'r') as file:
+ content = file.read()
+ name_ = re.search("name: \'(.+)\'", content).group(1)
+ version_ = re.search("version: \'(.+)\'", content).group(1)
+ cpeMatch = re.search("cpe: \'(.+)\'", content)
+ cpe_ = None
+ if cpeMatch is not None:
+ cpe_ = cpeMatch.group(1)
+ licenses_ = re.search("licenses: \'(.+)\'", content).group(1)
+ comp = Component(
+ name = name_,
+ version = version_,
+ cpe = cpe_,
+ licenses = [lcFac.make_with_license(licenses_)],
+ bom_ref = name_ + "@" + version_
+ )
+ bom.components.add(comp)
+
+serializedJSON = JsonV1Dot4(bom).output_as_string()
+print(serializedJSON)
+
--
2.25.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages
2023-09-13 16:05 [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Simon Falsig
2023-09-13 16:05 ` [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages Simon Falsig
2023-09-13 16:05 ` [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support Simon Falsig
@ 2023-09-13 21:16 ` Christian Melki
2023-09-14 6:46 ` Simon Falsig
2023-09-15 10:14 ` [ptxdist] [PATCH] " Simon Falsig
2 siblings, 2 replies; 15+ messages in thread
From: Christian Melki @ 2023-09-13 21:16 UTC (permalink / raw)
To: Simon Falsig; +Cc: ptxdist
On 9/13/23 18:05, Simon Falsig wrote:
> From: Simon Falsig <sfalsig@verity.ch>
>
> If a package specifies a CPE, this is extracted into the fast report for
> that package. If no CPE is specified, then no value is added.
>
> The CPE (Common Platform Enumerator) allows matching CVEs to specific
> packages, and see if these apply to a specific deployment.
Hi Simon.
I think this is a good thing going forward, but some minor nag.
My objection would be that sticking full versioned CPE strings straight
into the .make as an only-source just creates clutter.
As an full CPE override, absolutely though.
I suggest that some basic CPE modelling should be done by ptxdist, with
possibly trivial hinting or nameing in the .make, with complete
overrides as a last resort. That way ptxdist could start by filling most
stuff and people could override on demand.
I'd primarily poke the vendor:product tuple. Maybe ptxdist could do
packagename:packagename as default. If you specify the smaller override
it could be something like APPL_CPE_VENDOR and APPL_CPE_PRODUCT. Here
you could use * or other strings. Overriding any of them or both.
APPL_CPE would serve as the full override.
That could help in hiding CPE format or other usages (subject to
changes) in a lot of places. Hopefully, most packages won't require
extra information to match.
Regards,
Christian
> ---
> rules/post/ptxd_make_world_common.make | 1 +
> scripts/lib/ptxd_make_world_report.sh | 1 +
> 2 files changed, 2 insertions(+)
>
> diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make
> index 08120607a..6c646fb16 100644
> --- a/rules/post/ptxd_make_world_common.make
> +++ b/rules/post/ptxd_make_world_common.make
> @@ -78,6 +78,7 @@ world/env/impl = \
> pkg_PKG="$(call ptx/escape,$(1))" \
> pkg_pkg="$(call ptx/escape,$($(1)))" \
> pkg_version="$(call ptx/escape,$($(1)_VERSION))" \
> + pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \
> pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \
> pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))" \
> pkg_path="$(call ptx/escape,$($(1)_PATH))" \
> diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh
> index dbdae5736..2bfe4c201 100644
> --- a/scripts/lib/ptxd_make_world_report.sh
> +++ b/scripts/lib/ptxd_make_world_report.sh
> @@ -39,6 +39,7 @@ ptxd_make_world_report_yaml() {
> do_list "rundeps:" "${pkg_run_deps}"
> do_echo "config:" "${pkg_config}"
> do_echo "version:" "${pkg_version}"
> + do_echo "cpe:" "${pkg_cpe}"
> do_list "url:" "${pkg_url}"
> do_echo "md5:" "${pkg_md5}"
> do_echo "source:" "${pkg_src}"
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages
2023-09-13 21:16 ` [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Christian Melki
@ 2023-09-14 6:46 ` Simon Falsig
2023-09-15 10:14 ` [ptxdist] [PATCH] " Simon Falsig
1 sibling, 0 replies; 15+ messages in thread
From: Simon Falsig @ 2023-09-14 6:46 UTC (permalink / raw)
To: christian.melki; +Cc: ptxdist
Hi Christian,
> From: Christian Melki <christian.melki@t2data.com>
> Sent: Wednesday, September 13, 2023 23:17
>
> On 9/13/23 18:05, Simon Falsig wrote:
> > From: Simon Falsig <sfalsig@verity.ch>
> >
> > If a package specifies a CPE, this is extracted into the fast report
> > for that package. If no CPE is specified, then no value is added.
> >
> > The CPE (Common Platform Enumerator) allows matching CVEs to specific
> > packages, and see if these apply to a specific deployment.
>
> Hi Simon.
>
> I think this is a good thing going forward, but some minor nag.
> My objection would be that sticking full versioned CPE strings straight
> into the .make as an only-source just creates clutter.
> As an full CPE override, absolutely though.
>
> I suggest that some basic CPE modelling should be done by ptxdist, with
> possibly trivial hinting or nameing in the .make, with complete overrides
> as a last resort. That way ptxdist could start by filling most stuff and
> people could override on demand.
>
> I'd primarily poke the vendor:product tuple. Maybe ptxdist could do
> packagename:packagename as default. If you specify the smaller override it
> could be something like APPL_CPE_VENDOR and APPL_CPE_PRODUCT. Here you
> could use * or other strings. Overriding any of them or both.
> APPL_CPE would serve as the full override.
>
> That could help in hiding CPE format or other usages (subject to
> changes) in a lot of places. Hopefully, most packages won't require extra
> information to match.
>
> Regards,
> Christian
>
I fully agree that the complete CPE string in the makefile is not desirable.
When I was preparing the patch I was considering other ways, but in the end
decided to just start with the simplest, to avoid implementing too complex
logic before the first review...
Also, I was struggling to find the right tradeoff between granularity and
user-friendliness. I guess requiring either none or both of _CPE_VENDOR and
_CPE_PRODUCT could work (to reduce the risk of wrong CPEs due to incorrect
generic defaults - so only generate a CPE if both exist) - and then allowing
a full override if _CPE is specified.
In most cases that should be enough (for now at least), I think. I'll have
a look at an implementation one of the next days.
Thanks for the input!
- Simon
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages
2023-09-13 21:16 ` [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Christian Melki
2023-09-14 6:46 ` Simon Falsig
@ 2023-09-15 10:14 ` Simon Falsig
2023-09-15 10:39 ` Michael Olbrich
1 sibling, 1 reply; 15+ messages in thread
From: Simon Falsig @ 2023-09-15 10:14 UTC (permalink / raw)
To: ptxdist; +Cc: Simon Falsig
From: Simon Falsig <sfalsig@verity.ch>
If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is
extracted into the fast report for that package. If no CPE is
specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is
added.
By default, the existing VERSION is used, but can be overridden with
CPE_VERSION.
Constructed CPEs are validated against the official CPE regex.
The CPE (Common Platform Enumerator) allows matching CVEs to specific
packages, and see if these apply to a specific deployment.
---
rules/post/ptxd_make_world_common.make | 4 ++++
scripts/lib/ptxd_make_world_report.sh | 29 ++++++++++++++++++++++++++
2 files changed, 33 insertions(+)
diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make
index 08120607a..0804f0b81 100644
--- a/rules/post/ptxd_make_world_common.make
+++ b/rules/post/ptxd_make_world_common.make
@@ -78,6 +78,10 @@ world/env/impl = \
pkg_PKG="$(call ptx/escape,$(1))" \
pkg_pkg="$(call ptx/escape,$($(1)))" \
pkg_version="$(call ptx/escape,$($(1)_VERSION))" \
+ pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))" \
+ pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))" \
+ pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))" \
+ pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \
pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \
pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))" \
pkg_path="$(call ptx/escape,$($(1)_PATH))" \
diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh
index dbdae5736..11f17b405 100644
--- a/scripts/lib/ptxd_make_world_report.sh
+++ b/scripts/lib/ptxd_make_world_report.sh
@@ -31,6 +31,30 @@ ptxd_make_world_report_yaml() {
awk "BEGIN { RS=\" \" } { if (\$1) print \"- '\" \$1 \"'\" }" <<<"${2}"
fi
}
+ do_build_cpe() {
+ prefix="${1}"
+ cpe="${2}"
+ vendor="${3}"
+ product="${4}"
+ version="${5}"
+ if [ -n "${cpe}" ]; then
+ # If a cpe is fully specified, then use that
+ :
+ elif [ -n "${vendor}" -a -n "${product}" -a -n "${version}" ]; then
+ # Otherwise, if we have vendor, product and version, then build a CPE2.3 string from it
+ cpe="cpe:2.3:a:${vendor}:${product}:${version}:*:*:*:*:*:*:*"
+ fi
+ if [ -n "$cpe" ]; then
+ # Validate the resulting CPE string
+ # Regex taken from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd
+ if echo "$cpe" | grep -Eq 'cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&'\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&''\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){4}'; then
+ echo "${prefix} ${cpe}"
+ else
+ >&2 echo "Error! $cpe is not valid CPE format string"
+ return 1
+ fi
+ fi
+ }
do_echo "name:" "${pkg_label}"
do_echo "rulefile:" "${pkg_makefile}"
do_list "extra-rulefiles:" "${pkg_extra_makefiles}"
@@ -39,6 +63,11 @@ ptxd_make_world_report_yaml() {
do_list "rundeps:" "${pkg_run_deps}"
do_echo "config:" "${pkg_config}"
do_echo "version:" "${pkg_version}"
+ if [ ! -n "${pkg_cpe_version}" ]; then
+ # Default to using pkg_version for the CPE string, unless _CPE_VERSION is explicitly specified
+ pkg_cpe_version="${pkg_version}";
+ fi
+ do_build_cpe "cpe:" "${pkg_cpe}" "${pkg_cpe_vendor}" "${pkg_cpe_product}" "${pkg_cpe_version}"
do_list "url:" "${pkg_url}"
do_echo "md5:" "${pkg_md5}"
do_echo "source:" "${pkg_src}"
--
2.25.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ptxdist] [PATCH] RFC: Add CPE for a few packages
2023-09-13 16:05 ` [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages Simon Falsig
@ 2023-09-15 10:15 ` Simon Falsig
0 siblings, 0 replies; 15+ messages in thread
From: Simon Falsig @ 2023-09-15 10:15 UTC (permalink / raw)
To: ptxdist; +Cc: Simon Falsig
From: Simon Falsig <sfalsig@verity.ch>
Just to see how this could look for a handful of packages. Note that all
of these have a different way of specifying the vendor ID (one is
$PACKAGE_project, one is just $PACKAGE, one is something completely
different).
---
rules/acl.make | 1 +
rules/busybox.make | 2 ++
rules/wpa_supplicant.make | 3 +++
3 files changed, 6 insertions(+)
diff --git a/rules/acl.make b/rules/acl.make
index daa5837c9..10769681b 100644
--- a/rules/acl.make
+++ b/rules/acl.make
@@ -21,6 +21,7 @@ ACL_SUFFIX := tar.gz
ACL_URL := http://download.savannah.gnu.org/releases/acl/$(ACL).$(ACL_SUFFIX)
ACL_SOURCE := $(SRCDIR)/$(ACL).$(ACL_SUFFIX)
ACL_DIR := $(BUILDDIR)/$(ACL)
+ACL_CPE := cpe:2.3:a:acl_project:acl:$(ACL_VERSION):*:*:*:*:*:*:*
ACL_LICENSE := LGPL-2.1-or-later
ACL_LICENSE_FILES:= file://doc/COPYING.LGPL;md5=9e9a206917f8af112da634ce3ab41764
ifdef PTXCONF_ACL_TOOLS
diff --git a/rules/busybox.make b/rules/busybox.make
index 413ad735c..b3ed2ff7a 100644
--- a/rules/busybox.make
+++ b/rules/busybox.make
@@ -22,6 +22,8 @@ BUSYBOX_URL := https://www.busybox.net/downloads/$(BUSYBOX).$(BUSYBOX_SUFFIX)
BUSYBOX_SOURCE := $(SRCDIR)/$(BUSYBOX).$(BUSYBOX_SUFFIX)
BUSYBOX_DIR := $(BUILDDIR)/$(BUSYBOX)
BUSYBOX_KCONFIG := $(BUSYBOX_DIR)/Config.in
+BUSYBOX_CPE_VENDOR := busybox
+BUSYBOX_CPE_PRODUCT := busybox
BUSYBOX_LICENSE := GPL-2.0-only
BUSYBOX_LICENSE_FILES := file://LICENSE;md5=de10de48642ab74318e893a61105afbb
diff --git a/rules/wpa_supplicant.make b/rules/wpa_supplicant.make
index 55d43b9b2..2f4d9c607 100644
--- a/rules/wpa_supplicant.make
+++ b/rules/wpa_supplicant.make
@@ -26,6 +26,9 @@ WPA_SUPPLICANT_SUBDIR := $(WPA_SUPPLICANT_NAME)
# Use '=' to delay $(shell ...) calls until this is needed
WPA_SUPPLICANT_CONFIG = $(call ptx/get-alternative, config/wpasupplicant, defconfig)
WPA_SUPPLICANT_DOTCONFIG:= $(BUILDDIR)/$(WPA_SUPPLICANT)/$(WPA_SUPPLICANT_SUBDIR)/.config
+WPA_SUPPLICANT_CPE_VENDOR := w1.fi
+WPA_SUPPLICANT_CPE_PRODUCT := wpa_supplicant
+WPA_SUPPLICANT_CPE_VERSION := $(WPA_SUPPLICANT_VERSION)
WPA_SUPPLICANT_LICENSE := GPL-2.0-only
# ----------------------------------------------------------------------------
--
2.25.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages
2023-09-15 10:14 ` [ptxdist] [PATCH] " Simon Falsig
@ 2023-09-15 10:39 ` Michael Olbrich
2023-09-18 14:29 ` Simon Falsig
2023-09-18 14:37 ` Simon Falsig
0 siblings, 2 replies; 15+ messages in thread
From: Michael Olbrich @ 2023-09-15 10:39 UTC (permalink / raw)
To: Simon Falsig; +Cc: ptxdist, Simon Falsig
On Fri, Sep 15, 2023 at 12:14:30PM +0200, Simon Falsig wrote:
> From: Simon Falsig <sfalsig@verity.ch>
>
> If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is
> extracted into the fast report for that package. If no CPE is
> specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is
> added.
>
> By default, the existing VERSION is used, but can be overridden with
> CPE_VERSION.
>
> Constructed CPEs are validated against the official CPE regex.
>
> The CPE (Common Platform Enumerator) allows matching CVEs to specific
> packages, and see if these apply to a specific deployment.
> ---
> rules/post/ptxd_make_world_common.make | 4 ++++
> scripts/lib/ptxd_make_world_report.sh | 29 ++++++++++++++++++++++++++
> 2 files changed, 33 insertions(+)
>
> diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make
> index 08120607a..0804f0b81 100644
> --- a/rules/post/ptxd_make_world_common.make
> +++ b/rules/post/ptxd_make_world_common.make
> @@ -78,6 +78,10 @@ world/env/impl = \
> pkg_PKG="$(call ptx/escape,$(1))" \
> pkg_pkg="$(call ptx/escape,$($(1)))" \
> pkg_version="$(call ptx/escape,$($(1)_VERSION))" \
> + pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))" \
> + pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))" \
> + pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))" \
> + pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \
> pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \
> pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))" \
> pkg_path="$(call ptx/escape,$($(1)_PATH))" \
> diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh
> index dbdae5736..11f17b405 100644
> --- a/scripts/lib/ptxd_make_world_report.sh
> +++ b/scripts/lib/ptxd_make_world_report.sh
> @@ -31,6 +31,30 @@ ptxd_make_world_report_yaml() {
> awk "BEGIN { RS=\" \" } { if (\$1) print \"- '\" \$1 \"'\" }" <<<"${2}"
> fi
> }
> + do_build_cpe() {
> + prefix="${1}"
> + cpe="${2}"
> + vendor="${3}"
> + product="${4}"
> + version="${5}"
> + if [ -n "${cpe}" ]; then
> + # If a cpe is fully specified, then use that
> + :
> + elif [ -n "${vendor}" -a -n "${product}" -a -n "${version}" ]; then
> + # Otherwise, if we have vendor, product and version, then build a CPE2.3 string from it
> + cpe="cpe:2.3:a:${vendor}:${product}:${version}:*:*:*:*:*:*:*"
> + fi
Hmmm, I think we should preserve the original data in the report. Building
the cpe string should happen in the SBOM script. So:
cpe: ....
or:
cpe-vendor: ...
cpe-product: ...
and maybe:
cpe-version: ...
> + if [ -n "$cpe" ]; then
> + # Validate the resulting CPE string
> + # Regex taken from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd
> + if echo "$cpe" | grep -Eq 'cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&'\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!"#$$%&''\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){4}'; then
> + echo "${prefix} ${cpe}"
> + else
> + >&2 echo "Error! $cpe is not valid CPE format string"
> + return 1
> + fi
Hmmm, I'm not sure where the validation should take place. Here or the SBOM
script. I don't mind either way.
FYI, your indention is wrong. Please check the rest of the script. I know
the style is a bit strange, but lets keep things consistent.
Michael
> + fi
> + }
> do_echo "name:" "${pkg_label}"
> do_echo "rulefile:" "${pkg_makefile}"
> do_list "extra-rulefiles:" "${pkg_extra_makefiles}"
> @@ -39,6 +63,11 @@ ptxd_make_world_report_yaml() {
> do_list "rundeps:" "${pkg_run_deps}"
> do_echo "config:" "${pkg_config}"
> do_echo "version:" "${pkg_version}"
> + if [ ! -n "${pkg_cpe_version}" ]; then
> + # Default to using pkg_version for the CPE string, unless _CPE_VERSION is explicitly specified
> + pkg_cpe_version="${pkg_version}";
> + fi
> + do_build_cpe "cpe:" "${pkg_cpe}" "${pkg_cpe_vendor}" "${pkg_cpe_product}" "${pkg_cpe_version}"
> do_list "url:" "${pkg_url}"
> do_echo "md5:" "${pkg_md5}"
> do_echo "source:" "${pkg_src}"
> --
> 2.25.1
>
>
>
--
Pengutronix e.K. | |
Steuerwalder Str. 21 | http://www.pengutronix.de/ |
31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages
2023-09-15 10:39 ` Michael Olbrich
@ 2023-09-18 14:29 ` Simon Falsig
2023-09-18 14:37 ` Simon Falsig
1 sibling, 0 replies; 15+ messages in thread
From: Simon Falsig @ 2023-09-18 14:29 UTC (permalink / raw)
To: ptxdist; +Cc: Simon Falsig
From: Simon Falsig <sfalsig@verity.ch>
If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is
extracted into the fast report for that package. If no CPE is
specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is
added.
By default, the existing VERSION is used, but can be overridden with
CPE_VERSION.
Constructed CPEs are validated against the official CPE regex.
The CPE (Common Platform Enumerator) allows matching CVEs to specific
packages, and see if these apply to a specific deployment.
---
rules/post/ptxd_make_world_common.make | 4 ++++
scripts/lib/ptxd_make_world_report.sh | 9 +++++++++
2 files changed, 13 insertions(+)
diff --git a/rules/post/ptxd_make_world_common.make b/rules/post/ptxd_make_world_common.make
index 08120607a..0804f0b81 100644
--- a/rules/post/ptxd_make_world_common.make
+++ b/rules/post/ptxd_make_world_common.make
@@ -78,6 +78,10 @@ world/env/impl = \
pkg_PKG="$(call ptx/escape,$(1))" \
pkg_pkg="$(call ptx/escape,$($(1)))" \
pkg_version="$(call ptx/escape,$($(1)_VERSION))" \
+ pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))" \
+ pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))" \
+ pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))" \
+ pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \
pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \
pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))" \
pkg_path="$(call ptx/escape,$($(1)_PATH))" \
diff --git a/scripts/lib/ptxd_make_world_report.sh b/scripts/lib/ptxd_make_world_report.sh
index dbdae5736..dea25635b 100644
--- a/scripts/lib/ptxd_make_world_report.sh
+++ b/scripts/lib/ptxd_make_world_report.sh
@@ -39,6 +39,15 @@ ptxd_make_world_report_yaml() {
do_list "rundeps:" "${pkg_run_deps}"
do_echo "config:" "${pkg_config}"
do_echo "version:" "${pkg_version}"
+ if [ ! -n "${pkg_cpe_version}" -a ! -n "${pkg_cpe}" ]; then
+ # Default to using pkg_version for the CPE string, unless _CPE_VERSION or _CPE are explicitly
+ # specified. In the case of the latter, there's no need to keep track of the version separately.
+ pkg_cpe_version="${pkg_version}"
+ fi
+ do_echo "cpe:" "${pkg_cpe}"
+ do_echo "cpe_vendor:" "${pkg_cpe_vendor}"
+ do_echo "cpe_product:" "${pkg_cpe_product}"
+ do_echo "cpe_version:" "${pkg_cpe_version}"
do_list "url:" "${pkg_url}"
do_echo "md5:" "${pkg_md5}"
do_echo "source:" "${pkg_src}"
--
2.25.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ptxdist] [PATCH] RFC: sbom_report: Add support
2023-09-13 16:05 ` [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support Simon Falsig
@ 2023-09-18 14:33 ` Simon Falsig
2023-10-21 13:52 ` Bruno Thomsen
0 siblings, 1 reply; 15+ messages in thread
From: Simon Falsig @ 2023-09-18 14:33 UTC (permalink / raw)
To: ptxdist; +Cc: Simon Falsig
From: Simon Falsig <sfalsig@verity.ch>
This provides support for building SBOMs in CycloneDX format.
A target is added alongside the other reports, that (based on the
fast-bsp-report) extracts name, version, cpe and license of each target
package, and puts these into a final sbom-report in CycloneDX/JSON
format.
This requires a working Python3 setup with the cyclonedx-bom package
installed.
---
bin/ptxdist | 3 +-
rules/post/ptxd_make_report.make | 15 ++++++--
scripts/lib/ptxd_make_report.sh | 16 +++++++++
scripts/lib/ptxd_make_sbom_report.py | 54 ++++++++++++++++++++++++++++
4 files changed, 85 insertions(+), 3 deletions(-)
create mode 100644 scripts/lib/ptxd_make_sbom_report.py
diff --git a/bin/ptxdist b/bin/ptxdist
index dfb619cbd..15be851f5 100755
--- a/bin/ptxdist
+++ b/bin/ptxdist
@@ -780,6 +780,7 @@ Misc:
full-bsp-report generate a yaml file that describes the BSP and
all packages. More data but will build all
packages if necessary.
+ sbom-report generate a CycloneDX json SBOM
print <var> print the contents of a variable, in the way
it is known by "make"
printnext <var> assumes that the contents of <var> is another
@@ -1807,7 +1808,7 @@ EOF
ptxd_make_log export_src EXPORTDIR="${1}"
exit
;;
- fast-bsp-report|full-bsp-report)
+ fast-bsp-report|full-bsp-report|sbom-report)
check_premake_compiler &&
ptxd_make_log "${cmd}"
exit
diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make
index eecd2a577..ffa398c95 100644
--- a/rules/post/ptxd_make_report.make
+++ b/rules/post/ptxd_make_report.make
@@ -10,7 +10,9 @@ ptx/report-env = \
$(image/env) \
ptx_report_target="$(strip $(1))" \
ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \
- ptx_image_packages="$(IMAGE_PACKAGES)"
+ ptx_image_packages="$(IMAGE_PACKAGES)" \
+ ptx_target_packages="$(PACKAGES)"
+
PHONY += full-bsp-report
full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml
@@ -26,13 +28,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \
@$(call ptx/report-env, $@) ptxd_make_full_bsp_report
@$(call finish)
+
PHONY += fast-bsp-report
fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml
-
$(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED)))
@$(call targetinfo)
@$(call ptx/report-env, $@) ptxd_make_fast_bsp_report
@$(call finish)
+
+PHONY += sbom-report
+sbom-report: $(RELEASEDIR)/sbom-report.json
+
+$(RELEASEDIR)/sbom-report.json: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES)))
+ @$(call targetinfo)
+ @$(call ptx/report-env, $@) ptxd_make_sbom_report
+ @$(call finish)
+
# vim: syntax=make
diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh
index a363ca5b3..e2da4c05f 100644
--- a/scripts/lib/ptxd_make_report.sh
+++ b/scripts/lib/ptxd_make_report.sh
@@ -144,3 +144,19 @@ ptxd_make_fast_bsp_report() {
}
export -f ptxd_make_fast_bsp_report
+ptxd_make_sbom_report() {
+ local -a ptxd_reply
+ local pkg_lic pkg
+
+ ptxd_make_layer_init || return
+
+ echo "Generating $(ptxd_print_path "${ptx_report_target}") ..."
+ echo
+
+ mkdir -p "$(dirname "${ptx_report_target}")" &&
+ python3 ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report &&
+ mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" ||
+ ptxd_bailout "failed to create SBOM report"
+}
+export -f ptxd_make_sbom_report
+
diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py
new file mode 100644
index 000000000..cc6a6f703
--- /dev/null
+++ b/scripts/lib/ptxd_make_sbom_report.py
@@ -0,0 +1,54 @@
+from cyclonedx.factory.license import LicenseFactory
+from cyclonedx.factory.license import LicenseChoiceFactory
+from cyclonedx.model.bom import Bom
+from cyclonedx.model.component import Component
+from cyclonedx.output.json import JsonV1Dot4
+import sys
+import re
+
+lFac = LicenseFactory()
+lcFac = LicenseChoiceFactory(license_factory=lFac)
+bom = Bom()
+
+for i in range(2, len(sys.argv)):
+ pkg_report = sys.argv[1] + sys.argv[i] + ".yaml"
+ with open(pkg_report, 'r') as file:
+ content = file.read()
+ name_ = re.search("name: \'(.+)\'", content).group(1)
+ version_ = re.search("version: \'(.+)\'", content).group(1)
+
+ # First see if we have a full CPE specified, then use that
+ cpe_match = re.search("cpe: \'(.+)\'", content)
+ cpe_ = None
+ if cpe_match is not None:
+ cpe_ = cpe_match.group(1)
+ else:
+ # See if we have the individual components
+ cpe_vendor_match = re.search("cpe_vendor: \'(.+)\'", content)
+ cpe_product_match = re.search("cpe_product: \'(.+)\'", content)
+ cpe_version_match = re.search("cpe_version: \'(.+)\'", content)
+ if cpe_vendor_match is not None and cpe_product_match is not None and cpe_version_match is not None:
+ cpe_ = "cpe:2.3:a:{vendor}:{product}:{version}:*:*:*:*:*:*:*".format(vendor=cpe_vendor_match.group(1),
+ product=cpe_product_match.group(1),
+ version=cpe_version_match.group(1))
+
+ if cpe_ is not None:
+ # We have a CPE, let's validate it. Regex from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd
+ if not re.fullmatch("cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;"
+ "<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)"
+ "|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;<=>@\[\]\^`\{\|}~]"
+ "))+(\?*|\*?))|[\*\-])){4}", cpe_):
+ raise ValueError("Constructed CPE is not valid: {cpe}".format(cpe=cpe_))
+
+ licenses_ = re.search("licenses: \'(.+)\'", content).group(1)
+ comp = Component(
+ name=name_,
+ version=version_,
+ cpe=cpe_,
+ licenses=[lcFac.make_with_license(licenses_)],
+ bom_ref=name_ + "@" + version_
+ )
+ bom.components.add(comp)
+
+serializedJSON = JsonV1Dot4(bom).output_as_string()
+print(serializedJSON)
--
2.25.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [ptxdist] [PATCH] RFC: ptxd_make_world: Extract CPE for packages
2023-09-15 10:39 ` Michael Olbrich
2023-09-18 14:29 ` Simon Falsig
@ 2023-09-18 14:37 ` Simon Falsig
1 sibling, 0 replies; 15+ messages in thread
From: Simon Falsig @ 2023-09-18 14:37 UTC (permalink / raw)
To: ptxdist
Hi Michael,
> From: ptxdist <ptxdist-bounces@pengutronix.de> On Behalf Of Michael
> Olbrich
> Sent: Friday, September 15, 2023 12:39
>
> On Fri, Sep 15, 2023 at 12:14:30PM +0200, Simon Falsig wrote:
> > From: Simon Falsig <sfalsig@verity.ch>
> >
> > If a package specifies a CPE or CPE_VENDOR and CPE_PRODUCT, this is
> > extracted into the fast report for that package. If no CPE is
> > specified, or not both of CPE_VENDOR and CPE_PRODUCT, then no value is
> > added.
> >
> > By default, the existing VERSION is used, but can be overridden with
> > CPE_VERSION.
> >
> > Constructed CPEs are validated against the official CPE regex.
> >
> > The CPE (Common Platform Enumerator) allows matching CVEs to specific
> > packages, and see if these apply to a specific deployment.
> > ---
> > rules/post/ptxd_make_world_common.make | 4 ++++
> > scripts/lib/ptxd_make_world_report.sh | 29 ++++++++++++++++++++++++++
> > 2 files changed, 33 insertions(+)
> >
> > diff --git a/rules/post/ptxd_make_world_common.make
> > b/rules/post/ptxd_make_world_common.make
> > index 08120607a..0804f0b81 100644
> > --- a/rules/post/ptxd_make_world_common.make
> > +++ b/rules/post/ptxd_make_world_common.make
> > @@ -78,6 +78,10 @@ world/env/impl = \
> > pkg_PKG="$(call ptx/escape,$(1))" \
> > pkg_pkg="$(call ptx/escape,$($(1)))" \
> > pkg_version="$(call ptx/escape,$($(1)_VERSION))" \
> > + pkg_cpe_vendor="$(call ptx/escape,$($(1)_CPE_VENDOR))"
> \
> > + pkg_cpe_product="$(call ptx/escape,$($(1)_CPE_PRODUCT))" \
> > + pkg_cpe_version="$(call ptx/escape,$($(1)_CPE_VERSION))" \
> > + pkg_cpe="$(call ptx/escape,$($(1)_CPE))" \
> > pkg_config="$(call ptx/escape,$($(1)_CONFIG))" \
> > pkg_ref_config="$(call ptx/escape,$($(1)_REF_CONFIG))"
> \
> > pkg_path="$(call ptx/escape,$($(1)_PATH))" \
> > diff --git a/scripts/lib/ptxd_make_world_report.sh
> > b/scripts/lib/ptxd_make_world_report.sh
> > index dbdae5736..11f17b405 100644
> > --- a/scripts/lib/ptxd_make_world_report.sh
> > +++ b/scripts/lib/ptxd_make_world_report.sh
> > @@ -31,6 +31,30 @@ ptxd_make_world_report_yaml() {
> > awk "BEGIN { RS=\" \" } { if (\$1) print \"- '\" \$1 \"'\" }"
> <<<"${2}"
> > fi
> > }
> > + do_build_cpe() {
> > + prefix="${1}"
> > + cpe="${2}"
> > + vendor="${3}"
> > + product="${4}"
> > + version="${5}"
> > + if [ -n "${cpe}" ]; then
> > + # If a cpe is fully specified, then use that
> > + :
> > + elif [ -n "${vendor}" -a -n "${product}" -a -n "${version}" ];
> then
> > + # Otherwise, if we have vendor, product and version, then
> build a CPE2.3 string from it
> > +
> cpe="cpe:2.3:a:${vendor}:${product}:${version}:*:*:*:*:*:*:*"
> > + fi
>
> Hmmm, I think we should preserve the original data in the report. Building
> the cpe string should happen in the SBOM script. So:
>
> cpe: ....
>
> or:
>
> cpe-vendor: ...
> cpe-product: ...
>
> and maybe:
>
> cpe-version: ...
>
Makes sense - changed.
> > + if [ -n "$cpe" ]; then
> > + # Validate the resulting CPE string
> > + # Regex taken from:
> https://csrc.nis/
> t.gov%2Fschema%2Fcpe%2F2.3%2Fcpe-
> naming_2.3.xsd&data=05%7C01%7Csfalsig%40verity.net%7Cc9dea8f344e64c0f2a570
> 8dbb5daf98c%7C06487c727d884632bf56071603defa0a%7C1%7C0%7C63830372431499528
> 3%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1h
> aWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RQxWOHS3iGwu%2BUXaR%2Bc1FZRzo4rHk
> XX8U4fjSmWtalQ%3D&reserved=0
> > + if echo "$cpe" | grep -Eq 'cpe:2\.3:[aho\*\-
> ](:(((\?*|\*?)([a-zA-Z0-9\-
> \._]|(\\[\\\*\?!"#$$%&'\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-
> ])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)|[\*\-
> ]))(:(((\?*|\*?)([a-zA-Z0-9\-
> \._]|(\\[\\\*\?!"#$$%&''\''\(\)\+,/:;<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-
> ])){4}'; then
> > + echo "${prefix} ${cpe}"
> > + else
> > + >&2 echo "Error! $cpe is not valid CPE format string"
> > + return 1
> > + fi
>
> Hmmm, I'm not sure where the validation should take place. Here or the
> SBOM script. I don't mind either way.
>
I've switched to do the validation in the SBOM script. Matches better with
having the original cpe_... values in the individual reports, then building
and validating the complete CPE when doing the full report.
> FYI, your indention is wrong. Please check the rest of the script. I know
> the style is a bit strange, but lets keep things consistent.
Argh, sorry. I've tried fixing things up now - hope it's good!
>
> Michael
Thanks for the input!
- Simon
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [ptxdist] [PATCH] RFC: sbom_report: Add support
2023-09-18 14:33 ` [ptxdist] [PATCH] " Simon Falsig
@ 2023-10-21 13:52 ` Bruno Thomsen
2023-11-03 7:34 ` Simon Falsig
0 siblings, 1 reply; 15+ messages in thread
From: Bruno Thomsen @ 2023-10-21 13:52 UTC (permalink / raw)
To: ptxdist; +Cc: Simon Falsig
Den man. 18. sep. 2023 kl. 16.34 skrev Simon Falsig <sfalsig@verity.net>:
>
> From: Simon Falsig <sfalsig@verity.ch>
>
> This provides support for building SBOMs in CycloneDX format.
>
> A target is added alongside the other reports, that (based on the
> fast-bsp-report) extracts name, version, cpe and license of each target
> package, and puts these into a final sbom-report in CycloneDX/JSON
> format.
>
> This requires a working Python3 setup with the cyclonedx-bom package
> installed.
Hi Simon,
I have tested this together with GitLab Dependency Scanning in Ultimate SaaS,
and it seems to be working well.
Tested-by: Bruno Thomsen <bruno.thomsen@gmail.com>
.gitlab-ci.yml example snippet:
-------------8<-------------
ptxdist sbom:
stage: build
script:
- cd ptxdist
- ./p sbom-report
artifacts:
reports:
dependency_scanning: <platform-dir>/release/sbom-report.json
-------------8<-------------
Thanks for working on this.
/Bruno
> ---
> bin/ptxdist | 3 +-
> rules/post/ptxd_make_report.make | 15 ++++++--
> scripts/lib/ptxd_make_report.sh | 16 +++++++++
> scripts/lib/ptxd_make_sbom_report.py | 54 ++++++++++++++++++++++++++++
> 4 files changed, 85 insertions(+), 3 deletions(-)
> create mode 100644 scripts/lib/ptxd_make_sbom_report.py
>
> diff --git a/bin/ptxdist b/bin/ptxdist
> index dfb619cbd..15be851f5 100755
> --- a/bin/ptxdist
> +++ b/bin/ptxdist
> @@ -780,6 +780,7 @@ Misc:
> full-bsp-report generate a yaml file that describes the BSP and
> all packages. More data but will build all
> packages if necessary.
> + sbom-report generate a CycloneDX json SBOM
> print <var> print the contents of a variable, in the way
> it is known by "make"
> printnext <var> assumes that the contents of <var> is another
> @@ -1807,7 +1808,7 @@ EOF
> ptxd_make_log export_src EXPORTDIR="${1}"
> exit
> ;;
> - fast-bsp-report|full-bsp-report)
> + fast-bsp-report|full-bsp-report|sbom-report)
> check_premake_compiler &&
> ptxd_make_log "${cmd}"
> exit
> diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make
> index eecd2a577..ffa398c95 100644
> --- a/rules/post/ptxd_make_report.make
> +++ b/rules/post/ptxd_make_report.make
> @@ -10,7 +10,9 @@ ptx/report-env = \
> $(image/env) \
> ptx_report_target="$(strip $(1))" \
> ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \
> - ptx_image_packages="$(IMAGE_PACKAGES)"
> + ptx_image_packages="$(IMAGE_PACKAGES)" \
> + ptx_target_packages="$(PACKAGES)"
> +
>
> PHONY += full-bsp-report
> full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml
> @@ -26,13 +28,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \
> @$(call ptx/report-env, $@) ptxd_make_full_bsp_report
> @$(call finish)
>
> +
> PHONY += fast-bsp-report
> fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml
>
> -
> $(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED)))
> @$(call targetinfo)
> @$(call ptx/report-env, $@) ptxd_make_fast_bsp_report
> @$(call finish)
>
> +
> +PHONY += sbom-report
> +sbom-report: $(RELEASEDIR)/sbom-report.json
> +
> +$(RELEASEDIR)/sbom-report.json: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES)))
> + @$(call targetinfo)
> + @$(call ptx/report-env, $@) ptxd_make_sbom_report
> + @$(call finish)
> +
> # vim: syntax=make
> diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh
> index a363ca5b3..e2da4c05f 100644
> --- a/scripts/lib/ptxd_make_report.sh
> +++ b/scripts/lib/ptxd_make_report.sh
> @@ -144,3 +144,19 @@ ptxd_make_fast_bsp_report() {
> }
> export -f ptxd_make_fast_bsp_report
>
> +ptxd_make_sbom_report() {
> + local -a ptxd_reply
> + local pkg_lic pkg
> +
> + ptxd_make_layer_init || return
> +
> + echo "Generating $(ptxd_print_path "${ptx_report_target}") ..."
> + echo
> +
> + mkdir -p "$(dirname "${ptx_report_target}")" &&
> + python3 ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report &&
> + mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" ||
> + ptxd_bailout "failed to create SBOM report"
> +}
> +export -f ptxd_make_sbom_report
> +
> diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py
> new file mode 100644
> index 000000000..cc6a6f703
> --- /dev/null
> +++ b/scripts/lib/ptxd_make_sbom_report.py
> @@ -0,0 +1,54 @@
> +from cyclonedx.factory.license import LicenseFactory
> +from cyclonedx.factory.license import LicenseChoiceFactory
> +from cyclonedx.model.bom import Bom
> +from cyclonedx.model.component import Component
> +from cyclonedx.output.json import JsonV1Dot4
> +import sys
> +import re
> +
> +lFac = LicenseFactory()
> +lcFac = LicenseChoiceFactory(license_factory=lFac)
> +bom = Bom()
> +
> +for i in range(2, len(sys.argv)):
> + pkg_report = sys.argv[1] + sys.argv[i] + ".yaml"
> + with open(pkg_report, 'r') as file:
> + content = file.read()
> + name_ = re.search("name: \'(.+)\'", content).group(1)
> + version_ = re.search("version: \'(.+)\'", content).group(1)
> +
> + # First see if we have a full CPE specified, then use that
> + cpe_match = re.search("cpe: \'(.+)\'", content)
> + cpe_ = None
> + if cpe_match is not None:
> + cpe_ = cpe_match.group(1)
> + else:
> + # See if we have the individual components
> + cpe_vendor_match = re.search("cpe_vendor: \'(.+)\'", content)
> + cpe_product_match = re.search("cpe_product: \'(.+)\'", content)
> + cpe_version_match = re.search("cpe_version: \'(.+)\'", content)
> + if cpe_vendor_match is not None and cpe_product_match is not None and cpe_version_match is not None:
> + cpe_ = "cpe:2.3:a:{vendor}:{product}:{version}:*:*:*:*:*:*:*".format(vendor=cpe_vendor_match.group(1),
> + product=cpe_product_match.group(1),
> + version=cpe_version_match.group(1))
> +
> + if cpe_ is not None:
> + # We have a CPE, let's validate it. Regex from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd
> + if not re.fullmatch("cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;"
> + "<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)"
> + "|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;<=>@\[\]\^`\{\|}~]"
> + "))+(\?*|\*?))|[\*\-])){4}", cpe_):
> + raise ValueError("Constructed CPE is not valid: {cpe}".format(cpe=cpe_))
> +
> + licenses_ = re.search("licenses: \'(.+)\'", content).group(1)
> + comp = Component(
> + name=name_,
> + version=version_,
> + cpe=cpe_,
> + licenses=[lcFac.make_with_license(licenses_)],
> + bom_ref=name_ + "@" + version_
> + )
> + bom.components.add(comp)
> +
> +serializedJSON = JsonV1Dot4(bom).output_as_string()
> +print(serializedJSON)
> --
> 2.25.1
>
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [ptxdist] [PATCH] RFC: sbom_report: Add support
2023-10-21 13:52 ` Bruno Thomsen
@ 2023-11-03 7:34 ` Simon Falsig
0 siblings, 0 replies; 15+ messages in thread
From: Simon Falsig @ 2023-11-03 7:34 UTC (permalink / raw)
To: Bruno Thomsen, ptxdist
> -----Original Message-----
> From: Bruno Thomsen <bruno.thomsen@gmail.com>
> Sent: Saturday, October 21, 2023 15:52
> Subject: Re: [ptxdist] [PATCH] RFC: sbom_report: Add support
>
> Den man. 18. sep. 2023 kl. 16.34 skrev Simon Falsig <sfalsig@verity.net>:
> >
> > From: Simon Falsig <sfalsig@verity.ch>
> >
> > This provides support for building SBOMs in CycloneDX format.
> >
> > A target is added alongside the other reports, that (based on the
> > fast-bsp-report) extracts name, version, cpe and license of each
> > target package, and puts these into a final sbom-report in
> > CycloneDX/JSON format.
> >
> > This requires a working Python3 setup with the cyclonedx-bom package
> > installed.
>
> Hi Simon,
>
> I have tested this together with GitLab Dependency Scanning in Ultimate
> SaaS, and it seems to be working well.
>
> Tested-by: Bruno Thomsen <bruno.thomsen@gmail.com>
>
> .gitlab-ci.yml example snippet:
>
> -------------8<-------------
>
> ptxdist sbom:
> stage: build
> script:
> - cd ptxdist
> - ./p sbom-report
> artifacts:
> reports:
> dependency_scanning: <platform-dir>/release/sbom-report.json
>
> -------------8<-------------
>
> Thanks for working on this.
>
> /Bruno
>
Thanks, Bruno!
I've sadly not had time to work more on this lately, but am hoping to pick it up again later in November.
A bit stuck on how to properly provide / ensure that the needed Python cyclonedx libraries are available on the system. If anyone has any suggestions, I'd be happy to hear them :)
Best regards,
Simon
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support
2024-02-19 16:56 ` [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support Simon Falsig
@ 2024-03-04 17:27 ` Michael Olbrich
0 siblings, 0 replies; 15+ messages in thread
From: Michael Olbrich @ 2024-03-04 17:27 UTC (permalink / raw)
To: Simon Falsig; +Cc: ptxdist
On Mon, Feb 19, 2024 at 05:56:17PM +0100, Simon Falsig wrote:
> This provides support for building SBOMs in CycloneDX format.
>
> A target is added alongside the other reports, that (based on the
> fast-bsp-report) extracts name, version, cpe and license of each target
> package, and puts these into a final sbom-report in CycloneDX/JSON
> format.
>
> This requires a working Python3 setup with the cyclonedx-python-lib
> package installed.
> ---
> bin/ptxdist | 3 +-
> rules/host-system-python3.in | 3 ++
> rules/host-system-python3.make | 6 ++++
> rules/post/ptxd_make_report.make | 16 +++++++--
> rules/sbom-report.in | 9 +++++
> scripts/lib/ptxd_make_report.sh | 16 +++++++++
> scripts/lib/ptxd_make_sbom_report.py | 52 ++++++++++++++++++++++++++++
> 7 files changed, 102 insertions(+), 3 deletions(-)
> create mode 100644 rules/sbom-report.in
> create mode 100644 scripts/lib/ptxd_make_sbom_report.py
>
> diff --git a/bin/ptxdist b/bin/ptxdist
> index 77cad673b..9e62d2bc0 100755
> --- a/bin/ptxdist
> +++ b/bin/ptxdist
> @@ -792,6 +792,7 @@ Misc:
> full-bsp-report generate a yaml file that describes the BSP and
> all packages. More data but will build all
> packages if necessary.
> + sbom-report generate a CycloneDX json SBOM
So I'm not so sure if I want a separate command for this. I think it would
be better to build this with the images.
> print <var> print the contents of a variable, in the way
> it is known by "make"
> printnext <var> assumes that the contents of <var> is another
> @@ -1841,7 +1842,7 @@ EOF
> ptxd_make_log export_src EXPORTDIR="${1}"
> exit
> ;;
> - fast-bsp-report|full-bsp-report)
> + fast-bsp-report|full-bsp-report|sbom-report)
> check_premake_compiler &&
> ptxd_make_log "${cmd}"
> exit
> diff --git a/rules/host-system-python3.in b/rules/host-system-python3.in
> index 25201e93f..c158a295f 100644
> --- a/rules/host-system-python3.in
> +++ b/rules/host-system-python3.in
> @@ -35,4 +35,7 @@ config HOST_SYSTEM_PYTHON3_PYELFTOOLS
> config HOST_SYSTEM_PYTHON3_PYYAML
> bool
>
> +config HOST_SYSTEM_PYTHON3_CYCLONEDX
> + bool
> +
> endif
> diff --git a/rules/host-system-python3.make b/rules/host-system-python3.make
> index 3688cf09d..f4979a453 100644
> --- a/rules/host-system-python3.make
> +++ b/rules/host-system-python3.make
> @@ -88,6 +88,12 @@ ifdef PTXCONF_HOST_SYSTEM_PYTHON3_PYYAML
> ptxd_bailout "Python pyyaml module not found! \
> Please install python3-yaml (debian)";
> endif
> +ifdef PTXCONF_HOST_SYSTEM_PYTHON3_CYCLONEDX
> + @echo "Checking for Python cyclonedx-python-lib ..."
> + @$(SYSTEMPYTHON3) -c 'import cyclonedx.factory' 2>/dev/null || \
> + ptxd_bailout "Python cyclonedx-python-lib module not found! \
> + Please install cyclonedx-python-lib (pip)";
> +endif
>
> @$(call touch)
>
This part (or whatever replaces this) should be a separate patch.
> diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make
> index eecd2a577..529b1c78e 100644
> --- a/rules/post/ptxd_make_report.make
> +++ b/rules/post/ptxd_make_report.make
> @@ -10,7 +10,10 @@ ptx/report-env = \
> $(image/env) \
> ptx_report_target="$(strip $(1))" \
> ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \
> - ptx_image_packages="$(IMAGE_PACKAGES)"
> + ptx_image_packages="$(IMAGE_PACKAGES)" \
> + ptx_target_packages="$(PACKAGES)" \
This should not be needed. See below.
> + ptx_system_python3="$(SYSTEMPYTHON3)"
> +
Just the normal she-bang should to the right thing.
>
> PHONY += full-bsp-report
> full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml
> @@ -26,13 +29,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \
> @$(call ptx/report-env, $@) ptxd_make_full_bsp_report
> @$(call finish)
>
> +
> PHONY += fast-bsp-report
> fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml
>
> -
Unrelated changes.
> $(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED)))
> @$(call targetinfo)
> @$(call ptx/report-env, $@) ptxd_make_fast_bsp_report
> @$(call finish)
>
> +
> +PHONY += sbom-report
> +sbom-report: $(RELEASEDIR)/sbom-report.json
> +
> +$(RELEASEDIR)/sbom-report.json: $(STATEDIR)/host-system-python3.install $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES)))
> + @$(call targetinfo)
> + @$(call ptx/report-env, $@) ptxd_make_sbom_report
> + @$(call finish)
> +
So I don't think this should be one target like this. The old license
documents work like this. But it's not very flexible. What you typically
want is basically an SBOM for an image with all its dependencies or maybe
one for the whole BSP (including all host packages).
The implementation should be flexible enough to do this. I've done that for
the 'new' license documents in scripts/generate-report.py and I am
currently extending that to create an SPDX SBOM.
And the input is always the full-bsp-report.yaml (or maybe
fast-bsp-report.yaml) but never the individual fragments.
> # vim: syntax=make
> diff --git a/rules/sbom-report.in b/rules/sbom-report.in
> new file mode 100644
> index 000000000..311ad5a23
> --- /dev/null
> +++ b/rules/sbom-report.in
> @@ -0,0 +1,9 @@
> +## SECTION=ptxdist_options
> +
> +config ENABLE_SBOM_REPORT
> + bool
> + select HOST_SYSTEM_PYTHON3
> + select HOST_SYSTEM_PYTHON3_CYCLONEDX
> + prompt "enable sbom report generation"
> + help
> + This enables the generation of an SBOM report, through ptxdist sbom-report.
> diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh
> index 885f07828..f2e92e6be 100644
> --- a/scripts/lib/ptxd_make_report.sh
> +++ b/scripts/lib/ptxd_make_report.sh
> @@ -146,3 +146,19 @@ ptxd_make_fast_bsp_report() {
> }
> export -f ptxd_make_fast_bsp_report
>
> +ptxd_make_sbom_report() {
> + local -a ptxd_reply
> + local pkg_lic pkg
> +
> + ptxd_make_layer_init || return
> +
> + echo "Generating $(ptxd_print_path "${ptx_report_target}") ..."
> + echo
> +
> + mkdir -p "$(dirname "${ptx_report_target}")" &&
> + "${ptx_system_python3}" ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report &&
> + mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" ||
> + ptxd_bailout "failed to create SBOM report"
> +}
> +export -f ptxd_make_sbom_report
> +
> diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py
> new file mode 100644
> index 000000000..a9ec36697
> --- /dev/null
> +++ b/scripts/lib/ptxd_make_sbom_report.py
> @@ -0,0 +1,52 @@
> +from cyclonedx.factory.license import LicenseFactory
> +from cyclonedx.model.bom import Bom
> +from cyclonedx.model.component import Component
> +from cyclonedx.output.json import JsonV1Dot4
> +import sys
> +import re
> +
> +lFac = LicenseFactory()
> +bom = Bom()
> +
> +for i in range(2, len(sys.argv)):
> + pkg_report = sys.argv[1] + sys.argv[i] + ".yaml"
> + with open(pkg_report, 'r') as file:
> + content = file.read()
> + name_ = re.search("name: \'(.+)\'", content).group(1)
> + version_ = re.search("version: \'(.+)\'", content).group(1)
> +
> + # First see if we have a full CPE specified, then use that
> + cpe_match = re.search("cpe: \'(.+)\'", content)
> + cpe_ = None
> + if cpe_match is not None:
> + cpe_ = cpe_match.group(1)
> + else:
> + # See if we have the individual components
> + cpe_vendor_match = re.search("cpe_vendor: \'(.+)\'", content)
> + cpe_product_match = re.search("cpe_product: \'(.+)\'", content)
> + cpe_version_match = re.search("cpe_version: \'(.+)\'", content)
> + if cpe_vendor_match is not None and cpe_product_match is not None and cpe_version_match is not None:
> + cpe_ = "cpe:2.3:a:{vendor}:{product}:{version}:*:*:*:*:*:*:*".format(vendor=cpe_vendor_match.group(1),
> + product=cpe_product_match.group(1),
> + version=cpe_version_match.group(1))
> +
> + if cpe_ is not None:
> + # We have a CPE, let's validate it. Regex from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd
> + if not re.fullmatch("cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;"
> + "<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)"
> + "|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;<=>@\[\]\^`\{\|}~]"
> + "))+(\?*|\*?))|[\*\-])){4}", cpe_):
> + raise ValueError("Constructed CPE is not valid: {cpe}".format(cpe=cpe_))
> +
> + licenses_ = re.search("licenses: \'(.+)\'", content).group(1)
> + comp = Component(
> + name=name_,
> + version=version_,
> + cpe=cpe_,
> + licenses=[lFac.make_from_string(licenses_)],
> + bom_ref=name_ + "@" + version_
> + )
> + bom.components.add(comp)
> +
> +serializedJSON = JsonV1Dot4(bom).output_as_string()
> +print(serializedJSON)
This needs to be quite different. We have one yaml file with all the data
and that should be used.
As I said, I'm working on an SPDX SBOM. It's probably best if you wait for
that to hit ptxdist master and build on top of that. That will probably
happen this or next week, at least the initial version.
Regards,
Michael
--
Pengutronix e.K. | |
Steuerwalder Str. 21 | http://www.pengutronix.de/ |
31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support
2024-02-19 16:56 [ptxdist] [PATCH 1/3] " Simon Falsig
@ 2024-02-19 16:56 ` Simon Falsig
2024-03-04 17:27 ` Michael Olbrich
0 siblings, 1 reply; 15+ messages in thread
From: Simon Falsig @ 2024-02-19 16:56 UTC (permalink / raw)
To: ptxdist; +Cc: Simon Falsig
This provides support for building SBOMs in CycloneDX format.
A target is added alongside the other reports, that (based on the
fast-bsp-report) extracts name, version, cpe and license of each target
package, and puts these into a final sbom-report in CycloneDX/JSON
format.
This requires a working Python3 setup with the cyclonedx-python-lib
package installed.
---
bin/ptxdist | 3 +-
rules/host-system-python3.in | 3 ++
rules/host-system-python3.make | 6 ++++
rules/post/ptxd_make_report.make | 16 +++++++--
rules/sbom-report.in | 9 +++++
scripts/lib/ptxd_make_report.sh | 16 +++++++++
scripts/lib/ptxd_make_sbom_report.py | 52 ++++++++++++++++++++++++++++
7 files changed, 102 insertions(+), 3 deletions(-)
create mode 100644 rules/sbom-report.in
create mode 100644 scripts/lib/ptxd_make_sbom_report.py
diff --git a/bin/ptxdist b/bin/ptxdist
index 77cad673b..9e62d2bc0 100755
--- a/bin/ptxdist
+++ b/bin/ptxdist
@@ -792,6 +792,7 @@ Misc:
full-bsp-report generate a yaml file that describes the BSP and
all packages. More data but will build all
packages if necessary.
+ sbom-report generate a CycloneDX json SBOM
print <var> print the contents of a variable, in the way
it is known by "make"
printnext <var> assumes that the contents of <var> is another
@@ -1841,7 +1842,7 @@ EOF
ptxd_make_log export_src EXPORTDIR="${1}"
exit
;;
- fast-bsp-report|full-bsp-report)
+ fast-bsp-report|full-bsp-report|sbom-report)
check_premake_compiler &&
ptxd_make_log "${cmd}"
exit
diff --git a/rules/host-system-python3.in b/rules/host-system-python3.in
index 25201e93f..c158a295f 100644
--- a/rules/host-system-python3.in
+++ b/rules/host-system-python3.in
@@ -35,4 +35,7 @@ config HOST_SYSTEM_PYTHON3_PYELFTOOLS
config HOST_SYSTEM_PYTHON3_PYYAML
bool
+config HOST_SYSTEM_PYTHON3_CYCLONEDX
+ bool
+
endif
diff --git a/rules/host-system-python3.make b/rules/host-system-python3.make
index 3688cf09d..f4979a453 100644
--- a/rules/host-system-python3.make
+++ b/rules/host-system-python3.make
@@ -88,6 +88,12 @@ ifdef PTXCONF_HOST_SYSTEM_PYTHON3_PYYAML
ptxd_bailout "Python pyyaml module not found! \
Please install python3-yaml (debian)";
endif
+ifdef PTXCONF_HOST_SYSTEM_PYTHON3_CYCLONEDX
+ @echo "Checking for Python cyclonedx-python-lib ..."
+ @$(SYSTEMPYTHON3) -c 'import cyclonedx.factory' 2>/dev/null || \
+ ptxd_bailout "Python cyclonedx-python-lib module not found! \
+ Please install cyclonedx-python-lib (pip)";
+endif
@$(call touch)
diff --git a/rules/post/ptxd_make_report.make b/rules/post/ptxd_make_report.make
index eecd2a577..529b1c78e 100644
--- a/rules/post/ptxd_make_report.make
+++ b/rules/post/ptxd_make_report.make
@@ -10,7 +10,10 @@ ptx/report-env = \
$(image/env) \
ptx_report_target="$(strip $(1))" \
ptx_packages_selected="$(filter-out $(IMAGE_PACKAGES),$(PTX_PACKAGES_SELECTED))" \
- ptx_image_packages="$(IMAGE_PACKAGES)"
+ ptx_image_packages="$(IMAGE_PACKAGES)" \
+ ptx_target_packages="$(PACKAGES)" \
+ ptx_system_python3="$(SYSTEMPYTHON3)"
+
PHONY += full-bsp-report
full-bsp-report: $(RELEASEDIR)/full-bsp-report.yaml
@@ -26,13 +29,22 @@ $(RELEASEDIR)/full-bsp-report.yaml: \
@$(call ptx/report-env, $@) ptxd_make_full_bsp_report
@$(call finish)
+
PHONY += fast-bsp-report
fast-bsp-report: $(RELEASEDIR)/fast-bsp-report.yaml
-
$(RELEASEDIR)/fast-bsp-report.yaml: $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PTX_PACKAGES_SELECTED)))
@$(call targetinfo)
@$(call ptx/report-env, $@) ptxd_make_fast_bsp_report
@$(call finish)
+
+PHONY += sbom-report
+sbom-report: $(RELEASEDIR)/sbom-report.json
+
+$(RELEASEDIR)/sbom-report.json: $(STATEDIR)/host-system-python3.install $(addprefix $(STATEDIR)/,$(addsuffix .fast-report,$(PACKAGES)))
+ @$(call targetinfo)
+ @$(call ptx/report-env, $@) ptxd_make_sbom_report
+ @$(call finish)
+
# vim: syntax=make
diff --git a/rules/sbom-report.in b/rules/sbom-report.in
new file mode 100644
index 000000000..311ad5a23
--- /dev/null
+++ b/rules/sbom-report.in
@@ -0,0 +1,9 @@
+## SECTION=ptxdist_options
+
+config ENABLE_SBOM_REPORT
+ bool
+ select HOST_SYSTEM_PYTHON3
+ select HOST_SYSTEM_PYTHON3_CYCLONEDX
+ prompt "enable sbom report generation"
+ help
+ This enables the generation of an SBOM report, through ptxdist sbom-report.
diff --git a/scripts/lib/ptxd_make_report.sh b/scripts/lib/ptxd_make_report.sh
index 885f07828..f2e92e6be 100644
--- a/scripts/lib/ptxd_make_report.sh
+++ b/scripts/lib/ptxd_make_report.sh
@@ -146,3 +146,19 @@ ptxd_make_fast_bsp_report() {
}
export -f ptxd_make_fast_bsp_report
+ptxd_make_sbom_report() {
+ local -a ptxd_reply
+ local pkg_lic pkg
+
+ ptxd_make_layer_init || return
+
+ echo "Generating $(ptxd_print_path "${ptx_report_target}") ..."
+ echo
+
+ mkdir -p "$(dirname "${ptx_report_target}")" &&
+ "${ptx_system_python3}" ${PTXDIST_LIB_DIR}/ptxd_make_sbom_report.py "${ptx_report_dir}/fast/" ${ptx_target_packages} > ${PTXDIST_TEMPDIR}/sbom-report &&
+ mv "${PTXDIST_TEMPDIR}/sbom-report" "${ptx_report_target}" ||
+ ptxd_bailout "failed to create SBOM report"
+}
+export -f ptxd_make_sbom_report
+
diff --git a/scripts/lib/ptxd_make_sbom_report.py b/scripts/lib/ptxd_make_sbom_report.py
new file mode 100644
index 000000000..a9ec36697
--- /dev/null
+++ b/scripts/lib/ptxd_make_sbom_report.py
@@ -0,0 +1,52 @@
+from cyclonedx.factory.license import LicenseFactory
+from cyclonedx.model.bom import Bom
+from cyclonedx.model.component import Component
+from cyclonedx.output.json import JsonV1Dot4
+import sys
+import re
+
+lFac = LicenseFactory()
+bom = Bom()
+
+for i in range(2, len(sys.argv)):
+ pkg_report = sys.argv[1] + sys.argv[i] + ".yaml"
+ with open(pkg_report, 'r') as file:
+ content = file.read()
+ name_ = re.search("name: \'(.+)\'", content).group(1)
+ version_ = re.search("version: \'(.+)\'", content).group(1)
+
+ # First see if we have a full CPE specified, then use that
+ cpe_match = re.search("cpe: \'(.+)\'", content)
+ cpe_ = None
+ if cpe_match is not None:
+ cpe_ = cpe_match.group(1)
+ else:
+ # See if we have the individual components
+ cpe_vendor_match = re.search("cpe_vendor: \'(.+)\'", content)
+ cpe_product_match = re.search("cpe_product: \'(.+)\'", content)
+ cpe_version_match = re.search("cpe_version: \'(.+)\'", content)
+ if cpe_vendor_match is not None and cpe_product_match is not None and cpe_version_match is not None:
+ cpe_ = "cpe:2.3:a:{vendor}:{product}:{version}:*:*:*:*:*:*:*".format(vendor=cpe_vendor_match.group(1),
+ product=cpe_product_match.group(1),
+ version=cpe_version_match.group(1))
+
+ if cpe_ is not None:
+ # We have a CPE, let's validate it. Regex from: https://csrc.nist.gov/schema/cpe/2.3/cpe-naming_2.3.xsd
+ if not re.fullmatch("cpe:2\.3:[aho\*\-](:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;"
+ "<=>@\[\]\^`\{\|}~]))+(\?*|\*?))|[\*\-])){5}(:(([a-zA-Z]{2,3}(-([a-zA-Z]{2}|[0-9]{3}))?)"
+ "|[\*\-]))(:(((\?*|\*?)([a-zA-Z0-9\-\._]|(\\[\\\*\?!\"#$$%&'\(\)\+,/:;<=>@\[\]\^`\{\|}~]"
+ "))+(\?*|\*?))|[\*\-])){4}", cpe_):
+ raise ValueError("Constructed CPE is not valid: {cpe}".format(cpe=cpe_))
+
+ licenses_ = re.search("licenses: \'(.+)\'", content).group(1)
+ comp = Component(
+ name=name_,
+ version=version_,
+ cpe=cpe_,
+ licenses=[lFac.make_from_string(licenses_)],
+ bom_ref=name_ + "@" + version_
+ )
+ bom.components.add(comp)
+
+serializedJSON = JsonV1Dot4(bom).output_as_string()
+print(serializedJSON)
--
2.25.1
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2024-03-04 17:28 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-09-13 16:05 [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Simon Falsig
2023-09-13 16:05 ` [ptxdist] [PATCH 2/3] RFC: Add CPE for a few packages Simon Falsig
2023-09-15 10:15 ` [ptxdist] [PATCH] " Simon Falsig
2023-09-13 16:05 ` [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support Simon Falsig
2023-09-18 14:33 ` [ptxdist] [PATCH] " Simon Falsig
2023-10-21 13:52 ` Bruno Thomsen
2023-11-03 7:34 ` Simon Falsig
2023-09-13 21:16 ` [ptxdist] [PATCH 1/3] RFC: ptxd_make_world: Extract CPE for packages Christian Melki
2023-09-14 6:46 ` Simon Falsig
2023-09-15 10:14 ` [ptxdist] [PATCH] " Simon Falsig
2023-09-15 10:39 ` Michael Olbrich
2023-09-18 14:29 ` Simon Falsig
2023-09-18 14:37 ` Simon Falsig
2024-02-19 16:56 [ptxdist] [PATCH 1/3] " Simon Falsig
2024-02-19 16:56 ` [ptxdist] [PATCH 3/3] RFC: sbom_report: Add support Simon Falsig
2024-03-04 17:27 ` Michael Olbrich
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox