From: Michael Olbrich <mol@pengutronix.de>
To: ptxdist@pengutronix.de
Cc: u.oelmann@pengutronix.de
Subject: Re: [ptxdist] [v6,3/3] nodejs_packages: new package
Date: Fri, 27 Mar 2020 10:52:31 +0100 [thread overview]
Message-ID: <E1jHlfD-0000M8-4c@dude02.lab.pengutronix.de> (raw)
In-Reply-To: <20200318155406.27584-3-u.oelmann@pengutronix.de>
Thanks, applied.
Michael
[sent from post-receive hook]
On Fri, 27 Mar 2020 10:52:31 +0100, Ulrich Ölmann <u.oelmann@pengutronix.de> wrote:
> From: Björn Esser <b.esser@pengutronix.de>
>
> This is a dummy package to specify and install required Node.js packages for the
> target system.
>
> The management of those packages and their dependencies is handled with
> Yarn (run through host-nodejs) to maintain a reproducible offline cache that is
> stored in local_src/nodejs_packages.
>
> Signed-off-by: Björn Esser <b.esser@pengutronix.de>
> [uol: adjusted commit message and Kconfig menu file's help texts, integrated
> Kconfig symbol NODEJS_PACKAGES_LICENSE, made NODEJS_PACKAGES_CACHE and YARN_LOCK
> aware of PTXdist's concept of layers, made yarn, mkdir, install & cp more
> verbose, suppressed echoing of diverse commands, removed one square bracket in a
> shell test, let PTXdist bail out if yarn's frozen lock file does not exist at
> all and wrapped install/cp into call of execute and world/execute]
> Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
> ---
> v3 --> v4: adjusted commit message and Kconfig menu file's help texts & fixed
> NODEJS_PACKAGES_URL
> v4 --> v5: adjusted commit message and Kconfig menu file's help texts,
> integrated Kconfig symbol NODEJS_PACKAGES_LICENSE, removed one square
> bracket in a shell test and wrapped install/cp into call of execute
> and world/execute
> v5 --> v6: adjusted commit message, reverted "fix" of NODEJS_PACKAGES_URL
> applied in v4, made NODEJS_PACKAGES_CACHE and YARN_LOCK aware of
> PTXdist's concept of layers, made yarn, mkdir, install & cp more
> verbose, suppressed echoing of diverse commands and let PTXdist bail
> out if yarn's frozen lock file does not exist at all
>
> Detailed range-diff of v6 against v5:
> 3: 1a73ebbd6e7e ! 3: ffc71b434eda nodejs_packages: new package
> @@ Commit message
>
> Signed-off-by: Björn Esser <b.esser@pengutronix.de>
> [uol: adjusted commit message and Kconfig menu file's help texts, integrated
> - Kconfig symbol NODEJS_PACKAGES_LICENSE, fixed NODEJS_PACKAGES_URL, removed one
> - square bracket in a shell test and wrapped install/cp into call of execute and
> - world/execute]
> + Kconfig symbol NODEJS_PACKAGES_LICENSE, made NODEJS_PACKAGES_CACHE and YARN_LOCK
> + aware of PTXdist's concept of layers, made yarn, mkdir, install & cp more
> + verbose, suppressed echoing of diverse commands, removed one square bracket in a
> + shell test, let PTXdist bail out if yarn's frozen lock file does not exist at
> + all and wrapped install/cp into call of execute and world/execute]
> Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
>
> ## rules/nodejs_packages.in (new) ##
> @@ rules/nodejs_packages.make (new)
> +NODEJS_PACKAGES_VERSION := 0.0.1
> +NODEJS_PACKAGES := nodejs_packages-$(NODEJS_PACKAGES_VERSION)
> +NODEJS_PACKAGES_LOCAL := local_src/nodejs_packages
> -+NODEJS_PACKAGES_URL := lndir://$(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_LOCAL)
> ++NODEJS_PACKAGES_URL := lndir://$(NODEJS_PACKAGES_LOCAL)
> +NODEJS_PACKAGES_DIR := $(BUILDDIR)/$(NODEJS_PACKAGES)
> -+NODEJS_PACKAGES_CACHE := $(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_LOCAL)/yarn_cache
> ++NODEJS_PACKAGES_CACHE := $(call ptx/in-path, PTXDIST_PATH_LAYERS, $(NODEJS_PACKAGES_LOCAL))/yarn_cache
> +NODEJS_PACKAGES_LICENSE := $(call remove_quotes, $(PTXCONF_NODEJS_PACKAGES_LICENSE))
> +
> +NODEJS_PACKAGES_LIST := $(call remove_quotes, $(PTXCONF_NODEJS_PACKAGES_LIST))
> +
> -+YARN_LOCK := $(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_LOCAL)/yarn.lock
> ++YARN_LOCK := $(call ptx/in-path, PTXDIST_PATH_LAYERS, $(NODEJS_PACKAGES_LOCAL))/yarn.lock
> +YARN_OPTS := \
> + --cwd "$(NODEJS_PACKAGES_DIR)" \
> + --cache-folder "$(NODEJS_PACKAGES_CACHE)" \
> + --link-duplicates \
> -+ --production=true
> ++ --production=true \
> ++ --verbose
> +
> +ifdef PTXCONF_NODEJS_PACKAGES_OFFLINE
> +YARN_OPTS += \
> @@ rules/nodejs_packages.make (new)
> +
> +$(STATEDIR)/nodejs_packages.compile:
> + @$(call targetinfo)
> -+ mkdir -p $(NODEJS_PACKAGES_CACHE)
> -+ printf "{ \
> ++ @mkdir -vp $(NODEJS_PACKAGES_CACHE)
> ++ @printf "{ \
> + \"name\": \"nodejs_packages\", \
> + \"version\": \"$(NODEJS_PACKAGES_VERSION)\", \
> + \"license\": \"UNLICENSED\", \
> + \"private\": true \
> + }" > $(NODEJS_PACKAGES_DIR)/package.json
> -+ if [ ! -f $(YARN_LOCK) ]; then \
> ++ @if [ ! -f $(YARN_LOCK) ]; then \
> ++ if [ $(PTXCONF_NODEJS_PACKAGES_OFFLINE) = y ]; then \
> ++ ptxd_bailout "Offline cache locked without existing 'yarn.lock'! \
> ++ Please unlock and compile before locking again."; \
> ++ fi; \
> + touch $(YARN_LOCK); \
> + ln -fs $(YARN_LOCK) $(NODEJS_PACKAGES_DIR)/yarn.lock; \
> + fi
> + yarn $(YARN_OPTS) add $(NODEJS_PACKAGES_LIST)
> -+ find $(NODEJS_PACKAGES_CACHE) -type f -name '.yarn-tarball.tgz' -delete
> ++ @find $(NODEJS_PACKAGES_CACHE) -type f -name '.yarn-tarball.tgz' -delete
> + @$(call touch)
> +
> +# ----------------------------------------------------------------------------
> @@ rules/nodejs_packages.make (new)
> +$(STATEDIR)/nodejs_packages.install:
> + @$(call targetinfo)
> + @$(call world/execute, HOST_UA_NODESET, \
> -+ install -dm 0755 $(NODEJS_PACKAGES_PKGDIR)/usr/lib)
> ++ install -vdm 0755 $(NODEJS_PACKAGES_PKGDIR)/usr/lib)
> + @$(call execute, HOST_UA_NODESET, \
> -+ cp -pr $(NODEJS_PACKAGES_DIR)/node_modules \
> ++ cp -vpr $(NODEJS_PACKAGES_DIR)/node_modules \
> + $(NODEJS_PACKAGES_PKGDIR)/usr/lib)
> + @$(call touch)
> +
>
> rules/nodejs_packages.in | 73 ++++++++++++++++++++++++++++
> rules/nodejs_packages.make | 98 ++++++++++++++++++++++++++++++++++++++
> 2 files changed, 171 insertions(+)
> create mode 100644 rules/nodejs_packages.in
> create mode 100644 rules/nodejs_packages.make
>
> diff --git a/rules/nodejs_packages.in b/rules/nodejs_packages.in
> new file mode 100644
> index 000000000000..aea44205da41
> --- /dev/null
> +++ b/rules/nodejs_packages.in
> @@ -0,0 +1,73 @@
> +## SECTION=bytecode_engines
> +
> +menuconfig NODEJS_PACKAGES
> + tristate
> + select HOST_YARN
> + select NODEJS
> + prompt "Node.js packages (target) "
> + help
> + Reproducibly install Node.js packages that will be available
> + system-wide on the target.
> +
> + You MUST specify at least one Node.js package in the submenu that is
> + shown after enabling this option.
> +
> +
> +if NODEJS_PACKAGES
> +
> +config NODEJS_PACKAGES_LIST
> + string
> + prompt "Node.js packages to install"
> + help
> + A white-space separated list of Node.js packages including their
> + versions (e.g. express@4.17.x or socket.io@'>=2.1.0 <=2.3.0'). NPM
> + semvers are fully supported, see [1].
> +
> + The packages are managed with Yarn and are guaranteed to be
> + reproducible and immutable as long as the download location stays
> + accessible or the offline cache is not altered.
> +
> + If the list of Node.js packages to be installed has been populated for
> + the first time or if it has been modified later on one needs to
> + (re-)build this package to download the desired Node.js packages and
> + update the offline cache. Afterwards it is highly recommended to
> + commit the path '${PTXDIST_WORKSPACE}/local_src/nodejs_packages/' into
> + the version control system and to lock the package cache.
> +
> + As a prerequisite the path '${PTXDIST_WORKSPACE}/local_src/nodejs_
> + packages/' must exist in the BSP *before* building this package.
> +
> + [1] https://legacy.yarnpkg.com/en/docs/dependency-versions/
> +
> +config NODEJS_PACKAGES_LICENSE
> + string
> + prompt "Licenses of to be installed Node.js packages"
> + default "unknown"
> + help
> + Manually compiled list of licenses that the to be installed Node.js
> + packages are distributed under, see description of the variable
> + *_LICENSE in [1].
> +
> + ATTENTION: be aware of the fact that Node.js packages usually depend
> + on additional Node.js packages that Yarn installs as well and that
> + this propagates recursively resulting in an unknown number of Node.js
> + packages on the target with an unknown set of licenses that they may
> + be utilized under. Hence you carefully need to examine all of them
> + individually to create a reliable list of licenses.
> +
> + [1] https://www.ptxdist.org/doc/dev_manual.html#some-notes-about-licenses
> +
> +config NODEJS_PACKAGES_OFFLINE
> + bool
> + prompt "Lock offline cache for offline/production use"
> + help
> + This option locks the offline cache for installing the Node.js
> + packages and allows their installation during the build of the BSP
> + without any need for internet connectivity.
> +
> + Enabling this option is highly recommended for production use.
> +
> + Do NOT forget to commit the path '${PTXDIST_WORKSPACE}/local_src/
> + nodejs_packages/' into the BSP's version control system!
> +
> +endif
> diff --git a/rules/nodejs_packages.make b/rules/nodejs_packages.make
> new file mode 100644
> index 000000000000..a20fff877719
> --- /dev/null
> +++ b/rules/nodejs_packages.make
> @@ -0,0 +1,98 @@
> +# -*-makefile-*-
> +#
> +# Copyright (C) 2020 by Bjoern Esser <bes@pengutronix.de>
> +#
> +# For further information about the PTXdist project and license conditions
> +# see the README file.
> +#
> +
> +#
> +# We provide this package
> +#
> +PACKAGES-$(PTXCONF_NODEJS_PACKAGES) += nodejs_packages
> +
> +#
> +# Paths and names
> +#
> +NODEJS_PACKAGES_VERSION := 0.0.1
> +NODEJS_PACKAGES := nodejs_packages-$(NODEJS_PACKAGES_VERSION)
> +NODEJS_PACKAGES_LOCAL := local_src/nodejs_packages
> +NODEJS_PACKAGES_URL := lndir://$(NODEJS_PACKAGES_LOCAL)
> +NODEJS_PACKAGES_DIR := $(BUILDDIR)/$(NODEJS_PACKAGES)
> +NODEJS_PACKAGES_CACHE := $(call ptx/in-path, PTXDIST_PATH_LAYERS, $(NODEJS_PACKAGES_LOCAL))/yarn_cache
> +NODEJS_PACKAGES_LICENSE := $(call remove_quotes, $(PTXCONF_NODEJS_PACKAGES_LICENSE))
> +
> +NODEJS_PACKAGES_LIST := $(call remove_quotes, $(PTXCONF_NODEJS_PACKAGES_LIST))
> +
> +YARN_LOCK := $(call ptx/in-path, PTXDIST_PATH_LAYERS, $(NODEJS_PACKAGES_LOCAL))/yarn.lock
> +YARN_OPTS := \
> + --cwd "$(NODEJS_PACKAGES_DIR)" \
> + --cache-folder "$(NODEJS_PACKAGES_CACHE)" \
> + --link-duplicates \
> + --production=true \
> + --verbose
> +
> +ifdef PTXCONF_NODEJS_PACKAGES_OFFLINE
> +YARN_OPTS += \
> + --frozen-lockfile \
> + --offline
> +endif
> +
> +# ----------------------------------------------------------------------------
> +# Compile
> +# ----------------------------------------------------------------------------
> +
> +$(STATEDIR)/nodejs_packages.compile:
> + @$(call targetinfo)
> + @mkdir -vp $(NODEJS_PACKAGES_CACHE)
> + @printf "{ \
> + \"name\": \"nodejs_packages\", \
> + \"version\": \"$(NODEJS_PACKAGES_VERSION)\", \
> + \"license\": \"UNLICENSED\", \
> + \"private\": true \
> + }" > $(NODEJS_PACKAGES_DIR)/package.json
> + @if [ ! -f $(YARN_LOCK) ]; then \
> + if [ $(PTXCONF_NODEJS_PACKAGES_OFFLINE) = y ]; then \
> + ptxd_bailout "Offline cache locked without existing 'yarn.lock'! \
> + Please unlock and compile before locking again."; \
> + fi; \
> + touch $(YARN_LOCK); \
> + ln -fs $(YARN_LOCK) $(NODEJS_PACKAGES_DIR)/yarn.lock; \
> + fi
> + yarn $(YARN_OPTS) add $(NODEJS_PACKAGES_LIST)
> + @find $(NODEJS_PACKAGES_CACHE) -type f -name '.yarn-tarball.tgz' -delete
> + @$(call touch)
> +
> +# ----------------------------------------------------------------------------
> +# Install
> +# ----------------------------------------------------------------------------
> +
> +$(STATEDIR)/nodejs_packages.install:
> + @$(call targetinfo)
> + @$(call world/execute, HOST_UA_NODESET, \
> + install -vdm 0755 $(NODEJS_PACKAGES_PKGDIR)/usr/lib)
> + @$(call execute, HOST_UA_NODESET, \
> + cp -vpr $(NODEJS_PACKAGES_DIR)/node_modules \
> + $(NODEJS_PACKAGES_PKGDIR)/usr/lib)
> + @$(call touch)
> +
> +# ----------------------------------------------------------------------------
> +# Target-Install
> +# ----------------------------------------------------------------------------
> +
> +$(STATEDIR)/nodejs_packages.targetinstall:
> + @$(call targetinfo)
> +
> + @$(call install_init, nodejs_packages)
> + @$(call install_fixup, nodejs_packages,PRIORITY,optional)
> + @$(call install_fixup, nodejs_packages,SECTION,base)
> + @$(call install_fixup, nodejs_packages,AUTHOR,"Bjoern Esser <bes@pengutronix.de>")
> + @$(call install_fixup, nodejs_packages,DESCRIPTION,missing)
> +
> + $(call install_tree, nodejs_packages, 0, 0, -, /usr/lib/node_modules/)
> +
> + @$(call install_finish, nodejs_packages)
> +
> + @$(call touch)
> +
> +# vim: syntax=make
>
_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
next prev parent reply other threads:[~2020-03-27 9:52 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-18 15:54 [ptxdist] [PATCH v6 1/3] nodejs: version bump 6.9.5 -> 12.14.1 LTS Ulrich Ölmann
2020-03-18 15:54 ` [ptxdist] [PATCH v6 2/3] host-yarn: new package Ulrich Ölmann
2020-03-27 9:52 ` [ptxdist] [v6,2/3] " Michael Olbrich
2020-03-18 15:54 ` [ptxdist] [PATCH v6 3/3] nodejs_packages: " Ulrich Ölmann
2020-03-27 9:52 ` Michael Olbrich [this message]
2020-03-27 9:52 ` [ptxdist] [v6,1/3] nodejs: version bump 6.9.5 -> 12.14.1 LTS Michael Olbrich
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=E1jHlfD-0000M8-4c@dude02.lab.pengutronix.de \
--to=mol@pengutronix.de \
--cc=ptxdist@pengutronix.de \
--cc=u.oelmann@pengutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox