From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from dude02.hi.pengutronix.de ([2001:67c:670:100:1d::28] helo=dude02.lab.pengutronix.de) by metis.ext.pengutronix.de with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1j2aMN-0004oE-Is for ptxdist@pengutronix.de; Fri, 14 Feb 2020 13:46:19 +0100 Received: from mol by dude02.lab.pengutronix.de with local (Exim 4.92) (envelope-from ) id 1j2aMN-0005K3-BC for ptxdist@pengutronix.de; Fri, 14 Feb 2020 13:46:19 +0100 Date: Fri, 14 Feb 2020 13:46:19 +0100 From: Michael Olbrich Message-ID: <20200214124619.GF7958@pengutronix.de> References: <20200204075359.20506-1-u.oelmann@pengutronix.de> <20200204075359.20506-3-u.oelmann@pengutronix.de> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200204075359.20506-3-u.oelmann@pengutronix.de> Subject: Re: [ptxdist] [PATCH v5 3/3] nodejs_packages: new package List-Id: PTXdist Development Mailing List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: ptxdist@pengutronix.de Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Errors-To: ptxdist-bounces@pengutronix.de Sender: "ptxdist" To: ptxdist@pengutronix.de On Tue, Feb 04, 2020 at 08:53:59AM +0100, Ulrich =D6lmann wrote: > From: Bj=F6rn Esser > = > This is a dummy package to specify and install required Node.js packages = for the > target system. > = > The management of those packages and their dependencies is handled with > Yarn (run through host-nodejs) to maintain a reproducible offline cache t= hat is > stored in local_src/nodejs_packages. > = > Signed-off-by: Bj=F6rn Esser > [uol: adjusted commit message and Kconfig menu file's help texts, integra= ted > Kconfig symbol NODEJS_PACKAGES_LICENSE, fixed NODEJS_PACKAGES_URL, remove= d one > square bracket in a shell test and wrapped install/cp into call of execut= e and > world/execute] > Signed-off-by: Ulrich =D6lmann > --- > v3 --> v4: adjusted commit message and Kconfig menu file's help texts & f= ixed > NODEJS_PACKAGES_URL > Range-diff of v5 against v4: > 3: ccd2b44a8d92 ! 3: 1a73ebbd6e7e nodejs_packages: new package > @@ Commit message > stored in local_src/nodejs_packages. > = > Signed-off-by: Bj=F6rn Esser > - [uol: adjusted commit message and Kconfig menu file's help texts= & fixed > - NODEJS_PACKAGES_URL] > + [uol: adjusted commit message and Kconfig menu file's help texts= , integrated > + Kconfig symbol NODEJS_PACKAGES_LICENSE, fixed NODEJS_PACKAGES_UR= L, removed one > + square bracket in a shell test and wrapped install/cp into call = of execute and > + world/execute] > Signed-off-by: Ulrich =D6lmann > = > ## rules/nodejs_packages.in (new) ## > @@ rules/nodejs_packages.in (new) > + > + [1] https://legacy.yarnpkg.com/en/docs/dependency-versions/ > + > ++config NODEJS_PACKAGES_LICENSE > ++ string > ++ prompt "Licenses of to be installed Node.js packages" > ++ default "unknown" > ++ help > ++ Manually compiled list of licenses that the to be installed Node= .js > ++ packages are distributed under, see description of the variable > ++ *_LICENSE in [1]. > ++ > ++ ATTENTION: be aware of the fact that Node.js packages usually de= pend > ++ on additional Node.js packages that Yarn installs as well and th= at > ++ this propagates recursively resulting in an unknown number of No= de.js > ++ packages on the target with an unknown set of licenses that they= may > ++ be utilized under. Hence you carefully need to examine all of th= em > ++ individually to create a reliable list of licenses. > ++ > ++ [1] https://www.ptxdist.org/doc/dev_manual.html#some-notes-about= -licenses > ++ > +config NODEJS_PACKAGES_OFFLINE > + bool > -+ prompt "lock offline cache for offline/production use" > ++ prompt "Lock offline cache for offline/production use" > + help > -+ This option locks the offline cache for installing the nodejs pa= ckages > -+ and allows their installation during the build of the BSP withou= t any > -+ need for internet connectivity. > ++ This option locks the offline cache for installing the Node.js > ++ packages and allows their installation during the build of the B= SP > ++ without any need for internet connectivity. > + > + Enabling this option is highly recommended for production use. > + > @@ rules/nodejs_packages.make (new) > +NODEJS_PACKAGES_URL :=3D lndir://$(PTXDIST_WORKSPACE)/$(NODEJS_PACK= AGES_LOCAL) > +NODEJS_PACKAGES_DIR :=3D $(BUILDDIR)/$(NODEJS_PACKAGES) > +NODEJS_PACKAGES_CACHE :=3D $(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_L= OCAL)/yarn_cache > -+NODEJS_PACKAGES_LICENSE :=3D unknown > ++NODEJS_PACKAGES_LICENSE :=3D $(call remove_quotes, $(PTXCONF_NODEJS= _PACKAGES_LICENSE)) > + > +NODEJS_PACKAGES_LIST :=3D $(call remove_quotes, $(PTXCONF_NODEJS_PA= CKAGES_LIST)) > + > @@ rules/nodejs_packages.make (new) > + \"license\": \"UNLICENSED\", \ > + \"private\": true \ > + }" > $(NODEJS_PACKAGES_DIR)/package.json > -+ if [[ ! -f $(YARN_LOCK) ]]; then \ > ++ if [ ! -f $(YARN_LOCK) ]; then \ > + touch $(YARN_LOCK); \ > + ln -fs $(YARN_LOCK) $(NODEJS_PACKAGES_DIR)/yarn.lock; \ > + fi > @@ rules/nodejs_packages.make (new) > + > +$(STATEDIR)/nodejs_packages.install: > + @$(call targetinfo) > -+ install -dm 0755 $(NODEJS_PACKAGES_PKGDIR)/usr/lib > -+ cp -pr $(NODEJS_PACKAGES_DIR)/node_modules $(NODEJS_PACKAGES_PKGDI= R)/usr/lib > ++ @$(call world/execute, HOST_UA_NODESET, \ > ++ install -dm 0755 $(NODEJS_PACKAGES_PKGDIR)/usr/lib) > ++ @$(call execute, HOST_UA_NODESET, \ > ++ cp -pr $(NODEJS_PACKAGES_DIR)/node_modules \ > ++ $(NODEJS_PACKAGES_PKGDIR)/usr/lib) > + @$(call touch) > + > +# -----------------------------------------------------------------= ----------- > = > rules/nodejs_packages.in | 73 ++++++++++++++++++++++++++++++ > rules/nodejs_packages.make | 93 ++++++++++++++++++++++++++++++++++++++ > 2 files changed, 166 insertions(+) > create mode 100644 rules/nodejs_packages.in > create mode 100644 rules/nodejs_packages.make > = > diff --git a/rules/nodejs_packages.in b/rules/nodejs_packages.in > new file mode 100644 > index 000000000000..aea44205da41 > --- /dev/null > +++ b/rules/nodejs_packages.in > @@ -0,0 +1,73 @@ > +## SECTION=3Dbytecode_engines > + > +menuconfig NODEJS_PACKAGES > + tristate > + select HOST_YARN > + select NODEJS > + prompt "Node.js packages (target) " > + help > + Reproducibly install Node.js packages that will be available > + system-wide on the target. > + > + You MUST specify at least one Node.js package in the submenu that is > + shown after enabling this option. > + > + > +if NODEJS_PACKAGES > + > +config NODEJS_PACKAGES_LIST > + string > + prompt "Node.js packages to install" > + help > + A white-space separated list of Node.js packages including their > + versions (e.g. express@4.17.x or socket.io@'>=3D2.1.0 <=3D2.3.0'). NPM > + semvers are fully supported, see [1]. > + > + The packages are managed with Yarn and are guaranteed to be > + reproducible and immutable as long as the download location stays > + accessible or the offline cache is not altered. > + > + If the list of Node.js packages to be installed has been populated for > + the first time or if it has been modified later on one needs to > + (re-)build this package to download the desired Node.js packages and > + update the offline cache. Afterwards it is highly recommended to > + commit the path '${PTXDIST_WORKSPACE}/local_src/nodejs_packages/' into > + the version control system and to lock the package cache. > + > + As a prerequisite the path '${PTXDIST_WORKSPACE}/local_src/nodejs_ > + packages/' must exist in the BSP *before* building this package. > + > + [1] https://legacy.yarnpkg.com/en/docs/dependency-versions/ > + > +config NODEJS_PACKAGES_LICENSE > + string > + prompt "Licenses of to be installed Node.js packages" > + default "unknown" > + help > + Manually compiled list of licenses that the to be installed Node.js > + packages are distributed under, see description of the variable > + *_LICENSE in [1]. > + > + ATTENTION: be aware of the fact that Node.js packages usually depend > + on additional Node.js packages that Yarn installs as well and that > + this propagates recursively resulting in an unknown number of Node.js > + packages on the target with an unknown set of licenses that they may > + be utilized under. Hence you carefully need to examine all of them > + individually to create a reliable list of licenses. > + > + [1] https://www.ptxdist.org/doc/dev_manual.html#some-notes-about-lice= nses > + > +config NODEJS_PACKAGES_OFFLINE > + bool > + prompt "Lock offline cache for offline/production use" > + help > + This option locks the offline cache for installing the Node.js > + packages and allows their installation during the build of the BSP > + without any need for internet connectivity. > + > + Enabling this option is highly recommended for production use. > + > + Do NOT forget to commit the path '${PTXDIST_WORKSPACE}/local_src/ > + nodejs_packages/' into the BSP's version control system! > + > +endif > diff --git a/rules/nodejs_packages.make b/rules/nodejs_packages.make > new file mode 100644 > index 000000000000..cdde30f38b0c > --- /dev/null > +++ b/rules/nodejs_packages.make > @@ -0,0 +1,93 @@ > +# -*-makefile-*- > +# > +# Copyright (C) 2020 by Bjoern Esser > +# > +# For further information about the PTXdist project and license conditio= ns > +# see the README file. > +# > + > +# > +# We provide this package > +# > +PACKAGES-$(PTXCONF_NODEJS_PACKAGES) +=3D nodejs_packages > + > +# > +# Paths and names > +# > +NODEJS_PACKAGES_VERSION :=3D 0.0.1 > +NODEJS_PACKAGES :=3D nodejs_packages-$(NODEJS_PACKAGES_VERSION) > +NODEJS_PACKAGES_LOCAL :=3D local_src/nodejs_packages > +NODEJS_PACKAGES_URL :=3D lndir://$(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_= LOCAL) > +NODEJS_PACKAGES_DIR :=3D $(BUILDDIR)/$(NODEJS_PACKAGES) > +NODEJS_PACKAGES_CACHE :=3D $(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_LOCAL)= /yarn_cache > +NODEJS_PACKAGES_LICENSE :=3D $(call remove_quotes, $(PTXCONF_NODEJS_PACK= AGES_LICENSE)) > + > +NODEJS_PACKAGES_LIST :=3D $(call remove_quotes, $(PTXCONF_NODEJS_PACKAGE= S_LIST)) > + > +YARN_LOCK :=3D $(PTXDIST_WORKSPACE)/$(NODEJS_PACKAGES_LOCAL)/yarn.lock > +YARN_OPTS :=3D \ > + --cwd "$(NODEJS_PACKAGES_DIR)" \ > + --cache-folder "$(NODEJS_PACKAGES_CACHE)" \ > + --link-duplicates \ > + --production=3Dtrue > + > +ifdef PTXCONF_NODEJS_PACKAGES_OFFLINE > +YARN_OPTS +=3D \ > + --frozen-lockfile \ > + --offline > +endif > + > +# ----------------------------------------------------------------------= ------ > +# Compile > +# ----------------------------------------------------------------------= ------ > + > +$(STATEDIR)/nodejs_packages.compile: > + @$(call targetinfo) > + mkdir -p $(NODEJS_PACKAGES_CACHE) > + printf "{ \ > + \"name\": \"nodejs_packages\", \ > + \"version\": \"$(NODEJS_PACKAGES_VERSION)\", \ > + \"license\": \"UNLICENSED\", \ > + \"private\": true \ > + }" > $(NODEJS_PACKAGES_DIR)/package.json > + if [ ! -f $(YARN_LOCK) ]; then \ > + touch $(YARN_LOCK); \ > + ln -fs $(YARN_LOCK) $(NODEJS_PACKAGES_DIR)/yarn.lock; \ > + fi I think we should fail here if NODEJS_PACKAGES_OFFLINE is set and the lock file is missing. Or will yarn produce a sensible error message in this case? > + yarn $(YARN_OPTS) add $(NODEJS_PACKAGES_LIST) > + find $(NODEJS_PACKAGES_CACHE) -type f -name '.yarn-tarball.tgz' -delete > + @$(call touch) > + > +# ----------------------------------------------------------------------= ------ > +# Install > +# ----------------------------------------------------------------------= ------ > + > +$(STATEDIR)/nodejs_packages.install: > + @$(call targetinfo) > + @$(call world/execute, HOST_UA_NODESET, \ > + install -dm 0755 $(NODEJS_PACKAGES_PKGDIR)/usr/lib) add -v to make it visible at runtime. > + @$(call execute, HOST_UA_NODESET, \ > + cp -pr $(NODEJS_PACKAGES_DIR)/node_modules \ > + $(NODEJS_PACKAGES_PKGDIR)/usr/lib) Here as well. Michael > + @$(call touch) > + > +# ----------------------------------------------------------------------= ------ > +# Target-Install > +# ----------------------------------------------------------------------= ------ > + > +$(STATEDIR)/nodejs_packages.targetinstall: > + @$(call targetinfo) > + > + @$(call install_init, nodejs_packages) > + @$(call install_fixup, nodejs_packages,PRIORITY,optional) > + @$(call install_fixup, nodejs_packages,SECTION,base) > + @$(call install_fixup, nodejs_packages,AUTHOR,"Bjoern Esser ") > + @$(call install_fixup, nodejs_packages,DESCRIPTION,missing) > + > + $(call install_tree, nodejs_packages, 0, 0, -, /usr/lib/node_modules/) > + > + @$(call install_finish, nodejs_packages) > + > + @$(call touch) > + > +# vim: syntax=3Dmake > -- = > 2.25.0 > = > = > _______________________________________________ > ptxdist mailing list > ptxdist@pengutronix.de -- = Pengutronix e.K. | | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | _______________________________________________ ptxdist mailing list ptxdist@pengutronix.de