[ptxdist] Node.js packages handling

[ptxdist] Node.js packages handling

[Markus Heidelberg via ptxdist]

Hi,

I started using Node.js, in particular Node-RED, and quickly ran into
several problems.

So I want to share some experience and am interested in how other users
handle specific issues.

1. Size of node_modules/

   Are there ways to reduce the size of the /usr/lib/node_module/
   directory in rootfs? Currently everything from
   local_src/nodejs_packages/yarn_cache/ will be installed except the
   .yarn-metadata.json file existing in each package. I'm not quite sure
   whether *.map, *.d.ts and *.ts and could be safely removed.

2. "_loc" line in .yarn-metadata.json

   .yarn-metadata.json created by "yarn add" during
   "nodejs_packages.compile" stage contains a line with a "_loc" key
   like this:

       "_loc": "/home/user/myproject/local_src/nodejs_packages/yarn_cache/v6/npm-node-red-4.1.6-2f7a2d4bf2565da191663b2751254e41dabe6d79-integrity/node_modules/node-red/package.json",

   The value of this key contains an absolute path to the module's
   package.json. It is not desired to keep such information in the
   project because

     - it reveals the username and directory name of the developer
     - it doesn't match when the repository is cloned by another person
       or to another location
     - it might change on adjustments by another person or after
       changing the development location

   I don't know what the "_loc" key is used for. Can it be removed
   without side-effects?
   I removed it as follows and didn't notice an impact:

	diff --git a/rules/nodejs_packages.make b/rules/nodejs_packages.make
	index bf533cc..38c6581 100644
	--- a/rules/nodejs_packages.make
	+++ b/rules/nodejs_packages.make
	@@ -61,6 +61,8 @@ $(STATEDIR)/nodejs_packages.compile:
		fi
		$(call node/env, yarn $(YARN_OPTS) add $(NODEJS_PACKAGES_LIST))
		@find $(NODEJS_PACKAGES_CACHE) -type f -name '.yarn-tarball.tgz' -delete
	+#	# remove "_loc" keys containing an absolute path to the module's package.json
	+	@find $(NODEJS_PACKAGES_CACHE) -type f -name '.yarn-metadata.json' -exec sed -i '/^ *"_loc"/d' {} \;
		@$(call touch)
	 
	 # ----------------------------------------------------------------------------

3. Patching Node.js packages

   How do you act if having to patch a Node.js package? It is not
   possible to use the traditional way with the patches/ directory. I
   manually fixed the files in the Yarn cache - each duplicate and in
   all variants (.js, *.min.js). It worked in my simple case, but is not
   a proper solution.

4. Build failure with precompiled native packages

   During "nodejs_packages.targetinstall" stage the build aborted
   because it tries to strip the module of the wrong architecture (x64
   instead of arm64) of a precompiled native node package:

   aarch64-v8a-linux-gnu-strip: Unable to recognise the format of the input file `.../platform-.../packages/nodejs_packages-0.0.1/usr/lib/node_modules/@node-rs/bcrypt-linux-x64-gnu/bcrypt.linux-x64-gnu.node'

   "yarn add" copies the x64 version during "nodejs_packages.compile"
   stage because the "cpu" field of package.json is compared against the
   "process.arch" node variable, which contains the host architecture.

   My workaround was to remove the optional "@node-rs/bcrypt*" packages
   from local_src/nodejs_packages/yarn.lock (but kept the downloaded
   packages) and fall back to the JavaScript implementation "bcryptjs".

   Has anyone experienced the same or a similar issue? I have the
   feeling it cannot be solved using Yarn v1.x, also the latest release
   v1.22.22 is two years ago.

5. Intl object missing

   Node.js is configured using "--with-intl=none" so it doesn't ship the
   "Intl" object. This caused a problem in Node-RED, so I wonder whether
   the configuration without intl is done by intent and which different
   effects changing this option would have.

Markus

Re: [ptxdist] Node.js packages handling

[Michael Olbrich]

Hi,

I don't really have any answers to your questions. The Node.js stuff in
PTXdist is basically unmaintained. I've long forgotten how the
nodejs_packages stuff works.
I've occasionally updated the version, but only because I needed a new one
for the host package because it is needed to build QtWebEngine.

What is really needed here is an update of nodejs to a newer LTS version
(the easy part) and a complete overhaul of the package management (the hard
part).
And I don't really know how this should look like. Probably something a bit
like what we do with rust/cargo where we introspect the dependencies,
generate extra rules for ptxdist, so that the download stage can download
all packages and later we extract it all in a way that cargo can use it.

This is all far from trivial and not something that I can do in my limited
time for general maintenance. This requires significant effort initially to
implement it and long-term, because stuff like this _will_ break with new
nodejs versions.

Michael

On Mon, Mar 23, 2026 at 04:01:32PM +0000, Markus Heidelberg via ptxdist wrote:
> I started using Node.js, in particular Node-RED, and quickly ran into
> several problems.
> 
> So I want to share some experience and am interested in how other users
> handle specific issues.
> 
> 1. Size of node_modules/
> 
>    Are there ways to reduce the size of the /usr/lib/node_module/
>    directory in rootfs? Currently everything from
>    local_src/nodejs_packages/yarn_cache/ will be installed except the
>    .yarn-metadata.json file existing in each package. I'm not quite sure
>    whether *.map, *.d.ts and *.ts and could be safely removed.
> 
> 2. "_loc" line in .yarn-metadata.json
> 
>    .yarn-metadata.json created by "yarn add" during
>    "nodejs_packages.compile" stage contains a line with a "_loc" key
>    like this:
> 
>        "_loc": "/home/user/myproject/local_src/nodejs_packages/yarn_cache/v6/npm-node-red-4.1.6-2f7a2d4bf2565da191663b2751254e41dabe6d79-integrity/node_modules/node-red/package.json",
> 
>    The value of this key contains an absolute path to the module's
>    package.json. It is not desired to keep such information in the
>    project because
> 
>      - it reveals the username and directory name of the developer
>      - it doesn't match when the repository is cloned by another person
>        or to another location
>      - it might change on adjustments by another person or after
>        changing the development location
> 
>    I don't know what the "_loc" key is used for. Can it be removed
>    without side-effects?
>    I removed it as follows and didn't notice an impact:
> 
> 	diff --git a/rules/nodejs_packages.make b/rules/nodejs_packages.make
> 	index bf533cc..38c6581 100644
> 	--- a/rules/nodejs_packages.make
> 	+++ b/rules/nodejs_packages.make
> 	@@ -61,6 +61,8 @@ $(STATEDIR)/nodejs_packages.compile:
> 		fi
> 		$(call node/env, yarn $(YARN_OPTS) add $(NODEJS_PACKAGES_LIST))
> 		@find $(NODEJS_PACKAGES_CACHE) -type f -name '.yarn-tarball.tgz' -delete
> 	+#	# remove "_loc" keys containing an absolute path to the module's package.json
> 	+	@find $(NODEJS_PACKAGES_CACHE) -type f -name '.yarn-metadata.json' -exec sed -i '/^ *"_loc"/d' {} \;
> 		@$(call touch)
> 	 
> 	 # ----------------------------------------------------------------------------
> 
> 3. Patching Node.js packages
> 
>    How do you act if having to patch a Node.js package? It is not
>    possible to use the traditional way with the patches/ directory. I
>    manually fixed the files in the Yarn cache - each duplicate and in
>    all variants (.js, *.min.js). It worked in my simple case, but is not
>    a proper solution.
> 
> 4. Build failure with precompiled native packages
> 
>    During "nodejs_packages.targetinstall" stage the build aborted
>    because it tries to strip the module of the wrong architecture (x64
>    instead of arm64) of a precompiled native node package:
> 
>    aarch64-v8a-linux-gnu-strip: Unable to recognise the format of the input file `.../platform-.../packages/nodejs_packages-0.0.1/usr/lib/node_modules/@node-rs/bcrypt-linux-x64-gnu/bcrypt.linux-x64-gnu.node'
> 
>    "yarn add" copies the x64 version during "nodejs_packages.compile"
>    stage because the "cpu" field of package.json is compared against the
>    "process.arch" node variable, which contains the host architecture.
> 
>    My workaround was to remove the optional "@node-rs/bcrypt*" packages
>    from local_src/nodejs_packages/yarn.lock (but kept the downloaded
>    packages) and fall back to the JavaScript implementation "bcryptjs".
> 
>    Has anyone experienced the same or a similar issue? I have the
>    feeling it cannot be solved using Yarn v1.x, also the latest release
>    v1.22.22 is two years ago.
> 
> 5. Intl object missing
> 
>    Node.js is configured using "--with-intl=none" so it doesn't ship the
>    "Intl" object. This caused a problem in Node-RED, so I wonder whether
>    the configuration without intl is done by intent and which different
>    effects changing this option would have.
> 
> Markus
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |