[ptxdist] Error building nodejs in 2023.10.0

[ptxdist] Error building nodejs in 2023.10.0

[Jean-Claude Monnin]

Hi,

When trying to build nodejs with the latest ptxdist, I faced an error about missing a python module `ModuleNotFoundError: No module named '_bz2'` (see [1]).

I'm not very proficient with ptxdist, so I would like to check if the solution I came up is correct. Feedback welcome. If solution is ok, feel free to apply it upstream or ask for a proper email patch.

I think it's due to following issues:
- The host python is missing the option to support BZIP2. I added an option for that.
- nodejs.in says it depends on HOST_SYSTEM_PYTHON3, however it seems like the build isn't using system python but host python built by ptxdist. I think it should depend on HOST_PYTHON3 and the new HOST_PYTHON3_BZIP2.

See [2] for the patch I applied.

Kind regards,
Jean-Claude


[1] Error message

----------------------
target: nodejs.prepare
----------------------

Node.js configure: Found Python 3.11.2...
Traceback (most recent call last):
  File "/home/jean-claude/code/test-ptxdist/platform-nanopi-rk3588/build-target/node-v18.13.0/./configure", line 28, in <module>
    import configure
  File "/home/jean-claude/code/test-ptxdist/platform-nanopi-rk3588/build-target/node-v18.13.0/configure.py", line 14, in <module>
    import bz2
  File "/home/jean-claude/code/test-ptxdist/platform-nanopi-rk3588/sysroot-target/usr/lib/python3.11/bz2.py", line 17, in <module>
    from _bz2 import BZ2Compressor, BZ2Decompressor
ModuleNotFoundError: No module named '_bz2'


[2] possible patch

diff --git a/rules/host-python3.in b/rules/host-python3.in
index e15b90571..64f34affa 100644
--- a/rules/host-python3.in
+++ b/rules/host-python3.in
@@ -6,6 +6,7 @@ config HOST_PYTHON3
        select HOST_LIBFFI
        select HOST_OPENSSL
        select HOST_ZLIB                if HOST_PYTHON3_ZLIB
+       select HOST_BZIP2               if HOST_PYTHON3_BZIP2
        default y if ALLYES
        help
          Python is an interpreted, interactive,
@@ -16,4 +17,7 @@ if HOST_PYTHON3
 config HOST_PYTHON3_ZLIB
        bool
 
+config HOST_PYTHON3_BZIP2
+       bool
+
 endif
diff --git a/rules/nodejs.in b/rules/nodejs.in
index 80c6aec21..c75480953 100644
--- a/rules/nodejs.in
+++ b/rules/nodejs.in
@@ -3,7 +3,8 @@
 config NODEJS
        tristate
        prompt "Node.js                       "
-       select HOST_SYSTEM_PYTHON3
+       select HOST_PYTHON3
+       select HOST_PYTHON3_BZIP2
        select HOST_QEMU
        select HOST_QEMU_USR
        select C_ARES

Re: [ptxdist] Error building nodejs in 2023.10.0

[Michael Olbrich]

Hallo,

On Wed, Nov 01, 2023 at 05:04:30PM +0100, Jean-Claude Monnin wrote:
> When trying to build nodejs with the latest ptxdist, I faced an error about missing a python module `ModuleNotFoundError: No module named '_bz2'` (see [1]).
> 
> I'm not very proficient with ptxdist, so I would like to check if the solution I came up is correct. Feedback welcome. If solution is ok, feel free to apply it upstream or ask for a proper email patch.
> 
> I think it's due to following issues:
> - The host python is missing the option to support BZIP2. I added an option for that.
> - nodejs.in says it depends on HOST_SYSTEM_PYTHON3, however it seems like the build isn't using system python but host python built by ptxdist. I think it should depend on HOST_PYTHON3 and the new HOST_PYTHON3_BZIP2.
> 
> See [2] for the patch I applied.

Python in PTXdist is a bit tricky. We don't want to compile Python itself
just to run a few Python scripts at build-time. That's what
"host-system-python3" is for. It does not build anything, it just checks
if Python3 and required packages are installed.

"host-python3" is used when building Python packages for the target.

In this case, it looks like the wrong Python is used. Which ptxdist version
are you using?

Michael

> Kind regards,
> Jean-Claude
> 
> 
> [1] Error message
> 
> ----------------------
> target: nodejs.prepare
> ----------------------
> 
> Node.js configure: Found Python 3.11.2...
> Traceback (most recent call last):
>   File "/home/jean-claude/code/test-ptxdist/platform-nanopi-rk3588/build-target/node-v18.13.0/./configure", line 28, in <module>
>     import configure
>   File "/home/jean-claude/code/test-ptxdist/platform-nanopi-rk3588/build-target/node-v18.13.0/configure.py", line 14, in <module>
>     import bz2
>   File "/home/jean-claude/code/test-ptxdist/platform-nanopi-rk3588/sysroot-target/usr/lib/python3.11/bz2.py", line 17, in <module>
>     from _bz2 import BZ2Compressor, BZ2Decompressor
> ModuleNotFoundError: No module named '_bz2'
> 
> 
> [2] possible patch
> 
> diff --git a/rules/host-python3.in b/rules/host-python3.in
> index e15b90571..64f34affa 100644
> --- a/rules/host-python3.in
> +++ b/rules/host-python3.in
> @@ -6,6 +6,7 @@ config HOST_PYTHON3
>         select HOST_LIBFFI
>         select HOST_OPENSSL
>         select HOST_ZLIB                if HOST_PYTHON3_ZLIB
> +       select HOST_BZIP2               if HOST_PYTHON3_BZIP2
>         default y if ALLYES
>         help
>           Python is an interpreted, interactive,
> @@ -16,4 +17,7 @@ if HOST_PYTHON3
>  config HOST_PYTHON3_ZLIB
>         bool
>  
> +config HOST_PYTHON3_BZIP2
> +       bool
> +
>  endif
> diff --git a/rules/nodejs.in b/rules/nodejs.in
> index 80c6aec21..c75480953 100644
> --- a/rules/nodejs.in
> +++ b/rules/nodejs.in
> @@ -3,7 +3,8 @@
>  config NODEJS
>         tristate
>         prompt "Node.js                       "
> -       select HOST_SYSTEM_PYTHON3
> +       select HOST_PYTHON3
> +       select HOST_PYTHON3_BZIP2
>         select HOST_QEMU
>         select HOST_QEMU_USR
>         select C_ARES
> 
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] Error building nodejs in 2023.10.0

[Jean-Claude Monnin]

Hi Michael,

Thanks for the reply. I'm using ptxdist-2023.10.0.
Python is called by the node build system, and it looks like it was using the host python in my case. Possibly it's because another package selects host python; maybe it works if host python isn't there at all.
I'm happy with the workaround, but I see your point of wanting to use system python instead. Is there any way to check/fix why it selects host python on my system?

Jean-Claude

On Fri, Nov 10, 2023, at 9:00 AM, Michael Olbrich wrote:
> Hallo,
>
> On Wed, Nov 01, 2023 at 05:04:30PM +0100, Jean-Claude Monnin wrote:
>> When trying to build nodejs with the latest ptxdist, I faced an error about missing a python module `ModuleNotFoundError: No module named '_bz2'` (see [1]).
>> 
>> I'm not very proficient with ptxdist, so I would like to check if the solution I came up is correct. Feedback welcome. If solution is ok, feel free to apply it upstream or ask for a proper email patch.
>> 
>> I think it's due to following issues:
>> - The host python is missing the option to support BZIP2. I added an option for that.
>> - nodejs.in says it depends on HOST_SYSTEM_PYTHON3, however it seems like the build isn't using system python but host python built by ptxdist. I think it should depend on HOST_PYTHON3 and the new HOST_PYTHON3_BZIP2.
>> 
>> See [2] for the patch I applied.
>
> Python in PTXdist is a bit tricky. We don't want to compile Python itself
> just to run a few Python scripts at build-time. That's what
> "host-system-python3" is for. It does not build anything, it just checks
> if Python3 and required packages are installed.
>
> "host-python3" is used when building Python packages for the target.
>
> In this case, it looks like the wrong Python is used. Which ptxdist version
> are you using?
>
> Michael
>
>> Kind regards,
>> Jean-Claude
>> 
>> 
>> [1] Error message
>> 
>> ----------------------
>> target: nodejs.prepare
>> ----------------------
>> 
>> Node.js configure: Found Python 3.11.2...
>> Traceback (most recent call last):
>>   File "/home/jean-claude/code/test-ptxdist/platform-nanopi-rk3588/build-target/node-v18.13.0/./configure", line 28, in <module>
>>     import configure
>>   File "/home/jean-claude/code/test-ptxdist/platform-nanopi-rk3588/build-target/node-v18.13.0/configure.py", line 14, in <module>
>>     import bz2
>>   File "/home/jean-claude/code/test-ptxdist/platform-nanopi-rk3588/sysroot-target/usr/lib/python3.11/bz2.py", line 17, in <module>
>>     from _bz2 import BZ2Compressor, BZ2Decompressor
>> ModuleNotFoundError: No module named '_bz2'
>> 
>> 
>> [2] possible patch
>> 
>> diff --git a/rules/host-python3.in b/rules/host-python3.in
>> index e15b90571..64f34affa 100644
>> --- a/rules/host-python3.in
>> +++ b/rules/host-python3.in
>> @@ -6,6 +6,7 @@ config HOST_PYTHON3
>>         select HOST_LIBFFI
>>         select HOST_OPENSSL
>>         select HOST_ZLIB                if HOST_PYTHON3_ZLIB
>> +       select HOST_BZIP2               if HOST_PYTHON3_BZIP2
>>         default y if ALLYES
>>         help
>>           Python is an interpreted, interactive,
>> @@ -16,4 +17,7 @@ if HOST_PYTHON3
>>  config HOST_PYTHON3_ZLIB
>>         bool
>>  
>> +config HOST_PYTHON3_BZIP2
>> +       bool
>> +
>>  endif
>> diff --git a/rules/nodejs.in b/rules/nodejs.in
>> index 80c6aec21..c75480953 100644
>> --- a/rules/nodejs.in
>> +++ b/rules/nodejs.in
>> @@ -3,7 +3,8 @@
>>  config NODEJS
>>         tristate
>>         prompt "Node.js                       "
>> -       select HOST_SYSTEM_PYTHON3
>> +       select HOST_PYTHON3
>> +       select HOST_PYTHON3_BZIP2
>>         select HOST_QEMU
>>         select HOST_QEMU_USR
>>         select C_ARES
>> 
>> 
>
> -- 
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

[ptxdist] [PATCH] nodejs: explicitly call configure with 'python3'

[Michael Olbrich]

This will result in explicitly using the system Python. Otherwise
host-python may be used instead and it may lack support for bz2.

Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
---

Can you try if this patch helps?

Michael

 rules/nodejs.make | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/rules/nodejs.make b/rules/nodejs.make
index 0053f0d71dbb..35521227536a 100644
--- a/rules/nodejs.make
+++ b/rules/nodejs.make
@@ -83,7 +83,8 @@ endif
 $(STATEDIR)/nodejs.prepare:
 	@$(call targetinfo)
 
-	@$(call world/prepare, NODEJS)
+	@$(call world/execute, NODEJS, \
+		python3 ./configure $(NODEJS_CONF_OPT))
 	@mkdir -p $(NODEJS_DIR)/out/Release/
 	@ ln -svf $(PTXDIST_SYSROOT_CROSS)/usr/bin/qemu-cross \
 		$(NODEJS_DIR)/out/Release/tool-wrapper
-- 
2.39.2