[ptxdist] Python 3.5 and the use of getrandom() system call

[ptxdist] Python 3.5 and the use of getrandom() system call

[David Jander]

Hi all,

I have noticed that on embedded Linux systems, the kernel can apparently take a
significant amount of time to gather enough entropy to successfully initialize
the "nonblocking random pool". This leads to any python (3.5) scripts or
applications started on boot to hang for that time, because the python
executable uses the getrandom() system call to initialize hashing functions
and whatnot.

Theoretically there are not so many use-cases I can think of that would
require crypto-quality randomness in a python application on an embedded
system, and probably fast boot times are more important than good randomness.

For this reason I would like to know if anyone knows a better solution than
adding the following patch:

--- /dev/null
+++ b/patches/Python-3.5.0/0003-force-use-of-dev-urandom.patch
@@ -0,0 +1,33 @@
+Index: Python-3.5.0/configure.ac
+===================================================================
+--- Python-3.5.0.orig/configure.ac
++++ Python-3.5.0/configure.ac
+@@ -5123,8 +5123,7 @@ AC_LINK_IFELSE(
+ AC_MSG_RESULT($have_getrandom_syscall)
+ 
+ if test "$have_getrandom_syscall" = yes; then
+-    AC_DEFINE(HAVE_GETRANDOM_SYSCALL, 1,
+-              [Define to 1 if the Linux getrandom() syscall is available])
++    echo "WARNING: forced use of /dev/urandom over syscall getrandom()" >&AS_MESSAGE_FD
+ fi
+ 
+ # generate output files
+@@ -5148,4 +5147,5 @@ echo "creating Makefile" >&AS_MESSAGE_FD
+ $SHELL $srcdir/Modules/makesetup -c $srcdir/Modules/config.c.in \
+ 			-s Modules Modules/Setup.config \
+ 			Modules/Setup.local Modules/Setup
++
+ mv config.c Modules
+Index: Python-3.5.0/configure
+===================================================================
+--- Python-3.5.0.orig/configure
++++ Python-3.5.0/configure
+@@ -16027,7 +16027,7 @@ $as_echo "$have_getrandom_syscall" >&6;
+ 
+ if test "$have_getrandom_syscall" = yes; then
+ 
+-$as_echo "#define HAVE_GETRANDOM_SYSCALL 1" >>confdefs.h
++echo "WARNING: forced use of /dev/urandom over syscall getrandom()" >&6
+ 
+ fi
+ 

Or similar, to force python to use /dev/urandom instead.

Or is there a way (that I ignore) to speed-up the generation of entropy in the
Linux kernel?

If everyone agrees that this patch is the way to go, I'll arrange a formal
patch to be sent to this list.

Best regards,

-- 
David Jander
Protonic Holland.

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Python 3.5 and the use of getrandom() system call

[Clemens Gruber]

Hi,

On Fri, Nov 18, 2016 at 02:21:36PM +0100, David Jander wrote:
> Or is there a way (that I ignore) to speed-up the generation of entropy in the
> Linux kernel?

Did you build your kernel with CONFIG_CRYPTO_JITTERENTROPY=y and
CONFIG_CRYPTO_DRBG_HASH=y ?

Also, depending on your platform, there might be hardware support.
We use an i.MX6 which also includes a hardware RNG in its Cryptographic
Acceleration And Assurance Module (CAAM).
So we enabled CONFIG_CRYPTO_DEV_FSL_CAAM and use rngd to feed /dev/hwrng
into the kernel's entropy pool.

Could you try with above options and rngd started early? There should be
much more entropy available in a short time.

It's probably a bad idea to just patch python for every PTXdist user.
Not everybody is willing to trade security for faster start times.
Besides, not having enough entropy will lead to other problems as well.

If you are using systemd, verify that
PTXCONF_SYSTEMD_DISABLE_RANDOM_SEED is not set.
systemd-random-seed restores the random seed of the system at early boot
and saves it at shutdown. So only the first boot would take longer.

Regards,
Clemens

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Python 3.5 and the use of getrandom() system call

[Roland Hieber]

[-- Attachment #1.1.1: Type: text/plain, Size: 930 bytes --]

On 18.11.2016 18:15, Clemens Gruber wrote:
> It's probably a bad idea to just patch python for every PTXdist user.
> Not everybody is willing to trade security for faster start times.

Judging from Python-3.5.0/Python/random.c:118, it will use the
getrandom() syscall with flags=0, and according to the getrandom(2) man
page, this will read from /dev/urandom, and also block if the
nonblocking entropy pool is not initialized yet. (The man page also
mentions explicitly that the random bytes returned by it can be used for
cryptographic purposes, for all else, see Filippo Valsordas talk at 32c3
[0] ;-)) So, even if it does not solve the original problem (long boot
times), I see nothing wrong with doing this for every user.

[0]: https://www.youtube.com/watch?v=Q8JAlZ-HJQI

> Besides, not having enough entropy will lead to other problems as well.

Oh, could you please elaborate on that? :-)

 - Roland


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

[-- Attachment #2: Type: text/plain, Size: 91 bytes --]

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Python 3.5 and the use of getrandom() system call

[David Jander]

On Fri, 18 Nov 2016 18:15:58 +0100
Clemens Gruber <clemens.gruber@pqgruber.com> wrote:

> Hi,
> 
> On Fri, Nov 18, 2016 at 02:21:36PM +0100, David Jander wrote:
> > Or is there a way (that I ignore) to speed-up the generation of entropy in the
> > Linux kernel?  
> 
> Did you build your kernel with CONFIG_CRYPTO_JITTERENTROPY=y and
> CONFIG_CRYPTO_DRBG_HASH=y ?

Yes I had those.

> Also, depending on your platform, there might be hardware support.
> We use an i.MX6 which also includes a hardware RNG in its Cryptographic
> Acceleration And Assurance Module (CAAM).
> So we enabled CONFIG_CRYPTO_DEV_FSL_CAAM and use rngd to feed /dev/hwrng
> into the kernel's entropy pool.

Ah. There it is. I was already wondering if there wasn't any CAAM support. I
expected it under drivers/char/hwrng. Instead it is hidden under /crypto/.

> Could you try with above options and rngd started early? There should be
> much more entropy available in a short time.

Will try that out.

> It's probably a bad idea to just patch python for every PTXdist user.

Yes, I was thinking about making that patch configurable somehow.

> Not everybody is willing to trade security for faster start times.

I agree.

> Besides, not having enough entropy will lead to other problems as well.
> 
> If you are using systemd, verify that
> PTXCONF_SYSTEMD_DISABLE_RANDOM_SEED is not set.
> systemd-random-seed restores the random seed of the system at early boot
> and saves it at shutdown. So only the first boot would take longer.

Thanks a lot.

Best regards,

-- 
David Jander
Protonic Holland.

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Python 3.5 and the use of getrandom() system call

[Alexander Dahl]

[-- Attachment #1.1: Type: text/plain, Size: 702 bytes --]

Hei hei,

On Fri, Nov 18, 2016 at 02:21:36PM +0100, David Jander wrote:
> Or is there a way (that I ignore) to speed-up the generation of
> entropy in the Linux kernel?

I heard other people use haveged for this. I have two experimental
patches for adding a haveged package, but did not test it thouroughly
yet. Maybe someone else wants to try?

https://github.com/LeSpocky/ptxdist/tree/haveged

Greets
Alex

-- 
»With the first link, the chain is forged. The first speech censured, 
the first thought forbidden, the first freedom denied, chains us all 
irrevocably.« (Jean-Luc Picard, quoting Judge Aaron Satie)
*** GnuPG-FP: C28E E6B9 0263 95CF 8FAF  08FA 34AD CD00 7221 5CC6 ***

[-- Attachment #1.2: Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 91 bytes --]

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Python 3.5 and the use of getrandom() system call

[Michael Olbrich]

Hi,

On Tue, Nov 29, 2016 at 09:56:59AM +0100, Alexander Dahl wrote:
> On Fri, Nov 18, 2016 at 02:21:36PM +0100, David Jander wrote:
> > Or is there a way (that I ignore) to speed-up the generation of
> > entropy in the Linux kernel?

I agree with the other that such a patch is not a good idea for ptxdist
upstream.
Collecting enough entropy is not easy. But working around it like this is
imho not something that should be the default.

> I heard other people use haveged for this. I have two experimental
> patches for adding a haveged package, but did not test it thouroughly
> yet. Maybe someone else wants to try?
> 
> https://github.com/LeSpocky/ptxdist/tree/haveged

I prefer a patch (hint, hint :-))...

Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Python 3.5 and the use of getrandom() system call

[Clemens Gruber]

Hi,

On Tue, Nov 29, 2016 at 09:56:59AM +0100, Alexander Dahl wrote:
> On Fri, Nov 18, 2016 at 02:21:36PM +0100, David Jander wrote:
> > Or is there a way (that I ignore) to speed-up the generation of
> > entropy in the Linux kernel?
> 
> I heard other people use haveged for this. I have two experimental
> patches for adding a haveged package, but did not test it thouroughly
> yet. Maybe someone else wants to try?

The URL for haveged seems to be down?

From the ArchLinux wiki: "Unless you have a specific reason to not trust
any hardware random number generator on your system, you should try to
use them with the rng-tools first and if it turns out not to be enough
(or if you do not have a hardware random number generator available),
then use Haveged."

Cheers,
Clemens

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Python 3.5 and the use of getrandom() system call

[Alexander Dahl]

[-- Attachment #1.1: Type: text/plain, Size: 922 bytes --]

Hei hei,

On Tue, Nov 29, 2016 at 02:14:04PM +0100, Clemens Gruber wrote:
> The URL for haveged seems to be down?

Same here.

> From the ArchLinux wiki: "Unless you have a specific reason to not trust
> any hardware random number generator on your system, you should try to
> use them with the rng-tools first and if it turns out not to be enough
> (or if you do not have a hardware random number generator available),
> then use Haveged."

This is why I played around with it on at91samg20, which has no
hardware rng. dropbear creates its keys on first start and it's more
or less early in the boot process.

Greets
Alex

-- 
»With the first link, the chain is forged. The first speech censured, 
the first thought forbidden, the first freedom denied, chains us all 
irrevocably.« (Jean-Luc Picard, quoting Judge Aaron Satie)
*** GnuPG-FP: C28E E6B9 0263 95CF 8FAF  08FA 34AD CD00 7221 5CC6 ***

[-- Attachment #1.2: Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 91 bytes --]

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de