[ptxdist] Segmentation fault with ptxdist-2020.05.0

[ptxdist] Segmentation fault with ptxdist-2020.05.0

[Mircea Ciocan]

Hello list,

I have the most strange error with the ptxdist-2020.05.0 while trying to 
compile a simple image for the iMX6ULL EVK, the shell segfaults !!!

Here are some information, that you could find useful:

Distribution: Ubuntu 18.04 LTS, 8GB RAM, current updates.

System shell set to bash.

The compilation of everything goes perfectly OK, but while building the 
root filesystem image it crashes, always in the same place (100% 
reproducible), when adding the net-dns-perl package, with the following 
message;

info: selected ptxconfig:
       'configs/platform-mcimx6ull-evk/ptxconfig.tiny'
ptxdist: warning: No dev packages found in 'Firmware/packages-MCIMX6ULL-EVK'

...[snip]...

[compilation goes OK]

----------------
target: root.tgz
----------------
...[snip]...

Configuring readline.
Configuring net-dns-perl.
environment: line 1: 31523 Done                    { echo "cd 
'${work_dir}' || exit"; ptxd_dopermissions 
"${ptxd_reply_perm_files[@]}"; echo ":"; }
      31524 Segmentation fault      (core dumped) | sh
./Software/ptxdist-2020.05.0/rules/image-root-tgz.make:29: recipe for 
target './Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz' failed

make: *** [./Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz] 
Error 139


In the system log I get something like:

"vmunix: [132753.581596] sh[10747]: segfault at 7ffe5bdf1fe8 ip 
00007fdb0b04930e sp 00007ffe5bdf1fe0 error 6 in 
libc-2.27.so[7fdb0afb5000+1e7000]"


I have to say that I rarely, if ever, see this type of crash, and the 
build finishes OK with ptxdist-2020.02.0. Any pointer on how can I fix 
or debug this are greatly appreciated.


  Best regards,

  Mircea



_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Segmentation fault with ptxdist-2020.05.0

[Mircea Ciocan]

More information about the subject:

I've repeated the compilation with all releases since 2020.02.0, the 
same config and machine as described under (sorry for top posting):

- 2020.02.0 - works OK

-2020.03.0 - works OK

-2020.04.0 - crashes, bash segfault.

-2020.05.0 - crashes, same bash segfault, in the same place.

So the regression is between 2020.03.0 and 2020.04.0, I'll try a diff, 
maybe I can spot what went wrong, but I could use your kind help.

The main problem is that the older releases as 2020.05.0 doesn't compile 
the toolchain correctly.

Best regards,

Mircea


On 14.05.20 15:36, Mircea Ciocan wrote:
> Hello list,
>
> I have the most strange error with the ptxdist-2020.05.0 while trying 
> to compile a simple image for the iMX6ULL EVK, the shell segfaults !!!
>
> Here are some information, that you could find useful:
>
> Distribution: Ubuntu 18.04 LTS, 8GB RAM, current updates.
>
> System shell set to bash.
>
> The compilation of everything goes perfectly OK, but while building 
> the root filesystem image it crashes, always in the same place (100% 
> reproducible), when adding the net-dns-perl package, with the 
> following message;
>
> info: selected ptxconfig:
>       'configs/platform-mcimx6ull-evk/ptxconfig.tiny'
> ptxdist: warning: No dev packages found in 
> 'Firmware/packages-MCIMX6ULL-EVK'
>
> ...[snip]...
>
> [compilation goes OK]
>
> ----------------
> target: root.tgz
> ----------------
> ...[snip]...
>
> Configuring readline.
> Configuring net-dns-perl.
> environment: line 1: 31523 Done                    { echo "cd 
> '${work_dir}' || exit"; ptxd_dopermissions 
> "${ptxd_reply_perm_files[@]}"; echo ":"; }
>      31524 Segmentation fault      (core dumped) | sh
> ./Software/ptxdist-2020.05.0/rules/image-root-tgz.make:29: recipe for 
> target './Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz' 
> failed
>
> make: *** [./Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz] 
> Error 139
>
>
> In the system log I get something like:
>
> "vmunix: [132753.581596] sh[10747]: segfault at 7ffe5bdf1fe8 ip 
> 00007fdb0b04930e sp 00007ffe5bdf1fe0 error 6 in 
> libc-2.27.so[7fdb0afb5000+1e7000]"
>
>
> I have to say that I rarely, if ever, see this type of crash, and the 
> build finishes OK with ptxdist-2020.02.0. Any pointer on how can I fix 
> or debug this are greatly appreciated.
>
>
>  Best regards,
>
>  Mircea



_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Segmentation fault with ptxdist-2020.05.0

[Michael Olbrich]

On Fri, May 15, 2020 at 09:40:45AM +0200, Mircea Ciocan wrote:
> More information about the subject:
> 
> I've repeated the compilation with all releases since 2020.02.0, the same
> config and machine as described under (sorry for top posting):
> 
> - 2020.02.0 - works OK
> 
> -2020.03.0 - works OK
> 
> -2020.04.0 - crashes, bash segfault.
> 
> -2020.05.0 - crashes, same bash segfault, in the same place.
> 
> So the regression is between 2020.03.0 and 2020.04.0, I'll try a diff, maybe
> I can spot what went wrong, but I could use your kind help.

I've mentioned the 'tee' in my other mail. The output there is probably
what you want to diff.

Michael

> The main problem is that the older releases as 2020.05.0 doesn't compile the
> toolchain correctly.
> 
> Best regards,
> 
> Mircea
> 
> 
> On 14.05.20 15:36, Mircea Ciocan wrote:
> > Hello list,
> > 
> > I have the most strange error with the ptxdist-2020.05.0 while trying to
> > compile a simple image for the iMX6ULL EVK, the shell segfaults !!!
> > 
> > Here are some information, that you could find useful:
> > 
> > Distribution: Ubuntu 18.04 LTS, 8GB RAM, current updates.
> > 
> > System shell set to bash.
> > 
> > The compilation of everything goes perfectly OK, but while building the
> > root filesystem image it crashes, always in the same place (100%
> > reproducible), when adding the net-dns-perl package, with the following
> > message;
> > 
> > info: selected ptxconfig:
> >       'configs/platform-mcimx6ull-evk/ptxconfig.tiny'
> > ptxdist: warning: No dev packages found in
> > 'Firmware/packages-MCIMX6ULL-EVK'
> > 
> > ...[snip]...
> > 
> > [compilation goes OK]
> > 
> > ----------------
> > target: root.tgz
> > ----------------
> > ...[snip]...
> > 
> > Configuring readline.
> > Configuring net-dns-perl.
> > environment: line 1: 31523 Done                    { echo "cd
> > '${work_dir}' || exit"; ptxd_dopermissions
> > "${ptxd_reply_perm_files[@]}"; echo ":"; }
> >      31524 Segmentation fault      (core dumped) | sh
> > ./Software/ptxdist-2020.05.0/rules/image-root-tgz.make:29: recipe for
> > target './Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz'
> > failed
> > 
> > make: *** [./Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz]
> > Error 139
> > 
> > 
> > In the system log I get something like:
> > 
> > "vmunix: [132753.581596] sh[10747]: segfault at 7ffe5bdf1fe8 ip
> > 00007fdb0b04930e sp 00007ffe5bdf1fe0 error 6 in
> > libc-2.27.so[7fdb0afb5000+1e7000]"
> > 
> > 
> > I have to say that I rarely, if ever, see this type of crash, and the
> > build finishes OK with ptxdist-2020.02.0. Any pointer on how can I fix
> > or debug this are greatly appreciated.
> > 
> > 
> >  Best regards,
> > 
> >  Mircea
> 
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Segmentation fault with ptxdist-2020.05.0

[Alexander Dahl]

[-- Attachment #1.1: Type: text/plain, Size: 963 bytes --]

Hello Mircea,

On Fri, May 15, 2020 at 09:40:45AM +0200, Mircea Ciocan wrote:
> The main problem is that the older releases as 2020.05.0 doesn't compile the
> toolchain correctly.

Which toolchain? If you compile the OSELAS Toolchain by yourself
(instead of e.g. using the prepackaged binary toolchains from [1]),
you should use _exactly_ the ptxdist version the toolchain project is
configured for. For example if you want to build OSELAS Toolchain
v2019.09.1 you should use exactly ptxdist-2020.02.0 to build that
toolchain.

You can (and probably should) use a newer ptxdist version to build
your BSP then.

Greets
Alex

[1] http://debian.pengutronix.de/

-- 
/"\ ASCII RIBBON | »With the first link, the chain is forged. The first
\ / CAMPAIGN     | speech censured, the first thought forbidden, the
 X  AGAINST      | first freedom denied, chains us all irrevocably.«
/ \ HTML MAIL    | (Jean-Luc Picard, quoting Judge Aaron Satie)

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 92 bytes --]

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Segmentation fault with ptxdist-2020.05.0

[Mircea Ciocan]

On 15.05.20 09:59, Alexander Dahl wrote:
> Hello Mircea,
>
> On Fri, May 15, 2020 at 09:40:45AM +0200, Mircea Ciocan wrote:
>> The main problem is that the older releases as 2020.05.0 doesn't compile the
>> toolchain correctly.
> Which toolchain? If you compile the OSELAS Toolchain by yourself
> (instead of e.g. using the prepackaged binary toolchains from [1]),
> you should use _exactly_ the ptxdist version the toolchain project is
> configured for. For example if you want to build OSELAS Toolchain
> v2019.09.1 you should use exactly ptxdist-2020.02.0 to build that
> toolchain.

This is exactly what I was doing, I've used 2020.02 because it was 
around for the toolchain, but wanted the updated stuff in the newer 
versions.

> You can (and probably should) use a newer ptxdist version to build
> your BSP then.

I've tried that and kind of failed as described :(, now I'll retry with 
the hint from Michael for data collection and see what's coming of it. 
Is not every day that one sees this kind of bug in bash.


> Greets
> Alex

Cheers,

Mircea




_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Segmentation fault with ptxdist-2020.05.0

[Michael Olbrich]

On Thu, May 14, 2020 at 03:36:44PM +0200, Mircea Ciocan wrote:
> Hello list,
> 
> I have the most strange error with the ptxdist-2020.05.0 while trying to
> compile a simple image for the iMX6ULL EVK, the shell segfaults !!!
> 
> Here are some information, that you could find useful:
> 
> Distribution: Ubuntu 18.04 LTS, 8GB RAM, current updates.
> 
> System shell set to bash.
> 
> The compilation of everything goes perfectly OK, but while building the root
> filesystem image it crashes, always in the same place (100% reproducible),
> when adding the net-dns-perl package, with the following message;
> 
> info: selected ptxconfig:
>       'configs/platform-mcimx6ull-evk/ptxconfig.tiny'
> ptxdist: warning: No dev packages found in 'Firmware/packages-MCIMX6ULL-EVK'
> 
> ...[snip]...
> 
> [compilation goes OK]
> 
> ----------------
> target: root.tgz
> ----------------
> ...[snip]...
> 
> Configuring readline.
> Configuring net-dns-perl.
> environment: line 1: 31523 Done                    { echo "cd '${work_dir}'
> || exit"; ptxd_dopermissions "${ptxd_reply_perm_files[@]}"; echo ":"; }
>      31524 Segmentation fault      (core dumped) | sh
> ./Software/ptxdist-2020.05.0/rules/image-root-tgz.make:29: recipe for target
> './Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz' failed
> 
> make: *** [./Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz] Error
> 139
> 
> 
> In the system log I get something like:
> 
> "vmunix: [132753.581596] sh[10747]: segfault at 7ffe5bdf1fe8 ip
> 00007fdb0b04930e sp 00007ffe5bdf1fe0 error 6 in
> libc-2.27.so[7fdb0afb5000+1e7000]"
> 
> 
> I have to say that I rarely, if ever, see this type of crash, and the build
> finishes OK with ptxdist-2020.02.0. Any pointer on how can I fix or debug
> this are greatly appreciated.

Huh, I've never seen anything like this. Some things to test.

Take a look at scripts/lib/ptxd_make_image_prepare_work_dir.sh in PTXdist.
The relevant code is there. Line 50ff.
If you change line 56 to something like this:

} | tee /tmp/ptxdist-sh-test.txt | sh &&

Then this will save the exact code that the shell executes when it crashed.
Maybe we can find some clues there. But It's probably a bug in 'sh'.

What's your /bin/sh anyways? For me it's a link to 'dash' but sometimes
it's a bash. You could explicitly try the other here instead of 'sh' to see
if that crashes as well.

Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Segmentation fault with ptxdist-2020.05.0

[Mircea Ciocan]

On 15.05.20 09:46, Michael Olbrich wrote:
> On Thu, May 14, 2020 at 03:36:44PM +0200, Mircea Ciocan wrote:
>> Hello list,
>>
>> I have the most strange error with the ptxdist-2020.05.0 while trying to
>> compile a simple image for the iMX6ULL EVK, the shell segfaults !!!
>>
>> Here are some information, that you could find useful:
>>
>> Distribution: Ubuntu 18.04 LTS, 8GB RAM, current updates.
>>
>> System shell set to bash.
>>
>> The compilation of everything goes perfectly OK, but while building the root
>> filesystem image it crashes, always in the same place (100% reproducible),
>> when adding the net-dns-perl package, with the following message;
>>
>> info: selected ptxconfig:
>>        'configs/platform-mcimx6ull-evk/ptxconfig.tiny'
>> ptxdist: warning: No dev packages found in 'Firmware/packages-MCIMX6ULL-EVK'
>>
>> ...[snip]...
>>
>> [compilation goes OK]
>>
>> ----------------
>> target: root.tgz
>> ----------------
>> ...[snip]...
>>
>> Configuring readline.
>> Configuring net-dns-perl.
>> environment: line 1: 31523 Done                    { echo "cd '${work_dir}'
>> || exit"; ptxd_dopermissions "${ptxd_reply_perm_files[@]}"; echo ":"; }
>>       31524 Segmentation fault      (core dumped) | sh
>> ./Software/ptxdist-2020.05.0/rules/image-root-tgz.make:29: recipe for target
>> './Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz' failed
>>
>> make: *** [./Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz] Error
>> 139
>>
>>
>> In the system log I get something like:
>>
>> "vmunix: [132753.581596] sh[10747]: segfault at 7ffe5bdf1fe8 ip
>> 00007fdb0b04930e sp 00007ffe5bdf1fe0 error 6 in
>> libc-2.27.so[7fdb0afb5000+1e7000]"
>>
>>
>> I have to say that I rarely, if ever, see this type of crash, and the build
>> finishes OK with ptxdist-2020.02.0. Any pointer on how can I fix or debug
>> this are greatly appreciated.
> Huh, I've never seen anything like this. Some things to test.
>
> Take a look at scripts/lib/ptxd_make_image_prepare_work_dir.sh in PTXdist.
> The relevant code is there. Line 50ff.
> If you change line 56 to something like this:
>
> } | tee /tmp/ptxdist-sh-test.txt | sh &&
>
> Then this will save the exact code that the shell executes when it crashed.
> Maybe we can find some clues there. But It's probably a bug in 'sh'.
>
> What's your /bin/sh anyways? For me it's a link to 'dash' but sometimes
> it's a bash. You could explicitly try the other here instead of 'sh' to see
> if that crashes as well.
>
> Michael
>
Hello Michael and thanks for the hint, first things first, the shell is 
the GNU bash, I've change it via the update-alternatives, because more 
than one time I've had issues with scripts that call /bin/sh, but assume 
that is bash.

The logging tap collected the following file:

"cd 
'./Software/Firmware/platform-MCIMX6ULL-EVK/build-target/image-root-tgz' 
|| exit
chmod 0755    './' &&
chown 0.0    './' &&
chmod 0755    './usr' &&
chown 0.0    './usr' &&
chmod 0755    './usr/sbin' &&
chown 0.0    './usr/sbin' &&

...

!!! Another >>>71940<<< similar lines !!!

...

chmod 0755    './usr/bin/cpuburn-a9' &&
chown 0.0    './usr/bin/cpuburn-a9' &&
:"

Is this just a "cool way" to execute a quatrillion of commands, by 
exploiting to the limit the AND_IF (&&) construct to exit on error, or 
there is a deeper purpose ?

I could see that some other cool programmer tried some optimization 
somewhere in bash and these two worlds collided hard, now that full Perl 
and Python packages adds a few tens of thousands of files to the image 
that need their permissions fixed.

I'll have a look if there was another way to do it (how was it done in 
2020.03.0 and lower, could it be that only two lines have been used 
like: chmod -R * && chown -R 0.0 * und fertig ;) ?!?) and see if I could 
reproduce it, to make an Ubuntu bug report.

Other ideas and solutions are most welcome.


Thanks for your help,

Mircea



_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Segmentation fault with ptxdist-2020.05.0

[Mircea Ciocan]

On 15.05.20 15:02, Mircea Ciocan wrote:
>
> On 15.05.20 09:46, Michael Olbrich wrote:
>> On Thu, May 14, 2020 at 03:36:44PM +0200, Mircea Ciocan wrote:
>>> Hello list,
>>>
>>> I have the most strange error with the ptxdist-2020.05.0 while 
>>> trying to
>>> compile a simple image for the iMX6ULL EVK, the shell segfaults !!!
>>>
>>> Here are some information, that you could find useful:
>>>
>>> Distribution: Ubuntu 18.04 LTS, 8GB RAM, current updates.
>>>
>>> System shell set to bash.
>>>
>>> The compilation of everything goes perfectly OK, but while building 
>>> the root
>>> filesystem image it crashes, always in the same place (100% 
>>> reproducible),
>>> when adding the net-dns-perl package, with the following message;
>>>
>>> info: selected ptxconfig:
>>>        'configs/platform-mcimx6ull-evk/ptxconfig.tiny'
>>> ptxdist: warning: No dev packages found in 
>>> 'Firmware/packages-MCIMX6ULL-EVK'
>>>
>>> ...[snip]...
>>>
>>> [compilation goes OK]
>>>
>>> ----------------
>>> target: root.tgz
>>> ----------------
>>> ...[snip]...
>>>
>>> Configuring readline.
>>> Configuring net-dns-perl.
>>> environment: line 1: 31523 Done                    { echo "cd 
>>> '${work_dir}'
>>> || exit"; ptxd_dopermissions "${ptxd_reply_perm_files[@]}"; echo ":"; }
>>>       31524 Segmentation fault      (core dumped) | sh
>>> ./Software/ptxdist-2020.05.0/rules/image-root-tgz.make:29: recipe 
>>> for target
>>> './Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz' failed
>>>
>>> make: *** 
>>> [./Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz] Error
>>> 139
>>>
>>>
>>> In the system log I get something like:
>>>
>>> "vmunix: [132753.581596] sh[10747]: segfault at 7ffe5bdf1fe8 ip
>>> 00007fdb0b04930e sp 00007ffe5bdf1fe0 error 6 in
>>> libc-2.27.so[7fdb0afb5000+1e7000]"
>>>
>>>
>>> I have to say that I rarely, if ever, see this type of crash, and 
>>> the build
>>> finishes OK with ptxdist-2020.02.0. Any pointer on how can I fix or 
>>> debug
>>> this are greatly appreciated.
>> Huh, I've never seen anything like this. Some things to test.
>>
>> Take a look at scripts/lib/ptxd_make_image_prepare_work_dir.sh in 
>> PTXdist.
>> The relevant code is there. Line 50ff.
>> If you change line 56 to something like this:
>>
>> } | tee /tmp/ptxdist-sh-test.txt | sh &&
>>
>> Then this will save the exact code that the shell executes when it 
>> crashed.
>> Maybe we can find some clues there. But It's probably a bug in 'sh'.
>>
>> What's your /bin/sh anyways? For me it's a link to 'dash' but sometimes
>> it's a bash. You could explicitly try the other here instead of 'sh' 
>> to see
>> if that crashes as well.
>>
>> Michael
>>
> Hello Michael and thanks for the hint, first things first, the shell 
> is the GNU bash, I've change it via the update-alternatives, because 
> more than one time I've had issues with scripts that call /bin/sh, but 
> assume that is bash.
>
> The logging tap collected the following file:
>
> "cd 
> './Software/Firmware/platform-MCIMX6ULL-EVK/build-target/image-root-tgz' 
> || exit
> chmod 0755    './' &&
> chown 0.0    './' &&
> chmod 0755    './usr' &&
> chown 0.0    './usr' &&
> chmod 0755    './usr/sbin' &&
> chown 0.0    './usr/sbin' &&
>
> ...
>
> !!! Another >>>71940<<< similar lines !!!
>
> ...
>
> chmod 0755    './usr/bin/cpuburn-a9' &&
> chown 0.0    './usr/bin/cpuburn-a9' &&
> :"
>
> Is this just a "cool way" to execute a quatrillion of commands, by 
> exploiting to the limit the AND_IF (&&) construct to exit on error, or 
> there is a deeper purpose ?
>
> I could see that some other cool programmer tried some optimization 
> somewhere in bash and these two worlds collided hard, now that full 
> Perl and Python packages adds a few tens of thousands of files to the 
> image that need their permissions fixed.
>
> I'll have a look if there was another way to do it (how was it done in 
> 2020.03.0 and lower, could it be that only two lines have been used 
> like: chmod -R * && chown -R 0.0 * und fertig ;) ?!?) and see if I 
> could reproduce it, to make an Ubuntu bug report.
>
> Other ideas and solutions are most welcome.
>
>
> Thanks for your help,
>
> Mircea 

OK, latest update, is definitly a (Ubuntu) bash bug, I've run the 
mega-list of commands with bash and dash, bash crashes immediately, dash 
takes a mounstrously large amount of time (almost 5min on a beefy 
machine with fast SSD), so it is clear why Michael did't see the bug.
A bit of stracing show something like a lot of threads dieing, so 
probably bash tries to execute this in parallel and hits something, just 
FYI here the last words of bash run strace:

read(255, "mod 0755\t'./usr' &&\nchown 0.0\t'."..., 8192) = 8192
brk(0x56519329b000)                     = 0x56519329b000
read(255, "tch_map' &&\nchmod 0755\t'./' &&\nc"..., 8192) = 2374
brk(0x5651932bc000)                     = 0x5651932bc000
brk(0x5651932dd000)                     = 0x5651932dd000
.... maaaany more similar lines, with different addresses ....
brk(0x565193805000)                     = 0x565193805000
brk(0x565193826000)                     = 0x565193826000
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, 
si_addr=0x7ffef981df98} ---
+++ killed by SIGSEGV (core dumped) +++
Segmentation fault (core dumped)

So because switching back to dash here it's a bit difficult, is there 
any way that this function could be made a bit more bash-friendly ?
I could gladly test your patches and ideas (vanilla Ubuntu 18.04, with 
bash as system shell, is not such a seldom encountered configuration), 
but I'm afraid that I'm not in the deep of the PTXdist internals to 
start hacking and don't produce conequences.

  Best regards,
  Mircea




_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] Segmentation fault with ptxdist-2020.05.0

[Michael Olbrich]

On Fri, May 15, 2020 at 03:36:30PM +0200, Mircea Ciocan wrote:
> 
> On 15.05.20 15:02, Mircea Ciocan wrote:
> > 
> > On 15.05.20 09:46, Michael Olbrich wrote:
> > > On Thu, May 14, 2020 at 03:36:44PM +0200, Mircea Ciocan wrote:
> > > > Hello list,
> > > > 
> > > > I have the most strange error with the ptxdist-2020.05.0 while
> > > > trying to
> > > > compile a simple image for the iMX6ULL EVK, the shell segfaults !!!
> > > > 
> > > > Here are some information, that you could find useful:
> > > > 
> > > > Distribution: Ubuntu 18.04 LTS, 8GB RAM, current updates.
> > > > 
> > > > System shell set to bash.
> > > > 
> > > > The compilation of everything goes perfectly OK, but while
> > > > building the root
> > > > filesystem image it crashes, always in the same place (100%
> > > > reproducible),
> > > > when adding the net-dns-perl package, with the following message;
> > > > 
> > > > info: selected ptxconfig:
> > > >        'configs/platform-mcimx6ull-evk/ptxconfig.tiny'
> > > > ptxdist: warning: No dev packages found in
> > > > 'Firmware/packages-MCIMX6ULL-EVK'
> > > > 
> > > > ...[snip]...
> > > > 
> > > > [compilation goes OK]
> > > > 
> > > > ----------------
> > > > target: root.tgz
> > > > ----------------
> > > > ...[snip]...
> > > > 
> > > > Configuring readline.
> > > > Configuring net-dns-perl.
> > > > environment: line 1: 31523 Done                    { echo "cd
> > > > '${work_dir}'
> > > > || exit"; ptxd_dopermissions "${ptxd_reply_perm_files[@]}"; echo ":"; }
> > > >       31524 Segmentation fault      (core dumped) | sh
> > > > ./Software/ptxdist-2020.05.0/rules/image-root-tgz.make:29:
> > > > recipe for target
> > > > './Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz' failed
> > > > 
> > > > make: ***
> > > > [./Software/Firmware/platform-MCIMX6ULL-EVK/images/root.tgz]
> > > > Error
> > > > 139
> > > > 
> > > > 
> > > > In the system log I get something like:
> > > > 
> > > > "vmunix: [132753.581596] sh[10747]: segfault at 7ffe5bdf1fe8 ip
> > > > 00007fdb0b04930e sp 00007ffe5bdf1fe0 error 6 in
> > > > libc-2.27.so[7fdb0afb5000+1e7000]"
> > > > 
> > > > 
> > > > I have to say that I rarely, if ever, see this type of crash,
> > > > and the build
> > > > finishes OK with ptxdist-2020.02.0. Any pointer on how can I fix
> > > > or debug
> > > > this are greatly appreciated.
> > > Huh, I've never seen anything like this. Some things to test.
> > > 
> > > Take a look at scripts/lib/ptxd_make_image_prepare_work_dir.sh in
> > > PTXdist.
> > > The relevant code is there. Line 50ff.
> > > If you change line 56 to something like this:
> > > 
> > > } | tee /tmp/ptxdist-sh-test.txt | sh &&
> > > 
> > > Then this will save the exact code that the shell executes when it
> > > crashed.
> > > Maybe we can find some clues there. But It's probably a bug in 'sh'.
> > > 
> > > What's your /bin/sh anyways? For me it's a link to 'dash' but sometimes
> > > it's a bash. You could explicitly try the other here instead of 'sh'
> > > to see
> > > if that crashes as well.
> > > 
> > > Michael
> > > 
> > Hello Michael and thanks for the hint, first things first, the shell is
> > the GNU bash, I've change it via the update-alternatives, because more
> > than one time I've had issues with scripts that call /bin/sh, but assume
> > that is bash.
> > 
> > The logging tap collected the following file:
> > 
> > "cd
> > './Software/Firmware/platform-MCIMX6ULL-EVK/build-target/image-root-tgz'
> > || exit
> > chmod 0755    './' &&
> > chown 0.0    './' &&
> > chmod 0755    './usr' &&
> > chown 0.0    './usr' &&
> > chmod 0755    './usr/sbin' &&
> > chown 0.0    './usr/sbin' &&
> > 
> > ...
> > 
> > !!! Another >>>71940<<< similar lines !!!
> > 
> > ...
> > 
> > chmod 0755    './usr/bin/cpuburn-a9' &&
> > chown 0.0    './usr/bin/cpuburn-a9' &&
> > :"
> > 
> > Is this just a "cool way" to execute a quatrillion of commands, by
> > exploiting to the limit the AND_IF (&&) construct to exit on error, or
> > there is a deeper purpose ?

It's just there to catch error. There shouldn't be any, but we don't want
to ignore the either.

> > I could see that some other cool programmer tried some optimization
> > somewhere in bash and these two worlds collided hard, now that full Perl
> > and Python packages adds a few tens of thousands of files to the image
> > that need their permissions fixed.
> > 
> > I'll have a look if there was another way to do it (how was it done in
> > 2020.03.0 and lower, could it be that only two lines have been used
> > like: chmod -R * && chown -R 0.0 * und fertig ;) ?!?) and see if I could

No, not all files have the same ownership/permissions. We save those from
the 'instal_*' commands in targetinstall and reapply here.
In this case, not everything is actually needed. So maybe we can shorten
the list.

> > reproduce it, to make an Ubuntu bug report.
> > 
> > Other ideas and solutions are most welcome.
> > 
> > 
> > Thanks for your help,
> > 
> > Mircea
> 
> OK, latest update, is definitly a (Ubuntu) bash bug, I've run the mega-list
> of commands with bash and dash, bash crashes immediately, dash takes a
> mounstrously large amount of time (almost 5min on a beefy machine with fast
> SSD), so it is clear why Michael did't see the bug.
> A bit of stracing show something like a lot of threads dieing, so probably
> bash tries to execute this in parallel and hits something, just FYI here the
> last words of bash run strace:
> 
> read(255, "mod 0755\t'./usr' &&\nchown 0.0\t'."..., 8192) = 8192
> brk(0x56519329b000)                     = 0x56519329b000
> read(255, "tch_map' &&\nchmod 0755\t'./' &&\nc"..., 8192) = 2374
> brk(0x5651932bc000)                     = 0x5651932bc000
> brk(0x5651932dd000)                     = 0x5651932dd000
> .... maaaany more similar lines, with different addresses ....
> brk(0x565193805000)                     = 0x565193805000
> brk(0x565193826000)                     = 0x565193826000
> --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x7ffef981df98}
> ---
> +++ killed by SIGSEGV (core dumped) +++
> Segmentation fault (core dumped)
> 
> So because switching back to dash here it's a bit difficult, is there any
> way that this function could be made a bit more bash-friendly ?
> I could gladly test your patches and ideas (vanilla Ubuntu 18.04, with bash
> as system shell, is not such a seldom encountered configuration), but I'm
> afraid that I'm not in the deep of the PTXdist internals to start hacking
> and don't produce conequences.

Hmmm, in this case, we can probably use 'set -e' instead. But I probably
want to avoid doing most of this stuff here in the first place. I need to
think about this.

Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

[ptxdist] [PATCH] ptxd_lib_dopermissions: use 'set -e' instead of &&

[Michael Olbrich]

It seems that some bash versions cannot handle very long '&&' lists.
So in this case, just use 'set -e' to abort on error.

Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
---

Hi,

So I ended up with just using 'set -e'. The other stuff is a bit more
complex and I need to figure out what exactly I want to do here.

Does this work for you?

Michael

 scripts/lib/ptxd_lib_dopermissions.awk          | 13 +++++++++----
 scripts/lib/ptxd_make_image_prepare_work_dir.sh |  1 -
 2 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/scripts/lib/ptxd_lib_dopermissions.awk b/scripts/lib/ptxd_lib_dopermissions.awk
index b0aefe1b7a18..bef1823317c5 100755
--- a/scripts/lib/ptxd_lib_dopermissions.awk
+++ b/scripts/lib/ptxd_lib_dopermissions.awk
@@ -2,18 +2,23 @@
 
 BEGIN {
 	FS = "\x1F";
+	print("set -e")
 }
 
 $1 ~ "f" {
-	printf("chmod %s	'.%s' &&\n" \
-	       "chown %s.%s	'.%s' &&\n", \
+	printf("chmod %s	'.%s'\n" \
+	       "chown %s.%s	'.%s'\n", \
 	       $5, $2, $3, $4, $2);
 }
 
 $1 ~ "n" {
 	printf("if [ ! \\( -b '.%s' -o -c '.%s' -o -p '.%s' \\) ]; then " \
-	       "  mknod -m %s	'.%s'	%s %s %s &&\n" \
+	       "  mknod -m %s	'.%s'	%s %s %s\n" \
 	       "  chown %s.%s	'.%s'\n" \
-	       "fi &&\n", \
+	       "fi\n", \
 	       $2, $2, $2, $5, $2, $6, $7, $8, $3, $4, $2);
 }
+
+END {
+	print("set +e")
+}
diff --git a/scripts/lib/ptxd_make_image_prepare_work_dir.sh b/scripts/lib/ptxd_make_image_prepare_work_dir.sh
index 8b08397653ab..adbf7797c1b4 100644
--- a/scripts/lib/ptxd_make_image_prepare_work_dir.sh
+++ b/scripts/lib/ptxd_make_image_prepare_work_dir.sh
@@ -52,7 +52,6 @@ ${list[*]}
     {
 	echo "cd '${work_dir}' || exit"
 	ptxd_dopermissions "${ptxd_reply_perm_files[@]}"
-	echo ":"
     } | sh &&
     check_pipe_status &&
 
-- 
2.20.1


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de