[ptxdist] Request for comments: CROSS_LIB_DIR handling etc.

[ptxdist] Request for comments: CROSS_LIB_DIR handling etc.

[Christian Melki]

Hello.

I've been correcting various hardcoded paths in ptxdist packages and 
whatnot lately. This started with ptxdist not coping well with 
toolchains that adhere to the ABI path for various architectures.

Yesterday, I dug into the problem that on x86_64 (/lib64), all .pc.in -> 
.pc file transformations looked broken with hardcoded paths. At first I 
thought it had something to do with autoconf or pkg-config but after a 
while I found this.

scripts/lib/ptxd_make_world_install_mangle_pc.awk

Which after a while made me realize that there is still a lot of code in 
ptxdist core stuff that assumes that lib-paths are only /lib and nothing 
else.

So. I'm presenting a two options here.

1. Fix all ptxdist core stuff, because really, ptxdist should be more 
flexible than hardcoded paths. Esp. for libs.

2. Split ptxd_get_lib_dir, because, ld.so path should not be assumed to 
be the same as main library install path. So ptxd_get_ld_lib_dir which 
does what it does today and install ld there and ptxd_get_lib_dir = /lib 
and be done with all the userspace library transformations.

So. Number one probably requires a lot of more work and a lot of 
headache. Number two should be rather straightforward, atleast in theory.

Any thoughts?

/Christian

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] Request for comments: CROSS_LIB_DIR handling etc.

[Michael Olbrich]

Hi,

On Wed, Oct 13, 2021 at 09:29:24AM +0200, Christian Melki wrote:
> I've been correcting various hardcoded paths in ptxdist packages and whatnot
> lately. This started with ptxdist not coping well with toolchains that
> adhere to the ABI path for various architectures.
> 
> Yesterday, I dug into the problem that on x86_64 (/lib64), all .pc.in -> .pc
> file transformations looked broken with hardcoded paths. At first I thought
> it had something to do with autoconf or pkg-config but after a while I found
> this.
> 
> scripts/lib/ptxd_make_world_install_mangle_pc.awk
> 
> Which after a while made me realize that there is still a lot of code in
> ptxdist core stuff that assumes that lib-paths are only /lib and nothing
> else.
> 
> So. I'm presenting a two options here.
> 
> 1. Fix all ptxdist core stuff, because really, ptxdist should be more
> flexible than hardcoded paths. Esp. for libs.
> 
> 2. Split ptxd_get_lib_dir, because, ld.so path should not be assumed to be
> the same as main library install path. So ptxd_get_ld_lib_dir which does
> what it does today and install ld there and ptxd_get_lib_dir = /lib and be
> done with all the userspace library transformations.

So, put everything in lib/ and only the ld.so in lib64/, right?

> So. Number one probably requires a lot of more work and a lot of headache.

And we'll probably break it more often.

> Number two should be rather straightforward, atleast in theory.

There will be some packages that will use lib64/ anyways, because they
query the toolchain directly. We have a platform-foo/sysroot-host/lib64
symlink because of that.

> Any thoughts?

I actually really like option 2. It sounds like a lot less work and
problems in the long run.

If necessary, we could also make lib64/ a symlink to lib/ (in
ptxd_make_world_install_prepare() and in the final rootfs). That way, it
wouldn't matter which one the packages are using.
But I'd like to avoid that if possible.

Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] Request for comments: CROSS_LIB_DIR handling etc.

[Christian Melki]

On 10/15/21 3:12 PM, Michael Olbrich wrote:
> Hi,
> 
> On Wed, Oct 13, 2021 at 09:29:24AM +0200, Christian Melki wrote:
>> I've been correcting various hardcoded paths in ptxdist packages and whatnot
>> lately. This started with ptxdist not coping well with toolchains that
>> adhere to the ABI path for various architectures.
>>
>> Yesterday, I dug into the problem that on x86_64 (/lib64), all .pc.in -> .pc
>> file transformations looked broken with hardcoded paths. At first I thought
>> it had something to do with autoconf or pkg-config but after a while I found
>> this.
>>
>> scripts/lib/ptxd_make_world_install_mangle_pc.awk
>>
>> Which after a while made me realize that there is still a lot of code in
>> ptxdist core stuff that assumes that lib-paths are only /lib and nothing
>> else.
>>
>> So. I'm presenting a two options here.
>>
>> 1. Fix all ptxdist core stuff, because really, ptxdist should be more
>> flexible than hardcoded paths. Esp. for libs.
>>
>> 2. Split ptxd_get_lib_dir, because, ld.so path should not be assumed to be
>> the same as main library install path. So ptxd_get_ld_lib_dir which does
>> what it does today and install ld there and ptxd_get_lib_dir = /lib and be
>> done with all the userspace library transformations.
> 
> So, put everything in lib/ and only the ld.so in lib64/, right?
> 

Yes. And depending on toolchain configuration.. + glibc, but not the 
rest of the userspace libs as you say. Since the copy_toolchain routines 
query the linker by itself afaiu.

Either way, the important bit is the hardcoded path to ld.so in the ABI 
declaration. That's the one you have to hit, otherwise you'll have 
bricked userspace. :) The rest can be solved with ld.so.conf.

>> So. Number one probably requires a lot of more work and a lot of headache.
> 
> And we'll probably break it more often.

Pretty much guaranteed, yes.

>> Number two should be rather straightforward, atleast in theory.
> 
> There will be some packages that will use lib64/ anyways, because they
> query the toolchain directly. We have a platform-foo/sysroot-host/lib64
> symlink because of that.
> 
>> Any thoughts?
> 
> I actually really like option 2. It sounds like a lot less work and
> problems in the long run.

While I'd like a more generic solution, I can absolutely value things 
that works and are simple. So, less problems it is. :)

> If necessary, we could also make lib64/ a symlink to lib/ (in
> ptxd_make_world_install_prepare() and in the final rootfs). That way, it
> wouldn't matter which one the packages are using.
> But I'd like to avoid that if possible.

Mmm. I botched my rules to always include a separate lib64 now, but 
that's just a hack.

> Michael
> 

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] Request for comments: CROSS_LIB_DIR handling etc.

[Christian Melki]

Bump.

No more comments? Action?

On 10/15/21 3:34 PM, Christian Melki wrote:
> On 10/15/21 3:12 PM, Michael Olbrich wrote:
>> Hi,
>>
>> On Wed, Oct 13, 2021 at 09:29:24AM +0200, Christian Melki wrote:
>>> I've been correcting various hardcoded paths in ptxdist packages and 
>>> whatnot
>>> lately. This started with ptxdist not coping well with toolchains that
>>> adhere to the ABI path for various architectures.
>>>
>>> Yesterday, I dug into the problem that on x86_64 (/lib64), all .pc.in 
>>> -> .pc
>>> file transformations looked broken with hardcoded paths. At first I 
>>> thought
>>> it had something to do with autoconf or pkg-config but after a while 
>>> I found
>>> this.
>>>
>>> scripts/lib/ptxd_make_world_install_mangle_pc.awk
>>>
>>> Which after a while made me realize that there is still a lot of code in
>>> ptxdist core stuff that assumes that lib-paths are only /lib and nothing
>>> else.
>>>
>>> So. I'm presenting a two options here.
>>>
>>> 1. Fix all ptxdist core stuff, because really, ptxdist should be more
>>> flexible than hardcoded paths. Esp. for libs.
>>>
>>> 2. Split ptxd_get_lib_dir, because, ld.so path should not be assumed 
>>> to be
>>> the same as main library install path. So ptxd_get_ld_lib_dir which does
>>> what it does today and install ld there and ptxd_get_lib_dir = /lib 
>>> and be
>>> done with all the userspace library transformations.
>>
>> So, put everything in lib/ and only the ld.so in lib64/, right?
>>
> 
> Yes. And depending on toolchain configuration.. + glibc, but not the 
> rest of the userspace libs as you say. Since the copy_toolchain routines 
> query the linker by itself afaiu.
> 
> Either way, the important bit is the hardcoded path to ld.so in the ABI 
> declaration. That's the one you have to hit, otherwise you'll have 
> bricked userspace. :) The rest can be solved with ld.so.conf.
> 
>>> So. Number one probably requires a lot of more work and a lot of 
>>> headache.
>>
>> And we'll probably break it more often.
> 
> Pretty much guaranteed, yes.
> 
>>> Number two should be rather straightforward, atleast in theory.
>>
>> There will be some packages that will use lib64/ anyways, because they
>> query the toolchain directly. We have a platform-foo/sysroot-host/lib64
>> symlink because of that.
>>
>>> Any thoughts?
>>
>> I actually really like option 2. It sounds like a lot less work and
>> problems in the long run.
> 
> While I'd like a more generic solution, I can absolutely value things 
> that works and are simple. So, less problems it is. :)
> 
>> If necessary, we could also make lib64/ a symlink to lib/ (in
>> ptxd_make_world_install_prepare() and in the final rootfs). That way, it
>> wouldn't matter which one the packages are using.
>> But I'd like to avoid that if possible.
> 
> Mmm. I botched my rules to always include a separate lib64 now, but 
> that's just a hack.
> 
>> Michael
>>
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de
> To unsubscribe, send a mail with subject "unsubscribe" to 
> ptxdist-request@pengutronix.de

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] Request for comments: CROSS_LIB_DIR handling etc.

[Michael Olbrich]

Hi,

On Thu, Nov 04, 2021 at 11:06:13AM +0100, Christian Melki wrote:
> Bump.

Right, this got lost on my side, sorry.

> No more comments? Action?
> 
> On 10/15/21 3:34 PM, Christian Melki wrote:
> > On 10/15/21 3:12 PM, Michael Olbrich wrote:
> > > Hi,
> > > 
> > > On Wed, Oct 13, 2021 at 09:29:24AM +0200, Christian Melki wrote:
> > > > I've been correcting various hardcoded paths in ptxdist packages
> > > > and whatnot
> > > > lately. This started with ptxdist not coping well with toolchains that
> > > > adhere to the ABI path for various architectures.
> > > > 
> > > > Yesterday, I dug into the problem that on x86_64 (/lib64), all
> > > > .pc.in -> .pc
> > > > file transformations looked broken with hardcoded paths. At
> > > > first I thought
> > > > it had something to do with autoconf or pkg-config but after a
> > > > while I found
> > > > this.
> > > > 
> > > > scripts/lib/ptxd_make_world_install_mangle_pc.awk
> > > > 
> > > > Which after a while made me realize that there is still a lot of code in
> > > > ptxdist core stuff that assumes that lib-paths are only /lib and nothing
> > > > else.
> > > > 
> > > > So. I'm presenting a two options here.
> > > > 
> > > > 1. Fix all ptxdist core stuff, because really, ptxdist should be more
> > > > flexible than hardcoded paths. Esp. for libs.
> > > > 
> > > > 2. Split ptxd_get_lib_dir, because, ld.so path should not be
> > > > assumed to be
> > > > the same as main library install path. So ptxd_get_ld_lib_dir which does
> > > > what it does today and install ld there and ptxd_get_lib_dir =
> > > > /lib and be
> > > > done with all the userspace library transformations.
> > > 
> > > So, put everything in lib/ and only the ld.so in lib64/, right?
> > > 
> > 
> > Yes. And depending on toolchain configuration.. + glibc, but not the
> > rest of the userspace libs as you say. Since the copy_toolchain routines
> > query the linker by itself afaiu.

Good point about copy_toolchain. I don't mind if glibc (etc.) is installed
in /lib64 as well. Whatever is easier to start with. We can change that
later if necessary.

> > Either way, the important bit is the hardcoded path to ld.so in the ABI
> > declaration. That's the one you have to hit, otherwise you'll have
> > bricked userspace. :) The rest can be solved with ld.so.conf.

So /usr/lib is not in the default search path and /etc/ld.so.conf is
necessary in this case?

> > > > So. Number one probably requires a lot of more work and a lot of
> > > > headache.
> > > 
> > > And we'll probably break it more often.
> > 
> > Pretty much guaranteed, yes.
> > 
> > > > Number two should be rather straightforward, atleast in theory.
> > > 
> > > There will be some packages that will use lib64/ anyways, because they
> > > query the toolchain directly. We have a platform-foo/sysroot-host/lib64
> > > symlink because of that.
> > > 
> > > > Any thoughts?
> > > 
> > > I actually really like option 2. It sounds like a lot less work and
> > > problems in the long run.
> > 
> > While I'd like a more generic solution, I can absolutely value things
> > that works and are simple. So, less problems it is. :)

:-)

> > > If necessary, we could also make lib64/ a symlink to lib/ (in
> > > ptxd_make_world_install_prepare() and in the final rootfs). That way, it
> > > wouldn't matter which one the packages are using.
> > > But I'd like to avoid that if possible.
> > 
> > Mmm. I botched my rules to always include a separate lib64 now, but
> > that's just a hack.

Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] Request for comments: CROSS_LIB_DIR handling etc.

[Christian Melki]

On 11/5/21 9:57 AM, Michael Olbrich wrote:
> Hi,
> 
> On Thu, Nov 04, 2021 at 11:06:13AM +0100, Christian Melki wrote:
>> Bump.
> 
> Right, this got lost on my side, sorry.
> 

Np. :)

>> No more comments? Action?
>>
>> On 10/15/21 3:34 PM, Christian Melki wrote:
>>> On 10/15/21 3:12 PM, Michael Olbrich wrote:
>>>> Hi,
>>>>
>>>> On Wed, Oct 13, 2021 at 09:29:24AM +0200, Christian Melki wrote:
>>>>> I've been correcting various hardcoded paths in ptxdist packages
>>>>> and whatnot
>>>>> lately. This started with ptxdist not coping well with toolchains that
>>>>> adhere to the ABI path for various architectures.
>>>>>
>>>>> Yesterday, I dug into the problem that on x86_64 (/lib64), all
>>>>> .pc.in -> .pc
>>>>> file transformations looked broken with hardcoded paths. At
>>>>> first I thought
>>>>> it had something to do with autoconf or pkg-config but after a
>>>>> while I found
>>>>> this.
>>>>>
>>>>> scripts/lib/ptxd_make_world_install_mangle_pc.awk
>>>>>
>>>>> Which after a while made me realize that there is still a lot of code in
>>>>> ptxdist core stuff that assumes that lib-paths are only /lib and nothing
>>>>> else.
>>>>>
>>>>> So. I'm presenting a two options here.
>>>>>
>>>>> 1. Fix all ptxdist core stuff, because really, ptxdist should be more
>>>>> flexible than hardcoded paths. Esp. for libs.
>>>>>
>>>>> 2. Split ptxd_get_lib_dir, because, ld.so path should not be
>>>>> assumed to be
>>>>> the same as main library install path. So ptxd_get_ld_lib_dir which does
>>>>> what it does today and install ld there and ptxd_get_lib_dir =
>>>>> /lib and be
>>>>> done with all the userspace library transformations.
>>>>
>>>> So, put everything in lib/ and only the ld.so in lib64/, right?
>>>>
>>>
>>> Yes. And depending on toolchain configuration.. + glibc, but not the
>>> rest of the userspace libs as you say. Since the copy_toolchain routines
>>> query the linker by itself afaiu.
> 
> Good point about copy_toolchain. I don't mind if glibc (etc.) is installed
> in /lib64 as well. Whatever is easier to start with. We can change that
> later if necessary.
> 

Ok. I'll prepare a patch for hardcoding ptxdist to /lib for non 
toolchain related things. I will be naive/stupid/simple, but the 
discussion about configurability etc can start there.

>>> Either way, the important bit is the hardcoded path to ld.so in the ABI
>>> declaration. That's the one you have to hit, otherwise you'll have
>>> bricked userspace. :) The rest can be solved with ld.so.conf.
> 
> So /usr/lib is not in the default search path and /etc/ld.so.conf is
> necessary in this case?
> 

Hmm. If we do nothing then I guess that depends on the toolchain. If the 
toolchain is configured with an extended "user-defined-trusted-dirs" 
option you probably don't need an ld.so.conf for the default search 
paths. But otherwise, I'd say yes? Can't remember. I use an ld.so.conf 
that always includes the usual suspects and prepare a cache when I boot 
(after ld.so has been hit).

To me it looks like glibc expects the default usr search path to be the 
usr/ld.so path (/lib64, /usr/lib64)
This is what it looks like during glibc generation:
---
(glibc) Makefile:
# Build a file mentioning all trustworthy directories to look for shared 
 
 
 

# libraries when using LD_LIBRARY_PATH in a setuid program.  The user 
can 
 
 

# add directories to the list by defining $(user-defined-trusted-dirs) 
 
 
 

# before starting make. 
 
 
 

$(objpfx)trusted-dirs.h: $(objpfx)trusted-dirs.st; @:
$(objpfx)trusted-dirs.st: Makefile $(..)Makeconfig
         $(make-target-directory)
         echo "$(subst :, ,$(default-rpath) 
$(user-defined-trusted-dirs))"    \
         | $(AWK) -f gen-trusted-dirs.awk > ${@:st=T};
	echo '#define DL_DST_LIB "$(notdir $(slibdir))"' >> ${@:st=T}
	$(move-if-change) ${@:st=T} ${@:st=h}
	touch $@

---
(non-extended, generated glibc header, on x86_64) trusted-dirs.h:
#define SYSTEM_DIRS \
   "/lib64/" "\0" "/usr/lib64/"

#define SYSTEM_DIRS_LEN \
   7, 11

#define SYSTEM_DIRS_MAX_LEN     11
#define DL_DST_LIB "lib64"


>>>>> So. Number one probably requires a lot of more work and a lot of
>>>>> headache.
>>>>
>>>> And we'll probably break it more often.
>>>
>>> Pretty much guaranteed, yes.
>>>
>>>>> Number two should be rather straightforward, atleast in theory.
>>>>
>>>> There will be some packages that will use lib64/ anyways, because they
>>>> query the toolchain directly. We have a platform-foo/sysroot-host/lib64
>>>> symlink because of that.
>>>>
>>>>> Any thoughts?
>>>>
>>>> I actually really like option 2. It sounds like a lot less work and
>>>> problems in the long run.
>>>
>>> While I'd like a more generic solution, I can absolutely value things
>>> that works and are simple. So, less problems it is. :)
> 
> :-)
> 
>>>> If necessary, we could also make lib64/ a symlink to lib/ (in
>>>> ptxd_make_world_install_prepare() and in the final rootfs). That way, it
>>>> wouldn't matter which one the packages are using.
>>>> But I'd like to avoid that if possible.
>>>
>>> Mmm. I botched my rules to always include a separate lib64 now, but
>>> that's just a hack.
> 
> Michael
> 

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de

Re: [ptxdist] Request for comments: CROSS_LIB_DIR handling etc.

[Michael Olbrich]

Hi,

On Fri, Nov 05, 2021 at 10:31:24AM +0100, Christian Melki wrote:
> On 11/5/21 9:57 AM, Michael Olbrich wrote:
> > On Thu, Nov 04, 2021 at 11:06:13AM +0100, Christian Melki wrote:
> > > On 10/15/21 3:34 PM, Christian Melki wrote:
> > > > On 10/15/21 3:12 PM, Michael Olbrich wrote:
> > > > > On Wed, Oct 13, 2021 at 09:29:24AM +0200, Christian Melki wrote:
> > > > > > I've been correcting various hardcoded paths in ptxdist packages
> > > > > > and whatnot
> > > > > > lately. This started with ptxdist not coping well with toolchains that
> > > > > > adhere to the ABI path for various architectures.
> > > > > > 
> > > > > > Yesterday, I dug into the problem that on x86_64 (/lib64), all
> > > > > > .pc.in -> .pc
> > > > > > file transformations looked broken with hardcoded paths. At
> > > > > > first I thought
> > > > > > it had something to do with autoconf or pkg-config but after a
> > > > > > while I found
> > > > > > this.
> > > > > > 
> > > > > > scripts/lib/ptxd_make_world_install_mangle_pc.awk
> > > > > > 
> > > > > > Which after a while made me realize that there is still a lot of code in
> > > > > > ptxdist core stuff that assumes that lib-paths are only /lib and nothing
> > > > > > else.
> > > > > > 
> > > > > > So. I'm presenting a two options here.
> > > > > > 
> > > > > > 1. Fix all ptxdist core stuff, because really, ptxdist should be more
> > > > > > flexible than hardcoded paths. Esp. for libs.
> > > > > > 
> > > > > > 2. Split ptxd_get_lib_dir, because, ld.so path should not be
> > > > > > assumed to be
> > > > > > the same as main library install path. So ptxd_get_ld_lib_dir which does
> > > > > > what it does today and install ld there and ptxd_get_lib_dir =
> > > > > > /lib and be
> > > > > > done with all the userspace library transformations.
> > > > > 
> > > > > So, put everything in lib/ and only the ld.so in lib64/, right?
> > > > > 
> > > > 
> > > > Yes. And depending on toolchain configuration.. + glibc, but not the
> > > > rest of the userspace libs as you say. Since the copy_toolchain routines
> > > > query the linker by itself afaiu.
> > 
> > Good point about copy_toolchain. I don't mind if glibc (etc.) is installed
> > in /lib64 as well. Whatever is easier to start with. We can change that
> > later if necessary.
> > 
> 
> Ok. I'll prepare a patch for hardcoding ptxdist to /lib for non toolchain
> related things. I will be naive/stupid/simple, but the discussion about
> configurability etc can start there.
> 
> > > > Either way, the important bit is the hardcoded path to ld.so in the ABI
> > > > declaration. That's the one you have to hit, otherwise you'll have
> > > > bricked userspace. :) The rest can be solved with ld.so.conf.
> > 
> > So /usr/lib is not in the default search path and /etc/ld.so.conf is
> > necessary in this case?
> > 
> 
> Hmm. If we do nothing then I guess that depends on the toolchain. If the
> toolchain is configured with an extended "user-defined-trusted-dirs" option
> you probably don't need an ld.so.conf for the default search paths. But
> otherwise, I'd say yes? Can't remember. I use an ld.so.conf that always
> includes the usual suspects and prepare a cache when I boot (after ld.so has
> been hit).
> 
> To me it looks like glibc expects the default usr search path to be the
> usr/ld.so path (/lib64, /usr/lib64)

So, I've picked this up myself and I've pushed something to master.

CROSS_LIB_DIR is now always /lib and CROSS_LINKER_LIB_DIR has its previous
value. If they differ, then /$(CROSS_LINKER_LIB_DIR) is created as a
symlink to /lib (which is a symlink to /usr/lib). And everything is
installed to /usr/lib including the dynamic linker.

This avoids any issues with the search path.

I've removed all uses of CROSS_LIB_DIR in PTXdist itself, but it's still
there for compatibility for now.

I've tested this with a crosstool-ng x86_64 toolchain, but it would be
great if you could take a look at it with your setup.

Michael


-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |