[ptxdist] /usr merge

[ptxdist] /usr merge

[Michael Olbrich]

Hi,

I've just pushed a large series of commits for /usr merge. This means that
/bin, /sbin and /lib are now symlinks to the corresponding directories in
/usr. All files are installed accordingly.

I'm not going to rehash all the arguments for this. My main motivations
are:
- It's where the big distros are going, so it's better tested in the long
  term.
- We've had several problems in the past where programs where searched in
  the wrong location. And busybox didn't make that any easier.

How it works in PTXdist:
- install_copy and friends will transparently move files to /usr where
  appropriate. So if you have packages that install files to /bin etc. it
  should just work. If not, please sent a mail to this list.
- If install_alternative etc. installs a file from /usr/... and finds the
  a file in the corresponding path without /usr it will fail.
  This is to make sure that you move all overwrites in projectroot/.

I've done some build and runtime testing and there are currently no known
problems. But this is a big change and I can't test everything. It would be
great if some more people could test this before the next release.

Regards,
Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] /usr merge

[Enrico Weigelt, metux IT consult]

On 11.04.2017 12:07, Michael Olbrich wrote:

Hi,

> I've just pushed a large series of commits for /usr merge. This means that
> /bin, /sbin and /lib are now symlinks to the corresponding directories in
> /usr. All files are installed accordingly.

Just curious: what's the actual problem to solve ?

For my embedded projects, I haven't had an actual usecase for having
them separated, but on server systems lots of them over the past 20yrs.

Note that this separation indeed has serious reaons: separating the
base system (IOW: everything required to boot up at least to single
user mode and mount /usr - whereever it might come from). Most of the
practical usecases might be obsolete now (eg. storage is so cheap that
people rarely have the need to share /usr within clusters, etc), but
still there're lots of systems out there relying on that (maybe quite
irrelevant to ptxdist, but certainly for standard like Debian)

I'm personally agnostic to that, but I've already seen such setups
(eg. having /usr on separate SDcard) in embedded systems. So, it seems
better if that remains configurable.

> I'm not going to rehash all the arguments for this. My main motivations
> are:
> - It's where the big distros are going, so it's better tested in the long
>   term.

Note that this already caused a lot of trouble there. (eg. there've
been huge flamewars @debian). IIRC that all started w/ the systemd
mess, which suddenly introduced hard dependencies to things in /usr -
the reaction of Lennart+friends was just ranting aaginst who relies
on the separation.

> - We've had several problems in the past where programs where searched in
>   the wrong location. And busybox didn't make that any easier.

Correct me if I'm wrong, but isn't that what $PATH is for ?

> How it works in PTXdist:
> - install_copy and friends will transparently move files to /usr where
>   appropriate. So if you have packages that install files to /bin etc. it
>   should just work. If not, please sent a mail to this list.

I really have a bad feeling about such magic in existing commands,
which now suddenly change semantics. At least that should be
configurable.


--mtx


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] /usr merge

[Michael Olbrich]

On Wed, Apr 12, 2017 at 10:27:30AM +0200, Enrico Weigelt, metux IT consult wrote:
> On 11.04.2017 12:07, Michael Olbrich wrote:
> > I've just pushed a large series of commits for /usr merge. This means that
> > /bin, /sbin and /lib are now symlinks to the corresponding directories in
> > /usr. All files are installed accordingly.
> 
> Just curious: what's the actual problem to solve ?

The issues below.

> For my embedded projects, I haven't had an actual usecase for having
> them separated, but on server systems lots of them over the past 20yrs.
> 
> Note that this separation indeed has serious reaons: separating the
> base system (IOW: everything required to boot up at least to single
> user mode and mount /usr - whereever it might come from). Most of the
> practical usecases might be obsolete now (eg. storage is so cheap that
> people rarely have the need to share /usr within clusters, etc), but
> still there're lots of systems out there relying on that (maybe quite
> irrelevant to ptxdist, but certainly for standard like Debian)
> 
> I'm personally agnostic to that, but I've already seen such setups
> (eg. having /usr on separate SDcard) in embedded systems. So, it seems
> better if that remains configurable.

I haven't seen any done with PTXdist. And while I don't know the exact
use-case, I think starting with an initrd and then switching to the rootfs
on the SDcard make more sense.

A separate /usr requires a lot of work. I'm quite certain, that we had
several programs in /bin and /sbin that linked to libraries in /usr and
nobody ever complained about it here.

> > I'm not going to rehash all the arguments for this. My main motivations
> > are:
> > - It's where the big distros are going, so it's better tested in the long
> >   term.
> 
> Note that this already caused a lot of trouble there. (eg. there've
> been huge flamewars @debian). IIRC that all started w/ the systemd
> mess, which suddenly introduced hard dependencies to things in /usr -
> the reaction of Lennart+friends was just ranting aaginst who relies
> on the separation.

Yes, flamewars and rants in all directions. Are there any real discussions
to solve the problems? After all, a separate /usr is one solution to a set
of problems, but why should it be the only solution?

> > - We've had several problems in the past where programs where searched in
> >   the wrong location. And busybox didn't make that any easier.
> 
> Correct me if I'm wrong, but isn't that what $PATH is for ?

No. $PATH only really works in a shell. exec*() requires the full path to
the executable and those are built into the binaries. These paths may be
hard coded in the source, or maybe detected with AC_PATH_PROG. That's
especially tricky because, unless the correct environment variable is set,
the path from the build host is used and it may work on one distro but not
the other. And then there is busybox with its own idea where the programs
should be. If that differs from the corresponding 'real' program, then
there isn't even a "correct" path to use.
And this has happened before and it's really hard to get this right.

> > How it works in PTXdist:
> > - install_copy and friends will transparently move files to /usr where
> >   appropriate. So if you have packages that install files to /bin etc. it
> >   should just work. If not, please sent a mail to this list.
> 
> I really have a bad feeling about such magic in existing commands,
> which now suddenly change semantics.

That depends on your definition of changed semantics. After all, the files
are still reachable at the specified path.

> At least that should be configurable.

Doing that is a lot of work. Because that only makes sense, if a separate
/usr _really_ works. And that is more than making the recent changes
optional. It starts with making sure that nothing outside /usr needs
anything in /usr and continues with a clear concept of what should be
possible without /usr mounted.
I certainly don't have the time to do the initial work or maintain this
longterm.

We have quite a few contributors that add new packages and keep packages up
to date. But I'm the only one that actually works on the core stuff.
So unless someone steps up to do this including the maintenance, it's not
going to happen.

Regards,
Michael

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] /usr merge

[Ladislav Michl]

On Wed, Apr 12, 2017 at 11:24:35AM +0200, Michael Olbrich wrote:
> On Wed, Apr 12, 2017 at 10:27:30AM +0200, Enrico Weigelt, metux IT consult wrote:
> > On 11.04.2017 12:07, Michael Olbrich wrote:
> > > I've just pushed a large series of commits for /usr merge. This means that
> > > /bin, /sbin and /lib are now symlinks to the corresponding directories in
> > > /usr. All files are installed accordingly.
> > 
> > Just curious: what's the actual problem to solve ?
> 
> The issues below.
> 
> > For my embedded projects, I haven't had an actual usecase for having
> > them separated, but on server systems lots of them over the past 20yrs.
> > 
> > Note that this separation indeed has serious reaons: separating the
> > base system (IOW: everything required to boot up at least to single
> > user mode and mount /usr - whereever it might come from). Most of the
> > practical usecases might be obsolete now (eg. storage is so cheap that
> > people rarely have the need to share /usr within clusters, etc), but
> > still there're lots of systems out there relying on that (maybe quite
> > irrelevant to ptxdist, but certainly for standard like Debian)
> > 
> > I'm personally agnostic to that, but I've already seen such setups
> > (eg. having /usr on separate SDcard) in embedded systems. So, it seems
> > better if that remains configurable.

That's also seen on many consumer electronic devices, but given poor
quality (hack until it works and then it's done), I wouldn't take it
as any measure.

Btw, this is quite a nice summary:
http://lists.busybox.net/pipermail/busybox/2010-December/074114.html

[snip]
> > I really have a bad feeling about such magic in existing commands,
> > which now suddenly change semantics.
> 
> That depends on your definition of changed semantics. After all, the files
> are still reachable at the specified path.
> 
> > At least that should be configurable.
> 
> Doing that is a lot of work. Because that only makes sense, if a separate
> /usr _really_ works. And that is more than making the recent changes
> optional. It starts with making sure that nothing outside /usr needs
> anything in /usr and continues with a clear concept of what should be
> possible without /usr mounted.
> I certainly don't have the time to do the initial work or maintain this
> longterm.
> 
> We have quite a few contributors that add new packages and keep packages up
> to date. But I'm the only one that actually works on the core stuff.
> So unless someone steps up to do this including the maintenance, it's not
> going to happen.

...and also does not need to happen ;-)
Anyway (and back to more constructive note), I updated and tested all my five
configurations and didn't find any issues. Nice work!

Thanks,
	ladis

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] /usr merge

[Enrico Weigelt, metux IT consult]

On 12.04.2017 11:24, Michael Olbrich wrote:

Hi,

> I haven't seen any done with PTXdist. 

I did, quite some years ago, no idea whether it still exists.

> And while I don't know the exact use-case, I think starting with an
> initrd and then switching to the rootfs on the SDcard make more sense.

Sure, that would be an option (and what several large distros do),
and for new systems I'd choose either that approach, or even having
an entirely separate rescue system.

My concern is that there might be lots of systems in the field, that
can't be easily migrated. So, there it should be configurable, at least
for some time.

Anyways, I wonder why the merge hasn't been done exactly the opposite
way and perhaps symlinking /usr/bin to /bin, etc.

> Yes, flamewars and rants in all directions. Are there any real discussions
> to solve the problems? 

I'd always start w/ asking what the real problem to solve actually is.
Unfortunately, in our branch seems to have the tendency to provide
lots of solutions for unknown problems ;-)

> After all, a separate /usr is one solution to a set
> of problems, but why should it be the only solution?

Sure. But we could ask the same question for any other solution.
My point here is that having separate /usr has a long tradition (even
longer than Linux itself exists), so we should be quite cautious w/
dropping that, w/o giving a proper fallback.

By the way: in Linux embedded world, we also have people splitting
up between OS and their own applications, leading to the interesting
questions, where the border line actually is (on which side do eg.
libxml or boost belong ?). I'd rather like not to open that can of
worms, but IMHO it seems to root back to the same fundamental problem.

>> Correct me if I'm wrong, but isn't that what $PATH is for ?
> 
> No. $PATH only really works in a shell. exec*() requires the full path to
> the executable and those are built into the binaries. 

Okay, forgot that, as I'm usually not hardcoding any exec pathes
(or even use system()+friends, if applicable).

> These paths may be hard coded in the source, or maybe detected with AC_PATH_PROG.

Hardcoded in the source seems a really, really bad idea.
OTOH, is AC_PATH_PROG even cross/sysroot-aware ?

I just dont remember any case where some package expects a program in
/usr/bin, which traditionally is in /bin. But there're cases in the
opposite direction, eg. the recent busybox-tar issue here on the list.

> That's especially tricky because, unless the correct environment variable
> is set, the path from the build host is used and it may work on one
> distro but not the other.

Looks like all the problems we had w/ autotools (and autotool'ed
packages) in the last decades, simply working on false assumptions,
just like AC_TRY_RUN() and friends. Merging /usr doesn't solve the
underlying problem - it just reduces the set of actual breaks for a
little bit.

The fundamentally wrong assumption is that build and target system
have anything to do w/ each other (and things like crosscompiling
were just be some weird corner cases) - that also hits standard
distros like Debian (IIRC, they still run most of the cross builds
in qemu).

> And then there is busybox with its own idea where the programs
> should be. If that differs from the corresponding 'real' program, then
> there isn't even a "correct" path to use.

So, could busybox just be wrong, and it should be fixed there ?
Why not letting busybox install the corresponding programs
(actually, symlinks) into the standard places ?

Merging /usr for that reason reminds me to the old Cinderella
story ... :o

"There they go, there they go!
There is blood on her shoe;
The shoe is too small,
Not the right bride at all!"

> That depends on your definition of changed semantics. After all,
> the files are still reachable at the specified path.

The change is that they aren't in /bin + friends anymore, just
symlinks to the actual programs somwhere under /usr.

If we're already breaking FHS, we could also do it in the exactly
opposive direction - moving /usr/bin/* to /bin/, etc. Or symlink
the whole directories (instead of individual files). Or put each
package into it's own prefix and use unions put everything together,
like Plan9 does.

I wonder when anybody proposes drive letters ...
(we already have the registry, so that doesn't seem to unlikely :o)

> Doing that is a lot of work. Because that only makes sense, if a separate
> /usr _really_ works. And that is more than making the recent changes
> optional. It starts with making sure that nothing outside /usr needs
> anything in /usr and continues with a clear concept of what should be
> possible without /usr mounted.

Once upon a time, it had been so. (it's just a few years ago since
distros start breaking that, eg. makeing desktop tools part of the
early boot process)

> I certainly don't have the time to do the initial work or maintain this
> longterm.

I don't remeber anybody asking you to do that. No idea how many people
are around here, but I'm sure we've got enough manpower here to
distribute the load onto many shoulders and solve the problem at the
root, once and for all. Add me to the list of volounteers. (my time is
limited, too, but I believe it's an area worth solving once and for
all, instead of adding one workaround atop another)


--mtx


_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] /usr merge

[Clemens Gruber]

On Wed, Apr 12, 2017 at 02:36:58PM +0200, Enrico Weigelt, metux IT consult wrote:
> I just dont remember any case where some package expects a program in
> /usr/bin, which traditionally is in /bin. But there're cases in the
> opposite direction, eg. the recent busybox-tar issue here on the list.

I recently experienced this when using some tools from coreutils instead
of busybox.

The Fedora folks made a good case for the usr merge, here:
https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/

ArchLinux did the /usr merge four years ago, and I really like the idea
of no longer needing to know for each binary, where it was put.
It is especially useful when writing systemd units because ExecStart
needs the full path.

I can remember several problems which would not have happened with a
usr merge. Most of them wrong paths in ExecStart(Pre), because for
example coreutils rm is installed to /usr/bin while busybox rm is in
/bin.

Does someony have a system in production, where PTXdist is updated
somewhat regularily and the usr split is required / using an initrd is
not possible?

Thanks,
Clemens

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] /usr merge

[Alexander Dahl]

[-- Attachment #1.1: Type: text/plain, Size: 816 bytes --]

Hei hei,

On Wed, Apr 12, 2017 at 02:36:58PM +0200, Enrico Weigelt, metux IT consult wrote:
> By the way: in Linux embedded world, we also have people splitting
> up between OS and their own applications, leading to the interesting
> questions, where the border line actually is (on which side do eg.
> libxml or boost belong ?). I'd rather like not to open that can of
> worms, but IMHO it seems to root back to the same fundamental problem.

We put our own applications to /usr/local/bin or /usr/local/sbin …
O:-)

A

-- 
»With the first link, the chain is forged. The first speech censured, 
the first thought forbidden, the first freedom denied, chains us all 
irrevocably.« (Jean-Luc Picard, quoting Judge Aaron Satie)
*** GnuPG-FP: C28E E6B9 0263 95CF 8FAF  08FA 34AD CD00 7221 5CC6 ***

[-- Attachment #1.2: Type: application/pgp-signature, Size: 819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 91 bytes --]

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de