[ptxdist] Replace DYLD_FALLBACK_LIBRARY_PATH by something that works

[ptxdist] Replace DYLD_FALLBACK_LIBRARY_PATH by something that works

[Bernhard Walle]

I don't know why I was able to build host-qemu in the OSELAS Generic BSP in the
first place. Maybe I just had DYLD_FALLBACK_LIBRARY_PATH set in my work shell.

Anyway, that patch series removes setting DYLD_FALLBACK_LIBRARY_PATH and replaces
it by editing the binaries after installation. See the description of the second
patch for more details.

Since it doesn't do anything on Linux, it cannot break Linux, so it should be
quite safe for most users. :)


-- 
ptxdist mailing list
ptxdist@pengutronix.de

[ptxdist] [PATCH 1/2] Remove DYLD_FALLBACK_LIBRARY_PATH (Darwin)

[Bernhard Walle]

This has been introduced by commit 6ab38b6d244e0de21b9dd94052e0da30adc4aff3.
The problem with the environment variable is that you only can run the
compiled host tools when the environment is set. So one has to add the
environment all over in many places and even if it's set in ptxdist
globally, you can't run the programs manually.

The next patch provides a better solution.

Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
---
 rules/pre/Rules.make |    7 +------
 1 Datei geändert, 1 Zeile hinzugefügt(+), 6 Zeilen entfernt(-)

diff --git a/rules/pre/Rules.make b/rules/pre/Rules.make
index 522c90e..1d09a18 100644
--- a/rules/pre/Rules.make
+++ b/rules/pre/Rules.make
@@ -321,10 +321,6 @@ HOST_ENV_CXX		:= CXX="$(HOSTCXX)"
 HOST_ENV_CPPFLAGS	:= CPPFLAGS="$(HOST_CPPFLAGS)"
 HOST_ENV_LDFLAGS	:= LDFLAGS="$(HOST_LDFLAGS)"
 
-ifeq ($(shell uname -s),Darwin)
-HOST_ENV_LIBRARY_PATH	:= DYLD_FALLBACK_LIBRARY_PATH=$(PTXDIST_SYSROOT_HOST)/lib
-endif
-
 HOST_ENV_PKG_CONFIG	:= \
 	PKG_CONFIG_PATH="" \
 	PKG_CONFIG_LIBDIR="$(PTXDIST_SYSROOT_HOST)/lib/pkgconfig:$(PTXDIST_SYSROOT_HOST)/share/pkgconfig"
@@ -339,8 +335,7 @@ HOST_ENV	:= \
 	$(HOST_ENV_CPPFLAGS) \
 	$(HOST_ENV_LDFLAGS) \
 	$(HOST_ENV_PKG_CONFIG) \
-	$(HOST_ENV_PYTHONPATH) \
-	$(HOST_ENV_LIBRARY_PATH)
+	$(HOST_ENV_PYTHONPATH)
 
 
 HOST_AUTOCONF  := --prefix=
-- 
1.7.10.4


-- 
ptxdist mailing list
ptxdist@pengutronix.de

[ptxdist] [PATCH 2/2] ptxd_make_world_install_post: Darwin: Edit install name of binaries

[Bernhard Walle]

This patch should solve the problem that binaries in
$PTXDIST_SYSROOT_HOST/bin must be able to find their libraries in
$PTXDIST_SYSROOT_HOST/lib. On ELF systems that problem is solved
by using rpath. However, rpath doesn't exist (works different) on
Darwin that uses the Mach-O binary format.

After copying the files to $PTXDIST_SYSROOT_HOST, we do following:

 - For every library (lib/*.*.dylib), we change the install name
   from /lib/libfoo.x.dylib to $PTXDIST_SYSROOT_HOST/lib/libfoo.x.dylib.
   That way programs linked against that library can be executed
   immediately, even in the build directory before installation
   to $PTXDIST_SYSROOT_HOST.

   That's the call of "install_name_tool -id".

 - For every library (lib/*.*.dylib) and executable (bin/*), we
   change the install name of dependent libraries to point to
   $PTXDIST_SYSROOT_HOST/lib. The call of "otool -L" lists the
   dependent libraries (like "readelf -d" on ELF systems) and
   the call "install_name_tool -change" changes it. That step
   is necessary because multiple libraries and binaries can be
   built and installed at the same time, so the binary is already
   built before "install_name_tool -id" is invoked.

That way I'm finally able to build host-qemu with host-glib and
host-gettext-dummy (that provides -lintl which host-glib uses).

Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
---
 scripts/lib/ptxd_make_world_install.sh             |    5 ++
 .../lib/ptxd_make_world_install_library_path.sh    |   92 ++++++++++++++++++++
 2 Dateien geändert, 97 Zeilen hinzugefügt(+)
 create mode 100644 scripts/lib/ptxd_make_world_install_library_path.sh

diff --git a/scripts/lib/ptxd_make_world_install.sh b/scripts/lib/ptxd_make_world_install.sh
index 3d5d5e5..e0418dc 100644
--- a/scripts/lib/ptxd_make_world_install.sh
+++ b/scripts/lib/ptxd_make_world_install.sh
@@ -175,5 +175,10 @@ ptxd_make_world_install_post() {
     done &&
 
     cp -dprf -- "${pkg_pkg_dir}"/* "${pkg_sysroot_dir}"
+
+    # only for host packages
+    if [ "${pkg_sysroot_dir}" = "${PTXDIST_SYSROOT_HOST}" ] ; then
+	ptxd_make_world_install_library_path "${pkg_pkg_dir}"
+    fi
 }
 export -f ptxd_make_world_install_post
diff --git a/scripts/lib/ptxd_make_world_install_library_path.sh b/scripts/lib/ptxd_make_world_install_library_path.sh
new file mode 100644
index 0000000..32b4b45
--- /dev/null
+++ b/scripts/lib/ptxd_make_world_install_library_path.sh
@@ -0,0 +1,92 @@
+#!/bin/bash
+#
+# (c) 2012 Bernhard Walle <bernhard@bwalle.de>
+#
+
+# This function is Darwin (Mac OS) specific. Helper function for
+# ptxd_make_world_install_library_path.
+#
+# It should be invoked on one binary in ptxdist platform-*/sysroot-host (both
+# for executables and libraries after installation. For binaries and libraries,
+# edits the dependent shared library install name to point to sysroot-host/lib.
+# For libraries only it changes also the install name (-id). That way the
+# binaries linked against a library in sysroot-target/lib can be executed
+# directly without setting environment variables.
+#
+# Arguments: $1 - the binary
+ptxd_make_world_install_library_path_one()
+{
+    local binary="$1"
+    local filename installfile
+
+    if ! [ -r "${binary}" ] ; then
+	return
+    fi
+
+    for used_library in $(otool -LX "${binary}" | awk '{print $1}') ; do
+	filename=${used_library##*/}  # basename
+
+	# if the library exists in sysroot-host, we use that one
+	# using @executable_path even keeps the tree relocatable.
+	installfile="${PTXDIST_SYSROOT_HOST}/lib/${filename}"
+	if [ -r "${installfile}" ] ; then
+	    install_name_tool -change \
+		"${used_library}" "${installfile}" \
+		"${binary}"
+	fi
+
+	# if it's a library, change also the ID so that binaries
+	# that link against the binary get it right in the first
+	# place (so they can get executed before installation)
+	if [[ ${installfile} == *.dylib ]] ; then
+	    install_name_tool -id "${binary}" "${binary}"
+	fi
+    done
+}
+export -f ptxd_make_world_install_library_path_one
+
+# This function is Darwin (Mac OS) specific. It does nothing on other
+# operating systems.
+#
+# The function gets invoked in install.post stage after the binaries
+# have been copied from the package dir to the sysroot-host directory.
+# The function must be called with the package directory as argument.
+# For every binary (library and executable) in that package directory
+# it translates the path to the correspondent path after copying
+# to the sysroot-host and invokes ptxd_make_world_install_library_path_one
+# which finally edits the install path of libaries/executables.
+#
+# Arguments: $1 - pkgdir of a package
+ptxd_make_world_install_library_path()
+{
+    if [ "$(uname -s)" != Darwin ] ; then
+	return
+    fi
+
+    local pkgdir="$1"
+    local file
+    local installed_file
+
+    echo "install.post: Fixup library paths"
+    for file in ${pkgdir}/bin/* ${pkgdir}/lib/*.*.dylib ; do
+
+	# for unexpanded wildcards
+	if ! [ -r "${file}" ] ; then
+	    continue
+	fi
+
+	installed_file=$(echo $file | sed -e "s@${pkgdir}@${PTXDIST_SYSROOT_HOST}@g")
+
+	# sanity check
+	if ! [ -r "${installed_file}" ] ; then
+	    echo >&2 "File '${file}' exists but '${installed_file}' does not after coping???"
+	    continue
+	fi
+
+	echo "${installed_file}"
+	ptxd_make_world_install_library_path_one "${installed_file}"
+    done
+
+    echo "install.post: Fixup library paths: done"
+}
+export -f ptxd_make_world_install_library_path
-- 
1.7.10.4


-- 
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [PATCH 2/2] ptxd_make_world_install_post: Darwin: Edit install name of binaries

[Michael Olbrich]

On Sun, Jun 10, 2012 at 05:05:32PM +0200, Bernhard Walle wrote:
> This patch should solve the problem that binaries in
> $PTXDIST_SYSROOT_HOST/bin must be able to find their libraries in
> $PTXDIST_SYSROOT_HOST/lib. On ELF systems that problem is solved
> by using rpath. However, rpath doesn't exist (works different) on
> Darwin that uses the Mach-O binary format.
> 
> After copying the files to $PTXDIST_SYSROOT_HOST, we do following:
> 
>  - For every library (lib/*.*.dylib), we change the install name
>    from /lib/libfoo.x.dylib to $PTXDIST_SYSROOT_HOST/lib/libfoo.x.dylib.
>    That way programs linked against that library can be executed
>    immediately, even in the build directory before installation
>    to $PTXDIST_SYSROOT_HOST.
> 
>    That's the call of "install_name_tool -id".
> 
>  - For every library (lib/*.*.dylib) and executable (bin/*), we
>    change the install name of dependent libraries to point to
>    $PTXDIST_SYSROOT_HOST/lib. The call of "otool -L" lists the
>    dependent libraries (like "readelf -d" on ELF systems) and
>    the call "install_name_tool -change" changes it. That step
>    is necessary because multiple libraries and binaries can be
>    built and installed at the same time, so the binary is already
>    built before "install_name_tool -id" is invoked.
> 
> That way I'm finally able to build host-qemu with host-glib and
> host-gettext-dummy (that provides -lintl which host-glib uses).
> 
> Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
> ---
>  scripts/lib/ptxd_make_world_install.sh             |    5 ++
>  .../lib/ptxd_make_world_install_library_path.sh    |   92 ++++++++++++++++++++
>  2 Dateien geändert, 97 Zeilen hinzugefügt(+)
>  create mode 100644 scripts/lib/ptxd_make_world_install_library_path.sh
> 
> diff --git a/scripts/lib/ptxd_make_world_install.sh b/scripts/lib/ptxd_make_world_install.sh
> index 3d5d5e5..e0418dc 100644
> --- a/scripts/lib/ptxd_make_world_install.sh
> +++ b/scripts/lib/ptxd_make_world_install.sh
> @@ -175,5 +175,10 @@ ptxd_make_world_install_post() {
>      done &&
>  
>      cp -dprf -- "${pkg_pkg_dir}"/* "${pkg_sysroot_dir}"
> +
> +    # only for host packages
> +    if [ "${pkg_sysroot_dir}" = "${PTXDIST_SYSROOT_HOST}" ] ; then
> +	ptxd_make_world_install_library_path "${pkg_pkg_dir}"
> +    fi

    if [ "${pkg_type}" != "target" ]; then
	ptxd_make_world_install_library_path
    fi

for cross packages too, and use pkg_pkg_dir directly below.

>  }
>  export -f ptxd_make_world_install_post
> diff --git a/scripts/lib/ptxd_make_world_install_library_path.sh b/scripts/lib/ptxd_make_world_install_library_path.sh
> new file mode 100644
> index 0000000..32b4b45
> --- /dev/null
> +++ b/scripts/lib/ptxd_make_world_install_library_path.sh
> @@ -0,0 +1,92 @@
> +#!/bin/bash
> +#
> +# (c) 2012 Bernhard Walle <bernhard@bwalle.de>
> +#
> +
> +# This function is Darwin (Mac OS) specific. Helper function for
> +# ptxd_make_world_install_library_path.
> +#
> +# It should be invoked on one binary in ptxdist platform-*/sysroot-host (both
> +# for executables and libraries after installation. For binaries and libraries,
> +# edits the dependent shared library install name to point to sysroot-host/lib.
> +# For libraries only it changes also the install name (-id). That way the
> +# binaries linked against a library in sysroot-target/lib can be executed
> +# directly without setting environment variables.
> +#
> +# Arguments: $1 - the binary
> +ptxd_make_world_install_library_path_one()
> +{
> +    local binary="$1"
> +    local filename installfile
> +
> +    if ! [ -r "${binary}" ] ; then
> +	return
> +    fi

this was already checked, right?

> +
> +    for used_library in $(otool -LX "${binary}" | awk '{print $1}') ; do
> +	filename=${used_library##*/}  # basename

filename="$(basename "${used_library}")"

> +
> +	# if the library exists in sysroot-host, we use that one
> +	# using @executable_path even keeps the tree relocatable.
> +	installfile="${PTXDIST_SYSROOT_HOST}/lib/${filename}"
> +	if [ -r "${installfile}" ] ; then
> +	    install_name_tool -change \
> +		"${used_library}" "${installfile}" \
> +		"${binary}"
> +	fi
> +
> +	# if it's a library, change also the ID so that binaries
> +	# that link against the binary get it right in the first
> +	# place (so they can get executed before installation)
> +	if [[ ${installfile} == *.dylib ]] ; then
> +	    install_name_tool -id "${binary}" "${binary}"
> +	fi
> +    done
> +}
> +export -f ptxd_make_world_install_library_path_one
> +
> +# This function is Darwin (Mac OS) specific. It does nothing on other
> +# operating systems.
> +#
> +# The function gets invoked in install.post stage after the binaries
> +# have been copied from the package dir to the sysroot-host directory.
> +# The function must be called with the package directory as argument.
> +# For every binary (library and executable) in that package directory
> +# it translates the path to the correspondent path after copying
> +# to the sysroot-host and invokes ptxd_make_world_install_library_path_one
> +# which finally edits the install path of libaries/executables.
> +#
> +# Arguments: $1 - pkgdir of a package
> +ptxd_make_world_install_library_path()
> +{
> +    if [ "$(uname -s)" != Darwin ] ; then
> +	return
> +    fi
> +
> +    local pkgdir="$1"

use pkg_pkg_dir directly everywhere

> +    local file
> +    local installed_file
> +
> +    echo "install.post: Fixup library paths"
> +    for file in ${pkgdir}/bin/* ${pkgdir}/lib/*.*.dylib ; do
> +
> +	# for unexpanded wildcards
> +	if ! [ -r "${file}" ] ; then
> +	    continue
> +	fi

use "find "${pkgdir}" -type f" and 
file "${file}" | grep -q 'dynamically linked' || continue

or something like that. Otherwise you might miss something.

> +
> +	installed_file=$(echo $file | sed -e "s@${pkgdir}@${PTXDIST_SYSROOT_HOST}@g")

use ${pkg_sysroot_dir} (works for cross too)

> +
> +	# sanity check
> +	if ! [ -r "${installed_file}" ] ; then
> +	    echo >&2 "File '${file}' exists but '${installed_file}' does not after coping???"
> +	    continue
> +	fi

imho not needed. The files where copied just before calling this.

> +
> +	echo "${installed_file}"

remove.

Michael

> +	ptxd_make_world_install_library_path_one "${installed_file}"
> +    done
> +
> +    echo "install.post: Fixup library paths: done"
> +}
> +export -f ptxd_make_world_install_library_path
> -- 
> 1.7.10.4
> 
> 
> -- 
> ptxdist mailing list
> ptxdist@pengutronix.de
-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

-- 
ptxdist mailing list
ptxdist@pengutronix.de

[ptxdist] [PATCH 1/2] Remove DYLD_FALLBACK_LIBRARY_PATH (Darwin)

[Bernhard Walle]

This has been introduced by commit 6ab38b6d244e0de21b9dd94052e0da30adc4aff3.
The problem with the environment variable is that you only can run the
compiled host tools when the environment is set. So one has to add the
environment all over in many places and even if it's set in ptxdist
globally, you can't run the programs manually.

The next patch provides a better solution.

Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
---
 rules/pre/Rules.make |    7 +------
 1 Datei geändert, 1 Zeile hinzugefügt(+), 6 Zeilen entfernt(-)

diff --git a/rules/pre/Rules.make b/rules/pre/Rules.make
index a6b662f..2d1faa6 100644
--- a/rules/pre/Rules.make
+++ b/rules/pre/Rules.make
@@ -321,10 +321,6 @@ HOST_ENV_CXX		:= CXX="$(HOSTCXX)"
 HOST_ENV_CPPFLAGS	:= CPPFLAGS="$(HOST_CPPFLAGS)"
 HOST_ENV_LDFLAGS	:= LDFLAGS="$(HOST_LDFLAGS)"
 
-ifeq ($(shell uname -s),Darwin)
-HOST_ENV_LIBRARY_PATH	:= DYLD_FALLBACK_LIBRARY_PATH=$(PTXDIST_SYSROOT_HOST)/lib
-endif
-
 HOST_ENV_PKG_CONFIG	:= \
 	PKG_CONFIG_PATH="" \
 	PKG_CONFIG_LIBDIR="$(PTXDIST_SYSROOT_HOST)/lib/pkgconfig:$(PTXDIST_SYSROOT_HOST)/share/pkgconfig"
@@ -339,8 +335,7 @@ HOST_ENV	:= \
 	$(HOST_ENV_CPPFLAGS) \
 	$(HOST_ENV_LDFLAGS) \
 	$(HOST_ENV_PKG_CONFIG) \
-	$(HOST_ENV_PYTHONPATH) \
-	$(HOST_ENV_LIBRARY_PATH)
+	$(HOST_ENV_PYTHONPATH)
 
 
 HOST_AUTOCONF  := --prefix=
-- 
1.7.10.4


-- 
ptxdist mailing list
ptxdist@pengutronix.de

[ptxdist] [PATCH 2/2] ptxd_make_world_install_post: Darwin: Edit install name of binaries

[Bernhard Walle]

This patch should solve the problem that binaries in
$PTXDIST_SYSROOT_HOST/bin must be able to find their libraries in
$PTXDIST_SYSROOT_HOST/lib. On ELF systems that problem is solved
by using rpath. However, rpath doesn't exist (works different) on
Darwin that uses the Mach-O binary format.

After copying the files to $PTXDIST_SYSROOT_HOST, we do following:

 - For every library (lib/*.*.dylib), we change the install name
   from /lib/libfoo.x.dylib to $PTXDIST_SYSROOT_HOST/lib/libfoo.x.dylib.
   That way programs linked against that library can be executed
   immediately, even in the build directory before installation
   to $PTXDIST_SYSROOT_HOST.

   That's the call of "install_name_tool -id".

 - For every library (lib/*.*.dylib) and executable (bin/*), we
   change the install name of dependent libraries to point to
   $PTXDIST_SYSROOT_HOST/lib. The call of "otool -L" lists the
   dependent libraries (like "readelf -d" on ELF systems) and
   the call "install_name_tool -change" changes it. That step
   is necessary because multiple libraries and binaries can be
   built and installed at the same time, so the binary is already
   built before "install_name_tool -id" is invoked.

That way I'm finally able to build host-qemu with host-glib and
host-gettext-dummy (that provides -lintl which host-glib uses).

Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
---
 scripts/lib/ptxd_make_world_install.sh             |    5 ++
 .../lib/ptxd_make_world_install_library_path.sh    |   76 ++++++++++++++++++++
 2 Dateien geändert, 81 Zeilen hinzugefügt(+)
 create mode 100644 scripts/lib/ptxd_make_world_install_library_path.sh

diff --git a/scripts/lib/ptxd_make_world_install.sh b/scripts/lib/ptxd_make_world_install.sh
index 3d5d5e5..dcb6a61 100644
--- a/scripts/lib/ptxd_make_world_install.sh
+++ b/scripts/lib/ptxd_make_world_install.sh
@@ -175,5 +175,10 @@ ptxd_make_world_install_post() {
     done &&
 
     cp -dprf -- "${pkg_pkg_dir}"/* "${pkg_sysroot_dir}"
+
+    # host and cross packages
+    if [ "${pkg_type}" != "target" ]; then
+	ptxd_make_world_install_library_path
+    fi
 }
 export -f ptxd_make_world_install_post
diff --git a/scripts/lib/ptxd_make_world_install_library_path.sh b/scripts/lib/ptxd_make_world_install_library_path.sh
new file mode 100644
index 0000000..20e570c
--- /dev/null
+++ b/scripts/lib/ptxd_make_world_install_library_path.sh
@@ -0,0 +1,76 @@
+#!/bin/bash
+#
+# (c) 2012 Bernhard Walle <bernhard@bwalle.de>
+#
+
+# This function is Darwin (Mac OS) specific. Helper function for
+# ptxd_make_world_install_library_path.
+#
+# It should be invoked on one binary in ptxdist platform-*/sysroot-host (both
+# for executables and libraries after installation. For binaries and libraries,
+# edits the dependent shared library install name to point to sysroot-host/lib.
+# For libraries only it changes also the install name (-id). That way the
+# binaries linked against a library in sysroot-target/lib can be executed
+# directly without setting environment variables.
+#
+# Arguments: $1 - the binary
+ptxd_make_world_install_library_path_one()
+{
+    local binary="$1"
+    local filename installfile
+
+    for used_library in $(otool -LX "${binary}" | awk '{print $1}') ; do
+	filename="$(basename "${used_library}")"
+
+	# if the library exists in sysroot-host, we use that one
+	# using @executable_path even keeps the tree relocatable.
+	installfile="${PTXDIST_SYSROOT_HOST}/lib/${filename}"
+	if [ -r "${installfile}" ] ; then
+	    install_name_tool -change \
+		"${used_library}" "${installfile}" \
+		"${binary}"
+	fi
+
+	# if it's a library, change also the ID so that binaries
+	# that link against the binary get it right in the first
+	# place (so they can get executed before installation)
+	if [[ ${installfile} == *.dylib ]] ; then
+	    install_name_tool -id "${binary}" "${binary}"
+	fi
+    done
+}
+export -f ptxd_make_world_install_library_path_one
+
+# This function is Darwin (Mac OS) specific. It does nothing on other
+# operating systems.
+#
+# The function gets invoked in install.post stage after the binaries
+# have been copied from the package dir to the sysroot-host directory.
+# The function must be called with the package directory as argument.
+# For every binary (library and executable) in that package directory
+# it translates the path to the correspondent path after copying
+# to the sysroot-host and invokes ptxd_make_world_install_library_path_one
+# which finally edits the install path of libaries/executables.
+ptxd_make_world_install_library_path()
+{
+    if [ "$(uname -s)" != Darwin ] ; then
+	return
+    fi
+
+    local file
+    local installed_file
+
+    echo "install.post: Fixup library paths"
+    for file in $(find "${pkg_pkg_dir}" -type f) ; do
+
+	# skip non-Mach-O files
+	file -b "${file}" | grep -q '^Mach-O' || continue
+
+	installed_file=$(echo $file | sed -e "s@${pkg_pkg_dir}@${pkg_sysroot_dir}@g")
+
+	ptxd_make_world_install_library_path_one "${installed_file}"
+    done
+
+    echo "install.post: Fixup library paths: done"
+}
+export -f ptxd_make_world_install_library_path
-- 
1.7.10.4


-- 
ptxdist mailing list
ptxdist@pengutronix.de

[ptxdist] [PATCH 1/2] Remove DYLD_FALLBACK_LIBRARY_PATH (Darwin)

[Bernhard Walle]

This has been introduced by commit 6ab38b6d244e0de21b9dd94052e0da30adc4aff3.
The problem with the environment variable is that you only can run the
compiled host tools when the environment is set. So one has to add the
environment all over in many places and even if it's set in ptxdist
globally, you can't run the programs manually.

The next patch provides a better solution.

Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
---
 rules/pre/Rules.make |    7 +------
 1 Datei geändert, 1 Zeile hinzugefügt(+), 6 Zeilen entfernt(-)

diff --git a/rules/pre/Rules.make b/rules/pre/Rules.make
index a6b662f..2d1faa6 100644
--- a/rules/pre/Rules.make
+++ b/rules/pre/Rules.make
@@ -321,10 +321,6 @@ HOST_ENV_CXX		:= CXX="$(HOSTCXX)"
 HOST_ENV_CPPFLAGS	:= CPPFLAGS="$(HOST_CPPFLAGS)"
 HOST_ENV_LDFLAGS	:= LDFLAGS="$(HOST_LDFLAGS)"
 
-ifeq ($(shell uname -s),Darwin)
-HOST_ENV_LIBRARY_PATH	:= DYLD_FALLBACK_LIBRARY_PATH=$(PTXDIST_SYSROOT_HOST)/lib
-endif
-
 HOST_ENV_PKG_CONFIG	:= \
 	PKG_CONFIG_PATH="" \
 	PKG_CONFIG_LIBDIR="$(PTXDIST_SYSROOT_HOST)/lib/pkgconfig:$(PTXDIST_SYSROOT_HOST)/share/pkgconfig"
@@ -339,8 +335,7 @@ HOST_ENV	:= \
 	$(HOST_ENV_CPPFLAGS) \
 	$(HOST_ENV_LDFLAGS) \
 	$(HOST_ENV_PKG_CONFIG) \
-	$(HOST_ENV_PYTHONPATH) \
-	$(HOST_ENV_LIBRARY_PATH)
+	$(HOST_ENV_PYTHONPATH)
 
 
 HOST_AUTOCONF  := --prefix=
-- 
1.7.10.4


-- 
ptxdist mailing list
ptxdist@pengutronix.de

[ptxdist] [PATCH 2/2] ptxd_make_world_install_post: Darwin: Edit install name of binaries

[Bernhard Walle]

This patch should solve the problem that binaries in
$PTXDIST_SYSROOT_HOST/bin must be able to find their libraries in
$PTXDIST_SYSROOT_HOST/lib. On ELF systems that problem is solved
by using rpath. However, rpath doesn't exist (works different) on
Darwin that uses the Mach-O binary format.

After copying the files to $PTXDIST_SYSROOT_HOST, we do following:

 - For every library (lib/*.*.dylib), we change the install name
   from /lib/libfoo.x.dylib to $PTXDIST_SYSROOT_HOST/lib/libfoo.x.dylib.
   That way programs linked against that library can be executed
   immediately, even in the build directory before installation
   to $PTXDIST_SYSROOT_HOST.

   That's the call of "install_name_tool -id".

 - For every library (lib/*.*.dylib) and executable (bin/*), we
   change the install name of dependent libraries to point to
   $PTXDIST_SYSROOT_HOST/lib. The call of "otool -L" lists the
   dependent libraries (like "readelf -d" on ELF systems) and
   the call "install_name_tool -change" changes it. That step
   is necessary because multiple libraries and binaries can be
   built and installed at the same time, so the binary is already
   built before "install_name_tool -id" is invoked.

That way I'm finally able to build host-qemu with host-glib and
host-gettext-dummy (that provides -lintl which host-glib uses).

Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
---
 scripts/lib/ptxd_make_world_install.sh             |    5 ++
 .../lib/ptxd_make_world_install_library_path.sh    |   76 ++++++++++++++++++++
 2 Dateien geändert, 81 Zeilen hinzugefügt(+)
 create mode 100644 scripts/lib/ptxd_make_world_install_library_path.sh

diff --git a/scripts/lib/ptxd_make_world_install.sh b/scripts/lib/ptxd_make_world_install.sh
index 3d5d5e5..dcb6a61 100644
--- a/scripts/lib/ptxd_make_world_install.sh
+++ b/scripts/lib/ptxd_make_world_install.sh
@@ -175,5 +175,10 @@ ptxd_make_world_install_post() {
     done &&
 
     cp -dprf -- "${pkg_pkg_dir}"/* "${pkg_sysroot_dir}"
+
+    # host and cross packages
+    if [ "${pkg_type}" != "target" ]; then
+	ptxd_make_world_install_library_path
+    fi
 }
 export -f ptxd_make_world_install_post
diff --git a/scripts/lib/ptxd_make_world_install_library_path.sh b/scripts/lib/ptxd_make_world_install_library_path.sh
new file mode 100644
index 0000000..20e570c
--- /dev/null
+++ b/scripts/lib/ptxd_make_world_install_library_path.sh
@@ -0,0 +1,76 @@
+#!/bin/bash
+#
+# (c) 2012 Bernhard Walle <bernhard@bwalle.de>
+#
+
+# This function is Darwin (Mac OS) specific. Helper function for
+# ptxd_make_world_install_library_path.
+#
+# It should be invoked on one binary in ptxdist platform-*/sysroot-host (both
+# for executables and libraries after installation. For binaries and libraries,
+# edits the dependent shared library install name to point to sysroot-host/lib.
+# For libraries only it changes also the install name (-id). That way the
+# binaries linked against a library in sysroot-target/lib can be executed
+# directly without setting environment variables.
+#
+# Arguments: $1 - the binary
+ptxd_make_world_install_library_path_one()
+{
+    local binary="$1"
+    local filename installfile
+
+    for used_library in $(otool -LX "${binary}" | awk '{print $1}') ; do
+	filename="$(basename "${used_library}")"
+
+	# if the library exists in sysroot-host, we use that one
+	# using @executable_path even keeps the tree relocatable.
+	installfile="${PTXDIST_SYSROOT_HOST}/lib/${filename}"
+	if [ -r "${installfile}" ] ; then
+	    install_name_tool -change \
+		"${used_library}" "${installfile}" \
+		"${binary}"
+	fi
+
+	# if it's a library, change also the ID so that binaries
+	# that link against the binary get it right in the first
+	# place (so they can get executed before installation)
+	if [[ ${installfile} == *.dylib ]] ; then
+	    install_name_tool -id "${binary}" "${binary}"
+	fi
+    done
+}
+export -f ptxd_make_world_install_library_path_one
+
+# This function is Darwin (Mac OS) specific. It does nothing on other
+# operating systems.
+#
+# The function gets invoked in install.post stage after the binaries
+# have been copied from the package dir to the sysroot-host directory.
+# The function must be called with the package directory as argument.
+# For every binary (library and executable) in that package directory
+# it translates the path to the correspondent path after copying
+# to the sysroot-host and invokes ptxd_make_world_install_library_path_one
+# which finally edits the install path of libaries/executables.
+ptxd_make_world_install_library_path()
+{
+    if [ "$(uname -s)" != Darwin ] ; then
+	return
+    fi
+
+    local file
+    local installed_file
+
+    echo "install.post: Fixup library paths"
+    for file in $(find "${pkg_pkg_dir}" -type f) ; do
+
+	# skip non-Mach-O files
+	file -b "${file}" | grep -q '^Mach-O' || continue
+
+	installed_file=$(echo $file | sed -e "s@${pkg_pkg_dir}@${pkg_sysroot_dir}@g")
+
+	ptxd_make_world_install_library_path_one "${installed_file}"
+    done
+
+    echo "install.post: Fixup library paths: done"
+}
+export -f ptxd_make_world_install_library_path
-- 
1.7.10.4


-- 
ptxdist mailing list
ptxdist@pengutronix.de

[ptxdist] [PATCH 1/2] Remove DYLD_FALLBACK_LIBRARY_PATH (Darwin)

[Bernhard Walle]

This has been introduced by commit 6ab38b6d244e0de21b9dd94052e0da30adc4aff3.
The problem with the environment variable is that you only can run the
compiled host tools when the environment is set. So one has to add the
environment all over in many places and even if it's set in ptxdist
globally, you can't run the programs manually.

The next patch provides a better solution.

Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
---
 rules/pre/Rules.make |    7 +------
 1 Datei geändert, 1 Zeile hinzugefügt(+), 6 Zeilen entfernt(-)

diff --git a/rules/pre/Rules.make b/rules/pre/Rules.make
index a6b662f..2d1faa6 100644
--- a/rules/pre/Rules.make
+++ b/rules/pre/Rules.make
@@ -321,10 +321,6 @@ HOST_ENV_CXX		:= CXX="$(HOSTCXX)"
 HOST_ENV_CPPFLAGS	:= CPPFLAGS="$(HOST_CPPFLAGS)"
 HOST_ENV_LDFLAGS	:= LDFLAGS="$(HOST_LDFLAGS)"
 
-ifeq ($(shell uname -s),Darwin)
-HOST_ENV_LIBRARY_PATH	:= DYLD_FALLBACK_LIBRARY_PATH=$(PTXDIST_SYSROOT_HOST)/lib
-endif
-
 HOST_ENV_PKG_CONFIG	:= \
 	PKG_CONFIG_PATH="" \
 	PKG_CONFIG_LIBDIR="$(PTXDIST_SYSROOT_HOST)/lib/pkgconfig:$(PTXDIST_SYSROOT_HOST)/share/pkgconfig"
@@ -339,8 +335,7 @@ HOST_ENV	:= \
 	$(HOST_ENV_CPPFLAGS) \
 	$(HOST_ENV_LDFLAGS) \
 	$(HOST_ENV_PKG_CONFIG) \
-	$(HOST_ENV_PYTHONPATH) \
-	$(HOST_ENV_LIBRARY_PATH)
+	$(HOST_ENV_PYTHONPATH)
 
 
 HOST_AUTOCONF  := --prefix=
-- 
1.7.10.4


-- 
ptxdist mailing list
ptxdist@pengutronix.de

[ptxdist] [PATCH 2/2] ptxd_make_world_install_post: Darwin: Edit install name of binaries

[Bernhard Walle]

This patch should solve the problem that binaries in
$PTXDIST_SYSROOT_HOST/bin must be able to find their libraries in
$PTXDIST_SYSROOT_HOST/lib. On ELF systems that problem is solved
by using rpath. However, rpath doesn't exist (works different) on
Darwin that uses the Mach-O binary format.

After copying the files to $PTXDIST_SYSROOT_HOST, we do following:

 - For every library (lib/*.*.dylib), we change the install name
   from /lib/libfoo.x.dylib to $PTXDIST_SYSROOT_HOST/lib/libfoo.x.dylib.
   That way programs linked against that library can be executed
   immediately, even in the build directory before installation
   to $PTXDIST_SYSROOT_HOST.

   That's the call of "install_name_tool -id".

 - For every library (lib/*.*.dylib) and executable (bin/*), we
   change the install name of dependent libraries to point to
   $PTXDIST_SYSROOT_HOST/lib. The call of "otool -L" lists the
   dependent libraries (like "readelf -d" on ELF systems) and
   the call "install_name_tool -change" changes it. That step
   is necessary because multiple libraries and binaries can be
   built and installed at the same time, so the binary is already
   built before "install_name_tool -id" is invoked.

That way I'm finally able to build host-qemu with host-glib and
host-gettext-dummy (that provides -lintl which host-glib uses).

Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
---
 scripts/lib/ptxd_make_world_install.sh             |    5 ++
 .../lib/ptxd_make_world_install_library_path.sh    |   76 ++++++++++++++++++++
 2 Dateien geändert, 81 Zeilen hinzugefügt(+)
 create mode 100644 scripts/lib/ptxd_make_world_install_library_path.sh

diff --git a/scripts/lib/ptxd_make_world_install.sh b/scripts/lib/ptxd_make_world_install.sh
index 3d5d5e5..dcb6a61 100644
--- a/scripts/lib/ptxd_make_world_install.sh
+++ b/scripts/lib/ptxd_make_world_install.sh
@@ -175,5 +175,10 @@ ptxd_make_world_install_post() {
     done &&
 
     cp -dprf -- "${pkg_pkg_dir}"/* "${pkg_sysroot_dir}"
+
+    # host and cross packages
+    if [ "${pkg_type}" != "target" ]; then
+	ptxd_make_world_install_library_path
+    fi
 }
 export -f ptxd_make_world_install_post
diff --git a/scripts/lib/ptxd_make_world_install_library_path.sh b/scripts/lib/ptxd_make_world_install_library_path.sh
new file mode 100644
index 0000000..20e570c
--- /dev/null
+++ b/scripts/lib/ptxd_make_world_install_library_path.sh
@@ -0,0 +1,76 @@
+#!/bin/bash
+#
+# (c) 2012 Bernhard Walle <bernhard@bwalle.de>
+#
+
+# This function is Darwin (Mac OS) specific. Helper function for
+# ptxd_make_world_install_library_path.
+#
+# It should be invoked on one binary in ptxdist platform-*/sysroot-host (both
+# for executables and libraries after installation. For binaries and libraries,
+# edits the dependent shared library install name to point to sysroot-host/lib.
+# For libraries only it changes also the install name (-id). That way the
+# binaries linked against a library in sysroot-target/lib can be executed
+# directly without setting environment variables.
+#
+# Arguments: $1 - the binary
+ptxd_make_world_install_library_path_one()
+{
+    local binary="$1"
+    local filename installfile
+
+    for used_library in $(otool -LX "${binary}" | awk '{print $1}') ; do
+	filename="$(basename "${used_library}")"
+
+	# if the library exists in sysroot-host, we use that one
+	# using @executable_path even keeps the tree relocatable.
+	installfile="${PTXDIST_SYSROOT_HOST}/lib/${filename}"
+	if [ -r "${installfile}" ] ; then
+	    install_name_tool -change \
+		"${used_library}" "${installfile}" \
+		"${binary}"
+	fi
+
+	# if it's a library, change also the ID so that binaries
+	# that link against the binary get it right in the first
+	# place (so they can get executed before installation)
+	if [[ ${installfile} == *.dylib ]] ; then
+	    install_name_tool -id "${binary}" "${binary}"
+	fi
+    done
+}
+export -f ptxd_make_world_install_library_path_one
+
+# This function is Darwin (Mac OS) specific. It does nothing on other
+# operating systems.
+#
+# The function gets invoked in install.post stage after the binaries
+# have been copied from the package dir to the sysroot-host directory.
+# The function must be called with the package directory as argument.
+# For every binary (library and executable) in that package directory
+# it translates the path to the correspondent path after copying
+# to the sysroot-host and invokes ptxd_make_world_install_library_path_one
+# which finally edits the install path of libaries/executables.
+ptxd_make_world_install_library_path()
+{
+    if [ "$(uname -s)" != Darwin ] ; then
+	return
+    fi
+
+    local file
+    local installed_file
+
+    echo "install.post: Fixup library paths"
+    for file in $(find "${pkg_pkg_dir}" -type f) ; do
+
+	# skip non-Mach-O files
+	file -b "${file}" | grep -q '^Mach-O' || continue
+
+	installed_file=$(echo $file | sed -e "s@${pkg_pkg_dir}@${pkg_sysroot_dir}@g")
+
+	ptxd_make_world_install_library_path_one "${installed_file}"
+    done
+
+    echo "install.post: Fixup library paths: done"
+}
+export -f ptxd_make_world_install_library_path
-- 
1.7.10.4


-- 
ptxdist mailing list
ptxdist@pengutronix.de

Re: [ptxdist] [PATCH 2/2] ptxd_make_world_install_post: Darwin: Edit install name of binaries

[Michael Olbrich]

On Mon, Jun 18, 2012 at 09:42:54PM +0200, Bernhard Walle wrote:
> This patch should solve the problem that binaries in
> $PTXDIST_SYSROOT_HOST/bin must be able to find their libraries in
> $PTXDIST_SYSROOT_HOST/lib. On ELF systems that problem is solved
> by using rpath. However, rpath doesn't exist (works different) on
> Darwin that uses the Mach-O binary format.
> 
> After copying the files to $PTXDIST_SYSROOT_HOST, we do following:
> 
>  - For every library (lib/*.*.dylib), we change the install name
>    from /lib/libfoo.x.dylib to $PTXDIST_SYSROOT_HOST/lib/libfoo.x.dylib.
>    That way programs linked against that library can be executed
>    immediately, even in the build directory before installation
>    to $PTXDIST_SYSROOT_HOST.
> 
>    That's the call of "install_name_tool -id".
> 
>  - For every library (lib/*.*.dylib) and executable (bin/*), we
>    change the install name of dependent libraries to point to
>    $PTXDIST_SYSROOT_HOST/lib. The call of "otool -L" lists the
>    dependent libraries (like "readelf -d" on ELF systems) and
>    the call "install_name_tool -change" changes it. That step
>    is necessary because multiple libraries and binaries can be
>    built and installed at the same time, so the binary is already
>    built before "install_name_tool -id" is invoked.
> 
> That way I'm finally able to build host-qemu with host-glib and
> host-gettext-dummy (that provides -lintl which host-glib uses).

Sorry about the delay. Both patches applied.

Michael

> Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
> ---
>  scripts/lib/ptxd_make_world_install.sh             |    5 ++
>  .../lib/ptxd_make_world_install_library_path.sh    |   76 ++++++++++++++++++++
>  2 Dateien geändert, 81 Zeilen hinzugefügt(+)
>  create mode 100644 scripts/lib/ptxd_make_world_install_library_path.sh
> 
> diff --git a/scripts/lib/ptxd_make_world_install.sh b/scripts/lib/ptxd_make_world_install.sh
> index 3d5d5e5..dcb6a61 100644
> --- a/scripts/lib/ptxd_make_world_install.sh
> +++ b/scripts/lib/ptxd_make_world_install.sh
> @@ -175,5 +175,10 @@ ptxd_make_world_install_post() {
>      done &&
>  
>      cp -dprf -- "${pkg_pkg_dir}"/* "${pkg_sysroot_dir}"
> +
> +    # host and cross packages
> +    if [ "${pkg_type}" != "target" ]; then
> +	ptxd_make_world_install_library_path
> +    fi
>  }
>  export -f ptxd_make_world_install_post
> diff --git a/scripts/lib/ptxd_make_world_install_library_path.sh b/scripts/lib/ptxd_make_world_install_library_path.sh
> new file mode 100644
> index 0000000..20e570c
> --- /dev/null
> +++ b/scripts/lib/ptxd_make_world_install_library_path.sh
> @@ -0,0 +1,76 @@
> +#!/bin/bash
> +#
> +# (c) 2012 Bernhard Walle <bernhard@bwalle.de>
> +#
> +
> +# This function is Darwin (Mac OS) specific. Helper function for
> +# ptxd_make_world_install_library_path.
> +#
> +# It should be invoked on one binary in ptxdist platform-*/sysroot-host (both
> +# for executables and libraries after installation. For binaries and libraries,
> +# edits the dependent shared library install name to point to sysroot-host/lib.
> +# For libraries only it changes also the install name (-id). That way the
> +# binaries linked against a library in sysroot-target/lib can be executed
> +# directly without setting environment variables.
> +#
> +# Arguments: $1 - the binary
> +ptxd_make_world_install_library_path_one()
> +{
> +    local binary="$1"
> +    local filename installfile
> +
> +    for used_library in $(otool -LX "${binary}" | awk '{print $1}') ; do
> +	filename="$(basename "${used_library}")"
> +
> +	# if the library exists in sysroot-host, we use that one
> +	# using @executable_path even keeps the tree relocatable.
> +	installfile="${PTXDIST_SYSROOT_HOST}/lib/${filename}"
> +	if [ -r "${installfile}" ] ; then
> +	    install_name_tool -change \
> +		"${used_library}" "${installfile}" \
> +		"${binary}"
> +	fi
> +
> +	# if it's a library, change also the ID so that binaries
> +	# that link against the binary get it right in the first
> +	# place (so they can get executed before installation)
> +	if [[ ${installfile} == *.dylib ]] ; then
> +	    install_name_tool -id "${binary}" "${binary}"
> +	fi
> +    done
> +}
> +export -f ptxd_make_world_install_library_path_one
> +
> +# This function is Darwin (Mac OS) specific. It does nothing on other
> +# operating systems.
> +#
> +# The function gets invoked in install.post stage after the binaries
> +# have been copied from the package dir to the sysroot-host directory.
> +# The function must be called with the package directory as argument.
> +# For every binary (library and executable) in that package directory
> +# it translates the path to the correspondent path after copying
> +# to the sysroot-host and invokes ptxd_make_world_install_library_path_one
> +# which finally edits the install path of libaries/executables.
> +ptxd_make_world_install_library_path()
> +{
> +    if [ "$(uname -s)" != Darwin ] ; then
> +	return
> +    fi
> +
> +    local file
> +    local installed_file
> +
> +    echo "install.post: Fixup library paths"
> +    for file in $(find "${pkg_pkg_dir}" -type f) ; do
> +
> +	# skip non-Mach-O files
> +	file -b "${file}" | grep -q '^Mach-O' || continue
> +
> +	installed_file=$(echo $file | sed -e "s@${pkg_pkg_dir}@${pkg_sysroot_dir}@g")
> +
> +	ptxd_make_world_install_library_path_one "${installed_file}"
> +    done
> +
> +    echo "install.post: Fixup library paths: done"
> +}
> +export -f ptxd_make_world_install_library_path
> -- 
> 1.7.10.4
> 
> 
> -- 
> ptxdist mailing list
> ptxdist@pengutronix.de

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

-- 
ptxdist mailing list
ptxdist@pengutronix.de