Re: [ptxdist] pkg-config-wrapper issue

Re: [ptxdist] pkg-config-wrapper issue

[Ladislav Michl]

On Mon, Oct 03, 2022 at 06:14:34PM +0000, Richard OGrady wrote:
> Hello,
> 
> We are using ptxdist 2022.07 and are building 2 packages, libimxvpuapi2 and libimxdmabuffer.
> libimxvpuapi2 depends on libimxdmabuffer and accordingly libimxvpuapi2.in selects libimxdmabuffer.

Those 'depends on' and 'select' are actual keywords used in *.in files
or just a description?

> The libimxvpuapi2 build fails during config.  The config file shows
> 
> Checking for 'libimxdmabuffer >= 0.9.0'
> ['/home/rich/imx6_3D104X_fsl6/Grayhill-iMX6/platform-3D50_std/sysroot-cross/bin/pkg-config', '--cflags', '--libs', 'libimxdmabuffer', '>=', '0.9.0']
> err: pkg-config: warning: blocking 'libimxdmabuffer': not selected by 'libimxvpuapi2'
> 
> So pkg-config-wrapper is blocking the libimxdmabuffer package.  The code that does this in pkg-config-wrapper is:
> 
>         for pkg in ${pkgs}; do
>            if [[ ! " ${whitelist} " =~ " ${pkg} " && ! "${pkg}" =~ '-uninstalled' ]]; then
>               echo "$(basename ${0}): warning: blocking '${pkg}': not selected by '${PKGCONFIG_WHITELIST_SRC}'" >&${ERROR_FD}
>               exit 1
>            fi
>        done
> 
> We can get rid of the issue by commenting out the code above in pkg-config-wrapper but it seems that then breaks some other packages.
> Libimxdmabuffer is version 1.1.2.  Both packages are using waf to build.
> 
> What could be going wrong?

It is a bit hard to tell without seeing actutal code, but from description
above I guess you created circular dependency which got dropped.

> Rich OGrady
> 
> 

Re: [ptxdist] pkg-config-wrapper issue

[Richard OGrady]

> -----Original Message-----
> From: ptxdist <ptxdist-bounces@pengutronix.de> On Behalf Of Ladislav Michl
> Sent: Monday, October 3, 2022 3:44 PM
> To: ptxdist@pengutronix.de
> Subject: Re: [ptxdist] pkg-config-wrapper issue
> 
> On Mon, Oct 03, 2022 at 06:14:34PM +0000, Richard OGrady wrote:
> > Hello,
> >
> > We are using ptxdist 2022.07 and are building 2 packages, libimxvpuapi2 and libimxdmabuffer.
> > libimxvpuapi2 depends on libimxdmabuffer and accordingly libimxvpuapi2.in selects libimxdmabuffer.
> 
> Those 'depends on' and 'select' are actual keywords used in *.in files or just a
> description?
> 
'depends on' is just a statement of fact, 'select' is keyword in libimxvpuapi2.in

> > The libimxvpuapi2 build fails during config.  The config file shows
> >
> > Checking for 'libimxdmabuffer >= 0.9.0'
> > ['/home/rich/imx6_3D104X_fsl6/Grayhill-iMX6/platform-3D50_std/sysroot-
> > cross/bin/pkg-config', '--cflags', '--libs', 'libimxdmabuffer', '>=',
> > '0.9.0']
> > err: pkg-config: warning: blocking 'libimxdmabuffer': not selected by 'libimxvpuapi2'
> >
> > So pkg-config-wrapper is blocking the libimxdmabuffer package.  The code that does this in pkg-config-wrapper is:
> >
> >         for pkg in ${pkgs}; do
> >            if [[ ! " ${whitelist} " =~ " ${pkg} " && ! "${pkg}" =~ '-uninstalled' ]]; then
> >               echo "$(basename ${0}): warning: blocking '${pkg}': not selected by '${PKGCONFIG_WHITELIST_SRC}'" >&${ERROR_FD}
> >               exit 1
> >            fi
> >        done
> >
> > We can get rid of the issue by commenting out the code above in pkg-config-wrapper but it seems that then breaks some other packages.
> > Libimxdmabuffer is version 1.1.2.  Both packages are using waf to build.
> >
> > What could be going wrong?
> 
> It is a bit hard to tell without seeing actutal code, but from description above I
> guess you created circular dependency which got dropped.
> 
I guess another way to ask the question is what is that code chunk in pkg-config-wrapper trying to do?
It seems that if ${whitelist} were to contain ${pkg} then it would not bail out.  Correct?
If so, then by what mechanism would ${whitelist} contain ${pkg}?
> > Rich OGrady
> >
> >

Re: [ptxdist] pkg-config-wrapper

[Michael Olbrich]

Hi,

On Wed, Oct 05, 2022 at 08:44:44PM +0000, Richard OGrady wrote:
> Debugging a prepare stage issue so trying to understand this chunk of code in pkg-config-wrapper.
> 
> for pkg in ${pkgs}; do
>    if [[ ! " ${whitelist} " =~ " ${pkg} " && ! "${pkg}" =~ '-uninstalled' ]]; then
>       echo "$(basename ${0}): warning: blocking '${pkg}': not selected by '${PKGCONFIG_WHITELIST_SRC}'" >&${ERROR_FD}
>       exit 1
>   fi
> done
> 
> Where does ${whitelist} come from?  Have tried to trace this back and best guess is it may come from ../state/${PKGCONFIG_WHITELIST_SRC}. pkgconfig .
> 
> If this is the case, perhaps pkg-config-wrapper can't be used during prepare as the .pkgconfig file isn't there yet?

${whitelist} is defined as PKGCONFIG_WHITELIST_HOST or
PKGCONFIG_WHITELIST_TARGET depending on whether the host or cross pkgconfig
is called. For example PKGCONFIG_WHITELIST_HOST is used by when the Linux
kernel looks for openssl to build a host tool.

These variables are filled by the .pkgconfig files from all packages that
are selected in the .in file. So those are already available during the
prepare stage.
During the install stage the new .pkgconfig for the package is created. It
is filled with the name of any .pc file the package itself provides and the
content of the .pkgconfig files of all direct dependencies. This way, the
.pkgconfig will contain all pkg-config packages of the ptxdist package and
all its dependencies.

So if you hit this then your package is not selecting the necessary
dependencies.
Btw, if you run ptxdist with '-v' then the full prepare command is logged
including PKGCONFIG_WHITELIST_HOST= / PKGCONFIG_WHITELIST_TARGET=
environment variables. Take a look, what's actually listed there.

Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] pkg-config-wrapper

[Michael Olbrich]

Hi,

please keep the mailing list in Cc.

On Fri, Oct 07, 2022 at 02:43:51PM +0000, Richard OGrady wrote:
> > On Wed, Oct 05, 2022 at 08:44:44PM +0000, Richard OGrady wrote:
> > > Debugging a prepare stage issue so trying to understand this chunk of code in pkg-config-wrapper.
> > >
> > > for pkg in ${pkgs}; do
> > >    if [[ ! " ${whitelist} " =~ " ${pkg} " && ! "${pkg}" =~ '-uninstalled' ]]; then
> > >       echo "$(basename ${0}): warning: blocking '${pkg}': not selected by
> > '${PKGCONFIG_WHITELIST_SRC}'" >&${ERROR_FD}
> > >       exit 1
> > >   fi
> > > done
> > >
> > > Where does ${whitelist} come from?  Have tried to trace this back and best guess is it may come
> > from ../state/${PKGCONFIG_WHITELIST_SRC}. pkgconfig .
> > >
> > > If this is the case, perhaps pkg-config-wrapper can't be used during prepare as the .pkgconfig file
> > > isn't there yet?
> > 
> > ${whitelist} is defined as PKGCONFIG_WHITELIST_HOST or PKGCONFIG_WHITELIST_TARGET
> > depending on whether the host or cross pkgconfig is called. For example
> > PKGCONFIG_WHITELIST_HOST is used by when the Linux kernel looks for openssl to build a host tool.
> > 
> > These variables are filled by the .pkgconfig files from all packages that are selected in the .in file. So
> > those are already available during the prepare stage.
> 
> I think this is where I am having a problem.  My file gstreamer2-imx.in
> select(s) the dependency libimxdmabuffer.  My file libimxdmabuffer.in
> only select(s) the dependency HOST_SYSTEM_PYTHON3 and its pkgconfig file
> is empty as expected.  So the dependency libimxdmabuffer isn't in
> PKGCONFIG_WHITELIST_TARGET when gstreamer2-imx is prepared and the
> pkg-config-wrapper blocks it.
> 
> From your description it seems that primary dependencies (like
> libimxdmabuffer) are not in PKGCONFIG_WHITELIST_TARGET.  Is that the
> case? I would think this is a common scenario so I must still be missing
> something.

When you build gstreamer2-imx, then libimxdmabuffer should be in
PKGCONFIG_WHITELIST_TARGET. But please note, that
PKGCONFIG_WHITELIST_TARGET does not contain the ptxdist package names but
the .pc file names without the .pc suffix.
And the .pkgconfig of libimxdmabuffer should contain something. Is the file
empty or is it not generated at all? What's the output of:

find platform-<your-platform>/packages/libimxdmabuffer-<version>/ -name *.pc

Replace the path with the directory that is filled by libimxdmabuffer.

Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] pkg-config-wrapper

[Richard OGrady]

> -----Original Message-----
> From: Michael Olbrich <m.olbrich@pengutronix.de>
> Sent: Friday, October 7, 2022 10:29 AM
> To: Richard OGrady <Rich_Ogrady@grayhill.com>
> Cc: ptxdist@pengutronix.de
> Subject: Re: [ptxdist] pkg-config-wrapper
> 
> On Fri, Oct 07, 2022 at 02:43:51PM +0000, Richard OGrady wrote:
> > > On Wed, Oct 05, 2022 at 08:44:44PM +0000, Richard OGrady wrote:
> > > > Debugging a prepare stage issue so trying to understand this chunk of code in pkg-config-wrapper.
> > > >
> > > > for pkg in ${pkgs}; do
> > > >    if [[ ! " ${whitelist} " =~ " ${pkg} " && ! "${pkg}" =~ '-uninstalled' ]]; then
> > > >       echo "$(basename ${0}): warning: blocking '${pkg}': not
> > > > selected by
> > > '${PKGCONFIG_WHITELIST_SRC}'" >&${ERROR_FD}
> > > >       exit 1
> > > >   fi
> > > > done
> > > >
> > > > Where does ${whitelist} come from?  Have tried to trace this back
> > > > and best guess is it may come
> > > from ../state/${PKGCONFIG_WHITELIST_SRC}. pkgconfig .
> > > >
> > > > If this is the case, perhaps pkg-config-wrapper can't be used
> > > > during prepare as the .pkgconfig file isn't there yet?
> > >
> > > ${whitelist} is defined as PKGCONFIG_WHITELIST_HOST or
> > > PKGCONFIG_WHITELIST_TARGET depending on whether the host or cross
> > > pkgconfig is called. For example PKGCONFIG_WHITELIST_HOST is used by when the Linux kernel looks for openssl to build a host tool.
> > >
> > > These variables are filled by the .pkgconfig files from all packages
> > > that are selected in the .in file. So those are already available during the prepare stage.
> >
> > I think this is where I am having a problem.  My file
> > gstreamer2-imx.in
> > select(s) the dependency libimxdmabuffer.  My file libimxdmabuffer.in
> > only select(s) the dependency HOST_SYSTEM_PYTHON3 and its pkgconfig
> > file is empty as expected.  So the dependency libimxdmabuffer isn't in
> > PKGCONFIG_WHITELIST_TARGET when gstreamer2-imx is prepared and the
> > pkg-config-wrapper blocks it.
> >
> > From your description it seems that primary dependencies (like
> > libimxdmabuffer) are not in PKGCONFIG_WHITELIST_TARGET.  Is that the
> > case? I would think this is a common scenario so I must still be
> > missing something.
> 
> When you build gstreamer2-imx, then libimxdmabuffer should be in PKGCONFIG_WHITELIST_TARGET.
> But please note, that PKGCONFIG_WHITELIST_TARGET does not contain the ptxdist package names but
> the .pc file names without the .pc suffix.
> And the .pkgconfig of libimxdmabuffer should contain something. Is the file empty or is it not
> generated at all? What's the output of:
> 
> find platform-<your-platform>/packages/libimxdmabuffer-<version>/ -name *.pc
> 
> Replace the path with the directory that is filled by libimxdmabuffer.
> 
> Michael
> 
That is the problem.  The .pc file did not exist.  That package is a waf configure/build/install.  
To fix I needed to explicitly add --destdir to the install command.  It works now.
Thank you.

Rich OGrady