[ptxdist] Globally define GIT_CEILING_DIRECTORIES?

[ptxdist] Globally define GIT_CEILING_DIRECTORIES?

[Sven Püschel]

I would suggest globally setting the GIT_CEILING_DIRECTORIES environment 
variable when building packages.

Besides lshw, sockperf is now the second time I've encountered the 
buildsystem to check the current git repository. But as Git also checks 
the parent directories when searching for Git repositories, these checks 
may result in getting an unwanted Git repository (usually the BSP), 
resulting in package versions with a Git commit/version that has nothing 
to do with the actual package.

With GIT_CEILING_DIRECTORIES set to the parent directory of the 
extracted sources, Git will stop searching for parent Git repositories, 
when it encounters this directory. See my sockperf v2 patch for an 
example of using this environment variable.

Given this is a global change, does anyone see a potential problem?


Best regards

Sven

Re: [ptxdist] Globally define GIT_CEILING_DIRECTORIES?

[Christian Melki]

Hi Sven.

On 2/13/23 2:58 PM, Sven Püschel wrote:
> I would suggest globally setting the GIT_CEILING_DIRECTORIES environment variable when building packages.
> 
> Besides lshw, sockperf is now the second time I've encountered the buildsystem to check the current git repository. But as Git also checks the parent directories when searching for Git repositories, these checks may result in getting an unwanted Git repository (usually the BSP), resulting in package versions with a Git commit/version that has nothing to do with the actual package.
> 
> With GIT_CEILING_DIRECTORIES set to the parent directory of the extracted sources, Git will stop searching for parent Git repositories, when it encounters this directory. See my sockperf v2 patch for an example of using this environment variable.
> 
> Given this is a global change, does anyone see a potential problem?
> 
> 
> Best regards
> 
> Sven
> 
> 

Just to summarize so I understand this.
You want to limit the searching for git repos,
so that if it isn't found in the current posix PWD (cwd), it's not a git repo?
I'm assuming it's expandable by doing an export to $PWD?

Have you tried this as an global?
Afaict, sockperf treats this locally?

Also, do you have any anecdotal or factual history from other build environments?
Some of them rely extensively on git hashes, ie, repos.
They've likely stumbled upon the same question.

Regards,
Christian

Re: [ptxdist] Globally define GIT_CEILING_DIRECTORIES?

[Sven Püschel]

Hi Christian,


> Just to summarize so I understand this.
> You want to limit the searching for git repos,
> so that if it isn't found in the current posix PWD (cwd), it's not a 
> git repo?
> I'm assuming it's expandable by doing an export to $PWD?

I only want to limit the parent search at a specific directory, not to 
the current directory. Practically I'm using DistroKit as a BSP with 
PTXDist. The target package (e.g. sockperf) is downloaded and extracted 
to /path/to/DistroKit/platform-v8a/build-target/sockperf-3.10 . 
Currently sockperf executes "git rev-parse HEAD" to get the current 
commit, that is attached to the version number. As the sockperf-3.10 
directory itself isn't a Git directory, git checks the parent 
directories until it finds the Git repository of DistroKit and returns 
it's commit hash. This results in sockperf having a Distrokit commit as 
part of it's version number. lshw as another example used the version 
tag of Distrokit as it's own version.

When setting 
GIT_CEILING_DIR=/path/to/DistroKit/platform-v8a/build-target , then Git 
will still backtrack, but stop when it reaches the build-target 
directory. E.g. when 
/path/to/Distrokit/platform-v8a/build-target/sockperf-3.10 is a Git 
repository and some buildscript calls git from 
/path/to/DistroKit/platform-v8a/build-target/sockperf-3.10/src , then 
Git still traverses the parent directory to find the sockperf Git 
repository. But if sockperf-3.10 isn't a git repo, it will stop 
backtracking to the build-target directory and return that it didn't 
find a git repository.

> Have you tried this as an global?
> Afaict, sockperf treats this locally?

I've added it locally to sockperf via

SOCKPERF_CONF_ENV       := $(CROSS_ENV) GIT_CEILING_DIRECTORIES="$(BUILDDIR)"
SOCKPERF_MAKE_ENV       := $(CROSS_ENV) GIT_CEILING_DIRECTORIES="$(BUILDDIR)"

I haven't tried this globally yet, as I'm not that familiar with the 
PTXDist internals and didn't wanted to spend multiple hours creating a 
patch and then get the feedback, that there might be a better way to do it.

> Also, do you have any anecdotal or factual history from other build 
> environments?
> Some of them rely extensively on git hashes, ie, repos.
> They've likely stumbled upon the same question.

Nope, PTXDist is my first embedded build environment.


Best regards

Sven

Re: [ptxdist] Globally define GIT_CEILING_DIRECTORIES?

[Christian Melki]

On 2/13/23 16:03, Sven Püschel wrote:
> Hi Christian,
> 
> 
>> Just to summarize so I understand this.
>> You want to limit the searching for git repos,
>> so that if it isn't found in the current posix PWD (cwd), it's not a 
>> git repo?
>> I'm assuming it's expandable by doing an export to $PWD?
> 
> I only want to limit the parent search at a specific directory, not to 
> the current directory. Practically I'm using DistroKit as a BSP with 
> PTXDist. The target package (e.g. sockperf) is downloaded and extracted 
> to /path/to/DistroKit/platform-v8a/build-target/sockperf-3.10 . 
> Currently sockperf executes "git rev-parse HEAD" to get the current 
> commit, that is attached to the version number. As the sockperf-3.10 
> directory itself isn't a Git directory, git checks the parent 
> directories until it finds the Git repository of DistroKit and returns 
> it's commit hash. This results in sockperf having a Distrokit commit as 
> part of it's version number. lshw as another example used the version 
> tag of Distrokit as it's own version.
> 
> When setting 
> GIT_CEILING_DIR=/path/to/DistroKit/platform-v8a/build-target , then Git 
> will still backtrack, but stop when it reaches the build-target 
> directory. E.g. when 
> /path/to/Distrokit/platform-v8a/build-target/sockperf-3.10 is a Git 
> repository and some buildscript calls git from 
> /path/to/DistroKit/platform-v8a/build-target/sockperf-3.10/src , then 
> Git still traverses the parent directory to find the sockperf Git 
> repository. But if sockperf-3.10 isn't a git repo, it will stop 
> backtracking to the build-target directory and return that it didn't 
> find a git repository.
>
>> Have you tried this as an global?
>> Afaict, sockperf treats this locally?
> 
> I've added it locally to sockperf via
> 
> SOCKPERF_CONF_ENV       := $(CROSS_ENV) GIT_CEILING_DIRECTORIES="$(BUILDDIR)"
> SOCKPERF_MAKE_ENV       := $(CROSS_ENV) GIT_CEILING_DIRECTORIES="$(BUILDDIR)"
> 
> I haven't tried this globally yet, as I'm not that familiar with the 
> PTXDist internals and didn't wanted to spend multiple hours creating a 
> patch and then get the feedback, that there might be a better way to do it.
> 

The different CROSS_ENVs are defined in rules/pre/Rules.make
Maybe test it there?

>> Also, do you have any anecdotal or factual history from other build 
>> environments?
>> Some of them rely extensively on git hashes, ie, repos.
>> They've likely stumbled upon the same question.
> 
> Nope, PTXDist is my first embedded build environment.
> 

\o/ :)

> 
> Best regards
> 
> Sven
> 

Re: [ptxdist] Globally define GIT_CEILING_DIRECTORIES?

[Michael Riesch]

Hi Sven,

On 2/13/23 14:58, Sven Püschel wrote:
> I would suggest globally setting the GIT_CEILING_DIRECTORIES environment
> variable when building packages.
> 
> Besides lshw, sockperf is now the second time I've encountered the
> buildsystem to check the current git repository. But as Git also checks
> the parent directories when searching for Git repositories, these checks
> may result in getting an unwanted Git repository (usually the BSP),
> resulting in package versions with a Git commit/version that has nothing
> to do with the actual package.
> 
> With GIT_CEILING_DIRECTORIES set to the parent directory of the
> extracted sources, Git will stop searching for parent Git repositories,
> when it encounters this directory. See my sockperf v2 patch for an
> example of using this environment variable.

We can add the libcamera project to that list. There is a script inside
libcamera that checks whether the sources are in a git tree. If yes, the
version obtained by git describe is used. As a fallback, the meson
project version is used. As a consequence, the version of our BSP is
leaked into libcamera.

I haven't tested this GIT_CEILING_DIRECTORIES approach but it seems to
me that it would do the trick.

> Given this is a global change, does anyone see a potential problem?

Can't be the judge of that, though.

Best regards,
Michael

Re: [ptxdist] Globally define GIT_CEILING_DIRECTORIES?

[Michael Olbrich]

Hi,

On Mon, Feb 13, 2023 at 02:58:49PM +0100, Sven Püschel wrote:
> I would suggest globally setting the GIT_CEILING_DIRECTORIES environment
> variable when building packages.
> 
> Besides lshw, sockperf is now the second time I've encountered the
> buildsystem to check the current git repository. But as Git also checks the
> parent directories when searching for Git repositories, these checks may
> result in getting an unwanted Git repository (usually the BSP), resulting in
> package versions with a Git commit/version that has nothing to do with the
> actual package.
> 
> With GIT_CEILING_DIRECTORIES set to the parent directory of the extracted
> sources, Git will stop searching for parent Git repositories, when it
> encounters this directory. See my sockperf v2 patch for an example of using
> this environment variable.
> 
> Given this is a global change, does anyone see a potential problem?

I've been thinking about similar issues from time to time. Mostly when I'm
digging into package sources. It happens occasionally that I do a 'git
grep', which succeeds without finding something... So I'd prefer a solution
that works here as well.

One idea is to create an empty platform-XXX/.git file. Git will stop here
with an error: "fatal: invalid gitfile format:  .../platform-XXX/.git".
It's not the best error, but it should solve both our issues.

Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] Globally define GIT_CEILING_DIRECTORIES?

[Michael Riesch]

Hi mol,

On 2/14/23 15:21, Michael Olbrich wrote:
> Hi,
> 
> On Mon, Feb 13, 2023 at 02:58:49PM +0100, Sven Püschel wrote:
>> I would suggest globally setting the GIT_CEILING_DIRECTORIES environment
>> variable when building packages.
>>
>> Besides lshw, sockperf is now the second time I've encountered the
>> buildsystem to check the current git repository. But as Git also checks the
>> parent directories when searching for Git repositories, these checks may
>> result in getting an unwanted Git repository (usually the BSP), resulting in
>> package versions with a Git commit/version that has nothing to do with the
>> actual package.
>>
>> With GIT_CEILING_DIRECTORIES set to the parent directory of the extracted
>> sources, Git will stop searching for parent Git repositories, when it
>> encounters this directory. See my sockperf v2 patch for an example of using
>> this environment variable.
>>
>> Given this is a global change, does anyone see a potential problem?
> 
> I've been thinking about similar issues from time to time. Mostly when I'm
> digging into package sources. It happens occasionally that I do a 'git
> grep', which succeeds without finding something... So I'd prefer a solution
> that works here as well.
> 
> One idea is to create an empty platform-XXX/.git file. Git will stop here
> with an error: "fatal: invalid gitfile format:  .../platform-XXX/.git".
> It's not the best error, but it should solve both our issues.

I gave this a quick test and it solves the problems with the libcamera
project.

Any idea when such a solution could land in ptxdist?

Thanks and regards,
mri

Re: [ptxdist] Globally define GIT_CEILING_DIRECTORIES?

[Michael Olbrich]

On Mon, Feb 27, 2023 at 10:22:43PM +0100, Michael Riesch wrote:
> On 2/14/23 15:21, Michael Olbrich wrote:
> > On Mon, Feb 13, 2023 at 02:58:49PM +0100, Sven Püschel wrote:
> >> I would suggest globally setting the GIT_CEILING_DIRECTORIES environment
> >> variable when building packages.
> >>
> >> Besides lshw, sockperf is now the second time I've encountered the
> >> buildsystem to check the current git repository. But as Git also checks the
> >> parent directories when searching for Git repositories, these checks may
> >> result in getting an unwanted Git repository (usually the BSP), resulting in
> >> package versions with a Git commit/version that has nothing to do with the
> >> actual package.
> >>
> >> With GIT_CEILING_DIRECTORIES set to the parent directory of the extracted
> >> sources, Git will stop searching for parent Git repositories, when it
> >> encounters this directory. See my sockperf v2 patch for an example of using
> >> this environment variable.
> >>
> >> Given this is a global change, does anyone see a potential problem?
> > 
> > I've been thinking about similar issues from time to time. Mostly when I'm
> > digging into package sources. It happens occasionally that I do a 'git
> > grep', which succeeds without finding something... So I'd prefer a solution
> > that works here as well.
> > 
> > One idea is to create an empty platform-XXX/.git file. Git will stop here
> > with an error: "fatal: invalid gitfile format:  .../platform-XXX/.git".
> > It's not the best error, but it should solve both our issues.
> 
> I gave this a quick test and it solves the problems with the libcamera
> project.
> 
> Any idea when such a solution could land in ptxdist?

It's in master now and will be in the next release, later today.

Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] Globally define GIT_CEILING_DIRECTORIES?

[Michael Riesch]

Hi mol,

On 3/3/23 08:25, Michael Olbrich wrote:
> On Mon, Feb 27, 2023 at 10:22:43PM +0100, Michael Riesch wrote:
>> On 2/14/23 15:21, Michael Olbrich wrote:
>>> On Mon, Feb 13, 2023 at 02:58:49PM +0100, Sven Püschel wrote:
>>>> I would suggest globally setting the GIT_CEILING_DIRECTORIES environment
>>>> variable when building packages.
>>>>
>>>> Besides lshw, sockperf is now the second time I've encountered the
>>>> buildsystem to check the current git repository. But as Git also checks the
>>>> parent directories when searching for Git repositories, these checks may
>>>> result in getting an unwanted Git repository (usually the BSP), resulting in
>>>> package versions with a Git commit/version that has nothing to do with the
>>>> actual package.
>>>>
>>>> With GIT_CEILING_DIRECTORIES set to the parent directory of the extracted
>>>> sources, Git will stop searching for parent Git repositories, when it
>>>> encounters this directory. See my sockperf v2 patch for an example of using
>>>> this environment variable.
>>>>
>>>> Given this is a global change, does anyone see a potential problem?
>>>
>>> I've been thinking about similar issues from time to time. Mostly when I'm
>>> digging into package sources. It happens occasionally that I do a 'git
>>> grep', which succeeds without finding something... So I'd prefer a solution
>>> that works here as well.
>>>
>>> One idea is to create an empty platform-XXX/.git file. Git will stop here
>>> with an error: "fatal: invalid gitfile format:  .../platform-XXX/.git".
>>> It's not the best error, but it should solve both our issues.
>>
>> I gave this a quick test and it solves the problems with the libcamera
>> project.
>>
>> Any idea when such a solution could land in ptxdist?
> 
> It's in master now and will be in the next release, later today.

Cool, thanks! I'll give it a try!

Best regards,
Michael

> 
> Michael
>