* [ptxdist] git ptxdist best practices @ 2014-01-08 2:32 Jon Ringle 2014-01-08 5:56 ` Alexander Dahl 2014-01-08 7:59 ` Jürgen Beisert 0 siblings, 2 replies; 20+ messages in thread From: Jon Ringle @ 2014-01-08 2:32 UTC (permalink / raw) To: ptxdist [-- Attachment #1.1: Type: text/plain, Size: 1087 bytes --] I'm looking for some best practices ideas as it relates to using ptxdist to build a complete system that pulls from various git repos. I have a clone Linux git repo that has local commits specific to our embedded hardware. I build an initramfs filesystem with ptxdist that gets embedded within the Linux kernel image. I build a rootfs with ptxdist for the real rootfs I also have a couple of libs and apps that have their own git repos as well that I'm getting ptxdist to build as well. I'm now at the point where I now need to make this available to other developers and also get setup for a build bot to build (bamboo). My initial thought is to create a git repo that holds the ptxconfig files and local rules/, and then configure the rules to somehow get the git repos by tag for building (as it seems that there is some sort of support for this in scripts/lib/ptxd_make_get.sh)... but I don't quite understand how to integrate this in my rules/*.make file. Perhaps there is a better way of doing this. I'd be interested to hear what others are doing in this regard. Thanks, Jon [-- Attachment #1.2: Type: text/html, Size: 1247 bytes --] [-- Attachment #2: Type: text/plain, Size: 48 bytes --] -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-08 2:32 [ptxdist] git ptxdist best practices Jon Ringle @ 2014-01-08 5:56 ` Alexander Dahl 2014-01-08 6:12 ` Jon Ringle 2014-01-08 7:59 ` Jürgen Beisert 1 sibling, 1 reply; 20+ messages in thread From: Alexander Dahl @ 2014-01-08 5:56 UTC (permalink / raw) To: ptxdist [-- Attachment #1.1: Type: text/plain, Size: 1250 bytes --] Hei hei, On 08.01.2014 03:32, Jon Ringle wrote: > My initial thought is to create a git repo that holds the ptxconfig files > and local rules/, and then configure the rules to somehow get the git repos > by tag for building (as it seems that there is some sort of support for > this in scripts/lib/ptxd_make_get.sh)... but I don't quite understand how > to integrate this in my rules/*.make file. This is exactly what we do, alas we do not use any ptxdist magic but two self written shell scripts residing in another git repository "scripts" included as submodule. Our rules/foo.make use one script for the get and one for the extract stage. Those scripts make are called with two extra variables from rules/foo.in as parameter. For each of our own packages those describe clone path and which git entity (branch, tag, changeset) to checkout after cloning. All these packages are cloned to local_src/foo and build with cmake in local_src/foo-git. I assume all this specific to our needs, but that's the way we do it. If you're interested in the scripts, tell me. However I'm also curious about other ways. Is there a nice ptxdist way to handle packages without tarball coming directly from some git repo? Greets Alex [-- Attachment #1.2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 259 bytes --] [-- Attachment #2: Type: text/plain, Size: 48 bytes --] -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-08 5:56 ` Alexander Dahl @ 2014-01-08 6:12 ` Jon Ringle 2014-01-08 9:21 ` Alexander Dahl 0 siblings, 1 reply; 20+ messages in thread From: Jon Ringle @ 2014-01-08 6:12 UTC (permalink / raw) To: ptxdist [-- Attachment #1.1: Type: text/plain, Size: 1506 bytes --] On Wed, Jan 8, 2014 at 12:56 AM, Alexander Dahl <post@lespocky.de> wrote: > Hei hei, > > On 08.01.2014 03:32, Jon Ringle wrote: > > My initial thought is to create a git repo that holds the ptxconfig files > > and local rules/, and then configure the rules to somehow get the git > repos > > by tag for building (as it seems that there is some sort of support for > > this in scripts/lib/ptxd_make_get.sh)... but I don't quite understand how > > to integrate this in my rules/*.make file. > > This is exactly what we do, alas we do not use any ptxdist magic but two > self written shell scripts residing in another git repository "scripts" > included as submodule. Our rules/foo.make use one script for the get and > one for the extract stage. Those scripts make are called with two extra > variables from rules/foo.in as parameter. For each of our own packages > those describe clone path and which git entity (branch, tag, changeset) > to checkout after cloning. All these packages are cloned to > local_src/foo and build with cmake in local_src/foo-git. I assume all > this specific to our needs, but that's the way we do it. If you're > interested in the scripts, tell me. > > Yes, I would be interested to see your scripts and how you use them in your *.make and *.in :) > However I'm also curious about other ways. Is there a nice ptxdist way > to handle packages without tarball coming directly from some git repo? > > Greets > Alex > > > > -- > ptxdist mailing list > ptxdist@pengutronix.de > > [-- Attachment #1.2: Type: text/html, Size: 2180 bytes --] [-- Attachment #2: Type: text/plain, Size: 48 bytes --] -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-08 6:12 ` Jon Ringle @ 2014-01-08 9:21 ` Alexander Dahl 2014-01-08 9:48 ` Alexander Dahl ` (2 more replies) 0 siblings, 3 replies; 20+ messages in thread From: Alexander Dahl @ 2014-01-08 9:21 UTC (permalink / raw) To: ptxdist Hei hei, Am 2014-01-08 07:12, schrieb Jon Ringle: > On Wed, Jan 8, 2014 at 12:56 AM, Alexander Dahl <post@lespocky.de> wrote: >> This is exactly what we do, alas we do not use any ptxdist magic but two >> self written shell scripts residing in another git repository "scripts" >> included as submodule. Our rules/foo.make use one script for the get and >> one for the extract stage. Those scripts make are called with two extra >> variables from rules/foo.in as parameter. For each of our own packages >> those describe clone path and which git entity (branch, tag, changeset) >> to checkout after cloning. All these packages are cloned to >> local_src/foo and build with cmake in local_src/foo-git. I assume all >> this specific to our needs, but that's the way we do it. If you're >> interested in the scripts, tell me. >> >> > Yes, I would be interested to see your scripts and how you use them in your > *.make and *.in :) Okay I edited them a little and attached it. Note: the scripts itself are put in a subfolder of the BSP here named "scripts". I decided to use the get stage for cloning a git repo or checking if it's correctly cloned and if fetching all branches and changesets without updating the working copy at this stage. The other script for the extract stage does the actual checkout then. It works. ;-) Greets Alex -- »With the first link, the chain is forged. The first speech censured, the first thought forbidden, the first freedom denied, chains us all irrevocably.« (Jean-Luc Picard, quoting Judge Aaron Satie) *** GnuPG-FP: 02C8 A590 7FE5 CA5F 3601 D1D5 8FBA 7744 CC87 10D0 *** -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-08 9:21 ` Alexander Dahl @ 2014-01-08 9:48 ` Alexander Dahl 2014-01-08 9:52 ` Jürgen Beisert 2014-01-08 17:56 ` Jon Ringle 2 siblings, 0 replies; 20+ messages in thread From: Alexander Dahl @ 2014-01-08 9:48 UTC (permalink / raw) To: ptxdist [-- Attachment #1: Type: text/plain, Size: 487 bytes --] Am 2014-01-08 10:21, schrieb Alexander Dahl: > Okay I edited them a little and attached it. Note: the scripts itself > are put in a subfolder of the BSP here named "scripts". Guess what I forgot to attach?! Alex -- »With the first link, the chain is forged. The first speech censured, the first thought forbidden, the first freedom denied, chains us all irrevocably.« (Jean-Luc Picard, quoting Judge Aaron Satie) *** GnuPG-FP: 02C8 A590 7FE5 CA5F 3601 D1D5 8FBA 7744 CC87 10D0 *** [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: get-git.sh --] [-- Type: text/x-shellscript; name=get-git.sh, Size: 1747 bytes --] #!/bin/sh # ---------------------------------------------------------------------- # Clone a git repository in get stage or check if it's the right one # in case already cloned. # # Authors: Alexander Dahl <post@lespocky.de> # # Copyright 2011,2012,2013 Alexander Dahl # ---------------------------------------------------------------------- #set -x #set -e _RED="\033[31m" _YELLOW="\033[33m\033[40m\033[1m" _NOCOLOR="\033[0m\033[49m" GITURL="$1" DESTDIR="$2" print_usage() { echo "Usage: $0 giturl destdir" } if [ -z "$1" -o -z "$2" ] then print_usage exit 1 fi CHECKOUTDIR="$(basename ${DESTDIR})" SRCDIR="$(dirname ${DESTDIR})" if [ ! -d "${SRCDIR}" ] then echo "folder ${_YELLOW}local_src${_NOCOLOR} does not exist, please create it!" exit 4 fi cd "${SRCDIR}" if [ ! -d "${CHECKOUTDIR}" ] then if [ -e "${CHECKOUTDIR}" ] then echo "${CHECKOUTDIR} exists but is no directory!" exit 2 else echo "${CHECKOUTDIR} does not exist." fi if git clone "${GITURL}" "${CHECKOUTDIR}" then echo 'git clone successful ...' else exit $? fi cd "${CHECKOUTDIR}" if git fetch --all then echo 'git fetch successful ...' else exit $? fi else if [ -L "${CHECKOUTDIR}" ] then echo "${_RED}WARNING: ${_YELLOW}${CHECKOUTDIR} is a symbolic link!${_NOCOLOR}" exit 0 fi cd "${CHECKOUTDIR}" REMOTEURL="$(git remote -v | awk '/^origin.+(fetch)/ {print $2;}')" if [ "${REMOTEURL}" = "${GITURL}" ] then echo "This already cloned repo is the right one." exit 0 else echo "Cloned repo remote is '${REMOTEURL}' but '${GITURL}' was expected!" exit 3 fi fi [-- Attachment #3: foo.make --] [-- Type: text/plain, Size: 2164 bytes --] # -*-makefile-*- # # Copyright (C) 2012 by Alexander Dahl <post@lespocky.de> # # See CREDITS for details about who has contributed to this project. # # For further information about the PTXdist project and license conditions # see the README file. # # # We provide this package # PACKAGES-$(PTXCONF_FOO) += foo # # Paths and names # FOO_VERSION := git #FOO_MD5 := FOO := foo-$(FOO_VERSION) #FOO_SUFFIX := #FOO_URL := /$(FOO).$(FOO_SUFFIX) #FOO_SOURCE := $(SRCDIR)/$(FOO).$(FOO_SUFFIX) #FOO_DIR := $(BUILDDIR)/$(FOO) FOO_LICENSE := proprietary FOO_SOURCE_LOCAL := $(PTXDIST_WORKSPACE)/local_src/$(FOO) FOO_GIT_URL := $(PTXCONF_FOO_CLONE_URL) FOO_CHECKOUT_ENTITY := $(PTXCONF_FOO_CHECKOUT_ENTITY) FOO_DIR := $(FOO_SOURCE_LOCAL) # ---------------------------------------------------------------------------- # Get # ---------------------------------------------------------------------------- $(STATEDIR)/foo.get: @$(call targetinfo) @$(PTXDIST_WORKSPACE)/scripts/get-git.sh "$(FOO_GIT_URL)" "$(FOO_SOURCE_LOCAL)" @$(call touch) # ---------------------------------------------------------------------------- # Extract # ---------------------------------------------------------------------------- $(STATEDIR)/foo.extract: @$(call targetinfo) @cd $(FOO_SOURCE_LOCAL) && \ $(PTXDIST_WORKSPACE)/scripts/extract-git.sh "$(FOO_CHECKOUT_ENTITY)" @$(call touch) # ---------------------------------------------------------------------------- # Prepare # ---------------------------------------------------------------------------- FOO_CONF_TOOL := cmake # ---------------------------------------------------------------------------- # Target-Install # ---------------------------------------------------------------------------- $(STATEDIR)/foo.targetinstall: @$(call targetinfo) @$(call install_init, foo) @$(call install_fixup, foo,PRIORITY,optional) @$(call install_fixup, foo,SECTION,base) @$(call install_fixup, foo,AUTHOR,"Alexander Dahl <post@lespocky.de>") @$(call install_fixup, foo,DESCRIPTION,missing) @$(call install_copy, foo, 0, 0, 0755, -, /usr/bin/foo) @$(call install_finish, foo) @$(call touch) # vim: ft=make noet [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: extract-git.sh --] [-- Type: text/x-shellscript; name=extract-git.sh, Size: 1104 bytes --] #!/bin/sh # ---------------------------------------------------------------------- # In a git "working copy" checkout the given entity (branch, tag, # changeset). In case of a branch pull the latest changes, in every # other case leave it in detached HEAD state. This script performs no # directory changes, so start it in the target directory! # # Authors: Alexander Dahl <post@lespocky.de> # # Copyright 2011,2013 Alexander Dahl # ---------------------------------------------------------------------- ENTITY="$1" BRANCH_EXISTS="no" print_usage() { echo "Usage: $0 checkoutentity" } if [ -z "$1" ] then print_usage exit 1 fi if git fetch then echo 'Git fetch successful!' else exit $? fi if git checkout "$1" then echo 'Git checkout successful!' else exit $? fi for branch in $(git branch | cut -c3-) do if [ "${ENTITY}" = "${branch}" ] then BRANCH_EXISTS="yes" fi done if [ "${BRANCH_EXISTS}" = "yes" ] then git pull fi if git submodule update --init --recursive then echo 'Git submodule update successful!' else exit $? fi exit 0 [-- Attachment #5: foo.in --] [-- Type: text/plain, Size: 578 bytes --] ## SECTION=project_specific menuconfig FOO tristate prompt "foo " select HOST_CMAKE default y help Foo like in bar. config FOO_CLONE_URL string depends on FOO prompt "Git url to clone from" default "git://git.example.com/git/foo.git" help Git URL to clone from. Defaults to local server, but can be overwritten with any local or remote URL. config FOO_CHECKOUT_ENTITY string depends on FOO prompt "Git entity to checkout." default "stable" help Check out a specific branch, tag or changeset. # vim: ft=kconfig noet tw=72 [-- Attachment #6: Type: text/plain, Size: 48 bytes --] -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-08 9:21 ` Alexander Dahl 2014-01-08 9:48 ` Alexander Dahl @ 2014-01-08 9:52 ` Jürgen Beisert 2014-01-08 17:56 ` Jon Ringle 2 siblings, 0 replies; 20+ messages in thread From: Jürgen Beisert @ 2014-01-08 9:52 UTC (permalink / raw) To: ptxdist On Wednesday 08 January 2014 10:21:53 Alexander Dahl wrote: > Am 2014-01-08 07:12, schrieb Jon Ringle: > > On Wed, Jan 8, 2014 at 12:56 AM, Alexander Dahl <post@lespocky.de> wrote: > >> This is exactly what we do, alas we do not use any ptxdist magic but two > >> self written shell scripts residing in another git repository "scripts" > >> included as submodule. Our rules/foo.make use one script for the get and > >> one for the extract stage. Those scripts make are called with two extra > >> variables from rules/foo.in as parameter. For each of our own packages > >> those describe clone path and which git entity (branch, tag, changeset) > >> to checkout after cloning. All these packages are cloned to > >> local_src/foo and build with cmake in local_src/foo-git. I assume all > >> this specific to our needs, but that's the way we do it. If you're > >> interested in the scripts, tell me. > > > > Yes, I would be interested to see your scripts and how you use them in > > your *.make and *.in :) > > Okay I edited them a little and attached it. <nothing> More coffee? :)) jbe -- Pengutronix e.K. | Juergen Beisert | Linux Solutions for Science and Industry | http://www.pengutronix.de/ | -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-08 9:21 ` Alexander Dahl 2014-01-08 9:48 ` Alexander Dahl 2014-01-08 9:52 ` Jürgen Beisert @ 2014-01-08 17:56 ` Jon Ringle 2014-01-08 20:43 ` Alexander Dahl 2 siblings, 1 reply; 20+ messages in thread From: Jon Ringle @ 2014-01-08 17:56 UTC (permalink / raw) To: ptxdist [-- Attachment #1.1: Type: text/plain, Size: 1625 bytes --] On Wed, Jan 8, 2014 at 4:21 AM, Alexander Dahl <post@lespocky.de> wrote: > Hei hei, > > Am 2014-01-08 07:12, schrieb Jon Ringle: > > On Wed, Jan 8, 2014 at 12:56 AM, Alexander Dahl <post@lespocky.de> > wrote: > >> This is exactly what we do, alas we do not use any ptxdist magic but two > >> self written shell scripts residing in another git repository "scripts" > >> included as submodule. Our rules/foo.make use one script for the get and > >> one for the extract stage. Those scripts make are called with two extra > >> variables from rules/foo.in as parameter. For each of our own packages > >> those describe clone path and which git entity (branch, tag, changeset) > >> to checkout after cloning. All these packages are cloned to > >> local_src/foo and build with cmake in local_src/foo-git. I assume all > >> this specific to our needs, but that's the way we do it. If you're > >> interested in the scripts, tell me. > >> > >> > > Yes, I would be interested to see your scripts and how you use them in > your > > *.make and *.in :) > > Okay I edited them a little and attached it. Note: the scripts itself > are put in a subfolder of the BSP here named "scripts". > > I decided to use the get stage for cloning a git repo or checking if > it's correctly cloned and if fetching all branches and changesets > without updating the working copy at this stage. The other script for > the extract stage does the actual checkout then. It works. ;-) > > Thanks! How do you deal with creating a rule/kernel.make file that does this if I build Linux using my own tag? It seems that kernel may need some special handling Jon [-- Attachment #1.2: Type: text/html, Size: 2286 bytes --] [-- Attachment #2: Type: text/plain, Size: 48 bytes --] -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-08 17:56 ` Jon Ringle @ 2014-01-08 20:43 ` Alexander Dahl 0 siblings, 0 replies; 20+ messages in thread From: Alexander Dahl @ 2014-01-08 20:43 UTC (permalink / raw) To: ptxdist [-- Attachment #1.1: Type: text/plain, Size: 727 bytes --] Hei Jon, On Wed, Jan 08, 2014 at 12:56:37PM -0500, Jon Ringle wrote: > How do you deal with creating a rule/kernel.make file that does this if I > build Linux using my own tag? It seems that kernel may need some special > handling We use a vanilla kernel and just compile some additional modules for our own hardware. I can check tomorrow, how this is done, but as far as I remember just a simple additional make rule. Greets Alex -- »With the first link, the chain is forged. The first speech censured, the first thought forbidden, the first freedom denied, chains us all irrevocably.« (Jean-Luc Picard, quoting Judge Aaron Satie) *** GnuPG-FP: 02C8 A590 7FE5 CA5F 3601 D1D5 8FBA 7744 CC87 10D0 *** [-- Attachment #1.2: Type: application/pgp-signature, Size: 198 bytes --] [-- Attachment #2: Type: text/plain, Size: 48 bytes --] -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-08 2:32 [ptxdist] git ptxdist best practices Jon Ringle 2014-01-08 5:56 ` Alexander Dahl @ 2014-01-08 7:59 ` Jürgen Beisert 2014-01-08 14:14 ` Jon Ringle 2014-01-09 9:03 ` Olbrich, Michael 1 sibling, 2 replies; 20+ messages in thread From: Jürgen Beisert @ 2014-01-08 7:59 UTC (permalink / raw) To: ptxdist; +Cc: Olbrich, Michael Hi Jon, On Wednesday 08 January 2014 03:32:41 Jon Ringle wrote: > I'm looking for some best practices ideas as it relates to using ptxdist to > build a complete system that pulls from various git repos. > > I have a clone Linux git repo that has local commits specific to our > embedded hardware. > I build an initramfs filesystem with ptxdist that gets embedded within the > Linux kernel image. > I build a rootfs with ptxdist for the real rootfs > I also have a couple of libs and apps that have their own git repos as well > that I'm getting ptxdist to build as well. > > I'm now at the point where I now need to make this available to other > developers and also get setup for a build bot to build (bamboo). > > My initial thought is to create a git repo that holds the ptxconfig files > and local rules/, and then configure the rules to somehow get the git repos > by tag for building (as it seems that there is some sort of support for > this in scripts/lib/ptxd_make_get.sh)... but I don't quite understand how > to integrate this in my rules/*.make file. > > Perhaps there is a better way of doing this. I'd be interested to hear what > others are doing in this regard. Do you want to use this BSP only by your own, or is it intended to provide it to other developers who use it as a base for their own work? In the latter case I would "release" the various components and create archives of it. And these archives can simply be part of the (finally) released BSP you provide to other developers (in contrast to the downloadable archives PTXdist uses for the remaining packages like Busybox, Qt and so on). If you still want to use git repositories for the external components we should wait for Michael, because he can explain how to use the PTXdist feature which deals with such repositories. Regards, jbe -- Pengutronix e.K. | Juergen Beisert | Linux Solutions for Science and Industry | http://www.pengutronix.de/ | -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-08 7:59 ` Jürgen Beisert @ 2014-01-08 14:14 ` Jon Ringle 2014-01-09 8:10 ` Jürgen Beisert 2014-01-09 9:03 ` Olbrich, Michael 1 sibling, 1 reply; 20+ messages in thread From: Jon Ringle @ 2014-01-08 14:14 UTC (permalink / raw) To: Jürgen Beisert; +Cc: Olbrich, Michael, ptxdist [-- Attachment #1.1: Type: text/plain, Size: 1065 bytes --] On Wed, Jan 8, 2014 at 2:59 AM, Jürgen Beisert <jbe@pengutronix.de> wrote: > Hi Jon, > Do you want to use this BSP only by your own, or is it intended to provide > it > to other developers who use it as a base for their own work? > > In the latter case I would "release" the various components and create > archives > of it. And these archives can simply be part of the (finally) released BSP > you > provide to other developers (in contrast to the downloadable archives > PTXdist > uses for the remaining packages like Busybox, Qt and so on). > > Yes, this will be for other developers to base their work off of. So, what exactly would be included in the "release" BSP? platform/sysroot-*/ and platform/root*/ ?? > If you still want to use git repositories for the external components we > should > wait for Michael, because he can explain how to use the PTXdist feature > which > deals with such repositories. > > Yes, I'd like to understand better ptxdist features to support externat components maintained by our own git repos :) [-- Attachment #1.2: Type: text/html, Size: 1573 bytes --] [-- Attachment #2: Type: text/plain, Size: 48 bytes --] -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-08 14:14 ` Jon Ringle @ 2014-01-09 8:10 ` Jürgen Beisert 0 siblings, 0 replies; 20+ messages in thread From: Jürgen Beisert @ 2014-01-09 8:10 UTC (permalink / raw) To: ptxdist; +Cc: Olbrich, Michael Hi Jon, On Wednesday 08 January 2014 15:14:26 Jon Ringle wrote: > On Wed, Jan 8, 2014 at 2:59 AM, Jürgen Beisert <jbe@pengutronix.de> wrote: > > Do you want to use this BSP only by your own, or is it intended to > > provide it to other developers who use it as a base for their own work? > > In the latter case I would "release" the various components and create > > archives of it. And these archives can simply be part of the (finally) > > released BSP you provide to other developers (in contrast to the > > downloadable archives PTXdist uses for the remaining packages like Busybox, > > Qt and so on). > > > Yes, this will be for other developers to base their work off of. > > So, what exactly would be included in the "release" BSP? > platform/sysroot-*/ and platform/root*/ ?? No. These directories exist at buildtime only and will be recreated and filled up at each PTXdist run. Their content depends on the selected toolchain and package selection. You should keep this behaviour for the user of the BSP. Maybe I read your first question wrong: You have some development made and the results are part of a few external git repositories, haven't you? Why not tar them (not the whole repository, just a released variant: git archive --format=tar ...) and include the archive in the BSP? The corresponding rule file uses a local URL pointing to this archive and you are done. > > If you still want to use git repositories for the external components we > > should wait for Michael, because he can explain how to use the PTXdist > > feature which deals with such repositories. > > > Yes, I'd like to understand better ptxdist features to support externat > components maintained by our own git repos :) But this can work only, if everyone using your BSP has access to these repositories. Regards, Juergen -- Pengutronix e.K. | Juergen Beisert | Linux Solutions for Science and Industry | http://www.pengutronix.de/ | -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-08 7:59 ` Jürgen Beisert 2014-01-08 14:14 ` Jon Ringle @ 2014-01-09 9:03 ` Olbrich, Michael 2014-01-09 15:23 ` Jon Ringle 1 sibling, 1 reply; 20+ messages in thread From: Olbrich, Michael @ 2014-01-09 9:03 UTC (permalink / raw) To: ptxdist On Wed, Jan 08, 2014 at 08:59:43AM +0100, Jürgen Beisert wrote: > On Wednesday 08 January 2014 03:32:41 Jon Ringle wrote: > > I'm looking for some best practices ideas as it relates to using ptxdist to > > build a complete system that pulls from various git repos. > > > > I have a clone Linux git repo that has local commits specific to our > > embedded hardware. > > I build an initramfs filesystem with ptxdist that gets embedded within the > > Linux kernel image. > > I build a rootfs with ptxdist for the real rootfs > > I also have a couple of libs and apps that have their own git repos as well > > that I'm getting ptxdist to build as well. > > > > I'm now at the point where I now need to make this available to other > > developers and also get setup for a build bot to build (bamboo). > > > > My initial thought is to create a git repo that holds the ptxconfig files > > and local rules/, and then configure the rules to somehow get the git repos > > by tag for building (as it seems that there is some sort of support for > > this in scripts/lib/ptxd_make_get.sh)... but I don't quite understand how > > to integrate this in my rules/*.make file. > > > > Perhaps there is a better way of doing this. I'd be interested to hear what > > others are doing in this regard. > > Do you want to use this BSP only by your own, or is it intended to provide it > to other developers who use it as a base for their own work? > > In the latter case I would "release" the various components and create archives > of it. And these archives can simply be part of the (finally) released BSP you > provide to other developers (in contrast to the downloadable archives PTXdist > uses for the remaining packages like Busybox, Qt and so on). > > If you still want to use git repositories for the external components we should > wait for Michael, because he can explain how to use the PTXdist feature which > deals with such repositories. It really depends on how you want to work with this. If the external repositories are integrated often, then add them as git submodules in local_src and use "file://$(PTXDIST_WORKSPACE)/local_src/foo/" as URL. If not then create a tag when integrating and use git://myhost.de/foo.git;tag=mytag1" as URL and foo-mytag1.tar.bz2 as SOURCE and update the tag as needed. This will clone the repository and create a tarball for the tag. Note: you cannot follow a branch here! It will just use the tarball and ignore any changes to the branch. This is only for projects that only provide git tags for releases but no tarballs. Michael -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-09 9:03 ` Olbrich, Michael @ 2014-01-09 15:23 ` Jon Ringle 2014-01-09 16:06 ` Michael Olbrich 0 siblings, 1 reply; 20+ messages in thread From: Jon Ringle @ 2014-01-09 15:23 UTC (permalink / raw) To: ptxdist [-- Attachment #1.1: Type: text/plain, Size: 1456 bytes --] On Thu, Jan 9, 2014 at 4:03 AM, Olbrich, Michael <m.olbrich@pengutronix.de>wrote: > It really depends on how you want to work with this. If the external > repositories are integrated often, then add them as git submodules in > local_src and use "file://$(PTXDIST_WORKSPACE)/local_src/foo/" as URL. If > I've not used git submodules before. Can I bind a git submodule to a specific tag? > not then create a tag when integrating and use > git://myhost.de/foo.git;tag=mytag1" as URL and foo-mytag1.tar.bz2 as > SOURCE > and update the tag as needed. This will clone the repository and create a > tarball for the tag. Note: you cannot follow a branch here! It will just > use the tarball and ignore any changes to the branch. This is only for > projects that only provide git tags for releases but no tarballs. > > One issue that I ran across with this is that I have: PTXCONF_SETUP_PTXMIRROR_ONLY=y PTXCONF_SETUP_PTXMIRROR="http://opensource/pool/" Where opensource is an internal server that keeps all tar balls used to build. I'm paranoid that someday a upstream tar ball won't be available and would cause a build break if someone tried to build and they didn't have that tar ball in ${PTXCONF_SETUP_SRCDIR} But for my purposes, if I wanted to use the git URL in a rule file I'd want it to use the URL as is. But it rewrites it trying to use ${PTXCONF_SETUP_PTXMIRROR} instead in scripts/lib/ptxd_make_get.sh:295 Is there away to work around this? [-- Attachment #1.2: Type: text/html, Size: 2301 bytes --] [-- Attachment #2: Type: text/plain, Size: 48 bytes --] -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-09 15:23 ` Jon Ringle @ 2014-01-09 16:06 ` Michael Olbrich 2014-01-09 16:37 ` Jon Ringle 0 siblings, 1 reply; 20+ messages in thread From: Michael Olbrich @ 2014-01-09 16:06 UTC (permalink / raw) To: ptxdist On Thu, Jan 09, 2014 at 10:23:58AM -0500, Jon Ringle wrote: > On Thu, Jan 9, 2014 at 4:03 AM, Olbrich, Michael > <m.olbrich@pengutronix.de>wrote: > > > It really depends on how you want to work with this. If the external > > repositories are integrated often, then add them as git submodules in > > local_src and use "file://$(PTXDIST_WORKSPACE)/local_src/foo/" as URL. If > > > > I've not used git submodules before. Can I bind a git submodule to a > specific tag? git submodules are always bound to a specific commit. > > not then create a tag when integrating and use > > git://myhost.de/foo.git;tag=mytag1" as URL and foo-mytag1.tar.bz2 as > > SOURCE > > and update the tag as needed. This will clone the repository and create a > > tarball for the tag. Note: you cannot follow a branch here! It will just > > use the tarball and ignore any changes to the branch. This is only for > > projects that only provide git tags for releases but no tarballs. > > > > > One issue that I ran across with this is that I have: > PTXCONF_SETUP_PTXMIRROR_ONLY=y > PTXCONF_SETUP_PTXMIRROR="http://opensource/pool/" > > Where opensource is an internal server that keeps all tar balls used to > build. I'm paranoid that someday a upstream tar ball won't be available and > would cause a build break if someone tried to build and they didn't have > that tar ball in ${PTXCONF_SETUP_SRCDIR} > > But for my purposes, if I wanted to use the git URL in a rule file I'd want > it to use the URL as is. But it rewrites it trying to use > ${PTXCONF_SETUP_PTXMIRROR} instead in scripts/lib/ptxd_make_get.sh:295 > > Is there away to work around this? put your git repositories in the mirror: FOO_URL := http://opensource/pool/foo.git;tag=bar The mechanism whitelists any URL starting with ${PTXCONF_SETUP_PTXMIRROR} Michael -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-09 16:06 ` Michael Olbrich @ 2014-01-09 16:37 ` Jon Ringle 2014-01-10 15:37 ` Michael Olbrich 0 siblings, 1 reply; 20+ messages in thread From: Jon Ringle @ 2014-01-09 16:37 UTC (permalink / raw) To: ptxdist [-- Attachment #1.1: Type: text/plain, Size: 1753 bytes --] On Thu, Jan 9, 2014 at 11:06 AM, Michael Olbrich <m.olbrich@pengutronix.de>wrote: > On Thu, Jan 09, 2014 at 10:23:58AM -0500, Jon Ringle wrote:> One issue > that I ran across with this is that I have: > > PTXCONF_SETUP_PTXMIRROR_ONLY=y > > PTXCONF_SETUP_PTXMIRROR="http://opensource/pool/" > > > > Where opensource is an internal server that keeps all tar balls used to > > build. I'm paranoid that someday a upstream tar ball won't be available > and > > would cause a build break if someone tried to build and they didn't have > > that tar ball in ${PTXCONF_SETUP_SRCDIR} > > > > But for my purposes, if I wanted to use the git URL in a rule file I'd > want > > it to use the URL as is. But it rewrites it trying to use > > ${PTXCONF_SETUP_PTXMIRROR} instead in scripts/lib/ptxd_make_get.sh:295 > > > > Is there away to work around this? > > put your git repositories in the mirror: > FOO_URL := http://opensource/pool/foo.git;tag=bar > > Unfortunately, I can't do that. The opensource server is under IT control and they've locked it down so that I can only write to it using scp or sftp and have no ssh shell access to it. The mechanism whitelists any URL starting with ${PTXCONF_SETUP_PTXMIRROR} > Would the following approach work: 1. In ptxdist/rules/post/ptxd_make_world_common.make add to world/env/impl: pkg_url_safe="$(call ptx/escape,$($(1)_URL_SAFE))" 2. In ptxdist/scripts/lib/ptxd_make_get.sh function ptxd_make_get change all occurances of: if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" ]; then with something like: if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" -o -n "${pkg_url_safe}" ]; then Then in the rule file you can do: FOO_URL_SAFE := y So that you can whitelist a specific URL if PTXCONF_SETUP_PTXMIRROR_ONLY is set [-- Attachment #1.2: Type: text/html, Size: 2705 bytes --] [-- Attachment #2: Type: text/plain, Size: 48 bytes --] -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-09 16:37 ` Jon Ringle @ 2014-01-10 15:37 ` Michael Olbrich 2014-01-10 15:41 ` Jon Ringle 2014-01-11 1:47 ` Jon Ringle 0 siblings, 2 replies; 20+ messages in thread From: Michael Olbrich @ 2014-01-10 15:37 UTC (permalink / raw) To: ptxdist On Thu, Jan 09, 2014 at 11:37:15AM -0500, Jon Ringle wrote: > On Thu, Jan 9, 2014 at 11:06 AM, Michael Olbrich > <m.olbrich@pengutronix.de>wrote: > > > On Thu, Jan 09, 2014 at 10:23:58AM -0500, Jon Ringle wrote:> One issue > > that I ran across with this is that I have: > > > PTXCONF_SETUP_PTXMIRROR_ONLY=y > > > PTXCONF_SETUP_PTXMIRROR="http://opensource/pool/" > > > > > > Where opensource is an internal server that keeps all tar balls used to > > > build. I'm paranoid that someday a upstream tar ball won't be available > > and > > > would cause a build break if someone tried to build and they didn't have > > > that tar ball in ${PTXCONF_SETUP_SRCDIR} > > > > > > But for my purposes, if I wanted to use the git URL in a rule file I'd > > want > > > it to use the URL as is. But it rewrites it trying to use > > > ${PTXCONF_SETUP_PTXMIRROR} instead in scripts/lib/ptxd_make_get.sh:295 > > > > > > Is there away to work around this? > > > > put your git repositories in the mirror: > > FOO_URL := http://opensource/pool/foo.git;tag=bar > > > > Unfortunately, I can't do that. The opensource server is under IT control > and they've locked it down so that I can only write to it using scp or sftp > and have no ssh shell access to it. > > The mechanism whitelists any URL starting with ${PTXCONF_SETUP_PTXMIRROR} > > > > Would the following approach work: > > 1. In ptxdist/rules/post/ptxd_make_world_common.make add to world/env/impl: > pkg_url_safe="$(call ptx/escape,$($(1)_URL_SAFE))" > > 2. In ptxdist/scripts/lib/ptxd_make_get.sh function ptxd_make_get > change all occurances of: > if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" ]; then > with something like: > if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" -o -n "${pkg_url_safe}" ]; > then > > Then in the rule file you can do: > FOO_URL_SAFE := y > > So that you can whitelist a specific URL if PTXCONF_SETUP_PTXMIRROR_ONLY is > set Hmmm, this is not so nice. How about making PTXCONF_SETUP_PTXMIRROR a list? Then you can add http://opensource/pool/ and your git server to the 'whitelist'. Michael -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-10 15:37 ` Michael Olbrich @ 2014-01-10 15:41 ` Jon Ringle 2014-01-11 1:47 ` Jon Ringle 1 sibling, 0 replies; 20+ messages in thread From: Jon Ringle @ 2014-01-10 15:41 UTC (permalink / raw) To: ptxdist [-- Attachment #1.1: Type: text/plain, Size: 2346 bytes --] On Fri, Jan 10, 2014 at 10:37 AM, Michael Olbrich <m.olbrich@pengutronix.de>wrote: > On Thu, Jan 09, 2014 at 11:37:15AM -0500, Jon Ringle wrote: > > On Thu, Jan 9, 2014 at 11:06 AM, Michael Olbrich > > <m.olbrich@pengutronix.de>wrote: > > > > > On Thu, Jan 09, 2014 at 10:23:58AM -0500, Jon Ringle wrote:> One issue > > > that I ran across with this is that I have: > > > > PTXCONF_SETUP_PTXMIRROR_ONLY=y > > > > PTXCONF_SETUP_PTXMIRROR="http://opensource/pool/" > > > > > > > > Where opensource is an internal server that keeps all tar balls used > to > > > > build. I'm paranoid that someday a upstream tar ball won't be > available > > > and > > > > would cause a build break if someone tried to build and they didn't > have > > > > that tar ball in ${PTXCONF_SETUP_SRCDIR} > > > > > > > > But for my purposes, if I wanted to use the git URL in a rule file > I'd > > > want > > > > it to use the URL as is. But it rewrites it trying to use > > > > ${PTXCONF_SETUP_PTXMIRROR} instead in > scripts/lib/ptxd_make_get.sh:295 > > > > > > > > Is there away to work around this? > > > > > > put your git repositories in the mirror: > > > FOO_URL := http://opensource/pool/foo.git;tag=bar > > > > > > Unfortunately, I can't do that. The opensource server is under IT > control > > and they've locked it down so that I can only write to it using scp or > sftp > > and have no ssh shell access to it. > > > > The mechanism whitelists any URL starting with ${PTXCONF_SETUP_PTXMIRROR} > > > > > > > Would the following approach work: > > > > 1. In ptxdist/rules/post/ptxd_make_world_common.make add to > world/env/impl: > > pkg_url_safe="$(call ptx/escape,$($(1)_URL_SAFE))" > > > > 2. In ptxdist/scripts/lib/ptxd_make_get.sh function ptxd_make_get > > change all occurances of: > > if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" ]; then > > with something like: > > if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" -o -n "${pkg_url_safe}" ]; > > then > > > > Then in the rule file you can do: > > FOO_URL_SAFE := y > > > > So that you can whitelist a specific URL if PTXCONF_SETUP_PTXMIRROR_ONLY > is > > set > > Hmmm, this is not so nice. How about making PTXCONF_SETUP_PTXMIRROR a list? > Then you can add http://opensource/pool/ and your git server to the > 'whitelist'. > > Does PTXCONF_SETUP_PTXMIRROR currently support supplying a list of URLs?-- [-- Attachment #1.2: Type: text/html, Size: 3475 bytes --] [-- Attachment #2: Type: text/plain, Size: 48 bytes --] -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-10 15:37 ` Michael Olbrich 2014-01-10 15:41 ` Jon Ringle @ 2014-01-11 1:47 ` Jon Ringle 2014-01-24 9:21 ` Michael Olbrich 1 sibling, 1 reply; 20+ messages in thread From: Jon Ringle @ 2014-01-11 1:47 UTC (permalink / raw) To: ptxdist [-- Attachment #1.1: Type: text/plain, Size: 2145 bytes --] On Fri, Jan 10, 2014 at 10:37 AM, Michael Olbrich <m.olbrich@pengutronix.de>wrote: > On Thu, Jan 09, 2014 at 11:37:15AM -0500, Jon Ringle wrote: > > Would the following approach work: > > > > 1. In ptxdist/rules/post/ptxd_make_world_common.make add to > world/env/impl: > > pkg_url_safe="$(call ptx/escape,$($(1)_URL_SAFE))" > > > > 2. In ptxdist/scripts/lib/ptxd_make_get.sh function ptxd_make_get > > change all occurances of: > > if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" ]; then > > with something like: > > if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" -o -n "${pkg_url_safe}" ]; > > then > > > > Then in the rule file you can do: > > FOO_URL_SAFE := y > > > > So that you can whitelist a specific URL if PTXCONF_SETUP_PTXMIRROR_ONLY > is > > set > > Hmmm, this is not so nice. How about making PTXCONF_SETUP_PTXMIRROR a list? > Then you can add http://opensource/pool/ and your git server to the > 'whitelist'. > > The approach you are proposing has several issue for which I don't have the necessary bash chops to figure out how to tackle: 1. In the 1st while loop in ptxd_make_get(), I don't know how to get the case to accept multiple array values for ${PTXCONF_SETUP_PTXMIRROR}: case "${url}" in ${PTXCONF_SETUP_PTXMIRROR}/*/*) # keep original URL, for stuff like glibc argv[${#argv[@]}]="${url}" mrd=true ;; ${PTXCONF_SETUP_PTXMIRROR}/*) # if mirror is given us to download, add it, but only once if ! ${mrd}; then argv[${#argv[@]}]="${url}" mrd=true fi ;; Replacing with ${PTXCONF_SETUP_PTXMIRROR[@]} clear won't help as the various options aren't separate by PIPE character |. 2. Where ptxmirror_url gets assigned, it would also need to be an array too 3. And then also where ${ptxmirror_url} gets added to the argv array Quite frankly, I really like my approach much better... and it does not preclude you from implementing your solution as well. If I created and tested a patch that allows FOO_URL_SAFE, would you consider it for inclusion to ptxdist? Thanks, Jon [-- Attachment #1.2: Type: text/html, Size: 3116 bytes --] [-- Attachment #2: Type: text/plain, Size: 48 bytes --] -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-11 1:47 ` Jon Ringle @ 2014-01-24 9:21 ` Michael Olbrich 2014-01-24 13:10 ` Jon Ringle 0 siblings, 1 reply; 20+ messages in thread From: Michael Olbrich @ 2014-01-24 9:21 UTC (permalink / raw) To: ptxdist On Fri, Jan 10, 2014 at 08:47:52PM -0500, Jon Ringle wrote: > On Fri, Jan 10, 2014 at 10:37 AM, Michael Olbrich > <m.olbrich@pengutronix.de>wrote: > > > On Thu, Jan 09, 2014 at 11:37:15AM -0500, Jon Ringle wrote: > > > Would the following approach work: > > > > > > 1. In ptxdist/rules/post/ptxd_make_world_common.make add to > > world/env/impl: > > > pkg_url_safe="$(call ptx/escape,$($(1)_URL_SAFE))" > > > > > > 2. In ptxdist/scripts/lib/ptxd_make_get.sh function ptxd_make_get > > > change all occurances of: > > > if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" ]; then > > > with something like: > > > if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" -o -n "${pkg_url_safe}" ]; > > > then > > > > > > Then in the rule file you can do: > > > FOO_URL_SAFE := y > > > > > > So that you can whitelist a specific URL if PTXCONF_SETUP_PTXMIRROR_ONLY > > is > > > set > > > > Hmmm, this is not so nice. How about making PTXCONF_SETUP_PTXMIRROR a list? > > Then you can add http://opensource/pool/ and your git server to the > > 'whitelist'. > > > > The approach you are proposing has several issue for which I don't have > the necessary bash chops to figure out how to tackle: > > 1. In the 1st while loop in ptxd_make_get(), I don't know how to get the > case to accept multiple array values for ${PTXCONF_SETUP_PTXMIRROR}: > case "${url}" in > ${PTXCONF_SETUP_PTXMIRROR}/*/*) > # keep original URL, for stuff like glibc > argv[${#argv[@]}]="${url}" > mrd=true > ;; > ${PTXCONF_SETUP_PTXMIRROR}/*) > # if mirror is given us to download, add it, but only once > if ! ${mrd}; then > argv[${#argv[@]}]="${url}" > mrd=true > fi > ;; > > Replacing with ${PTXCONF_SETUP_PTXMIRROR[@]} clear won't help as the > various options aren't separate by PIPE character |. > > 2. Where ptxmirror_url gets assigned, it would also need to be an array too > 3. And then also where ${ptxmirror_url} gets added to the argv array > > Quite frankly, I really like my approach much better... and it does not > preclude you from implementing your solution as well. > If I created and tested a patch that allows FOO_URL_SAFE, would you > consider it for inclusion to ptxdist? I've been thinking about this some more. The choice to only download from the mirror comes from the BSP user via 'setup'. I don't think allowing the BSP / ptxdist developer to overwrite that is a good idea. What about an explicit URL whitelist in 'setup'? Michael -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [ptxdist] git ptxdist best practices 2014-01-24 9:21 ` Michael Olbrich @ 2014-01-24 13:10 ` Jon Ringle 0 siblings, 0 replies; 20+ messages in thread From: Jon Ringle @ 2014-01-24 13:10 UTC (permalink / raw) To: ptxdist On Fri, Jan 24, 2014 at 4:21 AM, Michael Olbrich <m.olbrich@pengutronix.de> wrote: > On Fri, Jan 10, 2014 at 08:47:52PM -0500, Jon Ringle wrote: >> On Fri, Jan 10, 2014 at 10:37 AM, Michael Olbrich >> <m.olbrich@pengutronix.de>wrote: >> >> > On Thu, Jan 09, 2014 at 11:37:15AM -0500, Jon Ringle wrote: >> > > Would the following approach work: >> > > >> > > 1. In ptxdist/rules/post/ptxd_make_world_common.make add to >> > world/env/impl: >> > > pkg_url_safe="$(call ptx/escape,$($(1)_URL_SAFE))" >> > > >> > > 2. In ptxdist/scripts/lib/ptxd_make_get.sh function ptxd_make_get >> > > change all occurances of: >> > > if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" ]; then >> > > with something like: >> > > if [ -z "${PTXCONF_SETUP_PTXMIRROR_ONLY}" -o -n "${pkg_url_safe}" ]; >> > > then >> > > >> > > Then in the rule file you can do: >> > > FOO_URL_SAFE := y >> > > >> > > So that you can whitelist a specific URL if PTXCONF_SETUP_PTXMIRROR_ONLY >> > is >> > > set >> > >> > Hmmm, this is not so nice. How about making PTXCONF_SETUP_PTXMIRROR a list? >> > Then you can add http://opensource/pool/ and your git server to the >> > 'whitelist'. >> > >> > The approach you are proposing has several issue for which I don't have >> the necessary bash chops to figure out how to tackle: >> >> 1. In the 1st while loop in ptxd_make_get(), I don't know how to get the >> case to accept multiple array values for ${PTXCONF_SETUP_PTXMIRROR}: >> case "${url}" in >> ${PTXCONF_SETUP_PTXMIRROR}/*/*) >> # keep original URL, for stuff like glibc >> argv[${#argv[@]}]="${url}" >> mrd=true >> ;; >> ${PTXCONF_SETUP_PTXMIRROR}/*) >> # if mirror is given us to download, add it, but only once >> if ! ${mrd}; then >> argv[${#argv[@]}]="${url}" >> mrd=true >> fi >> ;; >> >> Replacing with ${PTXCONF_SETUP_PTXMIRROR[@]} clear won't help as the >> various options aren't separate by PIPE character |. >> >> 2. Where ptxmirror_url gets assigned, it would also need to be an array too >> 3. And then also where ${ptxmirror_url} gets added to the argv array >> >> Quite frankly, I really like my approach much better... and it does not >> preclude you from implementing your solution as well. >> If I created and tested a patch that allows FOO_URL_SAFE, would you >> consider it for inclusion to ptxdist? > > I've been thinking about this some more. The choice to only download from > the mirror comes from the BSP user via 'setup'. I don't think allowing the > BSP / ptxdist developer to overwrite that is a good idea. > What about an explicit URL whitelist in 'setup'? I wouldn't mind that. This isn't any different than what you suggested before. But as I mentioned before, I don't know how to implement this. Jon -- ptxdist mailing list ptxdist@pengutronix.de ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2014-01-24 13:10 UTC | newest] Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-01-08 2:32 [ptxdist] git ptxdist best practices Jon Ringle 2014-01-08 5:56 ` Alexander Dahl 2014-01-08 6:12 ` Jon Ringle 2014-01-08 9:21 ` Alexander Dahl 2014-01-08 9:48 ` Alexander Dahl 2014-01-08 9:52 ` Jürgen Beisert 2014-01-08 17:56 ` Jon Ringle 2014-01-08 20:43 ` Alexander Dahl 2014-01-08 7:59 ` Jürgen Beisert 2014-01-08 14:14 ` Jon Ringle 2014-01-09 8:10 ` Jürgen Beisert 2014-01-09 9:03 ` Olbrich, Michael 2014-01-09 15:23 ` Jon Ringle 2014-01-09 16:06 ` Michael Olbrich 2014-01-09 16:37 ` Jon Ringle 2014-01-10 15:37 ` Michael Olbrich 2014-01-10 15:41 ` Jon Ringle 2014-01-11 1:47 ` Jon Ringle 2014-01-24 9:21 ` Michael Olbrich 2014-01-24 13:10 ` Jon Ringle
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox