* [ptxdist] [PATCH] plugins: remove old code
@ 2021-06-09 20:05 Roland Hieber
0 siblings, 0 replies; only message in thread
From: Roland Hieber @ 2021-06-09 20:05 UTC (permalink / raw)
To: ptxdist; +Cc: Roland Hieber
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf8, Size: 40163 bytes --]
Both url_check and url_check_ng have been touched last in 2008 and 2009,
they are referenced nowhere in the repository, and have also been
superseded by the 'ptxdist urlcheck' command since commit
5d39eb69ac6e116fbe0f (2016-08-20, "ptxdist: add urlcheck command").
Link: https://git.pengutronix.de/cgit/ptxdist/commit/?id=5d39eb69ac6e116fbe0f
Signed-off-by: Roland Hieber <rhi@pengutronix.de>
---
plugins/url_check/README | 12 -
plugins/url_check/main | 314 -----------------
plugins/url_check/ptxlib.bash | 351 --------------------
plugins/url_check_ng/README | 27 --
plugins/url_check_ng/main | 283 ----------------
plugins/url_check_ng/ptxlib.bash | 351 --------------------
plugins/url_check_ng/robotparser2.py.diff | 44 ---
plugins/url_check_ng/robotparser2.py.readme | 1 -
8 files changed, 1383 deletions(-)
delete mode 100644 plugins/url_check/README
delete mode 100755 plugins/url_check/main
delete mode 100644 plugins/url_check/ptxlib.bash
delete mode 100644 plugins/url_check_ng/README
delete mode 100755 plugins/url_check_ng/main
delete mode 100644 plugins/url_check_ng/ptxlib.bash
delete mode 100644 plugins/url_check_ng/robotparser2.py.diff
delete mode 100644 plugins/url_check_ng/robotparser2.py.readme
diff --git a/plugins/url_check/README b/plugins/url_check/README
deleted file mode 100644
index 482cbe1e1284..000000000000
--- a/plugins/url_check/README
+++ /dev/null
@@ -1,12 +0,0 @@
-# this is a quick hack, but quite useful
-
-Usage: main OPTIONS
-
- --help, -h this help
- --check-all, -a check ALL packages
-
-plugins/url_check/main checks the availability of all needed source packages
-for the current project configuration. By default, packages
-will only be checked, if the corresponding switch in
-ptxdistrc is set to "y" (enabled in 'ptxdist menuconfig')
-
diff --git a/plugins/url_check/main b/plugins/url_check/main
deleted file mode 100755
index 5789d4059bd7..000000000000
--- a/plugins/url_check/main
+++ /dev/null
@@ -1,314 +0,0 @@
-#!/bin/bash
-# ----------------------------------------------------------
-#
-# Script: PTXdist URL Checker
-# Rev: 1
-# Description:
-# Written by: Bjørn Bürger <b.buerger@pengutronix.de>
-# Changed: 2006-09-21 bbu
-# Docs: inline
-# Manpage: none
-#
-# ----------------------------------------------------------
-PREFIX="`basename $0` "
-# ----------------------------------------------------------
-# Short Documentation / Comments
-# ----------------------------------------------------------
-#
-# TODO:
-# build URL_CHECKER as host-tool in ptxdist.
-# Problem -> uses python => big and fat
-# OR find a lightweight alternative.
-#
-# Please note, that wget has problems w/ ftp links.
-#
-#
-
-# ----------------------------------------------------------
-# generic script settings
-# ----------------------------------------------------------
-#
-# The script domains - chose one or more of:
-# - dumb_tool
-# - tool
-# - development
-# - system_management
-#
-# PTX_SCRIPT_DOMAINS="dumb_tool"
-
-PTX_LIB_VERSION="2"
-
-# ----------------------------------------------------------
-# Default Configuration Options
-# ----------------------------------------------------------
-
-MKTEMP="mktemp"
-WHICH="which"
-GREP="grep"
-EGREP="egrep"
-CAT="cat"
-SED="sed"
-MKDIR="mkdir"
-
-LINKCHECKER_BIN="linkchecker"
-
-URL_CHECKER="$LINKCHECKER_BIN --no-warnings"
-# URL_CHECKER="wget --spider"
-
-PTXCONFIG="selected_ptxconfig"
-
-logdir="`dirname $0`/log"
-logfile="$logdir/url_check_log"
-
-# ----------------------------------------------------------
-# Load ptx shell library and generic ptx configuration
-# ----------------------------------------------------------
-
-PTXLIB=`dirname $0`/ptxlib.bash
-
-if [ -e "$PTXLIB" ] ; then
- . $PTXLIB
-else
- echo "ERROR: ptxlib not found"
- exit 1
-fi
-
-# ==========================================================
-# Temporary files
-# ==========================================================
-
-TMPDIR="`$MKTEMP -d /tmp/url_check_plugin.XXXXXXXXXX`" || echo "could not create TMPDIR"
-
-ptx_debug "TMPDIR is $TMPDIR"
-
-# ==========================================================
-# Traps
-# ==========================================================
-
-[ -e "$TMPDIR/on_exit_reverse.sh" ] || echo "echo" > $TMPDIR/on_exit_reverse.sh
-
-if [ "$DEBUG" = "true" ]; then
- ptx_debug "trap function: deleting temporary files..."
- trap '[ -e "$TMPDIR/on_exit_reverse.sh" ] && sh $TMPDIR/on_exit_reverse.sh ; rm -rvf /tmp/$(basename $TMPDIR)' EXIT
-else
- trap '[ -e "$TMPDIR/on_exit_reverse.sh" ] && sh $TMPDIR/on_exit_reverse.sh ; rm -rf /tmp/$(basename $TMPDIR)' EXIT
-fi
-
-# ----------------------------------------------------------
-# Default Dependency check
-# ----------------------------------------------------------
-
-dependency_check_dirs_depends=""
-dependency_check_files_depends="$PTXCONFIG"
-dependency_check_tools_depends="$LINKCHECKER_BIN $MKTEMP"
-
-ptx_dependency_check
-
-# ==========================================================
-# Option Parser
-# ==========================================================
-
-Usage() {
-cat <<-EOF
-
-Usage: `basename "$0"` OPTIONS
-
- --help -h this help
- --check-all -a check ALL packages
- --modules -m check module packages
- --builtin -y check builtin packages
-
-$0 checks the availability of all needed source packages
-for the current project configuration. By default, packages
-will only be checked, if the corresponding switch in
-ptxdistrc is set to "y" (enabled in 'ptxdist menuconfig')
-
-EOF
-}
-
-# Parser
-# ------
-# option no argument
-# option: required argument
-# option:: optional argument
-
-TEMP=`getopt --options h,a,y,m \
- --longoptions="help,check-all,modules,builtin" \
- -n "$0" -- "$@"`
-if [ $? != 0 ] ; then echo "Terminating..." >&2 ; exit 1 ; fi
-eval set -- "$TEMP"
-
-while true ; do
- case "$1" in
- -h|--help)
- [ -z "$action" ]
- action="help" ;
- shift
- ;;
- -a|--check-all)
- [ -z "$action" ]
- action="check_all" ;
- shift
- ;;
- -m|--modules)
- [ -z "$action" ]
- action="check_modules" ;
- shift
- ;;
- -y|--builtin)
- [ -z "$action" ]
- action="check_builtin" ;
- shift
- ;;
- --) shift ; break ;;
- *) echo "Internal error!" ; exit 1 ;;
- esac
-done
-
-# ==========================================================
-# Script Variables
-# ==========================================================
-
-# none
-
-# ==========================================================
-# Script Functions
-# ==========================================================
-
-init(){
- # choose the right ptxdist version:
- echo "PATH is $PATH"
- PTXDIST_VERSION=$(echo `$GREP PTXCONF_CONFIGFILE_VERSION $PTXCONFIG` \; echo "\$PTXCONF_CONFIGFILE_VERSION" | sh)
- PTXDIST=$(dirname $0)/../../bin/ptxdist
- if [ "$($PTXDIST --version)" != "$PTXDIST_VERSION" ]; then
- PTXDIST=ptxdist-$PTXDIST_VERSION
- which $PTXDIST || PTXDIST=ptxdist
- fi
- PTXDIST_BIN="`$WHICH $PTXDIST`"
- ptx_debug "PTXDIST is: $PTXDIST ($PTXDIST_BIN)"
- if [ "$1" = "all" ]; then
- # get all package labels
- # (PACKAGES-y contains all activated packages)
- # (PACKAGES- contains all deactivated packages)
- YESPACKAGES=`$PTXDIST print PACKAGES-y`
- MODPACKAGES=`$PTXDIST print PACKAGES-m`
- NOPACKAGES=`$PTXDIST print PACKAGES-`
- elif [ "$1" = "active" ]; then
- # get only configured builtin package labels
- # (PACKAGES-y contains all active builtin packages)
- YESPACKAGES=`$PTXDIST print PACKAGES-y`
- MODPACKAGES=""
- NOPACKAGES=""
- elif [ "$1" = "modules" ]; then
- # get only configured MODULE package labels
- # (PACKAGES-m contains all module packages)
- YESPACKAGES=""
- MODPACKAGES=`$PTXDIST print PACKAGES-m`
- NOPACKAGES=""
- else
- # get all configured builtin and module package labels
- # (PACKAGES-y contains all activated packages)
- YESPACKAGES=`$PTXDIST print PACKAGES-y`
- MODPACKAGES=`$PTXDIST print PACKAGES-m`
- NOPACKAGES=""
- fi
- PACKAGES="$(echo $YESPACKAGES $MODPACKAGES $NOPACKAGES | tr "[a-z-]" "[A-Z_]")"
-}
-
-create_url_list(){
- echo "creating list of download URLS for `echo $PACKAGES | wc -w` packages"
- targets=`for target in $PACKAGES; do echo print-${target}_URL; done | sort -u`
- $PTXDIST make -i ${targets} 2>> $TMPDIR/errors > $TMPDIR/urllist
- if [ `cat "$TMPDIR/urllist" | wc -l` = "0" ]; then
- echo "Fast listing failed. This may take a while!"
- for target in $PACKAGES; do
- $PTXDIST print ${target}_URL 2>> $TMPDIR/errors >> $TMPDIR/urllist
- echo -n "."
- done
- fi
- echo "done"
-}
-
-test_urls(){
- egrep -v "^#|^$|^[[:space:]]" $TMPDIR/urllist \
- | while read line; do
- local count_ok=0
- local count_fail=0
- rm -f $TMPDIR/errormsg
- for url in $line; do
- $URL_CHECKER $url >>$TMPDIR/errormsg 2>&1
- case $? in
- 0)
- count_ok=$[count_ok+1]
- ;;
- *)
- count_fail=$[count_fail+1]
- ;;
- esac
- done
- if [ $count_fail -eq 0 ]; then
- echo " [ OK ] $line"
- elif [ $count_ok -eq 0 ]; then
- echo " [ FAIL ] $line"
- echo ""
- echo ""
- $CAT $TMPDIR/errormsg | $GREP -A 10 "Start checking at"
- echo ""
- else
- local count=$[count_ok+$count_fail]
- echo " [ $count_ok/$count ] $line"
- fi
- done
-}
-
-runner(){
- $MKDIR -p $logdir
- case $? in
- 0)
- test_urls | tee $logfile
- echo "LOGFILE is --> $logfile"
- ;;
- *)
- test_urls
- ;;
- esac
-}
-
-# ==========================================================
-# Script Main
-# ==========================================================
-
-case "$action" in
- help)
- Usage
- ;;
- check_all)
- init all
- create_url_list >&2
- runner
- ;;
- check_modules)
- init modules
- create_url_list >&2
- runner
- ;;
- check_builtin)
- init active
- create_url_list >&2
- runner
- ;;
- *)
- init
- create_url_list >&2
- runner
- ;;
-esac
-
-# ==========================================================
-# Cleanup
-# ==========================================================
-
-# unconfigured - done by trap function
-# ----------------------------------------------------------
-# End
-# ----------------------------------------------------------
diff --git a/plugins/url_check/ptxlib.bash b/plugins/url_check/ptxlib.bash
deleted file mode 100644
index 50249916d2b7..000000000000
--- a/plugins/url_check/ptxlib.bash
+++ /dev/null
@@ -1,351 +0,0 @@
-#!/bin/bash
-# -----------------------------------------------------
-# Description: Basic Script Functions
-# Author: Bjørn Bürger <b.buerger@pengutronix.de>
-# Date: Wed Mar 21 17:03:39 CET 2007
-# -----------------------------------------------------
-
-PTXLIB="true"
-MY_VERSION="2"
-
-if [ "$PTX_LIB_VERSION" -lt "$MY_VERSION" ] ; then
- echo "WARNING: ptxlib.bash not compatible"
- echo "WARNING: please audit your script"
- exit 1
-fi
-
-FULLARGS="$@"
-DEBUG=${DEBUG:="false"}
-
-#
-# customized exit functions
-#
-# $1 --> Error Message
-# $2 --> Exit Code
-#
-ptx_exit(){
- echo "$0: $1"
- exit $2
-}
-ptx_exit_silent(){
- ptx_debug "$0: $1"
- exit $2
-}
-
-#
-# print out error message and exit with status 1
-#
-# $1: error message
-# ${PREFIX}: to be printed before message
-#
-ptx_bailout () {
- echo "${PREFIX}error: $1" >&2
- exit 1
-}
-
-
-#
-# print out warning message
-#
-# $1: warning message
-# ${PREFIX}: to be printed before message
-#
-ptx_warning() {
- echo "${PREFIX}warning: $1" >&2
-}
-
-#
-# print out classic message
-#
-# $1: warning message
-# ${PREFIX}: to be printed before message
-#
-ptx_message() {
- echo "${PREFIX}: $1" >&1
-}
-
-ptx_message_n() {
- if [ -n "$PREFIX" ]; then
- echo -n "${PREFIX}: $1" >&1
- else
- echo -n "$1" >&1
- fi
-}
-
-#
-# Debugging Output
-#
-ptx_debug(){
- [ "$DEBUG" = "true" ] && echo "$0: $1" >&2
-}
-ptx_debug "Debugging is enabled - Turn off with DEBUG=false"
-
-
-#
-# check if a previously executed pipe returned an error
-#
-ptx_check_pipe_status() {
- for i in "${PIPESTATUS[@]}"; do [ $i -gt 0 ] && {
- echo
- echo "error: a command in the pipe returned $i, bailing out"
- echo
- exit $i
- }
- done
-}
-
-#
-# present a choice [y/n] and execute command
-#
-# $1: question / message
-# $2: command
-# $3: yes message
-# $4: no message
-# ${PREFIX}: to be printed before message
-#
-#
-ptx_yesno_choice(){
- ptx_debug "Message: $1"
- ptx_debug "Command: $2"
- dialog --yesno "$1" 0 0
- case $? in
- 0)
- ptx_message "${PREFIX} [YES] - $3"
- $2 || ptx_error "command $2 failed"
- ;;
- *)
- ptx_warning "${PREFIX} [NO] - $4"
- ;;
- esac
-}
-
-#
-# create a directory
-#
-ptx_check_create_dir(){
- [ -d "$1" ] || ptx_yesno_choice "$1 does not exist, create it?" "mkdir -pv $1" "[SUCCESS]" "[ABORTED]"
-}
-
-#
-# dependency check for needed files
-#
-# if any one of these variables is
-# set, a dependency check is performed:
-#
-# dependency_check_files_recommends
-# dependency_check_files_depends
-# dependency_check_files_conflicts
-#
-ptx_dependency_check_files(){
- [ -z "$dependency_check_files_recommends" ] && ptx_debug "[file check] no file recommendation defined"
- for file in $dependency_check_files_recommends; do
- test -e $file
- case $? in
- 0)
- ptx_debug "[file check] This Script recommends $file: OK"
- ;;
- *)
- ptx_warning "[file check] This Script recommends $file: FILE NOT FOUND"
- ;;
- esac
- done
- [ -z "$dependency_check_files_depends" ] && ptx_debug "[file check] no file dependency defined"
- for file in $dependency_check_files_depends; do
- test -e $file
- case $? in
- 0)
- ptx_debug "[file check] This Script depends on $file: OK"
- ;;
- *)
- ptx_bailout "[file check] This Script depends on $file: FILE NOT FOUND"
- ;;
- esac
- done
- [ -z "$dependency_check_files_conflicts" ] && ptx_debug "[file check] no file conflict defined"
- for file in $dependency_check_files_conflicts; do
- test -e $file
- case $? in
- 0)
- ptx_bailout "[file check] This Script conflicts with $file: FILE EXISTS"
- ;;
- *)
- ptx_debug "[file check] This Script conflicts with $file: OK (does not exist)"
- ;;
- esac
- done
-}
-
-#
-# dependency check for needed directories
-#
-# if any one of these variables is
-# set, a dependency check is performed:
-#
-# dependency_check_dirs_recommends
-# dependency_check_dirs_depends
-# dependency_check_dirs_conflicts
-#
-ptx_dependency_check_dirs(){
- [ -z "$dependency_check_dirs_recommends" ] && ptx_debug "[dir check] no directory recommendation defined"
- for directory in $dependency_check_dirs_recommends; do
- test -d $directory
- case $? in
- 0)
- ptx_debug "[dir check] This Script recommends $directory: OK"
- ;;
- *)
- ptx_warning "[dir check] This Script recommends $directory: directory NOT FOUND"
- ;;
- esac
- done
- [ -z "$dependency_check_dirs_depends" ] && ptx_debug "[dir check] no directory dependency defined"
- for directory in $dependency_check_dirs_depends; do
- test -d $directory
- case $? in
- 0)
- ptx_debug "[dir check] This Script depends on $directory: OK"
- ;;
- *)
- ptx_bailout "[dir check] This Script depends on $directory: directory NOT FOUND"
- ;;
- esac
- done
- [ -z "$dependency_check_dirs_conflicts" ] && ptx_debug "[dir check] no directory conflict defined"
- for directory in $dependency_check_dirs_conflicts; do
- test -d $directory
- case $? in
- 0)
- ptx_bailout "[dir check] This Script conflicts with $directory: directory EXISTS"
- ;;
- *)
- ptx_debug "[dir check] This Script conflicts with $directory: OK (does not exist)"
- ;;
- esac
- done
-}
-
-#
-# dependency check for needed tools
-#
-# if any one of these variables is
-# set, a dependency check is performed:
-#
-# dependency_check_tools_recommends
-# dependency_check_tools_depends
-# dependency_check_tools_conflicts
-#
-ptx_dependency_check_tools(){
- [ -z "$dependency_check_files_recommends" ] && ptx_debug "[file check] no tool recommendation defined"
- for tool in $dependency_check_tools_recommends; do
- which $tool >/dev/null 2>&1
- case $? in
- 0)
- ptx_debug "[tool check] This Script recommends $tool: OK"
- ;;
- *)
- ptx_warning "[tool check] This Script recommends $tool: MISSING"
- ;;
- esac
- done
- [ -z "$dependency_check_tools_depends" ] && ptx_debug "[file check] no tool dependency defined"
- for tool in $dependency_check_tools_depends; do
- which $tool >/dev/null 2>&1
- case $? in
- 0)
- ptx_debug "[tool check] This Script depends on $tool: OK"
- ;;
- *)
- ptx_bailout "[tool check] This Script depends on $tool: MISSING"
- ;;
- esac
- done
- [ -z "$dependency_check_tools_conflicts" ] && ptx_debug "[file check] no file conflict defined"
- for tool in $dependency_check_tools_conflicts; do
- which $tool >/dev/null 2>&1
- case $? in
- 0)
- ptx_bailout "[tool check] This Script conflicts with on $tool: CONFLICT FOUND"
- ;;
- *)
- ptx_debug "[tool check] This Script conflicts with $tool: OK"
- ;;
- esac
- done
-}
-
-ptx_dependency_check_hostname(){
- [ -z "$dependency_check_hostname_recommends" ] && ptx_debug "[hostname check] no hostname recommendation defined"
- for hostname in $dependency_check_hostname_recommends; do
- [ "`hostname`" = "$hostname" ] >/dev/null 2>&1
- case $? in
- 0)
- ptx_debug "[hostname check] This Script recommends host $hostname: OK"
- ;;
- *)
- ptx_warning "[hostname check] This Script recommends host $hostname: THIS IS NOT ME"
- ;;
- esac
- done
- [ -z "$dependency_check_hostname_depends" ] && ptx_debug "[hostname check] no hostname dependency defined"
- for hostname in $dependency_check_hostname_depends; do
- [ "`hostname`" = "$hostname" ] >/dev/null 2>&1
- case $? in
- 0)
- ptx_debug "[hostname check] This Script depends on host $hostname: OK"
- ;;
- *)
- ptx_bailout "[hostname check] This Script depends on host $hostname: THIS IS NOT ME"
- ;;
- esac
- done
- [ -z "$dependency_check_hostname_conflicts" ] && ptx_debug "[hostname check] no hostname conflict defined"
- for hostname in $dependency_check_hostname_conflicts; do
- [ "`hostname`" = "$hostname" ] >/dev/null 2>&1
- case $? in
- 0)
- ptx_bailout "[hostname check] This Script conflicts with host $hostname: CONFLICT FOUND"
- ;;
- *)
- ptx_debug "[hostname check] This Script conflicts with host $hostname: OK"
- ;;
- esac
- done
-}
-
-#
-# convenience wrapper for the functions above
-#
-# performs all available checks
-#
-ptx_dependency_check(){
- ptx_dependency_check_hostname
- ptx_dependency_check_dirs
- ptx_dependency_check_files
- ptx_dependency_check_tools
-}
-
-# service specific lib functions, based on script domains:
-#
-# - dumb_tool
-# - tool
-# - development
-# - system_management
-#
-
-if [ -n "$PTX_SCRIPT_DOMAINS" ]; then
-for ptx_script_domain in $PTX_SCRIPT_DOMAINS; do
- ptx_debug "loading config for domain: ${ptx_script_domain}"
- SCONFS="$HOME/.ptxsd-${ptx_script_domain}.conf \
- /usr/local/etc/ptxsd-${ptx_script_domain}.conf \
- /etc/ptxsd-${ptx_script_domain}.conf \
- `dirname $0`/ptxsd-${ptx_script_domain}.conf \
- `dirname $0`/../lib/ptxsd-${ptx_script_domain}.conf"
- for scriptconf in $SCONFS; do
- if [ -e "$scriptconf" ] ; then
- . $scriptconf && { ptx_debug "$scriptconf loaded"; break; }
- fi
- done
- [ "$PTXSD" = "true" ] || ptx_warning " No valid Configuration for Domain: ${ptx_script_domain} found \
- [ $conf (version $MY_VERSION) ] [ caller: $0 ]"
-done
-fi
diff --git a/plugins/url_check_ng/README b/plugins/url_check_ng/README
deleted file mode 100644
index f29656fd802c..000000000000
--- a/plugins/url_check_ng/README
+++ /dev/null
@@ -1,27 +0,0 @@
-# new urlcheck version, which uses m2b environment
-# for faster processing. This will speed up processing
-# ( ~about 30 seconds to 5 minutes ).
-
-Usage: main OPTIONS
-
- --help, -h this help
- --check-all, -a check ALL packages
-
-plugins/url_check_ng/main checks the availability of all
-needed source packages for the current project configuration.
-By default, packages will only be checked, if the
-corresponding switch in ptxdistrc is set to "y" (enabled
-in 'ptxdist menuconfig')
-
-Usage:
-
-<full_path>/plugins/url_check_ng/main [--help | --check-all ]
-
-OR
-
-user@host:YourPTXdistProject$ ptxdist make plugin-url_check_ng
-
-PTXdist will will try to find the plugin in
-a) $PTXDIST_WORKSPACE/plugins
-b) $PTXDIST_TOPDIR/plugins
-
diff --git a/plugins/url_check_ng/main b/plugins/url_check_ng/main
deleted file mode 100755
index d896e0a866e4..000000000000
--- a/plugins/url_check_ng/main
+++ /dev/null
@@ -1,283 +0,0 @@
-#!/bin/bash
-# ----------------------------------------------------------
-#
-# Script: PTXdist URL Checker
-# Rev: 1.1
-# Description:
-# Written by: Bjørn Bürger <b.buerger@pengutronix.de>
-# Changed: 2006-09-21 bbu
-# Docs: inline
-# Manpage: none
-#
-# ----------------------------------------------------------
-PREFIX="`basename $0` "
-# ----------------------------------------------------------
-# Short Documentation / Comments
-# ----------------------------------------------------------
-#
-# TODO:
-# build URL_CHECKER as host-tool in ptxdist.
-# Problem -> uses python => big and fat
-# OR find a lightweight alternative.
-#
-# Please note, that wget has problems w/ ftp links.
-#
-#
-
-# ----------------------------------------------------------
-# generic script settings
-# ----------------------------------------------------------
-#
-# The script domains - chose one or more of:
-# - dumb_tool
-# - tool
-# - development
-# - system_management
-#
-# PTX_SCRIPT_DOMAINS="dumb_tool"
-
-PTX_LIB_VERSION="2"
-
-# ----------------------------------------------------------
-# Default Configuration Options
-# ----------------------------------------------------------
-
-MKTEMP="mktemp"
-WHICH="which"
-GREP="grep"
-EGREP="egrep"
-CAT="cat"
-SED="sed"
-MKDIR="mkdir"
-
-LINKCHECKER_BIN="linkchecker"
-
-# If you are using the python based linkchecker, this
-# option may speed up things *a little bit* ;-)
-if [ "$LINKCHECKER_BIN" == "linkchecker" ]; then
- LINKCHECKER_FASTMODE=${LINKCHECKER_FASTMODE:=true}
- LINKCHECKER_THREADS=${LINKCHECKER_THREADS:=50}
-else
- LINKCHECKER_FASTMODE=${LINKCHECKER_FASTMODE:=false}
-fi
-
-URL_CHECKER="$LINKCHECKER_BIN --no-warnings"
-# URL_CHECKER="wget --spider"
-
-PTXCONFIG="ptxconfig"
-M2B_ENV="${PTXDIST_PLATFORMDIR}/state/environment"
-
-# ----------------------------------------------------------
-# Load ptx shell library and generic ptx configuration
-# ----------------------------------------------------------
-
-PTXLIB=`dirname $0`/ptxlib.bash
-
-if [ -e "$PTXLIB" ] ; then
- . $PTXLIB
-else
- echo "ERROR: ptxlib not found"
- exit 1
-fi
-
-get_ptxdist_var(){
- grep $1 $M2B_ENV | while read a b ; do echo $b ; done;
-}
-
-logdir="`get_ptxdist_var PTXDIST_WORKSPACE`/log"
-logfile="$logdir/url_check_ng_log"
-
-# ==========================================================
-# Temporary files
-# ==========================================================
-
-TMPDIR="`$MKTEMP -d /tmp/url_check_plugin.XXXXXXXXXX`" || echo "could not create TMPDIR"
-
-ptx_debug "TMPDIR is $TMPDIR"
-
-# ==========================================================
-# Traps
-# ==========================================================
-
-[ -e "$TMPDIR/on_exit_reverse.sh" ] || echo "echo" > $TMPDIR/on_exit_reverse.sh
-
-if [ "$DEBUG" = "true" ]; then
- ptx_debug "trap function: deleting temporary files..."
- trap '[ -e "$TMPDIR/on_exit_reverse.sh" ] && sh $TMPDIR/on_exit_reverse.sh ; rm -rvf /tmp/$(basename $TMPDIR)' EXIT
-else
- trap '[ -e "$TMPDIR/on_exit_reverse.sh" ] && sh $TMPDIR/on_exit_reverse.sh ; rm -rf /tmp/$(basename $TMPDIR)' EXIT
-fi
-
-# ----------------------------------------------------------
-# Default Dependency check
-# ----------------------------------------------------------
-
-dependency_check_dirs_depends=""
-dependency_check_files_depends="$PTXCONFIG"
-dependency_check_tools_depends="$LINKCHECKER_BIN $MKTEMP"
-
-ptx_dependency_check
-
-# ==========================================================
-# Option Parser
-# ==========================================================
-
-Usage() {
-cat <<-EOF
-
-Usage: `basename "$0"` OPTIONS
-
- --help, -h this help
- --check-all, -a check ALL packages
-
-$0 checks the availability of all needed source packages
-for the current project configuration. By default, packages
-will only be checked, if the corresponding switch in
-ptxdistrc is set to "y" (enabled in 'ptxdist menuconfig')
-
-EOF
-}
-
-# Parser
-# ------
-# option no argument
-# option: required argument
-# option:: optional argument
-
-TEMP=`getopt --options h,a \
- --longoptions="help,check-all" \
- -n "$0" -- "$@"`
-if [ $? != 0 ] ; then echo "Terminating..." >&2 ; exit 1 ; fi
-eval set -- "$TEMP"
-
-while true ; do
- case "$1" in
- -h|--help)
- [ -z "$action" ]
- action="help" ;
- shift
- ;;
- -a|--check-all)
- [ -z "$action" ]
- action="check_all" ;
- shift
- ;;
- --) shift ; break ;;
- *) echo "Internal error!" ; exit 1 ;;
- esac
-done
-
-# ==========================================================
-# Script Variables
-# ==========================================================
-
-# none
-
-# ==========================================================
-# Script Functions
-# ==========================================================
-
-init(){
- # choose the right ptxdist version:
- PTXDIST=$(echo `$GREP PTXCONF_CONFIGFILE_VERSION $PTXCONFIG` \; echo "ptxdist-\$PTXCONF_CONFIGFILE_VERSION" | sh)
- PTXDIST_BIN="`$WHICH $PTXDIST || $WHICH ptxdist || ptx_bailout 'NO PTXDIST FOUND'`"
- ptx_debug "PTXDIST is: $PTXDIST ($PTXDIST_BIN)"
- # init package label list if it does not exist
- [ -e "${M2B_ENV}" ] || $PTXDIST_BIN make dump || ptx_bailout "could not dump environment"
- while read line content; do
- case $line in
- "PACKAGES-y"*)
- # we don´t want to see kernel- and u-boot urls, so we just skip these packages
- YESPACKAGES=`echo "$content" | sed -e s/kernel\ //g -e s/u-boot\ //g | tr "a-z-" "A-Z_"`
- ;;
- "PACKAGES-"*)
- NOPACKAGES=`echo "$content" | sed -e s/kernel\ //g -e s/u-boot\ //g | tr "a-z-" "A-Z_"`
- ;;
- esac
- done < ${M2B_ENV}
- if [ "$1" = "all" ]; then
- PACKAGES="$YESPACKAGES $NOPACKAGES"
- else
- PACKAGES="$YESPACKAGES"
- fi
-}
-
-create_url_list(){
- echo "creating list of download URLS for `echo $PACKAGES | wc -w` packages"
- for target in $PACKAGES; do
- grep "^${target}_URL\ " ${M2B_ENV} | while read name location; do
- if [ -n "$location" ]; then
- echo "$location" >> $TMPDIR/urllist
- echo "<a href=\"$location\">PTXdist Target ${target}</a><br>" >> $TMPDIR/urllist.html
- fi
- done
- echo -n "."
- done
- echo "done"
-}
-
-test_urls(){
- if [ "$LINKCHECKER_FASTMODE" = "true" ] ; then
- $URL_CHECKER -t${LINKCHECKER_THREADS:=10} $TMPDIR/urllist.html > $TMPDIR/errormsg
- egrep -v "^URL|^Parent\ URL|^Check\ Time" $TMPDIR/errormsg | sed -n '/Start/,$p'
- else
- egrep -v "^#|^$|^[[:space:]]" $TMPDIR/urllist \
- | while read line; do
- $URL_CHECKER $line >$TMPDIR/errormsg 2>&1
- case $? in
- 0)
- echo " [ OK ] $line"
- ;;
- *)
- echo ""
- echo " [ FAIL ] $line"
- echo ""
- $CAT $TMPDIR/errormsg | $GREP -A 10 "Start checking at"
- echo ""
- ;;
- esac
- done
- fi
-}
-
-runner(){
- $MKDIR -p $logdir
- case $? in
- 0)
- test_urls | tee $logfile
- echo "LOGFILE is --> $logfile"
- ;;
- *)
- test_urls
- ;;
- esac
-}
-
-# ==========================================================
-# Script Main
-# ==========================================================
-
-case "$action" in
- help)
- Usage
- ;;
- check_all)
- init all
- create_url_list >&2
- runner
- ;;
- *)
- init
- create_url_list >&2
- runner
- ;;
-esac
-
-# ==========================================================
-# Cleanup
-# ==========================================================
-
-# unconfigured - done by trap function
-# ----------------------------------------------------------
-# End
-# ----------------------------------------------------------
diff --git a/plugins/url_check_ng/ptxlib.bash b/plugins/url_check_ng/ptxlib.bash
deleted file mode 100644
index 50249916d2b7..000000000000
--- a/plugins/url_check_ng/ptxlib.bash
+++ /dev/null
@@ -1,351 +0,0 @@
-#!/bin/bash
-# -----------------------------------------------------
-# Description: Basic Script Functions
-# Author: Bjørn Bürger <b.buerger@pengutronix.de>
-# Date: Wed Mar 21 17:03:39 CET 2007
-# -----------------------------------------------------
-
-PTXLIB="true"
-MY_VERSION="2"
-
-if [ "$PTX_LIB_VERSION" -lt "$MY_VERSION" ] ; then
- echo "WARNING: ptxlib.bash not compatible"
- echo "WARNING: please audit your script"
- exit 1
-fi
-
-FULLARGS="$@"
-DEBUG=${DEBUG:="false"}
-
-#
-# customized exit functions
-#
-# $1 --> Error Message
-# $2 --> Exit Code
-#
-ptx_exit(){
- echo "$0: $1"
- exit $2
-}
-ptx_exit_silent(){
- ptx_debug "$0: $1"
- exit $2
-}
-
-#
-# print out error message and exit with status 1
-#
-# $1: error message
-# ${PREFIX}: to be printed before message
-#
-ptx_bailout () {
- echo "${PREFIX}error: $1" >&2
- exit 1
-}
-
-
-#
-# print out warning message
-#
-# $1: warning message
-# ${PREFIX}: to be printed before message
-#
-ptx_warning() {
- echo "${PREFIX}warning: $1" >&2
-}
-
-#
-# print out classic message
-#
-# $1: warning message
-# ${PREFIX}: to be printed before message
-#
-ptx_message() {
- echo "${PREFIX}: $1" >&1
-}
-
-ptx_message_n() {
- if [ -n "$PREFIX" ]; then
- echo -n "${PREFIX}: $1" >&1
- else
- echo -n "$1" >&1
- fi
-}
-
-#
-# Debugging Output
-#
-ptx_debug(){
- [ "$DEBUG" = "true" ] && echo "$0: $1" >&2
-}
-ptx_debug "Debugging is enabled - Turn off with DEBUG=false"
-
-
-#
-# check if a previously executed pipe returned an error
-#
-ptx_check_pipe_status() {
- for i in "${PIPESTATUS[@]}"; do [ $i -gt 0 ] && {
- echo
- echo "error: a command in the pipe returned $i, bailing out"
- echo
- exit $i
- }
- done
-}
-
-#
-# present a choice [y/n] and execute command
-#
-# $1: question / message
-# $2: command
-# $3: yes message
-# $4: no message
-# ${PREFIX}: to be printed before message
-#
-#
-ptx_yesno_choice(){
- ptx_debug "Message: $1"
- ptx_debug "Command: $2"
- dialog --yesno "$1" 0 0
- case $? in
- 0)
- ptx_message "${PREFIX} [YES] - $3"
- $2 || ptx_error "command $2 failed"
- ;;
- *)
- ptx_warning "${PREFIX} [NO] - $4"
- ;;
- esac
-}
-
-#
-# create a directory
-#
-ptx_check_create_dir(){
- [ -d "$1" ] || ptx_yesno_choice "$1 does not exist, create it?" "mkdir -pv $1" "[SUCCESS]" "[ABORTED]"
-}
-
-#
-# dependency check for needed files
-#
-# if any one of these variables is
-# set, a dependency check is performed:
-#
-# dependency_check_files_recommends
-# dependency_check_files_depends
-# dependency_check_files_conflicts
-#
-ptx_dependency_check_files(){
- [ -z "$dependency_check_files_recommends" ] && ptx_debug "[file check] no file recommendation defined"
- for file in $dependency_check_files_recommends; do
- test -e $file
- case $? in
- 0)
- ptx_debug "[file check] This Script recommends $file: OK"
- ;;
- *)
- ptx_warning "[file check] This Script recommends $file: FILE NOT FOUND"
- ;;
- esac
- done
- [ -z "$dependency_check_files_depends" ] && ptx_debug "[file check] no file dependency defined"
- for file in $dependency_check_files_depends; do
- test -e $file
- case $? in
- 0)
- ptx_debug "[file check] This Script depends on $file: OK"
- ;;
- *)
- ptx_bailout "[file check] This Script depends on $file: FILE NOT FOUND"
- ;;
- esac
- done
- [ -z "$dependency_check_files_conflicts" ] && ptx_debug "[file check] no file conflict defined"
- for file in $dependency_check_files_conflicts; do
- test -e $file
- case $? in
- 0)
- ptx_bailout "[file check] This Script conflicts with $file: FILE EXISTS"
- ;;
- *)
- ptx_debug "[file check] This Script conflicts with $file: OK (does not exist)"
- ;;
- esac
- done
-}
-
-#
-# dependency check for needed directories
-#
-# if any one of these variables is
-# set, a dependency check is performed:
-#
-# dependency_check_dirs_recommends
-# dependency_check_dirs_depends
-# dependency_check_dirs_conflicts
-#
-ptx_dependency_check_dirs(){
- [ -z "$dependency_check_dirs_recommends" ] && ptx_debug "[dir check] no directory recommendation defined"
- for directory in $dependency_check_dirs_recommends; do
- test -d $directory
- case $? in
- 0)
- ptx_debug "[dir check] This Script recommends $directory: OK"
- ;;
- *)
- ptx_warning "[dir check] This Script recommends $directory: directory NOT FOUND"
- ;;
- esac
- done
- [ -z "$dependency_check_dirs_depends" ] && ptx_debug "[dir check] no directory dependency defined"
- for directory in $dependency_check_dirs_depends; do
- test -d $directory
- case $? in
- 0)
- ptx_debug "[dir check] This Script depends on $directory: OK"
- ;;
- *)
- ptx_bailout "[dir check] This Script depends on $directory: directory NOT FOUND"
- ;;
- esac
- done
- [ -z "$dependency_check_dirs_conflicts" ] && ptx_debug "[dir check] no directory conflict defined"
- for directory in $dependency_check_dirs_conflicts; do
- test -d $directory
- case $? in
- 0)
- ptx_bailout "[dir check] This Script conflicts with $directory: directory EXISTS"
- ;;
- *)
- ptx_debug "[dir check] This Script conflicts with $directory: OK (does not exist)"
- ;;
- esac
- done
-}
-
-#
-# dependency check for needed tools
-#
-# if any one of these variables is
-# set, a dependency check is performed:
-#
-# dependency_check_tools_recommends
-# dependency_check_tools_depends
-# dependency_check_tools_conflicts
-#
-ptx_dependency_check_tools(){
- [ -z "$dependency_check_files_recommends" ] && ptx_debug "[file check] no tool recommendation defined"
- for tool in $dependency_check_tools_recommends; do
- which $tool >/dev/null 2>&1
- case $? in
- 0)
- ptx_debug "[tool check] This Script recommends $tool: OK"
- ;;
- *)
- ptx_warning "[tool check] This Script recommends $tool: MISSING"
- ;;
- esac
- done
- [ -z "$dependency_check_tools_depends" ] && ptx_debug "[file check] no tool dependency defined"
- for tool in $dependency_check_tools_depends; do
- which $tool >/dev/null 2>&1
- case $? in
- 0)
- ptx_debug "[tool check] This Script depends on $tool: OK"
- ;;
- *)
- ptx_bailout "[tool check] This Script depends on $tool: MISSING"
- ;;
- esac
- done
- [ -z "$dependency_check_tools_conflicts" ] && ptx_debug "[file check] no file conflict defined"
- for tool in $dependency_check_tools_conflicts; do
- which $tool >/dev/null 2>&1
- case $? in
- 0)
- ptx_bailout "[tool check] This Script conflicts with on $tool: CONFLICT FOUND"
- ;;
- *)
- ptx_debug "[tool check] This Script conflicts with $tool: OK"
- ;;
- esac
- done
-}
-
-ptx_dependency_check_hostname(){
- [ -z "$dependency_check_hostname_recommends" ] && ptx_debug "[hostname check] no hostname recommendation defined"
- for hostname in $dependency_check_hostname_recommends; do
- [ "`hostname`" = "$hostname" ] >/dev/null 2>&1
- case $? in
- 0)
- ptx_debug "[hostname check] This Script recommends host $hostname: OK"
- ;;
- *)
- ptx_warning "[hostname check] This Script recommends host $hostname: THIS IS NOT ME"
- ;;
- esac
- done
- [ -z "$dependency_check_hostname_depends" ] && ptx_debug "[hostname check] no hostname dependency defined"
- for hostname in $dependency_check_hostname_depends; do
- [ "`hostname`" = "$hostname" ] >/dev/null 2>&1
- case $? in
- 0)
- ptx_debug "[hostname check] This Script depends on host $hostname: OK"
- ;;
- *)
- ptx_bailout "[hostname check] This Script depends on host $hostname: THIS IS NOT ME"
- ;;
- esac
- done
- [ -z "$dependency_check_hostname_conflicts" ] && ptx_debug "[hostname check] no hostname conflict defined"
- for hostname in $dependency_check_hostname_conflicts; do
- [ "`hostname`" = "$hostname" ] >/dev/null 2>&1
- case $? in
- 0)
- ptx_bailout "[hostname check] This Script conflicts with host $hostname: CONFLICT FOUND"
- ;;
- *)
- ptx_debug "[hostname check] This Script conflicts with host $hostname: OK"
- ;;
- esac
- done
-}
-
-#
-# convenience wrapper for the functions above
-#
-# performs all available checks
-#
-ptx_dependency_check(){
- ptx_dependency_check_hostname
- ptx_dependency_check_dirs
- ptx_dependency_check_files
- ptx_dependency_check_tools
-}
-
-# service specific lib functions, based on script domains:
-#
-# - dumb_tool
-# - tool
-# - development
-# - system_management
-#
-
-if [ -n "$PTX_SCRIPT_DOMAINS" ]; then
-for ptx_script_domain in $PTX_SCRIPT_DOMAINS; do
- ptx_debug "loading config for domain: ${ptx_script_domain}"
- SCONFS="$HOME/.ptxsd-${ptx_script_domain}.conf \
- /usr/local/etc/ptxsd-${ptx_script_domain}.conf \
- /etc/ptxsd-${ptx_script_domain}.conf \
- `dirname $0`/ptxsd-${ptx_script_domain}.conf \
- `dirname $0`/../lib/ptxsd-${ptx_script_domain}.conf"
- for scriptconf in $SCONFS; do
- if [ -e "$scriptconf" ] ; then
- . $scriptconf && { ptx_debug "$scriptconf loaded"; break; }
- fi
- done
- [ "$PTXSD" = "true" ] || ptx_warning " No valid Configuration for Domain: ${ptx_script_domain} found \
- [ $conf (version $MY_VERSION) ] [ caller: $0 ]"
-done
-fi
diff --git a/plugins/url_check_ng/robotparser2.py.diff b/plugins/url_check_ng/robotparser2.py.diff
deleted file mode 100644
index feb8a77ba09c..000000000000
--- a/plugins/url_check_ng/robotparser2.py.diff
+++ /dev/null
@@ -1,44 +0,0 @@
-linkchecker patch
------------------
-
-The following patch is a quick hack to prevent robots.txt
-restrictions from taking effect. This is somewhat ugly, but
-we need to *know exactly* if a source package is accessible
-or has moved. So we explicitly don't want to honor those
-restrictions. Unfortunately, the --robots-txt option is
-enabled per default and you cannot disable it from the
-command line.
-
-Since this hack is generally a bad thing, please behave
-reasponsible: Don't run url_checks too often (or at
-least ask the mirror maintainers for approval to do so).
-A server maintainer who doesn't want spiders to index
-his machine might have his reasons... ;-)
-
-bbu, 20071019
-
-----------------------------------------------------------------------------
-
---- robotparser2.py.orig 2007-10-19 14:59:06.000000000 +0200
-+++ robotparser2.py 2007-10-19 15:03:28.000000000 +0200
-@@ -344,19 +344,9 @@
- if not isinstance(url, str):
- url = url.encode("ascii", "ignore")
- if self.disallow_all:
-- return False
-+ return True
- if self.allow_all:
- return True
-- # search for given user agent matches
-- # the first match counts
-- url = urllib.quote(urlparse.urlparse(urllib.unquote(url))[2]) or "/"
-- for entry in self.entries:
-- if entry.applies_to(useragent):
-- return entry.allowance(url)
-- # try the default entry last
-- if self.default_entry is not None:
-- return self.default_entry.allowance(url)
-- # agent not found ==> access granted
- return True
-
- def get_crawldelay (self, useragent):
diff --git a/plugins/url_check_ng/robotparser2.py.readme b/plugins/url_check_ng/robotparser2.py.readme
deleted file mode 100644
index a0504219749b..000000000000
--- a/plugins/url_check_ng/robotparser2.py.readme
+++ /dev/null
@@ -1 +0,0 @@
-/usr/share/pycentral/linkchecker/site-packages/linkcheck
--
2.29.2
[-- Attachment #2: Type: text/plain, Size: 181 bytes --]
_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de
To unsubscribe, send a mail with subject "unsubscribe" to ptxdist-request@pengutronix.de
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2021-06-09 20:06 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-09 20:05 [ptxdist] [PATCH] plugins: remove old code Roland Hieber
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox