Re: [ptxdist] [PATCH] nginx: new package

[Michael Olbrich <m.olbrich@pengutronix.de>]

On Sun, Nov 06, 2016 at 08:12:18PM +0100, Clemens Gruber wrote:
> Add nginx package. Heavily inspired by Buildroot's nginx.mk, using their
> revised patches for 1.10.x and the endian patch from Marc Kleine-Budde,
> as well as the nginx.make from the patchset Robert sent, as a starting
> point.

Do you know if anyone has tried to upstream the cross-compile patches?

> But instead of building the modules by default, each module is
> selectable in PTXdist, to allow reducing the binary size.
> 
> Most popular modules are supported, systemd unit file and basic
> configuration files are included as well.
> Supports SSL, HTTP/2, HTTP Proxying, GZIP, Auth Basic, Rewrite, FastCGI,
> SCGI, uWSGI, etc.
> 
> Signed-off-by: Clemens Gruber <clemens.gruber@pqgruber.com>
> ---
>  ...izeof-rework-autotest-to-be-cross-compila.patch |  87 +++++++++
>  ...e-add-mechanism-allowing-to-force-feature.patch | 135 +++++++++++++
>  ...x_feature_run_force_result-for-each-featu.patch | 213 +++++++++++++++++++++
>  ...0004-auto-lib-libxslt-conf-use-pkg-config.patch |  31 +++
>  ...nix-make-sys_nerr-guessing-cross-friendly.patch | 138 +++++++++++++
>  ...0006-auto-lib-openssl-conf-use-pkg-config.patch |  31 +++
>  .../0007-auto-lib-libgd-conf-use-pkg-config.patch  |  31 +++
>  ...-ngx_linux_config.h-only-include-dlfcn.h-.patch |  33 ++++
>  ...ness-add-mechanism-allowing-to-force-resu.patch |  26 +++
>  patches/nginx-1.10.2/series                        |  11 ++
>  projectroot/etc/nginx/mime.types                   |  89 +++++++++

Why did you add this file? nginx installs it into pkgdir and
install_alternative will use that as a fallback.

>  projectroot/etc/nginx/nginx.conf                   |  39 ++++
>  projectroot/lib/systemd/system/nginx.service       |  14 ++
>  projectroot/usr/lib/tmpfiles.d/nginx.conf          |  10 +
>  rules/nginx.in                                     | 170 ++++++++++++++++
>  rules/nginx.make                                   | 209 ++++++++++++++++++++
>  16 files changed, 1267 insertions(+)
>  create mode 100644 patches/nginx-1.10.2/0001-auto-type-sizeof-rework-autotest-to-be-cross-compila.patch
>  create mode 100644 patches/nginx-1.10.2/0002-auto-feature-add-mechanism-allowing-to-force-feature.patch
>  create mode 100644 patches/nginx-1.10.2/0003-auto-set-ngx_feature_run_force_result-for-each-featu.patch
>  create mode 100644 patches/nginx-1.10.2/0004-auto-lib-libxslt-conf-use-pkg-config.patch
>  create mode 100644 patches/nginx-1.10.2/0005-auto-unix-make-sys_nerr-guessing-cross-friendly.patch
>  create mode 100644 patches/nginx-1.10.2/0006-auto-lib-openssl-conf-use-pkg-config.patch
>  create mode 100644 patches/nginx-1.10.2/0007-auto-lib-libgd-conf-use-pkg-config.patch
>  create mode 100644 patches/nginx-1.10.2/0008-src-os-unix-ngx_linux_config.h-only-include-dlfcn.h-.patch
>  create mode 100644 patches/nginx-1.10.2/0100-auto-endianness-add-mechanism-allowing-to-force-resu.patch
>  create mode 100644 patches/nginx-1.10.2/series
>  create mode 100644 projectroot/etc/nginx/mime.types
>  create mode 100644 projectroot/etc/nginx/nginx.conf
>  create mode 100644 projectroot/lib/systemd/system/nginx.service
>  create mode 100644 projectroot/usr/lib/tmpfiles.d/nginx.conf
>  create mode 100644 rules/nginx.in
>  create mode 100644 rules/nginx.make
> 
[...]
> diff --git a/projectroot/etc/nginx/nginx.conf b/projectroot/etc/nginx/nginx.conf
> new file mode 100644
> index 0000000..5801e12
> --- /dev/null
> +++ b/projectroot/etc/nginx/nginx.conf
> @@ -0,0 +1,39 @@
> +worker_processes 1;	# Enough for most webservers and incurs less latency
> +#worker_processes auto;	# Auto-detect based on CPU count
> +
> +events {
> +	use epoll;
> +	worker_connections 1024;
> +	#multi_accept on;	# Should be enabled if worker_processes > 1
> +}
> +
> +http {
> +	server_tokens off;
> +
> +	sendfile on;
> +
> +	include /etc/nginx/mime.types;
> +	default_type application/octet-stream;
> +
> +	access_log off;
> +	error_log stderr;
> +
> +	gzip on;
> +	gzip_disable "msie6";
> +
> +	server {
> +		listen 80;
> +		listen [::]:80;
> +		server_name _;
> +
> +		index index.html;
> +		root /var/www;
> +
> +		location / {
> +			# First attempt to serve request as file, then
> +			# as directory, then fall back to displaying a 404.
> +			try_files $uri $uri/ =404;
> +		}
> +	}
> +}
> +
> diff --git a/projectroot/lib/systemd/system/nginx.service b/projectroot/lib/systemd/system/nginx.service
> new file mode 100644
> index 0000000..a6a1a4d
> --- /dev/null
> +++ b/projectroot/lib/systemd/system/nginx.service
> @@ -0,0 +1,14 @@
> +[Unit]
> +Description=NGINX HTTP and reverse proxy server
> +After=syslog.target network.target nss-lookup.target
> +
> +[Service]
> +Type=forking
> +PIDFile=/run/nginx.pid
> +ExecStartPre=/usr/sbin/nginx -t
> +ExecStart=/usr/sbin/nginx
> +ExecReload=/bin/kill -s HUP $MAINPID
> +ExecStop=/bin/kill -s QUIT $MAINPID
> +
> +[Install]
> +WantedBy=multi-user.target
> diff --git a/projectroot/usr/lib/tmpfiles.d/nginx.conf b/projectroot/usr/lib/tmpfiles.d/nginx.conf
> new file mode 100644
> index 0000000..138b675
> --- /dev/null
> +++ b/projectroot/usr/lib/tmpfiles.d/nginx.conf
> @@ -0,0 +1,10 @@
> +# systemd tmpfile settings for nginx
> +# See tmpfiles.d(5) for details
> +
> +d /var/log/nginx 0755 root root -
> +d /var/tmp/nginx 0755 www www
> +d /var/tmp/nginx/client 0755 www www
> +d /var/tmp/nginx/proxy 0755 www www
> +d /var/tmp/nginx/fastcgi 0755 www www
> +d /var/tmp/nginx/scgi 0755 www www
> +d /var/tmp/nginx/uwsgi 0755 www www
> diff --git a/rules/nginx.in b/rules/nginx.in
> new file mode 100644
> index 0000000..00aa202
> --- /dev/null
> +++ b/rules/nginx.in
> @@ -0,0 +1,170 @@
> +## SECTION=networking
> +menuconfig NGINX
> +	tristate
> +	prompt "nginx                         "
> +	select GCCLIBS_GCC_S
> +	select LIBC_DL
> +	select LIBC_PTHREAD
> +	select LIBC_CRYPT
> +	select LIBPCRE			if NGINX_PCRE
> +	select OPENSSL			if NGINX_OPENSSL
> +	select ZLIB			if NGINX_HTTP_GZIP_MODULE || NGINX_HTTP_GZIP_STATIC_MODULE
> +	help
> +	  nginx ("engine X") is a high-performance web and reverse
> +	  proxy server created by Igor Sysoev. It can be used both as
> +	  a standalone web server and as a proxy to reduce the load on
> +	  backend servers.
> +
> +if NGINX
> +
> +menu "Base settings                 "
> +
> +config NGINX_OPENSSL
> +	bool
> +	prompt "OpenSSL support"
> +	help
> +	  Include OpenSSL support, which is required for the
> +	  ngx_http_ssl_module.
> +
> +config NGINX_PCRE
> +	bool
> +	select LIBPCRE

remove.

> +	prompt "PCRE support"
> +	help
> +	  Include pcre support, which is required for regular
> +	  expressions support in the location directive and for the
> +	  ngx_http_rewrite_module module.
> +
> +if NGINX_PCRE
> +
> +config NGINX_PCRE_JIT
> +	bool
> +	prompt "Enable PCRE JIT"
> +	help
> +	  Enables the use of just-in-time compilation for regular
> +	  expressions known by the time of configuration parsing.
> +	  PCRE JIT can significantly speed up processing of regular
> +	  expressions.
> +
> +endif
> +
> +config NGINX_THREADS
> +	bool
> +	prompt "Threads support"
> +	help
> +	  Enable multi-threading. Files can be read and sent without
> +	  blocking a worker process. Requires epoll as connection
> +	  processing method.
> +
> +endmenu
> +
> +menu "Modules                       "
> +
> +config NGINX_HTTP_ACCESS_MODULE
> +	bool
> +	prompt "ngx_http_access_module"
> +	help
> +	  ngx_http_access_module allows limiting access to certain
> +	  client addresses.
> +
> +config NGINX_HTTP_AUTH_BASIC_MODULE
> +	bool
> +	prompt "ngx_http_auth_basic_module"
> +	help
> +	  ngx_http_auth_basic_module allows limiting access to
> +	  resources by validating the user name and password using the
> +	  HTTP Basic Authentication protocol.
> +
> +config NGINX_HTTP_AUTOINDEX_MODULE
> +	bool
> +	prompt "ngx_http_autoindex_module"
> +	help
> +	  ngx_http_autoindex_module processes requests ending with the
> +	  slash character and produces a directory listing.
> +
> +config NGINX_HTTP_FASTCGI_MODULE
> +	bool
> +	prompt "ngx_http_fastcgi_module"
> +	help
> +	  ngx_http_fastcgi_module allows passing requests to a FastCGI
> +	  server.
> +
> +config NGINX_HTTP_GZIP_MODULE
> +	bool
> +	prompt "ngx_http_gzip_module"
> +	help
> +	  ngx_http_gzip_module is a filter that compresses responses
> +	  using gzip.
> +
> +config NGINX_HTTP_GZIP_STATIC_MODULE
> +	bool
> +	prompt "ngx_http_gzip_static_module"
> +	help
> +	  ngx_http_gzip_static_module allows sending precompressed files
> +	  with the .gz filename extension instead of regular files.
> +
> +config NGINX_HTTP_PROXY_MODULE
> +	bool
> +	prompt "ngx_http_proxy_module"
> +	help
> +	  ngx_http_proxy_module allows passing requests to another
> +	  server.
> +
> +config NGINX_HTTP_REWRITE_MODULE
> +	bool
> +	select NGINX_PCRE
> +	prompt "ngx_http_rewrite_module"
> +	help
> +	  ngx_http_rewrite_module module is used to change request URI
> +	  using PCRE regular expressions, return redirects, and
> +	  conditionally select configurations.
> +
> +config NGINX_HTTP_SCGI_MODULE
> +	bool
> +	prompt "ngx_http_scgi_module"
> +	help
> +	  ngx_http_scgi_module allows passing requests to a SCGI
> +	  server.
> +
> +config NGINX_HTTP_SSL_MODULE
> +	bool
> +	select NGINX_OPENSSL
> +	prompt "ngx_http_ssl_module"
> +	help
> +	  ngx_http_ssl_module provides the necessary support for HTTPS.
> +
> +config NGINX_HTTP_UWSGI_MODULE
> +	bool
> +	prompt "ngx_http_uwsgi_module"
> +	help
> +	  ngx_http_uwsgi_module allows passing requests to a uWSGI
> +	  server.
> +
> +config NGINX_HTTP_V2_MODULE
> +	bool
> +	select NGINX_OPENSSL
> +	prompt "ngx_http_v2_module"
> +	help
> +	  ngx_http_v2_module provides support for HTTP/2.
> +
> +endmenu
> +
> +menu "Startup                       "
> +
> +config NGINX_SYSTEMD_UNIT
> +	bool
> +	default y
> +	depends on SYSTEMD
> +	prompt "Install systemd unit files for nginx"
> +
> +endmenu
> +
> +config NGINX_POPULATE_TEST_WEBSITE
> +	bool
> +	default y
> +	prompt "Populate a generic test website"
> +	help
> +	  This install a generic website into /var/www/index.html
> +	  from <ptxdist-install>/projectroot/var/www/httpd.html.
> +
> +endif
> diff --git a/rules/nginx.make b/rules/nginx.make
> new file mode 100644
> index 0000000..4cc0dd2
> --- /dev/null
> +++ b/rules/nginx.make
> @@ -0,0 +1,209 @@
> +# -*-makefile-*-
> +#
> +# Copyright (C) 2015 by Marc Kleine-Budde <mkl@pengutronix.de>
> +# Copyright (C) 2016 by Clemens Gruber <clemens.gruber@pqgruber.com>
> +#
> +# See CREDITS for details about who has contributed to this project.
> +#
> +# For further information about the PTXdist project and license conditions
> +# see the README file.
> +#
> +
> +#
> +# We provide this package
> +#
> +PACKAGES-$(PTXCONF_NGINX) += nginx
> +
> +#
> +# Paths and names
> +#
> +NGINX_VERSION	:= 1.10.2
> +NGINX_MD5	:= e8f5f4beed041e63eb97f9f4f55f3085
> +NGINX		:= nginx-$(NGINX_VERSION)
> +NGINX_SUFFIX	:= tar.gz
> +NGINX_URL	:= https://nginx.org/download/$(NGINX).$(NGINX_SUFFIX)
> +NGINX_SOURCE	:= $(SRCDIR)/$(NGINX).$(NGINX_SUFFIX)
> +NGINX_DIR	:= $(BUILDDIR)/$(NGINX)
> +NGINX_LICENSE	:= BSD-2-Clause
> +
> +# ----------------------------------------------------------------------------
> +# Prepare
> +# ----------------------------------------------------------------------------
> +
> +NGINX_CONF_ENV := \
> +	ngx_force_c_compiler=yes \
> +	ngx_force_c99_have_variadic_macros=yes \
> +	ngx_force_gcc_have_variadic_macros=yes \
> +	ngx_force_gcc_have_atomic=yes \
> +	ngx_force_have_libatomic=no \
> +	ngx_force_have_epoll=yes \
> +	ngx_force_have_sendfile=yes \
> +	ngx_force_have_sendfile64=yes \
> +	ngx_force_have_pr_set_dumpable=yes \
> +	ngx_force_have_timer_event=yes \
> +	ngx_force_have_map_anon=yes \
> +	ngx_force_have_map_devzero=yes \
> +	ngx_force_have_sysvshm=yes \
> +	ngx_force_have_posix_sem=yes
> +
> +ifdef PTXCONF_ENDIAN_LITTLE
> +NGINX_CONF_ENV += ngx_force_have_little_endian=yes
> +endif
> +
> +NGINX_CONF_TOOL := autoconf
> +NGINX_CONF_OPT := \
> +	--crossbuild=Linux::$(PTXCONF_ARCH_STRING) \
> +	--with-cc=$(CROSS_CC) \
> +	--with-cpp=$(CROSS_CC) \
> +	--with-cc-opt=-O2 \
> +	--prefix=/usr/share/nginx \
> +	--conf-path=/etc/nginx/nginx.conf \
> +	--sbin-path=/usr/sbin/nginx \
> +	--pid-path=/run/nginx.pid \
> +	--lock-path=/var/lock/nginx.lock \
> +	--user=www \
> +	--group=www \
> +	--http-log-path=off \
> +	--error-log-path=stderr \
> +	--http-client-body-temp-path=/var/tmp/nginx/client-body \
> +	--http-proxy-temp-path=/var/tmp/nginx/proxy \
> +	--http-fastcgi-temp-path=/var/tmp/nginx/fastcgi \
> +	--http-scgi-temp-path=/var/tmp/nginx/scgi \
> +	--http-uwsgi-temp-path=/var/tmp/nginx/uwsgi \
> +	--$(call ptx/wwo, PTXCONF_NGINX_PCRE)-pcre \
> +	--without-http_browser_module \
> +	--without-http_charset_module \
> +	--without-http_empty_gif_module \
> +	--without-http_geo_module \
> +	--without-http_limit_conn_module \
> +	--without-http_limit_req_module \
> +	--without-http_map_module \
> +	--without-http_memcached_module \
> +	--without-http_referer_module \
> +	--without-http_split_clients_module \
> +	--without-http_ssi_module \
> +	--without-http_upstream_hash_module \
> +	--without-http_upstream_ip_hash_module \
> +	--without-http_upstream_keepalive_module \
> +	--without-http_upstream_least_conn_module \
> +	--without-http_upstream_zone_module \
> +	--without-http_userid_module \
> +	--without-mail_imap_module \
> +	--without-mail_pop3_module \
> +	--without-mail_smtp_module \
> +	--without-stream_access_module \
> +	--without-stream_limit_conn_module \
> +	--without-stream_upstream_hash_module \
> +	--without-stream_upstream_least_conn_module \
> +	--without-stream_upstream_zone_module
> +
> +# Note: Settings and module options are *not* symmetric.
> +#       If a module is on by default, a without option exists.
> +#       If it is off by default, a with option exists.
> +
> +# Opt-in settings
> +

NGINX_CONF_OPT += \
	$(call ptx/ifdef, PTXCONF_*,--with-*) \
	[...]

For each Opt-in/Opt-out etc. group. Or just one block with all modules. Or
make list + foreach magic like I handled the gstreamer plugins. Either way
is fine, I just want this to be more compact.

> +ifdef PTXCONF_GLOBAL_IPV6
> +NGINX_CONF_OPT += --with-ipv6
> +endif
> +
> +ifdef PTXCONF_NGINX_PCRE_JIT
> +NGINX_CONF_OPT += --with-pcre-jit
> +endif
> +
> +ifdef PTXCONF_NGINX_THREADS
> +NGINX_CONF_OPT += --with-threads
> +endif
> +
> +# Opt-in modules
> +
> +ifdef PTXCONF_NGINX_HTTP_GZIP_STATIC_MODULE
> +NGINX_CONF_OPT += --with-http_gzip_static_module
> +endif
> +
> +ifdef PTXCONF_NGINX_HTTP_SSL_MODULE
> +NGINX_CONF_OPT += --with-http_ssl_module
> +endif
> +
> +ifdef PTXCONF_NGINX_HTTP_V2_MODULE
> +NGINX_CONF_OPT += --with-http_v2_module
> +endif
> +
> +# Opt-out modules
> +
> +ifndef PTXCONF_NGINX_HTTP_ACCESS_MODULE
> +NGINX_CONF_OPT += --without-http_access_module
> +endif
> +
> +ifndef PTXCONF_NGINX_HTTP_AUTH_BASIC_MODULE
> +NGINX_CONF_OPT += --without-http_auth_basic_module
> +endif
> +
> +ifndef PTXCONF_NGINX_HTTP_AUTOINDEX_MODULE
> +NGINX_CONF_OPT += --without-http_autoindex_module
> +endif
> +
> +ifndef PTXCONF_NGINX_HTTP_FASTCGI_MODULE
> +NGINX_CONF_OPT += --without-http_fastcgi_module
> +endif
> +
> +ifndef PTXCONF_NGINX_HTTP_GZIP_MODULE
> +NGINX_CONF_OPT += --without-http_gzip_module
> +endif
> +
> +ifndef PTXCONF_NGINX_HTTP_PROXY_MODULE
> +NGINX_CONF_OPT += --without-http_proxy_module
> +endif
> +
> +ifndef PTXCONF_NGINX_HTTP_REWRITE_MODULE
> +NGINX_CONF_OPT += --without-http_rewrite_module
> +endif
> +
> +ifndef PTXCONF_NGINX_HTTP_SCGI_MODULE
> +NGINX_CONF_OPT += --without-http_scgi_module
> +endif
> +
> +ifndef PTXCONF_NGINX_HTTP_UWSGI_MODULE
> +NGINX_CONF_OPT += --without-http_uwsgi_module
> +endif
> +
> +
> +# ----------------------------------------------------------------------------
> +# Target-Install
> +# ----------------------------------------------------------------------------
> +
> +$(STATEDIR)/nginx.targetinstall:
> +	@$(call targetinfo)
> +
> +	@$(call install_init, nginx)
> +	@$(call install_fixup, nginx,PRIORITY,optional)
> +	@$(call install_fixup, nginx,SECTION,base)
> +	@$(call install_fixup, nginx,AUTHOR,"Clemens Gruber <clemens.gruber@pqgruber.com>")
> +	@$(call install_fixup, nginx,DESCRIPTION,missing)
> +
> +	@$(call install_copy, nginx, 0, 0, 0755, -, /usr/sbin/nginx)
> +
> +	@$(call install_alternative, nginx, 0, 0, 0644, /etc/nginx/nginx.conf)
> +	@$(call install_alternative, nginx, 0, 0, 0644, /etc/nginx/mime.types)
> +
> +ifdef PTXCONF_NGINX_POPULATE_TEST_WEBSITE
> +	@$(call install_copy, nginx, 12, 102, 0644, \

use user/group names, not numbers.

> +		$(PTXDIST_TOPDIR)/projectroot/var/www/httpd.html, \
> +		/var/www/index.html)

Use install_alternative here (untested):

	@$(call install_alternatice, nginx, www, www, 0644, \
		/projectroot/var/www/httpd.html, n, \
		/var/www/index.html)

> +endif
> +
> +ifdef PTXCONF_NGINX_SYSTEMD_UNIT
> +	@$(call install_alternative, nginx, 0, 0, 0644, /lib/systemd/system/nginx.service)
> +	@$(call install_link, nginx, ../nginx.service, \
> +		/lib/systemd/system/multi-user.target.wants/nginx.service)
> +endif
> +
> +ifdef PTXCONF_SYSTEMD

No, never use an option from a different package directly. Either install
this unconditionally (I'd prefer that) or add an options without prompt
that defaults to SYSTEMD.

Michael

> +	@$(call install_alternative, nginx, 0, 0, 0644, /usr/lib/tmpfiles.d/nginx.conf)
> +endif
> +
> +	@$(call install_finish, nginx)
> +
> +	@$(call touch)
> +
> +# vim: syntax=make
> -- 
> 2.10.2
> 
> 
> _______________________________________________
> ptxdist mailing list
> ptxdist@pengutronix.de

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
ptxdist mailing list
ptxdist@pengutronix.de