#903 Draft: lighttpd infra simplification
Closed 2 years ago by praiskup. Opened 2 years ago by gstrauss.
fedora-infra/ gstrauss/ansible lighttpd-infra-updates  into  main

lighttpd dirlisting backlink
Glenn Strauss • 2 years ago  
lighttpd infra simplification
Glenn Strauss • 2 years ago  
@@ -1,33 +0,0 @@ 

- #! /bin/bash

- 

- # Something like 'cat' fucntionality, but automatically re-open the output file

- # upon the SIGHUP signal.  We used to use cronolog instead in Copr, but (a) we

- # haven't used it for rotating at all, and (b) that tool doesn't react on

- # SIGHUP.  Long story in: https://pagure.io/copr/copr/issue/2001

- #

- # One may object this is slow, I tested this locally with throughput about

- # 12k lines per second:

- #

- #   $ for i in `seq 1000000`; do printf "%79d\n" "0" >> /tmp/test-file; done

- #   $ time cat /tmp/test-file | /tmp/logger /tmp/output-measured

- #   real    1m24.354s

- #   user    0m39.895s

- #   sys     1m6.402s

- #

- # But we would get much higher throughput if implemented in C.

- 

- logfile=$1

- cmd="$0 $*"

- 

- handler()

- {

-   exec 1>> "$logfile"

-   echo "=== start: $cmd ==="

- }

- 

- trap handler SIGHUP

- handler

- 

- while IFS= read -r line; do

-     echo "$line"

- done

@@ -1,15 +0,0 @@ 

- #! /bin/bash

- 

- main_pid=$(systemctl show --property MainPID --value lighttpd)

- for child in $(cat /proc/$main_pid/task/$main_pid/children); do

-     cmd_to_kill=$(readlink -f "/proc/$child/exe")

-     case $cmd_to_kill in

-         *bash)

-             echo "sending SIGHUP to PID $child as that's bash"

-             kill -HUP "$child"

-             ;;

-         *)

-             echo "Not sending to $cmd_to_kill"

-             ;;

-     esac

- done

@@ -5,5 +5,6 @@ 

  dir-listing.hide-header-file = "disable"

  dir-listing.show-header = "disable"

  dir-listing.hide-readme-file = "disable"

- dir-listing.show-readme = "disable"

- 

+ dir-listing.show-readme = "/etc/lighttpd/dirlisting.inc"

+ dir-listing.encode-readme = "disable"

+ dir-listing.cache = ( "path" => cache_dir, "max-age" => 15 )

@praiksup how much of the server load is due to directory listing?
How frequently are the directory listings requested?
How frequently are directories changing?
Is 15 seconds reasonable for caching a directory listing?
Is 60 seconds reasonable?
How large are these directories?
If only a few files, caching might only marginally improve performance.
If directories do not change frequently, or if they change once to compress logs,
then generating a static index.html when the directory contents are compressed,
and then allowing the index.html to be cached, is probably the most performant
solution.

I think that the configuration - as you prpopose - is fine. Lightttpd just doesn't
dload the inc file (deployed here https://copr-be-dev.cloud.fedoraproject.org/results/%40rocknsm/rocknsm-2.1/).

I think that the configuration - as you prpopose - is fine. Lightttpd just doesn't
dload the inc file (deployed here https://copr-be-dev.cloud.fedoraproject.org/results/%40rocknsm/rocknsm-2.1/).

I see that the link is not included. You might strace lighttpd to see if it is trying to open the file after generating the
dirlisting. You might also check lighttpd -f /etc/lighttpd/lighttpd.conf -p and look at the dir-listing.* options to
check that it is configured. Of course, also please also check that lighttpd has been properly restarted after any
config change.

As an aside, I poked around and saw .xml.gz files there. The lua code in this patch handling .log.gz
could be extended to handle more types. Back in 2017, I wrote a more pedantically correct bit of
lua to parse client Accept-Encoding and Accept-Language and to then choose a resource to send.
https://wiki.lighttpd.net/AbsoLUAtion#Content-Negotiation

doh! Sorry. You're running lighttpd 1.4.61. Latest lighttpd release is lighttpd 1.4.63. There was a bug in lighttpd 1.4.60 and lighttpd 1.4.61 which did not show the readme. https://git.lighttpd.net/lighttpd/lighttpd1.4/commit/cba6a1ab54f063609922ea0085d6ba15ed77fa6f

Answering your questions isn't easy, as it depends on the concrete directory,
it's not homogeneous load...
Though the caches are potentially good against DoS attempts at least?

Also, most of the traffic to lighty goes through CloudFronts so it's cached
there. And the load is pretty low nowadays, I would say.

I mean load forwarded to lighttpd is pretty low.

I think I answered my own question. During a build, the logs are being modified, so the caching interval should be brief, or disabled. If not too many people are watching live builds and nothing is polling in a tight loop, then there is little point to enabling the mod_dirlisting cache. That is true as long as once the build completes, an index.html is generated with the directory listing, so that the overhead of generating the directory listing never occurs again for that (completed) build.

lighttpd is not returning HTTP Cache-Control in the response headers, so I am unsure how your are controlling the caching at CloudFront. Have you disabled the CloudFront content re-validation? Once an index.html is generated in a directory, then we should allow that to be cached. A short lua script could be substituted for mod_indexfile, and could add appropriate Cache-Control to the response if the file was served from index.html, or if index.html was not present, allow mod_dirlisting to generate the response.

I think we could request an update of Lighttpd in Fedora 35... if going with JS and not Lua.

CloudFronts caches everything, and yes - it really leads to user confusion (people are asking us what happened, that they don't see the build finished, etc.). So I'm sure the directory listing should be controlled at least till we finish the build.

@@ -45,14 +45,14 @@ 

    ".html"         =>      "text/html",

    ".htm"          =>      "text/html",

    ".js"           =>      "text/javascript",

-   ".asc"          =>      "text/plain",

-   ".c"            =>      "text/plain",

-   ".cpp"          =>      "text/plain",

-   ".log"          =>      "text/plain",

-   ".conf"         =>      "text/plain",

-   ".text"         =>      "text/plain",

-   ".txt"          =>      "text/plain",

-   ".spec"         =>      "text/plain",

+   ".asc"          =>      "text/plain; charset=utf-8",

+   ".c"            =>      "text/plain; charset=utf-8",

+   ".cpp"          =>      "text/plain; charset=utf-8",

+   ".log"          =>      "text/plain; charset=utf-8",

+   ".conf"         =>      "text/plain; charset=utf-8",

+   ".text"         =>      "text/plain; charset=utf-8",

+   ".txt"          =>      "text/plain; charset=utf-8",

+   ".spec"         =>      "text/plain; charset=utf-8",

    ".dtd"          =>      "text/xml",

    ".xml"          =>      "text/xml",

    ".mpeg"         =>      "video/mpeg",
@@ -68,7 +68,7 @@ 

    ".tar.bz2"      =>      "application/x-bzip-compressed-tar",

    ".rpm"          =>      "application/x-rpm",

    # make the default mime type application/octet-stream.

-   ""              =>      "text/plain",

+   ""              =>      "text/plain; charset=utf-8",

roles/copr/backend/files/lighttpd/mime.conf was created in 2014 and the significant change from default /etc/lighttpd/conf.d/mime.conf is the default Content-Type "" => "text/plain", instead of "application/octet-stream". Is this still relevant for downloads? Are there a smaller set of extensions which should have "text/plain" forced or a smaller portion of the site for which this should apply? It is generally preferable to use /etc/lighttpd/conf.d/mime.conf unless there is a specific reason not to do so.

We should go with the default config, yes.

If the mime.conf is removed from this ansible package, how will the original from lighttpd-filesystem be restored? It is a config file, so it won't be overwritten by lighttpd package upgrades, will it?

This is easier to do manually:, remove the config file and dnf reinstall lighttpd. Or we can wait till we migrate to a new version of Fedora (year or so) - then we install from scratch.

It seems like a safer solution is to include /etc/lighttpd/conf.d/mime.conf from lighttpd-filesystem into this repository so that it overwrites existing installations. Then, some time later, after a round of deployment through dev, staging, and prod, then mime.conf can be removed from this package.

   )

  

  

@@ -9,7 +9,9 @@ 

          mode={{ item.mode | default('0600') }}

    with_items:

    - file: copr-be.cloud.fedoraproject.org.key

+     mode: "0640"

    - file: copr-be.cloud.fedoraproject.org.cert

+     mode: "0640"

    - file: copr-be.cloud.fedoraproject.org.pem

      mode: "0640"

    - file: copr-be.cloud.fedoraproject.org.intermediate.cert
@@ -42,6 +44,8 @@ 

    loop: "{{ files|product(perms)|list }}"

    vars:

      files:

+       - "/etc/lighttpd/copr-be.cloud.fedoraproject.org.key"

+       - "/etc/lighttpd/copr-be.cloud.fedoraproject.org.cert"

        - "/etc/lighttpd/copr-be.cloud.fedoraproject.org.pem"

        - "/etc/lighttpd/copr-be.cloud.fedoraproject.org.intermediate.cert"

      perms:

@@ -105,8 +105,11 @@ 

  

  - name: add gzip content-encoding header by lua script

    template: src="lighttpd/content-encoding-gzip-if-exists.lua" dest=/etc/lighttpd/content-encoding-gzip-if-exists.lua owner=root group=root mode=0644

-   notify:

-   - restart lighttpd

+   tags:

+   - config

+ 

+ - name: add dirlisting include

+   template: src="lighttpd/dirlisting.inc.j2" dest=/etc/lighttpd/dirlisting.inc owner=root group=root mode=0644

    tags:

    - config

  
@@ -142,27 +145,6 @@ 

    tags:

    - config

  

- - name: install custom lighttpd template for directory listings

-   template: src="lighttpd/dir-generator.php.j2" dest="/var/lib/copr/public_html/dir-generator.php" owner=copr group=copr mode=0755

- 

- - name: install the helper scripts for lighttpd log rotation

-   copy:

-     src: "{{ item }}"

-     dest: /usr/local/bin/{{ item }}

-     mode: 0755

-     owner: root

-     group: root

-   with_items:

-     - copr-lighty-logger

-     - copr-lighty-reopen-logs

-   register: logrotate_scripts

-   tags: logrotate

- 

- - name: fix selinux context on helper scripts

-   command: restorecon -irv /usr/local/bin/copr-lighty*

-   when: logrotate_scripts.changed

-   tags: logrotate

- 

  - name: install custom logrotate config for lighttpd

    template: src="logrotate/lighttpd.j2" dest=/etc/logrotate.d/lighttpd owner=root group=root mode=644

    tags: logrotate

@@ -1,3 +1,28 @@ 

- if (lighty.stat(lighty.env["physical.path"])) then

-     lighty.header["Content-Encoding"] = "gzip"

+ --

+ -- content negotiation for .log.gz or else .log

+ --

+ local r = lighty.r

+ local path = r.req_attr["physical.path"]

+ if (string.match(path, "%.log$")) then

+   if (lighty.c.stat(path .. ".gz")) then

+     -- .log.gz is available; NB: assumes client sends Accept-Encoding: gzip

+     r.resp_header["Content-Encoding"] = "gzip"

+     r.req_attr["physical.path"] = path = path .. ".gz"

+   elseif (lighty.c.stat(path)) then

+     -- log file may still be in process of being written

+     -- https://pagure.io/copr/copr/issue/762

+     r.resp_header["Cache-Control"] = "no-cache"

+   else

+     -- not found

+     return 0

+   end

+ elseif (string.match(path, "%.log.gz$") and lighty.c.stat(path)) then

+   -- .log.gz is available; NB: assumes client sends Accept-Encoding: gzip

+   r.resp_header["Content-Encoding"] = "gzip"

+ else

+   -- not .log or .log.gz, or not found

+   return 0

  end

+ r.resp_header["Content-Type"] = "text/plain; charset=utf-8"

+ r.resp_body:set({ { filename = path } })

+ return 200

@@ -1,449 +0,0 @@ 

- <?php

- 

- $VERSION = "0.3";

- 

- /*  Lighttpd Enhanced Directory Listing Script

-  *  ------------------------------------------

-  *  Author: Evan Fosmark

-  *  Version: 2008.08.07

-  *

-  *

-  *  GNU License Agreement

-  *  ---------------------

-  *  This program is free software; you can redistribute it and/or modify

-  *  it under the terms of the GNU General Public License as published by

-  *  the Free Software Foundation; either version 2 of the License, or

-  *  (at your option) any later version.

-  *

-  *  This program is distributed in the hope that it will be useful,

-  *  but WITHOUT ANY WARRANTY; without even the implied warranty of

-  *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

-  *  GNU General Public License for more details.

-  *

-  *  You should have received a copy of the GNU General Public License

-  *  along with this program; if not, write to the Free Software

-  *  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA

-  *

-  *  http://www.gnu.org/licenses/gpl.txt

-  */

- 

- /*  Revision by KittyKatt

-  *  ---------------------

-  *  E-Mail:  kittykatt@archlinux.us

-  *  Website: http://www.archerseven.com/kittykatt/

-  *  Version:  2010.03.01

-  *

-  *  Revised original code to include hiding for directories prefixed with a "." (or hidden

-  *  directories) as the script was only hiding files prefixed with a "." before. Also included more

-  *  file extensions/definitions.

-  *

-  */

- 

- $show_hidden_files = true;

- $calculate_folder_size = false;

- 

- // Various file type associations

- $movie_types = array('mpg','mpeg','avi','asf','mp3','wav','mp4','wma','aif','aiff','ram', 'midi','mid','asf','au','flac');

- $image_types = array('jpg','jpeg','gif','png','tif','tiff','bmp','ico');

- $archive_types = array('zip','cab','7z','gz','tar.bz2','tar.gz','tar','rar',);

- $document_types = array('txt','text','doc','docx','abw','odt','pdf','rtf','tex','texinfo',);

- $font_types = array('ttf','otf','abf','afm','bdf','bmf','fnt','fon','mgf','pcf','ttc','tfm','snf','sfd');

- 

- 

- // Get the path (cut out the query string from the request_uri)

- list($path) = explode('?', $_SERVER['REQUEST_URI']);

- 

- 

- // Get the path that we're supposed to show.

- $path = ltrim(rawurldecode($path), '/');

- 

- 

- if(strlen($path) == 0) {

- 	$path = "./";

- }

- 

- 

- // Can't call the script directly since REQUEST_URI won't be a directory

- if($_SERVER['PHP_SELF'] == '/'.$path) {

- 	die("Unable to call " . $path . " directly.");

- }

- 

- 

- // Make sure it is valid.

- if(!is_dir($path)) {

- 	die("<b>" . $path . "</b> is not a valid path.");

- }

- 

- 

- //

- // Get the size in bytes of a folder

- //

- function foldersize($path) {

- 	$size = 0;

- 	if($handle = @opendir($path)){

- 		while(($file = readdir($handle)) !== false) {

- 			if(is_file($path."/".$file)){

- 				$size += filesize($path."/".$file);

- 			}

- 

- 			if(is_dir($path."/".$file)){

- 				if($file != "." && $file != "..") {

- 					$size += foldersize($path."/".$file);

- 				}

- 			}

- 		}

- 	}

- 

- 	return $size;

- }

- 

- 

- //

- // This function returns the file size of a specified $file.

- //

- function format_bytes($size, $precision=0) {

-     $sizes = array('B', 'B', 'E', 'P', 'B', 'G', 'M', 'K', 'B');

-     $total = count($sizes);

- 

-     while($total-- && $size > 1024) $size /= 1024;

-     return sprintf('%.'.$precision.'f', $size).$sizes[$total];

- }

- 

- 

- //

- // This function returns the mime type of $file.

- //

- function get_file_type($file) {

- 	global $image_types, $movie_types;

- 

- 	$pos = strrpos($file, ".");

- 	if ($pos === false) {

- 		return "Text File";

- 	}

- 

- 	$ext = rtrim(substr($file, $pos+1), "~");

- 	if(in_array($ext, $image_types)) {

- 		$type = "Image File";

- 

- 	} elseif(in_array($ext, $movie_types)) {

- 		$type = "Video File";

- 

- 	} elseif(in_array($ext, $archive_types)) {

- 		$type = "Compressed Archive";

- 

- 	} elseif(in_array($ext, $document_types)) {

- 		$type = "Type Document";

- 

- 	} elseif(in_array($ext, $font_types)) {

- 		$type = "Type Font";

- 

- 	} else {

- 		$type = "File";

- 	}

- 

- 	return(strtoupper($ext) . " " . $type);

- }

- 

- 

- 

- // Print the heading stuff

- $vpath = ($path != "./")?$path:"";

- print "<!DOCTYPE html>

- 	<head>

- 		<title>Index of /" .$vpath. "</title>

- 		<style type='text/css'>

- 		a, a:active {text-decoration: none; color: blue;}

- 		a:visited {color: #48468F;}

- 		a:hover, a:focus {text-decoration: underline; color: red;}

- 		body {background-color: #F5F5F5;}

- 		h2 {margin-bottom: 12px;}

- 		table {margin-left: 12px;}

- 		th, td { font-family: monospace; font-size: 9pt; text-align: left;}

- 		th { font-weight: bold; padding-right: 14px; padding-bottom: 3px;}

- 		td {padding-right: 14px;}

- 		td.s, th.s {text-align: right;}

- 		div.list { background-color: white; border-top: 1px solid #646464; border-bottom: 1px solid #646464; padding-top: 10px; padding-bottom: 14px;}

- 		div.foot, div.script_title { font-family: 'Courier New', Courier, monospace; font-size: 10pt; color: #787878; padding-top: 4px;}

- 		div.script_title {float:right;text-align:right;font-size:8pt;color:#999;}

- 		</style>

- 	</head>

- 	<body>

- 	<h2>Index of /" . $vpath ."</h2>

- 	<div class='list'>

- 	<table summary='Directory Listing' cellpadding='0' cellspacing='0'>";

- 

- 

- 

- 

- // Get all of the folders and files.

- $folderlist = array();

- $filelist = array();

- if($handle = @opendir($path)) {

- 	while(($item = readdir($handle)) !== false) {

- 		if(is_dir($path.'/'.$item) and $item != '.' and $item != '..') {

- 			if( $show_hidden_files == "false" ) {

- 				if(substr($item, 0, 1) == "." or substr($item, -1) == "~") {

- 				  continue;

- 				}

- 			}

- 			$folderlist[] = array(

- 				'name' => $item,

- 				'size' => (($calculate_folder_size)?foldersize($path.'/'.$item):0),

- 				'modtime'=> filemtime($path.'/'.$item),

- 				'file_type' => "Directory"

- 			);

- 		}

- 

- 		elseif(is_file($path.'/'.$item)) {

- 			if( $show_hidden_files == "false" ) {

- 				if(substr($item, 0, 1) == "." or substr($item, -1) == "~") {

- 				  continue;

- 				}

- 			}

- 			$filelist[] = array(

- 				'name'=> $item,

- 				'size'=> filesize($path.'/'.$item),

- 				'modtime'=> filemtime($path.'/'.$item),

- 				'file_type' => get_file_type($path.'/'.$item)

- 			);

- 		}

- 	}

- 	fclose($handle);

- }

- 

- // Show sort methods

- print "<thead><tr>";

- 

- $header = array();

- $header['name'] = "Name";

- $header['modtime'] = "Last Modified";

- $header['size'] = "Size";

- $header['file_type'] = "Type";

- 

- foreach($header as $key=>$item) {

- 	if($_GET['sort'] == $key) {

- 		print "<th class='n'>".$item."</th>";

- 	} else {

- 		print "<th class='n'>".$item."</th>";

- 	}

- }

- print "</tr></thead><tbody>";

- 

- 

- 

- // Parent directory link

- if($path != "./") {

- 	print "<tr><td class='n'><a href='..'>..</a>/</td>";

- 	print "<td class='m'>&nbsp;</td>";

- 	print "<td class='s'>&nbsp;</td>";

- 	print "<td class='t'>Directory</td></tr>";

- }

- 

- 

- 

- // Print folder information

- foreach($folderlist as $folder) {

- 	print "<tr><td class='n'><a href='" . addslashes($folder['name']). "'>" .htmlentities($folder['name']). "</a>/</td>";

- 	print "<td class='m'>" . date('Y-M-d H:i:s', $folder['modtime']) . "</td>";

- 	print "<td class='s'>" . (($calculate_folder_size)?format_bytes($folder['size'], 2):'--') . "&nbsp;</td>";

- 	print "<td class='t'>" . $folder['file_type']                    . "</td></tr>";

- }

- 

- 

- 

- 

- // Print file information

- foreach($filelist as $file) {

- 	print "<tr><td class='n'><a href='" . addslashes($file['name']). "'>" .htmlentities($file['name']). "</a></td>";

- 	print "<td class='m'>" . date('Y-M-d H:i:s', $file['modtime'])   . "</td>";

- 	print "<td class='s'>" . format_bytes($file['size'],2)           . "&nbsp;</td>";

- 	print "<td class='t'>" . $file['file_type']                      . "</td></tr>";

- }

- 

- 

- $frontend_baseurl = '{{ frontend_base_url }}';

- $path = explode('/', $_SERVER['REQUEST_URI']);

- if (count($path) >= 7) {

- 	$owner = preg_replace('/^(@|%40)(.*)$/', 'g/$2', $path[2]);

- 	$project = $path[3];

- 	$buildid = ltrim(explode('-', $path[5])[0], '0');

- 	$frontend_url = implode('/', [$frontend_baseurl, 'coprs', $owner, $project, 'build', $buildid]);

- 	$frontend_url_text = 'Go to COPR frontend build';

- } else if (count($path) >= 5) {

- 	$owner = preg_replace('/^(@|%40)(.*)$/', 'g/$2', $path[2]);

- 	$project = $path[3];

- 	$frontend_url = implode('/', [$frontend_baseurl, 'coprs', $owner, $project]);

- 	$frontend_url_text = 'Go to COPR frontend project';

- } else {

- 	$frontend_url = $frontend_baseurl;

- 	$frontend_url_text = 'Go to COPR frontend';

- }

- 

- // Print ending stuff

- print "</tbody>

- 	</table>

- 	</div>

- 	<div class='script_title'>Lighttpd Enhanced Directory Listing Script</div>

- 	<div class='script_title' style='float:left; font-size:10pt'><a href=".$frontend_url.">".$frontend_url_text."</a></div>

- 	<div class='foot'>". $_ENV['SERVER_SOFTWARE'] . "</div>

- 	<script type='text/javascript'>

- 	// <!--

- 

- 	var click_column;

- 	var name_column = 0;

- 	var date_column = 1;

- 	var size_column = 2;

- 	var type_column = 3;

- 	var prev_span = null;

- 

- 	if (typeof(String.prototype.localeCompare) === 'undefined') {

- 	 String.prototype.localeCompare = function(str, locale, options) {

- 	   return ((this == str) ? 0 : ((this > str) ? 1 : -1));

- 	 };

- 	}

- 

- 	if (typeof(String.prototype.toLocaleUpperCase) === 'undefined') {

- 	 String.prototype.toLocaleUpperCase = function() {

- 	  return this.toUpperCase();

- 	 };

- 	}

- 

- 	function get_inner_text(el) {

- 	 if((typeof el == 'string')||(typeof el == 'undefined'))

- 	  return el;

- 	 if(el.innerText)

- 	  return el.innerText;

- 	 else {

- 	  var str = '';

- 	  var cs = el.childNodes;

- 	  var l = cs.length;

- 	  for (i=0;i<l;i++) {

- 	   if (cs[i].nodeType==1) str += get_inner_text(cs[i]);

- 	   else if (cs[i].nodeType==3) str += cs[i].nodeValue;

- 	  }

- 	 }

- 	 return str;

- 	}

- 

- 	function isdigit(c) {

- 	 return (c >= '0' && c <= '9');

- 	}

- 

- 	function unit_multiplier(unit) {

- 	 return (unit=='K') ? 1000

- 	      : (unit=='M') ? 1000000

- 	      : (unit=='G') ? 1000000000

- 	      : (unit=='T') ? 1000000000000

- 	      : (unit=='P') ? 1000000000000000

- 	      : (unit=='E') ? 1000000000000000000 : 1;

- 	}

- 

- 	function sortfn_then_by_name(a,b,sort_column) {

- 	 if (sort_column == name_column || sort_column == type_column) {

- 	  var ad = (a.cells[type_column].innerHTML === 'Directory');

- 	  var bd = (b.cells[type_column].innerHTML === 'Directory');

- 	  if (ad != bd) return (ad ? -1 : 1);

- 	 }

- 	 var at = get_inner_text(a.cells[sort_column]);

- 	 var bt = get_inner_text(b.cells[sort_column]);

- 	 var cmp;

- 	 if (a.cells[sort_column].className == 'int') {

- 	  cmp = parseInt(at)-parseInt(bt);

- 	 } else if (sort_column == date_column) {

- 	  cmp = Date.parse(at.replace(/-/g, '/'))

- 	      - Date.parse(bt.replace(/-/g, '/'));

- 	  var ad = isdigit(at.substr(0,1));

- 	  var bd = isdigit(bt.substr(0,1));

- 	  if (ad != bd) return (!ad ? -1 : 1);

- 	 } else if (sort_column == size_column) {

- 	  var ai = parseInt(at, 10) * unit_multiplier(at.substr(-1,1));

- 	  var bi = parseInt(bt, 10) * unit_multiplier(bt.substr(-1,1));

- 	  if (at.substr(0,1) == '-') ai = -1;

- 	  if (bt.substr(0,1) == '-') bi = -1;

- 	  cmp = ai - bi;

- 	 } else {

- 	  cmp = at.toLocaleUpperCase().localeCompare(bt.toLocaleUpperCase());

- 	  if (0 != cmp) return cmp;

- 	  cmp = at.localeCompare(bt);

- 	 }

- 	 if (0 != cmp || sort_column == name_column) return cmp;

- 	 return sortfn_then_by_name(a,b,name_column);

- 	}

- 

- 	function sortfn(a,b) {

- 	 return sortfn_then_by_name(a,b,click_column);

- 	}

- 

- 	function resort(lnk) {

- 	 var span = lnk.childNodes[1];

- 	 var table = lnk.parentNode.parentNode.parentNode.parentNode;

- 	 var rows = new Array();

- 	 for (j=1;j<table.rows.length;j++)

- 	  rows[j-1] = table.rows[j];

- 	 click_column = lnk.parentNode.cellIndex;

- 	 rows.sort(sortfn);

- 

- 	 if (prev_span != null) prev_span.innerHTML = '';

- 	 if (span.getAttribute('sortdir')=='down') {

- 	  span.innerHTML = '&uarr;';

- 	  span.setAttribute('sortdir','up');

- 	  rows.reverse();

- 	 } else {

- 	  span.innerHTML = '&darr;';

- 	  span.setAttribute('sortdir','down');

- 	 }

- 	 for (i=0;i<rows.length;i++)

- 	  table.tBodies[0].appendChild(rows[i]);

- 	 prev_span = span;

- 	}

- 

- 	function init_sort(init_sort_column, ascending) {

- 	 var tables = document.getElementsByTagName('table');

- 	 for (var i = 0; i < tables.length; i++) {

- 	  var table = tables[i];

- 	  //var c = table.getAttribute('class')

- 	  //if (-1 != c.split(' ').indexOf('sort')) {

- 	   var row = table.rows[0].cells;

- 	   for (var j = 0; j < row.length; j++) {

- 	    var n = row[j];

- 	    if (n.childNodes.length == 1 && n.childNodes[0].nodeType == 3) {

- 	     var link = document.createElement('a');

- 	     var title = n.childNodes[0].nodeValue.replace(/:$/, '');

- 	     link.appendChild(document.createTextNode(title));

- 	     link.setAttribute('href', '#');

- 	     link.setAttribute('class', 'sortheader');

- 	     link.setAttribute('onclick', 'resort(this);return false;');

- 	     var arrow = document.createElement('span');

- 	     arrow.setAttribute('class', 'sortarrow');

- 	     arrow.appendChild(document.createTextNode(':'));

- 	     link.appendChild(arrow)

- 	     n.replaceChild(link, n.firstChild);

- 	    }

- 	   }

- 	   var lnk = row[init_sort_column].firstChild;

- 	   if (ascending) {

- 	    var span = lnk.childNodes[1];

- 	    span.setAttribute('sortdir','down');

- 	   }

- 	   resort(lnk);

- 	  //}

- 	 }

- 	}

- 

- 	init_sort(0, 0);

- 

- 	// -->

- 	</script>

- 

- 	</body>

- 	</html>";

- 

- 

- /* -------------------------------------------------------------------------------- *\

- 	I hope you enjoyed my take on the enhanced directory listing script!

- 	If you have any questions, feel free to contact me.

- 

- 	Regards,

- 	   Evan Fosmark < me@evanfosmark.com >

- \* -------------------------------------------------------------------------------- */

- ?>

@@ -0,0 +1,20 @@ 

+ 

+ <div id='backlink' class='script_title' style='float:left; font-size:10pt'><a href="{{ frontend_base_url }}">Go to COPR frontend</a></div>

+ 

+ <script type='text/javascript'>

+ // <!--

+ const parts = window.location.pathname.split('/');

+ if (parts.length >= 5) {

+   owner = parts[2].replace(/^(@|%40)/, 'g/');

+   project = parts[3];

+   backlink = '{{ frontend_base_url }}/coprs/'+owner+'/'+project;

+   if (parts.length >= 7) {

+     backlink += parts[5].replace(/^0*([^-]+).*$/,'/build/$1'); // buildid

+     document.getElementById("backlink").innerHTML = "<a href="+backlink+">Go to COPR frontend build</a>"

+   }

+   else {

+     document.getElementById("backlink").innerHTML = "<a href="+backlink+">Go to COPR frontend project</a>"

+   }

+ }

+ // -->

+ </script>

This looks good, but I fail to make it work. Would you mind adding it as a separate patch?

2021-12-10 23:50:03: (response.c.407) -- parsed Request-URI
2021-12-10 23:50:03: (response.c.409) Request-URI     : /results/%40copr/copr-dev/epel-7-ppc64le
2021-12-10 23:50:03: (response.c.411) URI-scheme      : http
2021-12-10 23:50:03: (response.c.413) URI-authority   : copr-be-dev.cloud.fedoraproject.org
2021-12-10 23:50:03: (response.c.415) URI-path (clean): /results/@copr/copr-dev/epel-7-ppc64le
2021-12-10 23:50:03: (response.c.417) URI-query       : 
2021-12-10 23:50:03: (response.c.495) -- logical -> physical
2021-12-10 23:50:03: (response.c.497) Doc-Root     : /var/lib/copr/public_html
2021-12-10 23:50:03: (response.c.499) Basedir      : /var/lib/copr/public_html
2021-12-10 23:50:03: (response.c.501) Rel-Path     : /results/@copr/copr-dev/epel-7-ppc64le
2021-12-10 23:50:03: (response.c.503) Path         : /var/lib/copr/public_html/results/@copr/copr-dev/epel-7-ppc64le
2021-12-10 23:50:04: (response.c.407) -- parsed Request-URI
2021-12-10 23:50:04: (response.c.409) Request-URI     : /fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.411) URI-scheme      : https
2021-12-10 23:50:04: (response.c.413) URI-authority   : copr-be-dev.cloud.fedoraproject.org
2021-12-10 23:50:04: (response.c.415) URI-path (clean): /fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.417) URI-query       : 
2021-12-10 23:50:04: (response.c.495) -- logical -> physical
2021-12-10 23:50:04: (response.c.497) Doc-Root     : /var/lib/copr/public_html
2021-12-10 23:50:04: (response.c.499) Basedir      : /var/lib/copr/public_html
2021-12-10 23:50:04: (response.c.501) Rel-Path     : /fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.503) Path         : /var/lib/copr/public_html/fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.522) -- handling subrequest
2021-12-10 23:50:04: (response.c.524) Path         : /var/lib/copr/public_html/fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.526) URI          : /fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.528) Pathinfo     : 
2021-12-10 23:50:04: (mod_dirlisting.c.1245) -- handling the request as Dir-Listing
2021-12-10 23:50:04: (mod_dirlisting.c.1247) URI          : /fedora-rawhide-x86_64/01998568-deepin-system-monitor/

And still no readme file ^^^ appended...

@@ -18,7 +18,6 @@ 

  var.state_dir   = "/run"

  var.home_dir    = "/var/lib/lighttpd"

  var.conf_dir    = "/etc/lighttpd"

- var.log_pipe = "| /usr/local/bin/copr-lighty-logger "

  

  ##

  ## run the server chrooted.
@@ -83,9 +82,7 @@ 

    "mod_magnet",

    "mod_setenv",

    "mod_redirect",

-   "mod_rewrite",

    "mod_indexfile",

-   "mod_cgi",

    "mod_openssl"

  )

  
@@ -149,7 +146,7 @@ 

  ##

  ## Path to the error log file

  ##

- server.errorlog = log_pipe + log_root + "/error.log"

+ server.errorlog = log_root + "/error.log"

  

  ##

  ## If you want to log to syslog you have to unset the
@@ -173,6 +170,16 @@ 

  ##

  ## corresponding documentation:

  ## http://www.lighttpd.net/documentation/performance.html

+ 

+ ##

+ ## immediate graceful restart

+ ## (safe as long as dynamic backends are run independently

+ ##  i.e. *.server directives *do not* define "bin-path")

+ ##

+ #server.feature-flags += ( "server.graceful-shutdown-timeout" => 8 )

+ server.feature-flags += ( "server.graceful-restart-bg" => "enable" )

+ server.systemd-socket-activation = "enable"

+ 

  ##

  ## set the event-handler (read the performance section in the manual)

  ##
@@ -184,7 +191,7 @@ 

  ##

  ## linux-sysepoll is recommended on kernel 2.6.

  ##

- server.event-handler = "linux-sysepoll"

+ #server.event-handler = "linux-sysepoll"

  

  ##

  ## The basic network interface for all platforms at the syscalls read()
@@ -194,7 +201,7 @@ 

  ## linux-sendfile - is recommended for small files.

  ## writev         - is recommended for sending many large files

  ##

- server.network-backend = "linux-sendfile"

+ #server.network-backend = "linux-sendfile"

  

  ##

  ## As lighttpd is a single-threaded server, its main resource limit is
@@ -212,7 +219,7 @@ 

  ##

  ## With SELinux enabled, this is denied by default and needs to be allowed

  ## by running the following once : setsebool -P httpd_setrlimit on

- server.max-fds = 2048

+ #server.max-fds = 2048

Is this really expected?

Yes. The default is 4096 since lighttpd 1.4.56. For busy systems and for benchmarking lighttpd,
I recommend server.max-fds = 16384, and that is reflected in the sample config changed in
lighttpd 1.4.62 lighttpd1.4/doc/config/lighttpd.conf. Restricting lighttpd by number of fds is
typically not a good solution to a more specific problem as fds are used for many different things.
Instead, adjusting server.max-connections is a more precise setting.
https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_Performance

I commented out things that were defaults, or were suboptimal to the defaults.

  

  ##

  ## Stat() call caching.
@@ -222,7 +229,7 @@ 

  ## possible values are:

  ## disable, simple or fam.

  ##

- server.stat-cache-engine = "simple"

+ #server.stat-cache-engine = "simple"

  

  ##

  ## Fine tuning for the request handling
@@ -304,26 +311,6 @@ 

  ##                                 "index.htm", "default.htm" )

  ##

  

- # Warning: This is sooo ugly.

- #

- # We only ever want to enable PHP CGI for the index generator file

- # 'dir-generator.php', nothing else - so we are sure our users can not execute

- # their own scripts.  Therefore non-conditional 'cgi.assign' config can not be

- # used.

- #

- # But to make the 'index-file.names' work correctly, we still have to set

- # 'cgi.assign' conditionally - the trick is to do this only on paths that both

- # (a) end with slash (= directories) and (b) do not contain 'dir-generator'

- # string (guard against '/some/dir-generator.php/' or alike hacks).

- # index-file.names = (

- #   "/dir-generator.php"

- # )

- # $HTTP["url"] =~ "/$" {

- #   $HTTP["url"] !~ "dir-generator" {

- #     cgi.assign = ( "/dir-generator.php" => "/usr/bin/php-cgi" )

- #   }

- # }

This is very interesting. This config did not work with new lighty: f736ed7
I did not report it, but was it a security issue in previous version of lighty?

I am not aware of any such issue with lighttpd 1.4.60 or lighttpd 1.4.61.

However, I also do not understand what this mess is trying to accomplish.
I would suggest looking at lighttpd mod_alias to alias "/dir-generator.php" to a root-owned path,
and using a simple condition like $HTTP["url"] =~ "^/dirlisting.php" { cgi.assign = ( "" => "/usr/bin/php-cgi" ) }

Then again, I also found that this specific CGI script was wasteful and unnecessary, and could be
replaced by lighttpd mod_dirlisting and a small include file. I strongly recommend preferring a
solution using lighttpd mod_dirlisting, as doing so is simpler and eliminates any and all of the
security concerns expressed in the comment above.

I used that condition with the actual version (basically, see the commit) because it worked!
But with previous version, thetested that $HTTP["url"] =~ regexp did not
match with the dirlisting.php string but the directory listing URI being substitued with
the dirlisting. Therefore it did not enable the cgi.assign properly. I had to use
url =~ $/ condition to employ cgi.assign => which in turn caused that any custom
dirlisting.php was loaded. Therefore I limitted it using the !~.

- 

  ##

  ## deny access the file-extensions

  ##
@@ -333,14 +320,6 @@ 

  url.access-deny             = ( "~", ".inc" )

  

  ##

- ## disable range requests for pdf files

- ## workaround for a bug in the Acrobat Reader plugin.

- ##

- $HTTP["url"] =~ "\.pdf$" {

-   server.range-requests = "disable"

- }

- 

- ##

  ## url handling modules (rewrite, redirect)

  ##

  #url.rewrite                = ( "^/$"             => "/server-status" )
@@ -385,7 +364,7 @@ 

  ##

  ## Should lighttpd follow symlinks?

  ##

- server.follow-symlink = "enable"

+ #server.follow-symlink = "enable"

  

  ##

  ## force all filenames to be lowercase?
@@ -403,149 +382,74 @@ 

  

  #######################################################################

  ##

- ##  SSL Support

- ## -------------

- ##

- ## To enable SSL for the whole server you have to provide a valid

- ## certificate and have to enable the SSL engine.::

- ##

- ##   ssl.engine = "enable"

- ##   ssl.pemfile = "/path/to/server.pem"

- ##

- ## The HTTPS protocol does not allow you to use name-based virtual

- ## hosting with SSL. If you want to run multiple SSL servers with

- ## one lighttpd instance you must use IP-based virtual hosting: ::

- ##

- ##   $SERVER["socket"] == "10.0.0.1:443" {

- ##     ssl.engine                  = "enable"

- ##     ssl.pemfile                 = "/etc/ssl/private/www.example.com.pem"

- ##     #

- ##     # Mitigate BEAST attack:

- ##     #

- ##     # A stricter base cipher suite. For details see:

- ##     # http://blog.ivanristic.com/2011/10/mitigating-the-beast-attack-on-tls.html

- ##     #

- ##     ssl.cipher-list             = "ECDHE-RSA-AES256-SHA384:AES256-SHA256:RC4-SHA:RC4:HIGH:!MD5:!aNULL:!EDH:!AESGCM"

- ##     #

- ##     # Make the server prefer the order of the server side cipher suite instead of the client suite.

- ##     # This is necessary to mitigate the BEAST attack (unless you disable all non RC4 algorithms).

- ##     # This option is enabled by default, but only used if ssl.cipher-list is set.

- ##     #

- ##     # ssl.honor-cipher-order = "enable"

- ##     #

- ##     # Mitigate CVE-2009-3555 by disabling client triggered renegotation

- ##     # This is enabled by default.

- ##     #

- ##     # ssl.disable-client-renegotiation = "enable"

- ##     #

- ##     server.name                 = "www.example.com"

+ ## custom includes like vhosts.

  ##

- ##     server.document-root        = "/srv/www/vhosts/example.com/www/"

- ##   }

+ #include "conf.d/config.conf"

+ #include_shell "cat /etc/lighttpd/vhosts.d/*.conf"

  ##

+ #######################################################################

  

- ## If you have a .crt and a .key file, cat them together into a

- ## single PEM file:

- ## $ cat /etc/ssl/private/lighttpd.key /etc/ssl/certs/lighttpd.crt \

- ##   > /etc/ssl/private/lighttpd.pem

- ##

- #ssl.pemfile = "/etc/ssl/private/lighttpd.pem"

  

+ #######################################################################

  ##

- ## optionally pass the CA certificate here.

- ##

- ##

- #ssl.ca-file = ""

- 

+ ##  SSL Support

+ ## -------------

  ##

- #######################################################################

- 

- #######################################################################

+ ## https://wiki.lighttpd.net/Docs_SSL

  ##

- ## custom includes like vhosts.

+ ## To enable SSL for the whole server you have to provide a valid

+ ## certificate and have to enable the SSL engine.::

  ##

- #include "conf.d/config.conf"

- #include_shell "cat /etc/lighttpd/vhosts.d/*.conf"

+ ## ssl.pemfile = "/path/to/site/fullchain.pem"

+ ## ssl.privkey = "/path/to/site/privkey.pem"

+ ## $SERVER["socket"] == "0.0.0.0:443" { ssl.engine = "enable" }

+ ## $SERVER["socket"] == "[::]:443"    { ssl.engine = "enable" }

  ##

  #######################################################################

- 

- {% for https_bind in ["0.0.0.0:443", "[::]:443"] %}

- # we need to have this twice, once for IPv4 and once for IPv6, see:

- # https://redmine.lighttpd.net/projects/lighttpd/wiki/IPv6-Config#Recommended-IPv6-setup

+ $SERVER["socket"] == "0.0.0.0:443" { ssl.engine = "enable" }

+ $SERVER["socket"] == "[::]:443"    { ssl.engine = "enable" }

  

  {% if letsencrypt is not defined %}

  # production still uses normal configuration

- $SERVER["socket"] == "{{ https_bind }}" {

-   ssl.engine = "enable"

-   ssl.pemfile = "/etc/lighttpd/copr-be.cloud.fedoraproject.org.pem"

-   ssl.ca-file = "/etc/lighttpd/copr-be.cloud.fedoraproject.org.intermediate.cert"

-   ssl.disable-client-renegotiation = "enable"

-   ssl.cipher-list             = "ECDHE-RSA-AES256-SHA384:AES256-SHA256:RC4-SHA:RC4:HIGH:!MD5:!aNULL:!EDH:!AESGCM"

- }

+ ssl.privkey = "/etc/lighttpd/copr-be.cloud.fedoraproject.org.key"

+ ssl.pemfile = "/etc/lighttpd/copr-be.cloud.fedoraproject.org.cert"

+ ssl.ca-file = "/etc/lighttpd/copr-be.cloud.fedoraproject.org.intermediate.cert"

  {% else %}

  # devel runs on letsencrypt

- 

- # Enable HTTPS

- $SERVER["socket"] == "{{ https_bind }}" {

-     ssl.engine              = "enable"

  {% for hostname, _ in letsencrypt.certificates.items()  %}

-     ssl.ca-file             = "/etc/letsencrypt/live/{{ hostname }}/chain.pem"

-     ssl.pemfile             = "/etc/letsencrypt/live/{{ hostname }}/combined.pem"

- {% endfor %}

-     ssl.honor-cipher-order  = "enable"

-     # The following is OPTIONAL

-     ssl.cipher-list         = "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:ECDHE-RSA-AES128-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA128:DHE-RSA-AES128-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA128:ECDHE-RSA-AES128-SHA384:ECDHE-RSA-AES128-SHA128:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA128:DHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA384:AES128-GCM-SHA128:AES128-SHA128:AES128-SHA128:AES128-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4"

- }

- {% endif %}

+ ssl.privkey = "/etc/letsencrypt/live/{{ hostname }}/privkey.pem"

+ ssl.pemfile = "/etc/letsencrypt/live/{{ hostname }}/fullchain.pem"

This would be nice to have as a separate patch, too. I believe the cipher-list is replaced with "default" which should be PROFILE=SYSTEM by default on Fedora.

  {% endfor %}

- 

- {% if letsencrypt is defined %}

  # Used for letsencrypt validation

- $HTTP["url"] =~ "^/.well-known/" {

-     dir-listing.activate = "enable"

- }

+ # (?redundant since already enabled in "conf.d/dirlisting.conf"?)

+ #$HTTP["url"] =~ "^/.well-known/" {

+ #    dir-listing.activate = "enable"

+ #}

  {% endif %}

  

+ 

+ # ??? obsolete rule ??? committed in 2015

  url.redirect               = ( "^/results/sgallagh/cockpit-preview/(.+)" => "/results/@cockpit/cockpit-preview/$1" )

  

+ # ??? obsolete rule ??? committed in 2016  Are there still i586 logs? (instead of i686?)

@praiskup, are these obsolete? Did spacewalk archives move to a different server? Did cockpit-preview? Do i586 build logs exist?

Spacewall/cockipt: I don't know personally, perhaps @xsuchy?
Mageia? There's cd4e0a7 but commit message doesn't say why.
But is there a reason to touch those rules?

But is there a reason to touch those rules?

Yes: bit rot. Since we're looking in lighttpd.conf, it is useful to remove cruft if it is easy to identify.
Next time (in the future) that someone needs to a make a modification, a simpler config is
easier to test and they do not have to guess why something is there and if the change they are
making will affect it.

  url.redirect += ( "^/results/(.*)/(.*)/mageia-(.*)-i386(.*)" => "/results/$1/$2/mageia-$3-i586$4" )

  

- url.redirect += ( "^/(.*)/redirect-builder-live.log$" => "/$1/builder-live.log.gz" )

- url.rewrite-if-not-file = ("^/(.*)/builder-live.log$" => "/$1/redirect-builder-live.log")

- 

- url.redirect += ( "^/(.*)/redirect-backend.log$" => "/$1/backend.log.gz" )

- url.rewrite-if-not-file += ("^/(.*)/backend.log$" => "/$1/redirect-backend.log")

This is really interesting, the lua replacement. But we need to have this properly tested. A separate patch would be nice so we can test in izolation.

- 

- $HTTP["url"] =~ "^/archive/spacewalk($|/$)" {

-   dir-listing.activate = "disable"

-   index-file.names   = ( "index.html" )

-   accesslog.filename = ""

- }

- $HTTP["url"] =~ "^/archive/spacewalk/" {

+ # ??? did spacewalk archives move to a different server?

+ #     e.g. https://listman.redhat.com/archives/spacewalk-devel/

+ $HTTP["url"] =~ "^/archive/spacewalk($|/)" {

+   #$HTTP["url"] == "/archive/spacewalk/" {

+     dir-listing.activate = "disable"

+   #}

    index-file.names   = ( "index.html" )

-   accesslog.filename = ""

+   #accesslog.filename = ""

  }

- 

- # This may break many things, per: https://redmine.lighttpd.net/projects/1/wiki/Server_max-workerDetails

- # At least we have to use cronolog or alternative: https://redmine.lighttpd.net/projects/1/wiki/Docs_ModAccesslog

- # But we need logrotate for logcounter, see /etc/logrotate.d/lighttpd, and

- # logrotate doesn't work well with logcounter.

- server.max-worker = 6

- 

- $HTTP["url"] !~ "^/archive/spacewalk($|/)" {

-   accesslog.filename = log_pipe + log_root + "/access.log"

-   $HTTP["url"] =~ "\.log\.gz$" {

+ else {

+   accesslog.filename = log_root + "/access.log"

+   $HTTP["url"] =~ "\.log(?:\.gz)?$" {

      magnet.attract-physical-path-to = ( "/etc/lighttpd/content-encoding-gzip-if-exists.lua" )

-     mimetype.assign = ("" => "text/plain" )

    }

  }

  

- # https://pagure.io/copr/copr/issue/762

- $HTTP["url"] =~ "\.log$" {

-     setenv.add-response-header  = ( "Cache-Control" => "no-store")

-     mimetype.assign = ("" => "text/plain; charset=utf-8" )

- }

- 

  $HTTP["url"] =~ "/repomd.xml" {

      setenv.add-response-header  = ( "Cache-Control" => "no-cache")

  }

@@ -2,25 +2,6 @@ 

  # option.  This means that hitcounter is run only once too (which is good).  In

  # (likely impossible) situations when access.log is empty and error.log not, we

  # would call hitcounter too on an empty file.

- #

- # Note that lighttpd server runs with max-workers, and thus we should pipe the

- # logs through cronolog to have consistent logs (see mod_accesslog docs).  On

- # the other hand, we still use logrotate.  That's needed because it supports

- # compressison, and because it can trigger the hitcounter script below.

- #

- # Running both "log rotation" programs turned out to be very complicated:

- # - sending -HUP to lighty doesn't restart the piped cronolog, so it never

- #   reopens the file after rotation

- # - cronolog doesn't handle -HUP correctly, it just terminates

- # - when cronolog terminates, lighttpd restarts all the child processes (not

- #   just cronolog processes), leading to whole bunch of problems (but there's at

- #   least short downtime)

- # There's no reasonable way-around this than logrotate's copytruncate option

- # (which is racy by definition).  Lighty is innocent here (doesn't ever expect

- # the log pipe to terminate, and if it happens it's good reason to restart

- # everything).  Cronolog is innocent here, it doesn't actaully do any rotation.

- # Lograte is innocent as well.

- # See https://pagure.io/copr/copr/issue/2001 for more info.

  

  /var/log/lighttpd/*log {

      rotate 5
@@ -31,9 +12,13 @@ 

      delaycompress

      sharedscripts

      prerotate

+ {% if devel %}

          /usr/bin/copr_log_hitcounter.py /var/log/lighttpd/access.log --ignore-subnets 172.25.144.0/20 209.132.184.33/24 &>>/var/log/copr-backend/hitcounter-logrotate.log || :

+ {% else %}

+         /usr/bin/copr_log_hitcounter.py /var/log/lighttpd/access.log --ignore-subnets 172.25.80.0/20 209.132.184.33/24 &>>/var/log/copr-backend/hitcounter-logrotate.log || :

+ {% endif %}

      endscript

      postrotate

-         /usr/local/bin/copr-lighty-reopen-logs || :

+         /usr/bin/kill -HUP $(systemctl show --property MainPID --value lighttpd) || :

After my quick testing, I tend to agree that max-worker = N option doesn't help us.
I'll post the numbers once I can test the javascript variant ... (but honestly, I don't think
Javascript is better than server-side script in this case, just faster). This needs to be
discussed a bit in our team before we move.

From a secdevops perspective, eliminating dir-generator.php and server-side scripts reduces
exposure. See my comments above about the misguided configuration that attempted to
secure dir-generator.php. Eliminating dir-generator.php also reduces maintenance burden
(as lighttpd handles it instead of needing PHP-FPM config), and reduces server load. I am
surprised and disappointed that someone like yourself in infrastructure has overlooked all
of these benefits.

[Edited for word wrapping.
Why doesn't pagure word wrap comments in the Files Changed view?]

There was some bug in lighttpd forcing us to "secure" it that way (v1.4.55 on Fedora 33).
I see the benefits of the JS solution :-) and I tend to go the JS way (just going to discuss
this with team). But really, if the lua support (we use for adjusting headers) could
calculate the stuff added to the listing footer, it would be better.

There was some bug in lighttpd forcing us to "secure" it that way (v1.4.55 on Fedora 33).

Is there a ticket somewhere on pagure.io describing this? Was it reported upstream?

But really, if the lua support (we use for adjusting headers) could
calculate the stuff added to the listing footer, it would be better.

lighttpd can do lots of things. Since lighttpd 1.4.60, you could implement dir-generator.php
entirely in lua inside lighttpd. However, that is probably not an optimal solution on a busy site.
A lua script run from lighttpd mod_magnet magnet.attract-response-start-to hook can also
modify the response to add the link, but I still think the client-side is a trivial place to generate
this link. I hope you'll spend a few more cycles looking into why the include file is not being
included in the lighttpd mod_dirlisting output.

Is there a ticket somewhere on pagure.io describing this? Was it reported upstream?
Was it reported upstream?

No :-( I fell into an assumption the previous behavior was expected.

lighttpd can do lots of things. Since lighttpd 1.4.60

Ok, I think I can take a look at script how to append the footer? It
sounds like something that shouldn't eat too much CPU, right?

      endscript

  }

Draft: lighttpd infra simplification

sketching updates based on recent issues
https://pagure.io/copr/copr/issue/2001
https://redmine.lighttpd.net/issues/3123
and older issues
https://pagure.io/copr/copr/issue/921

A number of items have comments for discussion and validation. @praiskup

One example: roles/copr/backend/files/lighttpd/mime.conf was created in 2014 and the significant change from default /etc/lighttpd/conf.d/mime.conf is the default Content-Type "" => "text/plain", instead of "application/octet-stream". Is this still relevant for downloads? Are there a smaller set of extensions which should have "text/plain" forced or a smaller portion of the site for which this should apply? It is generally preferable to use /etc/lighttpd/conf.d/mime.conf unless there is a specific reason not to do so.

rebased onto 487542e

2 years ago

Not yet part of this PR: what types of responses are cachable? lighttpd.conf does not use mod_expire to set any Cache-Control headers, but probably should.

This looks good, but I fail to make it work. Would you mind adding it as a separate patch?

2021-12-10 23:50:03: (response.c.407) -- parsed Request-URI
2021-12-10 23:50:03: (response.c.409) Request-URI     : /results/%40copr/copr-dev/epel-7-ppc64le
2021-12-10 23:50:03: (response.c.411) URI-scheme      : http
2021-12-10 23:50:03: (response.c.413) URI-authority   : copr-be-dev.cloud.fedoraproject.org
2021-12-10 23:50:03: (response.c.415) URI-path (clean): /results/@copr/copr-dev/epel-7-ppc64le
2021-12-10 23:50:03: (response.c.417) URI-query       : 
2021-12-10 23:50:03: (response.c.495) -- logical -> physical
2021-12-10 23:50:03: (response.c.497) Doc-Root     : /var/lib/copr/public_html
2021-12-10 23:50:03: (response.c.499) Basedir      : /var/lib/copr/public_html
2021-12-10 23:50:03: (response.c.501) Rel-Path     : /results/@copr/copr-dev/epel-7-ppc64le
2021-12-10 23:50:03: (response.c.503) Path         : /var/lib/copr/public_html/results/@copr/copr-dev/epel-7-ppc64le
2021-12-10 23:50:04: (response.c.407) -- parsed Request-URI
2021-12-10 23:50:04: (response.c.409) Request-URI     : /fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.411) URI-scheme      : https
2021-12-10 23:50:04: (response.c.413) URI-authority   : copr-be-dev.cloud.fedoraproject.org
2021-12-10 23:50:04: (response.c.415) URI-path (clean): /fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.417) URI-query       : 
2021-12-10 23:50:04: (response.c.495) -- logical -> physical
2021-12-10 23:50:04: (response.c.497) Doc-Root     : /var/lib/copr/public_html
2021-12-10 23:50:04: (response.c.499) Basedir      : /var/lib/copr/public_html
2021-12-10 23:50:04: (response.c.501) Rel-Path     : /fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.503) Path         : /var/lib/copr/public_html/fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.522) -- handling subrequest
2021-12-10 23:50:04: (response.c.524) Path         : /var/lib/copr/public_html/fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.526) URI          : /fedora-rawhide-x86_64/01998568-deepin-system-monitor/
2021-12-10 23:50:04: (response.c.528) Pathinfo     : 
2021-12-10 23:50:04: (mod_dirlisting.c.1245) -- handling the request as Dir-Listing
2021-12-10 23:50:04: (mod_dirlisting.c.1247) URI          : /fedora-rawhide-x86_64/01998568-deepin-system-monitor/

And still no readme file ^^^ appended...

This would be nice to have as a separate patch, too. I believe the cipher-list is replaced with "default" which should be PROFILE=SYSTEM by default on Fedora.

This is really interesting, the lua replacement. But we need to have this properly tested. A separate patch would be nice so we can test in izolation.

This is very interesting. This config did not work with new lighty: f736ed7
I did not report it, but was it a security issue in previous version of lighty?

After my quick testing, I tend to agree that max-worker = N option doesn't help us.
I'll post the numbers once I can test the javascript variant ... (but honestly, I don't think
Javascript is better than server-side script in this case, just faster). This needs to be
discussed a bit in our team before we move.

Yes. The default is 4096 since lighttpd 1.4.56. For busy systems and for benchmarking lighttpd,
I recommend server.max-fds = 16384, and that is reflected in the sample config changed in
lighttpd 1.4.62 lighttpd1.4/doc/config/lighttpd.conf. Restricting lighttpd by number of fds is
typically not a good solution to a more specific problem as fds are used for many different things.
Instead, adjusting server.max-connections is a more precise setting.
https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_Performance

I commented out things that were defaults, or were suboptimal to the defaults.

I am not aware of any such issue with lighttpd 1.4.60 or lighttpd 1.4.61.

However, I also do not understand what this mess is trying to accomplish.
I would suggest looking at lighttpd mod_alias to alias "/dir-generator.php" to a root-owned path,
and using a simple condition like $HTTP["url"] =~ "^/dirlisting.php" { cgi.assign = ( "" => "/usr/bin/php-cgi" ) }

Then again, I also found that this specific CGI script was wasteful and unnecessary, and could be
replaced by lighttpd mod_dirlisting and a small include file. I strongly recommend preferring a
solution using lighttpd mod_dirlisting, as doing so is simpler and eliminates any and all of the
security concerns expressed in the comment above.

From a secdevops perspective, eliminating dir-generator.php and server-side scripts reduces
exposure. See my comments above about the misguided configuration that attempted to
secure dir-generator.php. Eliminating dir-generator.php also reduces maintenance burden
(as lighttpd handles it instead of needing PHP-FPM config), and reduces server load. I am
surprised and disappointed that someone like yourself in infrastructure has overlooked all
of these benefits.

[Edited for word wrapping.
Why doesn't pagure word wrap comments in the Files Changed view?]

roles/copr/backend/files/lighttpd/mime.conf was created in 2014 and the significant change from default /etc/lighttpd/conf.d/mime.conf is the default Content-Type "" => "text/plain", instead of "application/octet-stream". Is this still relevant for downloads? Are there a smaller set of extensions which should have "text/plain" forced or a smaller portion of the site for which this should apply? It is generally preferable to use /etc/lighttpd/conf.d/mime.conf unless there is a specific reason not to do so.

@praiskup, are these obsolete? Did spacewalk archives move to a different server? Did cockpit-preview? Do i586 build logs exist?

@praiksup how much of the server load is due to directory listing?
How frequently are the directory listings requested?
How frequently are directories changing?
Is 15 seconds reasonable for caching a directory listing?
Is 60 seconds reasonable?
How large are these directories?
If only a few files, caching might only marginally improve performance.
If directories do not change frequently, or if they change once to compress logs,
then generating a static index.html when the directory contents are compressed,
and then allowing the index.html to be cached, is probably the most performant
solution.

I split the changes in this PR into multiple commits on a different branch, rebased on current main branch: https://pagure.io/fork/gstrauss/fedora-infra/ansible/commits/lighttpd-smaller-commits

I could rebase/force-push to this PR, but that might make it more difficult to follow the various conversations above.

Regarding your question about dirlisting:

This looks good, but I fail to make it work. [...]
And still no readme file ^^^ appended...

You will not see any trace in the lighttpd error log for the included file, but should see the link at the bottom of the resulting page. I had tested the mod_dirlisting settings in my local lighttpd test environment, which is not the same as your ansible environment, so please check that /etc/lighttpd/dirlisting.inc was installed and has proper permissions so that lighttpd mod_dirlisting can read it. Also, please make sure that /etc/lighttpd/conf.d/dirlisting.conf contains the settings to enable the file include in the directory listing.

I used that condition with the actual version (basically, see the commit) because it worked!
But with previous version, thetested that $HTTP["url"] =~ regexp did not
match with the dirlisting.php string but the directory listing URI being substitued with
the dirlisting. Therefore it did not enable the cgi.assign properly. I had to use
url =~ $/ condition to employ cgi.assign => which in turn caused that any custom
dirlisting.php was loaded. Therefore I limitted it using the !~.

There was some bug in lighttpd forcing us to "secure" it that way (v1.4.55 on Fedora 33).
I see the benefits of the JS solution :-) and I tend to go the JS way (just going to discuss
this with team). But really, if the lua support (we use for adjusting headers) could
calculate the stuff added to the listing footer, it would be better.

We should go with the default config, yes.

I think that the configuration - as you prpopose - is fine. Lightttpd just doesn't
dload the inc file (deployed here https://copr-be-dev.cloud.fedoraproject.org/results/%40rocknsm/rocknsm-2.1/).

Spacewall/cockipt: I don't know personally, perhaps @xsuchy?
Mageia? There's cd4e0a7 but commit message doesn't say why.
But is there a reason to touch those rules?

But is there a reason to touch those rules?

Yes: bit rot. Since we're looking in lighttpd.conf, it is useful to remove cruft if it is easy to identify.
Next time (in the future) that someone needs to a make a modification, a simpler config is
easier to test and they do not have to guess why something is there and if the change they are
making will affect it.

There was some bug in lighttpd forcing us to "secure" it that way (v1.4.55 on Fedora 33).

Is there a ticket somewhere on pagure.io describing this? Was it reported upstream?

But really, if the lua support (we use for adjusting headers) could
calculate the stuff added to the listing footer, it would be better.

lighttpd can do lots of things. Since lighttpd 1.4.60, you could implement dir-generator.php
entirely in lua inside lighttpd. However, that is probably not an optimal solution on a busy site.
A lua script run from lighttpd mod_magnet magnet.attract-response-start-to hook can also
modify the response to add the link, but I still think the client-side is a trivial place to generate
this link. I hope you'll spend a few more cycles looking into why the include file is not being
included in the lighttpd mod_dirlisting output.

I think that the configuration - as you prpopose - is fine. Lightttpd just doesn't
dload the inc file (deployed here https://copr-be-dev.cloud.fedoraproject.org/results/%40rocknsm/rocknsm-2.1/).

I see that the link is not included. You might strace lighttpd to see if it is trying to open the file after generating the
dirlisting. You might also check lighttpd -f /etc/lighttpd/lighttpd.conf -p and look at the dir-listing.* options to
check that it is configured. Of course, also please also check that lighttpd has been properly restarted after any
config change.

As an aside, I poked around and saw .xml.gz files there. The lua code in this patch handling .log.gz
could be extended to handle more types. Back in 2017, I wrote a more pedantically correct bit of
lua to parse client Accept-Encoding and Accept-Language and to then choose a resource to send.
https://wiki.lighttpd.net/AbsoLUAtion#Content-Negotiation

doh! Sorry. You're running lighttpd 1.4.61. Latest lighttpd release is lighttpd 1.4.63. There was a bug in lighttpd 1.4.60 and lighttpd 1.4.61 which did not show the readme. https://git.lighttpd.net/lighttpd/lighttpd1.4/commit/cba6a1ab54f063609922ea0085d6ba15ed77fa6f

If the mime.conf is removed from this ansible package, how will the original from lighttpd-filesystem be restored? It is a config file, so it won't be overwritten by lighttpd package upgrades, will it?

Answering your questions isn't easy, as it depends on the concrete directory,
it's not homogeneous load...
Though the caches are potentially good against DoS attempts at least?

Also, most of the traffic to lighty goes through CloudFronts so it's cached
there. And the load is pretty low nowadays, I would say.

I mean load forwarded to lighttpd is pretty low.

This is easier to do manually:, remove the config file and dnf reinstall lighttpd. Or we can wait till we migrate to a new version of Fedora (year or so) - then we install from scratch.

Future enhancement:
While poking around some subdirectories in https://copr-be-dev.cloud.fedoraproject.org/results/%40rubygems/rubygems/fedora-rawhide-x86_64/, it apppears to me that once a build has completed, the contents of the directory do not change. As a future enhancement, when the logs are in the completed build are compressed, an index.html could be generated into the directory (and the index.html could be excluded from the directory listing)

Is there a ticket somewhere on pagure.io describing this? Was it reported upstream?
Was it reported upstream?

No :-( I fell into an assumption the previous behavior was expected.

lighttpd can do lots of things. Since lighttpd 1.4.60

Ok, I think I can take a look at script how to append the footer? It
sounds like something that shouldn't eat too much CPU, right?

I think I answered my own question. During a build, the logs are being modified, so the caching interval should be brief, or disabled. If not too many people are watching live builds and nothing is polling in a tight loop, then there is little point to enabling the mod_dirlisting cache. That is true as long as once the build completes, an index.html is generated with the directory listing, so that the overhead of generating the directory listing never occurs again for that (completed) build.

lighttpd is not returning HTTP Cache-Control in the response headers, so I am unsure how your are controlling the caching at CloudFront. Have you disabled the CloudFront content re-validation? Once an index.html is generated in a directory, then we should allow that to be cached. A short lua script could be substituted for mod_indexfile, and could add appropriate Cache-Control to the response if the file was served from index.html, or if index.html was not present, allow mod_dirlisting to generate the response.

I think we could request an update of Lighttpd in Fedora 35... if going with JS and not Lua.

Ok, I think I can take a look at script how to append the footer? It
sounds like something that shouldn't eat too much CPU, right?

While you could, I don't think it is the best solution, and it would load the whole response into memory for lua to be able to rewrite it, though that would only really matter for very large directory listings.

I think using mod_dirlisting with the include file (and using lighttpd 1.4.62 or later) is a superior solution, along with my longer-term suggestion to generate index.html in the build directory after the build completes and logs are compressed.

CloudFronts caches everything, and yes - it really leads to user confusion (people are asking us what happened, that they don't see the build finished, etc.). So I'm sure the directory listing should be controlled at least till we finish the build.

I think we could request an update of Lighttpd in Fedora 35... if going with JS and not Lua.

I am discussing that with Gwyn. There was an snafu with SELinux disallowing setrlimit() in lighttpd, which has temporarily held up Gwyn from packaging lighttpd 1.4.62 and lighttpd 1.4.63.

lighttpd 1.4.64 is scheduled to be released in the first week of Jan and Gwyn will likely backport it to Fedora 35. I might suggest cherry-picking a couple patches to add to lighttpd 1.4.63 so that could be deployed sooner.

I think using mod_dirlisting with the include file (and using lighttpd 1.4.62 or later) is a superior solution, along with my longer-term suggestion to generate index.html in the build directory after the build completes and logs are compressed.

I don't think we want to take care of generating the index.html, unless it is done automatically on a lighttpd cache layer. The builds can also be removed, a few RPMs in the directory removed, etc, so we need to regenerate sometimes.

CloudFronts caches everything

I mean, we configured it so it caches everything, except for the smallest repomd.xml files
in the repodata directories. This guarantees that DNF clients always use the newest
RPM metadata.

That being said, we haven't spent too much time on this. Directory listing is not a priority here,
but still it confuses our users and long term we should resolve this.

It seems like a safer solution is to include /etc/lighttpd/conf.d/mime.conf from lighttpd-filesystem into this repository so that it overwrites existing installations. Then, some time later, after a round of deployment through dev, staging, and prod, then mime.conf can be removed from this package.

I think using mod_dirlisting with the include file (and using lighttpd 1.4.62 or later) is a superior solution, along with my longer-term suggestion to generate index.html in the build directory after the build completes and logs are compressed.

I don't think we want to take care of generating the index.html, unless it is done automatically on a lighttpd cache layer. The builds can also be removed, a few RPMs in the directory removed, etc, so we need to regenerate sometimes.

Ok. The mod_dirlisting caching is controlled with dir-listing.cache = ( "path" => cache_dir, "max-age" => 15 ) (and is disabled by default)
So what is a reasonable max-age value? 15 seconds? 30 seconds? Remember that this value is only for mod_dirlisting internally caching the result of reading the directory. If you have CloudFront caching this result for 30 seconds (or longer!), then you might set CloudFront to 5 seconds for directory caching ("/$"), and change "max-age" for dir-listing.cache to 30 seconds.

The builds can also be removed, a few RPMs in the directory removed, etc, so we need to regenerate sometimes.

Who/what removes these? Could whatever removes these also regenerate an index.html?

The builds can also be removed, a few RPMs in the directory removed, etc, so we need to regenerate sometimes.

Who/what removes these? Could whatever removes these also regenerate an index.html?

Alternatively, you could have a cron job which runs every 5 minutes and scans the directory tree checking that if index.html is present, that it is the same timestamp as the containing directory modification timestamp, or else regenerate it.

It seems like a safer solution is to include
/etc/lighttpd/conf.d/mime.conf from lighttpd-filesystem into this
repository so that it overwrites existing installations. Then, some
time later, after a round of deployment through dev, staging, and prod,
then mime.conf can be removed from this package.

This is not a package, just a git repo used for deployment. And we
"replace" the config shipped in lighttpd-filesystem.

I don't think we want to take care of generating the index.html,
unless it is done automatically on a lighttpd cache layer. The builds
can also be removed, a few RPMs in the directory removed, etc, so we
need to regenerate sometimes.

Ok. The mod_dirlisting caching is controlled with dir-listing.cache = ( "path" => cache_dir, "max-age" => 15 ) (and is disabled by default)
So what is a reasonable max-age value? 15 seconds? 30 seconds?
Remember that this value is only for mod_dirlisting internally caching
the result of reading the directory

I tend to think we are over-optimizing now, and too early. But generally
I think going with some value like 1 minute should decrease the confusion
a lot.

But if we want to be precise, the max-age is not something we should set
globally for all directories, I think. The build directory should have
a pretty low max-age value (at least when the build is still being done).

Two levels up (users' project directories) I think we can have a longer
max-age, and three levels up it can be a much, much longer cache.

If you have CloudFront caching this result for 30 seconds (or longer!),
then you might set CloudFront to 5 seconds for directory caching ("/$"),
and change "max-age" for dir-listing.cache to 30 seconds.

I'm not quite sure how CloudFronts is configured. Not even if there's
some "max-age" configuration ... or if we use it. As I understood it,
Cloudfronts should respect the max-age header given by lighty, though.

The builds can also be removed, a few RPMs in the directory removed,
etc, so we need to regenerate sometimes.

Who/what removes these? Could whatever removes these also regenerate an
index.html?

End-users can remove the whole build directories and the particular RPMs
are removed by an automatic garbage collector. Failed builds (with no
binary RPMs) have their own logic for garbage collecting .. and there are
requests to actually configure the period of how long some of the build
logs are kept, etc.

I don't like this idea at all, it is too much work for a low value. Also,
if we decided to change the footer format, would we have to crawl the 10T+
storage and re-build all the index.html files?

I tend to think we are over-optimizing now, and too early. But generally
I think going with some value like 1 minute should decrease the confusion
a lot.
...
Two levels up (users' project directories) I think we can have a longer
max-age, and three levels up it can be a much, much longer cache.

Ok. So I'll keepdir-listing.cache = ( "path" => cache_dir, "max-age" => 15 ) in this patch.
It is an internal max-age for mod_dirlisting, and applies only to the directory listing. It prevents people from mashing the refresh button from causing undue load re-reading the directory over and over since the result is cached in a file. HTTP caching response headers (Cache-Control) are separate and not sent from here.

Separately, it sounds like something like the following could be added to lighttpd.conf:

$HTTP["url"] =~ "^/results(?:/[^/]+){0,2}/$" {
    #setenv.set-response-header  = ( "Cache-Control" => "max-age=600")
    expire.url = ( "" => "access plus 10 minutes" )
}
else $HTTP["url"] =~ "^/results(?:/[^/]+){3,}/$" {
    #setenv.set-response-header  = ( "Cache-Control" => "max-age=120")
    expire.url = ( "" => "access plus 2 minutes" )
}

where /results/ is the frontend_base_url. (Alternatively, lighttpd mod_expires is more correct for generating caching header and could be used to provide similar results.)

The above could also be a short lua script if you wanted to check the next depth of directory and emit a longer Cache-Control if the build has completed (e.g. if there is a "success" file present in the directory)

@praiskup: So that you could continue testing using lighttpd 1.4.61, I pushed a commit to the top of https://pagure.io/fork/gstrauss/fedora-infra/ansible/commits/lighttpd-smaller-commits which abuses lighttpd dir-listing.set-footer to insert the javascript. I hard-coded a path to https://copr.fedorainfracloud.org/ rather than updating templates to use '{{ frontend_base_url }}'

But is there a reason to touch those rules?

Yes: bit rot. Since we're looking in lighttpd.conf, it is useful to remove cruft if it is easy to identify.
Next time (in the future) that someone needs to a make a modification, a simpler config is
easier to test and they do not have to guess why something is there and if the change they are
making will affect it.

@praiskup please check access logs for HTTP 301 responses to requests matching

  • "^/results/sgallagh/cockpit-preview/(.+)"
  • "^/results/(.)/(.)/mageia-(.)-i386(.)"

and please check access logs for requests to

  • "^/archive/spacewalk($|/)"

If there have been none in 2021, then these rules in copr lighttpd.conf can very likely be retired.

Thanks, I'm moving this here, we need to take a look at the proposal step by step:
https://pagure.io/copr/copr/issue/2011

Pull-Request has been closed by praiskup

2 years ago

Ok. I am going to force-push my https://pagure.io/fork/gstrauss/fedora-infra/ansible/commits/lighttpd-smaller-commits branch here, since that has the latest updates.