PR#985: backend: do not provide the builder-live.log stub - copr/copr

copr / copr

#985 backend: do not provide the builder-live.log stub

Merged 4 years ago by dturecek. Opened 4 years ago by praiskup.

Unknown source builder-live-log-compressed into master

backend: do not provide the builder-live.log stub

Pavel Raiskup • 4 years ago

df2fb85

backend/backend/mockremote/__init__.py

file modified

+11 -7

		`@@ -277,18 +277,22 @@`
		`def compress_live_log(self, job):`
		`log_basename = "builder-live.log"`
		`src = os.path.join(job.results_dir, log_basename)`
		`-`
		`+`
		`+ # For automatic redirect from log to log.gz, consider configuring`
		`+ # lighttpd like:`
		`+ # url.rewrite-if-not-file = ("^/(.*)/builder-live.log$" => "/$1/redirect-live.log")`
		`+ # url.redirect("^/(.*)/redirect-live.log$" => "/$1/builder-live.log.gz")`
		`+ # or apache by:`
		`+ # <Files builder-live.log>`
		`+ # RewriteEngine on`
		`+ # RewriteCond %{REQUEST_FILENAME} !-f`
		`+ # RewriteRule ^(.*)$ %{REQUEST_URI}.gz [R]`
		`+ # </files>`
		`self.log.info("Compressing {} by gzip".format(src))`
		`if subprocess.call(["gzip", src]) not in [0, 2]:`
		`self.log.error("Unable to compress file {}".format(src))`
		`return`

		`- try:`
		`- with open(src, "w") as f_src:`
		`- f_src.write("{} was moved to {}.gz\n".format(log_basename, log_basename))`
		`- except Exception as e:`
		`- self.log.exception(e)`
		`-`
		`def reattach_to_pkg_build(self):`
		`"""`
		`:raises VmError, BuilderError`

backend/run/copr-compress-live-logs

file added

+23

		`@@ -0,0 +1,23 @@`
		`+ #! /bin/sh`
		`+`
		`+ # Traverse the given directory and try to find files named 'builder-live.log'`
		`+ # and gzip them (or remove, if the corresponding gzipped file already exists).`
		`+`
		`+ die() { echo "$0: FATAL: $*" ; exit 1 ; }`
		`+`
		`+ test "$#" -lt 1 && die "<dir> argument expected"`
		`+`
		`+ dir=$1`
		`+`
		`+ test -d "$dir" \|\| die "'$dir' is not a directory"`
		`+`
		`+ # make sure we use -mtime to not hit logs for actually running builds`
		`+ find "$dir" -name builder-live.log -mtime 7 -type f \| \`
		`+ while read -r uncompressed; do`
		`+ compressed="$uncompressed.gz"`
		`+ if test -f "$compressed"; then`
		`+ rm -f "$uncompressed"`
		`+ else`
		`+ gzip "$uncompressed"`
		`+ fi`
		`+ done`

praiskup commented 4 years ago

Turns out that my original idea for PR#788 wasn't ideal and it is better
to not provide builder-live.log stub. Especially after
4aa9324 (UI generates appropriate link), so the http server
redirect configuration is mostly voluntary.

The thing is that it is very complicated (if possible at all) to
configure lighttpd (Fedora Copr deployment case) to redirect
automatically from 'builder-live.log' to 'builder-live.log.gz' when the
'builder-live.log.gz' exists. And OTOH it is pretty trivial to
redirect to if 'builder-live.log' doesn't exist.

Also, notable benefit of this change is that we'll save some inode
quota.

While I'm on it, I'm adding single-shot script which will traverse the
build directory and (a) remove the stub logs we created before, and (b)
compress the non-stub uncompressed builder-live.log files (from the
times we did not compress at all). This is needed to have working
'builder-live.log.gz' links from UI.

Relates: #921, PR#788, #941

rebased onto 0395012955ad3a197895aa9788b8bf3ddb5fe878

4 years ago

rebased onto fe7a3eb2f50a6862df8415ff5501c730a088f7e3

4 years ago

Metadata Update from @praiskup:
- Request assigned

4 years ago