PR#1796: backend: fix copr_prune_results logging once more - copr/copr

copr / copr

#1796 backend: fix copr_prune_results logging once more

Merged 3 years ago by frostyx. Opened 3 years ago by praiskup.

Unknown source prunerepo-logging-once-more into main

backend: fix copr_prune_results logging once more

Pavel Raiskup • 3 years ago

5aa40bb

backend/copr-backend.spec

file modified

+3 -2

		`@@ -2,6 +2,7 @@`
		`%global _pkgdocdir %{_docdir}/%{name}-%{version}`
		`%endif`

		`+ %global prunerepo_version 1.20`
		`%global tests_version 2`
		`%global tests_tar test-data-copr-backend`

		`@@ -54,7 +55,7 @@`
		`BuildRequires: python3-sphinx`
		`BuildRequires: python3-tabulate`
		`BuildRequires: modulemd-tools >= 0.6`
		`- BuildRequires: prunerepo >= 1.19`
		`+ BuildRequires: prunerepo >= %prunerepo_version`
		`BuildRequires: dnf`

		`Requires: (copr-selinux if selinux-policy-targeted)`
		`@@ -68,7 +69,7 @@`
		`Requires: mock`
		`Requires: obs-signd`
		`Requires: openssh-clients`
		`- Requires: prunerepo >= 1.19`
		`+ Requires: prunerepo >= %prunerepo_version`
		`Requires: python3-copr`
		`Requires: python3-copr-common >= 0.10.1.dev`
		`Requires: python3-copr-messaging`

backend/run/copr_prune_results.py

file modified

+8 -7

		`@@ -65,10 +65,9 @@`
		`call_copr_repo(directory=chroot_path, rpms_to_remove=rpms,`
		`logger=LOG)`
		`clean_copr(chroot_path, prune_days, verbose=True)`
		`- except Exception as err: # pylint: disable=broad-except`
		`- LOG.exception(err)`
		`- LOG.error("Error pruning chroot %s/%s/%s", username, projectdir,`
		`- sub_dir_name)`
		`+ except Exception: # pylint: disable=broad-except`
		`+ LOG.exception("Error pruning chroot %s/%s/%s", username, projectdir,`
		`+ sub_dir_name)`
schlupov commented 3 years ago Do we want to use `LOG.exception`? The backend log for pruner will be full of Traceback (most recent call last): File "/opt/copr/backend/run/copr_prune_results.py", line 62, in run_prunerepo rpms = get_rpms_to_remove(chroot_path, days=prune_days, log=LOG) File "/usr/lib/python3.9/site-packages/prunerepo/helpers.py", line 178, in get_rpms_to_remove latest_rpms = get_rpms(get_latest_packages_cmd, directory, log) File "/usr/lib/python3.9/site-packages/prunerepo/helpers.py", line 68, in get_rpms stdout = run_cmd(repoquery_cmd, log) # returns srpms as well File "/usr/lib/python3.9/site-packages/prunerepo/helpers.py", line 51, in run_cmd raise PrunerepoException("Command {} failed".format(str_cmd)) prunerepo.helpers.PrunerepoException: Command dnf repoquery --repofrompath=prunerepo_query,/var/lib/copr/public_html/ results/user/project/fedora-32-x86_64 --repo=prunerepo_query --refresh --queryformat=%{location} --quiet --setopt= skip_if_unavailable=False --latest-limit=1 failed when `repoquery` fail. Edited 3 years ago by schlupov
praiskup commented 3 years ago I didn't want to remove the exception when it already was there ..., I just de-duplicated the output.

		`LOG.info("Pruning finished for projectdir %s/%s/%s",`
		`username, projectdir, sub_dir_name)`
		`@@ -105,6 +104,11 @@`
		`self.prune_project(project_path, username, projectdir)`
		`LOG.info("--------------------------------------------")`

		`+`
		`+ LOG.info("Pruning tasks are delegated to background workers, waiting.")`
		`+ self.pool.close()`
		`+ self.pool.join()`
		`+`
		`LOG.info("Setting final_prunerepo_done for deactivated chroots")`
		`chroots_to_prune = []`
		`for chroot, active in self.chroots.items():`
		`@@ -112,10 +116,7 @@`
		`chroots_to_prune.append(chroot)`
		`self.frontend_client.post("final-prunerepo-done", chroots_to_prune)`

		`- self.pool.close()`
		`- self.pool.join()`
		`LOG.info("--------------------------------------------")`
		`- LOG.info("Pruning finished")`

		`def prune_project(self, project_path, username, projectdir):`
		`LOG.info("Going to prune %s/%s", username, projectdir)`

praiskup commented 3 years ago

The real problem is going to be fixed in:
https://pagure.io/prunerepo/pull-request/13

Closes: #1794
Closes: #1791

Metadata Update from @praiskup:
- Pull-request tagged with: release-blocker

3 years ago

zuul commented 3 years ago

Build succeeded.

csdiff-pylint : SUCCESS in 1m 39s

praiskup commented 3 years ago

This requires prunerepo 1.20, that needs to be released first ... I can do that if anyone promptly reviews the needed pull request there.

schlupov commented on line 11 of backend/run/copr_prune_results.py 3 years ago

Do we want to use LOG.exception? The backend log for pruner will be full of

Traceback (most recent call last):
File "/opt/copr/backend/run/copr_prune_results.py", line 62, in run_prunerepo
rpms = get_rpms_to_remove(chroot_path, days=prune_days, log=LOG)
File "/usr/lib/python3.9/site-packages/prunerepo/helpers.py", line 178, in get_rpms_to_remove
latest_rpms = get_rpms(get_latest_packages_cmd, directory, log)
File "/usr/lib/python3.9/site-packages/prunerepo/helpers.py", line 68, in get_rpms
stdout = run_cmd(repoquery_cmd, log)  # returns srpms as well
File "/usr/lib/python3.9/site-packages/prunerepo/helpers.py", line 51, in run_cmd
raise PrunerepoException("Command {} failed".format(str_cmd))
prunerepo.helpers.PrunerepoException: Command dnf repoquery --repofrompath=prunerepo_query,/var/lib/copr/public_html/
results/user/project/fedora-32-x86_64 --repo=prunerepo_query --refresh --queryformat=%{location} --quiet --setopt=
skip_if_unavailable=False --latest-limit=1 failed

when repoquery fail.

Edited 3 years ago by schlupov

praiskup commented on line 11 of backend/run/copr_prune_results.py 3 years ago

I didn't want to remove the exception when it already was there ..., I just de-duplicated the output.

schlupov commented 3 years ago

+1, works fine with prunerepo 1.20

praiskup commented 3 years ago

[copr-build]