PR#93: Move back our Zuul CI jobs from the zuul-fedora-jobs repository. - fedora-infra/ansible

fedora-infra / ansible

#93 Move back our Zuul CI jobs from the zuul-fedora-jobs repository.

Merged 3 years ago by nphilipp. Opened 3 years ago by nphilipp.

fedora-infra/ nphilipp/ansible master--move-back-zuul-jobs into master

Fix some problems found by ansible-review.

Pierre-Yves Chibon • 3 years ago

fb4bc2c

Zuul: Verify that local jobs are namespaced

Nils Philippsen • 3 years ago

42170b4

Zuul: use repo-local job playbooks again

Nils Philippsen • 3 years ago

97be9cd

.zuul.yaml

file modified

+22 -1

		`@@ -1,5 +1,26 @@`
		`---`
		`+ - job:`
		`+ name: fi-ansible--zuul-jobs-verify-namespace`
		`+ run: ci/zuul-jobs-verify-namespace.yaml`
		`+ files:`
		`+ - ^\.zuul\.yaml$`
		`+`
		`+ - job:`
		`+ name: fi-ansible--ansible-review-base`
		`+ pre-run: ci/ansible-review-base.yaml`
		`+`
		`+ - job:`
		`+ name: fi-ansible--ansible-review-everything`
		`+ parent: fi-ansible--ansible-review-base`
		`+ run: ci/ansible-review-everything.yaml`
		`+`
		`+ - job:`
		`+ name: fi-ansible--ansible-review-diff`
		`+ parent: ansible-review-base`
fbo commented 3 years ago Should it be fi-ansible--ansible-review-base ?
nphilipp commented 3 years ago Good catch!
		`+ run: ci/ansible-review-diff.yaml`
		`+`
		`- project:`
		`check:`
		`jobs:`
		`- - ansible-review-diff`
		`+ - fi-ansible--zuul-jobs-verify-namespace`
		`+ - fi-ansible--ansible-review-diff`

ci/ansible-review-base.yaml

file added

+14

		`@@ -0,0 +1,14 @@`
		`+ - hosts: all`
		`+ tasks:`
		`+ - name: Install ansible-review`
		`+ package:`
		`+ name: python3-ansible-review`
		`+ state: latest`
		`+`
		`+ - name: Install invocation script`
		`+ copy:`
		`+ src: ansible-review-filtered.sh`
		`+ dest: /usr/local/bin/ansible-review-filtered.sh`
		`+ owner: root`
		`+ group: root`
		`+ mode: '0755'`

ci/ansible-review-diff.yaml

file added

		`@@ -0,0 +1,8 @@`
		`+ - hosts: all`
		`+ tasks:`
		`+ - name: Run ansible-review on the changes in the PR`
		`+ shell: >`
		`+ git diff origin/{{ zuul.branch\|quote }}...{{ zuul.branch\|quote }}`
		`+ \| ansible-review-filtered.sh`
		`+ args:`
		`+ chdir: '{{ zuul.project.src_dir }}'`

ci/ansible-review-everything.yaml

file added

+11

		`@@ -0,0 +1,11 @@`
		`+ - hosts: all`
		`+ tasks:`
		`+ - name: Run ansible-review on the playbooks and roles`
		`+ shell: >`
		`+ exitcode=0;`
		`+ while read pbrulefile; do`
		`+ ansible-review-filtered.sh "$pbrulefile" \|\| exitcode=1;`
		`+ done <<<$(find playbooks roles -name ".yml" -o -name ".yaml");`
		`+ exit "$exitcode"`
		`+ args:`
		`+ chdir: '{{ zuul.project.src_dir }}'`

ci/ansible-review-filtered.sh

file added

+10

		`@@ -0,0 +1,10 @@`
		`+ #!/bin/bash`
		`+`
		`+ # Remove warnings on stderr about missing configuration and used default rules.`
		`+`
		`+ SEDSCRIPT='`
		`+ 0,/^WARN: No configuration file found at/{/^WARN: No configuration file found at/d;};`
		`+ 1,/^WARN: Using example .* found at/{/^WARN: Using example .* found at/d;};`
		`+ '`
		`+`
		`+ exec ansible-review "$@" 2> >(sed -e "$SEDSCRIPT" >&2)`

ci/zuul-jobs-verify-namespace.py

file added

+36

		`@@ -0,0 +1,36 @@`
		`+ #!/usr/bin/env python3`
		`+`
		`+ import sys`
		`+ import traceback`
		`+`
		`+ import yaml`
		`+`
		`+`
		`+ errors_found = False`
		`+`
		`+ for zuul_yaml_file in sys.argv[1:]:`
		`+ print(f"Processing {zuul_yaml_file!r}...", flush=True)`
		`+ with open(zuul_yaml_file, "r") as yaml_stream:`
		`+ for doc_no, yaml_doc in enumerate(yaml.safe_load_all(yaml_stream), 1):`
		`+ print(f"Verifying YAML doc #{doc_no}...", flush=True)`
		`+ yaml_doc_errors_found = False`
		`+ try:`
		`+ for item in yaml_doc:`
		`+ if "job" in item:`
		`+ job = item["job"]`
		`+ if not job["name"].startswith("fi-ansible--"):`
		`+ yaml_doc_errors_found = errors_found = True`
		`+ print(`
		`+ f"Locally defined Zuul job {job['name']!r} must be named"`
		`+ " 'fi-ansible-...'"`
		`+ )`
		`+ except Exception:`
		`+ yaml_doc_errors_found = errors_found = True`
		`+ traceback.print_exc()`
		`+`
		`+ if yaml_doc_errors_found:`
		`+ print(f"Verification failed.", flush=True)`
		`+ else:`
		`+ print(f"Verification passed.", flush=True)`
		`+`
		`+ sys.exit(errors_found)`

ci/zuul-jobs-verify-namespace.yaml

file added

+19

		`@@ -0,0 +1,19 @@`
		`+ - hosts: all`
		`+ tasks:`
		`+ - name: Install Python YAML package`
		`+ package:`
		`+ name: python3-pyyaml`
		`+ state: latest`
		`+`
		`+ - name: Install invocation script`
		`+ copy:`
		`+ src: zuul-jobs-verify-namespace.py`
		`+ dest: /usr/local/bin/zuul-jobs-verify-namespace.py`
		`+ owner: root`
		`+ group: root`
		`+ mode: '0755'`
		`+`
		`+ - name: Verify that local Zuul jobs use our namespace`
		`+ command: zuul-jobs-verify-namespace.py .zuul.yaml`
		`+ args:`
		`+ chdir: '{{ zuul.project.src_dir }}'`

playbooks/manual/copr/copr-backend-upgrade.yml

file modified

-1

		`@@ -29,7 +29,6 @@`
		`dnf: state=latest`
		`name=createrepo_c`
		`enablerepo=updates-testing`
		`- when:`
		`when: ansible_facts.packages['createrepo_c'][0].version is version('0.15.5', '<')`

		`handlers:`

nphilipp commented 3 years ago

This is a follow-up to #54 and prefixes job names with fi-ansible-- to distinguish them from the (un-namespaced) jobs in the zuul-fedora-jobs repository. Additionally, it fixes a bug in a playbook that was found in our tests in the original PR and verifies that local Zuul jobs comply with the naming convention.

Fixes: fedora-infrastructure#8932

rebased onto c58d9a24d4daf64beaf10ef16110a5c6cea4f472

3 years ago

zuul commented 3 years ago

Build succeeded.

fi-ansible--ansible-review-diff : SUCCESS in 1m 17s

zuul commented 3 years ago

Build succeeded.

fi-ansible--ansible-review-diff : SUCCESS in 1m 28s

1 new commit added

Zuul: Verify that local jobs are namespaced

3 years ago

7 new commits added

Zuul: Verify that local jobs are namespaced
Fix some problems found by ansible-review.
Zuul: Compartmentalize ansible-review crashes
Zuul: Filter initial warnings of ansible-review
Zuul: Run ansible-review on changes in PRs
Zuul: Rename job and split out shared actions
Add basic zuul configuration to run ansible-review

3 years ago

nphilipp commented 3 years ago

recheck

zuul commented 3 years ago

Build succeeded.

fi-ansible--ansible-review-diff : SUCCESS in 1m 36s

7 new commits added

Zuul: Verify that local jobs are namespaced
Fix some problems found by ansible-review.
Zuul: Compartmentalize ansible-review crashes
Zuul: Filter initial warnings of ansible-review
Zuul: Run ansible-review on changes in PRs
Zuul: Rename job and split out shared actions
Add basic zuul configuration to run ansible-review

3 years ago

7 new commits added

Zuul: Verify that local jobs are namespaced
Fix some problems found by ansible-review.
Zuul: Compartmentalize ansible-review crashes
Zuul: Filter initial warnings of ansible-review
Zuul: Run ansible-review on changes in PRs
Zuul: Rename job and split out shared actions
Add basic zuul configuration to run ansible-review

3 years ago

zuul commented 3 years ago

Build succeeded.

fi-ansible--ansible-review-diff : SUCCESS in 1m 24s

7 new commits added

Zuul: Verify that local jobs are namespaced
Fix some problems found by ansible-review.
Zuul: Compartmentalize ansible-review crashes
Zuul: Filter initial warnings of ansible-review
Zuul: Run ansible-review on changes in PRs
Zuul: Rename job and split out shared actions
Add basic zuul configuration to run ansible-review

3 years ago

zuul commented 3 years ago

Build succeeded.

fi-ansible--ansible-review-diff : SUCCESS in 1m 20s

7 new commits added

Zuul: Verify that local jobs are namespaced
Fix some problems found by ansible-review.
Zuul: Compartmentalize ansible-review crashes
Zuul: Filter initial warnings of ansible-review
Zuul: Run ansible-review on changes in PRs
Zuul: Rename job and split out shared actions
Add basic zuul configuration to run ansible-review

3 years ago

zuul commented 3 years ago

Build succeeded.

fi-ansible--ansible-review-diff : SUCCESS in 1m 34s

7 new commits added

Zuul: Verify that local jobs are namespaced
Fix some problems found by ansible-review.
Zuul: Compartmentalize ansible-review crashes
Zuul: Filter initial warnings of ansible-review
Zuul: Run ansible-review on changes in PRs
Zuul: Rename job and split out shared actions
Add basic zuul configuration to run ansible-review

3 years ago

zuul commented 3 years ago

Build succeeded.

fi-ansible--ansible-review-diff : SUCCESS in 1m 24s

7 new commits added

Zuul: Verify that local jobs are namespaced
Fix some problems found by ansible-review.
Zuul: Compartmentalize ansible-review crashes
Zuul: Filter initial warnings of ansible-review
Zuul: Run ansible-review on changes in PRs
Zuul: Rename job and split out shared actions
Add basic zuul configuration to run ansible-review

3 years ago

7 new commits added

Zuul: Verify that local jobs are namespaced
Fix some problems found by ansible-review.
Zuul: Compartmentalize ansible-review crashes
Zuul: Filter initial warnings of ansible-review
Zuul: Run ansible-review on changes in PRs
Zuul: Rename job and split out shared actions
Add basic zuul configuration to run ansible-review

3 years ago

zuul commented 3 years ago

Build succeeded.

fi-ansible--zuul-jobs-verify-namespace : SUCCESS in 1m 13s
fi-ansible--ansible-review-diff : SUCCESS in 1m 24s

zuul commented 3 years ago

Build succeeded.

fi-ansible--zuul-jobs-verify-namespace : SUCCESS in 1m 30s
fi-ansible--ansible-review-diff : SUCCESS in 1m 22s

zuul commented 3 years ago

Build succeeded.

fi-ansible--zuul-jobs-verify-namespace : SUCCESS in 1m 17s
fi-ansible--ansible-review-diff : SUCCESS in 1m 23s

rebased onto 411f3773dcea49d78bbbec8f7f6f05ccbdc2e5a6

3 years ago

zuul commented 3 years ago

Build succeeded.

fi-ansible--zuul-jobs-verify-namespace : SUCCESS in 1m 13s
fi-ansible--ansible-review-diff : SUCCESS in 1m 46s

zuul commented 3 years ago

@nphilipp hi, let me know when I could delete https://pagure.io/fedora-zuul-jobs/blob/master/f/zuul.d/fedora-infra-jobs.yaml and release the jobs names :)

nphilipp commented 3 years ago

@zuul (@fbo? :wink:) Let me submit a PR to fedora-zuul-jobs to remove these jobs, depending on this one. This should work, shouldn't it?

fbo commented 3 years ago

yes it's me :) (forgot to un-login as zuul)

Yes you can do this. But well I think the depends-on won't have any specific effect, excepted that Zuul will prevent the fedora-zuul-job's PR to merge if this one is not merged yet. Also it also give a hint to humans about the cross repo PR dependency between both PRs.

This could be done the other way too, but assuming this PR is refactored first to use the final job names (the one defined in fedora-zuul-jobs).

But both way are valid :)

nphilipp commented 3 years ago

But well I think the depends-on won't have any specific effect, excepted that Zuul will prevent the fedora-zuul-job's PR to merge if this one is not merged yet.

This is what I wanted to accomplish. :wink: But first I have to wait on Pagure creating my fork of fedora-zuul-jobs.

rebased onto 90db1f716ea1e09eb81d3dd8633eef3f2b99de00

3 years ago

zuul commented 3 years ago

Build succeeded.