PR#61: two new parameters in schedule.conf.sample - fedora-qa/fedora_openqa

fedora-qa / fedora_openqa

#61 two new parameters in schedule.conf.sample

Opened 7 years ago by michelmno. Modified 3 years ago

fedora-qa/ michelmno/fedora_openqa new_schedule_conf_parms into main

Do not schedule update tests for ppc64(BE) after F28

Michel Normand • 6 years ago

c66f896

Add compose url location in warning log

Michel Normand • 6 years ago

50156bd

New 'not_scheduled_events' list in schedule.conf.sample

Michel Normand • 6 years ago

e890ecb

new 'update_arches' in schedule.conf.sample

Michel Normand • 6 years ago

e76c99d

fedora_openqa/config.py

file modified

		`@@ -71,6 +71,9 @@`
		`CONFIG.set('consumers', 'test_wiki_hostname', 'stg.fedoraproject.org')`
		`# As the default location is public staging, keep this as false`
		`CONFIG.set('consumers', 'test_wiki_report', 'false')`
		`+ # list of arches to process on 'update' type message`
		`+ CONFIG.set('consumers', 'update_arches', 'x86_64')`
		`+ CONFIG.set('consumers', 'not_scheduled_events', '')`

		`CONFIG.read('/etc/fedora-openqa/schedule.conf')`
		`CONFIG.read('{0}/.config/fedora-openqa/schedule.conf'.format(os.path.expanduser('~')))`

fedora_openqa/consumer.py

file modified

+67 -8

		`@@ -50,6 +50,20 @@`
		`"""openQA base URL."""`
		`return CONFIG.get('consumers', 'prod_oqa_baseurl')`

		`+ @property`
		`+ def openqa_update_arches(self):`
		`+ """openQA list of supported arches for update msg."""`
		`+ archstr = CONFIG.get('consumers', 'update_arches')`
		`+ arches = archstr.replace(' ','').split(',')`
		`+ return arches`
		`+`
		`+ @property`
		`+ def openqa_not_scheduled_events(self):`
		`+ """openQA list of events not to be scheduled."""`
		`+ strcomp = CONFIG.get('consumers', 'not_scheduled_events')`
		`+ if strcomp:`
		`+ strcomp = strcomp.replace(' ','').split(',')`
		`+ return strcomp`

		`class OpenQAStagingConsumer(fedmsg.consumers.FedmsgConsumer):`
		`"""Staging mixin for both wiki and ResultsDB reporters so they can`
		`@@ -67,6 +81,20 @@`
		`"""openQA base URL."""`
		`return CONFIG.get('consumers', 'stg_oqa_baseurl')`

		`+ @property`
		`+ def openqa_update_arches(self):`
		`+ """openQA list of supported arches for update msg."""`
		`+ archstr = CONFIG.get('consumers', 'update_arches')`
		`+ arches = archstr.replace(' ','').split(',')`
		`+ return arches`
		`+`
		`+ @property`
		`+ def openqa_not_scheduled_events(self):`
		`+ """openQA list of events not to be scheduled."""`
		`+ strcomp = CONFIG.get('consumers', 'not_scheduled_events')`
		`+ if strcomp:`
		`+ strcomp = strcomp.replace(' ','').split(',')`
		`+ return strcomp`

		`class OpenQATestConsumer(fedmsg.consumers.FedmsgConsumer):`
		`"""Test mixin for both wiki and ResultsDB reporters so they can`
		`@@ -85,6 +113,20 @@`
		`"""openQA base URL."""`
		`return CONFIG.get('consumers', 'test_oqa_baseurl')`

		`+ @property`
		`+ def openqa_update_arches(self):`
		`+ """openQA list of supported arches for update msg."""`
		`+ archstr = CONFIG.get('consumers', 'update_arches')`
		`+ arches = archstr.replace(' ','').split(',')`
		`+ return arches`
		`+`
		`+ @property`
		`+ def openqa_not_scheduled_events(self):`
		`+ """openQA list of events not to be scheduled."""`
		`+ strcomp = CONFIG.get('consumers', 'not_scheduled_events')`
		`+ if strcomp:`
		`+ strcomp = strcomp.replace(' ','').split(',')`
		`+ return strcomp`

		`# SCHEDULER CLASSES`

		`@@ -112,6 +154,12 @@`
		`self._log('info', "Not scheduling jobs for modular compose {0}".format(compstr))`
		`return`

		`+ for compname in self.openqa_not_scheduled_events:`
		`+ if compname in compstr:`
		`+ self._log('warning', "Not scheduling jobs for {0} compose "`
		`+ "{1}".format(compname, compstr))`
		`+ return`
		`+`
		`if 'FINISHED' in status and location:`
		`# We have a complete pungi4 compose`
		`self._log('info', "Scheduling openQA jobs for compose {0}".format(compstr))`
		`@@ -145,6 +193,11 @@`
		`# to None, means 'run all tests'.`
		`flavors = []`

		`+ if 'update' in self.openqa_not_scheduled_events:`
		`+ self._log('warning', "Not scheduling jobs for update event "`
		`+ "{0}".format(advisory))`
		`+ return`
		`+`
		`# if update is critpath, always run all update tests`
		`if critpath and advisory and version and idpref == 'FEDORA':`
		`self._log('info', "Scheduling openQA jobs for critical path update {0}".format(advisory))`
		`@@ -182,14 +235,20 @@`
		`# being None here results in our desired behaviour (jobs will`
		`# be created for all flavors)`
		`# pylint: disable=no-member`
		`- jobs = schedule.jobs_from_update(`
		`- advisory, version, flavors=flavors, openqa_hostname=self.openqa_hostname, force=True)`
		`- if jobs:`
		`- self._log('info', "openQA jobs run on update {0}: "`
		`- "{1}".format(advisory, ' '.join(str(job) for job in jobs)))`
		`- else:`
		`- self._log('warning', "No openQA jobs run!")`
		`- return`
		`+ for onearch in self.openqa_update_arches:`
		`+ # f28 is the last release to support ppc64 (BE)`
		`+ if onearch == 'ppc64' and str(version).isdigit() and int(version) > 28:`
		`+ self._log('info',`
		`+ "No update tests for {0} and F{1}".format(onearch, version))`
		`+ continue`
		`+ jobs = schedule.jobs_from_update(`
		`+ advisory, version, flavors=flavors, openqa_hostname=self.openqa_hostname, arch=onearch, force=True)`
		`+ if jobs:`
		`+ self._log('info', "openQA jobs run on update {0} {1}: "`
		`+ "{2}".format(advisory, onearch, ' '.join(str(job) for job in jobs)))`
		`+ else:`
		`+ self._log('warning', "No openQA jobs run for %s", onearch)`
		`+ return`

		`self._log('debug', "Finished")`
		`return`

fedora_openqa/schedule.py

file modified

+1 -1

		`@@ -296,7 +296,7 @@`
		`images = [img for img in images if img[1] in arches]`

		`if len(images) == 0:`
		`- raise TriggerException("Compose found, but no available images")`
		`+ raise TriggerException("Compose found, but no available images at {0}".format(location))`
		`jobs = []`
		`univs = {}`

schedule.conf.sample

file modified

		`@@ -22,6 +22,14 @@`
		`# Settings in this section affect the behaviour of the Fedmsg consumers`
		`# in consumer.py`

		`+ # list of arches to process on 'update' type message`
		`+ # default in fedora_openqa/config.py`
		`+ update_arches: x86_64, ppc64le, ppc64`
		`+`
		`+ # list of fedmsg events not to be scheduled`
		`+ # default is an empty list`
		`+ #not_scheduled_events: update, Fedora-Cloud, Fedora-Modular, Fedora-Atomic, Fedora-Docker, Fedora-Rawhide`
		`+`
		`# openQA host to query for results (for production reporters) or`
		`# create jobs on (for production scheduler)`
		`prod_oqa_hostname: openqa.fedoraproject.org`

michelmno commented 7 years ago

two new parameters in schedule.conf.sample as detailed in two commits below
and a third commit only for warning log.

Add compose url location in warning log
new 'not_scheduled_events' list in schedule.conf.sample
new 'update_arches' in schedule.conf.sample

adamwill commented 7 years ago

This seems about as good as any other way of doing it, yeah. Could you please cover this in the test suite, though? Tests are important. :) Thanks!

adamwill commented 7 years ago

@michelmno ping?

michelmno commented 7 years ago

I saw the previous request to add tests, I will do it, but do not have yet a target date for delivery.

rebased onto 0ab348a

7 years ago

adamwill commented 6 years ago

@michelmno ping? :)

michelmno commented 6 years ago

Hello Adam,
I completely stopped to work on the tests associated to this PR, because my python skill is not sufficient
to have tests to work on a PowerPC machine
to write tests that work for those new parameters

Nevertheless we are using daily those 3 commits in our working branch without obvious failure.

adamwill commented 6 years ago

OK, if it's significant to you, I'll try and find a bit of time to see if I can write some tests. I really want to keep the test coverage high.

rebased onto d01941a

6 years ago

rebased onto 15b2ae3

6 years ago

rebased onto e76c99d

6 years ago

1 new commit added

Do not schedule update tests for ppc64(BE) after F28

6 years ago

zuul commented 3 years ago

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.