#4 profile existing PUATP tests in AutoQA for input/output needs
Closed: Fixed None Opened 10 years ago by tflink.

As we start working on input and output methods for tasks, one important question is what our existing supported tests already need and use. This will likely change over time as taskbot evolves but it's a good place to start.


= Current Watchers =

== compose ==

  • rats_install (disabled since 2012-05-24)

=== Stored 'extra data' ===

rats_install:
* arch

== git-post-receive ==

Unused

== yum-repo ==

  • conflicts (disabled since 2012_10_02)
  • rats_sanity
  • repoclosure

The watcher watches for changes in repomd.xml file(s) for specifed repos. If the change occurs, then a job is created, and the repo is identified by its name (like 'rawhide') and URL.
Separate testruns are scheduled for each repo (if multiple repos changed).

=== Stored 'extra data' ===

rats_sanity + repoclosure:
* arch
* baseurl (e.g. http://download.fedoraproject.org/pub/fedora/linux/development/rawhide/x86_64/os)

== koji-bodhi ==

  • rpmlint (1 run per build, ENVR on input)
  • rpmguard (1 run per build, ENVR on input)
  • depcheck (1 run per multiple builds, list of ENVRs on input)
  • upgradepath (1 run per multiple builds, list of ENVRs on input)

Koji-Bodhi watcher operates on per-build basis. The tests which react on the events produced by the watcher are either per-build tests (rpmlint, rpmguard), or operate on a whole set of "new builds since last check" (depcheck, upgradepath).

The watcher provides a list of (e)NVRs based on changes in koji-tags. If the tests (upgradepath, depcheck) need to get information about which builds belong to which update, it is done explicitly in the test by querying Bodhi.

=== Stored 'extra data' ===

rpmlint + rpmguard:
* envr
* kojitag (e.g. f17-updates-candidate)
* arch (always 'noarch')

upgradepath:
* envr (1+)
* kojitag (e.g. f17-updates-pending)
* arch (always 'noarch')
* update (name of the update, not update_id)

depcheck:
* envr (1+)
* kojitag (e.g. f17-updates-pending)
* arch (arch dependent)
* update (name of the update, not update_id)

IMHO the kojitag is mostly useless (especially if we hook up on the Bodhi's fedmsg, see below).

= Taskbot =

== koji-bodhi ==

I believe, that (for PUAPT purposes) we could use the Bodhi's fedmsg events (most probably bodhi.update.{request, complete}.testing) to spawn the tests. The results would still be identified on NVR basis, but the results could be grouped in 'jobs' (since we'd start a job per update). I'm not sure whether we'd want to update the results (or the job) with the update_id (once it's available) or not, but I'd most deffinitely store both envr & update_title in the extradata.

If we decided on testing on per-build, instead of per-update (or both) we probably could use some of the buildsys's messages (but I was quite unable to decipher what they mean, so no concrete suggestion here).

Thanks for getting this taken care of.

Actually, the current proof-of-concept for taskbot is using buildsys messages to trigger rpmlint jobs so we're already doing that. I had been planning to extend that to bodhi messages once we got to checks that took updates for input.

The major question I have is whether we need to be worrying about zmq's non-guaranteed message delivery or not, but that investigation is ongoing and is being tracked in #2.

Login to comment on this ticket.

Metadata