Look over Taskotron output over a period of time (several days, spread out) to make sure that the output we expect exists and is correct:
== Output Types to Check == Bodhi Comments Output in ResultsDB
This should be put off until shortly before release
To clarify a bit, go through all the output from our tasks (rpmlint, depcheck, upgradepath - ignore examplebodhi) and make sure that:
Bodhi comments are being reported to a [[https://taskotron-stg.fedoraproject.org/fakefedorainfra/boji/|fake_fedorainfra instance in stg]]. The comments url is a bit broken until I do a new build with D157 but everything is still accessible if you do a bit of manual url hackery.
I have found some problems so far, I have created tickets this ticket and this ticket.
Frontend for resultsdb is missing information about package (item) in result index, for example [[ http://resultsdb-stg.cloud.fedoraproject.org/results?testcase_name=upgradepath | here ]]. I think that there are two possible solutions for this - either add "item" (and item type) field into this index, or change tasks to show meaningful messages in Summary field, like [[ http://resultsdb-stg.cloud.fedoraproject.org/results?testcase_name=rpmlint | rpmlint does ]].
Another thing that is missing is link to log output in result detail. I have already [[ https://bitbucket.org/fedoraqa/resultsdb_frontend/commits/b84171fb279bcc305a00896fd3a0200f7386ccd7 | created the patch ]] for resultsdb_frontend.
Also, upgradepath test is failing with:
usage: runtask [-h] [--debug] KOJI_TAG runtask: error: Koji tags ending in "-testing" are not supported at the moment
is it bug or is it desired behaviour?
And there are still problems with space on disk, for example [[ https://taskotron-stg.fedoraproject.org/taskmaster/builders/x86_64/builds/6876/steps/runtask/logs/stdio | here ]].
! In #222#10, @jsedlak wrote: Also, upgradepath test is failing with: usage: runtask [-h] [--debug] KOJI_TAG runtask: error: Koji tags ending in "-testing" are not supported at the moment is it bug or is it desired behaviour?
! In #222#10, @jsedlak wrote: Also, upgradepath test is failing with:
It's sub-optimal but expected behavior. The messages don't show up in resultsdb
Yeah, #257 should address that issue. I've cleaned off the disks manually in the mean time
! In #222#9, @jsedlak wrote: Frontend for resultsdb is missing information about package (item) in result index, for example [[ http://resultsdb-stg.cloud.fedoraproject.org/results?testcase_name=upgradepath | here ]]. I think that there are two possible solutions for this - either add "item" (and item type) field into this index, or change tasks to show meaningful messages in Summary field, like [[ http://resultsdb-stg.cloud.fedoraproject.org/results?testcase_name=rpmlint | rpmlint does ]].
It's probably worth putting something like "upgradepath for f20-updates" in the summary field for upgradepath and a similar change for depcheck. All of our tasks need to be updated to use the checkname field for TAP output, so this could be done at the same time.
Ah, I had forgotten about that. Will get a new build out to stg today.
By the way, I'm seeing a lot of depcheck errors, but it seems that it is fixed in D171.
! In #222#14, @jsedlak wrote: By the way, I'm seeing a lot of depcheck errors, but it seems that it is fixed in D171.
stg has been updated with latest tasks, that should be going away now
The last thing is, it's weird that comments in fedora infrastructure are showing that "depcheck PASSED on x86_64", but link to log is to i686 build machine, for example [[ https://taskotron-stg.fedoraproject.org/fakefedorainfra/boji/comment/360 | in this comment ]].
! In #222#16, @jsedlak wrote: The last thing is, it's weird that comments in fedora infrastructure are showing that "depcheck PASSED on x86_64", but link to log is to i686 build machine, for example [[ https://taskotron-stg.fedoraproject.org/fakefedorainfra/boji/comment/360 | in this comment ]].
depcheck is written such that i386 and x86_64 runs are done on the same machine. Until yesterday, depcheck was being scheduled on both i386 and x86_64 builders so this is expected for results before yesterday. i386 results on x86_64 machines are expected going forward
OK, I haven't found other problems. I cannot compare upgradepath (because every upgradepath run fails), it seems that depcheck has the same output as AutoQA's depcheck. AutoQA's rpmlint is testing fc21 packages (rawhide I guess), but taskotron's isn't.
I don't think that this task is still relevant, so I'm marking it as resolved.
Login to comment on this ticket.