#141 "no tests results found" update status not updated after tests are run
Closed 4 years ago Opened 4 years ago by decathorpe.

I submitted two updates to fedora {f27,f26}-testing about 13 days ago:

Both are still blocked from being pushed to stable because "no test results found", but browsing the "Automated Tests" tab shows test results, and manually querying resultdb also shows that all tests were indeed run and all checks passed:

It looks like the "no test results found" status for those updates just was never updated once the test results were available.

I previously reported this issue against bodhi here, but was told that this is an issue in greenwave.


Hi @decathorpe. :)

Try out this python script, which poses the same query to greenwave that bodhi does.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
#!/usr/bin/env python
""" Ask a question of greenwave.  """
import pprint
import sys

import requests

for nvr in ['golang-github-xtaci-smux-1.0.7-1.fc27']:
    url = (
    'https://greenwave-web-greenwave.app.os.fedoraproject.org/'
    'api/v1.0/decision')
    payload = dict(
    #verbose=True,
    decision_context='bodhi_update_push_stable',
    product_version='fedora-27',
    subject=[{'item': nvr, 'type': 'koji_build'}],
    )
    response = requests.post(url, json=payload, verify=False)
    print("-" * 40)
    print(nvr, response, response.status_code)
    data = response.json()
    print(pprint.pformat(data))

When I run it, I get this output:

('golang-github-xtaci-smux-1.0.7-1.fc27', <Response [200]>, 200)
{u'applicable_policies': [u'taskotron_release_critical_tasks_for_stable'],
 u'policies_satisfied': False,
 u'summary': u'no test results found',
 u'unsatisfied_requirements': [{u'item': {u'item': u'golang-github-xtaci-smux-1.0.7-1.fc27',
                      u'type': u'koji_build'},
                u'scenario': None,
                u'testcase': u'dist.rpmdeplint',
                u'type': u'test-result-missing'}]}

Which shows that greenwave is requiring the dist.rpmdeplint test to run, but it cannot find a result for it (neither pass nor failure).

You posted some queries to taskotron above. Take one, and let's filter it down to look for only dist.rpmdeplint results.

So, then we bounce you along. It seems like there really is no rpmdeplint result for golang-github-xtaci-smux-1.0.7-1.fc27. @kparal, where should we refer @decathorpe about this?


In the meantime, @decathorpe, you should be able to get your update moving by "waiving" the absence of that test result.

$ waiverdb-cli -p "fedora-27" -t dist.rpmdeplint -s '{"item": "golang-github-xtaci-smux-1.0.7-1.fc27", "type": "koji_build"}' -c "I dunno what's wrong, but I really want to ship my update."

All that aside, there is something that ends up being misleading about the "no test results found" string. Really, greenwave found some test results, it's just that none of them applied to the list of tests that it required. It should be possible to make that more clear on greenwave's side.

Ok, Thanks! However: Running that command doesn't work, I get this error message:

Error: The config option "resultsdb_api_url" is required

There's no information on where to set this settings key (I'm guessing in /etc/waiverdb/client.conf?), but I have no idea which value I should put there.

Hello @decathorpe.
You can find how to configure "resultsdb_api_url" at the end of this section:
https://fedoraproject.org/wiki/Package_update_HOWTO#Handling_feedback_from_automated_tests

Basically, it is the URL of the instance of resultsdb.
So I guess for you it will be something like:
resultsdb_api_url=https://taskotron.fedoraproject.org/resultsdb_api/api/v2.0

Whoops. The resultsdb_api_url on the wiki page there was incorrect (for a few weeks!) It should be fixed now.

So, then we bounce you along. It seems like there really is no rpmdeplint result for golang-github-xtaci-smux-1.0.7-1.fc27. @kparal, where should we refer @decathorpe about this?

Recently our resultsdb instance hasn't been able to keep up with the load and is often unresponsive, which means tests are unable to post results. So it's quite common for results to be missing. We will need to do something about it, unfortunately I don't know what and who can do it. This is especially problematic combined with the fact that we can't reschedule tests to be run again. I can only recommend @decathorpe to test dependencies manually and submit a waiver, so that he can push to stable updates. I don't have anything better at the moment, sorry :/

It's okay, @kparal.

Let's get Bodhi to stop displaying results directly from resultsdb as a first step (so that the 1000 users don't all query resultsdb themselves).

Today, greenwave supplies the list of all results it saw (even the "irrelevant" ones) in its API response to Bodhi, if Bodhi passes verbose=True in the question. Bodhi could cache this and display it to users, which I expect would greatly reduce the load on resultsdb.

/cc @bowlofeggs

There's a Bodhi ticket where I've discussed having Bodhi cache the test results in its DB:

https://github.com/fedora-infra/bodhi/issues/2138

OK, after adding the api endpoint to the .conf file, filing the waivers worked, and I guess now I have to wait until bodhi updates the status.

Small suggestion: Maybe the default resultdb API path should be added to the default configuration, so not everybody has to find that wiki page and add the thing manually?

Small suggestion: Maybe the default resultdb API path should be added to the default configuration, so not everybody has to find that wiki page and add the thing manually?

Yeah you are totally right. Ralph did fix this in waiverdbPR#144 but it hasn't made it into a Fedora update just yet. Sorry about that.

OK - there's a Bodhi change to make about increasing the frequency of the sync from Greenwave (best when driven by the message bus).

There's another bit in here that will live on in #145.

Metadata Update from @ralph:
- Issue status updated to: Closed (was: Open)

4 years ago

Yeah you are totally right. Ralph did fix this in waiverdbPR#144 but it hasn't made it into a Fedora update just yet. Sorry about that.

It's nice to see that this is already done - I didn't know that when I stated my suggestion above.

And thanks for your help with troubleshooting my issue :)

No problem. Thanks for your patience (and for pursuing the report!)

Login to comment on this ticket.

Metadata