#8148 Seems ci-results-updater is not ingesting CI pipeline messages to resultsdb
Closed: Fixed 2 months ago by pingou. Opened 2 months ago by mvadkert.

As far as @kevin and I knew.. loopabull had nothing to do with the CI pipeline. Did this change and what would we look for?

I was wrong, loopabull is not involved. This is the service:

So is this working? Or we still need to dig more?

Metadata Update from @kevin:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: ci

2 months ago

I'm still seeing problems. From my perspective -- as an ignorant end user with no clue of what greenwave, loopabull, or any of those terms are -- the issue is that between [1] and [2] the test status changed from "tests failed" to "test results missing". I am still seeing "missing" on [3] and [4], but those are only 2 and 3 hours old respectively, and in the past it has taken up to four hours to actually get results - so I can't be sure if the problem still exists, or if it's just the usual slowness from before.

[1] https://bodhi.fedoraproject.org/updates/FEDORA-2019-bfff3112b5
[2] https://bodhi.fedoraproject.org/updates/FEDORA-2019-8d97c8b9e6

[3] https://bodhi.fedoraproject.org/updates/FEDORA-2019-8dce0aaefc
[4] https://bodhi.fedoraproject.org/updates/FEDORA-2019-4670e94e9b

@pingou yeah sorry for the confusion with loopabull, this is about the regular builds ... so ci-resultsdb-updater ....

I deleted the confusing comments and stand corrected

The links from the description are still relevant though ... the last resultsdb results were created 4 days ago +-

the last resultsdb results were created 4 days ago +-

Hm that sounds like a different problem then, let me check this

Ok rdbsync had failed to restart after the last reboot.

I've restarted the service and it seems messages are back in resultsdb: https://taskotron.fedoraproject.org/resultsdb/results?testcases=org.centos.prod.ci.pipeline.allpackages-build.complete :)

Metadata Update from @pingou:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

2 months ago

@pingou is there a way we could monitor these services? We are about to setup prometheus in upstream to monitor some critical parts of the infra ...

@mvadkert I use this script https://pagure.io/fedora-ci/monitor-gating to monitor the entire pipeline, it's a start but not yet enough.

This is definitely something we'll want to improve/fix with automation and proper reporting.

Login to comment on this ticket.