#158 Fedora CI pending for 3 hours on test which should finish in minutes
Closed 3 years ago by mvadkert. Opened 3 years ago by plautrba.

In the time of reporting, https://src.fedoraproject.org/tests/selinux/pull-request/120 Fedora CI is pending for 3 hours. It's apparently running but there's no way how to check what's going on.
It points to https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/dist-git-pipeline/job/master/491/ which shows almost full progress bar and "Started 3 hr 49 min ago"

Console output - https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/dist-git-pipeline/job/master/491/console - cycles on

[Pipeline] retry
[Pipeline] {
[Pipeline] httpRequest
HttpMethod: GET
URL: https://api.dev.testing-farm.io/v0.1/requests/5aa3ed23-0345-46ae-84ef-0164586ffcaf
Content-Type: application/json
Sending request to url: https://api.dev.testing-farm.io/v0.1/requests/5aa3ed23-0345-46ae-84ef-0164586ffcaf

Response Code: HTTP/1.1 200 OK
Response: 
{"id": "5aa3ed23-0345-46ae-84ef-0164586ffcaf", "test": {"fmf": null, "script": null, "sti": {"extra_variables": null, "playbooks": ["tests-pr.yml"], "ref": "daf4df16e8571aae112f0d3078149095dc425c77", "url": "https://src.fedoraproject.org/forks/mmalik/tests/selinux"}}, "state": "running", "environments_requested": [{"arch": "x86_64", "artifacts": [], "os": {"compose": "Fedora-Rawhide"}, "pool": null, "variables": null}], "notes": [{"level": "info", "message": "tf-sti/dispatch-1603439184-6739cbce"}], "result": null, "run": null, "created": "2020-10-23 07:46:23.354941", "updated": "2020-10-23 07:46:23.354952"}
Success code from [200‥200]
[Pipeline] }
[Pipeline] // retry
[Pipeline] echo
Testing Farm Artifacts URL: http://artifacts.dev.testing-farm.io/5aa3ed23-0345-46ae-84ef-0164586ffcaf
[Pipeline] sleep
Sleeping for 20 sec

https://api.dev.testing-farm.io/v0.1/requests/5aa3ed23-0345-46ae-84ef-0164586ffcaf

{"id": "5aa3ed23-0345-46ae-84ef-0164586ffcaf", "test": {"fmf": null, "script": null, "sti": {"extra_variables": null, "playbooks": ["tests-pr.yml"], "ref": "daf4df16e8571aae112f0d3078149095dc425c77", "url": "https://src.fedoraproject.org/forks/mmalik/tests/selinux"}}, "state": "running", "environments_requested": [{"arch": "x86_64", "artifacts": [], "os": {"compose": "Fedora-Rawhide"}, "pool": null, "variables": null}], "notes": [{"level": "info", "message": "tf-sti/dispatch-1603439184-6739cbce"}], "result": null, "run": null, "created": "2020-10-23 07:46:23.354941", "updated": "2020-10-23 07:46:23.354952"}

I haven't found any useful information why the simple test is running so long


@plaurtba seems to me it is hanging somewhere, I was able to download the artifacts from the machine

I shared the artifacts with you internally

@plautrba this are the contexts for /var/tmp/artifacts from the machine where the test run against .... trying to find which test this is hanging on

but seems the tests are running actually ...

currently the test which is running is .../selinux-policy-pam_timestamp-and-related-err

We have https://src.fedoraproject.org/tests/selinux/blob/master/f/tests-pr.yml which should run only tests affected by a PR, but it runs all tests. Why? It used to work before migration to testing farm. Please fix it ASAP - it blocks our review process

Metadata Update from @mvadkert:
- Issue assigned to mvadkert

3 years ago

We have found a solution, trying to fix it now

deploying a hot fix that will solve the problem, but seems it will require that the fork's master is in sync with the target repo master

@plautrba I will close this one and open a specific issue to the underlying problem we were workarounding

Metadata Update from @mvadkert:
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata
Attachments 1
Attached 3 years ago View Comment