#9718 Job stuck in 'pending -> testing'
Closed: Fixed 3 years ago by kevin. Opened 3 years ago by sergesanspaille.

This update:

https://bodhi.fedoraproject.org/updates/FEDORA-2021-793a5cd627

has been stuck in pending -> testing state for 9 hours. Before that it was stuck in -> stable state for 3 days.

There seems to be an issue, I can just submit a new update while removing that one, but if there's a better path, I'll happily take it.


So, we can try and figure out what happened here. ;(

I am assuming that you made a side tag, built everything in that tag and then used the 'create update from sidetag' button on bodhi? Or did you use some other workflow?

Normally things should go to testing in a few minutes, then stable (if all the gating passes) a few minutes after gating finishes, so this is definitely not normal.

The newer builds you were adding, you were just building those normally? Or in the side tag? (Although bodhi should remove the sidetag when it merges it).

I am assuming that you made a side tag, built everything in that tag and then used the 'create update from sidetag' button on bodhi?

yes

The newer builds you were adding, you were just building those normally? Or in the side tag?

in the side tag, except one that got build by someone else and I had to tag it manually

Metadata Update from @smooge:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: bodhi, medium-gain, medium-trouble, releng

3 years ago

FYI : This one is blocking LLVM 12.0.0.rc2, based on @kevin feedback, I'll probably reset the update in ~6h

What do you mean by 'reset the update' ? bodhi will not let you delete updates, or remove the last package from them, so making a new update sounds like a pretty bad plan to me. ;(

I see the update is obsolte now. Are you going to try and make a new one? Or do you want me to try and get the old one to move forward?

Yeah, I had to move forward. Don't bother handling the old one.

ok. Dug through a bunch of logs.

Mostly things look normal until:

2021-03-09 13:41:35,410 WARNI [bodhi.server][MainThread] Cannot find release associated with build: clang-12.0.0-0.3.rc2.fc35, tags: ['f35-build-side-38412']
2021-03-09 13:45:45,128 WARNI [bodhi.server][MainThread] Cannot find release associated with build: clang-12.0.0-0.3.rc2.fc35, tags: ['f35-build-side-38412']

but that seems cosmetic.

Looking further I see:

2021-03-05 08:42:05,467 INFO  [bodhi.server][MainThread] Checking against clang-12.0.0-0.1.rc1.fc35
2021-03-05 08:42:05,848 INFO  [bodhi.server][MainThread] Checking against compiler-rt-12.0.0-0.1.rc1.fc35
2021-03-05 08:42:06,244 INFO  [bodhi.server][MainThread] Checking against flang-12.0.0-0.1.rc1.fc35
2021-03-05 08:42:06,380 INFO  [bodhi.server][MainThread] Checking against flang-11.1.0-0.3.rc1.fc35
2021-03-05 08:42:06,477 INFO  [bodhi.server][MainThread] Checking against libcxxabi-12.0.0-0.1.rc1.fc35
2021-03-05 08:42:06,657 INFO  [bodhi.server][MainThread] Checking against libomp-12.0.0-0.1.rc1.fc35
2021-03-05 08:42:06,800 INFO  [bodhi.server][MainThread] Checking against lld-12.0.0-0.1.rc1.fc35
2021-03-05 08:42:06,946 INFO  [bodhi.server][MainThread] Checking against lldb-12.0.0-1.rc1.fc35
2021-03-05 08:42:07,185 INFO  [bodhi.server][MainThread] Checking against llvm-12.0.0-0.1.rc1.fc35
2021-03-05 08:42:07,381 INFO  [bodhi.server][MainThread] Checking against llvm-test-suite-12.0.0-0.1.rc1.f
c35
2021-03-05 08:42:07,893 INFO  [bodhi.server][MainThread] Checking against mlir-12.0.0-0.1.rc1.fc35
2021-03-05 08:42:08,339 INFO  [bodhi.server][MainThread] Checking against pocl-1.7-0.1.20210218gita2d016c.
fc35
2021-03-05 08:42:08,708 INFO  [bodhi.server][MainThread] Checking against polly-12.0.0-0.1.rc1.fc35
2021-03-05 08:42:09,142 INFO  [bodhi.server][MainThread] Checking against sourcetrail-2020.2.43-10.fc35
2021-03-05 08:42:09,307 INFO  [bodhi.server][MainThread] Unable to set request for FEDORA-2021-793a5cd627 to stable due to failed requirements: No checks required.
2021-03-01 19:47:06,205 INFO  [bodhi.server][MainThread] Editing update: FEDORA-2021-793a5cd627
2021-03-01 19:47:06,236 INFO  [bodhi.server][MainThread] Unpushing clang-12.0.0-0.2.rc2.fc35
2021-03-01 19:47:06,283 INFO  [bodhi.server][MainThread] Unpushing lldb-12.0.0-2.rc2.fc35
2021-03-01 19:47:06,522 INFO  [bodhi.server][MainThread] Sending mail to sguelton@redhat.com: [Fedora Update] [comment] compiler-rt-12.0.0-0.2.rc2.fc35, flang-12.0.0-0.2.rc2.fc35, and 12 more
2021-03-01 19:47:06,642 INFO  [bodhi.server][MainThread] Untagging FEDORA-2021-793a5cd627
2021-03-01 19:47:07,678 INFO  [bodhi.server][MainThread] Skipping tag that we don't know about: f35-build-side-37846
2021-03-01 19:47:08,340 INFO  [bodhi.server][MainThread] Updating test gating status of FEDORA-2021-793a5cd627
2021-03-01 19:47:09,512 INFO  [bodhi.server][MainThread] Deferring working on bugs and fetching test cases to celery
2021-03-01 19:47:09,521 INFO  [bodhi.server][MainThread] Done editing FEDORA-2021-793a5cd627

So, the problem seems to be gating tests somehow and "failed requirements: No checks required."

@bookwar any ideas on what could be going wrong here?

Although I think the underlying issue has not been resolved, this update has been obsoleted and thus the original issue is "solved".

Do we want to keep this ticket open? Or should we close it and see if it happens again?

I guess we close it...

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata