#1700 weird failure for thunderbird src.rpm uploads
Closed: Fixed 3 years ago by praiskup. Opened 3 years ago by praiskup.


It seems that the tarball is re-downloaded from frontend several times,
always afeter 'git commit' failure. That is though done through
calling dist-git => rpkg => GitPython tools. See several patterns:

commit
Running: git commit -m automatic import of thunderbird -a
Failed to execute command.

Metadata Update from @praiskup:
- Issue priority set to: High

3 years ago

Metadata Update from @praiskup:
- Issue assigned to frostyx

3 years ago

FTR, we discussed the issue off-list with @xhorak,
and I cannot find a reproducer.

I tried to download and submit the exact SRPM that @xhorak used in the original build and the import succeeded. The overall build failed because of an unrelated error but it was after 20 minutes of building the RPM package itself. SRPM phase was just fine.

Then I tried several builds from SRPMs that I found in koji and after adjusting the build timeout, they all succeeded.

https://copr.fedorainfracloud.org/coprs/frostyx/thunderbird/builds/

This issue happened the day we peaked our queue at the all-time max 20k builds, so my guess is that some of our services were temporarily unavailable and the connection somehow died.

@xhorak, did you try to resubmit the package since?

Hmm, there's rpmbuild --nodeps --rebuild %{SOURCE50} ... rpmbuild is claimed to be non-reentrant, so you shouldn't execute rpm* binaries from rpmbuild process (that's what Copr does through Mock). Not sure if this breaks the build itself, or only the fact that the source RPM is part of another source RPM. @frostyx, wdyt?

I mean, the rpmbuild command is executed inside %build.

@praiskup, that may very well be it. The thunderbird-78.8.1-1.fc33.src.rpm that I tried previously and it succeeded to build, doesn't do such shenanigans in the %build phase.

Well, one thing is that this is almost for sure unsupported RPM case,
it doesn't cause real-life troubles elsewhere (Koji/Brew, etc). I'm not
sure if we want to fix that.... but...

What is happening there is that we silently fail to import that package,
and we don't report the issue back to the frontend. It means that the
the task stays in queue and copr-dist-git re-tries to import it again ...
and again ... till some situation breaks this infinite loop (I'm not actually
sure how that break happens).

Can you at least try to debug what is going on there? And perhaps
fix the logging and break the loop explicitly? I'm motivated enough
to find a work-around later if we know the real reason for import
failure...

Weird, so is this FE => DG downloading problem? Or client => FE uploading problem?

Metadata Update from @praiskup:
- Issue assigned to praiskup (was: frostyx)

3 years ago

FTR, fedpkg import breaks git directory in a weird way, I reported this:
https://bugzilla.redhat.com/show_bug.cgi?id=1963954

Todd Zullinger 2021-05-25 00:02:43 CEST

I filed an issue and submitted a fix upstream:

https://github.com/gitpython-developers/GitPython/issues/1253
https://github.com/gitpython-developers/GitPython/pull/1254

I tested that on dev instance, and it indeed helps. I'd rather wait till
upstream accepts that patch, though.

Log in to comment on this ticket.

Metadata
Attachments 1
Attached 3 years ago View Comment