#1295 Lots of pending, no tasks running
Closed: Duplicate 4 years ago by iucar. Opened 4 years ago by iucar.

I triggered the background mass-rebuild I mentioned in #1294, plus some non-background tasks that are pending and are part of the daily update at 00:00 h UTC. I see two issues:

  1. I thought that these non-background tasks should have higher priority, so they should have been processed by now, and yet they are sitting in the pending queue.
  2. There are no tasks running for iucar/cran, and it seems that this has been the case during the last 16 hours. Also, I see this strange behaviour in which a bunch of tasks are imported every hour at XX:00 h, and then almost nothing changes for an hour:

Screenshot_20200301_102719.png

  • Is there some kind of throttling enabled for task importing?
  • Why are there no tasks running during this long import period? More specifically, why are the non-background tasks stuck in the pending queue?

I triggered the background mass-rebuild I mentioned in #1294, plus some
non-background tasks that are pending and are part of the daily update
at 00:00 h UTC. I see two issues:

I keep repeating this; the build queue is very unfair ATM, and the
background flag has almost no effect (it only hides the builds a bit so
other users won't complain that much). See e.g. #817 and #307.

I thought that these non-background tasks should have higher priority,
so they should have been processed by now, and yet they are sitting in
the pending queue.

Note https://pagure.io/copr/copr/issue/295 ; if you trigger thousands of
builds - you need to build src.rpms first ; and those builds have
priority.

There are no tasks running for iucar/cran, and it seems that this has
been the case during the last 16 hours. Also, I see this strange
behaviour in which a bunch of tasks are imported every hour at XX:00 h,
and then almost nothing changes for an hour:

I don't know how to explain this offhand ... but please treat the graphs
as informative thing. Perhaps there's some issue with calculating or
caching of calculation of graphs. We need to look at this.

Is there some kind of throttling enabled for task importing?

No, what do you mean by importing? Should there be? One thing is to
build src.rpm, and the other is o import the srpm into dist-git .. I'm not
sure what we talk about.

Why are there no tasks running during this long import period? More
specifically, why are the non-background tasks stuck in the pending
queue?

There are tasks running ... source RPMs are built.

There are tasks running ... source RPMs are built.

I mean, if you upload src.rpm to frontend, builder still goes and downloads the
src.rpm from frontend first ... before it gets imported.

There are no tasks running for iucar/cran, and it seems that this has
been the case during the last 16 hours. Also, I see this strange
behaviour in which a bunch of tasks are imported every hour at XX:00 h,
and then almost nothing changes for an hour:

I don't know how to explain this offhand ... but please treat the graphs
as informative thing. Perhaps there's some issue with calculating or
caching of calculation of graphs. We need to look at this.

Is there some kind of throttling enabled for task importing?

No, what do you mean by importing? Should there be? One thing is to
build src.rpm, and the other is o import the srpm into dist-git .. I'm not
sure what we talk about.

When I say "importing", I mean importing, i.e., all the stuff until the task is sitting in "peding" waiting for a builder to produce the RPM.

Why are there no tasks running during this long import period? More
specifically, why are the non-background tasks stuck in the pending
queue?

There are tasks running ... source RPMs are built.

At the time you were writing this comment, because the importing process of all the builds I requested was finished. But during the ~20 hours it took, not a single RPM was built. I checked. And maybe I'm wrong, but I'm pretty sure that wasn't the case for other batches I sent, i.e. I saw tasks running while the pending queue was still being populated.

But during the ~20 hours it took, not a single RPM was built.

That was several thousands of builds you requested; and since building of src.rpm
has always priority (at least at this moment), yes ... it is completely valid that no
RPM build (!= SRPM) were processed. You had huge queue of SRPM builds
that are not in logs, that is #295.

You seem to be uploading src.rpm, but still ... those are not just downloaded and
imported, but builder machines still download the srpm first.

Does this make sense? Feel free to ping me on IRC so we can close this ... any
help with this problem (patches for the cited issues above) is very welcome.

Ok, so basically background stuff should have a lower priority than normal stuff for the same repo, and that's #817. Closing this.

Metadata Update from @iucar:
- Issue close_status updated to: Duplicate
- Issue status updated to: Closed (was: Open)

4 years ago

Login to comment on this ticket.

Metadata
Attachments 1