#7582 Make koji-prunesigs cron job use lock-wrapper
Closed: Fixed 5 years ago by smooge. Opened 5 years ago by mizdebsk.

After F30 mass rebuild koji-prunesigs cron job was/is running for more than a day. This causes several instances of the same job running at the same time. This in turns causes OOM on koji02.

I think it would be good to use lock-wrapper to wrap the cron job to make sure at most one instance is running at the same time.


I was going to put this in place a long time ago, but I was told it was fine for multiple jobs to be running at the same time.

I suspect the reason we are hitting OOM now is that mass rebuild just happpened so we signed everything and rebuilt everything so all the old stuff is getting gc'ed...

Anyhow, I think it would still be a good idea. Perhaps we could ask koji folks their thoughts?

@mikem should we put a lock wrapper around the koji-gc jobs? Or no?

Note that this is not about koji-gc jobs, but koji-prunesigs job, that runs koji --noauth prune-signed-copies --verbose

I have put lock wrappers on each one. We had one for every day since beginning of February running with various ones oom'd out in the past. I think the pile of ones needing to be pruned is really long and needs some time to process.

Metadata Update from @smooge:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

5 years ago

Login to comment on this ticket.

Metadata