#942 kojira: Needed tag refers to unknown task
Closed: Fixed 5 years ago Opened 5 years ago by mikem.

In the code, a comment says this case "should not happen". Yet, I see it in my kojira logs.

This looks like a minor race condition. The 'unknown' task appears in the kojira logs earlier. It finishes very shortly before the warning. Either we're not cleaning something up, or just not cleaning it up fast enough. When this happens, kojira will make a duplicate newRepo task, so it is a wasteful bug.


The code segment here is in regenRepos:

order = self.needed_tags.values()
order.sort(key=lambda t:t['score'])
for tag in order:
    ### -SNIP- ###
    task_id = tag.get('task_id')
    if task_id:
        if task_id in self.tasks:
            # we already have a task
            continue
        else:
            # should not happen
            logger.warning('Needed tag refers to unknown task. '
                    '%s -> %i', tagname, task_id)

Looks like the the order of events is:

  • the self.tasks entry is added in regenRepos (which is its own thread now)
  • the 'task_id' field for the tag entry in self.needed_tags is set by regenRepos when it creates the repo task
  • the entry is deleted in checkTasks, called from updateRepos, in the main thread
  • regenRepos hits the above segment near the beginning of its loop. Note that it is looping over a sorted copy of the needed tags values
  • the self.needed_tags entry is deleted in checkNeeded (main thread) when it notices the new repo.

So this can leave a gap where we have a stale needed_tags entry.

Not critical, but definitely a bug we should fix

Metadata Update from @mikem:
- Issue set to the milestone: 1.17

5 years ago

Ways we might address this

  1. Clean up the needed_tags entry when the task finishes. I'm not sure this is quite right though. We might not want to trust that task completion == new repo.
  2. Hold onto the tasks entry for a period of time after it finishes so that we can see that the task is not unknown. After a reasonable timeout, log a warning and fix the entry.

I don't think we should just drop the check. If task fails to produce a valid repo, we might get into a state we never properly regenerate a needed tag.

Leaning towards number 2

Login to comment on this ticket.

Metadata