#2612 kojira: startup can take few hours
Closed: Fixed 2 years ago by tkopecek. Opened 3 years ago by tkopecek.

If there is a lot of recently expired repos, startup phase can take even few hours. Initial readCurrentRepo could be done in a thread, so other threads can start meanwhile (not sure - haven't checked if it is really possible).


Some numbers:
I've currently about 160000 in getActiveRepos, while kojira is picking about 500 per minute, so startup would be 5 hours.

Metadata Update from @tkopecek:
- Custom field Size adjusted to None

3 years ago

My initial thought is that there's not much Kojira can do without a successful run of readCurrentRepos. Getting a meaningful gain here will probably require more than simply splitting this out.

More likely, the answer is to make readCurrentRepos faster. It's basically one call to getActiveRepos, but this gets complicated by the additional tag data that is needed to set up the ManagedRepo instances. Once we get going, we're caching a lot of our getTag calls, but at the start, we have not cache. Also in ManagedRepo.__init__() we end up calling getFullInheritance.

Before we get too far into it, it might be good to profile this, but perhaps we could defer the getFullInheritance call by making ManagedRepo.taglist a caching property.

Also, we might try to prepopulate the cache used by getTag with a single listTags call.

Prepopulating + property works well for me.

Metadata Update from @tkopecek:
- Issue tagged with: testing-ready

2 years ago

Metadata Update from @jcupova:
- Issue tagged with: testing-done

2 years ago

Login to comment on this ticket.

Metadata
Related Pull Requests
  • #2764 Merged 2 years ago