#129 Koji request response is slow for kojira user query
Closed: Dropped 2 years ago by tkopecek. Opened 7 years ago by pnemade.

Can it be possible to make koji serve faster response for queries like ​http://koji.fedoraproject.org/koji/tasks?owner=kojira&state=all ?
I am using currently Fedora 24 machine using Firefox browser I found this request or changing page to older tasks list request also takes around 5-7 minutes for me.

Please ask for any more input if needed.
Thanks.


Note that this tracker is for the upstream Koji project, not the particular Koji instance at koji.fedoraproject.org. There are numerous factors that influence the performance of this query.

That being said, there may be some things we can do on the code side. Possibly we could add more indexes. It might also help to introduce garbage collection for old tasks (kojira has created a lot of tasks over the past 9+ years, and I don't know if we really need a record of createrepo jobs from 2007).

Also, possibly, this patch might help: https://pagure.io/koji/pull-request/90
(or it might not, or maybe only for the right postgres)

On the admin side, Fedora may need to beef up the db server that koji is using, and/or tune the options a bit.

Also, fetching http://koji.fedoraproject.org/koji/tasks?view=toplevel&state=all is impossible (does not return anything in ~15 minutes).

Side note: Brew is a similarly large Koji instance and this page comes back reasonably quickly.

$ time curl -o /dev/null  'https://brewweb.engineering.redhat.com/brew/tasks?state=all&owner=kojira&view=toplevel&method=all&order=-id'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  229k  100  229k    0     0  37989      0  0:00:06  0:00:06 --:--:-- 53281

real    0m6.194s
user    0m0.083s
sys     0m0.099s

I wonder of there is something off with the db indexes on koji.fp.o

Metadata Update from @yulwang:
- Issue tagged with: groomed

4 years ago

No response in two years, so let's close this one. If you still have problems with this, please reopen.

Metadata Update from @tkopecek:
- Issue untagged with: groomed

4 years ago

Metadata Update from @tkopecek:
- Issue tagged with: groomed

4 years ago

Metadata Update from @tkopecek:
- Issue close_status updated to: Invalid
- Issue status updated to: Closed (was: Open)

4 years ago

I still have problems. Currently I am even getting problems with my own user: https://koji.fedoraproject.org/koji/tasks?owner=churchyard&state=all

This page loads for minutes. (HTTP request sent, awaiting response...) Users with fewer builds don't have such problems.

@lbalhar loads for 4 seconds.
@churchyard loads for 3 minutes.

Please reopen.

Note that this tracker is for the upstream Koji project, not the particular Koji instance at koji.fedoraproject.org.

Whta is the proper place to get this resolved? https://pagure.io/fedora-infrastructure/issues or https://pagure.io/releng/issues ?

Metadata Update from @dgregor:
- Custom field Size adjusted to None
- Issue status updated to: Open (was: Closed)

4 years ago

For issues specific to koji.fedoraproject.org, I would suggest https://pagure.io/fedora-infrastructure/issues

That said, I just re-ran Mike's query from a couple years ago and got a much different result:

$ time curl -o /tmp/kojira 'https://brewweb.engineering.redhat.com/brew/tasks?state=all&owner=kojira&view=toplevel&method=all&order=-id'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 326k 100 326k 0 0 1476 0 0:03:46 0:03:46 --:--:-- 92941

real 3m46.303s
user 0m0.096s
sys 0m0.155s

It's worth taking a look at the SQL being used for this page and see if it can be improved.

Should I open an infra ticket or wait?

It wouldn't hurt to open an infra ticket. They may be able to resolve this by adding an index or optimizing the postgresql query planner.

Metadata Update from @tkopecek:
- Issue close_status updated to: Dropped
- Issue status updated to: Closed (was: Open)

2 years ago

Login to comment on this ticket.

Metadata