#8562 koji unresponsive due to database autovacuum
Closed: Fixed 6 months ago by kevin. Opened 6 months ago by kevin.

koji is current not very responsive, due to it doing an autovacuum of the buildroot_listing table (200+GB) to prevent transaction wrap around:

 koji    | buildroot_listing_rpms                          |               | RowExclusiveLock         | t       | postgre
s | autovacuum: VACUUM public.buildroot_listing (to prevent wraparound)                                                  



                                                                                         | 2020-01-20 15:43:37.448043+00 
| 01:16:41.053496 | 31992
 koji    |                                                 |               | ExclusiveLock            | t       | postgre
s | autovacuum: VACUUM public.buildroot_listing (to prevent wraparound)                                                  



                                                                                         | 2020-01-20 15:43:37.448043+00 
| 01:16:41.053496 | 31992
 koji    | buildroot_listing_buildroot_id_key              |               | RowExclusiveLock         | t       | postgre
s | autovacuum: VACUUM public.buildroot_listing (to prevent wraparound)                                                  



                                                                                         | 2020-01-20 15:43:37.448043+00 
| 01:16:41.053496 | 31992
 koji    | buildroot_listing                               |               | ShareUpdateExclusiveLock | t       | postgre
s | autovacuum: VACUUM public.buildroot_listing (to prevent wraparound)                                                  



                                                                                         | 2020-01-20 15:43:37.448043+00

This is keeping a lock on that table.

Investigating if there's anything to do. If we restart the db it will just start again, so we ened to adjust something or let it finish.


I've adjusted autovacuum_freeze_max_age so it puts off the run for now and restarted things.

ok, it was fine for a few, then it started autovacuuming the public.sessions table and load skyrocketed again.

I stopped everything and did a manual vacuum on that table and restarted everything.

And it did it again. ;) This time the task table. That is all the big ones... so we will see now...

I went ahead and upgraded to koji 1.20.0... (some of the work there with bulk commits should help db).

So far, it's been ok since then.

I'm going to close this now and re-open if it happens again... I really hope we are back to stable now.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

6 months ago

Login to comment on this ticket.

Metadata