1b767b9 frontend: cache 'Batch.finished' and 'Batch.blocked'

2 files Authored by praiskup 3 years ago, Committed by praiskup 3 years ago,
    frontend: cache 'Batch.finished' and 'Batch.blocked'
    
    Batches caused a serious slowdown on /backend/pending-jobs/ before.  The
    reason was that every single task there was checked whether it is or is
    not blocked/finished.  It for BuildRoots means that
    'build_chroot.build.batch.builds[].build_chroots[]' was evaluated for
    every single row, and ditto for every single row in "source builds".
    The worst thing wasn't CPU calculation, but that the info was mostly
    lazy-loaded on demand without practical reason.  This is now mitigated.
    
    The trick implemented in this patch is that only **one batch** in each
    **tree of batches** has to be expensively checked for the finished
    state.  That's given by those facts:
      - Batch is blocked if parent batch is blocked, and ditto for grand
        parents.  So in most cases we can go to descendants and check for
        blocked state there only.
      - We don't have to check for finished batches all the time, but we can
        cache (1 hour in this commit) the finished status.  This is because
        we never add new builds into already finished batch.
    IOW, we only have to check the status for the batch in tree that is
    currently being processed.
    
    While we are on it, it doesn't even make much sense to re-calculate the
    'Build.finished' property all the time.  If batch B depends on batch A,
    then asking if 'B.finished' means that we have to calculate 'A.finished'
    right-away.  Later, when explicitly asking for 'A.finished', we don't
    have to recalculate (among possibly thousands of builds and
    build_chroots in batch A) A.finished - we can use the in-memory cache.
    That's why 'Batch.finished' uses 'Flask-Caching', and only if not cached
    we fallback to 'Batch.finished_slow'.