e66473e Only scan centos stream 9 primary mirror if there are changes

Authored and Committed by adrian 2 years ago
    Only scan centos stream 9 primary mirror if there are changes
    
    Currently we still see some errors with scanning the primary centos
    stream 9 mirror.
    
    The current assumption is that because of '--delay-updates' rsync
    updates the primary mirror timestamps even if the actual files are moved
    to the final location much later. This breaks our assumption during
    scanning that we only have to check files in directories with changed
    timestamps.
    
    Currently we might scan the primary mirror and see updated timestamps
    but the files are not yet moved to the final location because of
    '--delay-updates'. After our scan, rsync finishes on the primary mirror
    and the files are moved to the final location, but the timestamps of the
    directories do not change. That results in us not detecting new files.
    
    This change checks the timestamp of the COMPOSE_ID file and only runs
    the scan if the timestamp is newer than during the last scan. The hope
    is that the timestamp of COMPOSE_ID is only updated once the actual new
    file is replacing the old file.
    
    In addition to hopefully fixing the primary mirror scanning this should
    also reduce the I/O load on the primary mirror because we do not run a
    full rsync directory listing if COMPOSE_ID has not changed.
    
    Signed-off-by: Adrian Reber <adrian@lisas.de>