028b0c8 Fix #343: Allow configuring multiple target directories.

13 files Authored by jkaluza 4 years ago, Committed by lsedlar 4 years ago,
    Fix #343: Allow configuring multiple target directories.
    
    ODCS currently always store resulting compose in the single
    `conf.target_dir` directory. There are at least two use-cases
    for which it would make sense to store some composes in different
    locations:
    
    - The compose contains data which cannot be accessed publicly.
      This is needed by Fedora releng team to generate internal-only
      composes with packages which cannot be publicly available because
      of licensing issues.
    - The ODCS backend generating the compose runs in different geo-location
      and uses completely different NFS storage for composes. This might
      be used in the future to allow CI infrastructure to use ODCS and
      store the resulting composes closer to the CI VMs.
    
    This commit allows choosing the resulting target_dir by implementing
    following points:
    
    - New `conf.extra_target_dirs` option is introduced.
    - The `conf.clients` or `conf.admins` options can allow certain users/groups
      to use particular target dir defined in `conf.extra_target_dirs`.
    - New `target_dir` API variable is introduced to choose the target_dir.
      If unset, default `conf.target_dir` is used.
    - The `target_dir` is stored in DB using the `Compose` model.
    - The code to remove expired composes now checks composes in all configured
      target directories and tries to remove them. If the target directory
      is not available on particular backend, the compose removal is skipped.
      This means that only backends which have access to particular target_dir
      are able to remove expired composes.
    - The KojiTagCache code is changed to support multiple target directories.
    
    Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
    
        
file modified
+17 -12
file modified
+25 -19
file modified
+6 -0
file modified
+14 -3
file modified
+5 -4
file modified
+12 -2
file modified
+19 -8
file modified
+1 -0
file modified
+54 -1