7c1aa44 Add option to generate a short file list for fedfind

Authored and Committed by adamwill 8 years ago
    Add option to generate a short file list for fedfind
    
    See https://pagure.io/fedora-qa/fedfind/issue/2 . At present,
    fedfind parses rsync output to discover images for composes for
    which we have no reliable metadata (any compose older than
    Fedora 24, and any Fedora 24+ compose that's synced to the
    mirrors and split into two or more parts and has its metadata
    removed). This is bad for all the reasons quick-fedora-mirror
    cites - it's very heavy on both the server and the client.
    
    stickster suggested having an option for fedfind to parse the
    `fullfilelist` that `create-filelist` produces, for things that
    would use fedfind frequently and have fast connections to the
    server (so they wouldn't care about the size). But because all
    fedfind wants is images, we can actually do better, by creating
    a smaller file list which filters out packages (.rpm, .drpm),
    ARM Device Tree boot files (.dtb), and HTML files (.html). This
    should never filter out anything fedfind cares about, but makes
    far smaller lists, small enough that we could just have fedfind
    always use these lists and remove the rsync scraping code.
    
        
file modified
+9 -1