92d588f Add support for testing updates

Authored and Committed by adamwill 7 years ago
    Add support for testing updates
    
    Summary:
    This adds an entirely new workflow for testing distribution
    updates. The `ADVISORY` variable is introduced: when set,
    `main.pm` will load an early post-install test that sets up
    a repository containing the packages from the specified update,
    runs `dnf -y update`, and reboots. A new templates file is
    added, `templates-updates`, which adds two new flavors called
    `updates-server` and `updates-workstation`, each containing
    job templates for appropriate post-install tests. Scheduler is
    expected to post `ADVISORY=(update ID) HDD_1=(base image)
    FLAVOR=updates-(server|workstation)`, where (base image) is one
    of the stable release base disk images produced by `createhdds`
    and usually used for upgrade testing. This will result in the
    appropriate job templates being loaded.
    
    We rejig postinstall test loading and static network config a
    bit so that this works for both the 'compose' and 'updates' test
    flows: we have to ensure we bring up networking for the tap
    tests before we try and install the updates, but still allow
    later adjustment of the configuration. We take advantage of the
    openQA feature that was added a few months back to run the same
    module multiple times, so the `_advisory_update` module can
    reboot after installing the updates and the modules that take
    care of bootloader, encryption and login get run again. This
    looks slightly wacky in the web UI, though - it doesn't show the
    later runs of each module.
    
    We also use the recently added feature to specify `+HDD_1` in
    the test suites which use a disk image uploaded by an earlier
    post-install test, so the test suite value will take priority
    over the value POSTed by the scheduler for those tests, and we
    will use the uploaded disk image (and not the clean base image
    POSTed by the scheduler) for those tests.
    
    My intent here is to enhance the scheduler, adding a consumer
    which listens out for critpath updates, and runs this test flow
    for each one, then reports the results to ResultsDB where Bodhi
    could query and display them. We could also add a list of other
    packages to have one or both sets of update tests run on it, I
    guess.
    
    Test Plan:
    Try a post something like:
    HDD_1=disk_f25_server_3_x86_64.img DISTRI=fedora VERSION=25
    FLAVOR=updates-server ARCH=x86_64 BUILD=FEDORA-2017-376ae2b92c
    ADVISORY=FEDORA-2017-376ae2b92c CURRREL=25 PREVREL=24
    
    Pick an appropriate `ADVISORY` (ideally, one containing some
    packages which might actually be involved in the tests), and
    matching `FLAVOR` and `HDD_1`. The appropriate tests should run,
    a repo with the update packages should be created and enabled
    (and dnf update run), and the tests should work properly. Also
    test a regular compose run to make sure I didn't break anything.
    
    Reviewers: jskladan, jsedlak
    
    Reviewed By: jsedlak
    
    Subscribers: tflink
    
    Differential Revision: https://phab.qa.fedoraproject.org/D1143
    
        
file modified
+1 -1
file modified
+4 -1
file modified
+13 -2
file modified
+45 -1
file modified
+31 -8
file modified
+8 -2
file added
+349
tests/_anaconda_network_static.pm tests/_network_static.pm
file renamed
file was renamed with no change to the file
file modified
+33 -25
file modified
+0 -8
file modified
+0 -8
file modified
+2 -6
file modified
+0 -6