89abc3d Implement Ansible-based tasks

Authored and Committed by kparal 6 years ago
33 files changed. 911 lines added. 2694 lines removed.
.gitignore
file modified
+3 -0
README.rst
file modified
+90 -0
data/ansible/ansible.cfg
file added
+11
data/ansible/library/__init__.py
file added
+0
data/ansible/library/acquire_subject.py
file added
+182
data/ansible/library/taskotron_koji.py
file added
+169
data/ansible/runner.yml
file added
+76
data/ansible/tasks_generic.yml
file added
+21
data/ansible/tasks_si.yml
file added
+55
data/report_templates/html.j2libtaskotron/report_templates/html.j2
file renamed
+0 -0
libtaskotron.spec
file modified
+22 -26
libtaskotron/__init__.py
file modified
+1 -1
libtaskotron/config.py
file modified
+8 -1
libtaskotron/config_defaults.py
file modified
+5 -1
libtaskotron/directives/create_report_directive.py
file modified
+3 -2
libtaskotron/directives/resultsdb_directive.py
file modified
+8 -11
libtaskotron/executor.py
file modified
+142 -172
libtaskotron/image_utils.py
file added
+88
libtaskotron/main.py
file modified
+16 -23
libtaskotron/minion.py
file removed
-244
libtaskotron/overlord.py
file removed
-100
libtaskotron/remote_exec.py
file removed
-362
libtaskotron/taskformula.py
file removed
-246
requirements.txt
file modified
+1 -1
testing/functest_executor.py
file removed
-68
testing/functest_main.py
file removed
-35
testing/test_executor.py
file removed
-454
testing/test_exitcode_directive.py
file removed
-109
testing/test_main.py
file modified
+10 -14
testing/test_minion.py
file removed
-159
testing/test_overlord.py
file removed
-123
testing/test_remote_exec.py
file removed
-283
testing/test_taskformula.py
file removed
-259
    Implement Ansible-based tasks
    
    This allows to run Ansible-based tasks which implement Standard
    Interface [1] or their slight variation for generic tasks.
    
    Thanks everyone involved.
    
    Fixes: https://pagure.io/taskotron/libtaskotron/issue/380
    
    [1] https://fedoraproject.org/wiki/CI/Tests
    [1] https://fedoraproject.org/wiki/Changes/InvokingTests
    
    Squashed commit of the following:
    
    commit b1223559678dbb41ed6524c5e369554e42d0447f
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Mon Jan 22 14:49:22 2018 +0100
    
        added files for general subject acquirer
    
    commit ec93c136dcb640f7e141ad497a86fd14a0eabf98
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Mon Jan 22 09:55:52 2018 +0100
    
        basic general subject acquirer for SI
    
    commit 49456cfeb2710159075a5a034747b79d27ea0d1b
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Jan 18 15:16:25 2018 +0100
    
        readme: task-mtf-containers has been ansiblized
    
    commit dc24b36e70a7524e4688e01d2199a35956f6e444
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Jan 17 16:53:02 2018 +0100
    
        readme: check_modulemd has been ansiblized
    
    commit 97cbbd7bfc6f2fd59fe18cc14e58f7849d08c9ea
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Jan 17 12:40:21 2018 +0100
    
        tasks_si: forward taskotron_arch
    
    commit ba72030ca91417643692c742958adb336a3c2b35
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Tue Jan 16 15:31:26 2018 +0100
    
        readme: task-rpmdeplint is ansiblized
    
    commit c2969f4907105eb7c80ec254ddcbeea2e47b1fa9
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Tue Jan 16 13:31:21 2018 +0100
    
        bump release
    
    commit 092bcc2a4e3975fb449294cd43dc150949f27569
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Tue Jan 16 13:26:16 2018 +0100
    
        expose taskotron_arch variable
    
    commit a75c823ec1cf7187f59c3d7b730df969260e90d5
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Fri Jan 12 14:29:23 2018 +0100
    
        readme: taskotron-python-versions ansiblized
    
    commit e07dc1e7be7a68d99907f0f82c36bb46f5eb4a8e
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Fri Jan 12 13:09:47 2018 +0100
    
        syntax check for test playbooks
    
    commit 060dc73ee3441b6b577a71f689f7f3141fc77803
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Jan 11 16:46:39 2018 +0100
    
        readme: task-abicheck has been converted
    
    commit 25b7046c8a73cd247943589fe8cf65523ae76fae
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Jan 11 13:54:52 2018 +0100
    
        bump release
    
    commit 337d6c7ea7c2ff0846feff013153df0f513a176d
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Jan 11 13:29:25 2018 +0100
    
        document playbook variables
    
    commit da574335df5ab433b7af3c9a0bf00e4e3436eed2
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Jan 11 12:56:01 2018 +0100
    
        tasks_si: better temporary testcase
    
    commit 18052912aa114bbe8787628af72c723cf6a874e7
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Jan 10 16:18:55 2018 +0100
    
        readme: update ansiblize instructions
    
    commit cf90b0f09791b91e4fb7614b8877c8b462608df1
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Thu Jan 11 12:32:45 2018 +0100
    
        fixed storing artifacts for multiple tests*.yml
    
    commit 8073f88868020717213ccb9314a320eeb2af7712
    Merge: 8903dc7 2a8d50b
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Jan 10 15:17:11 2018 +0100
    
        Merge remote-tracking branch 'origin/feature/ansiblize' into feature/ansiblize
    
    commit 2a8d50bfc905a06b2621939bab3cdbe1277fd76b
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Wed Jan 10 15:12:59 2018 +0100
    
        support for multiple tests*.yml playbooks
    
    commit 8903dc7235f4d54afd161c8659871d4abdc92573
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Jan 10 14:17:20 2018 +0100
    
        bump release
    
    commit 37f828703e99ad65cc62bd88e0d63b03502a0d85
    Merge: 3840c9a c959bb4
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Jan 10 14:15:53 2018 +0100
    
        Merge branch 'develop' into feature/ansiblize
    
    commit 3840c9a8ccdc66c95838ee36a1820965d2d0044b
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Tue Jan 9 12:50:48 2018 +0100
    
        ansible: use import_tasks
    
        The 'include' keyword is now deprecated.
    
    commit 1e6ab1a067c36c45d570a193348430315f905085
    Merge: ef899d6 02a394c
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Tue Dec 12 12:59:28 2017 +0100
    
        Merge branch 'develop' into feature/ansiblize
    
    commit ef899d613db47a1b79378f3121481e8fe20adb3c
    Author: František Zatloukal <fzatlouk@redhat.com>
    Date:   Wed Dec 6 12:10:17 2017 +0100
    
        bump release
    
    commit a7070f05dcf16dec6f73d99188b5265a934d4888
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Wed Dec 6 11:03:55 2017 +0100
    
        Disable StrictHostKeyChecking
    
    commit ba04e28529b162d3bded8554e3ec0729bf87129a
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Mon Nov 27 15:30:30 2017 +0100
    
        fix permission issues by disabling .retry files
    
    commit fb6b94a97e19a888cc4836bf21f7684897aa3c6c
    Merge: 248491f 9afa9cf
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Nov 23 13:14:48 2017 +0100
    
        Merge branch 'develop' into feature/ansiblize
    
    commit 248491f6d049d7ed130c5ad851217b9be6c4191f
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Tue Nov 21 16:15:44 2017 +0100
    
        bump version
    
    commit 4f415a5b6741f22cac12debeb45cf5dbc93695b7
    Merge: ea94853 c4b2617
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Tue Nov 21 16:14:05 2017 +0100
    
        Merge branch 'develop' into feature/ansiblize
    
    commit ea94853a14e5939a13ccb064a75e69c55bbe96d6
    Merge: 36bdf1c 5a67334
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Nov 15 11:34:07 2017 +0100
    
        Merge branch 'develop' into feature/ansiblize
    
    commit 36bdf1cc44529d7ee0b6596565608075e245fed9
    Merge: 88e6981 d9fe106
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Oct 26 17:01:54 2017 +0200
    
        Merge branch 'develop' into feature/ansiblize
    
    commit 88e69814ff227f6d4e3292cc2e5c8300c1ba8c95
    Merge: 7a80779 318c548
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Fri Oct 6 15:47:36 2017 +0200
    
        Merge branch 'develop' into feature/ansiblize
    
    commit 7a8077924c5d5ec441fb4398f09dc05305d8d82c
    Merge: 687aa1c 245584e
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Tue Sep 26 16:06:35 2017 +0200
    
        Merge branch 'develop' into feature/ansiblize
    
    commit 687aa1cc914c8b1362b4141e1889193987ac8c16
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Sep 14 14:53:43 2017 +0200
    
        readme: fix branch name
    
    commit 82de0386efb4bc9fcce457d530ccd927d774e484
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Sep 14 11:00:00 2017 +0200
    
        add copr link
    
    commit 968cbe5da07eebb98a382b5634b29d85e4ee9e07
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Wed Sep 13 16:11:06 2017 +0200
    
        readability fix: long line folding
    
    commit b1807a543f135ed0a57c87ecf4ed0aec8d3a47f0
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Sep 13 15:44:25 2017 +0200
    
        use {{artifacts}}/taskotron subdir for our files
    
    commit 11ee42662da810be1d040bd13dc26a5068520dbb
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Sep 13 15:22:10 2017 +0200
    
        simplify setting outcome from exit code
    
    commit 850209ea54f43fd3b3f893ae49eb2dd962663ea9
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Sep 13 14:28:28 2017 +0200
    
        fix formatting
    
    commit 75b11d2dfa0b65230f756f51ca2302299e58502f
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Sep 13 14:24:46 2017 +0200
    
        add "what changed" info to readme
    
    commit ac0f745104902db76c13c507b4ab3f06fb0a21ef
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Sep 13 13:01:24 2017 +0200
    
        have a single runner.yml playbook
    
        Include either generic or SI tasks.
    
    commit 650271b74cf0a38427ee523c326f7269da89f127
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Sep 13 12:35:07 2017 +0200
    
        sync runner_generic and runner_si
    
    commit 7612ed39b95a675c095406443c9793bb9cb2ae74
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Wed Sep 13 12:27:09 2017 +0200
    
        rename koji ansible module to taskotron_koji module
    
        Otherwise the ansible module name clashed with `koji` python module and
        there were errors during import.
    
    commit b4c1eaea1c9c1cd37df59307a3bf137f59d8364b
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Wed Sep 13 12:06:11 2017 +0200
    
        SI fixes and polishing
    
    commit a38b1ceea32b2a30f617553beddaa08c69006948
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Tue Sep 12 17:07:22 2017 +0200
    
        don't download src.rpm from koji for si tests
    
    commit ba48e7817848b4d9130f10ee0c4fdca2b4cf7141
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Mon Sep 11 14:08:10 2017 +0200
    
        fix --inventory in SI runner
    
    commit a3869761f53e93f1ad3d137fdd1ecbe207e8205a
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Mon Sep 11 14:03:51 2017 +0200
    
        add missing comments
    
    commit d9c990c1a8a19c343a0396e6384c99e7747d64d1
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Mon Sep 11 13:53:04 2017 +0200
    
        added logic to decide what playbook to use
    
    commit 6f957b1612efb66bf3799c5200b3b67a59203dc2
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Fri Sep 8 16:34:17 2017 +0200
    
        improve ansible output
    
    commit 22ced9ca76ec0bf690d7b963f3f18e110a401188
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Fri Sep 8 13:24:10 2017 +0200
    
        bump version
    
    commit 0466dc65e3cdd5f3523647a33a66b05c59543ac4
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Fri Sep 8 12:46:28 2017 +0200
    
        fix becoming root
    
    commit 2c41c589cbf402ed467a527fab775a56fa1ad215
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Sep 7 16:27:45 2017 +0200
    
        spec file fixes for data files
    
    commit b35fd9e28bbd1556c68049000b92fce5a1da1879
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Sep 7 13:34:29 2017 +0200
    
        move data files to /usr/share/libtaskotron
    
    commit f9d1467a4236cefa1aadd0e1e2951e8d58ebbaf6
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Thu Sep 7 13:09:24 2017 +0200
    
        reporting to resultsdb
    
    commit 32ae22a30f3a2c30eabecbc04914f4582afcc411
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Sep 7 12:59:26 2017 +0200
    
        fix taskdir with relative paths
    
    commit 7e08f16fd951c819a98452be95a8377a8a44cc9f
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Thu Sep 7 11:27:04 2017 +0200
    
        move all ansible files into a subdirectory
    
    commit 49a52c6f80cfbf9fc19dcb2c925c5bfca577e9ab
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Wed Sep 6 14:30:15 2017 +0200
    
        ansiblizing cleanup
    
    commit afa906b532285b62da90855376e9c605d456672b
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Wed Aug 30 16:54:17 2017 +0200
    
        WIP: generating results for resultsdb
    
    commit 2b05db04ce86b7661d5805c7fdf20618874b5f7c
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Tue Aug 29 20:59:24 2017 +0200
    
        WIP: moar STI compatibility
    
    commit 1cf5e6265a5db2506c1ee6567efc71c9c746ed95
    Author: Lili Nie <lnie@redhat.com>
    Date:   Tue Aug 29 06:59:40 2017 +0000
    
        don't fail when ssh key is not provided
    
        Fix a bug that caused connection to fail when --ssh-privkey option was
        not used. Instead, the default ssh keys should be used.
    
        Merges #394.
    
    commit c9d42b4b00b2bbf2c3c4603a3be7bf7cfad1c2cb
    Merge: af984b6 61b1716
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Tue Aug 29 14:21:02 2017 +0200
    
        Merge branch 'develop' into feature/ansiblize
    
    commit af984b6803434fc4ae1d3c04ed386df87417ed61
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Fri Aug 25 11:02:16 2017 +0200
    
        WIP: STI compatibility
    
    commit 6d5ad44536bd641bf253d68d0d2bb0ccb6eb75e2
    Author: Lukas Brabec <lbrabec@redhat.com>
    Date:   Mon Jul 31 14:47:11 2017 +0200
    
        create artifacts dir when not local execution
    
    commit 81f65eecb111de36d77b980ae2b7c0c1cd6d7f08
    Merge: db85bdf 52ca1b4
    Author: Kamil Páral <kparal@redhat.com>
    Date:   Tue Jun 20 11:52:45 2017 +0200
    
        Merge branch 'develop' into feature/ansiblize
    
    commit db85bdff95f0692daa958b813eed80c93d739436
    Author: Martin Krizek <mkrizek@redhat.com>
    Date:   Thu May 4 18:11:46 2017 +0200
    
        Support for Ansible Tasks
    
        This patch removes libtaskotron formula format for writing tasks and
        introduces support for running tasks in form of an ansible playbook.
    
        The following parts are removed:
        * task formula handling code
        * remote execution (Paramiko Wrapper)
        * concept of overlord and minion
        * the exitcode directive (hasn't been really used)
    
        When running locally, user is asked for sudo password since tasks are ran
        as root.
    
        https://fedoraproject.org/wiki/Changes/InvokingTestsAnsible
    
        Test Plan:
        $ git clone https://pagure.io/task-rpmlint-ansible
        $ runtask -d -i gzip -t koji_build -a x86_64 task-rpmlint-ansible/run_tests.yml
        $ runtask -d -i gzip -t koji_build -a x86_64 --ssh $IP_ADDRESS --ssh-privkey conf/id_taskotron task-rpmlint-ansible/run_tests.yml
        $ runtask -d -i gzip -t koji_build -a x86_64 --libvirt --ssh-privkey conf/id_taskotron task-rpmlint-ansible/run_tests.yml
    
        Differential Revision: https://phab.qa.fedoraproject.org/D1195
    
        
file modified
+3 -0
file modified
+90 -0
empty file added
data/report_templates/html.j2 libtaskotron/report_templates/html.j2
file renamed
file was renamed with no change to the file
file modified
+22 -26
file modified
+1 -1
file modified
+8 -1
file modified
+142 -172
file modified
+16 -23
file removed
-244
file removed
-100
file modified
+1 -1
file removed
-454
file modified
+10 -14
file removed
-159
file removed
-123