#97 MTF (meta test family) standard test role
Opened 6 years ago by jscotka. Modified 6 years ago
jscotka/standard-test-roles str_mtf  into  master

@@ -0,0 +1,12 @@ 

+ # Ansible role for MTF tests


+ This role is intended for [Meta Test Family](https://github.com/fedora-modularity/meta-test-family) tests

+ Test set has to be defined via [metadata definition](https://github.com/fedora-modularity/meta-test-family/tree/devel/mtf/metadata)



+ * Put this role in your test_local.yml playbook.

+ * You'll need to have the following variables defined:

+     * `type`: A string representing which artifact type will be tested (eg. docker, nspawn)

+     * `testsubject_location`: string represeting repository or docker image name

+     * `artifacts`: An artifacts directory


@@ -0,0 +1,2 @@ 

+ ---

+ artifacts: "{{ lookup('env', 'TEST_ARTIFACTS')|default('./artifacts', true) }}"

@@ -0,0 +1,22 @@ 

+ ---

+ - name: Enable Copr repo with MTF

+   shell: dnf -y copr enable phracek/meta-test-family-devel


+ - name: Install the MTF requirements

+   package: name={{item}} state=latest

+   with_items:

+   - meta-test-family


+ - name: Make artifacts directory

+   file: path={{ artifacts }} state=directory owner=root mode=755 recurse=yes


+ - block:

+   - name: Execute the MTF tests

+     shell: exec 2>>{{artifacts}}/test.log 1>>{{artifacts}}/test.log; mtf --setup --metadata --module {{type}} --url {{testsubject_location}} --job-results-dir {{artifacts}}

the main difference b/w your role and the one I propose that you explicitly exec mtf while I want to leave this up to the user so the user is not locked


+ - always:

+   - name: Pull out the logs

+     fetch:

+       dest: "{{artifacts}}/"

+       src: "{{artifacts}}/"

+       flat: yes

@@ -0,0 +1,3 @@ 

+ ---

+ type: ""

+ testsubject_location: ""

Initial draft how MTF can be handled via standard test roles.
Unclear part for me is how testsubject_location will be pasted, because I expect to test for example docker_image from outside.

Workflow for docker test subject is:
* start docker daemon
* import there test subject (docker image defined via testsubject_location)
* be able to test test subject as blackbox and optionaly as whitebox, means that I'm not able to test test subject (image) itself from inside, because container provide some services based on various environment definition and I have to be able to start in its own turn.

maybe @bgoncalv could be interested because it is same also for module testing.

@jscotka how do you think, whether it possible to extend some existing role to handle this case?
Does it make sense to spawn more roles for each individual case?

MTF can be run with help of standard-test-basic role

yes, that could be part of some another role in case it make sense. I'm not sure what it better solution. if there will be more specialized roles, or less universal ones, but harder to config them - need to setup more variables there.
I'm not so familiar with standard test roles. so I let it up to you.

from that perspective there are 3. important parts:
* install additional requirement
* use test subject as an parameter (I dont know how to do it now - {{testsubject_location}} variable set maybe via TEST_SUBJECT env var in case it everytime contains this), and test subject is docker cotainer image (somewhere built)
* run special command: mtf --setup --metadata --module {{type}} --url {{testsubject_location}} --job-results-dir {{artifacts}}

and there is another important part, and this is test metadata - our tool solves this via --metadata option, so a do not expect to have test set defined via standard test role. But this is more less implementation detail how we gather test sets


Installing additional packages is very often task. It is part of common role now. Will be released with next build.

About "TEST_SUBJECT" can we discuss it face-to-face?

the main difference b/w your role and the one I propose that you explicitly exec mtf while I want to leave this up to the user so the user is not locked