PR#3: [WIP] - example how modulatirity testing framework could be integratedinside ansible - ansible_based_tests

		`@@ -0,0 +1,12 @@`
		`+ ---`
		`+ - hosts: localhost`
		`+ remote_user: root`
		`+ tasks:`
		`+ - name: Setup repositories`
		`+ shell: dnf copr -y enable jscotka/modularity-testing-framework`
pingou commented 7 years ago While fine for the moment, this would be something that we would need to document if we allow or not. COPR by nature can be very volatile so I would be not in favor of allowing it. But that is something to worry about later, this is fine for now :)
		`+ - name: Install the requirements`
		`+ package: name={{item}} state=latest`
		`+ with_items:`
		`+ - modularity-testing-framework`
		`+ - name: Execute the testsuite for testmodule downloaded by run-them.sh script`
		`+ shell: /usr/share/moduleframework/tools/run-them.sh testmodule testmodule-master-20170410111210 taskotron`
pingou commented 7 years ago `testmodule-master-20170410111210` is hard-coded? Could there be a way to retrieve this?

jscotka commented 7 years ago

HI @stefw
I've cteated PoC how MTF can be integrated into ansible.
It schedule tests via special script run-them, what is able to handle various modules and koji builds for modules (actually trigerred by MBS change messages (status done))
If you would like, I can copy tests here, to have tests explicitly here and run then explicitly via avocado, not via magic run-them.sh script :-)

it is done in this run-them.sh script steps like:

parse module build
set enviromental variables
avocado run /usr/share/moduleframework/examples/testing-module/*

I've created this PR as an feedback for section: Example: Modularity testing Framework
on page https://fedoraproject.org/wiki/Changes/InvokingTestsAnsible
For me it is working well and ansible could be used for invocation MTF tests very simple.

Actually there is just issue and it is how to interpret results, there could be something what is able to parse for example xunit output, or tap output, or whatever what avocado framework is able to produce.
Tests are scheduled by avocado as whole testsuite. I do not expect to run tests sepatatelly, to have there more ansible 'phases' with unique tests, but it is also possible to list tests and then run them separately, change command (you have to imagine that it is ansible 'for' loop :-)):

avocado run /usr/share/moduleframework/examples/testing-module/*
->
for foo in /usr/share/moduleframework/examples/testing-module/*; do
    avocado run $foo
done

pingou commented on line 7 of tests/mtf-testmodule/run_tests.yml 7 years ago

While fine for the moment, this would be something that we would need to document if we allow or not. COPR by nature can be very volatile so I would be not in favor of allowing it.
But that is something to worry about later, this is fine for now :)

pingou commented on line 13 of tests/mtf-testmodule/run_tests.yml 7 years ago

testmodule-master-20170410111210 is hard-coded? Could there be a way to retrieve this?

jscotka commented 7 years ago

@pingou yep, this package will be in fedora, it is in progress, so that this is just workaround to allow using newest version as soon as possible :-)

I actually does not have testmodule-master-20170410111210 hardcoded anywhere, it is just example. This is identification produced by MBS via fedora fedmsg. So I expect to pass this params from outside (CI, manual trigger) I'm not familiar with ansible, so I've hardcoded it there to show how it can work.

Edited 7 years ago by jscotka

jscotka commented 7 years ago

I also don't exactly know how ansible will handle this testing:
Probably there is small confusing what you expect from ansible.

I expect that I'll install module on top of this ansible (rpm based, docker based, I'll prepare test env in my own), probably you expect that this is done by default?
So that I'm using some link to module build like testmodule-master-20170410111210 or could be used some local build of module, it is also possible

jscotka commented 7 years ago

@pingou probably I've found this issue, it is caused by my misunderstanding how it should work.
probably I should have module downloaded from dist-git (modules namespace like packages are in /modules/example_distgit.yml
so that I should add something similar for modules like:
fedpkg clone modules/testmodule, in that case it will test what is in dist-git and prepare env for that.

Metadata

Assignee

None

Tags

No Tags

Changes Summary 1

+12

file added

tests/mtf-testmodule/run_tests.yml

ansible_based_tests

Source Code

#3 [WIP] - example how modulatirity testing framework could be integratedinside ansible Opened 7 years ago by jscotka. Modified 7 years ago jscotka/ansible_based_tests mtf into master

Metadata

Changes Summary 1

#3 [WIP] - example how modulatirity testing framework could be integratedinside ansible

Opened 7 years ago by jscotka. Modified 7 years ago

jscotka/ansible_based_tests mtf into master