From a08cef950508b3f350cf3f8fe6ffb6edc00e48e9 Mon Sep 17 00:00:00 2001 From: Aleksandra Fedorova Date: Mar 12 2020 12:16:47 +0000 Subject: Add How to add simple test page --- diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 337ba4b..a186b51 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -1,4 +1,5 @@ * Guides +** xref:how-to-add-dist-git-test.adoc[How to add simple dist-git test] ** xref:quick-start-guide.adoc[Quick Start Guide] ** xref:pull-requests.adoc[Pull Requests] ** xref:share-test-code.adoc[Share Test Code] diff --git a/modules/ROOT/pages/how-to-add-dist-git-test.adoc b/modules/ROOT/pages/how-to-add-dist-git-test.adoc new file mode 100644 index 0000000..1187e7e --- /dev/null +++ b/modules/ROOT/pages/how-to-add-dist-git-test.adoc @@ -0,0 +1,169 @@ +:toc: + += How to add simple dist-git test for a package = + +== Test as a single command == +=== Task === + +There is a package *somepackage*, which contains a binary: +*/usr/bin/somebinary* + +The most simple and obvious way to test this binary is to run it: +*somebinary --help* and check the exit status of the result. + +How to add this test for the package? + +=== Solution === + +https://docs.fedoraproject.org/en-US/ci/standard-test-roles/[Standard +Test Roles] framework provides solution for the task. It is fully +supported in Fedora Rawhide. + +==== Step 1 - Add test configuration ==== + +Create file `tests/tests.yml` in the dist-git of the package: + +.rpms/somepackage.git: +---- +. +├── 0001-something.patch +├── somepackage.spec +├── sources +└── tests + └── tests.yml +---- + +*tests.yml* is an ansible playbook, where you describe test environment +and steps to run your tests. + +There are many options and +https://fedoraproject.org/wiki/CI/Examples[examples available], but +let's focus on the task at hand: we want to run just one command. + +Then, the content of the file should look as follows: + +.rpms/somepackage.git:tests/test.yml +[source,yaml] +---- +- hosts: localhost + roles: + - role: standard-test-basic // <1> + tags: + - classic + tests: + - simple: + dir: . + run: "somebinary --help" // <2> +---- +<1> this is a standard test role, it takes care of the test environment, logging, archiving results, etc +<2> this is your test command, its exit code defines the outcome of the test + +=== Step 2 - ... === + +Actually there is none. It is done. + +Submit your change as a pull request to see test results in Pagure +interface, or push it directly to dist-git and get new test results +every time you build package in Koji. + +NOTE: Test will be running in a *non-blocking* mode until you configure +gating for it. + +== Tests in a (sub)package == + +=== Task === + +There is a package `somepackage`, and there is an integration test suite +for it packaged in a separate `somepackage-tests` subpackage, which provides the +`/usr/bin/somepackage-tests` binary. + +There goal is to trigger this binary as a test for a main package. + +=== Solution === + +Same as above, you need to create a `tests.yml` configuration file with +the following content: + +.rpms/my-package.git:tests/test.yml +[source,yaml] +---- +--- +- hosts: localhost + roles: + - role: standard-test-basic + tags: + - classic + required_packages: + - somepackage-tests # <1> + + tests: + - subpackage_tests: # <2> + dir: . + run: somepackage-runtest # <3> +---- +<1> additional package which needs to be installed in the test environment +<2> any string, will be used as a name for artifacts and test results +<3> test execution command, provided by the somepackage-tests +subpackage. If -tests subpackage doesn't provide binary in the system +PATH, use full path here. + +== Questions == + +=== What if I want to run not one but a sequence of commands? === + +Put a bash script in **tests/scripts/** folder and run it from the +playbook. + +.rpms/somepackage.git: +---- +. +├── 0001-something.patch +├── somepackage.spec +├── sources +└── tests + ├── scripts + │ └── run_tests.sh # <1> + └── tests.yml +---- +<1> your custom test scenario + +Configure dist-git test to run this script: +[source,yaml] +.... +- hosts: localhost + roles: + - role: standard-test-basic # <1> + tags: + - classic + tests: + - simple: + dir: scripts # <2> + run: ./run_tests.sh # <3> +.... +<1> same standard role +<2> switch to subfolder (path is relative to `tests/` folder) +<3> this is the test script, its exit code is the outcome of the test + +=== What is under the hood? === + +To test the build we: + +* checkout dist-git repo +* take latest qcow image of Fedora Rawhide +* install all packages from the koji build on it +* run ansible playbook defined in tests.yml + +=== How do I verify my configuration? === + +It is possible to run and debug standard test roles locally. But we +highly recommend to use the pull-request workflow for it: simply create +a pull-request and wait for CI to react on it. + +We trigger almost the same CI machinery for PR testing as it is used for +gating of new builds. + +And as soon as result of the test is ready, it will appear on the pull +request page in http://src.fedoraproject.org/[Fedora Pagure] + +To restart the test add a comment to PR in Pagure, with the following +content: `[citest]` \ No newline at end of file