PR#1474: Bootstrap integration tests - fm-orchestrator

fm-orchestrator

#1474 Bootstrap integration tests

Merged 4 years ago by csomh. Opened 4 years ago by csomh.

csomh/fm-orchestrator integration-tests into master

Hunor Csomortáni • 4 years ago

0066c79

.gitignore

file modified

		`@@ -23,3 +23,4 @@`
		`requirements.txt.orig`
		`mbstest.db`
		`htmlcov/`
		`+ test.env.yaml`

tests/integration/README.rst

file added

+33

		`@@ -0,0 +1,33 @@`
		`+ ==============================================`
		`+ Integration tests for the Module Build Service`
		`+ ==============================================`
		`+`
		`+ This directory stores the integration tests for MBS.`
		`+`
		`+ Configuration`
		`+ =============`
		`+`
		+ The tests should be configured by a ``test.env.yaml`` file placed in the
		`+ top-level directory of this repository. This can be changed to a different`
		+ path by setting ``MBS_TEST_CONFIG``.
		`+`
		+ See `tests/integration/example.test.env.yaml`_ for the list of configuration
		`+ options and examples.`
		`+`
		`+ Running the tests`
		`+ =================`
		`+`
		`+ Tests can be triggered from the top-level directory of this repository with::`
		`+`
		`+ tox -e integration`
		`+`
		+ Note, that the ``integration`` environment is not part of the default ``tox``
		`+ envlist.`
		`+`
		+ ``REQUESTS_CA_BUNDLE`` is passed in ``tox.ini`` for the ``integration``
		`+ environment in order to enable running the tests against MBS instances which`
		`+ have self-signed certificates. Example usage::`
		`+`
		`+ REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -e integration`
		`+`
		`+ .. _tests/integration/example.test.env.yaml: example.test.env.yaml`

tests/integration/conftest.py

file added

+39

		`@@ -0,0 +1,39 @@`
		`+ # -- coding: utf-8 --`
		`+ # SPDX-License-Identifier: MIT`
		`+`
		`+ import os`
		`+`
		`+ import yaml`
		`+ import pytest`
		`+`
		`+ from utils import MBS, Git, Koji`
		`+`
		`+`
		`+ def load_test_env():`
		`+ """Load test environment configuration`
		`+`
		`+ :return: Test environment configuration.`
		`+ :rtype: dict`
		`+ """`
		`+ config_file = os.getenv("MBS_TEST_CONFIG", "test.env.yaml")`
		`+ with open(config_file) as f:`
		`+ env = yaml.safe_load(f)`
		`+ return env`
		`+`
		`+`
		`+ test_env = load_test_env()`
		`+`
		`+`
		`+ @pytest.fixture(scope="session")`
		`+ def mbs():`
		`+ return MBS(test_env["mbs_api"])`
		`+`
		`+`
		`+ @pytest.fixture(scope="session")`
		`+ def git():`
		`+ return Git(test_env["git_url"])`
		`+`
		`+`
		`+ @pytest.fixture(scope="session")`
		`+ def koji():`
		`+ return Koji(**test_env["koji"])`

tests/integration/example.test.env.yaml

file added

		`@@ -0,0 +1,9 @@`
		`+ ---`
		`+ # API endpoint of the MBS instance under test.`
		`+ mbs_api: https://mbs.fedoraproject.org/module-build-service/2/module-builds/`
		`+ # Git instance used by the build system.`
		`+ git_url: https://src.fedoraproject.org/`
		`+ # Koji instance the MBS instance under test communicates with.`
		`+ koji:`
		`+ server: https://koji.fedoraproject.org/kojihub`
		`+ topurl: https://kojipkgs.fedoraproject.org/`

tests/integration/test_normal_build.py

file added

		`@@ -0,0 +1,8 @@`
		`+ # -- coding: utf-8 --`
		`+ # SPDX-License-Identifier: MIT`
		`+`
		`+`
		`+ def test_normal_build():`
		`+ """ TODO(csomh): implement this test`
		`+ """`
		`+ assert True`

tests/integration/utils.py

file added

+21

		`@@ -0,0 +1,21 @@`
		`+ # -- coding: utf-8 --`
		`+ # SPDX-License-Identifier: MIT`
		`+`
		`+`
		`+ class MBS:`
		`+`
		`+ def __init__(self, api):`
		`+ self._api = api`
		`+`
		`+`
		`+ class Git:`
mprahl commented 4 years ago Optional: I'm not sure these will ever be run with Python 2, but the MBS codebase and unit tests are Python 2 and 3 compliant. To make this consistent, it might make sense to make these new style classes by having them inherit `object`.
		`+`
		`+ def __init__(self, url):`
		`+ self._url = url`
		`+`
		`+`
		`+ class Koji:`
		`+`
		`+ def __init__(self, server, topurl):`
		`+ self._server = server`
		`+ self._topurl = topurl`

tox.ini

file modified

+19 -2

		`@@ -27,6 +27,10 @@`
		`deps = -r{toxinidir}/test-requirements.txt`
		`commands =`
		`py.test -v \`
		`+ --ignore tests/integration \`
		`+ --cov module_build_service \`
		`+ --cov-report html \`
		`+ --cov-report term \`
		`-W "ignore:Use .persist_selectable:DeprecationWarning" \`
		`-W "ignore:The ConnectionEvents.dbapi_error() event is deprecated and will be removed in a future release. Please refer to the ConnectionEvents.handle_error() event.:DeprecationWarning" \`
		`{posargs}`
		`@@ -55,5 +59,18 @@`
		`/bin/bash -c "bandit -r -ll $(find . -mindepth 1 -maxdepth 1 ! -name tests ! -name \.\* -type d -o -name \*.py)"`
		`ignore_outcome = True`

		`- [pytest]`
		`- addopts = --cov module_build_service --cov-report html --cov-report term`
		`+ [testenv:integration]`
		`+ basepython = python3`
		`+ skipsdist = true`
		`+ skip_install = true`
		`+ sitepackages = false`
		`+ # let's handle integration test deps separately`
		`+ deps =`
		`+ pytest`
		`+ requests`
		`+ PyYAML`
		`+ # Set this to /etc/pki/tls/certs/ca-bundle.crt, for example,`
		`+ # if the instance tested has a self-signed certificate.`
		`+ passenv = REQUESTS_CA_BUNDLE MBS_TEST_CONFIG`
		`+ commands =`
		`+ pytest -vv --confcutdir=tests/integration {posargs:tests/integration}`

csomh commented 4 years ago

This is the first step to have some tests, that we could run against an
MBS instance, to check that it's functionally correct. Ultimately, these
will replace the test scripts (contrib/test-*).

This doesn't really do anything yet, but I would like to make sure that
everyone is on the same page regarding how this will be set up.

Signed-off-by: Hunor Csomortáni csomh@redhat.com

csomh commented 4 years ago

@mprahl ptal

mprahl commented on line 1 of tests/integration/README.md 4 years ago

Optional: I personally prefer markdown, but the documentation outside of the openshift directory is all RST. It might be worth making this RST as well to be consistent.

mprahl commented on line 1 of tests/integration/test_normal_build.py 4 years ago

Could you add the license header to this file?

mprahl commented on line 12 of tests/integration/utils.py 4 years ago

Optional: I'm not sure these will ever be run with Python 2, but the MBS codebase and unit tests are Python 2 and 3 compliant. To make this consistent, it might make sense to make these new style classes by having them inherit object.

mprahl commented 4 years ago

:thumbsup: once the license header is added. It's up to you if you want to address the optional comments.

cqi commented 4 years ago

Ultimately, these will replace the test scripts in contrib/

Script under contrib/ is to run unit tests inside container. Any reason to replace it with the integration tests?

rebased onto 1da861d46aa5da7b40056f1179db6ed90a864831

4 years ago

rebased onto 0066c79

4 years ago

csomh commented 4 years ago

Optional: I personally prefer markdown, but the documentation outside of the openshift directory is all RST. It might be worth making this RST as well to be consistent.

I also prefer markdown, but converted to RST for consistency.

Could you add the license header to this file?

Added.

Optional: I'm not sure these will ever be run with Python 2, but the MBS codebase and unit tests are Python 2 and 3 compliant. To make this consistent, it might make sense to make these new style classes by having them inherit object.

As the integration tests are not tightly coupled with the rest of the code, and will usually run from a different host than MBS is deployed, I choose to give up on Python 2 compatibility.