PR#1613: Add test_import_module - fm-orchestrator

fm-orchestrator

#1613 Add test_import_module

Merged 3 years ago by breilly. Opened 4 years ago by jobrauer.

jobrauer/fm-orchestrator test_import_module into master

jobrauer • 4 years ago

98ed3ab

tests/integration/example.test.env.yaml

file modified

		`@@ -96,3 +96,5 @@`
		`# with koji_tag_with_modules from testmodule's platform (stream 8.1.0.z)`
		`module: testmodule`
		`branch: test-reuse-tagged-module`
		`+ import_module:`
		`+ scmurl: "git://src.stg.fedoraproject.org/modules/testmodule?#test-import-module"`

tests/integration/test_import_module.py

file added

+14

		`@@ -0,0 +1,14 @@`
		`+`
		`+ def test_import_module(scenario, mbs):`
		`+ """`
		`+ Test import module functionality.`
		`+`
		`+ Steps:`
		`+ * Request module import with scmurl provided by test.env.yaml.`
		`+ Checks:`
		`+ * Non-error response.`
		`+ """`
		`+ scmurl = scenario.get("scmurl")`
		`+ assert scmurl, "No SCM URL specified in test.env.yaml file."`
		`+`
		`+ mbs.import_module(scmurl)`

tests/integration/utils.py

file modified

+43 -15

		`@@ -1,17 +1,20 @@`
		`# -- coding: utf-8 --`
		`# SPDX-License-Identifier: MIT`

		`+ import os`
		`import re`
		`import sys`
		`import time`

		`from kobo import rpmlib`
		`+ import json`
		`import koji`
		`- import yaml`
		`+ import pytest`
		`import requests`
		`- import tempfile`
		`+ import requests_kerberos`
		`import sh`
		`- import os`
		`+ import tempfile`
		`+ import yaml`

		`our_sh = sh(_out=sys.stdout, _err=sys.stderr, _tee=True)`
		`from our_sh import Command, git, pushd # noqa`
		`@@ -85,10 +88,9 @@`
		`return self._session.getTag(tag_info)`

		`def get_macro_specfile(self, build):`
		`- """`
		`- Download macro src.rpm and extract spec file .`
		`+ """Download macro src.rpm and extract spec file .`

		`- :param build: build object`
		`+ :param Build build: build object`
		`:return: content of module-build-macros.spec`
		`:rtype: str`
		`"""`
		`@@ -142,8 +144,7 @@`

		`@property`
		`def platform(self):`
		`- """`
		`- List of platforms in the modulemd file, obtaining values differs on version`
		`+ """List of platforms in the modulemd file, obtaining values differs on version`

		`:return: List of platforms in the modulemd file`
		`:rtype: list of strings`
		`@@ -334,8 +335,7 @@`
		`return {comp["package"]: comp["task_id"] for comp in self.components()}`

		`def batches(self):`
		`- """`
		`- Components of the module build separated in sets according to batches`
		`+ """Components of the module build separated in sets according to batches`

		`:return: list of components according to batches`
		`:rtype: list of sets`
		`@@ -401,8 +401,7 @@`
		`return False`

		`def get_modulemd(self):`
		`- """`
		`- Get module's metadata (from MBS API)`
		`+ """Get module's metadata (from MBS API)`

		`:return: module's metadata`
		`:rtype: dict`
		`@@ -469,9 +468,9 @@`
		`def get_builds(self, module, stream, order_desc_by=None):`
		`"""Get list of Builds objects via mbs api.`

		`- :attribute string module: Module name`
		`- :attribute string stream: Stream name`
		`- :attribute string order_desc_by: Optional sorting parameter e.g. "version"`
		`+ :param str module: Module name`
		`+ :param str stream: Stream name`
		`+ :param str order_desc_by: Optional sorting parameter e.g. "version"`
		`:return: list of Build objects`
		`:rtype: list`
		`"""`
		`@@ -483,6 +482,35 @@`
		`r.raise_for_status()`
		`return [Build(self._mbs_api, build["id"]) for build in r.json()["items"]]`

		`+ def import_module(self, scmurl):`
		`+ """Import module from SCM URL (modulemd).`
		`+`
		`+ :param str scmurl:`
		`+ :return: requests response`
		`+ :rtype: requests response object`
		`+ """`
		`+ url = f"{self._mbs_api}import-module/"`
		`+ headers = {"Content-Type": "application/json"}`
		`+ data = json.dumps({'scmurl': scmurl})`
		`+`
		`+ # (!) User executing this request must be allowed to do so on the target MBS instance.`
		`+ # MBS server does not support mutual auth, so make it optional (inspired by mbs-cli).`
		`+ auth = requests_kerberos.HTTPKerberosAuth(mutual_authentication=requests_kerberos.OPTIONAL)`
		`+`
		`+ response = requests.post(`
		`+ url,`
		`+ auth=auth,`
		`+ headers=headers,`
		`+ verify=False,`
		`+ data=data`
		`+ )`
		`+ try:`
		`+ response.raise_for_status()`
		`+ return response`
		`+ except requests.exceptions.HTTPError:`
		`+ # response message contains useful information, which requests module omits`
		`+ pytest.fail(response.text)`
		`+`
		`def get_module_builds(self, **kwargs):`
		`"""`
		`Query MBS API on module-builds endpoint`

tox.ini

file modified

		`@@ -80,6 +80,7 @@`
		`# rpm is an optional dependency for kobo, rpm-py-installer makes it work`
		`rpm-py-installer`
		`sh`
		`+ requests_kerberos`
		`# Set this to /etc/pki/tls/certs/ca-bundle.crt, for example,`
		`# if the instance tested has a self-signed certificate.`
		`passenv = REQUESTS_CA_BUNDLE MBS_TEST_CONFIG MBS_TEST_WORKERS HOME`

no initial comment

pretty please pagure-ci rebuild

4 years ago

rebased onto caada06a175bdfe020a23e4d963ee680010e5fc4

4 years ago

rebased onto 6d1d61c5f2f486e33caac88e3b94efa91c13b0c3

4 years ago

breilly commented on line 6 of tests/integration/example.test.env.yaml 4 years ago

I don't think we should use an internal link for testing; could we use a different host? Perhaps a pagure repo for it.

jobrauer commented 4 years ago

@breilly I thought that's the reason why we have a separate internal repo for test configuration (place where test.env is located) - to hide internal 'things'. Maybe you simply object to the private link being in the test.env example?

If not, I've been trying to import some modules from 'https://src.fedoraproject.org/', but I wasn't able to quickly find a suitable one, as there are different expectations from modulemd while doing an import.. Perhaps you'd be able to help here?

breilly commented 4 years ago

@jobrauer Right yes, it's just the private link used for the scmurl here, I think it should be using hosting that is publicly available, in case someone outside Red Hat wants to try to run these integration tests on their own deployments.

breilly commented 4 years ago

Or perhaps even make it configurable if that is an easier solution.

jobrauer commented 4 years ago

@breilly I believe that's the case now. Integration tests were designed to be only executable with an test.env.yaml configuration. Hence anybody wanting to execute these tests must provide it (in this case with whatever scmurl they wish).

breilly commented 4 years ago

Ah, right, I see. Well, if the expectation is that this is just an example which they will have to configure on their own, perhaps we could use some fake link?
Like "git://src.stg.fedoraproject.org/modules/testmodule?#test-import-module" or something along those lines.

rebased onto df2afd936bf00e3ae911bfcf4a8e77c4411be4a8

4 years ago

rebased onto 98ed3ab

4 years ago

jobrauer commented 4 years ago

@breilly done.

jobrauer commented 3 years ago

@breilly Is this alright now? Could we merge? Thanks.

breilly commented 3 years ago

LGTM, merging.

breilly commented 3 years ago

Commit 4195f06 fixes this pull-request