#49718 Issue 49717 - Add conftest.py for tests
Closed 3 years ago by spichugi. Opened 5 years ago by vashirov.
vashirov/389-ds-base conftest  into  master

@@ -0,0 +1,46 @@ 

+ import pytest

+ import subprocess

+ 

+ pkgs = ['389-ds-base', 'nss', 'nspr', 'openldap', 'cyrus-sasl']

+ 

+ 

+ def get_rpm_version(pkg):

+     try:

+         result = subprocess.check_output(['rpm', '-q', '--queryformat',

+                                           '%{VERSION}-%{RELEASE}', pkg])

+     except:

+         result = b"not installed"

+ 

+     return result.decode('utf-8')

+ 

+ 

+ def is_fips():

+     # Are we running in FIPS mode?

+     with open('/proc/sys/crypto/fips_enabled', 'r') as f:

+         return f.readline()

+ 

+ 

+ @pytest.fixture(autouse=True)

+ def _environment(request):

+     if "_metadata" in dir(request.config):

+         for pkg in pkgs:

+             request.config._metadata[pkg] = get_rpm_version(pkg)

+         request.config._metadata['FIPS'] = is_fips()

+ 

+ 

+ def pytest_report_header(config):

+     header = ""

+     for pkg in pkgs:

+         header += pkg + ": " + get_rpm_version(pkg) + "\n"

+     header += "FIPS: " + is_fips()

+     return header

+ 

+ 

+ @pytest.mark.optionalhook

+ def pytest_html_results_table_header(cells):

+     cells.pop()

+ 

+ 

+ @pytest.mark.optionalhook

+ def pytest_html_results_table_row(report, cells):

+     cells.pop()

Bug Description:
From the test's output it's not possible to tell what version of
389-ds-base was used during the test, what are the versions of
libraries that we depend on, etc.

Fix Description:
We can use conftest.py to print this useful information. This also will
be reflected in a html report, if py.test was used with --html option.

https://pagure.io/389-ds-base/issue/49717

Reviewed by: ???

Sample output:

============================================= test session starts ==============================================
platform linux -- Python 3.6.3, pytest-3.6.0, py-1.5.3, pluggy-0.6.0 -- /opt/rh/rh-python36/root/usr/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.6.3', 'Platform': 'Linux-3.10.0-862.el7.x86_64-x86_64-with-redhat-7.5-Maipo', 'Packages': {'pytest': '3.6.0', 'py': '1.5.3', 'pluggy': '0.6.0'}, 'Plugins': {'metadata': '1.7.0', 'html': '1.18.0'}}
389-ds-base: 1.3.7.5-22.el7_5
nss: 3.34.0-4.el7
nspr: 4.17.0-1.el7
openldap: 2.4.44-13.el7
cyrus-sasl: 2.1.26-23.el7
FIPS: 0

rootdir: /export/ds/dirsrvtests, inifile:
plugins: metadata-1.7.0, html-1.18.0
collected 0 items                                                                                              

========================================= no tests ran in 0.15 seconds =========================================

Looks good to me. Ack.

P.S. we were using this in QE for a while. It has proved itself very useful.

Hey there.

I really like the idea of this, but there may be some issues. First, what about "non rpm" systems? What about sudo make install etc?

Why not read the version string from defaults.py. I'm pretty sure that I had it written in there for exactly this purpose.

It would be much simpler, and "vendor neutral".

So "no ack" from me, I think there is a simpler way :)

First, what about "non rpm" systems?

Do we run tests on non-rpm systems? If somebody does that, I'd be glad to add support for that. In current state it won't fail, just will display that package is not installed. I don't want to add support for something that currently nobody uses.

What about sudo make install etc?

Yes, that's a corner case. Other dependencies will be listed though. Also this change is to simplify upstream and downstream testing, to use common tools for CI. Officially we support only rpm installs there anyway. Rpms is what we ship to customers, not making them run sudo make install.

Why not read the version string from defaults.py. I'm pretty sure that I had it written in there for exactly this purpose.

Because it lists only version string, but not full version string, including prerel, like 1.4.0.9-20180525git0123abcde, only 1.4.0.9. In CI that's useless, because we want to know version down to commit.

It would be much simpler, and "vendor neutral".

I'm sorry, but there is no easy way to support it in a vendor neutral way, unless we change the versioning mechanism used by DS. For example, "true" version of DS is something like this:

        389-Directory/1.4.0.9 B2018.135.1811
        server.example.com:389 (/etc/dirsrv/slapd-server)

that is logged during the startup.

And you don't have to use it. In your development environment you can always pass parameter to disable it:

--noconftest          Don't load any conftest.py files.

I liked this change and it is useful.
Under the Results table , currently Links column is empty,also the test links are not clickable, can we make them hyperlinks which can point to the test case code (may be covered by another pagure issue). Just a minor suggestion.

Links column is added by pytest-html plugin, we can remove it altogether, since it's not needed. As for clickable test names, it's possible to override default report, I'll see what I can do there.

rebased onto 67f9bc63e4678b342a4d1595c3ae2a259f1b6d56

5 years ago

I've removed Links column.
Clickable test names introduce additional complexity (handling test parameters, relative/absolute file location, repo location, etc.) to the report generation, that I think is not worth it. This report is not a replacement for a test case management system, where this information is more appropriate, IMO.

If no one else objects, I'd like to merge this. Any thoughts?

rebased onto 08b2d88

5 years ago

Pull-Request has been merged by vashirov

5 years ago

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This pull request has been cloned to Github as issue and is available here:
- https://github.com/389ds/389-ds-base/issues/2777

If you want to continue to work on the PR, please navigate to the github issue,
download the patch from the attachments and file a new pull request.

Thank you for understanding. We apologize for all inconvenience.

Pull-Request has been closed by spichugi

3 years ago
Metadata