| |
@@ -0,0 +1,397 @@
|
| |
+ .. highlight:: none
|
| |
+
|
| |
+ *********************************
|
| |
+ Running and developing SSSD tests
|
| |
+ *********************************
|
| |
+
|
| |
+ SSSD is a complex piece of software with a long development
|
| |
+ history. Therefore, there are several layers of tests with different goals
|
| |
+ and using different frameworks. This page shows how to run the tests
|
| |
+ and how to add new ones or modify the existing tests.
|
| |
+
|
| |
+ Existing test tiers
|
| |
+ ===================
|
| |
+
|
| |
+ Each test is different. Sometimes, you want to test the whole system end-to-end,
|
| |
+ sometimes the test should exercise some corner case for which special input
|
| |
+ and environment must be simulated. This section should give you a better idea
|
| |
+ what kind of tests already exist in SSSD so that you can choose where to add
|
| |
+ a new test and also provides a general overview.
|
| |
+
|
| |
+ Unit tests
|
| |
+ ----------
|
| |
+ Unit tests typically run a function or a tevent request without running
|
| |
+ the full deamon. The unit tests in SSSD are developed using either the
|
| |
+ `check <https://libcheck.github.io/check/>`__ library or the `cmocka
|
| |
+ <https://cmocka.org>`__ library.
|
| |
+
|
| |
+ It might sound strange that two different C unit testing libraries are used.
|
| |
+ The reason is mostly historical - when SSSD was started, the check library
|
| |
+ was the best choice, but it had become unmaintained for some time. While
|
| |
+ check has seen some development happening since then, the SSSD team had
|
| |
+ moved to using cmocka in the meantime. In addition, cmocka has support for
|
| |
+ mocking values using the ``mock`` and ``will_return`` functions. Therefore,
|
| |
+ cmocka should be used for any new unit tests added to SSSD.
|
| |
+
|
| |
+ The unit tests are fast to execute and in general this is where corner
|
| |
+ cases are typically easiest to test as you can provide false or unexpected
|
| |
+ input to the code under test. Unit tests are also often used to test a library's
|
| |
+ API.
|
| |
+
|
| |
+ An important part of many tests using cmocka is wrapping a function provided
|
| |
+ by an external library using the ``ld`` linker's ``--wrap`` feature. You
|
| |
+ can learn more about cmocka and this feature in a `lwn.net article the cmocka
|
| |
+ developers contributed <https://lwn.net/Articles/558106/>`__. In the SSSD
|
| |
+ source tree, the unit tests reside under ``src/tests/*.c`` (check-based tests)
|
| |
+ and ``src/tests/cmocka/*.c``.
|
| |
+
|
| |
+ To run the tests, make sure both the cmocka and check libraries are installed
|
| |
+ on your system. On Fedora/RHEL, the package names are ``libcmocka-devel``
|
| |
+ and ``check-devel``. Running ``make check`` from your build directory will
|
| |
+ then execute all the unit tests.
|
| |
+
|
| |
+ Testing for talloc context memory growth
|
| |
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
| |
+ Talloc can be a double-edged sword sometimes. On the one hand, talloc greatly
|
| |
+ simplifies memory management, on the other hand, using talloc creates a
|
| |
+ risk that a memory related to some operation is allocated using a top-level
|
| |
+ memory context and outlives the lifetime of the related request. To make
|
| |
+ sure we catch errors like this, our tests contain several useful functions
|
| |
+ that record the amount of memory a talloc context takes before an operation
|
| |
+ begins and compares that amount of memory after the operation finishes. The
|
| |
+ functions to set up and tear down the memory leak detection are called
|
| |
+ ``leak_check_setup`` and ``leak_check_teardown.`` For every operation,
|
| |
+ you'll want to record the amount of memory taken before the operation with
|
| |
+ ``check_leaks_push`` and then check the amount of memory taken after the
|
| |
+ operation with ``check_leaks_pop``.
|
| |
+
|
| |
+ Examples of what can be tested by unit tests
|
| |
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
| |
+ A typical use-case is the sysdb API tests at
|
| |
+ e.g. ``src/tests/sysdb-tests.c``.
|
| |
+
|
| |
+ A less typical use-case is testing of the NSS or PAM responders in isolation.
|
| |
+ The NSS responder test is located at ``src/tests/cmocka/test_nss_srv.c``.
|
| |
+ Normally, the NSS responder would require a client such as getent to talk
|
| |
+ to it through the nss_sss module and would send requests to and receive
|
| |
+ replies from a back end. In unit tests, the NSS client's input is simulatd
|
| |
+ by calling the ``sss_cmd_execute`` directly, but with mocked input (see
|
| |
+ e.g. ``mock_input_user_or_group``. The test even fakes communication
|
| |
+ to the Data Provider by mocking the ``sss_dp_get_account_send`` and
|
| |
+ ``sss_dp_get_account_recv`` request that normally talks to the Data Provider
|
| |
+ over D-Bus (see e.g. the ``test_nss_getpwnam_search`` test).
|
| |
+
|
| |
+ Integration tests
|
| |
+ -----------------
|
| |
+ SSSD integration tests run the deamon at the same machine you are developing
|
| |
+ on with the help from the `cwrap <https://cwrap.org>` libraries. The
|
| |
+ integration tests are half-way between the unit tests that call APIs or
|
| |
+ run a single component in isolation and between the multihost tests that
|
| |
+ run on a dedicated VM. During the integration tests, a build of SSSD is
|
| |
+ compiled and installed into an environment set up with the help of the
|
| |
+ ``fakeroot`` program. Then, the cwrap libraries are preloaded into the
|
| |
+ test environment. The socket_wrapper library provides networking through
|
| |
+ UNIX pipes, the uid_wrapper library provides the notion of running as root
|
| |
+ and the nss_wrapper library allows to route requests for users and groups
|
| |
+ through the NSS module under test.
|
| |
+
|
| |
+ The advantage over the unit tests is obvious, the full deamon is
|
| |
+ ran and you can talk to the SSSD using the same interfaces as a user
|
| |
+ would do in production, e.g. resolve a user with ``getpwnam``. Because
|
| |
+ the tests are ran on the same machine as the developer works on,
|
| |
+ is is much faster to compile a new SSSD version for the tests to
|
| |
+ run and so the develop-test-fix cycle is generally quite fast. The
|
| |
+ integration tests also offer a simple way to add a "breakpoint" to the
|
| |
+ tests and connect to the tests using ``screen(1)``. Finally,
|
| |
+ since the tests run on the same machine, they can trivially run on
|
| |
+ any OS release or any distribution with little to no changes, even
|
| |
+ in build systems that typically have no network connectivity as part of
|
| |
+ the SSSD build.
|
| |
+
|
| |
+ The disadvantages also stem from running the tests on the local machine.
|
| |
+ SSSD relies on whatever server it is connecting to to also run in the
|
| |
+ test environment provided by the cwrap libraries, but in many cases that
|
| |
+ is so difficult that we even haven't done the work (e.g. FreeIPA) or
|
| |
+ outright impossible (Active Directory). Even within the tests themselves,
|
| |
+ we sometimes stretch the limits of the cwrap libraries. As an example,
|
| |
+ the socket_wrapper library doesn't support faking the client credentials
|
| |
+ that the SSSD reads using the ``getsockopt`` call with the ``SO_PEERCRED``
|
| |
+ parameter.
|
| |
+
|
| |
+ Running integration tests
|
| |
+ ^^^^^^^^^^^^^^^^^^^^^^^^^
|
| |
+ The easiest way to run the integration tests is by running: ::
|
| |
+
|
| |
+ make intgcheck
|
| |
+
|
| |
+ This makefile target consists of two targets, actually::
|
| |
+
|
| |
+ make intgcheck-prepare
|
| |
+ make intgcheck-run
|
| |
+
|
| |
+ The former builds the special SSSD build and creates the environment
|
| |
+ for tests. The latter actually runs the tests.
|
| |
+
|
| |
+ Running the complete suite of tests may be overkill for debugging.
|
| |
+ Running individual tests from the suite can be done according to the
|
| |
+ following examples: ::
|
| |
+
|
| |
+ make intgcheck-prepare
|
| |
+ INTGCHECK_PYTEST_ARGS="-k test_netgroup.py" make intgcheck-run
|
| |
+ INTGCHECK_PYTEST_ARGS="test_netgroup.py -k test_add_empty_netgroup" make intgcheck-run
|
| |
+
|
| |
+ The ``INTGCHECK_PYTEST_ARGS`` format can be checked in the `PyTest
|
| |
+ official
|
| |
+ documentation <http://doc.pytest.org/en/latest/contents.html>`__.
|
| |
+
|
| |
+ Sometimes, during test development, you find out that the code needs to
|
| |
+ be fixed and then you'd like to re-run some tests. To do so, you need
|
| |
+ to first have the environment prepared by running ``intgcheck-prepare``.
|
| |
+ This needs to be done only once per "debugging session". Then, after you've
|
| |
+ done the required changes to the SSSD code, navigate into the ``intg/bld``
|
| |
+ subdirectory in your build directory and recompile and re-install the
|
| |
+ test build::
|
| |
+
|
| |
+ cd intg/bld
|
| |
+ make
|
| |
+ make -j1 install # Sometimes parallel installation causes issues
|
| |
+
|
| |
+ Now, re-running make intgcheck-run (optionally with any parameters, like
|
| |
+ only a subset of tests) would run your modified code!
|
| |
+
|
| |
+ Debugging integration tests
|
| |
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
| |
+ There are three basic ways to debug the integration tests - add print
|
| |
+ statements to the test, read the SSSD logs from the test directory and
|
| |
+ insert a breakpoint.
|
| |
+
|
| |
+ Print statements can be useful to know what's going on in the test code
|
| |
+ itself, but not the SSSD. As a general note, the tests remove the logs
|
| |
+ after a successful run and also suppress stdout during a successful run,
|
| |
+ so in order to make use of either print statements or the logs, you might
|
| |
+ need to fail the test on purpose e.g. by adding::
|
| |
+
|
| |
+ assert 1 == 0
|
| |
+
|
| |
+ The debug logs might be useful to get an insight into the SSSD. Let's
|
| |
+ pretend we want to debug the test called ``test_add_empty_netgroup``.
|
| |
+ We would add the dummy assert to fail the test first. Then, in the test
|
| |
+ fixture, we'd locate the function that generates the ``sssd.conf`` (often
|
| |
+ the function is called ``format_basic_conf`` in many tests) and we'd
|
| |
+ add the ``debug_level`` parameter::
|
| |
+
|
| |
+ --- a/src/tests/intg/test_netgroup.py
|
| |
+ +++ b/src/tests/intg/test_netgroup.py
|
| |
+ @@ -109,6 +109,7 @@ def format_basic_conf(ldap_conn, schema):
|
| |
+ disable_netlink = true
|
| |
+
|
| |
+ [nss]
|
| |
+ + debug_level = 10
|
| |
+
|
| |
+ [domain/LDAP]
|
| |
+ {schema_conf}
|
| |
+
|
| |
+ Next, we can run the test, expecting it to fail::
|
| |
+
|
| |
+ INTGCHECK_PYTEST_ARGS="-k add_empty_netgroup" make intgcheck-run
|
| |
+
|
| |
+ In the test output, we locate the test directory which always starts with
|
| |
+ ``/tmp/sssd-intg-*``. This director contains the fake root and we can then
|
| |
+ do useful things such as read the logs from outside the build environment::
|
| |
+
|
| |
+ less /tmp/sssd-intg.1ifu0f6n/var/log/sssd/sssd_nss.log
|
| |
+
|
| |
+ The final option is to insert a breakpoint into the test and jump into the
|
| |
+ test environment with ``screen(1)``. The breakpoint is inserted by calling
|
| |
+ the ``run_shell()`` function from the ``util`` package. Again, using the
|
| |
+ ``test_add_empty_netgroup`` test as an example, we need to first import
|
| |
+ ``run_shell``::
|
| |
+
|
| |
+ from util import run_shell
|
| |
+
|
| |
+ Next, we call ``run_shell()`` from the test function and invoke
|
| |
+ ``intgcheck-run`` again. You will see that the test started, but did not
|
| |
+ finish with either pass or fail, it seemingly hangs. This is when we can
|
| |
+ check that there is a screen instance running and connect to it::
|
| |
+
|
| |
+ $ screen -ls
|
| |
+ There is a screen on:
|
| |
+ 21302.sssd_cwrap_session (Detached)
|
| |
+ 1 Socket in /run/screen/S-jhrozek.
|
| |
+ $ screen -r sssd_cwrap_session
|
| |
+
|
| |
+ From within the screen session, you can attach ``gdb`` to the SSSD processes,
|
| |
+ call ``getent`` to resolve users or groups ``ldbsearch`` the cache etc.
|
| |
+ To finish the debugging session, simply exit all the terminals in the tabs.
|
| |
+
|
| |
+ Examples
|
| |
+ ^^^^^^^^
|
| |
+ The tests themselves are located under ``src/tests/intg``. Each file corresponds
|
| |
+ to one "test area", like testing the LDAP provider or testing the KCM responder.
|
| |
+
|
| |
+ To see an example of adding test cases to existing tests, see commit
|
| |
+ ``76ce965fc3abfdcf3a4a9518e57545ea060033d6`` or for an example of
|
| |
+ adding a whole new test, including faking the client library (which
|
| |
+ should also illustrate the limits of the cwrap testing), see commit
|
| |
+ ``5d838e13351d3062346ca449e00845750b9447da`` and the two preceding it.
|
| |
+
|
| |
+ Multihost tests
|
| |
+ ---------------
|
| |
+ SSSD multihost tests are the closest our tests get to running SSSD
|
| |
+ in the real world. The multihost tests utilize a VM the tests are ran
|
| |
+ at, so no part of the setup is faked. This is also the test's biggest
|
| |
+ advantage, as long as you can prepare the test environment, the tests
|
| |
+ can then be used to test even Active Directory or FreeIPA integration.
|
| |
+ Also, unlike the cwrap tests or the unit tests, the multihost tests
|
| |
+ are typically good enough for distribution QE teams, so the multihost
|
| |
+ tests allow a collaboration between the team that typically just
|
| |
+ develops SSSD and the team that tests it.
|
| |
+
|
| |
+ The disadvantage of the tests is that setting up the environment can
|
| |
+ be complex and the development loop (the time between modifying test,
|
| |
+ modifying the SSSD sources, deploying them to the test environment and
|
| |
+ running the tests) is much longer than with the cwrap based tests.
|
| |
+
|
| |
+ Please note that at the time this page is written, the multihost
|
| |
+ tests are still work in progress. Running the tests is not as
|
| |
+ easy as it should be. This page documents the manual steps, but we
|
| |
+ do acknowledge that the setup should be automated further.
|
| |
+
|
| |
+ Running multihost tests
|
| |
+ ^^^^^^^^^^^^^^^^^^^^^^^
|
| |
+ First, the infrastructure does not yet concern itself with provisioning
|
| |
+ at all. You need to set up a VM to run the tests on yourself. As an example,
|
| |
+ we'll be running the tests on a Fedora 28 machine which we will create
|
| |
+ using the `vagrant <https://www.vagrantup.com>`__ tool. To make things
|
| |
+ at least a little more palatable, the tests are expected to clean up
|
| |
+ after themselves, so it's possible to reuse the same provisioned VM
|
| |
+ for multiple test runs.
|
| |
+
|
| |
+ You can start with initializing the vagrant environment::
|
| |
+
|
| |
+ $ vagrant init fedora/28-cloud-base
|
| |
+
|
| |
+ Next, assign your test VM some address and host name in the ``Vagrantfile``::
|
| |
+
|
| |
+ SERVER_HOSTNAME="testmachine.sssd.test"
|
| |
+ SERVER_IP_ADDRESS="192.168.122.101"
|
| |
+
|
| |
+ config.vm.define "testmachine" do |testmachine|
|
| |
+ testmachine.vm.network "private_network", ip: "#{SERVER_IP_ADDRESS}"
|
| |
+ testmachine.vm.hostname = "#{SERVER_HOSTNAME}"
|
| |
+ end
|
| |
+
|
| |
+ The multihost tests ssh to the test VM as root and generally expect
|
| |
+ to know the root password. Start the machine and change the password::
|
| |
+
|
| |
+ $ vagrant up
|
| |
+ $ vagrant ssh
|
| |
+ [vagrant@testmachine ~]$ sudo passwd root
|
| |
+
|
| |
+ I'll be using ``Secret123`` as the root password in this document.
|
| |
+
|
| |
+ Next, you need to make sure the host (i.e. your laptop) can resolve the
|
| |
+ guest. Provided that you use ``libvirt`` as your VM management, you can
|
| |
+ just add a line with the VM's host name and IP address to ``/etc/hosts``
|
| |
+ followed by sending the ``HUP`` signal to the ``dnsmasq`` daemon::
|
| |
+
|
| |
+ $ grep testmachine /etc/hosts
|
| |
+ 192.168.122.101 testmachine.sssd.test
|
| |
+ $ sudo pkill -HUP dnsmasq
|
| |
+ $ ping testmachine.sssd.test
|
| |
+ PING testmachine.sssd.test (192.168.122.101) 56(84) bytes of data.
|
| |
+ 64 bytes from testmachine.sssd.test (192.168.122.101): icmp_seq=1 ttl=64 time=0.371 ms
|
| |
+ ^C
|
| |
+ $ ssh root@testmachine.sssd.test # Use Secret123
|
| |
+
|
| |
+ Now that we have the test VM prepared, we can proceed to setting up the
|
| |
+ tests. For some reason, the tests run in a Python virtual environment
|
| |
+ and download some packages from PIP instead of relying on distribution
|
| |
+ packages. This is again something we should change at the very least to
|
| |
+ make it possible to run the multihost tests easily using a make target.
|
| |
+
|
| |
+ Nonetheless, let's describe the current state. Make sure the following
|
| |
+ packages are installed::
|
| |
+
|
| |
+ dnf install python3-pip python3-virtualenv
|
| |
+
|
| |
+ Create the Python virtual environment. The directory name is arbitrary::
|
| |
+
|
| |
+ $ virtualenv-3 /tmp/abc
|
| |
+ Using base prefix '/usr'
|
| |
+ New python executable in /tmp/abc/bin/python3
|
| |
+ Not overwriting existing python script /tmp/abc/bin/python (you must use /tmp/abc/bin/python3)
|
| |
+ Installing setuptools, pip, wheel...done.
|
| |
+
|
| |
+ Activate the virtual environment::
|
| |
+
|
| |
+ $ source /tmp/abc/bin/activate
|
| |
+ $ (abc) [root@master-7740 bin]#
|
| |
+
|
| |
+ You can verify the environment is active by inspecting the ``PATH``
|
| |
+ variable, the ``/tmp/abc/bin`` directory should be the first one.
|
| |
+
|
| |
+ Install the ``sssd-testlib`` into your virtualenv::
|
| |
+
|
| |
+ $ pwd
|
| |
+ # You should be at your SSSD checkout now
|
| |
+ $ cd src/tests/python
|
| |
+ $ python setup.py install
|
| |
+
|
| |
+ Install the required Python packages into the virtual environment::
|
| |
+
|
| |
+ $ pip install pytest pytest-multihost paramiko python-ldap PyYAML
|
| |
+
|
| |
+ We're almost there. The next step is to configure the test by
|
| |
+ telling the tests where to run. There is a template YAML file at
|
| |
+ ``src/tests/multihost/basic/mhc.yaml``. You can copy the file and
|
| |
+ add the details of your test machine like this::
|
| |
+
|
| |
+ $ cat /tmp/mhc.yaml
|
| |
+ windows_test_dir: '/home/Administrator'
|
| |
+ root_password: 'Secret123'
|
| |
+ domains:
|
| |
+ - name: testmachine.sssd.test
|
| |
+ type: sssd
|
| |
+ hosts:
|
| |
+ - name: testmachine.sssd.test
|
| |
+ external_hostname: testmachine.sssd.test
|
| |
+ role: master
|
| |
+
|
| |
+ Now we can finally move on to running the test!. Navigate to the
|
| |
+ ``src/tests/multihost`` directory and run::
|
| |
+
|
| |
+ py.test -s -v --multihost-config=/tmp/mhc.yaml
|
| |
+
|
| |
+ You can also add the ``-v`` switch to ``py.test`` to see more debug messages,
|
| |
+ including the commands that are executed on the test VM.
|
| |
+
|
| |
+ Shortening the development loop
|
| |
+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
| |
+ As you may have noticed, the tests run whatever packages the VM can install
|
| |
+ from its repositories. This is fine for testing of stable distributions or
|
| |
+ for usage from a CI engine, where the packages can be fetched from e.g. a
|
| |
+ COPR repository.
|
| |
+
|
| |
+ But for developers hacking on SSSD, normally what you want
|
| |
+ is to compile and install SSSD from your git checkout. One
|
| |
+ example of a workflow might be to use the `Vagrant shared folders
|
| |
+ <https://www.vagrantup.com/docs/synced-folders/>`__ to share the SSSD
|
| |
+ sources from the host machine. This allows you to use your favorite editor
|
| |
+ or IDE on your host machine and just compile and run the SSSD on the test VM.
|
| |
+
|
| |
+ There are several kinds of shared folders, but I've found that the sshfs
|
| |
+ shared folder has the best ease of use to performance ratio. Start by
|
| |
+ installing the ``vagrant-sshfs`` plugin. On Fedora, it is normally present
|
| |
+ in the repos.
|
| |
+
|
| |
+ Then, you can define the folder in your Vagrantfile::
|
| |
+
|
| |
+ SSSD_SRC="/home/remote/jhrozek/devel/sssd"
|
| |
+ testmachine.vm.synced_folder "#{SSSD_SRC}", "/sssd", type: "sshfs", sshfs_opts_append: "-o cache=no"
|
| |
+
|
| |
+ Note the ``-o cache=no`` option. This causes some extra network traffic,
|
| |
+ but since the VM is local, this is OK and makes sure that the changes are
|
| |
+ propagated from the host to the VMs immediately. Then, using this setup,
|
| |
+ you'll have the SSSD sources mounted at ``/sssd`` and you can build and
|
| |
+ install SSSD on the machine using the usual steps.
|
| |
jhrozek
commented 5 years ago
Describes what kind of tests are there in SSSD and how to run them, including the new multihost tests.