#77 Refactor the documentation for test automation development
Opened 6 years ago by spichugi. Modified 4 years ago

Revise our upstream documentation and describe the optimal way to use lib389 and pytest.


Metadata Update from @spichugi:
- Issue assigned to spichugi

6 years ago

Metadata Update from @spichugi:
- Custom field Origin adjusted to QE
- Issue priority set to: Normal

6 years ago

I've attached the draft version of the guide.
My proposal is:

  • create index.rst with welcome words and table of contents for whole lib389 documentation
  • put these chapters in reStructuredText format to lib389 as Quick start guide

    Basic workflow
    A ways to make your code better in a pytest way
        Fixtures
            Scope
            Parametrizing
        Test cases
            Parametrizing
            Marking test functions and selecting them for a run
            Asserting
    lib389 and python-ldap functions
        Constants
            Basic constants
        Add, Modify, and Delete Operations
        Search and Bind Operations
        Basic instance operations
        Setting up SSL/TLS
        Replication
            Basic configuration
            Replication tools
    
  • add docstrings to every module in lib389 and generate the rest of the docs with Sphinx

  • create 'example script' which contains most of the lib389 features that you can execute and see how it works (probably, I'll create a separate issue for that)
  • put all in a nice format to index.rst

A super tiny fix on USN:

# Optionaly you can specify 'suffix', 'backend' and 'maxusn_to_delete'
usnplugin.cleanup(suffix=None, backend=None, max_usn=None)

It is necessary to specify either suffix or backend otherwise this method will fail.
I had a manual check about this in the same method, but we decided to move it inside task's _validate() method instead. Maybe it would be good to put it here as well in order to avoid confusion?

git remote add $(whoami) git+ssh://git.engineering.redhat.com/srv/git/users/$(whoami)/389-ds-base.git

I don't think applies in a public guide ;) similar for mojo links.

Use ./dirsrvtests/create_test.py tool to generate new test.py file. Usage:

Might be a good note here to say to avoid the "tickets" folder?

Go to the "beaker" branch of dirsrv-tests repo, to the beaker/upstream-tests dir

    http://git.app.eng.bos.redhat.com/git/dirsrv-tests.git

^ Internal only.

For more info check the source code at https://pagure.io/lib389/blob/master/f/lib389/_constants.py . If you need a constant, use, do not import all of them:

    from lib389._constants import CONSTNATS_YOU_NEED

Could we worth showing the tuple import syntax,

Add, Modify, and Delete Operations

^ "Manual add, modify ..."

Should probably recommend dsldapobjects over this though.

Setting up SSL/TLS

This section is showing CERT external auth. You can remove a fair bit of this section to "focus" it on the SSL/TLS setup. AS well, worth mentioning that standlanoe will detect ldaps if you set secureport.

DSLdapObject operations

...
# Other parameters are carried out - it takes 'rdn' from properties, and 'basedn' from parent class
users.create(properties=user_properties)

# Get one DSLdapObject
# with 'selector'
user = users.get('testuser')

You can shortcut with user = users.create .... Because we return the newly created object.

# Delete
user.delete()

This probably needs a bigger comment warning ;)

Memory Leak Testing (valgrind)

I'd say here that ASAN+LSAN are faster and detect more, plus are 100% seamless to tests. No need for valgrind.

Password

Looks like we are missing PBKDF2_SHA256 here ;)

Otherwise mate, this is looking great. A lot of these can easily become pydoc comments for the related objects. Great work!

I've applied the comments and have added more chapters that were missing (after some IRC discussions)
After the review, I'll create rST formatted guide and I'll start to document the code.

The parts with internal actions won't be included in the upsteam rST guide.

Guidelines-for-using-pytest-and-lib389_47541162.html

Metadata Update from @spichugi:
- Custom field Review Status adjusted to None

6 years ago

I've applied the comments and have added more chapters that were missing (after some IRC discussions)
After the review, I'll create rST formatted guide and I'll start to document the code.

Can't open provided link :(

Can't open provided link :(

After clicking it will be downloaded and you can open it with your browser.
Also, you can just review the internal version.

Most of the comments were applied. Thanks!

Now the layout is:
- index.rst - welcoming word, Table of Contents and Contact Us.
- guidelines.rst - is included into index.rst and contains basic test development workflow, pytest features overview, and basic lib389 and python-ldap features.
- replica.rst, agreement.rst, aci.rst, etc. - each contains the respective documentation.

Currently I do add docstrings to lib389 and rst files with references. It will be the next patch.
Now, please, check if you agree with the existing format.

The generated example - https://fedorapeople.org/~spichugi/html/

Metadata Update from @spichugi:
- Custom field Review Status adjusted to review (was: None)

6 years ago

Metadata Update from @mreynolds:
- Custom field Review Status adjusted to ack (was: review)

6 years ago

commit 3200669
Author: Simon Pichugin spichugi@redhat.com
Date: Thu Aug 24 23:05:08 2017 +0200

I just read this, this is great. Thank you so much @spichugi this is great work!

Rewriting docstrings. Part 1.

It includes:
_mapped_object.py, aci.py, agreement.py, backend.py,
changelog.py, mappingTree.py, replica.py, repltools.py

Also, I've added a script for links to the code in the docs.

You can check it also here:
https://fedorapeople.org/~spichugi/html/index.html

That looks great. There is a lot of content here, so I think it will take time to review to get it "right".

What's the process if I update a module, does it automatically just use the docstrings from the objects?

I think I'm happy to ack this, because if we start using it, we'll iron out any issues as we go.

Metadata Update from @spichugi:
- Custom field Review Status adjusted to review (was: ack)

6 years ago

What's the process if I update a module, does it automatically just use the docstrings from the objects?

Yeah, so basically we have this main part in .rst file:

.. autoclass:: lib389.replica.Replica
    :members:
    :inherited-members:

:members: will parse all docstrings from public modules (it won't parse _method, and __func__)
:inherited-members: will additionaly parse all methods that we have in parent class (but if the method is in our class, it will skip it in the parent).

I think I'm happy to ack this, because if we start using it, we'll iron out any issues as we go.

Sure, feel free to set ack, if you're okay (I forgot to set 'review' yesterday)

Metadata Update from @firstyear:
- Custom field Review Status adjusted to ack (was: review)

6 years ago

commit 3a498ae
Author: Simon Pichugin spichugi@redhat.com
Date: Tue Aug 29 19:29:15 2017 +0200

Somehow this broke the tests. dbdir not found in DirSrv

py.test -v -x -s tests/suites/setup_ds/setup_ds_test.py 
================================================================================= test session starts =================================================================================
platform linux2 -- Python 2.7.13, pytest-2.9.2, py-1.4.34, pluggy-0.3.1 -- /usr/bin/python2
cachedir: tests/suites/setup_ds/.cache
rootdir: /home/mareynol/source/ds389/389-ds-base/dirsrvtests/tests/suites/setup_ds, inifile: 
collected 2 items 

tests/suites/setup_ds/setup_ds_test.py::test_slapd_InstScriptsEnabled[true] INFO:setup_ds_test:set SER_INST_SCRIPTS_ENABLED to true
INFO:setup_ds_test:create_instance - Installs the instance and Sets the value of InstScriptsEnabled to true OR false.
INFO:setup_ds_test:Set up the instance and set the config_attr
FAILED

====================================================================================== FAILURES =======================================================================================
_________________________________________________________________________ test_slapd_InstScriptsEnabled[true] _________________________________________________________________________

config_attr = 'true'

    @pytest.mark.parametrize("config_attr", ('true', 'false'))
    def test_slapd_InstScriptsEnabled(config_attr):
        """Tests InstScriptsEnabled attribute with "True" and "False" options

        :id: 02faac7f-c44d-4a3e-bf2d-1021e51da1ed

        :setup: Standalone instance with slapd.InstScriptsEnabled option as "True" and "False"

        :steps:
             1. Execute setup-ds.pl with slapd.InstScriptsEnabled option as "True".
             2. Check if /usr/lib64/dirsrv/slapd-instance instance script directory is created or not.
             3. Execute setup-ds.pl with slapd.InstScriptsEnabled option as "False".
             4. Check if /usr/lib64/dirsrv/slapd-instance instance script directory is created or not.

        :expectedresults:
             1. Instance should be created.
             2. /usr/lib64/dirsrv/slapd-instance instance script directory should be created.
             3. Instance should be created.
             4. /usr/lib64/dirsrv/slapd-instance instance script directory should not be created.
        """

        log.info('set SER_INST_SCRIPTS_ENABLED to {}'.format(config_attr))
>       standalone = create_instance(config_attr)

tests/suites/setup_ds/setup_ds_test.py:62: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/suites/setup_ds/setup_ds_test.py:23: in create_instance
    standalone = DirSrv(verbose=False)
../../../lib389/lib389/__init__.py:400: in __init__
    self.__add_brookers__()
../../../lib389/lib389/__init__.py:318: in __add_brookers__
    self.changelog = Changelog(self)
../../../lib389/lib389/changelog.py:29: in __init__
    self.changelogdir = os.path.join(os.path.dirname(self.conn.dbdir), DEFAULT_CHANGELOG_DB)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <lib389.DirSrv object at 0x7f7ae2a22110>, name = 'dbdir'

    def __getattr__(self,name):
      if self.CLASSATTR_OPTION_MAPPING.has_key(name):
        return self.get_option(self.CLASSATTR_OPTION_MAPPING[name])
      elif self.__dict__.has_key(name):
        return self.__dict__[name]
      else:
        raise AttributeError,'%s has no attribute %s' % (
>         self.__class__.__name__,repr(name)
        )
E       AttributeError: DirSrv has no attribute 'dbdir'

Metadata Update from @spichugi:
- Custom field Review Status adjusted to review (was: ack)

6 years ago

Thanks this patch seems to resolve the issue, ack

Metadata Update from @mreynolds:
- Custom field Review Status adjusted to ack (was: review)

6 years ago

Thanks! Sorry for creating it...

commit bde1530
Author: Simon Pichugin spichugi@redhat.com
Date: Tue Sep 12 07:58:23 2017 +0200

Thanks! Sorry for creating it...

No problem, but this points out that we need to run the DS tests before pushing lib389 changes. Now, if we move lib389 inside of the DS source code then that will also cover the testing once 389-ds-base starts using PRs/jenkins, but we aren't there yet :-)

No problem, but this points out that we need to run the DS tests before pushing lib389 changes. Now, if we move lib389 inside of the DS source code then that will also cover the testing once 389-ds-base starts using PRs/jenkins, but we aren't there yet :-)

Yes, I agree, I have the exact same thoughts. :)
It was a big patch and I forgot about the change... I thought my patch was only about docs that I've executed successfully.

@mreynolds, @firstyear, hi.
Where do you think is the best place to store the documentation?

Unfortunately, readthedocs.ie is not an option. It has issue with C libs that python-ldap depends on.
http://docs.readthedocs.io/en/latest/faq.html#i-get-import-errors-on-libraries-that-depend-on-c-modules
To make it work we need:

If such libraries are installed via setup.py, you also will need to remove all the C-dependent libraries from your install_requires in the RTD environment.

And it's not an option I think.

Manually, the documentation can be generated with the next command:

$ cd src/lib389/doc/ || make html

And then we can upload the directory 'src/lib389/doc/build/html' somewhere on a web server.
Probably, we can use Openshift instance for auto-generation per commit.

What do you think?

ack on the patch

@mreynolds, hi.
Where do you think is the best place to store the documentation?

Eventually the admin guide :-)

It can be generated with the next command from 'src/lib389/doc/':
$ make html
And then we can upload the directory 'src/lib389/doc/build/html' somewhere.
Probably, we can use Openshift instance for it.
Maybe you have something else in your mind?

The wiki would be good for now (after we get the migration done).

Well, I'd love all our docs in an rpm, then we can just "install the rpm" and host on openshift ...

So that would be a good place to get to I think?

I am okay with either of your ideas.
Ok, lets put it first to wiki.

commit 5493b8686de9c88975aed07ec6ca9a85f6cd1b76
Author: Simon Pichugin spichugi@redhat.com
Date: Fri Nov 3 16:43:32 2017 +0100

Metadata Update from @spichugi:
- Assignee reset

4 years ago

Login to comment on this ticket.

Metadata