If this is required, isn't it better to inherit from TestCase instead of object and use simple inheritance in tests?

@tkopecek
The original idea of CliTestLib is to design a helper library. However it have to be manually initialized before use it. That's why I designed it as an independent class rather than inherited class.

@tkopecek
I made some changes to improve the library.
Changes include:
* Rename CliTestLib to CliTestCase.
* CliTestCase now is inherited from TestCase
* initialize() takes no argument (previous was target test function but it is not needed any more)
* simplify assert_system_exit interface and add help

:thumbsup:

Does __assert_lib_inited really need to be strictly private (the double underscore)? Makes it harder if we want to subclass in the future.

Does __assert_callable need to be private at all? For that matter, is it really any better than just using assert callable(callableObj)?

I get the desire to unify some of the common code here, but is there more of a goal here? As it is, this patch adds more lines than it removes, which seems somewhat counter-intuitive.

Does __assert_lib_inited really need to be strictly private (the double underscore)? Makes it harder if we want to subclass in the future.

The __assert_lib_inited is only used in the CliTestCase. in case we forget to call initialize() before using the member methods.

Does __assert_callable need to be private at all? For that matter, is it really any better than just using assert callable(callableObj)?

__assert_callable like __assert_lib_inited, are used for internal check, I don't think they should be
public

Beside, I quickly realized the assert_func used in assert_system_exit needs callable check. Since there are two places need to check, and it might be useful in the future, I would like to keep __assert_callable exist.

I get the desire to unify some of the common code here, but is there more of a goal here? As it is, this patch adds more lines than it removes, which seems somewhat counter-intuitive.

When I was designing for other CLI unit tests, I realized there are many codes I can share in different tests. Therefore I create this patch and modify my previous tests to leverage CliTestCase.

There are several new test cases that are queued in my working repo and they all uses the class.

I also wonder about __error_format as strict private. It seems odd to me to make it private and build the setter method when it so much easier to just use attribute access.

@mikem
After reviewing my commit again, I decided to make some change:
1) It seems the __init and initialize() are not necessary because all the attributes can be initialized during instance creation. So I decide to remove __init and initialize(), it makes the core more concise.
2) __error_format now is a public attribute and the setter is removed.

New commit include:
1) remove debug_print and add print_message in CliTestCase. The print_message() print message on stdout without any influence from mock.
2) Add more unit tests for CLI (current coverage: 31%)
test_add_tag.py
test_disable_host.py
test_enable_host.py
test_import.py
test_import_cg.py
test_import_sig.py
test_list_tasks.py
test_mock_config.py
test_restart_host.py
test_search.py
test_set_pkg_arches.py
test_set_pkg_owner.py
test_set_task_priority.py
test_taskinfo.py
test_unblock_pkg.py
test_write_signed_rpm.py

seems to be throwing an error

 PYTHONPATH=hub/.:plugins/hub/.:plugins/builder/.:plugins/cli/.:cli/.:www/lib nosetests tests/test_cli -x
.........................................................................................................................................................................................F
======================================================================
FAIL: Test anon_handle_search function with argument error
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/temp-work.CGsTR8/koji/tests/test_cli/test_search.py", line 75, in test_anon_handle_search_argument_error
    activate_session=None)
  File "/tmp/temp-work.CGsTR8/koji/tests/test_cli/utils.py", line 181, in assert_system_exit
    callableObj(*test_args, **test_kwargs)
AssertionError: SystemExit not raised

----------------------------------------------------------------------
Ran 186 tests in 1.017s

FAILED (failures=1)

and a few like this for make test3

======================================================================
ERROR: test_printTaskInfo_create_repo (tests.test_cli.test_taskinfo.TestPrintTaskInfo)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "/tmp/temp-work.CGsTR8/koji/tests/test_cli/test_taskinfo.py", line 427, in test_printTaskInfo_create_repo
    _printTaskInfo(session, 1, '/mnt/koji')
  File "/tmp/temp-work.CGsTR8/koji/cli/koji_cli/commands.py", line 4520, in _printTaskInfo
    children.sort(cmp=lambda a, b: cmp(a['id'], b['id']))
TypeError: 'cmp' is an invalid keyword argument for this function

cmp keyword in sorted()/list.sort() has been removed in python3,
I had filed another PR for fixing this issue.
https://pagure.io/koji/pull-request/714

and a few like this for make test3

======================================================================
ERROR: test_printTaskInfo_create_repo (tests.test_cli.test_taskinfo.TestPrintTaskInfo)

---

Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/mock/mock.py", line 1305, in patched
return func(args, *keywargs)
File "/tmp/temp-work.CGsTR8/koji/tests/test_cli/test_taskinfo.py", line 427, in test_printTaskInfo_create_repo
_printTaskInfo(session, 1, '/mnt/koji')
File "/tmp/temp-work.CGsTR8/koji/cli/koji_cli/commands.py", line 4520, in _printTaskInfo
children.sort(cmp=lambda a, b: cmp(a['id'], b['id']))
TypeError: 'cmp' is an invalid keyword argument for this function

seems to be throwing an error
PYTHONPATH=hub/.:plugins/hub/.:plugins/builder/.:plugins/cli/.:cli/.:www/lib nosetests tests/test_cli -x
.........................................................................................................................................................................................F
======================================================================
FAIL: Test anon_handle_search function with argument error

---

Traceback (most recent call last):
File "/tmp/temp-work.CGsTR8/koji/tests/test_cli/test_search.py", line 75, in test_anon_handle_search_argument_error
activate_session=None)
File "/tmp/temp-work.CGsTR8/koji/tests/test_cli/utils.py", line 181, in assert_system_exit
callableObj(test_args, *test_kwargs)
AssertionError: SystemExit not raised

---

Ran 186 tests in 1.017s

FAILED (failures=1)

In order to fix https://pagure.io/koji/issue/706,
The PR#707 has been filed, and this unit test is required fixed version.

@mikem
When I was designing the unit tests, I figured out and filed some issue and PR separately:
https://pagure.io/koji/issue/706 PR:#707
https://pagure.io/koji/issue/713 PR:#714

In order to run unit tests successfully, these PRs needs to be applied.

Commit #8cae09a
Add unit test for koji image-build command

Commit b260aab fixes this pull-request