PR#49699: Ticket 49683 - Add support for JSON option in lib389 CLI tools - 389-ds-base

389-ds-base

#49699 Ticket 49683 - Add support for JSON option in lib389 CLI tools

Closed 3 years ago by spichugi. Opened 5 years ago by mreynolds.

mreynolds/389-ds-base ticket49683 into master

Ticket 49683 - Add support for JSON option in lib389 CLI tools

Mark Reynolds • 5 years ago

593a73e

src/cockpit/389-console/README.md

file modified

+21 -15

		`@@ -37,7 +37,7 @@`

		`## Security`

		`- - Enable Security and configureation settings`
		`+ - Enable Security and configuration settings`
		`- Manage Certificate Database`
		`- Manage ciphers`

		`@@ -139,10 +139,27 @@`
		`- Drop in 389's cockpit plugin bundle under /usr/share/cockpit/ --> /usr/share/cockpit/389-console/`
		`- Done`

		`- ## setup-ds-cockpit`

		`- -Install script, see "man setup-ds-cockpit"`
		`+ # lib389 json representations`
		`+ --------------------------------`

		`+ Entry:`
		`+`
		`+ {`
		`+ "type": "entry",`
		`+ "dn": DN,`
		`+ "attrs": {`
		`+ ATTR: [val],`
		`+ ATTR: [val, val, ...],`
		`+ }`
		`+ }`
		`+`
		`+ List:`
		`+`
		`+ {`
		`+ "type": "list",`
		`+ "items": []`
		`+ }`

		`# Misc`
		`---------------------`
		`@@ -182,21 +199,10 @@`
		`- Edit plugin ...`
		`- Add/edit SASL Mapiing`
		`- Import/Export Certification (file location)`
		`+ - All the cert stuff`

		`- - Add/edit schema (attrs & objectclasses)`
		`- - Add/edit local password policy`
		`-`
		`-`
		`- ## Monitoring page`
		`-`
		`- - Nothing has been done yet`
		`- - Some monitoring features need RFE that have yet to be coded`

		`- ## Nice to have/fix`

		`- - Get "Cipher" Datatable to sort on checkbox (other wise we need a "yes/no" 'Enabled" column and a button (in its own column) to toggle it.`
		`- - Figure out why the ds-flex page changes width between sever configuraton buttons (Server Configuration vs SASL)`
		`- - Remove panel tab highlighting after slection`

src/cockpit/389-console/js/ds.js

file modified

+13

		`@@ -35,6 +35,19 @@`
		`);`
		`}`

		`+ // POC - REMOVE!!!!!`
		`+ function test_json_and_dsconf () {`
		`+ var cmd = ['/home/mareynol/source/ds389/389-ds-base/src/lib389/cli/dsconf',`
		`+ '-j', 'ldapi://%2fvar%2frun%2fslapd-localhost.socket','backend',`
		`+ 'list']`
		`+ cockpit.spawn(cmd, { superuser: true, "err": "message", "environ": ["PYTHONPATH=/home/mareynol/source/ds389/389-ds-base/src/lib389"]}).done(function(data) {`
		`+ var obj = JSON.parse(data);`
		`+ console.log("backend: " + obj['items']);`
		`+ }).fail(function(data) {`
		`+ console.log("failed: " + data.message);`
		`+ });`
		`+ }`
		`+`
		`function sort_list (sel) {`
		`var opts_list = sel.find('option');`
		`opts_list.sort(function(a, b) { return $(a).text() > $(b).text() ? 1 : -1; });`

src/cockpit/389-console/js/servers.js

file modified

-3

		`@@ -169,9 +169,6 @@`
		`check_for_389();`
		`$("#server-tab").css( 'color', '#228bc0');`

		`-`
		`-`
		`-`
		`$("#server-content").load("servers.html", function () {`
		`// Initial page setup`
		`$('.disk-monitoring').hide();`

src/lib389/cli/dsconf

file modified

+15 -6

		`@@ -54,7 +54,7 @@`
		`)`
		`parser.add_argument('-D', '--binddn',`
		`help="The account to bind as for executing operations",`
		`- default=None,`
		`+ default=None`
		`)`
		`parser.add_argument('-b', '--basedn',`
		`help="Basedn (root naming context) of the instance to manage",`
		`@@ -64,6 +64,10 @@`
		`help="Connect with StartTLS",`
		`default=False, action='store_true'`
		`)`
		`+ parser.add_argument('-j', '--json',`
		`+ help="Return result in JSON object",`
		`+ default=False, action='store_true'`
		`+ )`

		`subparsers = parser.add_subparsers(help="resources to act upon")`

		`@@ -110,15 +114,20 @@`
		`if args.verbose:`
		`inst = connect_instance(dsrc_inst=dsrc_inst, verbose=args.verbose)`
		`args.func(inst, None, log, args)`
		`- log.info("Command successful.")`
		`+ if not args.json:`
		`+ log.info("Command successful.")`
		`else:`
		`try:`
		`inst = connect_instance(dsrc_inst=dsrc_inst, verbose=args.verbose)`
		`args.func(inst, None, log, args)`
		`- log.info("Command successful.")`
		`- except Exception as e:`
		`- log.debug(e, exc_info=True)`
		`- log.error("Error: %s" % e)`
		`+ if not args.json:`
		`+ log.info("Command successful.")`
		`+ except ldap.LDAPError as e:`
		`+ #log.debug(e, exc_info=True)`
		`+ if args and args.json:`
		`+ print(e)`
		`+ else:`
		`+ log.error("Error!: %s" % str(e))`
		`disconnect_instance(inst)`

		`# Done!`

src/lib389/cli/dscreate

file modified

+1 -1

		`@@ -22,7 +22,7 @@`

		`parser.add_argument('-v', '--verbose',`
		`help="Display verbose operation tracing during command execution",`
		`- action='store_true', default=False`
		`+ action='store_true', default=False, dest='verbose'`
		`)`

		`subparsers = parser.add_subparsers(help="action")`

src/lib389/cli/dsctl

file modified

		`@@ -32,6 +32,10 @@`
		`parser.add_argument('instance',`
		`help="The name of the instance to act upon",`
		`)`
		`+ parser.add_argument('-j', '--json',`
		`+ help="Return result in JSON object",`
		`+ default=False, action='store_true'`
		`+ )`

		`subparsers = parser.add_subparsers(help="action")`

src/lib389/cli/dsidm

file modified

+5 -1

		`@@ -49,12 +49,16 @@`
		`)`
		`parser.add_argument('-D', '--binddn',`
		`help="The account to bind as for executing operations",`
		`- default=None,`
		`+ default=None`
		`)`
		`parser.add_argument('-Z', '--starttls',`
		`help="Connect with StartTLS",`
		`default=False, action='store_true'`
		`)`
		`+ parser.add_argument('-j', '--json',`
		`+ help="Return result in JSON object",`
		`+ default=False, action='store_true'`
		`+ )`

		`subparsers = parser.add_subparsers(help="resources to act upon")`

src/lib389/lib389/_mapped_object.py

file modified

+7 -5

		`@@ -10,6 +10,7 @@`
		`import ldap.dn`
		`from ldap import filter as ldap_filter`
		`import logging`
		`+ import json`
		`from functools import partial`

		`from lib389._entry import Entry`
		`@@ -167,9 +168,7 @@`
		`for k in attrs:`
		`str_attrs[ensure_str(k)] = ensure_list_str(attrs[k])`

		`- response = { "dn": ensure_str(self._dn), "attrs" : str_attrs }`
		`- print('json response')`
		`- print(response)`
		`+ response = json.dumps({ "type": "entry", "dn": ensure_str(self._dn), "attrs" : str_attrs });`

		`return response`

		`@@ -860,7 +859,7 @@`
		`insts = []`
		`return insts`

		`- def get(self, selector=[], dn=None):`
		`+ def get(self, selector=[], dn=None, json=False):`
		`"""Get a child entry (DSLdapObject, Replica, etc.) with dn or selector`
		`using a base DN and objectClasses of our object (DSLdapObjects, Replicas, etc.)`

		`@@ -884,7 +883,10 @@`
		`raise ldap.NO_SUCH_OBJECT("No object exists given the filter criteria %s" % selector)`
		`if len(results) > 1:`
		`raise ldap.UNWILLING_TO_PERFORM("Too many objects matched selection criteria %s" % selector)`
		`- return self._entry_to_instance(results[0].dn, results[0])`
		`+ if json:`
		`+ return self._entry_to_instance(results[0].dn, results[0]).get_all_attrs_json()`
		`+ else:`
		`+ return self._entry_to_instance(results[0].dn, results[0])`

		`def _get_dn(self, dn):`
		`# This will yield and & filter for objectClass with as many terms as needed.`

src/lib389/lib389/cli_base/__init__.py

file modified

+32 -19

		`@@ -8,11 +8,12 @@`

		`import logging`
		`import sys`
		`+ import json`

		`from getpass import getpass`
		`from lib389 import DirSrv`
		`from lib389.utils import assert_c`
		`- from lib389.properties import SER_LDAP_URL, SER_ROOT_DN, SER_ROOT_PW`
		`+ from lib389.properties import *`

		`MAJOR, MINOR, _, _, _ = sys.version_info`

		`@@ -21,7 +22,7 @@`
		`if MAJOR >= 3:`
		`return input(msg)`
		`else:`
		`- return raw_input(msg)`
		`+ return input(msg)`


		`def _get_arg(args, msg=None, hidden=False, confirm=False):`
		`@@ -82,15 +83,12 @@`

		`# We'll need another of these that does a "connect via instance name?"`
		`def connect_instance(dsrc_inst, verbose):`
		`- dsargs = {`
		`- SER_LDAP_URL: dsrc_inst['uri'],`
		`- SER_ROOT_DN: dsrc_inst['binddn'],`
		`- }`
		`+ dsargs = dsrc_inst['args']`
		`ds = DirSrv(verbose=verbose)`
		`ds.allocate(dsargs)`
		`if not ds.can_autobind() and dsrc_inst['binddn'] is not None:`
		`dsargs[SER_ROOT_PW] = getpass("Enter password for %s on %s : " % (dsrc_inst['binddn'], dsrc_inst['uri']))`
		`- elif dsrc_inst['binddn'] is None:`
		`+ elif not ds.can_autobind() and dsrc_inst['binddn'] is None:`
		`raise Exception("Must provide a binddn to connect with")`
		`ds.allocate(dsargs)`
		`ds.open(saslmethod=dsrc_inst['saslmech'],`
		`@@ -109,41 +107,56 @@`
		`for attr in attributes:`
		`parser.add_argument('--%s' % attr, nargs='?', help="Value of %s" % attr)`

		`- def _generic_list(inst, basedn, log, manager_class, **kwargs):`
		`+ def _generic_list(inst, basedn, log, manager_class, args=None):`
		`mc = manager_class(inst, basedn)`
		`ol = mc.list()`
		`if len(ol) == 0:`
		`- log.info("No objects to display")`
		`+ if args and args.json:`
		`+ print(json.dumps({"type": "list", "items": []}))`
		`+ else:`
		`+ log.info("No objects to display")`
		`elif len(ol) > 0:`
		`# We might sort this in the future`
		`+ if args and args.json:`
		`+ json_result = {"type": "list", "items": []}`
		`for o in ol:`
		`o_str = o.__unicode__()`
		`- log.info(o_str)`
		`+ if args and args.json:`
		`+ json_result['items'].append(o_str)`
		`+ else:`
		`+ log.info(o_str)`
		`+ if args and args.json:`
		`+ print(json.dumps(json_result))`

		`# Display these entries better!`
		`- def _generic_get(inst, basedn, log, manager_class, selector):`
		`+ def _generic_get(inst, basedn, log, manager_class, selector, args=None):`
		`mc = manager_class(inst, basedn)`
		`- o = mc.get(selector)`
		`- o_str = o.display()`
		`- log.info(o_str)`
		`+ if args and args.json:`
		`+ o = mc.get(selector, json=True)`
		`+ print(o);`
		`+ else:`
		`+ o = mc.get(selector)`
		`+ o_str = o.display()`
		`+ log.info(o_str)`

		`- def _generic_get_dn(inst, basedn, log, manager_class, dn):`
		`+ def _generic_get_dn(inst, basedn, log, manager_class, dn, args=None):`
		`mc = manager_class(inst, basedn)`
		`o = mc.get(dn=dn)`
		`o_str = o.display()`
		`log.info(o_str)`

		`- def _generic_create(inst, basedn, log, manager_class, kwargs):`
		`+ def _generic_create(inst, basedn, log, manager_class, kwargs, args=None):`
		`mc = manager_class(inst, basedn)`
		`o = mc.create(properties=kwargs)`
		`o_str = o.__unicode__()`
		`- log.info('Sucessfully created %s' % o_str)`

		`- def _generic_delete(inst, basedn, log, object_class, dn):`
		`+ log.info('Successfully created %s' % o_str)`
		`+`
		`+ def _generic_delete(inst, basedn, log, object_class, dn, args=None):`
		`# Load the oc direct`
		`o = object_class(inst, dn)`
		`o.delete()`
		`- log.info('Sucessfully deleted %s' % dn)`
		`+ log.info('Successfully deleted %s' % dn)`


		`class LogCapture(logging.Handler):`

src/lib389/lib389/cli_base/dsrc.py

file modified

+21

		`@@ -9,6 +9,7 @@`
		`import sys`
		`import os`
		`import ldap`
		`+ from lib389.properties import *`

		`MAJOR, MINOR, _, _, _ = sys.version_info`

		`@@ -42,7 +43,16 @@`
		`'tls_key': None,`
		`'tls_reqcert': ldap.OPT_X_TLS_HARD,`
		`'starttls': args.starttls,`
		`+ 'args': {}`
		`}`
		`+ # Now gather the args`
		`+ new_dsrc_inst['args'][SER_LDAP_URL] = new_dsrc_inst['uri']`
		`+ new_dsrc_inst['args'][SER_ROOT_DN] = new_dsrc_inst['binddn']`
		`+ if new_dsrc_inst['uri'][0:8] == 'ldapi://':`
		`+ new_dsrc_inst['args'][SER_LDAPI_ENABLED] = "on"`
		`+ new_dsrc_inst['args'][SER_LDAPI_SOCKET] = new_dsrc_inst['uri'][9:]`
		`+ new_dsrc_inst['args'][SER_LDAPI_AUTOBIND] = "on"`
		`+`
		`# Make new`
		`return new_dsrc_inst`
		`# overlay things.`
		`@@ -89,6 +99,8 @@`
		`return None`

		`dsrc_inst = {}`
		`+ dsrc_inst['args'] = {}`
		`+`
		`# Read all the values`
		`dsrc_inst['uri'] = config.get(instance_name, 'uri')`
		`dsrc_inst['basedn'] = config.get(instance_name, 'basedn', fallback=None)`
		`@@ -111,6 +123,15 @@`
		`dsrc_inst['tls_reqcert'] = ldap.OPT_X_TLS_HARD`
		`dsrc_inst['starttls'] = config.getboolean(instance_name, 'starttls', fallback=False)`

		`+ # Now gather the args`
		`+ dsrc_inst['args'][SER_LDAP_URL] = dsrc_inst['uri']`
		`+ dsrc_inst['args'][SER_ROOT_DN] = dsrc_inst['binddn']`
		`+ if dsrc_inst['uri'][0:8] == 'ldapi://':`
		`+ dsrc_inst['args'][SER_LDAPI_ENABLED] = "on"`
		`+ dsrc_inst['args'][SER_LDAPI_SOCKET] = dsrc_inst['uri'][9:]`
		`+ dsrc_inst['args'][SER_LDAPI_AUTOBIND] = "on"`
		`+`
		`+`
		`# Return the dict.`
		`log.debug("dsrc completed with %s" % dsrc_inst)`
		`return dsrc_inst`

src/lib389/lib389/cli_conf/backend.py

file modified

+6 -5

		`@@ -27,25 +27,26 @@`
		`RDN = 'cn'`

		`def backend_list(inst, basedn, log, args):`
		`- _generic_list(inst, basedn, log.getChild('backend_list'), MANY)`
		`+ _generic_list(inst, basedn, log.getChild('backend_list'), MANY, args)`

		`def backend_get(inst, basedn, log, args):`
		`rdn = _get_arg( args.selector, msg="Enter %s to retrieve" % RDN)`
		`- _generic_get(inst, basedn, log.getChild('backend_get'), MANY, rdn)`
		`+ _generic_get(inst, basedn, log.getChild('backend_get'), MANY, rdn, args)`

		`def backend_get_dn(inst, basedn, log, args):`
		`dn = _get_arg( args.dn, msg="Enter dn to retrieve")`
		`- _generic_get_dn(inst, basedn, log.getChild('backend_get_dn'), MANY, dn)`
		`+ _generic_get_dn(inst, basedn, log.getChild('backend_get_dn'), MANY, dn, args)`

		`def backend_create(inst, basedn, log, args):`
		`+ args.json = True`
		`kwargs = _get_attributes(args, Backend._must_attributes)`
		`- _generic_create(inst, basedn, log.getChild('backend_create'), MANY, kwargs)`
		`+ _generic_create(inst, basedn, log.getChild('backend_create'), MANY, kwargs, args)`

		`def backend_delete(inst, basedn, log, args, warn=True):`
		`dn = _get_arg( args.dn, msg="Enter dn to delete")`
		`if warn:`
		`_warn(dn, msg="Deleting %s %s" % (SINGULAR.__name__, dn))`
		`- _generic_delete(inst, basedn, log.getChild('backend_delete'), SINGULAR, dn)`
		`+ _generic_delete(inst, basedn, log.getChild('backend_delete'), SINGULAR, dn, args)`


		`def create_parser(subparsers):`

src/lib389/lib389/cli_conf/plugin.py

file modified

+3 -3

		`@@ -27,15 +27,15 @@`


		`def plugin_list(inst, basedn, log, args):`
		`- _generic_list(inst, basedn, log.getChild('plugin_list'), MANY)`
		`+ _generic_list(inst, basedn, log.getChild('plugin_list'), MANY, args)`

		`def plugin_get(inst, basedn, log, args):`
		`rdn = _get_arg( args.selector, msg="Enter %s to retrieve" % RDN)`
		`- _generic_get(inst, basedn, log.getChild('plugin_get'), MANY, rdn)`
		`+ _generic_get(inst, basedn, log.getChild('plugin_get'), MANY, rdn, args)`

		`def plugin_get_dn(inst, basedn, log, args):`
		`dn = _get_arg( args.dn, msg="Enter dn to retrieve")`
		`- _generic_get_dn(inst, basedn, log.getChild('plugin_get_dn'), MANY, dn)`
		`+ _generic_get_dn(inst, basedn, log.getChild('plugin_get_dn'), MANY, dn, args)`

		`# Plugin enable`
		`def plugin_enable(inst, basedn, log, args):`

src/lib389/lib389/cli_conf/plugins/automember.py

file modified

+24 -13

		`@@ -7,7 +7,7 @@`
		`# --- END COPYRIGHT BLOCK ---`

		`import ldap`
		`-`
		`+ import json`
		`from lib389.plugins import AutoMembershipPlugin, AutoMembershipDefinition, AutoMembershipDefinitions`
		`from lib389.cli_conf.plugin import add_generic_plugin_parsers`

		`@@ -25,14 +25,23 @@`
		`automembers = AutoMembershipDefinitions(inst)`

		`if args.name is not None:`
		`- automember = automembers.get(args.name)`
		`- log.info(automember.display())`
		`+ if args.json:`
		`+ print(automembers.get_all_attrs_json(args.name))`
		`+ else:`
		`+ automember = automembers.get(args.name)`
		`+ log.info(automember.display())`
		`else:`
		`all_definitions = automembers.list()`
		`-`
		`+ if args.json:`
		`+ result = {'type': 'list', 'items': []}`
		`for definition in all_definitions:`
		`- log.info(definition.display())`
		`+ if args.json:`
		`+ result['items'].append(definition)`
		`+ else:`
		`+ log.info(definition.display())`

		`+ if args.json:`
		`+ print(json.dumps(result))`

		`def create_definition(inst, basedn, log, args):`
		`"""`
		`@@ -63,9 +72,12 @@`

		`automembers = AutoMembershipDefinitions(inst)`

		`- automember = automembers.create(properties=automember_prop)`
		`- log.info("Automember definition created successfully!")`
		`-`
		`+ try:`
		`+ automember = automembers.create(properties=automember_prop)`
		`+ log.info("Automember definition created successfully!")`
		`+ except Exception as e:`
		`+ log.info("Failed to create Automember definition: {}".format(str(e)))`
		`+ raise e`

		`def edit_definition(inst, basedn, log, args):`
		`"""`
		`@@ -94,7 +106,6 @@`
		`automember.replace("automemberdefaultgroup", args.defaultgroup)`
		`if args.groupattr is not None:`
		`automember.replace("automembergroupingattr", args.groupattr)`
		`-`
		`log.info("Definition updated successfully.")`


		`@@ -111,8 +122,8 @@`
		`automember = automembers.get(args.name)`

		`automember.delete()`
		`- log.info("Definition deleted successfully!")`
		`-`
		`+ log.info("Definition deleted successfully.")`
		`+`

		`def create_parser(subparsers):`
		`automember_parser = subparsers.add_parser('automember', help="Manage and configure automember plugin")`
		`@@ -124,7 +135,7 @@`
		`create_parser = subcommands.add_parser('create', help='Create automember definition.')`
		`create_parser.set_defaults(func=create_definition)`

		`- create_parser.add_argument("name", nargs='?', required=True ,help='Set cn for group entry.')`
		`+ create_parser.add_argument("name", help='Set cn for group entry.')`

		`create_parser.add_argument("--groupattr", help='Set member attribute in group entry.', default='member:dn')`

		`@@ -137,7 +148,7 @@`
		`show_parser = subcommands.add_parser('list', help='List automember definition.')`
		`show_parser.set_defaults(func=list_definition)`

		`- show_parser.add_argument("name", nargs='?', help='Set cn for group entry.')`
		`+ show_parser.add_argument("name", help='Set cn for group entry.')`

		`edit_parser = subcommands.add_parser('edit', help='Edit automember definition.')`
		`edit_parser.set_defaults(func=edit_definition)`

src/lib389/lib389/cli_conf/schema.py

file modified

+69 -12

		`@@ -11,22 +11,64 @@`


		`def list_attributetype(inst, basedn, log, args):`
		`- for attributetype in inst.schema.get_attributetypes():`
		`- print(attributetype)`
		`+ if args is not None and args.json:`
		`+ print(inst.schema.get_attributetypes(json=True))`
		`+ else:`
		`+ for attributetype in inst.schema.get_attributetypes():`
		`+ print(attributetype)`
		`+`
		`+`
		`+ def list_objectclasses(inst, basedn, log, args):`
		`+ if args is not None and args.json:`
		`+ print(inst.schema.get_objectclasses(json=True))`
		`+ else:`
		`+ for oc in inst.schema.get_objectclasses():`
		`+ print(oc)`
		`+`
		`+`
		`+ def list_matchingrules(inst, basedn, log, args):`
		`+ if args is not None and args.json:`
		`+ print(inst.schema.get_matchingrules(json=True))`
		`+ else:`
		`+ for mr in inst.schema.matchingrules():`
		`+ print(mr)`
		`+`

		`def query_attributetype(inst, basedn, log, args):`
		`# Need the query type`
		`attr = _get_arg(args.attr, msg="Enter attribute to query")`
		`- attributetype, must, may = inst.schema.query_attributetype(attr)`
		`- print(attributetype)`
		`- print("")`
		`- print("MUST")`
		`- for oc in must:`
		`- print(oc)`
		`- print("")`
		`- print("MAY")`
		`- for oc in may:`
		`- print(oc)`
		`+ if args.json:`
		`+ print(inst.schema.query_attributetype(attr, json=True))`
		`+ else:`
		`+ attributetype, must, may = inst.schema.query_attributetype(attr)`
		`+ print(attributetype)`
		`+ print("")`
		`+ print("MUST")`
		`+ for oc in must:`
		`+ print(oc)`
		`+ print("")`
		`+ print("MAY")`
		`+ for oc in may:`
		`+ print(oc)`
		`+`
		`+`
		`+ def query_objectclass(inst, basedn, log, args):`
		`+ # Need the query type`
		`+ oc = _get_arg(args.attr, msg="Enter objectclass to query")`
		`+ if args.json:`
		`+ print(inst.schema.query_objectclass(oc, json=True))`
		`+ else:`
		`+ print("Not done")`
		`+`
		`+`
		`+ def query_matchingrule(inst, basedn, log, args):`
		`+ # Need the query type`
		`+ attr = _get_arg(args.attr, msg="Enter attribute to query")`
		`+ if args.json:`
		`+ print(inst.schema.query_matchingrule(attr, json=True))`
		`+ else:`
		`+ print("Not done")`
		`+`

		`def reload_schema(inst, basedn, log, args):`
		`schema = Schema(inst)`
		`@@ -48,6 +90,21 @@`
		`query_attributetype_parser.set_defaults(func=query_attributetype)`
		`query_attributetype_parser.add_argument('attr', nargs='?', help='Attribute type to query')`

		`+ list_objectclass_parser = subcommands.add_parser('list_objectclasses', help='List avaliable objectclasses on this system')`
		`+ list_objectclass_parser.set_defaults(func=list_objectclasses)`
		`+`
		`+ query_objectclass_parser = subcommands.add_parser('query_objectclass', help='Query an objectclass')`
		`+ query_objectclass_parser.set_defaults(func=query_objectclass)`
		`+ query_objectclass_parser.add_argument('attr', nargs='?', help='Objectclass to query')`
		`+`
		`reload_parser = subcommands.add_parser('reload', help='Dynamically reload schema while server is running')`
		`reload_parser.set_defaults(func=reload_schema)`
		`reload_parser.add_argument('-d', '--schemadir', help="directory where schema files are located")`
		`+`
		`+ list_matchingrules_parser = subcommands.add_parser('list_matchingrules', help='List avaliable matching rules on this system')`
		`+ list_matchingrules_parser.set_defaults(func=list_matchingrules)`
		`+`
		`+ query_matchingrule_parser = subcommands.add_parser('query_matchingrule', help='Query a matchingrule')`
		`+ query_matchingrule_parser.set_defaults(func=query_matchingrule)`
		`+ query_matchingrule_parser.add_argument('attr', nargs='?', help='Matchingrule to query')`
		`+`

src/lib389/lib389/cli_idm/__init__.py

file modified

+45 -40

		`@@ -8,7 +8,8 @@`

		`from getpass import getpass`
		`from lib389 import DirSrv`
		`- from lib389.properties import SER_LDAP_URL, SER_ROOT_DN, SER_ROOT_PW`
		`+ from lib389.properties import *`
		`+ import json`


		`def _get_arg(args, msg=None):`
		`@@ -18,7 +19,8 @@`
		`else:`
		`return args`
		`else:`
		`- return raw_input("%s : " % msg)`
		`+ return input("%s : " % msg)`
		`+`

		`def _get_args(args, kws):`
		`kwargs = {}`
		`@@ -31,84 +33,87 @@`
		`if priv:`
		`kwargs[kw] = getpass("%s : " % msg)`
		`else:`
		`- kwargs[kw] = raw_input("%s : " % msg)`
		`+ kwargs[kw] = input("%s : " % msg)`
		`return kwargs`

		`- # This is really similar to get_args, but generates from the MUST_ATTRIBUTES array`
		`+`
		`+ # This is really similar to get_args, but generates from an array`
		`def _get_attributes(args, attrs):`
		`kwargs = {}`
		`for attr in attrs:`
		`- if args is not None and len(args) > 0:`
		`- kwargs[attr] = args.pop(0)`
		`+ # Python can't represent a -, so it replaces it to _`
		`+ # in many places, so we have to normalise this.`
		`+ attr_normal = attr.replace('-', '_')`
		`+ if args is not None and hasattr(args, attr_normal) and getattr(args, attr_normal) is not None:`
		`+ kwargs[attr] = getattr(args, attr_normal)`
		`else:`
		`if attr.lower() == 'userpassword':`
		`kwargs[attr] = getpass("Enter value for %s : " % attr)`
		`else:`
		`- kwargs[attr] = raw_input("Enter value for %s : " % attr)`
		`+ kwargs[attr] = input("Enter value for %s : " % attr)`
		`return kwargs`


		`def _warn(data, msg=None):`
		`if msg is not None:`
		`print("%s :" % msg)`
		`- if 'Yes I am sure' != raw_input("Type 'Yes I am sure' to continue: "):`
		`+ if 'Yes I am sure' != input("Type 'Yes I am sure' to continue: "):`
		`raise Exception("Not sure if want")`
		`return data`

		`- def connect_instance(ldapurl, binddn, verbose, starttls):`
		`- dsargs = {`
		`- SER_LDAP_URL: ldapurl,`
		`- SER_ROOT_DN: binddn`
		`- }`
		`- ds = DirSrv(verbose=verbose)`
		`- ds.allocate(dsargs)`
		`- if not ds.can_autobind() and binddn is not None:`
		`- dsargs[SER_ROOT_PW] = getpass("Enter password for %s on %s : " % (binddn, ldapurl))`
		`- elif binddn is None:`
		`- raise Exception("Must provide a binddn to connect with")`
		`- ds.allocate(dsargs)`
		`- ds.open(starttls=starttls)`
		`- print("")`
		`- return ds`
		`-`
		`- def disconnect_instance(inst):`
		`- if inst is not None:`
		`- inst.close()`
		`-`
		`- def _generic_list(inst, basedn, log, manager_class, **kwargs):`
		`+`
		`+ def _generic_list(inst, basedn, log, manager_class, args=None):`
		`mc = manager_class(inst, basedn)`
		`ol = mc.list()`
		`if len(ol) == 0:`
		`- print("No objects to display")`
		`+ if args and args.json:`
		`+ print(json.dumps({"type": "list", "items": []}))`
		`+ else:`
		`+ log.info("No objects to display")`
		`elif len(ol) > 0:`
		`# We might sort this in the future`
		`+ if args and args.json:`
		`+ json_result = {"type": "list", "items": []}`
		`for o in ol:`
		`o_str = o.__unicode__()`
		`- print(o_str)`
		`+ if args and args.json:`
		`+ json_result['items'].append(o_str)`
		`+ else:`
		`+ log.info(o_str)`
		`+ if args and args.json:`
		`+ print(json.dumps(json_result))`
		`+`

		`# Display these entries better!`
		`- def _generic_get(inst, basedn, log, manager_class, selector):`
		`+ def _generic_get(inst, basedn, log, manager_class, selector, args=None):`
		`mc = manager_class(inst, basedn)`
		`- o = mc.get(selector)`
		`- o_str = o.__unicode__()`
		`- print(o_str)`
		`+ if args and args.json:`
		`+ o = mc.get(selector, json=True)`
		`+ print(o);`
		`+ else:`
		`+ o = mc.get(selector)`
		`+ o_str = o.display()`
		`+ log.info(o_str)`

		`- def _generic_get_dn(inst, basedn, log, manager_class, dn):`
		`+`
		`+ def _generic_get_dn(inst, basedn, log, manager_class, dn, args=None):`
		`mc = manager_class(inst, basedn)`
		`o = mc.get(dn=dn)`
		`o_str = o.__unicode__()`
		`print(o_str)`

		`- def _generic_create(inst, basedn, log, manager_class, kwargs):`
		`+`
		`+ def _generic_create(inst, basedn, log, manager_class, kwargs, args=None):`
		`mc = manager_class(inst, basedn)`
		`o = mc.create(properties=kwargs)`
		`o_str = o.__unicode__()`
		`- print('Sucessfully created %s' % o_str)`
		`+ log.info('Sucessfully created %s' % o_str)`
		`+`

		`- def _generic_delete(inst, basedn, log, object_class, dn):`
		`+ def _generic_delete(inst, basedn, log, object_class, dn, args=None):`
		`# Load the oc direct`
		`o = object_class(inst, dn)`
		`o.delete()`
		`- print('Sucessfully deleted %s' % dn)`
		`+ log.info('Sucessfully deleted %s' % dn)`

		`# vim: tabstop=4 expandtab shiftwidth=4 softtabstop=4`

src/lib389/lib389/cli_idm/account.py

file modified

+2 -2

		`@@ -9,7 +9,7 @@`
		`import argparse`

		`from lib389.idm.account import Account, Accounts`
		`- from lib389.cli_base import (`
		`+ from lib389.cli_idm import (`
		`_generic_list,`
		`_get_arg,`
		`)`
		`@@ -17,7 +17,7 @@`
		`MANY = Accounts`

		`def list(inst, basedn, log, args):`
		`- _generic_list(inst, basedn, log.getChild('_generic_list'), MANY)`
		`+ _generic_list(inst, basedn, log.getChild('_generic_list'), MANY, args)`

		`def status(inst, basedn, log, args):`
		`dn = _get_arg( args.dn, msg="Enter dn to check")`

src/lib389/lib389/cli_idm/group.py

file modified

+7 -8

		`@@ -8,9 +8,8 @@`

		`import argparse`
		`from lib389.idm.group import Group, Groups, MUST_ATTRIBUTES`
		`-`
		`- from lib389.cli_base import (`
		`- populate_attr_arguments,`
		`+ from lib389.cli_base import populate_attr_arguments`
		`+ from lib389.cli_idm import (`
		`_generic_list,`
		`_generic_get,`
		`_generic_get_dn,`
		`@@ -29,25 +28,25 @@`
		`# These are a generic specification, try not to tamper with them`

		`def list(inst, basedn, log, args):`
		`- _generic_list(inst, basedn, log.getChild('_generic_list'), MANY)`
		`+ _generic_list(inst, basedn, log.getChild('_generic_list'), MANY, args)`

		`def get(inst, basedn, log, args):`
		`rdn = _get_arg( args.selector, msg="Enter %s to retrieve" % RDN)`
		`- _generic_get(inst, basedn, log.getChild('_generic_get'), MANY, rdn)`
		`+ _generic_get(inst, basedn, log.getChild('_generic_get'), MANY, rdn, args)`

		`def get_dn(inst, basedn, log, args):`
		`dn = _get_arg( args.dn, msg="Enter dn to retrieve")`
		`- _generic_get_dn(inst, basedn, log.getChild('_generic_get_dn'), MANY, dn)`
		`+ _generic_get_dn(inst, basedn, log.getChild('_generic_get_dn'), MANY, dn, args)`

		`def create(inst, basedn, log, args):`
		`kwargs = _get_attributes(args, MUST_ATTRIBUTES)`
		`- _generic_create(inst, basedn, log.getChild('_generic_create'), MANY, kwargs)`
		`+ _generic_create(inst, basedn, log.getChild('_generic_create'), MANY, kwargs, args)`

		`def delete(inst, basedn, log, args, warn=True):`
		`dn = _get_arg( args.dn , msg="Enter dn to delete")`
		`if warn:`
		`_warn(dn, msg="Deleting %s %s" % (SINGULAR.__name__, dn))`
		`- _generic_delete(inst, basedn, log.getChild('_generic_delete'), SINGULAR, dn)`
		`+ _generic_delete(inst, basedn, log.getChild('_generic_delete'), SINGULAR, dn, args)`

		`def members(inst, basedn, log, args):`
		`cn = _get_arg( args.cn, msg="Enter %s of group" % RDN)`

src/lib389/lib389/cli_idm/organizationalunit.py

file modified

+7 -8

		`@@ -8,9 +8,8 @@`

		`import argparse`
		`from lib389.idm.organizationalunit import OrganizationalUnit, OrganizationalUnits, MUST_ATTRIBUTES`
		`-`
		`- from lib389.cli_base import (`
		`- populate_attr_arguments,`
		`+ from lib389.cli_base import populate_attr_arguments`
		`+ from lib389.cli_idm import (`
		`_generic_list,`
		`_generic_get,`
		`_generic_get_dn,`
		`@@ -29,24 +28,24 @@`
		`# These are a generic specification, try not to tamper with them`

		`def list(inst, basedn, log, args):`
		`- _generic_list(inst, basedn, log.getChild('_generic_list'), MANY)`
		`+ _generic_list(inst, basedn, log.getChild('_generic_list'), MANY, args)`

		`def get(inst, basedn, log, args):`
		`rdn = _get_arg( args.selector, msg="Enter %s to retrieve" % RDN)`
		`- _generic_get(inst, basedn, log.getChild('_generic_get'), MANY, rdn)`
		`+ _generic_get(inst, basedn, log.getChild('_generic_get'), MANY, rdn, args)`

		`def get_dn(inst, basedn, log, args):`
		`dn = lambda args: _get_arg( args.dn, msg="Enter dn to retrieve")`
		`- _generic_get_dn(inst, basedn, log.getChild('_generic_get_dn'), MANY, dn)`
		`+ _generic_get_dn(inst, basedn, log.getChild('_generic_get_dn'), MANY, dn, args)`

		`def create(inst, basedn, log, args):`
		`kwargs = _get_attributes(args, MUST_ATTRIBUTES)`
		`- _generic_create(inst, basedn, log.getChild('_generic_create'), MANY, kwargs)`
		`+ _generic_create(inst, basedn, log.getChild('_generic_create'), MANY, kwargs, args)`

		`def delete(inst, basedn, log, args):`
		`dn = _get_arg( args, msg="Enter dn to delete")`
		`_warn(dn, msg="Deleting %s %s" % (SINGULAR.__name__, dn))`
		`- _generic_delete(inst, basedn, log.getChild('_generic_delete'), SINGULAR, dn)`
		`+ _generic_delete(inst, basedn, log.getChild('_generic_delete'), SINGULAR, dn, args)`

		`def create_parser(subparsers):`
		`ou_parser = subparsers.add_parser('organizationalunit', help='Manage organizational units')`

src/lib389/lib389/cli_idm/posixgroup.py

file modified

+7 -9

		`@@ -8,9 +8,8 @@`

		`import argparse`
		`from lib389.idm.posixgroup import PosixGroup, PosixGroups, MUST_ATTRIBUTES`
		`-`
		`- from lib389.cli_base import (`
		`- populate_attr_arguments,`
		`+ from lib389.cli_base import populate_attr_arguments`
		`+ from lib389.cli_idm import (`
		`_generic_list,`
		`_generic_get,`
		`_generic_get_dn,`
		`@@ -22,7 +21,6 @@`
		`_warn,`
		`)`

		`-`
		`SINGULAR = PosixGroup`
		`MANY = PosixGroups`
		`RDN = 'cn'`
		`@@ -30,24 +28,24 @@`
		`# These are a generic specification, try not to tamper with them`

		`def list(inst, basedn, log, args):`
		`- _generic_list(inst, basedn, log.getChild('_generic_list'), MANY)`
		`+ _generic_list(inst, basedn, log.getChild('_generic_list'), MANY, args)`

		`def get(inst, basedn, log, args):`
		`rdn = _get_arg( args.selector, msg="Enter %s to retrieve" % RDN)`
		`- _generic_get(inst, basedn, log.getChild('_generic_get'), MANY, rdn)`
		`+ _generic_get(inst, basedn, log.getChild('_generic_get'), MANY, rdn, args)`

		`def get_dn(inst, basedn, log, args):`
		`dn = lambda args: _get_arg( args.dn, msg="Enter dn to retrieve")`
		`- _generic_get_dn(inst, basedn, log.getChild('_generic_get_dn'), MANY, dn)`
		`+ _generic_get_dn(inst, basedn, log.getChild('_generic_get_dn'), MANY, dn, args)`

		`def create(inst, basedn, log, args):`
		`kwargs = _get_attributes(args.extra, MUST_ATTRIBUTES)`
		`- _generic_create(inst, basedn, log.getChild('_generic_create'), MANY, kwargs)`
		`+ _generic_create(inst, basedn, log.getChild('_generic_create'), MANY, kwargs, args)`

		`def delete(inst, basedn, log, args):`
		`dn = _get_arg( args, msg="Enter dn to delete")`
		`_warn(dn, msg="Deleting %s %s" % (SINGULAR.__name__, dn))`
		`- _generic_delete(inst, basedn, log.getChild('_generic_delete'), SINGULAR, dn)`
		`+ _generic_delete(inst, basedn, log.getChild('_generic_delete'), SINGULAR, dn, args)`

		`def create_parser(subparsers):`
		`posixgroup_parser = subparsers.add_parser('posixgroup', help='Manage posix groups')`

src/lib389/lib389/cli_idm/user.py

file modified

+7 -8

		`@@ -8,9 +8,8 @@`

		`import argparse`
		`from lib389.idm.user import nsUserAccount, nsUserAccounts`
		`-`
		`- from lib389.cli_base import (`
		`- populate_attr_arguments,`
		`+ from lib389.cli_base import populate_attr_arguments`
		`+ from lib389.cli_idm import (`
		`_generic_list,`
		`_generic_get,`
		`_generic_get_dn,`
		`@@ -29,25 +28,25 @@`
		`# These are a generic specification, try not to tamper with them`

		`def list(inst, basedn, log, args):`
		`- _generic_list(inst, basedn, log.getChild('_generic_list'), MANY)`
		`+ _generic_list(inst, basedn, log.getChild('_generic_list'), MANY, args)`

		`def get(inst, basedn, log, args):`
		`rdn = _get_arg( args.selector, msg="Enter %s to retrieve" % RDN)`
		`- _generic_get(inst, basedn, log.getChild('_generic_get'), MANY, rdn)`
		`+ _generic_get(inst, basedn, log.getChild('_generic_get'), MANY, rdn, args)`

		`def get_dn(inst, basedn, log, args):`
		`dn = lambda args: _get_arg( args.dn, msg="Enter dn to retrieve")`
		`- _generic_get_dn(inst, basedn, log.getChild('_generic_get_dn'), MANY, dn)`
		`+ _generic_get_dn(inst, basedn, log.getChild('_generic_get_dn'), MANY, dn, args)`

		`def create(inst, basedn, log, args):`
		`kwargs = _get_attributes(args, SINGULAR._must_attributes)`
		`- _generic_create(inst, basedn, log.getChild('_generic_create'), MANY, kwargs)`
		`+ _generic_create(inst, basedn, log.getChild('_generic_create'), MANY, kwargs, args)`

		`def delete(inst, basedn, log, args, warn=True):`
		`dn = _get_arg( args.dn, msg="Enter dn to delete")`
		`if warn:`
		`_warn(dn, msg="Deleting %s %s" % (SINGULAR.__name__, dn))`
		`- _generic_delete(inst, basedn, log.getChild('_generic_delete'), SINGULAR, dn)`
		`+ _generic_delete(inst, basedn, log.getChild('_generic_delete'), SINGULAR, dn, args)`

		`def status(inst, basedn, log, args):`
		`uid = _get_arg( args.uid, msg="Enter %s to check" % RDN)`

src/lib389/lib389/schema.py

file modified

+96 -21

		`@@ -12,6 +12,8 @@`
		`"""`
		`import glob`
		`import ldap`
		`+ from json import dumps as dump_json`
		`+ from operator import itemgetter`
		`from ldap.schema.models import AttributeType, ObjectClass, MatchingRule`

		`from lib389._constants import *`
		`@@ -125,38 +127,75 @@`
		`ent = ents[0]`
		`return ent.getValue('nsSchemaCSN')`

		`- def get_objectclasses(self):`
		`+`
		`+ def get_objectclasses(self, json=False):`
		`"""Returns a list of ldap.schema.models.ObjectClass objects for all`
		`objectClasses supported by this instance.`
		`"""`
		`attrs = ['objectClasses']`
		`results = self.conn.search_s(DN_SCHEMA, ldap.SCOPE_BASE,`
		`'objectclass=*', attrs)[0]`
		`- objectclasses = [ObjectClass(oc) for oc in`
		`- results.getValues('objectClasses')]`
		`- return objectclasses`
		`-`
		`- def get_attributetypes(self):`
		`+ if json:`
		`+ objectclasses = [vars(ObjectClass(oc)) for oc in`
		`+ results.getValues('objectClasses')]`
		`+ for oc in objectclasses:`
		`+ # Add normalized name for sorting`
		`+ oc['name'] = oc['names'][0].lower()`
		`+ objectclasses = sorted(objectclasses, key=itemgetter('name'))`
		`+ result = {'type': 'list', 'items': objectclasses}`
		`+ return dump_json(result)`
		`+ else:`
		`+ objectclasses = [ObjectClass(oc) for oc in`
		`+ results.getValues('objectClasses')]`
		`+ return objectclasses`
		`+`
		`+ def get_attributetypes(self, json=False):`
		`"""Returns a list of ldap.schema.models.AttributeType objects for all`
		`attributeTypes supported by this instance.`
		`"""`
		`attrs = ['attributeTypes']`
		`results = self.conn.search_s(DN_SCHEMA, ldap.SCOPE_BASE,`
		`'objectclass=*', attrs)[0]`
		`- attributetypes = [AttributeType(at) for at in`
		`- results.getValues('attributeTypes')]`
		`- return attributetypes`

		`- def get_matchingrules(self):`
		`+ if json:`
		`+ attributetypes = [vars(AttributeType(at)) for at in`
		`+ results.getValues('attributeTypes')]`
		`+ for attr in attributetypes:`
		`+ # Add normalized name for sorting`
		`+ attr['name'] = attr['names'][0].lower()`
		`+ attributetypes = sorted(attributetypes, key=itemgetter('name'))`
		`+ result = {'type': 'list', 'items': attributetypes}`
		`+ return dump_json(result)`
		`+ else:`
		`+ attributetypes = [AttributeType(at) for at in`
		`+ results.getValues('attributeTypes')]`
		`+ return attributetypes`
		`+`
		`+`
		`+ def get_matchingrules(self, json=False):`
		`"""Return a list of the server defined matching rules"""`
		`attrs = ['matchingrules']`
		`results = self.conn.search_s(DN_SCHEMA, ldap.SCOPE_BASE,`
		`'objectclass=*', attrs)[0]`
		`- matchingRules = [MatchingRule(mr) for mr in`
		`+ if json:`
		`+ matchingRules = [vars(MatchingRule(mr)) for mr in`
		`+ results.getValues('matchingRules')]`
		`+ for mr in matchingRules:`
		`+ # Add normalized name for sorting`
		`+ if mr['names']:`
		`+ mr['name'] = mr['names'][0].lower()`
		`+ else:`
		`+ mr['name'] = ""`
		`+ matchingRules = sorted(matchingRules, key=itemgetter('name'))`
		`+ result = {'type': 'list', 'items': matchingRules}`
		`+ return dump_json(result)`
		`+ else:`
		`+ matchingRules = [MatchingRule(mr) for mr in`
		`results.getValues('matchingRules')]`
		`- return matchingRules`
		`+ return matchingRules`

		`- def query_matchingrule(self, mr_name):`
		`+`
		`+ def query_matchingrule(self, mr_name, json=False):`
		`"""Returns a single matching rule instance that matches the mr_name.`
		`Returns None if the matching rule doesn't exist.`

		`@@ -171,11 +210,18 @@`
		`list(map(str.lower, mr.names))]`
		`if len(matchingRule) != 1:`
		`# This is an error.`
		`- return None`
		`+ if json:`
		`+ raise ValueError('Could not find matchingrule: ' + objectclassname)`
		`+ else:`
		`+ return None`
		`matchingRule = matchingRule[0]`
		`- return matchingRule`
		`+ if json:`
		`+ result = {'type': 'schema', 'mr': vars(matchingRule)}`
		`+ return dump_json(result)`
		`+ else:`
		`+ return matchingRule`

		`- def query_objectclass(self, objectclassname):`
		`+ def query_objectclass(self, objectclassname, json=False):`
		`"""Returns a single ObjectClass instance that matches objectclassname.`
		`Returns None if the objectClass doesn't exist.`

		`@@ -192,11 +238,18 @@`
		`list(map(str.lower, oc.names))]`
		`if len(objectclass) != 1:`
		`# This is an error.`
		`- return None`
		`+ if json:`
		`+ raise ValueError('Could not find objectcass: ' + objectclassname)`
		`+ else:`
		`+ return None`
		`objectclass = objectclass[0]`
		`- return objectclass`
		`+ if json:`
		`+ result = {'type': 'schema', 'oc': vars(objectclass)}`
		`+ return dump_json(result)`
		`+ else:`
		`+ return objectclass`

		`- def query_attributetype(self, attributetypename):`
		`+ def query_attributetype(self, attributetypename, json=False):`
		`"""Returns a tuple of the AttributeType, and what objectclasses may or`
		`must take this attributeType. Returns None if attributetype doesn't`
		`exist.`
		`@@ -223,7 +276,10 @@`
		`list(map(str.lower, at.names))]`
		`if len(attributetype) != 1:`
		`# This is an error.`
		`- return None`
		`+ if json:`
		`+ raise ValueError('Could not find attribute: ' + attributetypename)`
		`+ else:`
		`+ return None`
		`attributetype = attributetype[0]`
		`# Get the primary name of this attribute`
		`attributetypename = attributetype.names[0]`
		`@@ -233,4 +289,23 @@`
		`# Build a set if they have must.`
		`must = [oc for oc in objectclasses if attributetypename.lower() in`
		`list(map(str.lower, oc.must))]`
		`- return (attributetype, must, may)`
		`+`
		`+ if json:`
		`+ # convert Objectclass class to dict, then sort each list`
		`+ may = [vars(oc) for oc in may]`
		`+ must = [vars(oc) for oc in must]`
		`+ # Add normalized 'name' for sorting`
		`+ for oc in may:`
		`+ oc['name'] = oc['names'][0].lower()`
		`+ for oc in must:`
		`+ oc['name'] = oc['names'][0].lower()`
		`+ may = sorted(may, key=itemgetter('name'))`
		`+ must = sorted(must, key=itemgetter('name'))`
		`+ result = {'type': 'schema',`
		`+ 'at': vars(attributetype),`
		`+ 'may': may,`
		`+ 'must': must}`
		`+ return dump_json(result)`
		`+ else:`
		`+ return (attributetype, must, may)`
		`+`

src/lib389/lib389/utils.py

file modified

+6 -1

		`@@ -939,7 +939,12 @@`

		`def ensure_str(val):`
		`if val != None and type(val) != str:`
		`- return val.decode('utf-8')`
		`+ try:`
		`+ result = val.decode('utf-8')`
		`+ except UnicodeDecodeError:`
		`+ # binary value, just return str repr?`
firstyear commented 5 years ago No, because you may corrupt the data here. str in python 3 IS unicode, so you need to either return it as bytes, or just ALLOW the exception to be raised.
firstyear commented 5 years ago IE: Just let the exception be raised because we can't help you after this happens.
mreynolds commented 5 years ago So this happens when we try and get the nsState attribute from a replica. I know it can not read it, but the value should NOT cause an error. This was a really hack fix to get around it, but I'm not sure how to address it correctly.
		`+ result = str(val)`
		`+ return result`
		`return val`

mreynolds commented 5 years ago

Description: This is the initial patch for adding a json option to
the CLI tools. This patch also addresses issues with
using LDAPI with the CLI.

https://pagure.io/389-ds-base/issue/49683

Reviewed by: ?

vashirov commented on line 28 of src/lib389/lib389/cli_idm/user.py 5 years ago

argsn - typo?

rebased onto 6e78608370d2481893e2258d589496a34d0a8ea6

5 years ago

mreynolds commented 5 years ago

Nice catch, fixed!

spichugi commented on line 6 of src/lib389/cli/dsconf 5 years ago

There is no need for this kind of changes. Module 'argparse' takes care about it and takes the variable name from stripping '--' arg. '--verbose' in this case.
https://docs.python.org/3.6/library/argparse.html#dest

mreynolds commented on line 6 of src/lib389/cli/dsconf 5 years ago

The way I read that doc was that you had to set dest if you used "-v" AND "--verbose" Only if it's just "--verbose" is it automatically included. Have I misread it?

spichugi commented on line 6 of src/lib389/cli/dsconf 5 years ago

Hmm, I refer to this part of the doc:
"For optional argument actions, the value of dest is normally inferred from the option strings. ArgumentParser generates the value of dest by taking the first long option string and stripping away the initial -- string. If no long option strings were supplied, dest will be derived from the first short option string by stripping the initial - character. "

Also, I've tested your patch without "dest='verbose'" change and it works. :)
Still, it is not a bit issue. I am okay if it will stay like this.

mreynolds commented 5 years ago

I can change it, no problem since you verified it still works....

rebased onto d38eac91064d505f8ce336896396373466927757

5 years ago

mreynolds commented 5 years ago

Changes made, please review...

spichugi commented on line 20 of src/lib389/lib389/cli_idm/__init__.py 5 years ago

ldap_url - looks like a typo

spichugi commented 5 years ago

The 'list' method works

[root@qeos-55 ds]# dsidm -D "cn=Directory manager" --json -b "dc=example,dc=com" ldap://localhost:389 user list
Enter password for cn=Directory manager on ldap://localhost:389 :
{"type": "list", "items": ["user1", "user2"]}
Command successful.

But I have an error with 'get' method:

[root@qeos-55 ds]# dsidm -D "cn=Directory manager" --json -b "dc=example,dc=com" ldap://localhost:389 user get
Enter password for cn=Directory manager on ldap://localhost:389 :
Enter uid to retrieve : user1
Error: 'nsUserAccounts' object has no attribute 'get_all_attrs_json'

Do I need to apply some another patch? Or it will be just added later? I see you said it is an initial patch.

mreynolds commented 5 years ago

Weird, so the _generic_get() is being called from lib389/cli_base/init.py in your case, but I would expect _generic_get() should get called from lib389/cli_idm/init.py when using dsidm (which uses get_json() not get_all_attrs_json()).

So is something broke with the model? Needs more investigation...

rebased onto 7c1502aa2061cfa23b324e6c61c7cd32c93274dc

5 years ago

rebased onto 4cbff2c7a6bec611984753c14a830bf64a7f1d19

5 years ago

mreynolds commented 5 years ago

Well I fixed the model but I haven't verified it yet. I will do so tomorrow...

rebased onto aa850417176afce146a1459092e1b7baf76ff578

5 years ago

spichugi commented 5 years ago

A couple of issues here...
First, you use _get_arg function and it fails on Python 3 because it uses raw_input() function which is not present. We should use input() built-in function instead.

Another thing, 'create' function fails

dsidm -v -D "cn=Directory manager" -b "dc=example,dc=com" ldap://localhost:389 user create

Traceback (most recent call last):
  File "/usr/sbin/dsidm", line 109, in <module>
    args.func(inst, dsrc_inst['basedn'], log, args)
  File "/usr/local/lib/python3.6/site-packages/lib389/cli_idm/user.py", line 42, in create
    kwargs = _get_attributes(args, SINGULAR._must_attributes)
  File "/usr/local/lib/python3.6/site-packages/lib389/cli_idm/__init__.py", line 41, in _get_attributes
    for attr in attrs:
TypeError: object of type 'Namespace' has no len()

args = Namespace(basedn='dc=example,dc=com', binddn='cn=Directory manager', cn=None, displayName=None, func=<function create at 0x7f3d1741e598>, gidNumber=None, homeDirectory=None, instance='ldap://localhost:389', json=False, starttls=False, uid=None, uidNumber=None, verbose=True)

But I think it is not the purpose of the patch so we can skip it for later. It is easy to fix though. :)

Edited 5 years ago by spichugi

spichugi commented on line 61 of src/lib389/lib389/cli_idm/__init__.py 5 years ago

I have an error here:
AttributeError: 'nsUserAccounts' object has no attribute 'get_json'

spichugi commented on line 64 of src/lib389/lib389/cli_base/__init__.py 5 years ago

It fails with 'NameError: name 'json' is not defined'. I think we miss 'import json' in this file.

mreynolds commented 5 years ago

Sorry I haven't verified my last patch yet. I wanted to get to it yesterday but that didn't happen, but I'll be working on this later today. Thanks for the testing though!

spichugi commented 5 years ago

Sorry I haven't verified my last patch yet. I wanted to get to it yesterday but that didn't happen, but I'll be working on this later today. Thanks for the testing though!

Oh, sorry... I've seen the rebase...

Okay, thank you!

Edited 5 years ago by spichugi

firstyear commented on line 5 of src/cockpit/389-console/js/ds.js 5 years ago

//POC REMOVE!!! :)

firstyear commented on line 15 of src/lib389/lib389/_mapped_object.py 5 years ago

You probably want to get rid of this line.

firstyear commented on line 95 of src/lib389/lib389/cli_conf/plugins/automember.py 5 years ago

Please don't do this. The point is to have the LOWEST exception propogate because it has the LDAP info about WHY this failed. If you re-raise, we lose that knowledge, and then it becomes creally hard to trace issues. Please NEVER try-reraise in python.

firstyear commented on line 61 of src/lib389/lib389/cli_idm/__init__.py 5 years ago

_json is a magic type that is suffixed to any function of DSLdapObject, so for example, get() can be used as get_json() BUT it relies on the function having a specific output type.

firstyear commented on line 9 of src/lib389/lib389/utils.py 5 years ago

No, because you may corrupt the data here. str in python 3 IS unicode, so you need to either return it as bytes, or just ALLOW the exception to be raised.

firstyear commented on line 9 of src/lib389/lib389/utils.py 5 years ago

IE: Just let the exception be raised because we can't help you after this happens.

firstyear commented on line 6 of src/lib389/lib389/cli_idm/__init__.py 5 years ago

A better spot to put this is probably in the dsrc handling, because "ldapurl" is a derivation of "instance name". So remove this, and move it to the dsrc.py so that if instance name == ldap, then you use the ldapi:// url of the instance instead. You could even do something like local_<instancename> and have that generate it. This hardcoded here is not right sorry.

firstyear commented on line 107 of src/lib389/lib389/cli_conf/plugins/automember.py 5 years ago

The CN is required though,

firstyear commented on line 103 of src/lib389/lib389/cli_base/__init__.py 5 years ago

You may want a "json success" type that you could return here instead.

firstyear commented on line 20 of src/lib389/lib389/cli_base/__init__.py 5 years ago

See belowe, this is the wrong location for this code,

firstyear commented on line 5 of src/lib389/lib389/__init__.py 5 years ago

You can probably remove this change if you change the belowe ldapi settings to be part of dsrc.

firstyear commented 5 years ago

Hey there, lots of comments for you. Hope it helps :)

mreynolds commented on line 5 of src/cockpit/389-console/js/ds.js 5 years ago

I want to keep this example in the code until we actually have functions that use it properly. Then I will remove :-p

mreynolds commented on line 15 of src/lib389/lib389/_mapped_object.py 5 years ago

already did, I'm working on this right now so you are commenting a bit early. Wait a few hours I'll have a more revised patch to look at.

mreynolds commented on line 103 of src/lib389/lib389/cli_base/__init__.py 5 years ago

cockpit will just use the result code from the cli tool to know if its a success or failure. I thought about doing a "result" JSON object, but its not needed, and I don't want to add the extra complexity at this stage. That being said I could jsonify it for the _generic results, but the UI won't use it.

mreynolds commented on line 95 of src/lib389/lib389/cli_conf/plugins/automember.py 5 years ago

Yeah I wasn't sure if I should of been doing that. I'll remove it...

mreynolds commented on line 107 of src/lib389/lib389/cli_conf/plugins/automember.py 5 years ago

Okay, well your can't "require" an argument what has nargs='?'. I'll look into this see if nargs should be changed so we can bring the requirement back.

mreynolds commented on line 61 of src/lib389/lib389/cli_idm/__init__.py 5 years ago

I've fixed this already... the issue is that in this context mc is nsUserAccounts - so get_json() doesn't work as its a list, not an entry. Like I said, I have a fix for this and it will be in the next rebase.

mreynolds commented on line 9 of src/lib389/lib389/utils.py 5 years ago

So this happens when we try and get the nsState attribute from a replica. I know it can not read it, but the value should NOT cause an error. This was a really hack fix to get around it, but I'm not sure how to address it correctly.

mreynolds commented on line 6 of src/lib389/lib389/cli_idm/__init__.py 5 years ago

I'll look into this next...

rebased onto 3e1010f7f34e6b216296d7c79d12b8ccfec92c12

5 years ago

mreynolds commented 5 years ago

@firstyear - about to add dsrc changes next...

rebased onto a95f55abcf85d26811e5a8f437b3d28ea39c9beb

5 years ago

rebased onto 119f428d8a7325455ff1e2c48e3c7e8cc2453ae6

5 years ago

spichugi commented on line 86 of src/lib389/lib389/cli_idm/__init__.py 5 years ago

dscreate and 'dsidm user get' work. But 'dsidm user list' fail with 'NameError: name 'json' is not defined' failure. We dont have an import here... Sorry if you are already up to this

rebased onto 71681f4dd855cf764812d825fd8d19efa16a8a72

5 years ago

mreynolds commented 5 years ago

Fixed!

rebased onto 9d46d200134195ee5a0b2f3d75c8fcb1dc16e683

5 years ago

mreynolds commented 5 years ago

Also started adding JSON schema support, and added missing dsconf options for getting objectclasses and matching rules.

We need to rework how we use query_attribute/query_objectclass. We need a way to convert the python-ldap AttributeType class to json object, same for objectclass and matching rule ldap classes. I'll be working on that next....

rebased onto 38779dea5922c21dc5daf14affd9b5194207a5b9

5 years ago

mreynolds commented 5 years ago

Solved the AttributeType/Objectclass class to json issue,. Still need to properly implement the query functions for JSON...

Edited 5 years ago by mreynolds

rebased onto 18122b276deeea5209df173c180a12cdeae1df1b

5 years ago

mreynolds commented 5 years ago

Ready for review...

spichugi commented 5 years ago

Tested... Looks good to me! Ack.

mreynolds commented 5 years ago

I'll merge this for now, and if can address any further issues in a new ticket/PR.

I'm still not happy about how I handled "nsState" binary value issue. I just treat it as a string on exception, but I'm not sure what is the best way to process it. The value is valid, it's not an error to be "handled". For the UI/CLI it's not actually an attribute we care about, but the json processing should not error out on it either.

rebased onto 593a73e

5 years ago

Pull-Request has been merged by mreynolds

5 years ago

spichugi commented 3 years ago

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This pull request has been cloned to Github as issue and is available here:
- https://github.com/389ds/389-ds-base/issues/2758

If you want to continue to work on the PR, please navigate to the github issue,
download the patch from the attachments and file a new pull request.

Thank you for understanding. We apologize for all inconvenience.