#7905 ipa-dnskeysync-replica should handle LDAP down gracefully
Closed: fixed 5 years ago by frenaud. Opened 5 years ago by cheimes.

Issue

The helper command ipa-dnskeysync-replica doesn't handle LDAP connection issues well. The command is executed by ipa-dnskeysyncd to synchronize DNSSEC key material. During maintenance cycles (e.g. updates) the LDAP server is shut down. When the script cannot connect to its local LDAP server, it fails with a visible exception. Instead it should fail with a clean error message and exit code > 0.

Steps to Reproduce

  1. stop 389-DS
  2. run SOFTHSM2_CONF=/etc/ipa/dnssec/softhsm2.conf sudo -E -u ods -- /usr/libexec/ipa/ipa-dnskeysync-replica

Actual behavior

ipa-dnskeysync-replica: DEBUG    Kerberos principal: ipa-dnskeysyncd/master.ipa.example
ipalib.install.kinit: DEBUG    Initializing principal ipa-dnskeysyncd/master.ipa.example using keytab /etc/ipa/dnssec/ipa-dnskeysyncd.keytab
ipalib.install.kinit: DEBUG    using ccache /tmp/ipa-dnskeysync-replica.ccache
ipalib.install.kinit: DEBUG    Attempt 1/5: success
ipa-dnskeysync-replica: DEBUG    Got TGT
ipa-dnskeysync-replica: DEBUG    Connecting to LDAP
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/ipapython/ipaldap.py", line 1023, in error_handler
    yield
  File "/usr/lib/python3.7/site-packages/ipapython/ipaldap.py", line 1189, in gssapi_bind
    '', auth_tokens, server_controls, client_controls)
  File "/usr/lib64/python3.7/site-packages/ldap/ldapobject.py", line 465, in sasl_interactive_bind_s
    return self._ldap_call(self._l.sasl_interactive_bind_s,who,auth,RequestControlTuples(serverctrls),RequestControlTuples(clientctrls),sasl_flags)
  File "/usr/lib64/python3.7/site-packages/ldap/ldapobject.py", line 329, in _ldap_call
    reraise(exc_type, exc_value, exc_traceback)
  File "/usr/lib64/python3.7/site-packages/ldap/compat.py", line 44, in reraise
    raise exc_value
  File "/usr/lib64/python3.7/site-packages/ldap/ldapobject.py", line 313, in _ldap_call
    result = func(*args,**kwargs)
ldap.SERVER_DOWN: {'desc': "Can't contact LDAP server", 'errno': 111, 'info': 'Connection refused'}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/libexec/ipa/ipa-dnskeysync-replica", line 168, in <module>
    ldap.gssapi_bind()
  File "/usr/lib/python3.7/site-packages/ipapython/ipaldap.py", line 1189, in gssapi_bind
    '', auth_tokens, server_controls, client_controls)
  File "/usr/lib64/python3.7/contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python3.7/site-packages/ipapython/ipaldap.py", line 1077, in error_handler
    error=info)
ipalib.errors.NetworkError: cannot connect to 'ldapi://%2fvar%2frun%2fslapd-IPA-EXAMPLE.socket': Connection refused

Expected behavior

script fails with an error message and exit code > 0.

Version/Release/Distribution

FreeIPA 4.7.2


ipa-dnskeysyncd has a similar problem:

# journalctl -u ipa-dnskeysyncd.service -o cat
...
ipa-dnskeysyncd: ERROR    syncrepl_poll: LDAP error ({'desc': "Can't contact LDAP server"})
Traceback (most recent call last):
  File "/usr/libexec/ipa/ipa-dnskeysyncd", line 116, in <module>
    while ldap_connection.syncrepl_poll(all=1, msgid=ldap_search):
  File "/usr/lib64/python3.7/site-packages/ldap/syncrepl.py", line 407, in syncrepl_poll
    all=0,
  File "/usr/lib64/python3.7/site-packages/ldap/ldapobject.py", line 756, in result4
    ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
  File "/usr/lib64/python3.7/site-packages/ldap/ldapobject.py", line 329, in _ldap_call
    reraise(exc_type, exc_value, exc_traceback)
  File "/usr/lib64/python3.7/site-packages/ldap/compat.py", line 44, in reraise
    raise exc_value
  File "/usr/lib64/python3.7/site-packages/ldap/ldapobject.py", line 313, in _ldap_call
    result = func(*args,**kwargs)
ldap.SERVER_DOWN: {'desc': "Can't contact LDAP server"}
ipa-dnskeysyncd.service: Main process exited, code=exited, status=1/FAILURE
ipa-dnskeysyncd.service: Failed with result 'exit-code'.
ipa-dnskeysyncd.service: Service RestartSec=1min expired, scheduling restart.
ipa-dnskeysyncd.service: Scheduled restart job, restart counter is at 1.
Stopped IPA key daemon.
Started IPA key daemon.
ipa-dnskeysyncd: CRITICAL Kerberos authentication failed: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (2529638972): Generic error (see e-text)
ipa-dnskeysyncd.service: Main process exited, code=exited, status=1/FAILURE
ipa-dnskeysyncd.service: Failed with result 'exit-code'.
ipa-dnskeysyncd.service: Service RestartSec=1min expired, scheduling restart.
ipa-dnskeysyncd.service: Scheduled restart job, restart counter is at 2.
Stopped IPA key daemon.
Started IPA key daemon.
...

WIth fix https://github.com/freeipa/freeipa/pull/3017 the script is now failing more gracefully.

...
ipalib.install.kinit: DEBUG    using ccache /tmp/ipa-dnskeysync-replica.ccache
ipalib.install.kinit: DEBUG    Attempt 1/5: success
ipa-dnskeysync-replica: DEBUG    Got TGT
ipa-dnskeysync-replica: DEBUG    Connecting to LDAP
ipa-dnskeysync-replica: ERROR    LDAP server is down: cannot connect to 'ldapi://%2fvar%2frun%2fslapd-IPA-EXAMPLE.socket': Connection refused
# echo $?
1

Metadata Update from @cheimes:
- Custom field on_review adjusted to https://github.com/freeipa/freeipa/pull/3017
- Issue assigned to cheimes

5 years ago

master:

  • 66873e2 Improve error handling in DNSSEC helpers

ipa-4-7:

  • 06426d0 Improve error handling in DNSSEC helpers

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

5 years ago

Login to comment on this ticket.

Metadata