There is an issue with starting IPA server when the upgrade the RPM packages happened during not running IPA server.
freeipa-server-4.3.90.201601291011GITd53c2f6-0.fc23.x86_64
Reproducer: 1. switch off the IPA server (systemctl stop ipa) 2. upgrade packages (dnf upgrade <IPA packages>), I used the master builds 3. systemctl start ipa
The problem is that IPA server needs to run "ipa-server-upgrade" before the starting it, what is correct, but it doesn't allow you to upgrade the server because IPA server is actually not running (cyclic dependency).
[root@master2 ~]# systemctl restart ipa Job for ipa.service failed because the control process exited with error code. See "systemctl status ipa.service" and "journalctl -xe" for details. [root@master2 ~]# systemctl status ipa.service ● ipa.service - Identity, Policy, Audit Loaded: loaded (/usr/lib/systemd/system/ipa.service; disabled; vendor preset: disabled) Active: failed (Result: exit-code) since Fri 2016-01-29 14:19:23 CET; 16s ago Process: 4000 ExecStart=/usr/sbin/ipactl start (code=exited, status=1/FAILURE) Main PID: 4000 (code=exited, status=1/FAILURE) Jan 29 14:19:21 master2.alich.work systemd[1]: Starting Identity, Policy, Audit... Jan 29 14:19:23 master2.alich.work ipactl[4000]: Upgrade required: please run ipa-server-upgrade command Jan 29 14:19:23 master2.alich.work ipactl[4000]: Aborting ipactl Jan 29 14:19:23 master2.alich.work systemd[1]: ipa.service: Main process exited, code=exited, status=1/FAILURE Jan 29 14:19:23 master2.alich.work systemd[1]: Failed to start Identity, Policy, Audit. Jan 29 14:19:23 master2.alich.work systemd[1]: ipa.service: Unit entered failed state. Jan 29 14:19:23 master2.alich.work systemd[1]: ipa.service: Failed with result 'exit-code'. [root@master2 ~]# ipa-server-upgrade session memcached servers not running IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. Cannot connect to LDAP server: cannot connect to 'ldapi://%2fvar%2frun%2fslapd-ALICH-WORK.socket': The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information [root@master2 ~]# cat /var/log/ipaupgrade.log - SNIP - 2016-01-29T13:19:58Z DEBUG Loading Index file from '/var/lib/ipa/sysrestore/sysrestore.index' 2016-01-29T13:19:58Z ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually. 2016-01-29T13:19:58Z DEBUG File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute return_value = self.run() File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py", line 48, in run raise admintool.ScriptError(str(e)) 2016-01-29T13:19:58Z DEBUG The ipa-server-upgrade command failed, exception: ScriptError: Cannot connect to LDAP server: cannot connect to 'ldapi://%2fvar%2frun%2fslapd-ALICH-WORK.socket': 2016-01-29T13:19:58Z ERROR Cannot connect to LDAP server: cannot connect to 'ldapi://%2fvar%2frun%2fslapd-ALICH-WORK.socket': 2016-01-29T13:19:58Z ERROR The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information
As workaround you would need to hack to make ldap server running or force to start ipa server (ipactl start -f) then the upgrade will pass.
[root@master2 ~]# ipactl start -f Skipping version check Starting Directory Service Starting krb5kdc Service Starting kadmin Service Starting ipa_memcached Service Starting httpd Service Starting ipa-custodia Service Starting pki-tomcatd Service Starting ipa-otpd Service ipa: INFO: The ipactl command was successful [root@master2 ~]# ipa-server-upgrade Upgrading IPA: [1/10]: stopping directory server [2/10]: saving configuration [3/10]: disabling listeners [4/10]: enabling DS global lock [5/10]: starting directory server [6/10]: updating schema [7/10]: upgrading server [8/10]: stopping directory server [9/10]: restoring configuration [10/10]: starting directory server Done. Update complete - SNIP - The IPA services were upgraded The ipa-server-upgrade command was successful
After the upgrade I ended up with (not running ipa)
dirsrv@ALICH-WORK.service loaded active running 389 Directory Server ALICH-WORK. <E2><97><8F> dirsrv@ALICH.WORK.service loaded failed failed 389 Directory Server ALICH.WORK. httpd.service loaded active running The Apache HTTP Server ipa-custodia.service loaded active running IPA Custodia Service <E2><97><8F> ipa.service loaded failed failed Identity, Policy, Audit ipa_memcached.service loaded active running IPA memcached daemon, increases IPA server performance kadmin.service loaded active running Kerberos 5 Password-changing and Administration
and after the ipa server restart (systemctl restart ipa) I got (running ipa and dirsrv)
dirsrv@ALICH-WORK.service loaded active running 389 Directory Server ALICH-WORK. <E2><97><8F> dirsrv@ALICH.WORK.service loaded failed failed 389 Directory Server ALICH.WORK. httpd.service loaded active running The Apache HTTP Server ipa-custodia.service loaded active running IPA Custodia Service ipa.service loaded active exited Identity, Policy, Audit ipa_memcached.service loaded active running IPA memcached daemon, increases IPA server performance kadmin.service loaded active running Kerberos 5 Password-changing and Administration
Regression introduced in bef0f4c
This bug breaks upgrades of IPA 4.2, 4.3, 4.4
master:
ipa-4-3:
ipa-4-2:
Linked to Bugzilla bug: https://bugzilla.redhat.com/show_bug.cgi?id=1298103 (Red Hat Enterprise Linux 7)
Linked to Bugzilla bug: https://bugzilla.redhat.com/show_bug.cgi?id=1296216 (Red Hat Enterprise Linux 7)
Metadata Update from @alich: - Issue assigned to mbabinsk - Issue set to the milestone: FreeIPA 4.2.4
Login to comment on this ticket.