During installation of IPA master / replica, systemctl fails to restart kadmin and the server installation fails.
The behaviour is non-deterministic, happens in about 1 out of 20 installations, reproduced on fedora 25 with freeipa copr and git master.
This issue makes the PR CI unstable.
git build: 7f4c2fb
2017-05-18T01:07:02Z DEBUG Configuring kadmin 2017-05-18T01:07:02Z DEBUG [1/2]: starting kadmin 2017-05-18T01:07:02Z DEBUG Starting external process 2017-05-18T01:07:02Z DEBUG args=/bin/systemctl is-active kadmin.service 2017-05-18T01:07:02Z DEBUG Process finished, return code=3 2017-05-18T01:07:02Z DEBUG stdout=inactive 2017-05-18T01:07:02Z DEBUG stderr= 2017-05-18T01:07:02Z DEBUG Loading StateFile from '/var/lib/ipa/sysrestore/sysrestore.state' 2017-05-18T01:07:02Z DEBUG Saving StateFile to '/var/lib/ipa/sysrestore/sysrestore.state' 2017-05-18T01:07:02Z DEBUG Starting external process 2017-05-18T01:07:02Z DEBUG args=/bin/systemctl restart kadmin.service 2017-05-18T01:07:02Z DEBUG Process finished, return code=1 2017-05-18T01:07:02Z DEBUG stdout= 2017-05-18T01:07:02Z DEBUG stderr=Job for kadmin.service failed because the control process exited with error code. See "systemctl status kadmin.service" and "journalctl -xe" for details. 2017-05-18T01:07:02Z DEBUG Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 504, in start_creation run_step(full_msg, method) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 494, in run_step method() File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 668, in __start self.restart() File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 404, in restart self.service.restart(instance_name, capture_output=capture_output, wait=wait) File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", line 322, in restart capture_output, wait) File "/usr/lib/python2.7/site-packages/ipaplatform/base/services.py", line 310, in _restart_base skip_output=not capture_output) File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 495, in run raise CalledProcessError(p.returncode, arg_string, str(output)) CalledProcessError: Command '/bin/systemctl restart kadmin.service' returned non-zero exit status 1
Complete logs:
ipa-replica-install:
Metadata Update from @tkrizek: - Issue priority set to: critical
kadmin.log
kadmind: Address already in use - Cannot bind server socket on 0.0.0.0.749 May 18 01:07:02 master.ipa.test kadmind[14187](Error): Failed setting up a RPC socket (for 0.0.0.0.749) kadmind: Address already in use - Error setting up network
Metadata Update from @pvoborni: - Issue set to the milestone: FreeIPA 4.6
After 135 test runs of simple replication test on 4.5.2, I ran into this issue again - https://fedorapeople.org/groups/freeipa/prci/jobs/96b4780a-650c-11e7-9e43-001a4a2315e4/
This issue happens in production PR CI every now and then. Also, IMO this should be fixed in 4.5 as well, since the issue happens there as well.
Metadata Update from @tkrizek: - Issue set to the milestone: FreeIPA 4.6.1 (was: FreeIPA 4.6)
Metadata Update from @tkrizek: - Issue set to the milestone: FreeIPA 4.6.2 (was: FreeIPA 4.6.1)
Metadata Update from @tdudlak: - Issue set to the milestone: FreeIPA 4.6.3 (was: FreeIPA 4.6.2)
Metadata Update from @rcritten: - Issue set to the milestone: FreeIPA 4.6.4 (was: FreeIPA 4.6.3)
FreeIPA 4.6.3 has been released, moving to FreeIPA 4.6.4 milestone
Metadata Update from @rcritten: - Issue set to the milestone: FreeIPA 4.6.5 (was: FreeIPA 4.6.4)
The port is sometimes blocked by rpcbind. There is nothing we can do about the issue from our side. I have opened https://bugzilla.redhat.com/show_bug.cgi?id=1592883 against rpcbind.
Metadata Update from @cheimes: - Issue priority set to: normal (was: important)
The BZ referenced by Christian is closed, marking this as fixed.
Metadata Update from @rcritten: - Issue close_status updated to: fixed - Issue status updated to: Closed (was: Open)
During uninstall using 4.8.10-6.fc33 this happens:
` File "/usr/lib/python3.9/site-packages/ipapython/install/core.py", line 421, in runner step() File "/usr/lib/python3.9/site-packages/ipapython/install/core.py", line 418, in <lambda> step = lambda: next(self.__gen) File "/usr/lib/python3.9/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from six.reraise(*exc_info) File "/usr/lib/python3.9/site-packages/six.py", line 703, in reraise raise value File "/usr/lib/python3.9/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from value = gen.send(prev_value) File "/usr/lib/python3.9/site-packages/ipapython/install/common.py", line 73, in _uninstall for unused in self._uninstaller(self.parent): File "/usr/lib/python3.9/site-packages/ipaserver/install/server/__init.py", line 581, in main uninstall(self) File "/usr/lib/python3.9/site-packages/ipaserver/install/server/install.py", line 272, in decorated func(installer) File "/usr/lib/python3.9/site-packages/ipaserver/install/server/install.py", line 1168, in uninstall krbinstance.KrbInstance(fstore).uninstall() File "/usr/lib/python3.9/site-packages/ipaserver/install/krbinstance.py", line 641, in uninstall self.kpasswd.uninstall() File "/usr/lib/python3.9/site-packages/ipaserver/install/service.py", line 824, in uninstall self.start() File "/usr/lib/python3.9/site-packages/ipaserver/install/service.py", line 495, in start self.service.start(instance_name, capture_output=capture_output, wait=wait) File "/usr/lib/python3.9/site-packages/ipaplatform/base/services.py", line 309, in start ipautil.run([paths.SYSTEMCTL, "start", File "/usr/lib/python3.9/site-packages/ipapython/ipautil.py", line 594, in run raise CalledProcessError(
2021-01-23T16:29:37Z DEBUG The ipa-server-install command failed, exception: CalledProcessError: CalledProcessError(Command ['/bin/systemctl', 'start', 'kadmin.service'] returned non-zero exit status 1: 'Job for kadmin.service failed because the control process exited with error code.\nSee "systemctl status kadmin.service" and "journalctl -xe" for details.\n')`
kadmin is likely not required post-install. The uninstaller tries to restore the system to its previous state and it must have recorded that kadmin was running at that time.
I suspect this is related to your issues in https://pagure.io/freeipa/issue/8675
Metadata Update from @rcritten: - Issue set to the milestone: None (was: FreeIPA 4.6.5)
kadmin is likely not required post-install. The uninstaller tries to restore the system to its previous state and it must have recorded that kadmin was running at that time. I suspect this is related to your issues in https://pagure.io/freeipa/issue/8675
Yes I agree. I just can't seem to get a clean uninstall and/or a clean reinstall. I'm happy to provide any logs/details.
Any other suggestions?
Login to comment on this ticket.