When running the test TestInstallWithCA_KRA_DNS2, it fails at the step test_replica2_ipa_kra_install, which installs a third KRA to a replica:
TestInstallWithCA_KRA_DNS2
test_replica2_ipa_kra_install
[2/9]: configuring KRA instance 2017-10-16T14:57:00Z DEBUG The ipa-kra-install command failed, exception: RuntimeError: KRA configuration failed. 2017-10-16T14:57:00Z ERROR KRA configuration failed. 2017-10-16T14:57:00Z ERROR The ipa-kra-install command failed. See /var/log/ipaserver-kra-install.log for more information
The same happens in the test TestInstallWithCA_KRA2. (FreeIPA version freeipa-server-4.5.90test-0.fc25.x86_64)
TestInstallWithCA_KRA2
<img alt="debug.gz" src="/freeipa/issue/raw/597a34c747d576925dd5a38a9f0848e346aa9592bad94558f0fc37fa1b4a9b69-debug.gz" /> <img alt="ipaserver-kra-install.log.gz" src="/freeipa/issue/raw/032ce7b5cd5b101a30abd3861a45e0f37deee8bacfb1ba77837285aaad1debe7-ipaserver-kra-install.log.gz" /> <img alt="pki-kra-spawn.20171016145515.log.gz" src="/freeipa/issue/raw/e4afed3a3c172f97ece2ee527d33374d4dfb0546bef113448b292dd249eeaf13-pki-kra-spawn.20171016145515.log.gz" />
@ftweedal Hi Fraser, is this something you would be able to take a look at? Thanks!
Metadata Update from @pvoborni: - Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1510341
Issue linked to bug 1510341
The interesting piece from debug log is Failed to update number range.:
Failed to update number range.
[16/Oct/2017:14:56:56][http-bio-8443-exec-3]: len is 4 [16/Oct/2017:14:56:56][http-bio-8443-exec-3]: hostname: <master.ipa.test> [16/Oct/2017:14:56:56][http-bio-8443-exec-3]: admin_port: <443> [16/Oct/2017:14:56:56][http-bio-8443-exec-3]: hostname: <replica1.ipa.test> [16/Oct/2017:14:56:56][http-bio-8443-exec-3]: admin_port: <443> [16/Oct/2017:14:56:56][http-bio-8443-exec-3]: hostname: <replica2.ipa.test> [16/Oct/2017:14:56:56][http-bio-8443-exec-3]: admin_port: <443> [16/Oct/2017:14:56:56][http-bio-8443-exec-3]: hostname: <replica3.ipa.test> [16/Oct/2017:14:56:56][http-bio-8443-exec-3]: admin_port: <443> [16/Oct/2017:14:56:56][http-bio-8443-exec-3]: === Subsystem Configuration === [16/Oct/2017:14:56:56][http-bio-8443-exec-3]: SystemConfigService: validate clone URI: https://replica1.ipa.test:443 [16/Oct/2017:14:56:56][http-bio-8443-exec-3]: SystemConfigService: get configuration entries from master [16/Oct/2017:14:56:57][http-bio-8443-exec-3]: updateNumberRange start host=replica1.ipa.test adminPort=443 eePort=443 [16/Oct/2017:14:56:57][http-bio-8443-exec-3]: ConfigurationUtils: POST https://replica1.ipa.test:443/kra/admin/kra/updateNumberRange [16/Oct/2017:14:56:57][http-bio-8443-exec-3]: Server certificate: [16/Oct/2017:14:56:57][http-bio-8443-exec-3]: - subject: CN=replica1.ipa.test,O=IPA.TEST [16/Oct/2017:14:56:57][http-bio-8443-exec-3]: - issuer: CN=Certificate Authority,O=IPA.TEST [16/Oct/2017:14:57:00][http-bio-8443-exec-3]: content from admin interface =<?xml version="1.0" encoding="UTF-8" standalone="no"?><XMLResponse><Status>1</Status><Error>Error: Failed to update number range.</Error></XMLResponse> [16/Oct/2017:14:57:00][http-bio-8443-exec-3]: updateNumberRange(): status=1 java.io.IOException: Error: Failed to update number range. at com.netscape.cms.servlet.csadmin.ConfigurationUtils.updateNumberRange(ConfigurationUtils.java:707) at com.netscape.cms.servlet.csadmin.ConfigurationUtils.getConfigEntriesFromMaster(ConfigurationUtils.java:572) at org.dogtagpki.server.rest.SystemConfigService.configureClone(SystemConfigService.java:865) at org.dogtagpki.server.rest.SystemConfigService.configureSubsystem(SystemConfigService.java:1002) at org.dogtagpki.server.rest.SystemConfigService.configure(SystemConfigService.java:147) at org.dogtagpki.server.rest.SystemConfigService.configure(SystemConfigService.java:104) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
It is already fixed on Dogtag side (pki-10.5.1).
Petr, FreeIPA master and 4.6 require Dogtag 10.5.1 already. Is the test case passing again?
(master / fedora26): ipa.test_integration.test_installation.TestInstallWithCA_KRA_DNS2.test_replica2_ipa_kra_install is now fixed in domain level 1, but ipa.test_integration.test_installation.TestInstallWithCA_KRA2.test_replica2_ipa_kra_install is still failing with "Failed to update number range".
master:
@frenaud what build of Dogtag was in use in that failed test run? And do you have the CA/KRA debug logs from replica and master?
Metadata Update from @pvoborni: - Issue priority set to: important - Issue set to the milestone: FreeIPA 4.6.2 - Issue tagged with: test-failure
@ftweedal I sent you a mail with a private link to the tests failure
Thanks @frenaud. The logs show that indeed the new version of Dogtag, which contains the "fix", was in use. Investigation is ongoing.
ipa-4-6:
Unfortunately the logs I need for thorough analysis are not collected in CI. Specifically, I would need to see logs for the replica from which the KRA replica is being created, but logs are only collected from the host where ipa-kra-install is being executed. I filed an issue: https://pagure.io/freeipa/issue/7310.
ipa-kra-install
@frenaud has commented that tests seem to be passing since this failure. It was possibly a transient failure. It was possibly caused by slowness that resulted in the default 5 second delay after security domain login to not be enough. The latter type of failure could occur again - the 5s delay which was introduced in pkispawn is a "workaround" to limit the risk of this kind of failure occurring. Since tests are usually passing it seems to be "good enough".
pkispawn
The 100% guaranteed fix is to use a kind of signed token rather than replicating security domain session data. We have a Dogtag ticket to track this but it has not been implemented yet: https://pagure.io/dogtagpki/issue/2831.
What is next? FreeIPA spec file already has the required version of Dogtag. So I guess it is a decision for QE about whether to remove the xfail on the relevant tests... or I might make a PR and we can see what happens.
xfail
edit: I did a PR that removes the failing-test annotations: https://github.com/freeipa/freeipa/pull/1366
Metadata Update from @tdudlak: - Issue set to the milestone: FreeIPA 4.6.3 (was: FreeIPA 4.6.2)
Metadata Update from @ftweedal: - Issue close_status updated to: fixed
Login to comment on this ticket.