#6016 ipa-ca-install on replica tries to connect to master:8443
Closed: Fixed None Opened 7 years ago by cheimes.

ipa-ca-install fails to install a CA when the master has port 8443/TCP blocked on its firewall. The problem is caused by an erroneous connection check in ipapython.dogtag.ca_status(). The replica waits for https://master:8443/ca/admin/ca/getStatus instead of https://replica1:8443/ca/admin/ca/getStatus.

I think the issues is caused by the fact that ipapython.dogtag.ca_status() is called without a host name. It falls back to api.env.ca_host, which is still the host name of the master.

freeipa-server-4.3.1-1.fc24.x86_64

[root@replica1 vagrant]# ipa-ca
ipa-cacert-manage  ipa-ca-install     
[root@replica1 vagrant]# ipa-ca-install 
WARNING: yacc table file version is out of date
Directory Manager (existing master) password:

Run connection check to master
Connection check OK
Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes 30 seconds
  [1/23]: creating certificate server user
  [2/23]: creating certificate server db
  [3/23]: setting up initial replication
Starting replication, please wait until this has completed.
Update in progress, 34 seconds elapsed
Update succeeded

  [4/23]: creating installation admin user
  [5/23]: setting up certificate server
  [6/23]: stopping instance to update CS.cfg
  [7/23]: backing up CS.cfg
  [8/23]: disabling nonces
  [9/23]: set up CRL publishing
  [10/23]: enable PKIX certificate path discovery and validation
  [11/23]: set up client auth to db
  [12/23]: destroying installation admin user
  [13/23]: starting instance

Installation doesn't succeed until I unblock 8443 on the master.

  [14/23]: importing CA chain to RA certificate database
  [15/23]: fixing RA database permissions
  [16/23]: setting up signing cert profile
  [17/23]: setting audit signing renewal to 2 years
  [18/23]: configure certificate renewals
  [19/23]: configure Server-Cert certificate renewal
  [20/23]: Configure HTTP to proxy connections
  [21/23]: updating IPA configuration
  [22/23]: Restart HTTP server to pick up changes
  [23/23]: enabling CA instance
Done configuring certificate server (pki-tomcatd).

/var/log/ipaserver-ca-install.log

2016-06-30T13:49:51Z DEBUG args=/usr/sbin/ipa-replica-conncheck --master master.ipa.example --auto-master-check --realm IPA.EXAMPLE --hostname replica1.ipa.example --ca-cert-file /etc/ipa/ca.crt
2016-06-30T13:50:08Z DEBUG Process finished, return code=0
2016-06-30T13:50:08Z DEBUG stdout=Check connection from replica to remote master 'master.ipa.example':
   Directory Service: Unsecure port (389): OK
   Directory Service: Secure port (636): OK
   Kerberos KDC: TCP (88): OK
   Kerberos Kpasswd: TCP (464): OK
   HTTP Server: Unsecure port (80): OK
   HTTP Server: Secure port (443): OK

The following list of ports use UDP protocol and would need to be
checked manually:
   Kerberos KDC: UDP (88): SKIPPED
   Kerberos Kpasswd: UDP (464): SKIPPED

Connection from replica to master is OK.
Start listening on required ports for remote master check
Get credentials to log in to remote master
Check RPC connection to remote master
Execute check on remote master
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13] Permission denied: 'lextab.py'
WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission denied: 'yacctab.py'
Check connection from master to remote replica 'replica1.ipa.example':
   Directory Service: Unsecure port (389): OK
   Directory Service: Secure port (636): OK
   Kerberos KDC: TCP (88): OK
   Kerberos KDC: UDP (88): WARNING
   Kerberos Kpasswd: TCP (464): OK
   Kerberos Kpasswd: UDP (464): WARNING
   HTTP Server: Unsecure port (80): OK
   HTTP Server: Secure port (443): OK
The following UDP ports could not be verified as open: 88, 464
This can happen if they are already bound to an application
and ipa-replica-conncheck cannot attach own UDP responder.

Connection from master to replica is OK.

...

2016-06-30T13:52:16Z DEBUG stderr=
2016-06-30T13:52:16Z DEBUG Service pki-tomcatd@pki-tomcat is not running, continue.
2016-06-30T13:52:16Z DEBUG   duration: 0 seconds
2016-06-30T13:52:16Z DEBUG   [12/23]: destroying installation admin user
2016-06-30T13:52:16Z DEBUG   duration: 0 seconds
2016-06-30T13:52:16Z DEBUG   [13/23]: starting instance
2016-06-30T13:52:16Z DEBUG Starting external process
2016-06-30T13:52:16Z DEBUG args=/bin/systemctl start pki-tomcatd@pki-tomcat.service
2016-06-30T13:52:16Z DEBUG Process finished, return code=0
2016-06-30T13:52:16Z DEBUG stdout=
2016-06-30T13:52:16Z DEBUG stderr=
2016-06-30T13:52:16Z DEBUG Starting external process
2016-06-30T13:52:16Z DEBUG args=/bin/systemctl is-active pki-tomcatd@pki-tomcat.service
2016-06-30T13:52:16Z DEBUG Process finished, return code=0
2016-06-30T13:52:16Z DEBUG stdout=active

2016-06-30T13:52:16Z DEBUG stderr=
2016-06-30T13:52:16Z DEBUG wait_for_open_ports: localhost [8080, 8443] timeout 300
2016-06-30T13:52:19Z DEBUG Waiting until the CA is running
2016-06-30T13:52:19Z DEBUG Starting external process
2016-06-30T13:52:19Z DEBUG args=/usr/bin/curl -o - --connect-timeout 30 -k https://master.ipa.example:8443/ca/admin/ca/getStatus
2016-06-30T13:52:19Z DEBUG Process finished, return code=7
2016-06-30T13:52:19Z DEBUG stdout=
2016-06-30T13:52:19Z DEBUG stderr=  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
^M  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to master.ipa.example port 8443: No route to host

2016-06-30T13:52:19Z DEBUG The CA status is: check interrupted due to error: Command '/usr/bin/curl -o - --connect-timeout 30 -k https://master.ipa.example:8443/ca/admin/ca/getStatus' returned non-zero exit status 7
2016-06-30T13:52:19Z DEBUG Waiting for CA to start...
2016-06-30T13:52:20Z DEBUG Starting external process
2016-06-30T13:52:20Z DEBUG args=/usr/bin/curl -o - --connect-timeout 30 -k https://master.ipa.example:8443/ca/admin/ca/getStatus
2016-06-30T13:52:21Z DEBUG Process finished, return code=7
2016-06-30T13:52:21Z DEBUG stdout=
2016-06-30T13:52:21Z DEBUG stderr=  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
^M  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to master.ipa.example port 8443: No route to host
...

After I unblocked 8443/TCP on the master

...
2016-06-30T13:54:02Z DEBUG Waiting for CA to start...
2016-06-30T13:54:56Z DEBUG Starting external process
2016-06-30T13:54:56Z DEBUG args=/usr/bin/curl -o - --connect-timeout 30 -k https://master.ipa.example:8443/ca/admin/ca/getStatus
2016-06-30T13:54:56Z DEBUG Process finished, return code=0
2016-06-30T13:54:56Z DEBUG stdout=<?xml version="1.0" encoding="UTF-8" standalone="no"?><XMLResponse><State>1</State><Type>CA</Type><Status>running</Status><Version>10.3.1-1.fc24</Version></XMLResponse>
2016-06-30T13:54:56Z DEBUG stderr=  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
^M  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0^M100   168  100   168    0     0   1958      0 --:--:-- --:--:-- --:--:--  1976

2016-06-30T13:54:56Z DEBUG The CA status is: running
2016-06-30T13:54:56Z DEBUG   duration: 159 seconds
2016-06-30T13:54:56Z DEBUG   [14/23]: importing CA chain to RA certificate database
2016-06-30T13:54:56Z DEBUG Starting external process
2016-06-30T13:54:56Z DEBUG args=/usr/bin/openssl pkcs7 -inform DER -print_certs
2016-06-30T13:54:56Z DEBUG Process finished, return code=0
2016-06-30T13:54:56Z DEBUG stdout=subject=/O=IPA.EXAMPLE/CN=Certificate Authority
issuer=/O=IPA.EXAMPLE/CN=Certificate Authority
...

First I tried to reset ca_host but api.env is read-only:

>>> from ipalib import api
>>> api.bootstrap()
>>> api.finalize()
>>> api.env.host
u'replica1.ipa.example'
>>> api.env.ca_host
u'master.ipa.example'
>>> api.env.ca_host = api.env.host
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/ipalib/config.py", line 229, in __setattr__
    self[name] = value
  File "/usr/lib/python2.7/site-packages/ipalib/config.py", line 242, in __setitem__
    (self.__class__.__name__, key, self.__d[key], value)
AttributeError: cannot override Env.ca_host value u'master.ipa.example' with u'replica1.ipa.example'

After some consideration I came to the conclusion that RedHatCAService.wait_until_running() should call dogtag.ca_status(api.env.host). It doesn't make sense to use ca_host here. The rest of the class waits for a local system service, so wait_until_running should always contact the local instance.

4.2 and 4.3 are also affected.

4.3.2 was released, moving to 4.3.3

master:

  • 1de92b1 RedHatCAService should wait for local Dogtag instance

ipa-4-3:

  • 16491d7 RedHatCAService should wait for local Dogtag instance

Metadata Update from @cheimes:
- Issue assigned to someone
- Issue set to the milestone: FreeIPA 4.3.3

7 years ago

Login to comment on this ticket.

Metadata