freeipa

Clone
Source Code
GIT

#9037 4.9.6 ca setup failure on Rocky 85 (FileNotFoundError: [Errno 2] No such file or directory: 'sysctl': 'sysctl')
Closed: worksforme 2 years ago by wolf-allywilson. Opened 2 years ago by wolf-allywilson.

Closed: worksforme
Reopen Issue

Issue

During setup of ipa-server, erroring out with the error in title.

Steps to Reproduce

  1. Install Rocky Linux 8.5
  2. dnf install @idm:DL1 -y
  3. dnf install ipa-server -y
  4. /sbin/ipa-server-install --debug --log-file=/opt/ipa-server-install.log --realm=MY.REALM.NAME --domain=my.realm.name --ds-password=SuperSecret123 --admin-password=DuperDecret234 --unattended

Actual behavior

The installations works fine, right up until the setup of the CA, it errors out with the following (debug log):

DEBUG: Command: systemctl start pki-tomcatd@pki-tomcat.service
INFO: Waiting for PKI server to start
INFO: Waiting for PKI server to start (1s)
INFO: Waiting for PKI server to start (2s)
INFO: PKI server started
INFO: Waiting for CA subsystem
FileNotFoundError: [Errno 2] No such file or directory: 'sysctl': 'sysctl'
File "/usr/lib/python3.6/site-packages/pki/server/pkispawn.py", line 575, in main
scriptlet.spawn(deployer)
File "/usr/lib/python3.6/site-packages/pki/server/deployment/scriptlets/configuration.py", line 713, in spawn
subsystem.wait_for_startup(deployer.startup_timeout, deployer.request_timeout)
File "/usr/lib/python3.6/site-packages/pki/server/subsystem.py", line 409, in wait_for_startup
fips_mode = pki.FIPS.is_enabled()
File "/usr/lib/python3.6/site-packages/pki/init.py", line 246, in is_enabled
output = subprocess.check_output(command, stderr=fnull).decode('utf-8')
File "/usr/lib64/python3.6/subprocess.py", line 356, in check_output
kwargs).stdout
File "/usr/lib64/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, kwargs) as process:
File "/usr/lib64/python3.6/subprocess.py", line 729, in init
restore_signals, start_new_session)
File "/usr/lib64/python3.6/subprocess.py", line 1364, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)

2021-11-17T10:06:49Z CRITICAL Failed to configure CA instance
2021-11-17T10:06:49Z CRITICAL See the installation logs and the following files/directories for > more information:
2021-11-17T10:06:49Z CRITICAL /var/log/pki/pki-tomcat
2021-11-17T10:06:49Z DEBUG Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/ipaserver/install/service.py", line 635, in start_creation
run_step(full_msg, method)
File "/usr/lib/python3.6/site-packages/ipaserver/install/service.py", line 621, in run_step
method()
File "/usr/lib/python3.6/site-packages/ipaserver/install/cainstance.py", line 629, in __spawn_instance
nolog_list=nolog_list
File "/usr/lib/python3.6/site-packages/ipaserver/install/dogtaginstance.py", line 213, in spawn_instance
self.handle_setup_error(e)
File "/usr/lib/python3.6/site-packages/ipaserver/install/dogtaginstance.py", line 567, in handle_setup_error
) from None
RuntimeError: CA configuration failed.

2021-11-17T10:06:49Z DEBUG [error] RuntimeError: CA configuration failed.
2021-11-17T10:06:49Z DEBUG Removing /root/.dogtag/pki-tomcat/ca
2021-11-17T10:06:49Z DEBUG File "/usr/lib/python3.6/site-packages/ipapython/admintool.py", line 180, in execute
return_value = self.run()
File "/usr/lib/python3.6/site-packages/ipapython/install/cli.py", line 342, in run
return cfgr.run()
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 360, in run
return self.execute()
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 386, in execute
for rval in self._executor():
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 431, in runner
exc_handler(exc_info)
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 460, in _handle_execute_exception
self._handle_exception(exc_info)
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 450, in _handle_exception
six.reraise(exc_info)
File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 421, in __runner
step()
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 418, in <lambda>
step = lambda: next(self.__gen)
File "/usr/lib/python3.6/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
six.reraise(exc_info)
File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/usr/lib/python3.6/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
value = gen.send(prev_value)
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 655, in _configure
next(executor)
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 431, in __runner
exc_handler(exc_info)
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 460, in _handle_execute_exception
self._handle_exception(exc_info)
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 518, in _handle_exception
self.__parent._handle_exception(exc_info)
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 450, in _handle_exception
six.reraise(exc_info)
File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 515, in _handle_exception
super(ComponentBase, self)._handle_exception(exc_info)
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 450, in _handle_exception
six.reraise(exc_info)
File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 421, in __runner
step()
File "/usr/lib/python3.6/site-packages/ipapython/install/core.py", line 418, in <lambda>
step = lambda: next(self.__gen)
File "/usr/lib/python3.6/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
six.reraise(*exc_info)
File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/usr/lib/python3.6/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
value = gen.send(prev_value)
File "/usr/lib/python3.6/site-packages/ipapython/install/common.py", line 65, in _install
for unused in self._installer(self.parent):
File "/usr/lib/python3.6/site-packages/ipaserver/install/server/__init.py", line 575, in main
master_install(self)
File "/usr/lib/python3.6/site-packages/ipaserver/install/server/install.py", line 275, in decorated
func(installer)
File "/usr/lib/python3.6/site-packages/ipaserver/install/server/install.py", line 909, in install
ca.install_step_0(False, None, options, custodia=custodia)
File "/usr/lib/python3.6/site-packages/ipaserver/install/ca.py", line 355, in install_step_0
pki_config_override=options.pki_config_override,
File "/usr/lib/python3.6/site-packages/ipaserver/install/cainstance.py", line 503, in configure_instance
self.start_creation(runtime=runtime)
File "/usr/lib/python3.6/site-packages/ipaserver/install/service.py", line 635, in start_creation
run_step(full_msg, method)
File "/usr/lib/python3.6/site-packages/ipaserver/install/service.py", line 621, in run_step
method()
File "/usr/lib/python3.6/site-packages/ipaserver/install/cainstance.py", line 629, in __spawn_instance
nolog_list=nolog_list
File "/usr/lib/python3.6/site-packages/ipaserver/install/dogtaginstance.py", line 213, in spawn_instance
self.handle_setup_error(e)
File "/usr/lib/python3.6/site-packages/ipaserver/install/dogtaginstance.py", line 567, in handle_setup_error
) from None

2021-11-17T10:06:49Z DEBUG The ipa-server-install command failed, exception: RuntimeError: CA configuration failed.
2021-11-17T10:06:49Z ERROR CA configuration failed.

Expected behavior

The CA to install correctly

Version/Release/Distribution

$ rpm -q freeipa-server freeipa-client ipa-server ipa-client 389-ds-base pki-ca krb5-server

package freeipa-server is not installed
package freeipa-client is not installed
ipa-server-4.9.6-6.module+el8.5.0+675+61f67439.x86_64
ipa-client-4.9.6-6.module+el8.5.0+675+61f67439.x86_64
389-ds-base-1.4.3.23-10.module+el8.5.0+700+370e33d5.x86_64
pki-ca-10.11.2-2.module+el8.5.0+701+8dc610e5.noarch
krb5-server-1.18.2-14.el8.x86_64

Additional info:

Disabled selinux to see if that was the cause, it's not. I see reference to FIPS being enabled, but it's not. This is not a replica, it's a master - but I do also get the same error when installing a replica.

Looks like you are missing sysctl utility? It is in procps-ng package. At least in Fedora it is required by the dogtag-pki-server package.

Please report this to Rocky. We as upstream have no influence over the downstream packages.

I'll report it to Rocky as well, but thought I would start here instead of spamming everyone.

sysctl is installed though:

sysctl -V && which sysctl

sysctl from procps-ng 3.3.15
/sbin/sysctl

Edit: Rocky case here https://bugs.rockylinux.org/show_bug.cgi?id=176
Edited 2 years ago by wolf-allywilson

I wonder what PATH dogtag uses. It calls just 'sysctl', not a fully-qualified path. which sysctl on my RHEL 8.something hackish install returns /usr/sbin/sysctl. Theoretically /sbin should be a symlink to /usr/sbin.

It looks dogtag this is trying to determine if FIPS is enabled. As a workaround you could hack code to get past this.

The call is in/usr/lib/python3.6/site-packages/pki/__init__.py

You could add a return, as appropriate for your environment, in is_enabled() to skip the execution.

I looked at a couple of my Rocky IPA domains and the path I get is /usr/sbin/sysctl - It almost seems like something is missing or something is misconfigured to cause /usr not show up as the path on the system itself.

According to the bug report on our end, it's noted that this was happening on our 8.5 AMI but not our 8.4. I created a simple minimal VM in my lab with 8.5 and it setup just fine. I'll roll out an 8.5 AMI to do some investigation there too.

Spun up an 8.5 AMI and this was the result:

[root@ip-172-16-12-82 ipa]# uname -a
Linux testing.resf.io 4.18.0-348.el8.0.2.x86_64 #1 SMP Sun Nov 14 00:51:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
[root@ip-172-16-12-82 ipa]# cat /etc/os-release
NAME="Rocky Linux"
VERSION="8.5 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.5"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.5 (Green Obsidian)"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:rocky:rocky:8.5:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
ROCKY_SUPPORT_PRODUCT="Rocky Linux"
ROCKY_SUPPORT_PRODUCT_VERSION="8"
[root@ip-172-16-12-82 ~]# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
[root@ip-172-16-12-82 ~]# yum module enable idm:DL1/{dns,server,common}
[root@ip-172-16-12-82 ~]# clear
[root@ip-172-16-12-82 ~]# yum install ipa-server ipa-server-dns -y
[root@ip-172-16-12-82 ~]# hostnamectl set-hostname testing.ipa.resf.org
[root@ip-172-16-12-82 ~]# ipa-server-install --setup-dns \
  --no-reverse \
  --domain ipa.resf.org \
  --realm IPA.RESF.ORG \
  --hostname testing.ipa.resf.org \
  --admin-password "ThisIsATest1!" \
  --ds-password "ThisIsATest1!" \
  --no-forwarders \
  --no-ntp \
  --unattended

...

==============================================================================
Setup complete

Next steps:
        1. You must make sure these network ports are open:
                TCP Ports:
                  * 80, 443: HTTP/HTTPS
                  * 389, 636: LDAP/LDAPS
                  * 88, 464: kerberos
                  * 53: bind
                UDP Ports:
                  * 88, 464: kerberos
                  * 53: bind

        2. You can now obtain a kerberos ticket using the command: 'kinit admin'
           This ticket will allow you to use the IPA tools (e.g., ipa user-add)
           and the web user interface.

Be sure to back up the CA certificates stored in /root/cacert.p12
These files are required to create replicas. The password for these
files is the Directory Manager password
The ipa-server-install command was successful

The installation worked fine for me on an 8.5 AMI of ours.

Is there anything extra that you could perhaps be doing when rolling out your system? Or installing something else before hand?

Thank you for all your help with this, it's much appreciated.

I've followed your steps, and can confirm it installs correctly.

This led me to believe it was what we were doing in our cloudformation userdata, specifically we run this:

alternatives --set python /usr/bin/python2

Which I thought was the culprit. However, after setting that to be python3 after cloudformation has finished hasn't fixed the issue. What is even stranger though, is that after it has failed the installation, if I then uninstall ipa-server and re-run the installation it works. It's driving me up the wall.

Here is the entire userdata commands we run:

dnf -y install python2 python3 unzip perf jq at bind-utils openldap-clients nmap
systemctl enable --now atd.service
alternatives --set python /usr/bin/python2
curl https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz -o /tmp/aws-cfn-bootstrap.tar.gz
cd /tmp
tar xzvf aws-cfn-bootstrap.tar.gz
cd aws-cfn-bootstrap-*
python2 setup.py build
python2 setup.py install
mkdir -p /opt/aws/bin
ln -s /usr/init/redhat/cfn-hup /etc/init.d/cfn-hup
chmod 775 /usr/init/redhat/cfn-hup
ln -s /usr/bin/cfn-hup /opt/aws/bin/cfn-hup
ln -s /usr/bin/cfn-signal /opt/aws/bin/cfn-signal
ln -s /usr/bin/cfn-init /opt/aws/bin/cfn-init
ln -s /usr/bin/cfn-get-metadata /opt/aws/bin/cfn-get-metadata
ln -s /usr/bin/cfn-send-cmd-event /opt/aws/bin/cfn-send-cmd-event
ln -s /usr/bin/cfn-send-cmd-result /opt/aws/bin/cfn-send-cmd-result
curl https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o /tmp/awscliv2.zip
cd /tmp
unzip -q awscliv2.zip
cd aws
./install
cd /tmp
rm -rf /tmp/aws\
dnf -y install https://s3.eu-west-1.amazonaws.com/amazoncloudwatch-agent-eu-west-1/centos/amd64/latest/amazon-cloudwatch-agent.rpm
echo "$(date --date='4 minutes' +%M) * * * * /bin/bash /opt/cwSetup.sh" | crontab -
echo "$(date --date='3 minutes' +%M) * * * * /bin/bash /opt/IdentityManagementServerScript.sh" | crontab -

The IdentityManagementServerScript.sh installs ipa-server. I still believe the issue is the python alias, but I'm drawing a blank as to why it would work the 2nd time it is run (the python alias is reverted to python3 at the beginning of the install script).

Edited 2 years ago by wolf-allywilson

Fixed. Updated the Rocky bug with how.

TL:DR: I was using cron to schedule the script to install ipa-server and cron does not have the same PATH.

Metadata Update from @wolf-allywilson:
- Issue close_status updated to: worksforme
- Issue status updated to: Closed (was: Open)
2 years ago

Login to comment on this ticket.

Metadata
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.