dogtagpki

Clone
Source Code
GIT

#3093 HSM support: sslserver uses wrong token
Closed: migrated 3 years ago by dmoluguw. Opened 5 years ago by cheimes.

Closed: migrated
Reopen Issue

There seems to be a regression in Dogtag 10.6. My IPA patch for HSM support and Dogtag with SoftHSM2 has been failing for a while. After further investigation it turns out that Dogtag misconfigures the token for the sslserver cert/key pair. It is explicitly configured to use the internal token, but the server ends up to use the HSM token instead.

pki-base-10.6.9-1.fc29.noarch
pki-base-java-10.6.9-1.fc29.noarch
pki-ca-10.6.9-1.fc29.noarch
pki-kra-10.6.9-1.fc29.noarch
pki-server-10.6.9-1.fc29.noarch
pki-symkey-10.6.9-1.fc29.x86_64
pki-tools-10.6.9-1.fc29.x86_64

pkispawn configuration

See ipa-server installation log

The config file for pkispawn has pki_sslserver_token = internal in the CA section:

[CA]
...
pki_ssl_server_token = internal
pki_sslserver_key_algorithm = SHA256withRSA
pki_sslserver_key_size = 2048
pki_sslserver_key_type = rsa
pki_sslserver_nickname = Server-Cert cert-pki-ca
pki_sslserver_subject_dn = cn=master.ipa.test,O=IPA.TEST
pki_sslserver_token = internal
...

Note I added pki_ssl_server_token for debugging. The absence or presence of pki_ssl_server_token doesn't affect the outcome.

pki-ca-spawn logs

pki_sslserver_token is None instead of internal

see pki-ca-spawn log

...
    'pki_ssl_server_key_algorithm': 'SHA256withRSA',
    'pki_ssl_server_key_size': '2048',
    'pki_ssl_server_key_type': 'rsa',
    'pki_ssl_server_nickname': 'Server-Cert cert-pki-tomcat',
    'pki_ssl_server_subject_dn': 'cn=master.ipa.test,ou=pki-tomcat,o=IPA',
    'pki_ssl_server_token': 'internal',
    'pki_sslserver_cert_path': '',
    'pki_sslserver_csr_path': '',
    'pki_sslserver_key_algorithm': 'SHA256withRSA',
    'pki_sslserver_key_size': '2048',
    'pki_sslserver_key_type': 'rsa',
    'pki_sslserver_nickname': 'Server-Cert cert-pki-ca',
    'pki_sslserver_subject_dn': 'cn=master.ipa.test,O=IPA.TEST',
    'pki_sslserver_tag': 'sslserver',
    'pki_sslserver_token': None,
...

CA debug

Finally the installation fails because Tomcat uses the wrong token for the server slot:

see ca debug log

2019-01-31 12:01:39 [main] FINE: CertUtils: verifySystemCerts() cert tag=sslserver
2019-01-31 12:01:39 [main] FINE: CertUtils: verifySystemCertByTag(sslserver)
2019-01-31 12:01:39 [main] FINE: Certutils.verifySystemCertValidityByNickname: (softhsm_token:Server-Cert cert-pki-ca)
2019-01-31 12:01:39 [main] SEVERE: Certutils.verifySystemCertValidityByNickname:  failed : Certificate not found: softhsm_token:Server-Cert cert-pki-ca
org.mozilla.jss.crypto.ObjectNotFoundException: Certificate not found: softhsm_token:Server-Cert cert-pki-ca
    at org.mozilla.jss.CryptoManager.findCertByNicknameNative(Native Method)
    at org.mozilla.jss.CryptoManager.findCertByNickname(CryptoManager.java:830)
    at com.netscape.cmscore.cert.CertUtils.verifySystemCertValidityByNickname(CertUtils.java:828)
    at com.netscape.cmscore.cert.CertUtils.verifySystemCertByTag(CertUtils.java:973)
    at com.netscape.cmscore.cert.CertUtils.verifySystemCerts(CertUtils.java:1095)
    at com.netscape.cms.selftests.common.SystemCertsVerification.runSelfTest(SystemCertsVerification.java:205)
    at com.netscape.cmscore.selftests.SelfTestSubsystem.runSelfTestsAtStartup(SelfTestSubsystem.java:852)
    at com.netscape.cmscore.selftests.SelfTestSubsystem.startup(SelfTestSubsystem.java:1777)
    at com.netscape.cmscore.apps.CMSEngine.startupSubsystems(CMSEngine.java:1556)

The regression may be in pki commit 17677ae4d2cda456b64ec67e2b25ba63f4a58a70.
I will make some related changes and test. Hopefully be able to confirm and have a patch soon.

Ping @cheimes.

Metadata Update from @ftweedal:
- Custom field component adjusted to None
- Custom field feature adjusted to None
- Custom field origin adjusted to None
- Custom field proposedmilestone adjusted to None
- Custom field proposedpriority adjusted to None
- Custom field reviewer adjusted to None
- Custom field type adjusted to None
- Custom field version adjusted to None
5 years ago

Hm, not as clear cut as I thought. The ca debug log (thanks for attaching it @cheimes!) shows that configuration request sent to the web app has sslserver token = null:

2019-01-31 12:00:38 [https-jsse-nio-8443-exec-10] FINE: SystemConfigService: request: ConfigurationRequest [pin=XXXX, token=softhsm_token, tokenPassword=XXXX, securityDomainType=newdomain, securityDomainUri=null, securityDomainName=IPA, securityDomainUser=null, securityDomainPassword=XXXX, securityDomainPostLoginSleepSeconds=null, isClone=false, cloneUri=null, subsystemName=CA master.ipa.test 8443, p12File=null, p12Password=XXXX, hierarchy=root, dsHost=master.ipa.test, dsPort=389, baseDN=o=ipaca, bindDN=cn=Directory Manager, bindpwd=XXXX, database=ipaca, secureConn=false, removeData=true, replicateSchema=null, masterReplicationPort=null, cloneReplicationPort=null, replicationSecurity=null, systemCertsImported=false, systemCerts=[SystemCertData[tag=signing, nickname=caSigningCert cert-pki-ca, token=softhsm_token, keyType=rsa, keyAlgorithm=SHA256withRSA, signingAlgorithm=SHA256withRSA, keySize=2048, keyCurveName=null, request=null, subjectDN=CN=Certificate Authority,O=IPA.TEST, cert=null, req_ext_oid=null, req_ext_critical=null, req_ext_data=null, san_for_server_cert=null], SystemCertData[tag=ocsp_signing, nickname=ocspSigningCert cert-pki-ca, token=softhsm_token, keyType=rsa, keyAlgorithm=SHA256withRSA, signingAlgorithm=SHA256withRSA, keySize=2048, keyCurveName=null, request=null, subjectDN=cn=OCSP Subsystem,O=IPA.TEST, cert=null, req_ext_oid=null, req_ext_critical=null, req_ext_data=null, san_for_server_cert=null], SystemCertData[tag=sslserver, nickname=Server-Cert cert-pki-ca, token=null, keyType=rsa, keyAlgorithm=SHA256withRSA, signingAlgorithm=null, keySize=2048, keyCurveName=null, request=null, subjectDN=cn=master.ipa.test,O=IPA.TEST, cert=null, req_ext_oid=null, req_ext_critical=null, req_ext_data=null, san_for_server_cert=null], SystemCertData[tag=subsystem, nickname=subsystemCert cert-pki-ca, token=softhsm_token, keyType=rsa, keyAlgorithm=SHA256withRSA, signingAlgorithm=null, keySize=2048, keyCurveName=null, request=null, subjectDN=cn=CA Subsystem,O=IPA.TEST, cert=null, req_ext_oid=null, req_ext_critical=null, req_ext_data=null, san_for_server_cert=null], SystemCertData[tag=audit_signing, nickname=auditSigningCert cert-pki-ca, token=softhsm_token, keyType=rsa, keyAlgorithm=SHA256withRSA, signingAlgorithm=SHA256withRSA, keySize=2048, keyCurveName=null, request=null, subjectDN=cn=CA Audit,O=IPA.TEST, cert=null, req_ext_oid=null, req_ext_critical=null, req_ext_data=null, san_for_server_cert=null]], issuingCA=null, generateServerCert=true, external=false, standAlone=false, authdbBaseDN=null, authdbHost=null, authdbPort=null, authdbSecureConn=null, caUri=null, kraUri=null, tksUri=null, enableServerSideKeyGen=null, importSharedSecret=null, generateSubsystemCert=true, sharedDB=false, sharedDBUserDN=null, createNewDB=true, setupReplication=null, subordinateSecurityDomainName=null, reindexData=null, startingCrlNumber=0, createSigningCertRecord=true, signingCertSerialNumber=1]

This is legitimate (null implies internal), so the bug must be on the Java side.

Thanks Fraser!

  • The Python function normalize_token() turns 'internal' into None
  • Something in Java interprets sslserver token NULL as default token instead of internal token.

The interesting thing about the snippet of logs posted above:

2019-01-31 12:01:39 [main] FINE: Certutils.verifySystemCertValidityByNickname: (softhsm_token:Server-Cert cert-pki-ca)
2019-01-31 12:01:39 [main] SEVERE: Certutils.verifySystemCertValidityByNickname:  failed : Certificate not found: softhsm_token:Server-Cert cert-pki-ca

Is that verifySystemCertValidityByNickname is passed the ca.cert.sslserver.nickname value from /etc/pki/pki-tomcat/ca/CS.cfg.

This, to me, implies that pkispawn sets the nickname incorrectly and that it should be on the default token, not the softhsm token.

@ftweedal pointed out this line earlier:

2019-01-31 12:00:38 [https-jsse-nio-8443-exec-10] FINE: SystemConfigService: request: ConfigurationRequest [pin=XXXX, token=softhsm_token, tokenPassword=XXXX, securityDomainType=newdomain, securityDomainUri=null, securityDomainName=IPA, securityDomainUser=null, securityDomainPassword=XXXX, securityDomainPostLoginSleepSeconds=null, isClone=false, cloneUri=null, subsystemName=CA master.ipa.test 8443, p12File=null, p12Password=XXXX, hierarchy=root, dsHost=master.ipa.test, dsPort=389, baseDN=o=ipaca, bindDN=cn=Directory Manager, bindpwd=XXXX, database=ipaca, secureConn=false, removeData=true, replicateSchema=null, masterReplicationPort=null, cloneReplicationPort=null, replicationSecurity=null, systemCertsImported=false, systemCerts=[SystemCertData[tag=signing, nickname=caSigningCert cert-pki-ca, token=softhsm_token, keyType=rsa, keyAlgorithm=SHA256withRSA, signingAlgorithm=SHA256withRSA, keySize=2048, keyCurveName=null, request=null, subjectDN=CN=Certificate Authority,O=IPA.TEST, cert=null, req_ext_oid=null, req_ext_critical=null, req_ext_data=null, san_for_server_cert=null], SystemCertData[tag=ocsp_signing, nickname=ocspSigningCert cert-pki-ca, token=softhsm_token, keyType=rsa, keyAlgorithm=SHA256withRSA, signingAlgorithm=SHA256withRSA, keySize=2048, keyCurveName=null, request=null, subjectDN=cn=OCSP Subsystem,O=IPA.TEST, cert=null, req_ext_oid=null, req_ext_critical=null, req_ext_data=null, san_for_server_cert=null], SystemCertData[tag=sslserver, nickname=Server-Cert cert-pki-ca, token=null, keyType=rsa, keyAlgorithm=SHA256withRSA, signingAlgorithm=null, keySize=2048, keyCurveName=null, request=null, subjectDN=cn=master.ipa.test,O=IPA.TEST, cert=null, req_ext_oid=null, req_ext_critical=null, req_ext_data=null, san_for_server_cert=null], SystemCertData[tag=subsystem, nickname=subsystemCert cert-pki-ca, token=softhsm_token, keyType=rsa, keyAlgorithm=SHA256withRSA, signingAlgorithm=null, keySize=2048, keyCurveName=null, request=null, subjectDN=cn=CA Subsystem,O=IPA.TEST, cert=null, req_ext_oid=null, req_ext_critical=null, req_ext_data=null, san_for_server_cert=null], SystemCertData[tag=audit_signing, nickname=auditSigningCert cert-pki-ca, token=softhsm_token, keyType=rsa, keyAlgorithm=SHA256withRSA, signingAlgorithm=SHA256withRSA, keySize=2048, keyCurveName=null, request=null, subjectDN=cn=CA Audit,O=IPA.TEST, cert=null, req_ext_oid=null, req_ext_critical=null, req_ext_data=null, san_for_server_cert=null]], issuingCA=null, generateServerCert=true, external=false, standAlone=false, authdbBaseDN=null, authdbHost=null, authdbPort=null, authdbSecureConn=null, caUri=null, kraUri=null, tksUri=null, enableServerSideKeyGen=null, importSharedSecret=null, generateSubsystemCert=true, sharedDB=false, sharedDBUserDN=null, createNewDB=true, setupReplication=null, subordinateSecurityDomainName=null, reindexData=null, startingCrlNumber=0, createSigningCertRecord=true, signingCertSerialNumber=1]
2019-01-31 12:00:38 [https-jsse-nio-8443-exec-10] FINE: === Token Configuration ===
2019-01-31 12:00:38 [https-jsse-nio-8443-exec-10] FINE: Setting preop.module.token=softhsm_token

@ftweedal is correct in that it has sslserver's token as null, but the next two lines in the log show that's where softhsm token is used: we assume there's one token for the entire request, versus changing it per-cert object. In particular, this assumption fails where, if cert's token is null, use the request's token instead.

This results in the following in the log:

2019-01-31 12:01:23 [https-jsse-nio-8443-exec-10] FINE: === Processing sslserver cert ===
2019-01-31 12:01:23 [https-jsse-nio-8443-exec-10] FINE: SystemConfigService.processKeyPair(sslserver)
2019-01-31 12:01:23 [https-jsse-nio-8443-exec-10] FINE: SystemConfigService: token: softhsm_token

Perhaps we just need to quit passing around internal as None...

Edited 5 years ago by cipherboy

I have a patch for this: https://github.com/frasertweedale/pki/tree/fix/3093-configuration-token-normalisation. I'll ask Christian to test it before creating a PR.

I have tested the patch, but if I understand everything correct, is does not fix the problem.
The Java code get correct token name, but not the pkispawn python job.

Getting the sslserver certificate right, i.e. in internal, is a blocking issue for supporting HSM, because dogtag will not work when it is in HSM, so it must be in internal/local.

sslserver cert inserted in token 'Dogtag'
$ less /var/log/pki/pki-ca-spawn.20190808212339.log
...
2019-08-08 21:44:23 configuration : INFO Removing temp SSL server cert from internal token: sslserver
2019-08-08 21:44:23 configuration : INFO Importing permanent SSL server cert into Dogtag token: sslserver
...

sslserver cert inserted in token '' (internal/None)
$ less /var/log/pki/pki-tomcat/ca/debug.2019-08-08.log
...
java.lang.Exception: Certutils.verifySystemCertValidityByNickname: faliled: nickname: sslservercause: org.mozilla.jss.crypto.ObjectNotFoundException: Certificate not found: sslserver
...

dogtag-10.7.0.tar.gz
Edited 4 years ago by magnuskkarlsson

I have documented all installation steps I took for verification here https://magnus-k-karlsson.blogspot.com/2019/08/installing-dogtag-on-fedora-30-with.html

To get pass this bug I hardcoded the sslserver to only use internal token, this is not pretty, but we need to have the sslserver internally to get any HSM to work. Dogtag will not work with having the sslserver on HSM.

$ vi /usr/lib/python3.7/site-packages/pki/server/deployment/scriptlets/configuration.py
...
def import_perm_sslserver_cert(self, deployer, instance, cert):
...
# BUG FIX hardcoded value
token = pki.nssdb.INTERNAL_TOKEN_NAME
...

And also having the sslserver only internally is better for performance and I don't think that is a security problem, since if you were able to get hold of the sslserver one intruder is already inside the CA and can do more harm than stealing the sslserver for Man in the Middle Attacks. But these are only mine thoughts.

@magnuskkarlsson this is a great help, thank you so much. I should be able to make progress on this, and will work on it later this week.

Metadata Update from @cheimes:
- Custom field component adjusted to IPA Integration (was: None)
- Issue tagged with: HSM
4 years ago

Dogtag PKI is moving from Pagure issues to GitHub issues. This means that existing or new
issues will be reported and tracked through Dogtag PKI's GitHub Issue tracker.

This issue has been cloned to GitHub and is available here:
https://github.com/dogtagpki/pki/issues/3210

If you want to receive further updates on the issue, please navigate to the
GitHub issue and click on Subscribe button.

Thank you for understanding, and we apologize for any inconvenience.

Metadata Update from @dmoluguw:
- Issue close_status updated to: migrated
- Issue status updated to: Closed (was: Open)
3 years ago

Login to comment on this ticket.

Metadata
None

HSM

None
None
major
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
IPA Integration
None
Attachments 1
dogtag-10.7.0.tar.gz
Attached 4 years ago View Comment
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.