Learn more about these different git repos.
Other Git URLs
I have a Dogtag setup that has a root CA with a subordinate CA.
auths.instance.flatFileAuth.deferOnFailure=false
ca.scep.allowedEncryptionAlgorithms=DES,DES3
ca.scep.allowedHashAlgorithms=MD5,SHA1,SHA256,SHA384,SHA512
ca.scep.enable=true
ca.scep.encryptionAlgorithm=DES
ca.scep.hashAlgorithm=MD5
Note: The algorithms are set as such due to #89
Certmonger is tested using the following steps:
getcert add-scep-ca -c Test -u https://$HOSTNAME:8443/ca/cgi-bin/pkiclient.exe -R /etc/pki/site-pki.pem
getcert request -c Test -k /etc/pki/my_cert.pem -f /etc/pki/my_cert.pub -I Host_Cert -R -w -L password
I've run certmonger from the command line to attempt to obtain debug information using /usr/libexec/certmonger/certmonger-session -n -F -d 100 -S 2>&1.
certmonger
/usr/libexec/certmonger/certmonger-session -n -F -d 100 -S 2>&1
The output from the test is as follows:
2018-02-02 14:55:45 [10281] Redirecting stdin and stderr to /dev/null, leaving stdout open for child "/usr/libexec/certmonger/scep-submit". 2018-02-02 14:55:45 [10105] Request1('Host_Cert') moved to state 'SUBMITTING' 2018-02-02 14:55:45 [10105] Will revisit Request1('Host_Cert') on traffic from 22. 2018-02-02 14:55:46 [10105] Certificate submission still ongoing. 2018-02-02 14:55:46 [10105] Will revisit Request1('Host_Cert') on traffic from 22. 2018-02-02 14:55:46 [10105] Certificate submission attempt complete. 2018-02-02 14:55:46 [10105] Child status = 3. 2018-02-02 14:55:46 [10105] Child output: "Error: failed to verify signature on server response. " 2018-02-02 14:55:46 [10105] Certificate not (yet?) issued. 2018-02-02 14:55:46 [10105] Request1('Host_Cert') moved to state 'CA_UNREACHABLE' 2018-02-02 14:55:46 [10105] Will revisit Request1('Host_Cert') in 604800 seconds.
I subsequently added printf statements to the code to determine exactly where the failure was happening and it resulted in isolation at https://pagure.io/certmonger/blob/master/f/src/pkcs7.c#_1065 (I don't know why the debug message there didn't print).
printf
I can patch the certmonger code to try different things but have been unable to determine how to successfully print the server response.
I have determined that the Dogtag installation is correct by using the sscep command to obtain certificates from the same CA.
sscep
The sscep.conf file is as follows:
sscep.conf
CACertFile=ca.crt CertReqFile=local.csr EncAlgorithm=des LocalCertFile=local.crt PrivateKeyFile=local.key SigAlgorithm=md5 URL=http://${HOSTNAME}:8080/ca/cgi-bin/pkiclient.exe
Using that config file:
sscep getca -f sscep.conf mkrequest -dns $HOSTNAME password sscep enroll -f sscep.conf
This results in the following output (and a valid, signed certificate):
sscep: Found private key local.key as file. If the engine can handle it, loading the file sscep: sending certificate request sscep: valid response from server sscep: reply transaction id: AA0FE652952D0FA691CF6CAAF656A520 sscep: pkistatus: SUCCESS
Thank you for the terrific reproduction steps. I've made a bit of progress in that I was able to get IPA to issue SCEP request. I duplicated the issue without having to create the subCA.
What I think is happening is certs is empty when calling PKCS7_verify(). The CA cert is only in the store variable which isn't searched for the signing cert (only for the chain to verify said cert in my reading of the man page).
I hacked pkcs7.c to insert a copy of roots into the certs variable and was able to successfully issue a cert over SCEP from certmonger. I'm not yet convinced this is the right solution. I need to trace the code further up to see what the difference is between certs and othercerts to certmonger. Only othercerts would be added to the certs variable in the call to PKCS7_verify() and in my case othercerts is empty. I suspect something is reversed further up the chain.
So basically the reason it couldn't verify the signer is it had no certs to verify with.
@rcritten Thanks for the update.
I noticed that, in the CA configuration saved by certmonger, there is a ca_encryption_issuer_cert and a ca_encryption_cert_pool variable pair that appear to have the certificates from my CAs in them.
ca_encryption_issuer_cert
ca_encryption_cert_pool
I'm not sure if this would make a difference in the root vs. sub-CA setup and your debugging.
I assume you are using the subCA with the -R command? Can you create another certmonger CA ( add-scep-ca) but use the main IPA CA instead?
@rcritten Will that work without an IPA server on the other end? I'm running pure Dogtag.
Sure I should have just said try the root CA to see if that helps.
I'd have to re-roll the setup. I don't add a KRA to my root CA since it's not supposed to be used for automatic key enrollment. Might not be able to get to it for a few days.
I mean just the root signing cert. You shouldn't need any additional services.
Tried the following with no difference: getcert add-scep-ca -c Test -u https://$HOSTNAME:8443/ca/cgi-bin/pkiclient.exe -R /etc/pki/site-pki.pem -r /etc/pki/site-pki.pem
getcert add-scep-ca -c Test -u https://$HOSTNAME:8443/ca/cgi-bin/pkiclient.exe -R /etc/pki/site-pki.pem -r /etc/pki/site-pki.pem
/etc/pki/site-pki.pem
I think I've tried all combinations of -r, -R, and -I with no luck. This includes using the entire certificate chain for -I and different combinations of the root and sub-CA certificates.
-r
-R
-I
Ok. In trying to get more information for you I discovered that what I thought were individual successful SCEP requests were in fact just retrieving the original cert I was issued (even if I revoked it). Continuing to look.
A bit of debugging advice.
Ensure you SCEP request is cleared out of certmonger. Remove the requesting key and cert (though there probably isn't one) and stop the existing tracking.
stop certmonger
strace -fF -s 4096 -o /tmp/out certmonger -d 9 -n
Submit your cert tracking request
In /tmp/out you'll be able to see the stdout messages that scep-submit is printing. This includes the PKCS#7 blog that is returned. Look for response_code =
If it is 200 (and it probably is) then you'll see other status messages and below that a results = blog which is the PKCS#7 message (just without the header/footer).
scep-submit writes its log messages to stderr.
So, this is what I'm seeing:
response_code
500
I haven't really duplicated your environment. I'm testing against a self-signed CA in an IPA environment.
Passing the IPA CA cert using -I is working for me.
Since you have already looked at instrumenting the code, this should do it to get the raw PKCS#7 signed data out:
diff --git a/src/pkcs7.c b/src/pkcs7.c index 991ef91d..864d37f9 100644 --- a/src/pkcs7.c +++ b/src/pkcs7.c @@ -911,6 +911,7 @@ cm_pkcs7_verify_signed(unsigned char *data, size_t l ength, char *s, buf[LINE_MAX], *p, *q; int ret = -1, i; long error; + FILE *fp; if (digest != NULL) { *digest = NULL; @@ -946,6 +947,9 @@ cm_pkcs7_verify_signed(unsigned char *data, size_t l ength, *payload_length = 0; } u = data; + fp = creat("/tmp/pkcs7.der", 0644); + write(fp, data, length); + close(fp); p7 = d2i_PKCS7(NULL, &u, length); if ((p7 == NULL) || (u != data + length)) { cm_log(1, "Error parsing what should be PKCS#7 signed-data.\n");
You can see who the signer is with:
# openssl asn1parse -inform der -in /tmp/pkcs7.der
@rcritten I can give you steps to spin up a full reproducer environment if you have somewhere you can run RVM and Vagrant. Let me know if you want me to post the steps.
Part of the key here is that I am not using IPA, just straight up Dogtag.
@rcritten Applied the patch that you supplied above and the code was never executed (the file was not created).
@rcritten I moved the block into https://pagure.io/certmonger/blob/master/f/src/pkcs7.c#_1065 and dumped the offending PKCS7 data.
Reading this in via the openssl command that you provided does not show any issues whatsoever.
openssl
It is a decodable PKCS#7 file but it should show you the signer of the blob which I hope will bring a eureka moment.
@rcritten The signer was just fine and as I expected.
What was the signer? The root CA or the subordinate CA? Can you show what the current dogtag_submit CA looks like?
If you could provide reproduction steps that would help a lot. I should mention that while I'm using an IPA-provisioned CA using SCEP talks directly to the CA and avoids all the IPA stuff. The real difference is the intermediate I think.
The signer was the root but it doesn't seem to matter.
The passwords are auto-generated garbage so don't let that worry you.
[DEFAULT] pki_instance_name=simp-pki-root pki_security_domain_name=SIMP pki_ds_bind_dn=cn=SIMP Directory Manager pki_ds_database=simp-pki-root-ca pki_ds_password=nEQ4pMEXY3TiQyRFzB46gYrUSTW148gOeXquLd4z20tXDBS7VSDIGcvEEXOqne3j [CA] pki_ds_hostname=127.0.0.1 pki_client_database_purge=False pki_client_database_password=mM8mpY0XLuf7VPPIkN1iV8nVULpoWQCYSOkY3enO08K5ciC2FcHEP3VNAokNzaGI pki_client_pkcs12_password=mM8mpY0XLuf7VPPIkN1iV8nVULpoWQCYSOkY3enO08K5ciC2FcHEP3VNAokNzaGI pki_http_port=4508 pki_https_port=4509 pki_admin_email=caadmin@ca.int.localdomain pki_admin_name=caadmin pki_admin_nickname=caadmin pki_admin_password=mM8mpY0XLuf7VPPIkN1iV8nVULpoWQCYSOkY3enO08K5ciC2FcHEP3VNAokNzaGI pki_admin_uid=caadmin
[DEFAULT] pki_instance_name=simp-site-pki pki_security_domain_name=SIMP pki_security_domain_hostname=ca.int.localdomain pki_security_domain_https_port=4509 pki_security_domain_user=caadmin pki_security_domain_password=mM8mpY0XLuf7VPPIkN1iV8nVULpoWQCYSOkY3enO08K5ciC2FcHEP3VNAokNzaGI pki_ds_bind_dn=cn=SIMP Directory Manager pki_ds_database=simp-site-pki-ca pki_ds_password=nEQ4pMEXY3TiQyRFzB46gYrUSTW148gOeXquLd4z20tXDBS7VSDIGcvEEXOqne3j [CA] pki_ds_hostname=127.0.0.1 pki_client_database_purge=False pki_client_database_password=lroWsjRudFF2NAO5PIpyWizT6xjaFlbsJISqUnhdtKKdSXpVazRYdinp30yIL8IW pki_client_pkcs12_password=lroWsjRudFF2NAO5PIpyWizT6xjaFlbsJISqUnhdtKKdSXpVazRYdinp30yIL8IW pki_subordinate=True pki_issuing_ca_hostname=ca.int.localdomain pki_issuing_ca_https_port=4509 pki_http_port=8080 pki_https_port=8443 pki_admin_email=caadmin@ca.int.localdomain pki_admin_name=caadmin pki_admin_nickname=caadmin pki_admin_password=lroWsjRudFF2NAO5PIpyWizT6xjaFlbsJISqUnhdtKKdSXpVazRYdinp30yIL8IW pki_admin_uid=caadmin [Tomcat] pki_ajp_port=8440 pki_tomcat_server_port=8441
Reproduction steps:
rvm install 2.4
rvm use 2.4 --default
git clone https://github.com/simp/simp_pki_service
cd simp_pki_service
bundle update
BEAKER_destroy=no rake beaker:suites
cd .vagrant/beaker_vagrant_files/default.yml
vagrant ssh ca
sudo su -
You can then do what you need to do.
Run vagrant destroy -f in the directory with the Vagrantfile to destroy the system.
vagrant destroy -f
Vagrantfile
Thanks for the steps. I'll try to reproduce.
I can't get it to install. I get a slew of ruby/gem errors and it fails to even try starting the vagrant images:
=== Suite 'default' Starting ===
/usr/local/rvm/rubies/ruby-2.4.1/bin/ruby -I/usr/local/rvm/gems/ruby-2.4.1/gems/rspec-core-3.7.1/lib:/usr/local/rvm/gems/ruby-2.4.1/gems/rspec-support-3.7.1/lib /usr/local/rvm/gems/ruby-2.4.1/gems/rspec-core-3.7.1/exe/rspec spec/acceptance/suites/default --color /usr/local/rvm/gems/ruby-2.4.1/gems/beaker-rspec-6.2.3/lib/beaker-rspec/helpers/serverspec.rb:43: warning: already initialized constant Module::VALID_OPTIONS_KEYS /usr/local/rvm/gems/ruby-2.4.1/gems/specinfra-2.67.3/lib/specinfra/configuration.rb:4: warning: previous definition of VALID_OPTIONS_KEYS was here Hypervisor for ca is vagrant Beaker::Hypervisor, found some vagrant boxes to create created Vagrantfile for VagrantHost ca
An error occurred while loading ./spec/acceptance/suites/default/00_default_spec.rb. Failure/Error: require 'beaker-rspec' RuntimeError: Failed to exec 'vagrant up'. Error was Ignoring ffi-1.9.18 because its extensions are not built. Try: gem pristine ffi --version 1.9.18 Ignoring json-2.1.0 because its extensions are not built. Try: gem pristine json --version 2.1.0 Ignoring unf_ext-0.0.7.4 because its extensions are not built. Try: gem pristine unf_ext --version 0.0.7.4 /usr/local/rvm/rubies/ruby-2.4.1/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in require': cannot load such file -- bundler/setup (LoadError) from /usr/local/rvm/rubies/ruby-2.4.1/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:inrequire'
require': cannot load such file -- bundler/setup (LoadError) from /usr/local/rvm/rubies/ruby-2.4.1/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in
and a ton more...
Note that the first go-around I did a gem install of each of these packages and no errors were reported and apparently they were installed.
@rcritten My guess is that you are using the EPEL vagrant, not the one from vagrantup.com. Try removing all Vagrants and then installing from vagrantup.com.
Also, try running as BEAKER_destroy=no bundle exec rake beaker:suites
BEAKER_destroy=no bundle exec rake beaker:suites
Ok, vagrant-2.0.2 is required. I was doing this on F-27 with 2.0.1. I have the image up.
One more question. Your initial report refers to /etc/pki/site-pki.pem. I don't have that on my system. I can certainly fetch the CA certificates but I'd like to reproduce as closely as possible to what you are seeing. Which certificate is here: the root, intermediate or both?
I realize you said you'd tried all combinations but again, just trying to start from the point you did and got from there.
@rcritten The VM is a base test harness that you can use the script that I posted before to test.
You'll have to extract the server PKI keys following the steps outlined here: https://github.com/simp/pupmod-simp-simp_pki_service#client-setup
Then you can use it the usual way.
I think I've got the setup done. I'm at least reproducing a similar looking failure.
Can you look at your pem files and see if there are headers like:
Bag Attributes 2.16.840.1.113730.5.1.1.1: CT,C,C localKeyID: 00 9B 92 61 B3 05 7F EE 42 9B 6A AF DE 5B 08 FC D2 E7 35 09 15 friendlyName: CA Signing Certificate - SIMP subject=/O=SIMP/OU=simp-pki-root/CN=CA Signing Certificate issuer=/O=SIMP/OU=simp-pki-root/CN=CA Signing Certificate
Please try stripping that out. This worked for me:
# getcert add-scep-ca -c Test -u https://hostname:8443/ca/cgi-bin/pkiclient.exe -I /etc/pki/site-sub.pem -v -R /etc/pki/site-pki.pem # getcert request -c Test -k /etc/pki/tls/private/test.key -f /etc/pki/tls/certs/test.pem -w -L 1234
hostname
Number of certificates and requests being tracked: 1. Request ID '20180301180935': status: MONITORING stuck: no key pair storage: type=FILE,location='/etc/pki/tls/private/test.key' certificate: type=FILE,location='/etc/pki/tls/certs/test.pem' signing request thumbprint (MD5): BC7AE9AD 6359D78A 31A6BB24 99D4727A signing request thumbprint (SHA1): 14A4E587 666E1626 52C570E8 C6E8C6D4 6287B675 CA: Test issuer: CN=CA Signing Certificate,OU=simp-site-pki,O=SIMP subject: CN=ca.example.com expires: 2020-02-13 21:19:42 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment eku: id-kp-clientAuth,id-kp-emailProtection pre-save command: post-save command: track: yes auto-renew: yes
Yeah, I'll give it a shot
@rcritten That did indeed work!
I'm surprised that the sscep code worked and this didn't since they appear to both call the same openssl routines.
Anyway, if this could be updated in the documentation that would be really helpful!
Thanks for helping with the debugging!
Thanks for confirming it works for you too. I don't think this extraneous data should cause certmonger to fail so I'm going to try to figure out what is causing it grief but at least you have a path forward now.
sscep is a completely different codebase using Java so who knows what cert processing it uses.
@rcritten sscep is C so you can probably just snag the code https://github.com/certnanny/sscep
Ah so it is. I don't recall installing the package and thought it came as a helper from dogtag. I'll take a look.
PR to fix finding certificates in a PEM file, https://pagure.io/certmonger/pull-request/100
master: 8e47658
Metadata Update from @rcritten: - Issue assigned to rcritten
Metadata Update from @rcritten: - Issue priority set to: major - Issue set to the milestone: 0.79
Metadata Update from @rcritten: - Issue close_status updated to: fixed - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.