Don't bother storing subsets of the pubkey info
    
    We used to hang on to just the public key, but then we started caching
    the entire PublicKeyInfo structure, which includes that key, so no point
    in hanging on to just the key.