#6540 Cannot ssh into koschei-stg
Closed: Fixed Opened by msimacek.

After koschei-{backend,web}-stg machines were respawned as Fedora 27, I cannot ssh there anymore.

msimacek ~ $ ssh -v koschei-backend-stg
OpenSSH_7.5p1, OpenSSL 1.1.0g-fips  2 Nov 2017
debug1: Reading configuration data /home/msimacek/.ssh/config
debug1: /home/msimacek/.ssh/config line 57: Applying options for koschei-backend-stg
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/05-redhat.conf
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug1: /etc/ssh/ssh_config.d/05-redhat.conf line 8: Applying options for *
debug1: Executing proxy command: exec ssh -W koschei-backend01.stg.phx2.fedoraproject.org:22 bastion.fedoraproject.org
debug1: permanently_drop_suid: 1000
debug1: identity file /home/msimacek/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/msimacek/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/msimacek/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/msimacek/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/msimacek/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/msimacek/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/msimacek/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/msimacek/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.5
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.5
debug1: match: OpenSSH_7.5 pat OpenSSH* compat 0x04000000
debug1: Authenticating to koschei-backend01.stg.phx2.fedoraproject.org:22 as 'msimacek'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ssh-rsa-cert-v01@openssh.com
debug1: kex: server->client cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256@libssh.org need=32 dh_need=32
debug1: kex: curve25519-sha256@libssh.org need=32 dh_need=32
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host certificate: ssh-rsa-cert-v01@openssh.com SHA256:mbRtw8J0G9UEdav2oI9C97XFzPF6h+DJuEa86bL1lHw, serial 1512067161 ID "koschei-backend01.stg.phx2.fedoraproject.org" CA ssh-rsa SHA256:pmw8O+j5VVfgpVIjMvWG+u7DizO2dyzrvxHWrLUg02s valid from 2017-11-30T19:39:21 to 2018-11-29T20:39:21
debug1: No matching CA found. Retry with plain key
debug1: Host 'koschei-backend01.stg.phx2.fedoraproject.org' is known and matches the RSA host key.
debug1: Found key in /home/msimacek/.ssh/known_hosts:60
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,null>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/msimacek/.ssh/id_rsa
debug1: Server accepts key: pkalg rsa-sha2-512 blen 279
Authentication failed.

It used to work and connecting to prod still works, so it shouldn't be misconfiguration on my side.

CC @mizdebsk


Oh yeah, this is due to:
https://bugzilla.redhat.com/show_bug.cgi?id=1513704
(selinux policy issue with f27 and nssdb).

I have put them both in permissve mode now, so you should be able to login.

Hopefully they will fix the above soon...

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

It started to happen again after reboot (part of phx2 maintenance), so I've made the SELinux workaround persistent. I hope that's fine.

Thats fine. If we don't get a real fix soon we can push our own selinux policy for it.

Metadata