I help manage a handful of workstations running RHEL8/OL8 in a development environment. The devs typically access these workstations remotely via SSH. Some of the devs are encountering a strange issue where authentication via SSH (password) fails even though they have entered the correct password.
Their accounts are not locked as reported by passwd -S. We have also tried resetting the password to something simple and it doesn't seem to help. One of the devs is a privileged user with root access, they are able to SSH in as root and su to their own user account without problems, but this is obviously not ideal.
Wondering if there is any insight on what could be the cause or where else to troubleshoot. My hunch is there potentially is a security setting somewhere which may be causing this but I am not sure where to look.
sshd log with verbosity:
Jan 25 13:54:10 pc123 sshd[883298]: debug3: fd 7 is not O_NONBLOCK
Jan 25 13:54:10 pc123 sshd[883298]: debug1: Forked child 883385.
Jan 25 13:54:10 pc123 sshd[883298]: debug3: send_rexec_state: entering fd = 10 config len 745
Jan 25 13:54:10 pc123 sshd[883298]: debug3: ssh_msg_send: type 0
Jan 25 13:54:10 pc123 sshd[883298]: debug3: send_rexec_state: done
Jan 25 13:54:10 pc123 sshd[883385]: debug3: oom_adjust_restore
Jan 25 13:54:10 pc123 sshd[883385]: debug1: Set /proc/self/oom_score_adj to 0
Jan 25 13:54:10 pc123 sshd[883385]: debug1: rexec start in 7 out 7 newsock 7 pipe 9 sock 10
Jan 25 13:54:10 pc123 sshd[883385]: debug1: inetd sockets after dupping: 5, 5
Jan 25 13:54:10 pc123 sshd[883385]: Connection from 10.123.45.67 port 59029 on 10.111.22.33 port 22
Jan 25 13:54:10 pc123 sshd[883385]: debug1: Local version string SSH-2.0-OpenSSH_8.0
Jan 25 13:54:10 pc123 sshd[883385]: debug1: Remote protocol version 2.0, remote software version OpenSSH_for_Windows_8.1
Jan 25 13:54:10 pc123 sshd[883385]: debug1: match: OpenSSH_for_Windows_8.1 pat OpenSSH* compat 0x04000000
Jan 25 13:54:10 pc123 sshd[883385]: debug2: fd 5 setting O_NONBLOCK
Jan 25 13:54:10 pc123 sshd[883385]: debug3: ssh_sandbox_init: preparing seccomp filter sandbox
Jan 25 13:54:10 pc123 sshd[883385]: debug2: Network child is on pid 883386
Jan 25 13:54:10 pc123 sshd[883385]: debug3: preauth child monitor started
Jan 25 13:54:10 pc123 sshd[883385]: debug1: SELinux support disabled [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: privsep user:group 74:74 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: permanently_set_uid: 74/74 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: ssh_sandbox_child: setting PR_SET_NO_NEW_PRIVS [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: ssh_sandbox_child: attaching seccomp filter program [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: list_hostkey_types: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: send packet: type 20 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: SSH2_MSG_KEXINIT sent [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: receive packet: type 20 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: SSH2_MSG_KEXINIT received [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: local server KEXINIT proposal [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: ciphers ctos: [email protected],[email protected],aes256-ctr,aes256-cbc,[email protected],aes128-ctr,aes128-cbc [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: ciphers stoc: [email protected],[email protected],aes256-ctr,aes256-cbc,[email protected],aes128-ctr,aes128-cbc [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha1,[email protected],hmac-sha2-512 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha1,[email protected],hmac-sha2-512 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: compression ctos: none,[email protected] [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: compression stoc: none,[email protected] [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: languages ctos: [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: languages stoc: [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: first_kex_follows 0 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: reserved 0 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: peer client KEXINIT proposal [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: host key algorithms: [email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],[email protected],[email protected],ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected] [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: compression ctos: none,[email protected],zlib [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: compression stoc: none,[email protected],zlib [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: languages ctos: [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: languages stoc: [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: first_kex_follows 0 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: reserved 0 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: kex: algorithm: curve25519-sha256 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: kex: host key algorithm: ecdsa-sha2-nistp256 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: kex: curve25519-sha256 need=64 dh_need=64 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_send entering: type 120 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive_expect entering: type 121 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive entering [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:10 pc123 sshd[883385]: debug3: monitor_read: checking request 120
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_send entering: type 121
Jan 25 13:54:10 pc123 sshd[883385]: debug1: kex: curve25519-sha256 need=64 dh_need=64 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_send entering: type 120 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive_expect entering: type 121 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive entering [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:10 pc123 sshd[883385]: debug3: monitor_read: checking request 120
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_send entering: type 121
Jan 25 13:54:10 pc123 sshd[883385]: debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: receive packet: type 30 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_sshkey_sign entering [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_send entering: type 6 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_sshkey_sign: waiting for MONITOR_ANS_SIGN [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive_expect entering: type 7 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive entering [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:10 pc123 sshd[883385]: debug3: monitor_read: checking request 6
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_answer_sign
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_answer_sign: hostkey proof signature 0x55ba15421190(99)
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_send entering: type 7
Jan 25 13:54:10 pc123 sshd[883385]: debug2: monitor_read: 6 used once, disabling now
Jan 25 13:54:10 pc123 sshd[883385]: debug3: send packet: type 31 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: send packet: type 21 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: set_newkeys: mode 1 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: rekey out after 134217728 blocks [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: SSH2_MSG_NEWKEYS sent [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: expecting SSH2_MSG_NEWKEYS [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: send packet: type 7 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: receive packet: type 21 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: SSH2_MSG_NEWKEYS received [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: set_newkeys: mode 0 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: rekey in after 134217728 blocks [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: KEX done [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: receive packet: type 5 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: send packet: type 6 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: receive packet: type 50 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: userauth-request for user qwer789 service ssh-connection method none [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug1: attempt 0 failures 0 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_getpwnamallow entering [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_send entering: type 8 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive_expect entering: type 9 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive entering [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:10 pc123 sshd[883385]: debug3: monitor_read: checking request 8
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_answer_pwnamallow
Jan 25 13:54:10 pc123 sshd[883385]: debug2: parse_server_config: config reprocess config len 745
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_answer_pwnamallow: sending MONITOR_ANS_PWNAM: 1
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_send entering: type 9
Jan 25 13:54:10 pc123 sshd[883385]: debug2: monitor_read: 8 used once, disabling now
Jan 25 13:54:10 pc123 sshd[883385]: debug2: input_userauth_request: setting up authctxt for qwer789 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_start_pam entering [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_send entering: type 100 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_inform_authserv entering [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_send entering: type 4 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_inform_authrole entering [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_send entering: type 80 [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug2: input_userauth_request: try method none [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: user_specific_delay: user specific delay 0.000ms [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: ensure_minimum_time_since: elapsed 0.967ms, delaying 4.541ms (requested 5.508ms) [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:10 pc123 sshd[883385]: debug3: monitor_read: checking request 100
Jan 25 13:54:10 pc123 sshd[883385]: debug1: PAM: initializing for "qwer789"
Jan 25 13:54:10 pc123 sshd[883385]: debug1: PAM: setting PAM_RHOST to "10.123.45.67"
Jan 25 13:54:10 pc123 sshd[883385]: debug1: PAM: setting PAM_TTY to "ssh"
Jan 25 13:54:10 pc123 sshd[883385]: debug2: monitor_read: 100 used once, disabling now
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:10 pc123 sshd[883385]: debug3: monitor_read: checking request 4
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_answer_authserv: service=ssh-connection, style=
Jan 25 13:54:10 pc123 sshd[883385]: debug2: monitor_read: 4 used once, disabling now
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:10 pc123 sshd[883385]: debug3: monitor_read: checking request 80
Jan 25 13:54:10 pc123 sshd[883385]: debug3: mm_answer_authrole: role=
Jan 25 13:54:10 pc123 sshd[883385]: debug2: monitor_read: 80 used once, disabling now
Jan 25 13:54:10 pc123 sshd[883385]: debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth]
Jan 25 13:54:10 pc123 sshd[883385]: debug3: send packet: type 51 [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: receive packet: type 50 [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug1: userauth-request for user qwer789 service ssh-connection method publickey [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug1: attempt 1 failures 0 [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug2: input_userauth_request: try method publickey [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug2: userauth_pubkey: valid user qwer789 attempting public key rsa-sha2-512 xxxxxxxxxx [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: userauth_pubkey: have rsa-sha2-512 signature for RSA SHA256:xxxxx [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: mm_key_allowed entering [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: mm_request_send entering: type 22 [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: mm_key_allowed: waiting for MONITOR_ANS_KEYALLOWED [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: mm_request_receive_expect entering: type 23 [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: mm_request_receive entering [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:20 pc123 sshd[883385]: debug3: monitor_read: checking request 22
Jan 25 13:54:20 pc123 sshd[883385]: debug3: mm_answer_keyallowed entering
Jan 25 13:54:20 pc123 sshd[883385]: debug3: mm_answer_keyallowed: key_from_blob: 0x55ba15429000
Jan 25 13:54:20 pc123 sshd[883385]: debug1: temporarily_use_uid: 2001/3000 (e=0/0)
Jan 25 13:54:20 pc123 sshd[883385]: debug1: trying public key file /home/qwer789/.ssh/authorized_keys
Jan 25 13:54:20 pc123 sshd[883385]: debug1: Could not open authorized keys '/home/qwer789/.ssh/authorized_keys': No such file or directory
Jan 25 13:54:20 pc123 sshd[883385]: debug1: restore_uid: 0/0
Jan 25 13:54:20 pc123 sshd[883385]: debug3: mm_answer_keyallowed: publickey authentication: RSA key is not allowed
Jan 25 13:54:20 pc123 sshd[883385]: Failed publickey for qwer789 from 10.123.45.67 port 59029 ssh2: RSA SHA256:xxxxx
Jan 25 13:54:20 pc123 sshd[883385]: debug3: mm_request_send entering: type 23
Jan 25 13:54:20 pc123 sshd[883385]: debug2: userauth_pubkey: authenticated 0 pkalg rsa-sha2-512 [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: user_specific_delay: user specific delay 0.000ms [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: ensure_minimum_time_since: elapsed 3.484ms, delaying 2.024ms (requested 5.508ms) [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth]
Jan 25 13:54:20 pc123 sshd[883385]: debug3: send packet: type 51 [preauth]
Jan 25 13:54:25 pc123 sshd[883385]: debug3: receive packet: type 50 [preauth]
Jan 25 13:54:25 pc123 sshd[883385]: debug1: userauth-request for user qwer789 service ssh-connection method password [preauth]
Jan 25 13:54:25 pc123 sshd[883385]: debug1: attempt 2 failures 1 [preauth]
Jan 25 13:54:25 pc123 sshd[883385]: debug2: input_userauth_request: try method password [preauth]
Jan 25 13:54:25 pc123 sshd[883385]: debug3: mm_auth_password entering [preauth]
Jan 25 13:54:25 pc123 sshd[883385]: debug3: mm_request_send entering: type 12 [preauth]
Jan 25 13:54:25 pc123 sshd[883385]: debug3: mm_auth_password: waiting for MONITOR_ANS_AUTHPASSWORD [preauth]
Jan 25 13:54:25 pc123 sshd[883385]: debug3: mm_request_receive_expect entering: type 13 [preauth]
Jan 25 13:54:25 pc123 sshd[883385]: debug3: mm_request_receive entering [preauth]
Jan 25 13:54:25 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:25 pc123 sshd[883385]: debug3: monitor_read: checking request 12
Jan 25 13:54:25 pc123 sshd[883385]: debug3: PAM: sshpam_passwd_conv called with 1 messages
Jan 25 13:54:25 pc123 sshd[883385]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.123.45.67 user=qwer789
Jan 25 13:54:25 pc123 sshd[883385]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.123.45.67 user=qwer789
Jan 25 13:54:27 pc123 sshd[883385]: debug1: PAM: password authentication failed for qwer789: Authentication failure
Jan 25 13:54:27 pc123 sshd[883385]: debug3: mm_answer_authpassword: sending result 0
Jan 25 13:54:27 pc123 sshd[883385]: debug3: mm_request_send entering: type 13
Jan 25 13:54:27 pc123 sshd[883385]: Failed password for qwer789 from 10.123.45.67 port 59029 ssh2
Jan 25 13:54:27 pc123 sshd[883385]: debug3: mm_auth_password: user not authenticated [preauth]
Jan 25 13:54:27 pc123 sshd[883385]: debug3: user_specific_delay: user specific delay 0.000ms [preauth]
Jan 25 13:54:27 pc123 sshd[883385]: debug3: ensure_minimum_time_since: elapsed 1460.110ms, delaying 1360.043ms (requested 5.508ms) [preauth]
Jan 25 13:54:28 pc123 sshd[883385]: debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth]
Jan 25 13:54:28 pc123 sshd[883385]: debug3: send packet: type 51 [preauth]
Jan 25 13:54:35 pc123 sshd[883385]: debug3: receive packet: type 50 [preauth]
Jan 25 13:54:35 pc123 sshd[883385]: debug1: userauth-request for user qwer789 service ssh-connection method password [preauth]
Jan 25 13:54:35 pc123 sshd[883385]: debug1: attempt 3 failures 2 [preauth]
Jan 25 13:54:35 pc123 sshd[883385]: debug2: input_userauth_request: try method password [preauth]
Jan 25 13:54:35 pc123 sshd[883385]: debug3: mm_auth_password entering [preauth]
Jan 25 13:54:35 pc123 sshd[883385]: debug3: mm_request_send entering: type 12 [preauth]
Jan 25 13:54:35 pc123 sshd[883385]: debug3: mm_auth_password: waiting for MONITOR_ANS_AUTHPASSWORD [preauth]
Jan 25 13:54:35 pc123 sshd[883385]: debug3: mm_request_receive_expect entering: type 13 [preauth]
Jan 25 13:54:35 pc123 sshd[883385]: debug3: mm_request_receive entering [preauth]
Jan 25 13:54:35 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:35 pc123 sshd[883385]: debug3: monitor_read: checking request 12
Jan 25 13:54:35 pc123 sshd[883385]: debug3: PAM: sshpam_passwd_conv called with 1 messages
Jan 25 13:54:37 pc123 sshd[883385]: debug1: PAM: password authentication failed for qwer789: Authentication failure
Jan 25 13:54:37 pc123 sshd[883385]: debug3: mm_answer_authpassword: sending result 0
Jan 25 13:54:37 pc123 sshd[883385]: debug3: mm_request_send entering: type 13
Jan 25 13:54:37 pc123 sshd[883385]: Failed password for qwer789 from 10.123.45.67 port 59029 ssh2
Jan 25 13:54:37 pc123 sshd[883385]: debug3: mm_auth_password: user not authenticated [preauth]
Jan 25 13:54:37 pc123 sshd[883385]: debug3: user_specific_delay: user specific delay 0.000ms [preauth]
Jan 25 13:54:37 pc123 sshd[883385]: debug3: ensure_minimum_time_since: elapsed 2167.493ms, delaying 652.660ms (requested 5.508ms) [preauth]
Jan 25 13:54:37 pc123 sshd[883385]: debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth]
Jan 25 13:54:37 pc123 sshd[883385]: debug3: send packet: type 51 [preauth]
Jan 25 13:54:47 pc123 sshd[883385]: debug3: receive packet: type 50 [preauth]
Jan 25 13:54:47 pc123 sshd[883385]: debug1: userauth-request for user qwer789 service ssh-connection method password [preauth]
Jan 25 13:54:47 pc123 sshd[883385]: debug1: attempt 4 failures 3 [preauth]
Jan 25 13:54:47 pc123 sshd[883385]: debug2: input_userauth_request: try method password [preauth]
Jan 25 13:54:47 pc123 sshd[883385]: debug3: mm_auth_password entering [preauth]
Jan 25 13:54:47 pc123 sshd[883385]: debug3: mm_request_send entering: type 12 [preauth]
Jan 25 13:54:47 pc123 sshd[883385]: debug3: mm_auth_password: waiting for MONITOR_ANS_AUTHPASSWORD [preauth]
Jan 25 13:54:47 pc123 sshd[883385]: debug3: mm_request_receive_expect entering: type 13 [preauth]
Jan 25 13:54:47 pc123 sshd[883385]: debug3: mm_request_receive entering [preauth]
Jan 25 13:54:47 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:47 pc123 sshd[883385]: debug3: monitor_read: checking request 12
Jan 25 13:54:47 pc123 sshd[883385]: debug3: PAM: sshpam_passwd_conv called with 1 messages
Jan 25 13:54:49 pc123 sshd[883385]: debug1: PAM: password authentication failed for qwer789: Authentication failure
Jan 25 13:54:49 pc123 sshd[883385]: debug3: mm_answer_authpassword: sending result 0
Jan 25 13:54:49 pc123 sshd[883385]: debug3: mm_request_send entering: type 13
Jan 25 13:54:49 pc123 sshd[883385]: Failed password for qwer789 from 10.123.45.67 port 59029 ssh2
Jan 25 13:54:49 pc123 sshd[883385]: debug3: mm_auth_password: user not authenticated [preauth]
Jan 25 13:54:49 pc123 sshd[883385]: debug3: user_specific_delay: user specific delay 0.000ms [preauth]
Jan 25 13:54:49 pc123 sshd[883385]: debug3: ensure_minimum_time_since: elapsed 2147.643ms, delaying 672.510ms (requested 5.508ms) [preauth]
Jan 25 13:54:49 pc123 sshd[883385]: debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth]
Jan 25 13:54:49 pc123 sshd[883385]: debug3: send packet: type 51 [preauth]
Jan 25 13:54:50 pc123 sshd[883385]: debug3: mm_request_send entering: type 122 [preauth]
Jan 25 13:54:50 pc123 sshd[883385]: debug3: mm_request_receive_expect entering: type 123 [preauth]
Jan 25 13:54:50 pc123 sshd[883385]: debug3: mm_request_receive entering [preauth]
Jan 25 13:54:50 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:50 pc123 sshd[883385]: debug3: monitor_read: checking request 122
Jan 25 13:54:50 pc123 sshd[883385]: debug3: mm_request_send entering: type 123
Jan 25 13:54:50 pc123 sshd[883385]: Connection reset by authenticating user qwer789 10.123.45.67 port 59029 [preauth]
Jan 25 13:54:50 pc123 sshd[883385]: debug1: do_cleanup [preauth]
Jan 25 13:54:50 pc123 sshd[883385]: debug3: PAM: sshpam_thread_cleanup entering [preauth]
Jan 25 13:54:50 pc123 sshd[883385]: debug3: mm_request_send entering: type 124 [preauth]
Jan 25 13:54:50 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:50 pc123 sshd[883385]: debug3: monitor_read: checking request 124
Jan 25 13:54:50 pc123 sshd[883385]: debug1: monitor_read_log: child log fd closed
Jan 25 13:54:50 pc123 sshd[883385]: debug3: mm_request_receive entering
Jan 25 13:54:50 pc123 sshd[883385]: debug1: do_cleanup
Jan 25 13:54:50 pc123 sshd[883385]: debug1: PAM: cleanup
Jan 25 13:54:50 pc123 sshd[883385]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.123.45.67 user=qwer789
Jan 25 13:54:50 pc123 sshd[883385]: debug3: PAM: sshpam_thread_cleanup entering
Jan 25 13:54:50 pc123 sshd[883385]: debug1: Killing privsep child 883386
Jan 25 13:54:50 pc123 sshd[883385]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.123.45.67 user=qwer789
sshd_config:
# $OpenBSD: sshd_config,v 1.103 2018/04/09 20:41:22 tj Exp $
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
# Ciphers and keying
#RekeyLimit default none
# Logging
#SyslogFacility AUTH
SyslogFacility AUTHPRIV
#LogLevel DEBUG3
# Authentication:
#LoginGraceTime 2m
PermitRootLogin yes
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10
#PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
#AuthorizedPrincipalsFile none
#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody
# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes
# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no
PasswordAuthentication yes
# Change to no to disable s/key passwords
#ChallengeResponseAuthentication yes
ChallengeResponseAuthentication no
# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no
#KerberosUseKuserok yes
# GSSAPI options
GSSAPIAuthentication yes
GSSAPICleanupCredentials no
#GSSAPIStrictAcceptorCheck yes
#GSSAPIKeyExchange no
#GSSAPIEnablek5users no
UsePAM yes
#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
PrintMotd no
#PrintLastLog yes
#TCPKeepAlive yes
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS no
#PidFile /var/run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none
# no default banner path
#Banner none
# Accept locale-related environment variables
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
# override default of no subsystems
Subsystem sftp /usr/libexec/openssh/sftp-server
/etc/pam.d/password-auth:
auth required pam_env.so
auth required pam_faildelay.so delay=2000000
auth [default=1 ignore=ignore success=ok] pam_usertype.so isregular
auth [default=1 ignore=ignore success=ok] pam_localuser.so
auth sufficient pam_unix.so nullok try_first_pass
auth [default=1 ignore=ignore success=ok] pam_usertype.so isregular
auth sufficient pam_sss.so forward_pass
auth required pam_deny.so
account required pam_unix.so
account sufficient pam_localuser.so
account sufficient pam_usertype.so issystem
account [default=bad success=ok user_unknown=ignore] pam_sss.so
account required pam_permit.so
password requisite pam_pwquality.so try_first_pass local_users_only
password sufficient pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password sufficient pam_sss.so use_authtok
password required pam_deny.so
session optional pam_keyinit.so revoke
session required pam_limits.so
-session optional pam_systemd.so
session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session required pam_unix.so
session optional pam_sss.so
/etc/sss/sssd.conf/var/log/sssd. I'll take another guess and say sssd is configured for ldap. Also look at/etc/pam.d/common-password. Thinking about it, it could just be they have different keyboards producing different scancodes. Perhaps someone it thinking they are keying in$, and on another keyboard this will be£or something similar.[kcm] [sec_get] (0x0040): Cannot retrieve the secret [2]: No such file or directory. There isn't a/etc/sssd/sssd.conffile so I'm not sure LDAP is configured for it (or anything for that matter). There aren't any/etc/pam.d/common-*files. The character encoding idea is a fair one, though all the systems are configured with the same US keyboard layout and the users are using the same. For now I have tried settingGSSAPIAuthentication noin sshd_config to see if anything changes..