Recently I have created a new user in one of my linux server and try to verify the credentials over ssh . But I used to get ” Permission denied” message even though verified the credentails from the local login itself. So it is something like weird situation and seeing the first time in my life. After checking the ssh log file (/var/log/secure) it is showing some detailed error report as shown below,
Nov 26 00:22:28 sshd: input_userauth_request: invalid user prod-write
Nov 26 00:22:36 sshd: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=220.127.116.11 user=prod-write
Nov 26 00:22:37 sshd: Failed password for invalid user prod-write from 18.104.22.168 port 54078 ssh2
I searched the /etc/ssh/sshd_config file and did not see any “AllowUsers” is set over there. Then I tried to update the SSH package and restart the ssh service. But I could not restart the ssh as it is fails while stopping it.
Stopping sshd: [FAILED]
Starting sshd: [ OK ]
I was seen that from the log((/var/log/secure)) below entries
Nov 26 23:11:43 sshd: fatal: Cannot bind any address.
Nov 26 23:13:31 : error: Bind to port 22 on 0.0.0.0 failed: Address already in use.
So I fall in trouble since I do not have any option to go since ssh is the only way to connect the server and I can not kill that process.
I have set a cronjob to restart ssh service on every 5 minutes and verified that it is working. After that I have killed all the ssh process by “killall -9 sshd”. After 4 minutes later I could able to login to the server either over root and the user which I created successfully.
Moral of the story
You can see from the ssh log file, there were lots of connection refusal requests are coming towards the server. This is the only reason while ssh is not being restarted since it is busy with refusing ssh connection. So I had late to identify it sadly ::( So set ssh firewall rules now itself and manage the access.