To only allow specific users to ssh to our server, you can add the following line to your sshd_config file:
AllowUsers usera userd userf
You can also use the AllowGroups option.
AllowGroups admins webmasters
In order for the changes to take effect, you will need to restart or reload sshd.
service sshd
