How to deny a specific user access via SSH when connecting on a specific interface?

Question

linux

Here is a question. There is a server with two interfaces (Ubuntu 14). One looks to the local network the other to the Internet. The server has 2 users admin and user . Worth SFTP. User must have access from the Internet to his folder via SFTP and not have access to the shell. Everything is set up and working fine.
But as a side effect, access from the Internet via ssh and sftp is also available from admin . I would like to close. If you add a line ListenAddress in /etc/ssh/sshd_config indicating the address of the local interface, then access from outside is closed, including for SFTP user , which is not good. If we formulate the task in short, then: It is required to prohibit
a specific user access via SSH when connecting on a specific interface. Any ideas?

Answer 1

Thanks to everyone for the advice, they pushed me to the right decision.
Actually the decision.
Everything is solved by editing the file /etc/ssh/sshd_config

Let's create a directive that allows the user user only SFTP and define a directory for it (I already had it)
Match User user
ChrootDirectory /ftp/user/
ForceCommand internal-sftp

Create a directive allowing SSH access to the admin user only if the client's ip address is in the range of local addresses
Match Address 192.168.0.0/24
AllowUsers admin
Maybe someone will come in handy.

Answer 2

man 5 sshd_config
DenyUsers etc.

Answer 3

How do you like the idea of ​​running two instances of sshd with different configs?
sshd -f config_file
And by the way, have you tried AllowUsers?