Problem with SWAP?

Question

linux

After migrating several sites to a new server with Ubuntu 16.04 LTS, large amount of RAM (1GB) and PHP7, there was a problem using RAM / swap.
The server has a LEMP stack and several simple sites with a total attendance of a maximum of 3-4k unique users per day. All applications (MySQL, Nginx, PHP-FPM, PHP 7, etc.) take up 300-400 MB (30-40%), but swap is used at 100%.

KiB Mem: 1012720 total, 14172 free, 302536 used, 696012 buff/cache 
KiB Swap: 262140 total, 0 free, 262140 used. 
629520 avail Mem

node:~$ free -m
              total        used        free      shared  buff/cache   available
Mem:            988         292         135          38         561         617
Swap:           255         255           0

Memory usage graph from Munin:

I know about the vm.swappiness parameter and it is 10 in my case:

node:~$ cat /proc/sys/vm/swappiness 
10

Given the vm.swappiness parameter, SWAP should be used if RAM is occupied by 90% or at least not clogged by 100%. Since the server has 40-50% free RAM, and SWAP is 100% busy, then vm.swappiness is not working properly or something is configured incorrectly.
There might be something wrong with the server configuration. Here is an example /etc/sysctl.conf:

kernel.panic=10
vm.panic_on_oom=1
vm.swappiness=10
vm.dirty_background_ratio=5
vm.dirty_ratio=10
net.ipv4.conf.all.accept_redirects=0
net.ipv6.conf.all.accept_redirects=0
net.ipv4.conf.default.accept_redirects=0
net.ipv6.conf.default.accept_redirects=0
net.ipv4.conf.all.secure_redirects=0
net.ipv4.conf.default.secure_redirects=0
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.icmp_ignore_bogus_error_responses=1
net.ipv4.conf.all.forwarding=0
net.ipv6.conf.all.forwarding=0
net.ipv4.conf.default.forwarding=0
net.ipv6.conf.default.forwarding=0
net.ipv4.ip_forward=0
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.default.send_redirects=0
net.ipv4.conf.all.accept_source_route=0
net.ipv6.conf.all.accept_source_route=0
net.ipv4.conf.default.accept_source_route=0
net.ipv6.conf.default.accept_source_route=0
net.ipv4.conf.all.rp_filter=1
net.ipv4.conf.default.rp_filter=1
net.ipv4.conf.all.proxy_arp=0
net.ipv4.ip_local_port_range=1024 65535
net.core.somaxconn=65536
net.core.netdev_max_backlog=30000
fs.file-max=2097152
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_tw_recycle=0
net.ipv4.tcp_fin_timeout=15
net.ipv4.tcp_max_tw_buckets=1440000
net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_timestamps=1
net.ipv4.tcp_sack=1
net.ipv4.tcp_rfc1337=1
net.ipv4.tcp_no_metrics_save=1
net.ipv4.tcp_mem=50576 64768 98152
net.ipv4.tcp_rmem=4096 524288 11960320
net.core.rmem_max=11960320
net.ipv4.tcp_wmem=4096 524288 11960320
net.core.wmem_max=11960320
net.core.optmem_max=65535
net.ipv4.tcp_max_orphans=65535
net.ipv4.tcp_orphan_retries=0
net.ipv4.tcp_keepalive_time=1800
net.ipv4.tcp_keepalive_intvl=25
net.ipv4.tcp_keepalive_probes=5
net.ipv4.tcp_syncookies=1
net.ipv4.tcp_syn_retries=2
net.ipv4.tcp_synack_retries=3
net.ipv4.tcp_max_syn_backlog=3240000

I can also show MySQL, PHP-FPM, Nginx, etc. settings.
Help, please, to understand what's the matter.
Thank you.

Answer 1

Here in your free output you can see:

total        used        free      shared  buff/cache   available
Mem:            988         292         135          38         561         617

That there are all sorts of caches and they have eaten up space. Most likely FS on the ext4 server and she knows how to cache and does it.
If after
you have freed up memory 2 times more than swap, then you can do it like this:
And if in the end everything worked, then the point is exactly that you have a lot of small files that the file system caches.
In this case, you need to optimize php (usually troubles because of it), enable and configure opcache, for example...

Answer 2

What does swap use?
top with swap- to studio
cat /proc/*/status for the process that takes the most swap