How to determine which operations will cause high disk load in elasticsearch?

Question

elasticsearch

There is a container with elasticsearch 5.6.13 (single) in the DC / OS cluster, which receives data from logstash, about 300 indexes, 1400 shards. Indexes from 5mb to 500mb.
After launch, it works correctly for a while, then the entry [oecmMetaDataMappingService] [ -n4ixxd
] [name of the index created today] update_mapping [name of the index created today] appears in the logs
they write something very actively / read from the disk. LA is under 80, it still works for a couple of hours, then messages from the GC begin, apparently the memory is slowly eaten away and the container falls.
Elastic settings - almost default, increased heap size to 4GB (8 allocated to the container itself).
bootstrap.memory_lock: true
refresh_interval - increased to 5 minutes

Answer 1

Since the elastic is written in Java, and in Java there are jmx, mbeans and jconsole, I don’t see any problems monitoring. You can even capture this data via logstash and shove it back into elastic for viewing in kibana. pavanrp1.blogspot.com

Answer 2

Not entirely on topic, of course, but I will say. Why do you need 1400 shards for a single elastic instance, especially with index sizes from 5 to 500mb? According to the manufacturer's recommendations, the size of the shard, which requires sharding (sorry for the tautology), is about 50GB... Again, for 4GB 300 indexes, this is not very weak, because elastic holds a lot of information in memory. Optimize or give it more resources.