ElasticSearch, how to do fast insertion of large dictionaries?

Question

elasticsearch

We need to insert 70,000 large dictionaries with 2-5 levels of nesting.
I don't use the bulk, because the constant was getting different errors, the last one was with a timeout
. The problem is that at first the insertion speed is acceptable for half a second, and then more and more.
elasticsearch client settings:
'settings': {
'index.number_of_shards': '5',
'index.refresh_interval': '1s',
'index.max_result_window': 1000000000,
'index.mapping.total_fields.limit': 1000000000
}
Tried use multiprocessing for this, there is no difference. Share your experience related to this task if any. Thank you.

Answer 1

It will come out a little sensible, the easiest way (but also requiring some knowledge) is to buy mikrotik of the younger series, the price is 600-700r, inside the ability to customize as you please.
Similarly, I sit on adsl and when uploading, the ping goes to infinity. The Internet solved the problem with this scheme
<-> D-Link DSL-2500U (pppoe) <-> Mikrotik hAP Lite (proxy, QoS, VPN, Wifi IoT) <-> Xiaomi WiFi3 (changed, basic wifi ac)
Maximum upload speed in 80% of the channel, now the ping does not jump and dropped to 10-15ms.
You can also reduce the ping at the cost of download speed loss by changing the adsl protocol

Answer 2

Firmware DD-Wrt
If you are not too lazy to read, you will succeed.