What can be the speed of ElasticSearch for a given case and are there any other alternatives?

Question

elasticsearch

Hello, I'm new to ElasticSearch, I want to understand if it suits me for the technical problem I'm solving.
I will describe the data and the case.

record {
customerId: "id",      <---- индексируется
binaryData: "data"   <---- не индексируется
}

Expectations:

• the average size of the binary record data is 1-2 kilobytes
  
• one customerId - up to 100 records
  
• number of entries - 500M
  
• record pattern #1: insert one record at a time
  
• recording pattern No. 2: in batches, it is possible in parallel, at a speed of at least 20M record per hour
  
• search pattern #1: search all records by customerId
  
• search pattern No. 2: search for all records by customerId group, it is possible in parallel, at a speed of at least 10M customerId per hour
  
• the data is not the most important, you can exchange some aspects of reliability for speed. The main thing is not to lie down to death and not to systematically lose pieces of data
  
• we suppose to work in AWS / GCP - and we don’t want to administer the system ourselves for sure
  

• Is it possible to achieve such speed indicators on ElasticSearch?
  
• What servers for such task can go?
  
• How is ElasticSearch doing with reliability in general? Can you expect good reliability if you use cloud services like AWS with replicas in different regions?
  
• Maybe there are other systems that are well suited for such requirements?
  

Thank you!

Answer 1

You don't need a search engine. This is a task for key-value storage. You can use AWS DynamoDB for this, for example.

Answer 2

I agree with Ivan Shumov - there is no point in running ElasticSearch for key-value.
Here is a good intro to different AWS options and I would start checking with DynamoDB:
https://cloudonaut.io/databases-on-aws/

Answer 3

clickhouse

Answer 4

search pattern #1: search all records by customerId

this is a script for DBMS key/value.
But:

search pattern #2: search for all records by customerId group,

this is a request like

SELECT blob FROM table WHERE customerId IN (100, 101, 200)

Not sure if DynamoDB supports such requests. Ivan Shumov ?
So I'm voting for RDS/Aurora for now. Here's what you need to test for:

recording pattern No. 2: in batches, it is possible in parallel, at a speed of at least 20M record per hour