I
I
Ilya T.2017-01-11 20:41:34
NoSQL
Ilya T., 2017-01-11 20:41:34

How to remove many records from elasticsearch?

Given, the base of hundreds of millions of records of the form:
ID, FILE_ID, SITE_ID, it is
necessary to optimally delete tens of thousands of records for which we know the FILE_ID, and SITE_ID within one request is the same.
ID - unique (but we don't know them initially)
FILE_ID - unique for each site
I.e. in SQL it would look something like this
delete from table where (FILE_ID=x or FILE_ID=y or ....) and (SITE_ID=z)

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
al_gon, 2017-01-11
@al_gon

Nothing smarter comes to mind.
https://www.elastic.co/guide/en/elasticsearch/refe...

{
    "query": {
        "query_string": {
            "query": "(FILE_ID:x OR FILE_ID:y OR ....) AND (SITE_ID:z)"
        }
    }
}

V
Vadim Stepanov, 2017-01-19
@Vdm17

For ES 5.0+ there is the DeleteByQuery API
https://www.elastic.co/guide/en/elasticsearch/ref...

POST index_name/type_name/_delete_by_query
{
  "query": {
    "bool": {
      "must": [
        {"term": {"SITE_ID": "z"}}
      ],
      "should": [
        {"term": {"FILE_ID": "x"}},
        {"term": {"FILE_ID": "y"}},
        ...
      ],
      "minimum_should_match": 1
    }
  }
}

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question