F
F
Fedor Sirotkin2020-08-18 17:59:55
elasticsearch
Fedor Sirotkin, 2020-08-18 17:59:55

How to convert Elasticsearch aggregation results to percentages and display a message based on the results?

There is a mapping:

PUT /status_email/_mapping
{
  "status_email" : {
    "mappings" : {
      "properties" : {
        "date_consume_queue" : {
          "type" : "date",
          "format" : "yyyy-MM-dd HH:mm:ss.SSSZZ||yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
        },
        "date_publish_queue" : {
          "type" : "date",
          "format" : "yyyy-MM-dd HH:mm:ss.SSSZZ||yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
        },
        "date_send_message" : {
          "type" : "date",
          "format" : "yyyy-MM-dd HH:mm:ss.SSSZZ||yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
        },
        "email_to" : {
          "type" : "text",
          "fielddata" : true
        },
        "ip" : {
          "type" : "ip"
        },
        "log_level" : {
          "type" : "text",
          "fielddata" : true
        },
        "login" : {
          "type" : "text",
          "fielddata" : true
        },
        "message_id" : {
          "type" : "text",
          "fielddata" : true
        },
        "response_code" : {
          "type" : "text",
          "fielddata" : true
        },
        "response_text" : {
          "type" : "text",
          "fielddata" : true
        },
        "status" : {
          "type" : "boolean"
        },
        "uuid" : {
          "type" : "text",
          "fielddata" : true
        }
      }
    }
  }
}

Based on it, documents are placed that contain the statuses of placing messages in the SMTP server.

Success and failure statuses must be aggregated. Convert the results of aggregation to a percentage, if the result does not exceed 10% of errors, then display "OK", otherwise "ERROR".

An example of one of the added documents:
{
  "date_publish_queue" : "2020-08-17 10:25:30.755+0300",
  "log_level" : "Info",
  "uuid" : "84d2cb67-dfca-44c0-bbfb-32ac8df39937",
  "email_to" : "[email protected]",
  "ip" : "::ffff:192.168.1.11",
  "login" : "test",
  "date_consume_queue" : "2020-08-17 10:25:31.075+0300",
  "response_code" : "250 2.0.0 Ok: queued as C7DF0122588",
  "date_send_message" : "2020-08-17 10:25:31.829+0300",
  "message_id" : "<[email protected]>",
  "response_text" : "send",
  "status" : true
}

Example of my request:
GET /status_email/_search?pretty
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "date_publish_queue": {
              "gte": "now-1h"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "smtp_bad": {
      "filter": {
        "bool": {
          "must_not": [
            {
              "exists": {
                "field": "status"
              }
            },
            {
              "match": {
                "status": "false"
              }
            },
            {
              "match_phrase": {
                "response_code": "250 2.0.0 Ok"
              }
            }
          ]
        }
      }
    },
    "smtp_good": {
      "filter": {
        "bool": {
          "must": [
            {
              "exists": {
                "field": "status"
              }
            },
            {
              "match": {
                "status": "true"
              }
            },
            {
              "match_phrase": {
                "response_code": "250 2.0.0 Ok"
              }
            }
          ]
        }
      }
    }
  }
}

An example response to such a request:
{
  "took" : 5,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 200,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "smtp_bad" : {
      "doc_count" : 7
    },
    "smtp_good" : {
      "doc_count" : 193
    }
  }
}

It is necessary to calculate percentages from these values. For example:
  • Failed emails sent = 7 / 200 * 100 = 3.5%
  • Successful sending of letters = 193 / 200 * 100 = 96.5%

If the percentage exceeds 10% of erroneous sendings, then print "ERROR", otherwise print "OK".
I tried to calculate the percentage of success and errors of sending emails:
GET /status_email/_search?pretty
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "date_publish_queue": {
              "gte": "now-1h"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "result": {
      "terms": {
        "script": {
          "source": "'this a hack'"
        }
      },
      "aggs": {
        "smtp_bad": {
          "filter": {
            "bool": {
              "must_not": [
                {
                  "exists": {
                    "field": "status"
                  }
                },
                {
                  "match": {
                    "status": "false"
                  }
                },
                {
                  "match_phrase": {
                    "response_code": "250 2.0.0 Ok"
                  }
                }
              ]
            }
          }
        },
        "smtp_good": {
          "filter": {
            "bool": {
              "must": [
                {
                  "exists": {
                    "field": "status"
                  }
                },
                {
                  "match": {
                    "status": "true"
                  }
                },
                {
                  "match_phrase": {
                    "response_code": "250 2.0.0 Ok"
                  }
                }
              ]
            }
          }
        },
        "percent_smtp_good": {
          "bucket_script": {
            "buckets_path": {
              "total": "_count",
              "good": "smtp_good._count"
            },
            "script": {
              "params": {
                "full_percent": 100
              },
              "source": "params.good / params.total * params.full_percent"
            }
          }
        },
        "percent_smtp_bad": {
          "bucket_script": {
            "buckets_path": {
              "total": "_count",
              "bad": "smtp_bad._count"
            },
            "script": {
              "params": {
                "full_percent": 100
              },
              "source": "params.bad / params.total * params.full_percent"
            }
          }
        }
      }
    }
  }
}

How can I organize the output of a string depending on the results of the aggregation?

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question