Fixing “Error writing alert info to Elasticsearch” with Elastalert

While developing a new Elastalert rule I’ve got an Error

elastalert[4772]: ERROR:root:Error writing alert info to Elasticsearch: RequestError(400, 'mapper_parsing_exception', "failed to parse field [match_body.foo.bar] of type [long] in document with id 'ax9_XngBcjZMAWIvYa3i'. Preview of field's value: '28P01’")

The error clearly tells us that the problem is in the mapping. But why? As we can see in the documentation ElastAlert uses Elasticsearch to store meta data in four indexes: elastalert, elastalert_status, elastalert_error, elastalert_silence.

Exploring further I found the difference between mappings for foo.bar in /elastalert/_mapping and another index. To fix this error, I reindexed the elastalert index:

1. First of all we need to stop the ElastAlert service and backup the index:

POST /elastalert/_clone/elastalert-backup

2. Check the index is cloned successfully

GET /_cat/indices/elastalert*

3. Get the mapping and copy the output to the text editor

GET /elastalert/_mapping

4. Fix field’s types in the editor

5. Delete /elastalert index

DELETE /elastalert

6. Create the new index with modified mapping:

PUT /elastalert
{
    "settings": {
        "number_of_shards": 1
    },
    "mappings" : {
        ...
    }
}

7. Reindex data from backup

POST _reindex
{
  "source": {
    "index": "elastalert-old"
  },
  "dest": {
    "index": "elastalert"
  }
}