logstash.outputs.elasticsearch retrying failed action with response code 403

在用Logstash收集应用数据到ElasticSearch中时，运行一段时间后就出错：

[root@appblog.cn logstash]# tail -f logs/logstash-plain.log 
[2021-02-20T17:09:26,878][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
[2021-02-20T17:09:26,878][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
[2021-02-20T17:09:26,879][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
[2021-02-20T17:09:26,879][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
[2021-02-20T17:09:26,879][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
[2021-02-20T17:09:26,880][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
[2021-02-20T17:09:26,880][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
[2021-02-20T17:09:26,880][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})
[2021-02-20T17:09:26,881][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

按照 Eleastisearch 官方文档给的解决办法是

An example of resetting the read-only index block on the twitter index:

PUT /twitter/_settings
{
  "index.blocks.read_only_allow_delete": null
}

官方链接： https://www.elastic.co/guide/en/elasticsearch/reference/current/disk-allocator.html

按照官方给的这个解决思路，在Linux上执行：

curl -XPUT -H 'Content-Type: application/json' http://192.168.1.8:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

其中_all可以更改为自己在创建 ElasticSearch 索引的时候的name，用来修改单个索引只读状态，当然用_all是修改所有的索引只读状态