ElasticSearch 常见异常

1. Cannot open file logs/gc.log due to Permission denied

不能用root账户启动ES,需要用ES账户启动su essearch ./bin/elasticsearch

如果出现以上问题,则是因为用root启动过导致。到ElasticSearch中的log中删除gc.log文件即可

2. ElasticSearch 启动不起来

当Logstash正大量拥堵要写入的数据时,ES是无法启动的,负载太高,需要临时停止Logstash作业

3. 健康状态一直处于警告,ES坏节点较多,无法分片

1
2
3
4
#重新分配
POST /_cluster/reroute?retry_failed=true
#查看集群状态
GET _cluster/health?pretty

4. logstash启动

1
nohup bin/logstash -f config/kafka2es.conf &>/dev/null &

5. Validation Failed: 1: this action would add [1] total shards, but this cluster currently has [1000]

1
[2021-04-12T03:41:01,723][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"appblog-2021.02.12", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x53ee69a7>], :response=>{"index"=>{"_index"=>"appblog-2021.02.12", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"validation_exception", "reason"=>"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [1000]/[1000] maximum shards open;"}}}}

ES报错无法创建多余的分片,可调整分片数

原因:ES集群默认有最大分片数1000的限制,达到最大数以后,无法添加新的分片,导致无法创建索引写入日志。

解决:在Kibana上的DEV TOOL工具里执行如下命令

1
2
3
4
5
6
7
8
PUT /_cluster/settings
{
"transient": {
"cluster": {
"max_shards_per_node":10000
}
}
}
1
2
3
4
5
6
7
8
9
{
"acknowledged" : true,
"persistent" : { },
"transient" : {
"cluster" : {
"max_shards_per_node" : "10000"
}
}
}