ES的索引每日都在增加的煩惱
elasticsearch本身有提供API去讓你處理index 當然我們可以自己去寫一個簡單的小程序去每日處理那些沒有用的index (e.g. 服務器日誌) 但是 Don’t reinvent the wheel Don’t reinvent the wheel 因為ES就有提供一個簡單方便的程序去做這件事 https://www.elastic.co/guide/en/elasticsearch/client/curator/5.5/fe_source.html
只要進行簡單的設定及以cronjob運行就可以了(我是用k8s中的cron job去運行,更加簡單) 設定config.yml (怎樣連接去es)
client:
hosts:
- http://your-es.host.com
port: 9200
url_prefix:
use_ssl: False
certificate:
client_cert:
client_key:
ssl_no_validate: False
http_auth:
timeout: 30
master_only: False
logging:
loglevel: INFO
logfile:
logformat: default
blacklist: ['elasticsearch', 'urllib3']
設定action_file.yml (決定哪一個index需要刪除)
actions:
1:
action: delete_indices
description: "Clean up ES by deleting old indices"
options:
continue_if_exception: False
disable_action: False
filters:
- filtertype: pattern
kind: regex
value: '^(prod-|stag-|dev-)kubernetes.*$'
- filtertype: age
source: name
direction: older
timestring: '%Y-%m-%h'
unit: days
unit_count: 30
exclude: False
注意filters中間的條件是AND 而不是OR的 這邊的設定簡單來說就是只要index的名字是stag/dev/prod而且含有kubernetes字元,產生日期是30天前就會刪掉