6.4.6.1 X-Pack on Elasticsearch

由于本文使用的es的版本为6.*，所以X-Pack不需要单独安装， 由于没有es的正式license请先查看6.4.6.4 xpack破解， 并保证license为生效状态， 所有相关参数配置elasticsearch.yml中，配置完成后使用systemctl restart elasticsearch.service 重启服务

Security setting in Elasticsearch (安全验证)

关键参数说明：

  • action.auto_create_index 是否开启自动创建index

  • xpack.security.enabled 是否开启安全认证, 默认为true

  • xpack.security.transport.ssl.enabled elasticsearch节点之前是否使用ssl传输数据， 默认false

  • xpack.security.transport.ssl.verification_mode 包含三种方式full(default) certificate none

    • full 检查证书是否由权威机构发布的， 还效验主机名是否和证书中的标识相同

    • certificate 检查证书是否由权威机构发布的， 但是不效验主机名

    • none 不执行任何效验

  • xpack.security.transport.ssl.keystore.path 证书文件存储全路径

  • xpack.security.transport.ssl.truststore.path 信任证书存储的全路径

(可选)配置自动创建索引

X-Pack需要在elasticsearch自动的创建一些索引， 默认elasticsearch允许自动创建索引, 如果关闭了自动创建索引必须在elasticsearch.yml中配置action.auto_create_index允许X-Pack自动创建如下的索引：

action.auto_create_index: .security,.monitoring*,.watches,.triggered_watches,.watcher-history*,.ml*

也可以这样配置action.auto_create_index: *允许创建所有索引

配置传输层安全性(TLS/SSL)

如果您拥有License，并且希望使用X-Pack安全性，则必须为节点间通信配置TLS。

  1. 创建一个CA， 生成文件名称为elastic-stack-ca.p12

bin/elasticsearch-certutil ca

  1. 生成证书和私钥

bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

  1. 将证书传输到各个节点的指定位置

mkdir /etc/elasticsearch/certs
mv bin/elastic-certificates.p12 /etc/elasticsearch/certs
mv bin/elastic-stack-ca.p12 /etc/elasticsearch/certs

开启X-Pack security

编辑elasticsearch.yml , 输入如下内容

xpack.security.enabled: true

开启集群中https传输

编辑elasticsearch.yml输入如下内容

xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

设置内建用户的密码

需要保留创建的用户名和密码， logstash 和 kibana 连接es时需要使用该密码

  • 手动方式设置所有内建用户的密码

# /usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y
Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [apm_system]: 
Reenter password for [apm_system]: 
Enter password for [kibana]: 
Reenter password for [kibana]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Enter password for [beats_system]: 
Reenter password for [beats_system]: 
Enter password for [remote_monitoring_user]: 
Reenter password for [remote_monitoring_user]: 
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

  • 使用自动方式设置所有内建用户的密码

 bin/elasticsearch-setup-passwords auto

保存console的输出内容以便于后续使用时查看

Auditing Security Settings（安全审计）

以下内容摘自官方文档， 更多配置请参考官方文档

关键参数说明

  • xpack.security.audit.enabled 是否开启自动创建index

  • xpack.security.audit.outputs 指定审计日志的输出位置， [ index, logfile ]

    • logfile(default) 输出审计日志到每个节点的<cluster-name>_access.log

    • index 输出审计日志到名字为.security_audit_log索引中

    PS: 应当尽量索引方式和日志方式同时使用避免审计日志的丢失

  • xpack.security.audit.logfile.events.include 指定审计日志输出的event, 默认值为:access_denied、access_granted、anonymous_access_denied、authentication_failed、connection_denied、tampered_request、run_as_denied、run_as_granted。

  • xpack.security.audit.logfile.events.emit_request_body 指定是否在某些事件类型(如authentication_failed)上包含来自REST请求的请求体。默认值为false。

  • xpack.security.audit.logfile.events.ignore_filters

    <policy_name> 自定义策略的名称

    • xpack.security.audit.logfile.events.ignore_filters.<policy_name>.users

      匹配的users将不输出审计日志，例如： ["logstash_internal"]

    • xpack.security.audit.logfile.events.ignore_filters.<policy_name>.realms

      A list of authentication realm names or wildcards. The specified policy will not print audit events for users in these realms.

    • xpack.security.audit.logfile.events.ignore_filters.<policy_name>.roles

      匹配的roles将不输出审计日志

    • xpack.security.audit.logfile.events.ignore_filters.<policy_name>.indices

      匹配的索引将不输出审计日志

开启审计

编辑elasticsearch.yml , 输入如下内容

  xpack.security.audit.enabled: true
  xpack.security.audit.logfile.events.emit_request_body: true

设置审计日志输出的位置

xpack.security.audit.outputs: [ logfile ]

定义filter不输出审计日志

忽略用户logstash_internal 不产生审计日志

xpack.security.audit.logfile.events.ignore_filters: 
  exclude_logstash_write:
    users: ["logstash_internal"]

Watcher Setting in Elasticsearch

可以在elasticsearch.yml中配置 xpack.notification 发送通知到email, HipChat, Slack, 或者 PagerDuty.

关键参数说明

  • xpack.watcher.enabled 设置为true或false 打开或者关闭Watcher

  • xpack.http.proxy.host 指定代理主机

  • xpack.http.proxy.port 指定代理主机连接的端口

  • Email Notification Settings

    • xpack.notification.email.account

      • profile;; The email profile to use to build the MIME messages that are sent from the account. Valid values: standard, gmail and outlook. Defaults to standard.

      • email_defaults.* An optional set of email attributes to use as defaults for the emails sent from the account. See Email Action Attributes for the supported attributes

      • smtp.auth 设置为true或者false， true使用认证， false不实用认证

      • smtp.host 指定smtp的地址

      • smtp.port 指定smtp 连接的端口

      • smtp.user 指定smtp认证的用户名

      • smtp.password 指定smtp.user的密码

      • smtp.starttls.enable 指定是否需要使用tls协议连接

  • xpack.notification.email.default_account 指定发送报警的账号

开启Watcher

编辑elasticsearch.yml , 输入如下内容

xpack.watcher.enabled: True

配置smtp账号信息

xpack.notification.email.account:
  exmail:
    profile: standard
    email_defaults:
      from: gongxiude888@163.com
    smtp:
      auth: true
      starttls.enable: true
      host: smtp.163.com
      port: 465
      user: gongxiude888@163.com
      password: 651201

配置notification的默认账号

xpack.notification.email.default_account: exmail

Monitoring Settings in Elasticsearch

Monitoring默认是打开状态， 可以配置monitoring在elasticsearch.yml文件中

xpack.monitoring.enabled: True

Machine Learning Settings in Elasticsearch

机器学习默认为开启状态, 因为没有具体使用将其关闭, 编辑elasticsearch.yml , 输入如下内容

xpack.ml.enabled: False

elasticsearch配置整合

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html

# Cluster / Node Basics
cluster.name: welog

# Node can have abritrary attributes we can use for routing
node.name: es-welog02cn-dw03-p014
node.max_local_storage_nodes: 1
node.master: False
node.data: True
node.ingest: True
node.attr.rack: warm 

# networking Settings
network.host: 10.40.9.154,127.0.0.1
http.enabled: false
transport.tcp.port: 9300

# dirs
path.data:
    - /data/elasticsearch/data-vdc
    - /data/elasticsearch/data-vde
    - /data/elasticsearch/data-vdd

path.logs: /data/elasticsearch/logs

# Minimum nodes alive to constitute an operational cluster
discovery.zen.minimum_master_nodes: 2

# Unicast Discovery (disable multicast)
discovery.zen.ping.unicast.hosts:
  - es-welog02cn-p001.pek3.example.net
  - es-welog02cn-p002.pek4.example.net
  - es-welog02cn-p003.pek3.example.net

# X-Pack
xpack.security.enabled: True
xpack.security.audit.enabled: true
xpack.security.audit.outputs: [ logfile ]
xpack.security.audit.logfile.events.emit_request_body: true
xpack.security.audit.logfile.events.ignore_filters:
  exclude_logstash_write:
    users: ["logstash_internal"]
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/certs/elastic-certificates.p12
xpack.watcher.enabled: True
xpack.monitoring.enabled: True
xpack.ml.enabled: False

xpack.notification.email.account:
  exmail:
    profile: standard
    email_defaults:
      from: log-monitor@exmail.qq.com
    smtp:
      auth: true
      starttls.enable: true
      host: smtp.exmail.qq.com
      port: 587
      user: 
      password: 

# X-Pack watcher email notification
xpack.notification.email.default_account: exmail
xpack.notification.email.account.alerting.smtp.host: "smtp.exmail.qq.com"
上一页6.4.6 enable-xpack下一页6.4.6.2 X-Pack on Logstash

最后更新于