6.4.2 安装

下面文档为基础的部署步骤， 后续请使用ansible进行部署

• 安装Elasticsearch

• 安装Logstash

• 部署Kibana

• 安装Cerebro

• 安装Curator

基础信息准备

由于ELK组建的更新换代速度很快， 请参考官方文档中的有关信息确定安装的具体版本

集群基本信息

es_version: 6.5.1 
jkd_version: 8 
es_cluster_name: welog

主机资源信息

IP	角色	生命周期角色	部署程序
10.40.58.153	master、data、client	hot	es logstash
10.40.58.154	master、data、client	cold	es kibana
10.40.61.116	master、data、client	cold	es cerebro curator

维护ansible hosts文件

➜ cat hosts

[elasticsearch-master]
py-modelo2o08cn-p003.pek3.example.net ansible_host=10.40.58.153

[elasticsearch-datanode]
py-modelo2o08cn-p004.pek3.example.net ansible_host=10.40.58.154
py-modelaud05cn-p005.pek3.example.net ansible_host=10.40.58.116

[logstash]
py-modelo2o08cn-p003.pek3.example.net ansible_host=10.40.58.153

[kibana]
py-modelo2o08cn-p004.pek3.example.net ansible_host=10.40.58.154

[kibana]
py-modelaud05cn-p005.pek3.example.net ansible_host=10.40.58.116

[curator]
py-modelaud05cn-p005.pek3.example.net ansible_host=10.40.58.116

安装jdk

Logstash Elasticsearch 需要JDK8， 需要检查主机上是否已经安装JDK8

[root@py-modelo2o08cn-p003 ~]# java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

安装Elasticsearch

安装部署请参照官方文档 查找对应的版本， 下面片段为官方网站

Download and install the RPM manually

The RPM for Elasticsearch v7.4.2 can be downloaded from the website and installed as follows:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.2-x86_64.rpm
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.2-x86_64.rpm.sha512
shasum -a 512 -c elasticsearch-7.4.2-x86_64.rpm.sha512 
sudo rpm --install elasticsearch-7.4.2-x86_64.rpm

Download and install the Debian package manually

The Debian package for Elasticsearch v7.4.2 can be downloaded from the website and installed as follows:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.2-amd64.deb
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.2-amd64.deb.sha512
shasum -a 512 -c elasticsearch-7.4.2-amd64.deb.sha512 
sudo dpkg -i elasticsearch-7.4.2-amd64.deb

Running Elasticsearch with systemd

To configure Elasticsearch to start automatically when the system boots up, run the following commands:

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service

Elasticsearch can be started and stopped as follows:

sudo systemctl start elasticsearch.service
sudo systemctl stop elasticsearch.service

These commands provide no feedback as to whether Elasticsearch was started successfully or not. Instead, this information will be written in the log files located in /var/log/elasticsearch/.

By default the Elasticsearch service doesn’t log information in the systemd journal. To enable journalctl logging, the --quiet option must be removed from the ExecStart command line in the elasticsearch.service file.

When systemd logging is enabled, the logging information are available using the journalctl commands:

To tail the journal:

sudo journalctl -f

To list journal entries for the elasticsearch service:

sudo journalctl --unit elasticsearch

To list journal entries for the elasticsearch service starting from a given time:

sudo journalctl --unit elasticsearch --since  "2016-10-30 18:17:16"

Check man journalctl or https://www.freedesktop.org/software/systemd/man/journalctl.html for more command line options.

安装Logstash

详细查看官方文档Install Logstash

YUM 方式安装

Download and install the public signing key:

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Add the following in your /etc/yum.repos.d/ directory in a file with a .repo suffix, for example logstash.repo

[logstash-6.x]
name=Elastic repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

And your repository is ready for use. You can install it with:

sudo yum install logstash

Running Logstash with systemd

To configure Logstash to start automatically when the system boots up, run the following commands:

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable logstash.service

Logstash can be started and stopped as follows:

sudo systemctl start logstash.service
sudo systemctl stop logstash.service

部署Kibana

更多的安装步骤请查看官方文档Install Kibana

Download and install the RPM manually

The RPM for Kibana v6.5.4 can be downloaded from the website and installed as follows:

wget https://artifacts.elastic.co/downloads/kibana/kibana-6.5.4-x86_64.rpm
shasum -a 512 kibana-6.5.4-x86_64.rpm 
sudo rpm --install kibana-6.5.4-x86_64.rpm

Running Kibana with systemd

To configure Kibana to start automatically when the system boots up, run the following commands:

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable kibana.service

Kibana can be started and stopped as follows:

sudo systemctl start kibana.service
sudo systemctl stop kibana.service

These commands provide no feedback as to whether Kibana was started successfully or not. Instead, this information will be written in the log files located in /var/log/kibana/.

安装Cerebro

cerebro支持节点状态监控 服务器磁盘空间，cpu，load，索引数量大小 ，数据分布位置等监控。 支持查询。项目托管于github

Download and install the RPM manually

rpm 包从github上进行下载， 操作命令如下：

wget https://github.com/lmenezes/cerebro/releases/download/v0.8.5/cerebro-0.8.5-1.noarch.rpm 
rpm --install cerebro-0.8.5-1.noarch.rpm

Running Cerebro with systemd

To configure cerebro to start automatically when the system boots up, run the following commands:

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable cerebro.service

cerebro can be started and stopped as follows:

sudo systemctl start cerebro.service
sudo systemctl stop cerebro.service

安装Curator

github 官方网站

• curator是一个用于管理es中的索引和快照的工具。

• curator是用Python写的，可以作为命令行工具，也能作为python的API。

Repository Configuration

Add the following in your /etc/yum.repos.d/ directory in a file with a .repo suffix, for example curator.repo

The repositories are different for CentOS/RHEL 6 and 7 due to library and path differences. Be sure to use the correct version for your system!

RHEL/CentOS 6:

[curator-5]
name=CentOS/RHEL 6 repository for Elasticsearch Curator 5.x packages
baseurl=https://packages.elastic.co/curator/5/centos/6
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

RHEL/CentOS 7:

[curator-5]
name=CentOS/RHEL 7 repository for Elasticsearch Curator 5.x packages
baseurl=https://packages.elastic.co/curator/5/centos/7
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

Binary Package Installation

The RPM version of Curator is a binary version. What this really means is that the source is frozen, and all required libraries are bundled with the curator binary, so there are no conflicts.

There are separate binary packages for RedHat variants. The binary packages resulting from the creation process have been tested on CentOS 6 & 7, with a different binary for each. They may work on similar variants and/or derivatives, but they have not been tested.

yum install elasticsearch-curator

This will install the necessary files into /opt/elasticsearch-curator and make a symlink at /usr/bin/curator that points to the curator binary in the aforementioned directory.