云网牛站
所在位置:首页 > Linux云服务器 > 在Ubuntu 18.04上安装三节点Elasticsearch集群的步骤

在Ubuntu 18.04上安装三节点Elasticsearch集群的步骤

2019-01-27 10:39:25作者:戴进稿源:云网牛站

本文将介绍在Ubuntu 18.04系统上安装部署三节点Elasticsearch集群,以确保高负载的高可用性和可扩展性,其中一个节点是主节点,另外两个节点作为两个节点。以下是具体操作步骤。

 

在Ubuntu 18.04上安装三节点Elasticsearch集群

我的设置有三个Elasticsearch节点,其中包含以下主机名:

10.10.5.10 elk-node-1

10.10.5.11 elk-node-2

10.10.5.12 elk-node-3

在每个服务器/etc/hosts文件中添加这些行。

以下仅供参考,该实验基于以下Vagrantfile:

# -*- mode: ruby -*-

# vim: set ft=ruby :

ENV['VAGRANT_DEFAULT_PROVIDER'] = 'libvirt'

# Check required plugins

REQUIRED_PLUGINS_LIBVIRT = %w(vagrant-libvirt)

exit unless REQUIRED_PLUGINS_LIBVIRT.all? do |plugin|

Vagrant.has_plugin?(plugin) || (

puts "The #{plugin} plugin is required. Please install it with:"

puts "$ vagrant plugin install #{plugin}"

false

)

end

Vagrant.configure("2") do |config|

config.vm.define "elk-01" do |node|

node.vm.hostname = "elk-01"

node.vm.box = "generic/ubuntu1804"

node.vm.box_check_update = false

#node.vm.synced_folder '.', '/vagrant', :disabled => true

node.vm.network "private_network", ip: "10.10.5.10"

node.vm.provider :libvirt do |domain|

domain.memory = 1024

domain.storage :file, :size => '10G'

end

end

config.vm.define "elk-02" do |node|

node.vm.hostname = "elk-02"

node.vm.box = "generic/ubuntu1804"

node.vm.box_check_update = false

#node.vm.synced_folder '.', '/vagrant', :disabled => true

node.vm.network "private_network", ip: "10.10.5.11"

node.vm.provider :libvirt do |domain|

domain.memory = 1024

domain.storage :file, :size => '10G'

end

end

config.vm.define "elk-03" do |node|

node.vm.hostname = "elk-03"

node.vm.box = "generic/ubuntu1804"

node.vm.box_check_update = false

#node.vm.synced_folder '.', '/vagrant', :disabled => true

node.vm.network "private_network", ip: "10.10.5.12"

node.vm.provider :libvirt do |domain|

domain.memory = 1024

domain.storage :file, :size => '10G'

end

end

end

请注意,我添加了一个10GB的辅助磁盘,用于存储Elasticsearch数据,理想的Elasticsearch设置将具有更大的磁盘大小,具体取决于你的存储要求,启动所有节点并使用正确的主机名和IP地址配置文件/etc/hosts。

 

第1步:为Elasticsearch数据创建分区

建议不要在根分区/下存储Elasticsearch数据,创建自己的分区并安装它,我们稍后会将数据路径更改为此分区安装点:

# lsblk  | grep vda

vda            252:0   0  10G  0 disk

为辅助磁盘创建GPT分区表,它可以是多个磁盘:

sudo parted -s -a optimal -- /dev/vda mklabel gpt

sudo parted -s -a optimal -- /dev/vda mkpart primary 0% 100%

sudo parted -s -- /dev/vda align-check optimal 1

然后创建LVM卷,这将使扩展分区变得容易:

$ sudo pvcreate  /dev/vda1

Physical volume "/dev/vda1" successfully created.

$ sudo vgcreate vg10 /dev/vda1

Volume group "vg10" successfully created

$ sudo lvcreate -n elasticsearch -l 100%FREE vg10

Logical volume "elasticsearch" created

在创建的逻辑卷上创建ext4文件系统:

$ sudo mkfs.ext4 /dev/mapper/vg10-elasticsearch 

mke2fs 1.44.1 (27-01-2019)

Creating filesystem with 2620416 4k blocks and 655360 inodes

Filesystem UUID: 5d769fb5-68fc-4ee3-a5fa-0f6b5cda5758

Superblock backups stored on blocks:

32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done

Writing inode tables: done

Creating journal (16384 blocks): done

Writing superblocks and filesystem accounting information: done

 

第2步:在Ubuntu 18.04上安装Elasticsearch

导入GPG密钥:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

然后添加Elasticsearch APT存储库,导入GPG密钥后,添加apt存储库,以便从以下位置安装Elasticsearch软件包。

1、对于Elasticsearch 5.x,请在下面添加repo:

echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list

2、对于Elasticsearch 6.x,请在下面添加repo:

echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

 

第3步:安装OpenJDK

Elasticsearch依赖于Java,需要先安装OpenJDK才能继续:

sudo apt update

sudo apt install apt-transport-https openjdk-8-jre-headless

参考:在Ubuntu 18.04.1系统中源代码编译安装OpenJDK 8的方法

 

第4步:安装Elasticsearch

现在运行apt-get update然后安装elasticsearch包:

sudo apt update

sudo apt install elasticsearch

安装Elasticsearch后,在文件/etc/fstab上配置挂载点,如果没有为Elasticsearch配置单独的分区,请忽略此项:

echo "/dev/mapper/vg10-elasticsearch /var/lib/elasticsearch/data ext4 defaults 0 0" | sudo tee -a /etc/fstab

mkdir /var/lib/elasticsearch/data

挂载分区并更改权限:

sudo mount -a 

sudo chown -R elasticsearch:elasticsearch /var/lib/elasticsearch/data

sudo chmod -R 775 /var/lib/elasticsearch/data

请注意,JVM的默认最小内存集为2gb,如果服务器的内存较小,请更改此值:

$ sudo vim /etc/elasticsearch/jvm.options

更改:

-Xms2g

-Xmx2g

并设置最小和最大内存分配的值,例如,将值设置为512mb的ram,使用:

-Xms512m

-Xmx512m

 

第5步:配置Elasticsearch集群

修改配置后,配置Elasticsearch集群:

sudo vim /etc/elasticsearch/elasticsearch.yml

首先设置集群名称(所有节点上都相同):

# ----------------- Cluster -----------------

#

# Use a descriptive name for your cluster:

#

cluster.name: graylog-cluster

#

在每个节点上,为节点设置描述性名称。

1、在节点1上:

# ----------------- Node -----------------

#

# Use a descriptive name for the node:

#

node.name: elk-node-1

#

2、在节点2上:

# ----------------- Node -----------------

#

# Use a descriptive name for the node:

#

node.name: elk-node-2

#

3、在节点3上:

# ----------------- Node -----------------

#

# Use a descriptive name for the node:

#

node.name: elk-node-3

#

将path.data设置修改为/var/lib/elasticsearch/data/:

# ----------------- Paths -----------------

#

# Path to directory where to store the data (separate multiple locations by comma):

#

path.data: /var/lib/elasticsearch/data

#

将绑定地址设置为特定IP。

1、在节点1上:

# ----------------- Network -----------------

#

# Set the bind address to a specific IP (IPv4 or IPv6):

#

network.host: 10.10.5.10

# Change accordingly for other nodes

2、在节点2上:

# ----------------- Network -----------------

#

# Set the bind address to a specific IP (IPv4 or IPv6):

#

network.host: 10.10.5.11

# Change accordingly for other nodes

3、在节点3上:

# ----------------- Network -----------------

#

# Set the bind address to a specific IP (IPv4 or IPv6):

#

network.host: 10.10.5.12

# Change accordingly for other nodes

通过指定所有节点IP地址(在所有节点上设置)来设置发现:

# ----------------- Discovery -----------------

#

# Pass an initial list of hosts to perform discovery when new node is started:

# The default list of hosts is ["127.0.0.1", "[::1]"]

#

discovery.zen.ping.unicast.hosts: ["10.10.5.10", "10.10.5.11", "10.10.5.12"]

#

指定符合条件的主节点数(在所有节点上设置):

discovery.zen.minimum_master_nodes: 2

此数字来自:[Master Eligible Node) / 2 + 1]

1]、将节点1定义为符合主要条件:

node.master: true

2]、将节点2和3定义为数据节点:

node.data: true

如果有活动防火墙,请打开端口9200和9300:

sudo ufw allow 9200

sudo ufw allow 9300

启动elasticsearch并启用服务以在启动时启动:

sudo systemctl daemon-reload

sudo systemctl enable elasticsearch.service

sudo systemctl restart elasticsearch.service

 

第6步:测试Elasticsearch集群

确保群集状态为绿色(green),这意味着它没问题,是OK的:

$ curl http://10.10.5.10:9200/_cluster/health?pretty

{

"cluster_name" : "graylog-cluster",

"status" : "green",

"timed_out" : false,

"number_of_nodes" : 3,

"number_of_data_nodes" : 3,

"active_primary_shards" : 0,

"active_shards" : 0,

"relocating_shards" : 0,

"initializing_shards" : 0,

"unassigned_shards" : 0,

"delayed_unassigned_shards" : 0,

"number_of_pending_tasks" : 0,

"number_of_in_flight_fetch" : 0,

"task_max_waiting_in_queue_millis" : 0,

"active_shards_percent_as_number" : 100.0

}

检查当前主节点:

$ curl http://10.10.5.10:9200/_cat/master

84GGSOjaRtCBuXrBFKXMcw 10.10.5.10 10.10.5.10 elk-node-1

检查工作节点:

$ curl http://10.10.5.10:9200/_cat/nodes?h=ip,port,heapPercent,name

10.10.5.10 9300 10 elk-node-1

10.10.5.11 9300 11 elk-node-2

10.10.5.12 9300 12 elk-node-3

现在在Ubuntu 18.04系统上有一个可用的Elasticsearch集群,要增加群集大小,使用此处提供的步骤配置额外节点,它加入群集而不会出现问题。

 

相关主题

在ubuntu中安装ELK Stack(Elasticsearch,Logstash和Kibana)

精选文章
热门文章