How to Install and Configure Elasticsearch Cluster with Multiple Nodes

In our previous elasticsearch tutorial, we discussed how to install and setup a stand-alone elasticsearch instance.

While stand-alone installation is good for dev/test, for production, it is recommended to setup elasticsearch cluster. Elasticsearch cluster has many advantages over stand-alone.

Advantages of Elastisearch Cluster

1. Distributed data: In cluster data is distributed, replicated to other server. So In case of failure of one node, data can be restored from replica node. It avoids single point of failure.
2. Dedicated Node roles: Every node has dedicated role assigned to it, that ensures specific role and role based load distribution hence increasing performance. Here are two important node roles
3. Data node: These nodes only stores data and do data related operations, search and data manipulation.
4. Master node: Master of all nodes, it holds responsibility of overall cluster, addition and removal of nodes from cluster, keeping track of alive nodes, master reselection in appropriate cases.
5. Scalability: Cluster model is easily scalable to multiple no of nodes. Thus increasing performance and reliability of elasticsearch.

In this tutorial, we’ll setup three node elasticsearch cluster.

Node 1 – Install and Configure

Let’s begin installing elasticsearch, I will show each and every steps during installation procedure

Follow our previous article How to install Elasticsearch and download elasticsearch zip archive, extract it on server or install it using deb repository.

Now open the Elasticsearch configuration file in extracted folder, or /etc/elasticsearch in case of installed directly from repo.

vim /etc/elasticsearch/elasticsearch.yml

Now we will have to configure it as cluster nodes, following entries need to be modified in the above file.

Set the cluster name:

cluster-name=TGS

Set the first node name:

node.name=tgs-1

Set whether this node will act as a master:

node.master=true

Set whether this node will act as a data-node:

node.data=true

Set first node’s ip and port:

network.host=192.168.101.51
http.port=9200

Cluster nodes discovery protocol and list of nodes:

discovery.zen.ping.unicast.hosts: ["192.168.101.51", "192.168.101.52","192.168.101.53"]

These are all the nodes part of cluster, this makes node discovery easy.

Now start elasticsearch server, following are important part of installation logs:

[INFO ][o.e.c.s.ClusterApplierService] [tgs-1] new_master {tgs-1}{7IqQvmfdSb66AFcP0u0AZQ}{OveP8d_SR0WIjVupZcW35g}{192.168.101.51}{192.168.101.51:9300}{ml.machine_memory=134986862592xpack.installed=trueml.max_open_jobs=20ml.enabled=true}reason: apply cluster state (from master [master {tgs-1}{7IqQvmfdSb66AFcP0u0AZQ}{OveP8d_SR0WIjVupZcW35g}{192.168.101.51}{192.168.101.51:9300}{ml.machine_memory=134986862592xpack.installed=trueml.max_open_jobs=20ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[INFO ][o.e.h.n.Netty4HttpServerTransport] [tgs-1] publish_address {192.168.101.51:9200}bound_addresses {192.168.101.51:9200}
[INFO ][o.e.n.Node               ] [tgs-1] started
[WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [tgs-1] Failed to clear cache for realms [[]]
[INFO ][o.e.l.LicenseService     ] [tgs-1] license [77777777-dddd-4444-aaaa-888888888888] mode [basic] - valid
[INFO ][o.e.g.GatewayService     ] [tgs-1] recovered [0] indices into cluster_state
[INFO ][o.e.c.s.MasterService    ] [tgs-1] zen-disco-node-join[{tgs-2}{S64iPm03Qtq0-bvJipk-_A}{eAChESS6TceLHHTHyfMU7g}{192.168.101.52}{192.168.101.52:9300}{ml.machine_memory=134986862592ml.max_open_jobs=20xpack.installed=trueml.enabled=true}]reason: added {{tgs-2}{S64iPm03Qtq0-bvJipk-_A}{eAChESS6TceLHHTHyfMU7g}{192.168.101.52}{192.168.101.52:9300}{ml.machine_memory=134986862592ml.max_open_jobs=20xpack.installed=trueml.enabled=true},}
[INFO ][o.e.c.s.ClusterApplierService] [tgs-1] added {{tgs-2}{S64iPm03Qtq0-bvJipk-_A}{eAChESS6TceLHHTHyfMU7g}{192.168.101.52}{192.168.101.52:9300}{ml.machine_memory=134986862592ml.max_open_jobs=20xpack.installed=trueml.enabled=true},}reason: apply cluster state (from master [master {tgs-1}{7IqQvmfdSb66AFcP0u0AZQ}{OveP8d_SR0WIjVupZcW35g}{192.168.101.51}{192.168.101.51:9300}{ml.machine_memory=134986862592xpack.installed=trueml.max_open_jobs=20ml.enabled=true} committed version [4] source [zen-disco-node-join[{tgs-2}{S64iPm03Qtq0-bvJipk-_A}{eAChESS6TceLHHTHyfMU7g}{192.168.101.52}{192.168.101.52:9300}{ml.machine_memory=134986862592ml.max_open_jobs=20xpack.installed=trueml.enabled=true}]]])

Node 2 – Install and Configure

login to second server, install elasticsearch or extract from downloaded zip archive

Edit configuration file accordingly.

Set the cluster name:

cluster-name=TGS

Set the second node name:

node.name=tgs-2

Set whether this node will act as a master:

node.master=true

Set whether this node will act as a data-node:

node.data=true

Set second node’s ip and port:

network.host=192.168.101.52
http.port=9200

Cluster nodes discovery protocol and list of nodes:

discovery.zen.ping.unicast.hosts: ["192.168.101.51", "192.168.101.52","192.168.101.53"]

These are all the nodes part of cluster, this makes node discovery easy.

Now start elasticsearch server, following are important part of installation logs

[INFO ][o.e.c.s.ClusterApplierService] [tgs-2] detected_master {tgs-1}
[WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [tgs-2] Failed to clear cache for realms [[]]
[INFO ][o.e.x.s.a.TokenService   ] [tgs-2] refresh keys
[INFO ][o.e.x.s.a.TokenService   ] [tgs-2] refreshed keys
[INFO ][o.e.l.LicenseService     ] [tgs-2] license [77777777-dddd-4444-aaaa-888888888888] mode [basic] - valid
[INFO ][o.e.h.n.Netty4HttpServerTransport] [tgs-2] publish_address {192.168.101.52:9200}, bound_addresses {192.168.101.52:9200}
[INFO ][o.e.n.Node               ] [tgs-2] started
[INFO ][o.e.c.s.ClusterApplierService] [tgs-2] added {{tgs-3}

Node 3 – Install and Configure

Login to third server and install elasticsearch or extract downloaded zip archive

Set the cluster name:

cluster-name=TGS

Set the third node name:

node.name=tgs-3

Set whether this node will act as a master:

node.master=true

Set whether this node will act as a data-node:

node.data=true

Set second node’s ip and port:

network.host=192.168.101.53
http.port=9200

Cluster nodes discovery protocol and list of nodes

discovery.zen.ping.unicast.hosts: ["192.168.101.51", "192.168.101.52","192.168.101.53"]

These are all the nodes part of cluster, this makes node discovery easy.

Now start elasticsearch server, following are important part of installation logs

[INFO ][o.e.c.s.ClusterApplierService] [tgs-3] detected_master {tgs-1}
[WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [tgs-3] Failed to clear cache for realms [[]]
[INFO ][o.e.x.s.a.TokenService   ] [tgs-3] refresh keys
[INFO ][o.e.x.s.a.TokenService   ] [tgs-3] refreshed keys
[INFO ][o.e.l.LicenseService     ] [tgs-3] license [77777777-dddd-4444-aaaa-888888888888] mode [basic] - valid
[INFO ][o.e.h.n.Netty4HttpServerTransport] [tgs-3] publish_address {192.168.101.53:9200}, bound_addresses {192.168.101.53:9200}
[INFO ][o.e.n.Node               ] [tgs-3] started

Verify Elasticsearch Cluster Status

Our cluster is setup and running, let’s verify this cluster setup, we could use elastic search Cluster API to check status of cluster (will cover in detail about cluster APIs in ES series articles)

curl -XGET 'http://localhost:9200/_cluster/state?pretty'

Output:

{
"cluster_name" : "TGS",
"cluster_uuid" : "vlt_bBfJRP268MFFA5XY4w",
"version" : 5,
"state_uuid" : "VxN1E51xRievom5n1BDiNQ",
"master_node" : "7IqQvmfdSb66AFcP0u0AZQ",
"blocks" : { },
"nodes" : {
"S64iPm03Qtq0-bvJipk-_A" : {
"name" : "tgs-2",
"ephemeral_id" : "eAChESS6TceLHHTHyfMU7g",
"transport_address" : "192.168.101.52:9300",
"attributes" : {
"ml.machine_memory" : "134986862592",
"ml.max_open_jobs" : "20",
"xpack.installed" : "true",
"ml.enabled" : "true"
}
},
"syxQbgRPS4q992NXkx3ptw" : {
"name" : "tgs-3",
"ephemeral_id" : "-uK6TjQ2Qz6o_6arfmRR3Q",
"transport_address" : "192.168.101.53:9300",
"attributes" : {
"ml.machine_memory" : "134986862592",
"ml.max_open_jobs" : "20",
"xpack.installed" : "true",
"ml.enabled" : "true"
}
},
"7IqQvmfdSb66AFcP0u0AZQ" : {
"name" : "tgs-1",
"ephemeral_id" : "OveP8d_SR0WIjVupZcW35g",
"transport_address" : "192.168.101.51:9300",
"attributes" : {
"ml.machine_memory" : "134986862592",
"xpack.installed" : "true",
"ml.max_open_jobs" : "20",
"ml.enabled" : "true"
}
}
},

This is the meaningful part of status API, it shows cluster, nodes, transport address and ids.

curl -XGET 'http://localhost:9200/_cluster/health'

Output:

{
"cluster_name" : "TGS",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 3,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}

The above output displays the overall health of the cluster and it’s current status is Green.