Skip to content

Commit 2c1c4f6

Browse files
pirespires
committed
Upgraded Elasticsearch to version 2.4.1 and Kubernetes to 1.4.3.
1 parent eaa9b3d commit 2c1c4f6

File tree

4 files changed

+76
-74
lines changed

4 files changed

+76
-74
lines changed

‎README.md

Lines changed: 73 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# kubernetes-elasticsearch-cluster
2-
Elasticsearch (2.3.4) cluster on top of Kubernetes made easy.
2+
Elasticsearch (2.4.1) cluster on top of Kubernetes made easy.
33

44
Elasticsearch best-practices recommend to separate nodes in three roles:
55
* `Master` nodes - intended for clustering management only, no data, no HTTP API
@@ -16,7 +16,7 @@ Given this, I'm hereby making possible for you to scale as needed. For instance,
1616

1717
## Pre-requisites
1818

19-
* Kubernetes cluster (tested with v1.3.5 on top of [Vagrant + CoreOS](https://github.com/pires/kubernetes-vagrant-coreos-cluster))
19+
* Kubernetes cluster (tested with v1.4.3 on top of [Vagrant + CoreOS](https://github.com/pires/kubernetes-vagrant-coreos-cluster))
2020
* `kubectl` configured to access your cluster master API Server
2121

2222
## Build images (optional)
@@ -51,36 +51,36 @@ Now, I leave up to you how to validate the cluster, but a first step is to wait
5151

5252
```
5353
$ kubectl get svc,deployment,pods
54-
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
55-
elasticsearch 10.100.166.126 <pending> 9200/TCP 36m
56-
elasticsearch-discovery 10.100.103.179 <none> 9300/TCP 36m
57-
kubernetes 10.100.0.1 <none> 443/TCP 42m
58-
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
59-
es-client 1 1 1 1 1m
60-
es-data 1 1 1 1 51s
61-
es-master 1 1 1 1 8m
62-
NAME READY STATUS RESTARTS AGE
63-
es-client-1532011931-ik7ld 1/1 Running 0 1m
64-
es-data-2129027932-9wyey 1/1 Running 0 51s
65-
es-master-3368190183-4ux7v 1/1 Running 0 8m
66-
```
67-
68-
```
69-
$ kubectl logs -f es-master-3368190183-4ux7v
70-
[2016-08-22 11:50:46,704][INFO ][node ] [The Destroyer] version[2.3.5], pid[11], build[90f439f/2016-07-27T10:36:52Z]
71-
[2016-08-22 11:50:46,705][INFO ][node ] [The Destroyer] initializing ...
72-
[2016-08-22 11:50:47,764][INFO ][plugins ] [The Destroyer] modules [reindex, lang-expression, lang-groovy], plugins [cloud-kubernetes], sites []
73-
[2016-08-22 11:50:47,824][INFO ][env ] [The Destroyer] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [13.5gb], net total_space [15.5gb], spins? [possibly], types [ext4]
74-
[2016-08-22 11:50:47,825][INFO ][env ] [The Destroyer] heap size [503.6mb], compressed ordinary object pointers [true]
75-
[2016-08-22 11:50:52,800][INFO ][node ] [The Destroyer] initialized
76-
[2016-08-22 11:50:52,800][INFO ][node ] [The Destroyer] starting ...
77-
[2016-08-22 11:50:53,053][INFO ][transport ] [The Destroyer] publish_address {10.244.72.2:9300}, bound_addresses {10.244.72.2:9300}
78-
[2016-08-22 11:50:53,067][INFO ][discovery ] [The Destroyer] myesdb/eO4Xb_xQSBGUvchOvMR_jA
79-
[2016-08-22 11:50:58,757][INFO ][cluster.service ] [The Destroyer] new_master {The Destroyer}{eO4Xb_xQSBGUvchOvMR_jA}{10.244.72.2}{10.244.72.2:9300}{data=false, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
80-
[2016-08-22 11:50:58,767][INFO ][node ] [The Destroyer] started
81-
[2016-08-22 11:50:58,806][INFO ][gateway ] [The Destroyer] recovered [0] indices into cluster_state
82-
[2016-08-22 11:57:33,414][INFO ][cluster.service ] [The Destroyer] added {{St. John Allerdyce}{6gSJOytcRXG_zYkBksGDEA}{10.244.74.2}{10.244.74.2:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{St. John Allerdyce}{6gSJOytcRXG_zYkBksGDEA}{10.244.74.2}{10.244.74.2:9300}{data=false, master=false}])
83-
[2016-08-22 11:58:21,927][INFO ][cluster.service ] [The Destroyer] added {{Amergin}{JdCvogNSR5KDuawdIP5QDQ}{10.244.83.2}{10.244.83.2:9300}{master=false},}, reason: zen-disco-join(join from node[{Amergin}{JdCvogNSR5KDuawdIP5QDQ}{10.244.83.2}{10.244.83.2:9300}{master=false}])
54+
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
55+
svc/elasticsearch 10.100.59.70 <pending> 9200/TCP 3m
56+
svc/elasticsearch-discovery 10.100.206.207 <none> 9300/TCP 3m
57+
svc/kubernetes 10.100.0.1 <none> 443/TCP 17h
58+
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
59+
deploy/es-client 1 1 1 1 2m
60+
deploy/es-data 1 1 1 1 51s
61+
deploy/es-master 1 1 1 1 3m
62+
NAME READY STATUS RESTARTS AGE
63+
po/es-client-893446138-z6v0v 1/1 Running 0 2m
64+
po/es-data-2462033339-dfede 1/1 Running 0 51s
65+
po/es-master-3681665862-0p6wc 1/1 Running 0 3m
66+
```
67+
68+
```
69+
$ kubectl logs -f es-master-3681665862-0p6wc
70+
[2016-10-19 10:27:31,629][INFO ][node ] [Cottonmouth] version[2.4.1], pid[11], build[c67dc32/2016-09-27T18:57:55Z]
71+
[2016-10-19 10:27:31,630][INFO ][node ] [Cottonmouth] initializing ...
72+
[2016-10-19 10:27:32,844][INFO ][plugins ] [Cottonmouth] modules [reindex, lang-expression, lang-groovy], plugins [cloud-kubernetes], sites []
73+
[2016-10-19 10:27:32,923][INFO ][env ] [Cottonmouth] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [13.1gb], net total_space [15.5gb], spins? [possibly], types [ext4]
74+
[2016-10-19 10:27:32,924][INFO ][env ] [Cottonmouth] heap size [247.6mb], compressed ordinary object pointers [true]
75+
[2016-10-19 10:27:38,469][INFO ][node ] [Cottonmouth] initialized
76+
[2016-10-19 10:27:38,475][INFO ][node ] [Cottonmouth] starting ...
77+
[2016-10-19 10:27:38,666][INFO ][transport ] [Cottonmouth] publish_address {10.244.36.2:9300}, bound_addresses {10.244.36.2:9300}
78+
[2016-10-19 10:27:38,676][INFO ][discovery ] [Cottonmouth] myesdb/B1gZ7EA0T1C0HBmQHqa-9w
79+
[2016-10-19 10:27:43,837][INFO ][cluster.service ] [Cottonmouth] new_master {Cottonmouth}{B1gZ7EA0T1C0HBmQHqa-9w}{10.244.36.2}{10.244.36.2:9300}{data=false, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
80+
[2016-10-19 10:27:43,854][INFO ][node ] [Cottonmouth] started
81+
[2016-10-19 10:27:43,912][INFO ][gateway ] [Cottonmouth] recovered [0] indices into cluster_state
82+
[2016-10-19 10:29:01,809][INFO ][cluster.service ] [Cottonmouth] added {{Steve Rogers}{lgXWvPt-TbKF8ZoWsH0QwQ}{10.244.95.2}{10.244.95.2:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{Steve Rogers}{lgXWvPt-TbKF8ZoWsH0QwQ}{10.244.95.2}{10.244.95.2:9300}{data=false, master=false}])
83+
[2016-10-19 10:30:32,536][INFO ][cluster.service ] [Cottonmouth] added {{Man-Thing}{9ZKhq0u5R4-Ry1mhEwcFJA}{10.244.18.2}{10.244.18.2:9300}{master=false},}, reason: zen-disco-join(join from node[{Man-Thing}{9ZKhq0u5R4-Ry1mhEwcFJA}{10.244.18.2}{10.244.18.2:9300}{master=false}])
8484
```
8585

8686
As you can assert, the cluster is up and running. Easy, wasn't it?
@@ -99,41 +99,42 @@ Did it work?
9999

100100
```
101101
$ kubectl get deployments,pods
102-
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
103-
es-client 2 2 2 2 3m
104-
es-data 2 2 2 2 3m
105-
es-master 3 3 3 3 10m
106-
NAME READY STATUS RESTARTS AGE
107-
es-client-1532011931-ik7ld 1/1 Running 2 3m
108-
es-client-1532011931-ps1ly 1/1 Running 0 1m
109-
es-data-2129027932-3asba 1/1 Running 1 1m
110-
es-data-2129027932-9wyey 1/1 Running 0 3m
111-
es-master-3368190183-4ux7v 1/1 Running 0 10m
112-
es-master-3368190183-n7tcq 1/1 Running 0 1m
113-
es-master-3368190183-v8yxv 1/1 Running 0 1m
102+
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
103+
deploy/es-client 2 2 2 2 5m
104+
deploy/es-data 2 2 2 2 4m
105+
deploy/es-master 3 3 3 3 6m
106+
NAME READY STATUS RESTARTS AGE
107+
po/es-client-893446138-rdb9y 1/1 Running 0 1m
108+
po/es-client-893446138-z6v0v 1/1 Running 0 5m
109+
po/es-data-2462033339-dfede 1/1 Running 0 4m
110+
po/es-data-2462033339-scgek 1/1 Running 0 30s
111+
po/es-master-3681665862-0p6wc 1/1 Running 0 6m
112+
po/es-master-3681665862-rljxr 1/1 Running 0 1m
113+
po/es-master-3681665862-zkhwi 1/1 Running 0 1m
114114
```
115115

116116
Let's take another look at the logs of one of the Elasticsearch `master` nodes:
117117

118118
```
119-
$ kubectl logs -f es-master-3368190183-4ux7v
120-
[2016-08-22 11:50:46,704][INFO ][node ] [The Destroyer] version[2.3.5], pid[11], build[90f439f/2016-07-27T10:36:52Z]
121-
[2016-08-22 11:50:46,705][INFO ][node ] [The Destroyer] initializing ...
122-
[2016-08-22 11:50:47,764][INFO ][plugins ] [The Destroyer] modules [reindex, lang-expression, lang-groovy], plugins [cloud-kubernetes], sites []
123-
[2016-08-22 11:50:47,824][INFO ][env ] [The Destroyer] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [13.5gb], net total_space [15.5gb], spins? [possibly], types [ext4]
124-
[2016-08-22 11:50:47,825][INFO ][env ] [The Destroyer] heap size [503.6mb], compressed ordinary object pointers [true]
125-
[2016-08-22 11:50:52,800][INFO ][node ] [The Destroyer] initialized
126-
[2016-08-22 11:50:52,800][INFO ][node ] [The Destroyer] starting ...
127-
[2016-08-22 11:50:53,053][INFO ][transport ] [The Destroyer] publish_address {10.244.72.2:9300}, bound_addresses {10.244.72.2:9300}
128-
[2016-08-22 11:50:53,067][INFO ][discovery ] [The Destroyer] myesdb/eO4Xb_xQSBGUvchOvMR_jA
129-
[2016-08-22 11:50:58,757][INFO ][cluster.service ] [The Destroyer] new_master {The Destroyer}{eO4Xb_xQSBGUvchOvMR_jA}{10.244.72.2}{10.244.72.2:9300}{data=false, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
130-
[2016-08-22 11:50:58,767][INFO ][node ] [The Destroyer] started
131-
[2016-08-22 11:50:58,806][INFO ][gateway ] [The Destroyer] recovered [0] indices into cluster_state
132-
[2016-08-22 11:57:33,414][INFO ][cluster.service ] [The Destroyer] added {{St. John Allerdyce}{6gSJOytcRXG_zYkBksGDEA}{10.244.74.2}{10.244.74.2:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{St. John Allerdyce}{6gSJOytcRXG_zYkBksGDEA}{10.244.74.2}{10.244.74.2:9300}{data=false, master=false}])
133-
[2016-08-22 11:58:21,927][INFO ][cluster.service ] [The Destroyer] added {{Amergin}{JdCvogNSR5KDuawdIP5QDQ}{10.244.83.2}{10.244.83.2:9300}{master=false},}, reason: zen-disco-join(join from node[{Amergin}{JdCvogNSR5KDuawdIP5QDQ}{10.244.83.2}{10.244.83.2:9300}{master=false}])
134-
[2016-08-22 12:04:31,859][INFO ][cluster.service ] [The Destroyer] added {{Beta Ray Bill}{oO8X5HIZQbeIb4guf8Yh_w}{10.244.74.2}{10.244.74.2:9300}{data=false, master=true},}, reason: zen-disco-join(join from node[{Beta Ray Bill}{oO8X5HIZQbeIb4guf8Yh_w}{10.244.74.2}{10.244.74.2:9300}{data=false, master=true}])
135-
[2016-08-22 12:04:32,034][INFO ][cluster.service ] [The Destroyer] added {{Tiger Shark}{T59rILwORPGvjmszgSBaCA}{10.244.83.3}{10.244.83.3:9300}{data=false, master=true},}, reason: zen-disco-join(join from node[{Tiger Shark}{T59rILwORPGvjmszgSBaCA}{10.244.83.3}{10.244.83.3:9300}{data=false, master=true}])
136-
[2016-08-22 12:05:50,791][INFO ][cluster.service ] [The Destroyer] added {{Doug and Jerry}{4j8AK5lvQPeXLGUeP7G17w}{10.244.74.3}{10.244.74.3:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{Doug and Jerry}{4j8AK5lvQPeXLGUeP7G17w}{10.244.74.3}{10.244.74.3:9300}{data=false, master=false}])
119+
$ kubectl logs -f es-master-3681665862-0p6wc
120+
[2016-10-19 10:27:31,629][INFO ][node ] [Cottonmouth] version[2.4.1], pid[11], build[c67dc32/2016-09-27T18:57:55Z]
121+
[2016-10-19 10:27:31,630][INFO ][node ] [Cottonmouth] initializing ...
122+
[2016-10-19 10:27:32,844][INFO ][plugins ] [Cottonmouth] modules [reindex, lang-expression, lang-groovy], plugins [cloud-kubernetes], sites []
123+
[2016-10-19 10:27:32,923][INFO ][env ] [Cottonmouth] using [1] data paths, mounts [[/data (/dev/sda9)]], net usable_space [13.1gb], net total_space [15.5gb], spins? [possibly], types [ext4]
124+
[2016-10-19 10:27:32,924][INFO ][env ] [Cottonmouth] heap size [247.6mb], compressed ordinary object pointers [true]
125+
[2016-10-19 10:27:38,469][INFO ][node ] [Cottonmouth] initialized
126+
[2016-10-19 10:27:38,475][INFO ][node ] [Cottonmouth] starting ...
127+
[2016-10-19 10:27:38,666][INFO ][transport ] [Cottonmouth] publish_address {10.244.36.2:9300}, bound_addresses {10.244.36.2:9300}
128+
[2016-10-19 10:27:38,676][INFO ][discovery ] [Cottonmouth] myesdb/B1gZ7EA0T1C0HBmQHqa-9w
129+
[2016-10-19 10:27:43,837][INFO ][cluster.service ] [Cottonmouth] new_master {Cottonmouth}{B1gZ7EA0T1C0HBmQHqa-9w}{10.244.36.2}{10.244.36.2:9300}{data=false, master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
130+
[2016-10-19 10:27:43,854][INFO ][node ] [Cottonmouth] started
131+
[2016-10-19 10:27:43,912][INFO ][gateway ] [Cottonmouth] recovered [0] indices into cluster_state
132+
[2016-10-19 10:29:01,809][INFO ][cluster.service ] [Cottonmouth] added {{Steve Rogers}{lgXWvPt-TbKF8ZoWsH0QwQ}{10.244.95.2}{10.244.95.2:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{Steve Rogers}{lgXWvPt-TbKF8ZoWsH0QwQ}{10.244.95.2}{10.244.95.2:9300}{data=false, master=false}])
133+
[2016-10-19 10:30:32,536][INFO ][cluster.service ] [Cottonmouth] added {{Man-Thing}{9ZKhq0u5R4-Ry1mhEwcFJA}{10.244.18.2}{10.244.18.2:9300}{master=false},}, reason: zen-disco-join(join from node[{Man-Thing}{9ZKhq0u5R4-Ry1mhEwcFJA}{10.244.18.2}{10.244.18.2:9300}{master=false}])
134+
[2016-10-19 10:32:29,849][INFO ][cluster.service ] [Cottonmouth] added {{Marduk Kurios}{5_Ds0dGvTqGNLCX6MKjWZQ}{10.244.95.3}{10.244.95.3:9300}{data=false, master=true},}, reason: zen-disco-join(join from node[{Marduk Kurios}{5_Ds0dGvTqGNLCX6MKjWZQ}{10.244.95.3}{10.244.95.3:9300}{data=false, master=true}])
135+
[2016-10-19 10:32:30,299][INFO ][cluster.service ] [Cottonmouth] added {{Wizard}{5V2ss-97TcmfcFqzMH73jw}{10.244.18.3}{10.244.18.3:9300}{data=false, master=true},}, reason: zen-disco-join(join from node[{Wizard}{5V2ss-97TcmfcFqzMH73jw}{10.244.18.3}{10.244.18.3:9300}{data=false, master=true}])
136+
[2016-10-19 10:32:50,862][INFO ][cluster.service ] [Cottonmouth] added {{Vavavoom}{LuomxPAjSKuGSEBL1bgKeQ}{10.244.36.3}{10.244.36.3:9300}{data=false, master=false},}, reason: zen-disco-join(join from node[{Vavavoom}{LuomxPAjSKuGSEBL1bgKeQ}{10.244.36.3}{10.244.36.3:9300}{data=false, master=false}])
137+
[2016-10-19 10:33:26,239][INFO ][cluster.service ] [Cottonmouth] added {{Karl Lykos}{BDJolawXRaCNqlBd3IgKig}{10.244.36.4}{10.244.36.4:9300}{master=false},}, reason: zen-disco-join(join from node[{Karl Lykos}{BDJolawXRaCNqlBd3IgKig}{10.244.36.4}{10.244.36.4:9300}{master=false}])
137138
138139
```
139140

@@ -143,28 +144,29 @@ $ kubectl logs -f es-master-3368190183-4ux7v
143144

144145
```
145146
$ kubectl get svc elasticsearch
146-
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
147-
elasticsearch 10.100.166.126 <pending> 9200/TCP 11m
147+
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
148+
elasticsearch 10.100.59.70 <pending> 9200/TCP 7m
148149
```
149150

150151
From any host on your cluster (that's running `kube-proxy`), run:
151152

152153
```
153-
curl http://10.100.166.126:9200
154+
curl http://10.100.59.70:9200
154155
```
155156

156157
You should see something similar to the following:
157158

158159
```json
159160
{
160-
"name" : "Doug and Jerry",
161+
"name" : "Steve Rogers",
161162
"cluster_name" : "myesdb",
163+
"cluster_uuid" : "oEklUx7mRMOBvG7MiAMjEA",
162164
"version" : {
163-
"number" : "2.3.5",
164-
"build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f",
165-
"build_timestamp" : "2016-07-27T11:24:31Z",
165+
"number" : "2.4.1",
166+
"build_hash" : "c67dc32e24162035d18d6fe1e952c4cbcbe79d16",
167+
"build_timestamp" : "2016-09-27T18:57:55Z",
166168
"build_snapshot" : false,
167-
"lucene_version" : "5.5.0"
169+
"lucene_version" : "5.5.2"
168170
},
169171
"tagline" : "You Know, for Search"
170172
}
@@ -173,7 +175,7 @@ You should see something similar to the following:
173175
Or if you want to see cluster information:
174176

175177
```
176-
curl http://10.100.166.126:9200/_cluster/health?pretty
178+
curl http://10.100.59.70:9200/_cluster/health?pretty
177179
```
178180

179181
You should see something similar to the following:

‎es-client.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ spec:
1919
capabilities:
2020
add:
2121
- IPC_LOCK
22-
image: quay.io/pires/docker-elasticsearch-kubernetes:2.3.5_1
22+
image: quay.io/pires/docker-elasticsearch-kubernetes:2.4.1
2323
imagePullPolicy: Always
2424
env:
2525
- name: NAMESPACE

‎es-data.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ spec:
1919
capabilities:
2020
add:
2121
- IPC_LOCK
22-
image: quay.io/pires/docker-elasticsearch-kubernetes:2.3.5_1
22+
image: quay.io/pires/docker-elasticsearch-kubernetes:2.4.1
2323
imagePullPolicy: Always
2424
env:
2525
- name: NAMESPACE

‎es-master.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ spec:
1919
capabilities:
2020
add:
2121
- IPC_LOCK
22-
image: quay.io/pires/docker-elasticsearch-kubernetes:2.3.5_1
22+
image: quay.io/pires/docker-elasticsearch-kubernetes:2.4.1
2323
imagePullPolicy: Always
2424
env:
2525
- name: NAMESPACE

0 commit comments

Comments
 (0)