github/kubernetes
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Upgrade Hazelcast example with the latest hazelcast-kubernetes-bootstrapper 0.7.0, using Deployments instead of Replication Controller.

Browse Source
This commit is contained in:
Andrea Cosentino
2017-01-08 12:23:10 +01:00
parent 9807cd7d06
commit 9ba27727e2
5 changed files with 125 additions and 172 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/examples_test.go
View File

@@ -241,7 +241,7 @@ func TestExampleObjectSchemas(t *testing.T) {
"pod": &api.Pod{},
},
"../examples/storage/hazelcast": {
"hazelcast-controller": &api.ReplicationController{},
"hazelcast-deployment": &extensions.Deployment{},
"hazelcast-service": &api.Service{},
},
"../examples/meteor": {

206
examples/storage/hazelcast/README.md
View File

@@ -4,7 +4,7 @@ The following document describes the development of a _cloud native_ [Hazelcast]
Any topology changes are communicated and handled by Hazelcast nodes themselves.
This document also attempts to describe the core components of Kubernetes: _Pods_, _Services_, and _Replication Controllers_.
This document also attempts to describe the core components of Kubernetes: _Pods_, _Services_, and _Deployments_.
### Prerequisites
@@ -65,52 +65,41 @@ $ kubectl create -f examples/storage/hazelcast/hazelcast-service.yaml
The real power of Kubernetes and Hazelcast lies in easily building a replicated, resizable Hazelcast cluster.
In Kubernetes a _[Replication Controller](../../../docs/user-guide/replication-controller.md)_ is responsible for replicating sets of identical pods. Like a _Service_ it has a selector query which identifies the members of it's set. Unlike a _Service_ it also has a desired number of replicas, and it will create or delete _Pods_ to ensure that the number of _Pods_ matches up with it's desired state.
In Kubernetes a _[_Deployment_](../../../docs/user-guide/deployments.md)_ is responsible for replicating sets of identical pods. Like a _Service_ it has a selector query which identifies the members of its set. Unlike a _Service_ it also has a desired number of replicas, and it will create or delete _Pods_ to ensure that the number of _Pods_ matches up with its desired state.
Replication Controllers will "adopt" existing pods that match their selector query, so let's create a Replication Controller with a single replica to adopt our existing Hazelcast Pod.
Deployments will "adopt" existing pods that match their selector query, so let's create a Deployment with a single replica to adopt our existing Hazelcast Pod.
<!-- BEGIN MUNGE: EXAMPLE hazelcast-controller.yaml -->
```yaml
apiVersion: v1
kind: ReplicationController
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: hazelcast
labels:
name: hazelcast
name: hazelcast
spec:
replicas: 1
selector:
name: hazelcast
template:
metadata:
labels:
name: hazelcast
spec:
containers:
- resources:
limits:
cpu: 0.1
image: quay.io/pires/hazelcast-kubernetes:0.6.1
name: hazelcast
- name: hazelcast
image: quay.io/pires/hazelcast-kubernetes:0.8.0
imagePullPolicy: Always
env:
- name: "DNS_DOMAIN"
value: "cluster.local"
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- containerPort: 5701
name: hazelcast
- name: hazelcast
containerPort: 5701
```
[Download example](hazelcast-controller.yaml?raw=true)
[Download example](hazelcast-deployment.yaml?raw=true)
<!-- END MUNGE: EXAMPLE hazelcast-controller.yaml -->
There are a few things to note in this description. First is that we are running the `quay.io/pires/hazelcast-kubernetes` image, tag `0.5`. This is a `busybox` installation with JRE 8 Update 45. However it also adds a custom [`application`](https://github.com/pires/hazelcast-kubernetes-bootstrapper) that finds any Hazelcast nodes in the cluster and bootstraps an Hazelcast instance accordingly. The `HazelcastDiscoveryController` discovers the Kubernetes API Server using the built in Kubernetes discovery service, and then uses the Kubernetes API to find new nodes (more on this later).
You may also note that we tell Kubernetes that the container exposes the `hazelcast` port. Finally, we tell the cluster manager that we need 1 cpu core.
You may note that we tell Kubernetes that the container exposes the `hazelcast` port.
The bulk of the replication controller config is actually identical to the Hazelcast pod declaration above, it simply gives the controller a recipe to use when creating new pods. The other parts are the `selector` which contains the controller's selector query, and the `replicas` parameter which specifies the desired number of replicas, in this case 1.
@@ -119,127 +108,124 @@ Last but not least, we set `DNS_DOMAIN` environment variable according to your K
Create this controller:
```sh
$ kubectl create -f examples/storage/hazelcast/hazelcast-controller.yaml
$ kubectl create -f examples/storage/hazelcast/hazelcast-deployment.yaml
```
After the controller provisions successfully the pod, you can query the service endpoints:
```sh
$ kubectl get endpoints hazelcast -o json
{
"kind": "Endpoints",
"apiVersion": "v1",
"metadata": {
"name": "hazelcast",
"namespace": "default",
"selfLink": "/api/v1/namespaces/default/endpoints/hazelcast",
"uid": "094e507a-2700-11e5-abbc-080027eae546",
"resourceVersion": "4094",
"creationTimestamp": "2015-07-10T12:34:41Z",
"labels": {
"name": "hazelcast"
}
},
"subsets": [
{
"addresses": [
{
"ip": "10.244.37.3",
"targetRef": {
"kind": "Pod",
"namespace": "default",
"name": "hazelcast-nsyzn",
"uid": "f57eb6b0-2706-11e5-abbc-080027eae546",
"resourceVersion": "4093"
}
}
],
"ports": [
{
"port": 5701,
"protocol": "TCP"
}
]
}
]
}
$ kubectl get endpoints hazelcast -o yaml
apiVersion: v1
kind: Endpoints
metadata:
creationTimestamp: 2016-12-16T08:57:27Z
labels:
name: hazelcast
name: hazelcast
namespace: default
resourceVersion: "11360"
selfLink: /api/v1/namespaces/default/endpoints/hazelcast
uid: 46447198-70eb-11e6-940c-0800278ab84d
subsets:
- addresses:
- ip: 10.244.37.2
targetRef:
kind: Pod
name: hazelcast-1790698550-3heau
namespace: default
resourceVersion: "11359"
uid: c9c3febd-70eb-11e6-940c-0800278ab84d
ports:
- port: 5701
protocol: TCP
```
You can see that the _Service_ has found the pod created by the replication controller.
Now it gets even more interesting.
Let's scale our cluster to 2 pods:
Now it gets even more interesting. Let's scale our cluster to 2 pods:
```sh
$ kubectl scale rc hazelcast --replicas=2
$ kubectl scale deployment hazelcast --replicas 2
```
Now if you list the pods in your cluster, you should see two hazelcast pods:
```sh
$ kubectl get pods
$ kubectl get deployment,pods
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deploy/hazelcast 2 2 2 2 1m
NAME READY STATUS RESTARTS AGE
hazelcast-nanfb 1/1 Running 0 40s
hazelcast-nsyzn 1/1 Running 0 2m
kube-dns-xudrp 3/3 Running 0 1h
po/hazelcast-3980717115-k1xsk 1/1 Running 0 1m
po/hazelcast-3980717115-pbhbq 1/1 Running 0 22s
```
To prove that this all works, you can use the `log` command to examine the logs of one pod, for example:
```sh
$ kubectl log hazelcast-nanfb hazelcast
2015-07-10 13:26:34.443 INFO 5 --- [ main] com.github.pires.hazelcast.Application : Starting Application on hazelcast-nanfb with PID 5 (/bootstrapper.jar started by root in /)
2015-07-10 13:26:34.535 INFO 5 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@42cfcf1: startup date [Fri Jul 10 13:26:34 GMT 2015]; root of context hierarchy
2015-07-10 13:26:35.888 INFO 5 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup
2015-07-10 13:26:35.924 INFO 5 --- [ main] c.g.p.h.HazelcastDiscoveryController : Asking k8s registry at https://kubernetes.default.svc.cluster.local..
2015-07-10 13:26:37.259 INFO 5 --- [ main] c.g.p.h.HazelcastDiscoveryController : Found 2 pods running Hazelcast.
2015-07-10 13:26:37.404 INFO 5 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.5] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [10.244.77.3, 10.244.37.3]
2015-07-10 13:26:37.405 INFO 5 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.5] Prefer IPv4 stack is true.
2015-07-10 13:26:37.415 INFO 5 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.5] Picked Address[10.244.77.3]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
2015-07-10 13:26:37.852 INFO 5 --- [ main] com.hazelcast.spi.OperationService : [10.244.77.3]:5701 [someGroup] [3.5] Backpressure is disabled
2015-07-10 13:26:37.879 INFO 5 --- [ main] c.h.s.i.o.c.ClassicOperationExecutor : [10.244.77.3]:5701 [someGroup] [3.5] Starting with 2 generic operation threads and 2 partition operation threads.
2015-07-10 13:26:38.531 INFO 5 --- [ main] com.hazelcast.system : [10.244.77.3]:5701 [someGroup] [3.5] Hazelcast 3.5 (20150617 - 4270dc6) starting at Address[10.244.77.3]:5701
2015-07-10 13:26:38.532 INFO 5 --- [ main] com.hazelcast.system : [10.244.77.3]:5701 [someGroup] [3.5] Copyright (c) 2008-2015, Hazelcast, Inc. All Rights Reserved.
2015-07-10 13:26:38.533 INFO 5 --- [ main] com.hazelcast.instance.Node : [10.244.77.3]:5701 [someGroup] [3.5] Creating TcpIpJoiner
2015-07-10 13:26:38.534 INFO 5 --- [ main] com.hazelcast.core.LifecycleService : [10.244.77.3]:5701 [someGroup] [3.5] Address[10.244.77.3]:5701 is STARTING
2015-07-10 13:26:38.672 INFO 5 --- [ cached1] com.hazelcast.nio.tcp.SocketConnector : [10.244.77.3]:5701 [someGroup] [3.5] Connecting to /10.244.37.3:5701, timeout: 0, bind-any: true
2015-07-10 13:26:38.683 INFO 5 --- [ cached1] c.h.nio.tcp.TcpIpConnectionManager : [10.244.77.3]:5701 [someGroup] [3.5] Established socket connection between /10.244.77.3:59951
2015-07-10 13:26:45.699 INFO 5 --- [ration.thread-1] com.hazelcast.cluster.ClusterService : [10.244.77.3]:5701 [someGroup] [3.5]
kubectl logs -f hazelcast-39807171
15-k1xsk
2017-01-30 12:42:50.774 INFO 6 --- [ main] com.github.pires.hazelcast.Application : Starting Application on hazelcast-3980717115-k1xsk with PID 6 (/bootstrapper.jar started by root in /)
2017-01-30 12:42:50.781 INFO 6 --- [ main] com.github.pires.hazelcast.Application : No active profile set, falling back to default profiles: default
2017-01-30 12:42:50.852 INFO 6 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@14514713: startup date [Mon Jan 30 12:42:50 GMT 2017]; root of context hierarchy
2017-01-30 12:42:52.304 INFO 6 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup
2017-01-30 12:42:52.323 INFO 6 --- [ main] c.g.p.h.HazelcastDiscoveryController : Asking k8s registry at https://kubernetes.default.svc.cluster.local..
2017-01-30 12:42:52.857 INFO 6 --- [ main] c.g.p.h.HazelcastDiscoveryController : Found 1 pods running Hazelcast.
2017-01-30 12:42:52.990 INFO 6 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.7.5] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [10.244.9.2]
2017-01-30 12:42:52.990 INFO 6 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.7.5] Prefer IPv4 stack is true.
2017-01-30 12:42:53.002 INFO 6 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.7.5] Picked [10.244.9.2]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
2017-01-30 12:42:53.032 INFO 6 --- [ main] com.hazelcast.system : [10.244.9.2]:5701 [someGroup] [3.7.5] Hazelcast 3.7.5 (20170124 - 111f332) starting at [10.244.9.2]:5701
2017-01-30 12:42:53.032 INFO 6 --- [ main] com.hazelcast.system : [10.244.9.2]:5701 [someGroup] [3.7.5] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
2017-01-30 12:42:53.032 INFO 6 --- [ main] com.hazelcast.system : [10.244.9.2]:5701 [someGroup] [3.7.5] Configured Hazelcast Serialization version : 1
2017-01-30 12:42:53.343 INFO 6 --- [ main] c.h.s.i.o.impl.BackpressureRegulator : [10.244.9.2]:5701 [someGroup] [3.7.5] Backpressure is disabled
2017-01-30 12:42:54.273 INFO 6 --- [ main] com.hazelcast.instance.Node : [10.244.9.2]:5701 [someGroup] [3.7.5] Creating TcpIpJoiner
2017-01-30 12:42:54.507 INFO 6 --- [ main] c.h.s.i.o.impl.OperationExecutorImpl : [10.244.9.2]:5701 [someGroup] [3.7.5] Starting 2 partition threads
2017-01-30 12:42:54.508 INFO 6 --- [ main] c.h.s.i.o.impl.OperationExecutorImpl : [10.244.9.2]:5701 [someGroup] [3.7.5] Starting 3 generic threads (1 dedicated for priority tasks)
2017-01-30 12:42:54.525 INFO 6 --- [ main] com.hazelcast.core.LifecycleService : [10.244.9.2]:5701 [someGroup] [3.7.5] [10.244.9.2]:5701 is STARTING
2017-01-30 12:42:54.529 INFO 6 --- [ main] c.h.n.t.n.NonBlockingIOThreadingModel : [10.244.9.2]:5701 [someGroup] [3.7.5] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads
2017-01-30 12:42:54.578 INFO 6 --- [ main] com.hazelcast.cluster.impl.TcpIpJoiner : [10.244.9.2]:5701 [someGroup] [3.7.5]
Members [2] {
Member [10.244.37.3]:5701
Member [10.244.77.3]:5701 this
Members [1] {
Member [10.244.9.2]:5701 - f9cae801-59da-49d9-b8de-7719abb53844 this
}
2015-07-10 13:26:47.722 INFO 5 --- [ main] com.hazelcast.core.LifecycleService : [10.244.77.3]:5701 [someGroup] [3.5] Address[10.244.77.3]:5701 is STARTED
2015-07-10 13:26:47.723 INFO 5 --- [ main] com.github.pires.hazelcast.Application : Started Application in 13.792 seconds (JVM running for 14.542)
2017-01-30 12:42:54.660 INFO 6 --- [ main] com.hazelcast.core.LifecycleService : [10.244.9.2]:5701 [someGroup] [3.7.5] [10.244.9.2]:5701 is STARTED
2017-01-30 12:42:54.662 INFO 6 --- [ main] com.github.pires.hazelcast.Application : Started Application in 5.078 seconds (JVM running for 5.771)
2017-01-30 12:44:08.780 INFO 6 --- [thread-Acceptor] c.h.nio.tcp.SocketAcceptorThread : [10.244.9.2]:5701 [someGroup] [3.7.5] Accepting socket connection from /10.244.93.3:45945
2017-01-30 12:44:08.814 INFO 6 --- [cached.thread-1] c.h.nio.tcp.TcpIpConnectionManager : [10.244.9.2]:5701 [someGroup] [3.7.5] Established socket connection between /10.244.9.2:5701 and /10.244.93.3:45945
2017-01-30 12:44:15.785 INFO 6 --- [ration.thread-0] c.h.internal.cluster.ClusterService : [10.244.9.2]:5701 [someGroup] [3.7.5]
Members [2] {
Member [10.244.9.2]:5701 - f9cae801-59da-49d9-b8de-7719abb53844 this
Member [10.244.93.3]:5701 - 4e15667b-ce17-40c2-b045-abe3fb25d48b
}
```
Now let's scale our cluster to 4 nodes:
```sh
$ kubectl scale rc hazelcast --replicas=4
$ kubectl scale deployment hazelcast --replicas 4
```
Examine the status again by checking the logs and you should see the 4 members connected.
Examine the status again by checking a node's logs and you should see the 4 members connected. Something like:
```
(...)
Members [4] {
Member [10.244.9.2]:5701 - f9cae801-59da-49d9-b8de-7719abb53844 this
Member [10.244.93.3]:5701 - 4e15667b-ce17-40c2-b045-abe3fb25d48b
Member [10.244.9.3]:5701 - e0f36fa4-16bf-4009-a034-d4e7a4105003
Member [10.244.93.4]:5701 - 7ac96b48-aa47-4410-885f-1ad0fc3690f0
}
```
### tl; dr;
For those of you who are impatient, here is the summary of the commands we ran in this tutorial.
```sh
# create a service to track all hazelcast nodes
kubectl create -f examples/storage/hazelcast/hazelcast-service.yaml
# create a replication controller to replicate hazelcast nodes
kubectl create -f examples/storage/hazelcast/hazelcast-controller.yaml
# scale up to 2 nodes
kubectl scale rc hazelcast --replicas=2
# scale up to 4 nodes
kubectl scale rc hazelcast --replicas=4
kubectl create -f service.yaml
kubectl create -f deployment.yaml
kubectl scale deployment hazelcast --replicas 2
kubectl scale deployment hazelcast --replicas 4
```
### Hazelcast Discovery Source

31
examples/storage/hazelcast/hazelcast-controller.yaml
View File

@@ -1,31 +0,0 @@
apiVersion: v1
kind: ReplicationController
metadata:
labels:
name: hazelcast
name: hazelcast
spec:
replicas: 1
selector:
name: hazelcast
template:
metadata:
labels:
name: hazelcast
spec:
containers:
- resources:
limits:
cpu: 0.1
image: quay.io/pires/hazelcast-kubernetes:0.6.1
name: hazelcast
env:
- name: "DNS_DOMAIN"
value: "cluster.local"
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- containerPort: 5701
name: hazelcast

22
examples/storage/hazelcast/hazelcast-deployment.yaml Normal file
View File

@@ -0,0 +1,22 @@
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: hazelcast
labels:
name: hazelcast
spec:
template:
metadata:
labels:
name: hazelcast
spec:
containers:
- name: hazelcast
image: quay.io/pires/hazelcast-kubernetes:0.8.0
imagePullPolicy: Always
env:
- name: "DNS_DOMAIN"
value: "cluster.local"
ports:
- name: hazelcast
containerPort: 5701

24
examples/storage/hazelcast/image/Dockerfile
View File

@@ -1,24 +0,0 @@
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
FROM quay.io/pires/docker-jre:8u45-2
EXPOSE 5701
RUN \
curl -Lskj https://github.com/pires/hazelcast-kubernetes-bootstrapper/releases/download/0.5/hazelcast-kubernetes-bootstrapper-0.5.jar \
-o /bootstrapper.jar
CMD java -jar /bootstrapper.jar