Skip to content

Commit 6ddc035

Browse files
ibrar-eurusibrar-eurus
committed
taint and tolaration
1 parent 90a3cd7 commit 6ddc035

File tree

2 files changed

+176
-1
lines changed

2 files changed

+176
-1
lines changed

‎taint and tolaration/deployment-toleration.yaml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
name: nginx-deployment
5+
spec:
6+
selector:
7+
matchLabels:
8+
app: nginx
9+
replicas: 10 # Run 10 pods matching the template below
10+
template:
11+
metadata:
12+
labels:
13+
app: nginx
14+
spec:
15+
containers:
16+
- name: nginx
17+
image: nginx:1.7.9
18+
ports:
19+
- containerPort: 80
20+
tolerations:
21+
- effect: NoSchedule
22+
operator: Exists

‎taint and tolaration/readme.md

Lines changed: 154 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,160 @@ By adding a taint to the worker node A and only giving the necessary toleration
1010

1111
## How to add Kubernetes taints
1212

13-
The kubectl taint command with the required taint allows us to add taints to nodes. The general syntax for the command is:
13+
The kubectl taint command with the required taint allows u# Taints and Tolerations
14+
Taints can be used for advanced scheduling in kubernetes. Tutorial below is a walk through of such usage.
15+
16+
## Concepts
17+
18+
### Taints
19+
Taint is a property of node that allows you to repel a set of pods unless those pods explicitly tolerates the said taint.
20+
21+
Taint has three parts. A key, a value and an effect.
22+
23+
For example,
24+
```
25+
kubectl taint nodes node1.compute.infracloud.io thisnode=HatesPods:NoSchedule
26+
```
27+
The above taint has key=thisnode, value=HatesPods and effect as NoSchedule. These key value pairs are configurable. Any pod that doesn't have a matching toleration to this taint will not be scheduled on node1.
28+
29+
To remove the above taint, we can run the following command
30+
```
31+
kubectl taint nodes node1.compute.infracloud.io thisnode:NoSchedule-
32+
```
33+
34+
What are some of the Taint effects?
35+
* NoSchedule - Doesn't schedule a pod without matching tolerations
36+
* PreferNoSchedule - Prefers that the pod without matching toleration be not scheduled on the node. It is a softer version of NoSchedule effect.
37+
* NoExecute - Evicts the pods that don't have matching tolerations.
38+
39+
A node can have multiple taints.
40+
For example, if any pod is to be scheduled on a node with multiple *NoExecute* effect taints, then that pod must tolerate all the taints. However, if the set of taints on a node is a combination of *NoExecute* and *PreferNoExecute* effects and the pod only tolerates *NoExecute* taints then kubernetes will prefer not to schedule the pod on that node, but will do it anyway if there's no alternative.
41+
42+
### Tolerations
43+
Nodes are tainted for a simple reason, to avoid running of workload. The similar outcome can be achieved by PodAffinity/PodAnti-Affinity, however, to reject a large workload taints are more efficient (In a sense that they only require tolerations to be added to the small workload that does run on the tainted nodes as opposed to podAffinity which would require every pod template to carry that information)
44+
45+
Toleration is simply a way to overcome a taint.
46+
47+
For example,
48+
In the above section, we have tainted thisnode.compute.infracloud.io
49+
50+
To schedule the pod on that node, we need a matching toleration. Below is the toleration that can be used to overcome the taint.
51+
52+
```
53+
tolerations:
54+
- key: "thisnode"
55+
operator: "Equal"
56+
value: "HatesPods"
57+
effect: "NoSchedule"
58+
```
59+
60+
What we are telling kubernetes here is that, on any node if you find that there's a taint with key *node1* and its value is *HatesPods* then that particular taint should not stop you from scheduling this pod on that node.
61+
62+
Toleration generally has four parts. A key, a value, an operator and an effect.
63+
Operator, if not specified, defaults to *Equal*
64+
65+
## Use cases
66+
* Taints can be used to group together a set of Nodes that only run a certain set of workload, like network pods or pods with special resource requirement.
67+
* Taints can also be used to evict a large set of pods from a node using taint with *NoExecute* effect.
68+
69+
## Examples:
70+
Follow through guide.
71+
72+
Let's begin with listing nodes.
73+
74+
```
75+
kubectl get nodes
76+
```
77+
You should be able to see the list of nodes available in the cluster,
78+
```
79+
NAME STATUS ROLES AGE VERSION
80+
node1.compute.infracloud.io Ready <none> 25m v1.9.4
81+
node2.compute.infracloud.io Ready <none> 25m v1.9.4
82+
node3.compute.infracloud.io Ready <none> 28m v1.9.4
83+
```
84+
85+
Now, let's taint node1 with *NoSchedule* effect.
86+
```
87+
kubectl taint nodes node1.compute.infracloud.io thisnode=HatesPods:NoSchedule
88+
```
89+
90+
You should be able to see that node1 is now tainted.
91+
```
92+
node "node1.compute.infracloud.io" tainted
93+
```
94+
95+
Let's run the deployment to see where pods are deployed.
96+
```
97+
kubectl create -f deployment.yaml
98+
```
99+
100+
Check the output using,
101+
```
102+
kubectl get pods -o wide
103+
```
104+
105+
You should be able that the pods aren't scheduled on node1
106+
```
107+
NAME READY STATUS RESTARTS AGE IP NODE
108+
nginx-deployment-6c54bd5869-g9rtf 1/1 Running 0 18s 10.20.32.2 node3.compute.infracloud.io
109+
nginx-deployment-6c54bd5869-v74m6 1/1 Running 0 18s 10.20.32.3 node3.compute.infracloud.io
110+
nginx-deployment-6c54bd5869-w5jxj 1/1 Running 0 18s 10.20.61.2 node2.compute.infracloud.io
111+
```
112+
113+
Now let's taint node3 with *NoExecute* effect, which will evict both the pods from node3 and schedule them on node2.
114+
```
115+
kubectl taint nodes node3.compute.infracloud.io thisnode=AlsoHatesPods:NoExecute
116+
```
117+
118+
In a few seconds you'll see that the pods are terminated on node3 and spawned on node2
119+
```
120+
kubectl get pods -o wide
121+
122+
NAME READY STATUS RESTARTS AGE IP NODE
123+
nginx-deployment-6c54bd5869-8vqvc 1/1 Running 0 33s 10.20.42.21 node2.compute.infracloud.io
124+
nginx-deployment-6c54bd5869-hsjhj 1/1 Running 0 33s 10.20.42.20 node2.compute.infracloud.io
125+
nginx-deployment-6c54bd5869-w5jxj 1/1 Running 0 2m 10.20.42.19 node2.compute.infracloud.io
126+
```
127+
128+
The above example demonstrates taint based evictions.
129+
130+
Let's delete the deployment and create new one with tolerations for the above taints.
131+
```
132+
kubectl delete deployment nginx-deployment
133+
```
134+
135+
```
136+
kubectl create -f taints/deployment-toleration.yaml
137+
```
138+
139+
You can check the output by running,
140+
```
141+
kubectl get pods -o wide
142+
```
143+
144+
You should be able to see that some of the pods are scheduled on node1 and some on node2. However, no pod is scheduled on node3. This is because, in the new deployment spec, we are tolerating taint *NoSchedule* effect. node3 is tainted with *NoExecute* effect which we have not tolerated so no pods will be scheduled there.
145+
146+
```
147+
NAME READY STATUS RESTARTS AGE IP NODE
148+
nginx-deployment-5699885bdb-4dz8z 1/1 Running 0 1m 10.20.34.3 node1.compute.infracloud.io
149+
nginx-deployment-5699885bdb-cr7p7 1/1 Running 0 1m 10.20.34.4 node1.compute.infracloud.io
150+
nginx-deployment-5699885bdb-kjxwv 1/1 Running 0 1m 10.20.34.5 node1.compute.infracloud.io
151+
nginx-deployment-5699885bdb-kvfw6 1/1 Running 0 1m 10.20.34.7 node1.compute.infracloud.io
152+
nginx-deployment-5699885bdb-lx2zv 1/1 Running 0 1m 10.20.34.6 node1.compute.infracloud.io
153+
nginx-deployment-5699885bdb-m686q 1/1 Running 0 1m 10.20.42.30 node2.compute.infracloud.io
154+
nginx-deployment-5699885bdb-x7c6z 1/1 Running 0 1m 10.20.42.31 node2.compute.infracloud.io
155+
nginx-deployment-5699885bdb-z8cwl 1/1 Running 0 1m 10.20.34.9 node1.compute.infracloud.io
156+
nginx-deployment-5699885bdb-z9c68 1/1 Running 0 1m 10.20.34.8 node1.compute.infracloud.io
157+
nginx-deployment-5699885bdb-zshst 1/1 Running 0 1m 10.20.34.2 node1.compute.infracloud.io
158+
```
159+
160+
To finish off, let's remove the taints from the nodes,
161+
```
162+
kubectl taint nodes node3.compute.infracloud.io thisnode:NoExecute-
163+
```
164+
```
165+
kubectl taint nodes node1.compute.infracloud.io thisnode:NoSchedule-
166+
```s to add taints to nodes. The general syntax for the command is:
14167
15168
```
16169

0 commit comments

Comments
 (0)