github/kubernetes
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Uncordon the node after upgrade

Browse Source 
Previous logic was relying on the node to recreate the node API object
and, as a side-effect, uncordon itself. A change went in that no
longer ensures the node recreates itself, so the bug in this logic was exposed.
This commit is contained in:
Kris
2018-05-09 11:11:09 -07:00
parent 1ed1cf6ef3
commit 231fe79332
1 changed files with 12 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
cluster/gce/upgrade.sh
View File

@@ -342,15 +342,21 @@ function do-single-node-upgrade() {
sleep 1 sleep 1
done done
# Wait for the node to not have SchedulingDisabled=True and also to have # Uncordon the node.
# Ready=True. echo "== Uncordon ${instance}. == " >&2
local uncordon_rc
"${KUBE_ROOT}/cluster/kubectl.sh" uncordon "${instance}" \
&& uncordon_rc=$? || uncordon_rc=$?
if [[ "${uncordon_rc}" != 0 ]]; then
echo "== FAILED to uncordon ${instance} =="
return ${uncordon_rc}
fi
# Wait for the node to have Ready=True.
echo "== Waiting for ${instance} to become ready. ==" >&2 echo "== Waiting for ${instance} to become ready. ==" >&2
while true; do while true; do
local cordoned=$("${KUBE_ROOT}/cluster/kubectl.sh" get node "${instance}" --output='jsonpath={.status.conditions[?(@.type == "SchedulingDisabled")].status}')
local ready=$("${KUBE_ROOT}/cluster/kubectl.sh" get node "${instance}" --output='jsonpath={.status.conditions[?(@.type == "Ready")].status}') local ready=$("${KUBE_ROOT}/cluster/kubectl.sh" get node "${instance}" --output='jsonpath={.status.conditions[?(@.type == "Ready")].status}')
if [[ "${cordoned}" == 'True' ]]; then if [[ "${ready}" != 'True' ]]; then
echo "Node ${instance} is still not ready: SchedulingDisabled=${ready}"
elif [[ "${ready}" != 'True' ]]; then
echo "Node ${instance} is still not ready: Ready=${ready}" echo "Node ${instance} is still not ready: Ready=${ready}"
else else
echo "Node ${instance} Ready=${ready}" echo "Node ${instance} Ready=${ready}"