github/kubernetes
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #59767 from sjenning/check-phase-transition

Browse Source 
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

kubelet: check for illegal phase transition

I have been unable to root cause the transition from `Failed` phase to `Succeeded`.  I have been unable to recreate as well.  However, our CI in Origin, where we have controllers that look for these transitions and rely on the phase transition rules being respected, obviously indicate that the illegal transition does occur.

This PR will prevent the illegal phase transition from propagating into the kubelet caches or the API server.

Fixes https://github.com/kubernetes/kubernetes/issues/58711

xref https://github.com/openshift/origin/issues/17595

@dashpole @yujuhong @derekwaynecarr @smarterclayton @tnozicka
This commit is contained in:
Kubernetes Submit Queue 2018-02-12 15:13:21 -08:00 committed by GitHub
commit 4efbb71bf6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 9 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
pkg/kubelet/kubelet_pods.go
View File

@ -1359,6 +1359,15 @@ func (kl *Kubelet) generateAPIPodStatus(pod *v1.Pod, podStatus *kubecontainer.Po
spec := &pod.Spec
allStatus := append(append([]v1.ContainerStatus{}, s.ContainerStatuses...), s.InitContainerStatuses...)
s.Phase = getPhase(spec, allStatus)
// Check for illegal phase transition
if pod.Status.Phase == v1.PodFailed || pod.Status.Phase == v1.PodSucceeded {
// API server shows terminal phase; transitions are not allowed
if s.Phase != pod.Status.Phase {
glog.Errorf("Pod attempted illegal phase transition from %s to %s: %v", pod.Status.Phase, s.Phase, s)
// Force back to phase from the API server
s.Phase = pod.Status.Phase
}
}
kl.probeManager.UpdatePodStatus(pod.UID, s)
s.Conditions = append(s.Conditions, status.GeneratePodInitializedCondition(spec, s.InitContainerStatuses, s.Phase))
s.Conditions = append(s.Conditions, status.GeneratePodReadyCondition(spec, s.ContainerStatuses, s.Phase))