kubernetes/pkg/kubelet/container
Sascha Grunert b296f82c69
Sort kubelet pods by their creation time
There is a corner case when blocking Pod termination via a lifecycle
preStop hook, for example by using this StateFulSet:

```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: ubi
  serviceName: "ubi"
  replicas: 1
  template:
    metadata:
      labels:
        app: ubi
    spec:
      terminationGracePeriodSeconds: 1000
      containers:
      - name: ubi
        image: ubuntu:22.04
        command: ['sh', '-c', 'echo The app is running! && sleep 360000']
        ports:
        - containerPort: 80
          name: web
        lifecycle:
          preStop:
            exec:
              command:
              - /bin/sh
              - -c
              - 'echo aaa; trap : TERM INT; sleep infinity & wait'
```

After creation, downscaling, forced deletion and upscaling of the
replica like this:

```
> kubectl apply -f sts.yml
> kubectl scale sts web --replicas=0
> kubectl delete pod web-0 --grace-period=0 --force
> kubectl scale sts web --replicas=1
```

We will end up having two pods running by the container runtime, while
the API only reports one:

```
> kubectl get pods
NAME    READY   STATUS    RESTARTS   AGE
web-0   1/1     Running   0          92s
```

```
> sudo crictl pods
POD ID              CREATED              STATE     NAME     NAMESPACE     ATTEMPT     RUNTIME
e05bb7dbb7e44       12 minutes ago       Ready     web-0    default       0           (default)
d90088614c73b       12 minutes ago       Ready     web-0    default       0           (default)
```

When now running `kubectl exec -it web-0 -- ps -ef`, there is a random chance that we hit the wrong
container reporting the lifecycle command `/bin/sh -c echo aaa; trap : TERM INT; sleep infinity & wait`.

This is caused by the container lookup via its name (and no podUID) at:
02109414e8/pkg/kubelet/kubelet_pods.go (L1905-L1914)

And more specifiy by the conversion of the pod result map to a slice in `GetPods`:
02109414e8/pkg/kubelet/kuberuntime/kuberuntime_manager.go (L407-L411)

We now solve that unexpected behavior by tracking the creation time of
the pod and sorting the result based on that. This will cause to always
match the most recently created pod.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
2022-10-13 16:32:44 +02:00
..
testing Merge pull request #111221 from inosato/remove-ioutil-from-kubelet 2022-09-17 21:56:28 -07:00
cache_test.go Fix int->string casts 2020-07-24 16:23:12 -04:00
cache.go Generate and format files 2022-07-26 13:14:05 -04:00
container_gc.go Structured Logging migration: modify volume and container part logs of kubelet. 2021-03-17 08:59:03 +08:00
container_hash_test.go Omit nil or empty field when calculating hash value 2019-08-22 13:46:52 +08:00
helpers_test.go Remove EphemeralContainers feature-gate checks 2022-07-26 02:55:30 +02:00
helpers.go kubelet: add GetUserNamespaceMappings to RuntimeHelper 2022-08-03 19:53:22 +02:00
os.go Remove ioutil in kubelet and its tests 2022-07-30 12:35:26 +09:00
ref_test.go Remove no-longer used selflink code from kubelet 2022-01-14 10:38:23 +01:00
ref.go Remove EphemeralContainers feature-gate checks 2022-07-26 02:55:30 +02:00
resize.go Fix golint failures for kubelet/container 2020-05-20 19:01:23 +00:00
runtime_cache_fake.go Fix golint failures for kubelet/container 2020-05-20 19:01:23 +00:00
runtime_cache_test.go
runtime_cache.go mockery to mockgen conversion 2021-09-25 16:15:08 +00:00
runtime.go Sort kubelet pods by their creation time 2022-10-13 16:32:44 +02:00
sync_result_test.go go-1.12: fix 'go vet' failures 2019-03-01 18:48:17 +02:00
sync_result.go fix golint issues in pkg/kubelet/container 2020-06-19 15:48:08 +00:00