should mark volume mount in actual state even if volume expansion fails so that
reconciler can tear down the volume when needed. To avoid pods start
using it, mark volume as uncertain instead of mounted.
Will add unit test after the logic is reviewed.
Change-Id: I5aebfa11ec93235a87af8f17bea7f7b1570b603d
A number of race conditions exist when pods are terminated early in
their lifecycle because components in the kubelet need to know "no
running containers" or "containers can't be started from now on" but
were relying on outdated state.
Only the pod worker knows whether containers are being started for
a given pod, which is required to know when a pod is "terminated"
(no running containers, none coming). Move that responsibility and
podKiller function into the pod workers, and have everything that
was killing the pod go into the UpdatePod loop. Split syncPod into
three phases - setup, terminate containers, and cleanup pod - and
have transitions between those methods be visible to other
components. After this change, to kill a pod you tell the pod worker
to UpdatePod({UpdateType: SyncPodKill, Pod: pod}).
Several places in the kubelet were incorrect about whether they
were handling terminating (should stop running, might have
containers) or terminated (no running containers) pods. The pod worker
exposes methods that allow other loops to know when to set up or tear
down resources based on the state of the pod - these methods remove
the possibility of race conditions by ensuring a single component is
responsible for knowing each pod's allowed state and other components
simply delegate to checking whether they are in the window by UID.
Removing containers now no longer blocks final pod deletion in the
API server and are handled as background cleanup. Node shutdown
no longer marks pods as failed as they can be restarted in the
next step.
See https://docs.google.com/document/d/1Pic5TPntdJnYfIpBeZndDelM-AbS4FN9H2GTLFhoJ04/edit# for details
Update mounter interface in volume manager's ActualStateOfWorld every time.
Otherwise kubelet uses the first mounter it gets, which may not have the
latest information.
This fixes set up of CSI volumes, which store information about SELinux
support in their `mounter` interface implementation. With each MountVolume()
retry, a new mounter is instantiated and only the final mounter that succeeds
has the right info if the volume supports SELinux or not and can later
return the right attributes on GetAttributes() call.
desiredStateOfWorldPopulator.findAndRemoveDeletedPods() should remove
volumes from DSW when a pod is deleted on the API server and the volume is
uncertain in ASW.
podVolumesExist() should consider also uncertain volumes (where kubelet
does not know if a volume was fully unmounted) when checking for pod's
volumes. Added GetPossiblyMountedVolumesForPod for that.
Adding uncertain mounts to GetMountedVolumesForPod would potentially break
other callers (e.g. `verifyVolumesMountedFunc`).
As discussed during the alpha review, the ReadOnly field is not really
needed because volume mounts can also be read-only. It's a historical
oddity that can be avoided for generic ephemeral volumes as part
of the promotion to beta.
Silently ignoring the unsupported volume type leads to:
Warning FailedMount 8s kubelet Unable to attach or mount volumes: unmounted volumes=[my-csi-volume default-token-bsnbz], unattached volumes=[my-csi-volume default-token-bsnbz]: failed to get Plugin from volumeSpec for volume "my-csi-volume" err=no volume plugin matched
The new message is easier to understand:
Warning FailedMount 6s (x5 over 49s) kubelet Unable to attach or mount volumes: unmounted volumes=[my-csi-volume], unattached volumes=[my-csi-volume default-token-rwlpp]: volume my-csi-volume is a generic ephemeral volume, but that feature is disabled in kubelet
fakeVolumeHost previously implemented both the KubeletVolumeHost and
AttachDetachVolumeHost interfaces. This design makes it difficult to test the
CSIAttacher since it behaves differently depending on what type of
VolumeHost is supplied.
The implementation consists of
- identifying all places where VolumeSource.PersistentVolumeClaim has
a special meaning and then ensuring that the same code path is taken
for an ephemeral volume, with the ownership check
- adding a controller that produces the PVCs for each embedded
VolumeSource.EphemeralVolume
- relaxing the PVC protection controller such that it removes
the finalizer already before the pod is deleted (only
if the GenericEphemeralVolume feature is enabled): this is
needed to break a cycle where foreground deletion of the pod
blocks on removing the PVC, which waits for deletion of the pod
The controller was derived from the endpointslices controller.
