Commit Graph

118 Commits

Author SHA1 Message Date
Jan Safranek
44b72d0348 Move new reconciler logic into reconciler_new.go
Move reconciler logic from reconstruct{new}.go to:
- reconciler.go - only the functionality used by the current (old)
  reconciler.
- reconciler_new.go - only the functionality used by the new reconciler.
- reconciler_common.go - common functions.
2022-11-03 16:55:13 +01:00
Jan Safranek
e0f3e5c457 Rework volume reconstruction
Subsequent SELinux work (see http://kep.k8s.io/1710) will need
ActualStateOfWorld populated around the time kubelet starts mounting
volumes.

Therefore reconstruct volumes before starting reconciler, but do not depend
on the desired state of world populated nor node.status - both need a
working API server, which may not be available at that time.

All reconstructed volumes are marked as Uncertain and reconciler will sort
them out - call SetUp to ensure the volume is really mounted when a pod
needs the volume or call TearDown then there is no such pod.

Finish the reconstruction when the API server becomes available:
- Clean up volumes that failed reconstruction and are not needed.

- Update devicePath of reconstructed volumes from node.status. Make sure
  not to overwrite devicePath that may have been updated when the volume
  was mounted by reconcile().

Hiding all this rework behind SELinuxMountReadWriteOncePod FeatureGate,
just to make sure we have a way back if this commit is buggy.
2022-11-03 16:55:12 +01:00
Jan Safranek
989e391d08 Move all volume reconstruction code into separate files
There is no code change, just moving code around and preparing for the
subsequent commit.
2022-11-02 15:58:21 +01:00
Jan Safranek
d37808faae Report error on a pod startup on SELinux mismatch
When a volume is already mounted with an unexpected SELinux label,
kubelet must unmount it first and then mount it back with the expected one.
Report an error to user, just in case the unmount takes too long.

In therory, this error should not happen too often, because two Pods with
different SELinux label will not enter Desired State of World, see
dsw.AddPodToVolume. It can happen when DSW and ASW SELinux labels only when
a volume has been deleted from DSW (= Pod was deleted) or a volume was
reconstructed after kubelet restart. In both cases, volume manager should
unmount the volume quickly.
2022-10-31 13:59:23 +01:00
Jan Safranek
a910d83070 Reduce log noise on SELinux mount mismatch
The Desired State of World can require a different SELinux mount context than
is in the Actual State of World and it's perfectly OK. For example when
user changes SELinux context of Pods or when the context is reconstructed
after kubelet restart.

Don't spam log and don't report errors to the user as event - reconciler
will do the right thing and unmount the old volume (with wrong context) and
mount a new one in the next reconciliation. It's not an error, it's
expected workflow.
2022-10-27 18:00:42 +02:00
Jan Safranek
1490d51028 Remove noisy log
The error would be logged every reconciler sync (100 ms).
2022-08-04 10:51:53 +02:00
Jan Safranek
48b0751269 Add SELinux context tracking to volume manager
Both ActualStateOfWorld and DesiredStateOfWorld must track SELinux context
of volume mounts.
2022-08-04 10:51:41 +02:00
Kubernetes Prow Robot
9ad4c5c0a0 Merge pull request #110670 from gnufied/fix-pod-deletion-terminating
Fix pod stuck in termination state when mount fails or gets skipped after kubelet restart
2022-07-27 06:31:29 -07:00
Davanum Srinivas
a9593d634c Generate and format files
- Run hack/update-codegen.sh
- Run hack/update-generated-device-plugin.sh
- Run hack/update-generated-protobuf.sh
- Run hack/update-generated-runtime.sh
- Run hack/update-generated-swagger-docs.sh
- Run hack/update-openapi-spec.sh
- Run hack/update-gofmt.sh

Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2022-07-26 13:14:05 -04:00
Hemant Kumar
835e8ccc76 Use CheckAndMarkAsUncertainViaReconstruction for uncertain volumes
Also only remove volumes from skippedDuringReconstruction only if
volume was marked as attached.
2022-07-22 20:11:37 -04:00
Hemant Kumar
6d43345c06 Remove volume from found during reconstruction if mounted
Add unit tests for removing reconstructed volumes from ASOW
2022-07-22 20:04:51 -04:00
Hemant Kumar
b455270f6e Add unit test for verifying if processReconstructedVolumes works as expected 2022-07-22 20:04:51 -04:00
Hemant Kumar
b8257e8c01 Address review comments 2022-07-22 20:04:51 -04:00
Hemant Kumar
eb071c2755 Fix code to process volumes which were skipped during reconstruction 2022-07-22 20:04:51 -04:00
Hemant Kumar
c8b85fb470 Keep track of each pod that uses a volume during reconstruction
Add tests for volume cleaning up
2022-07-22 20:04:51 -04:00
Kubernetes Prow Robot
1f90b7980b Merge pull request #108997 from dobsonj/issue79980
Fix volume reconstruction for CSI ephemeral volumes
2022-06-03 18:08:20 -07:00
Jonathan Dobson
daa181d92e kubelet: fix volume reconstruction for CSI ephemeral volumes
This resolves a couple of issues for CSI volume reconstruction.
1. IsLikelyNotMountPoint is known not to work for bind mounts and was
   causing problems for subpaths and hostpath volumes.
2. Inline volumes were failing reconstruction due to calling
   GetVolumeName, which only works when there is a PV spec.
2022-06-01 14:22:57 -06:00
yxxhero
4fac7486d4 remove ioutil in kubelet
Signed-off-by: yxxhero <aiopsclub@163.com>
2022-04-27 21:08:42 +08:00
Hemant Kumar
7a43406138 Do not update PVC if it already has updated size 2022-03-28 11:32:49 -04:00
Hemant Kumar
e4f62d6c41 Modify code to use new interface functions 2022-03-28 11:32:49 -04:00
Hemant Kumar
10f91a9951 Refactor volume attach code 2022-03-28 11:32:49 -04:00
Kubernetes Prow Robot
c239b406f0 Merge pull request #108929 from gnufied/move-expansion-feature-gate-ga
Move all volume expansion feature gates to GA
2022-03-25 18:08:16 -07:00
Hemant Kumar
966e1b6dd0 Fix code to not use the feature gate 2022-03-24 10:37:49 -04:00
Nikhil Sharma
4224b524d5 Managing nil pointer in VolumeManager 2022-03-22 22:04:24 +05:30
Hemant Kumar
7989f27044 use node informer to check volumes attachment status before backoff
fix unit tests
2021-12-20 11:57:05 -05:00
Hemant Kumar
5b7b2e2f6c When volume is not marked in-use, do not backoff 2021-12-07 11:50:15 -05:00
haoyun
b5409adaeb refactor: extract multiple ignore errors validate to ignoreError
Signed-off-by: haoyun <yun.hao@daocloud.io>
2021-11-16 20:43:50 +08:00
Kubernetes Prow Robot
a674fb496c Merge pull request #103261 from markusthoemmes/kubelet-volume-logs
Add pod context to volume lifecycle logs
2021-08-04 19:00:15 -07:00
Markus Thömmes
c820824711 Add pod context to volume lifecycle logs 2021-08-03 13:12:22 +02:00
Kubernetes Prow Robot
6aa160f3ba Merge pull request #103181 from 249043822/bugfix-volumemanager
Add sync reconstructed volume from desired state of world for volumemanager
2021-07-19 15:04:52 -07:00
KeZhang
65618bfd69 Add sync reconstructed volume from desired state of world for volumemanager 2021-07-13 12:51:37 +08:00
Jing Xu
0fa01c371c Mark volume mount as uncertain in case of volume expansion fails
should mark volume mount in actual state even if volume expansion fails so that
reconciler can tear down the volume when needed. To avoid pods start
using it, mark volume as uncertain instead of mounted.

Will add unit test after the logic is reviewed.

Change-Id: I5aebfa11ec93235a87af8f17bea7f7b1570b603d
2021-07-08 16:00:34 -07:00
sanwishe
9e257ec194 Optimization logging format for pkg/kubelet
Signed-off-by: sanwishe <jiang.mingzhi35@zte.com.cn>
2021-05-25 08:52:08 +08:00
Chris Henzie
f756bd5189 Fix nil ptr dereference in log line 2021-03-22 16:06:51 -07:00
JunYang
01a4e4face Structured Logging migration: modify volume and container part logs of kubelet.
Signed-off-by: JunYang <yang.jun22@zte.com.cn>
2021-03-17 08:59:03 +08:00
Kubernetes Prow Robot
6ac2930ef0 Merge pull request #94574 from auxten/pkg-kubelet-staticchecks
Fix pkg/kubelet static checks
2020-09-21 21:22:47 -07:00
Srini Brahmaroutu
fbe5daed73 Change code to use staging/k8s.io/mount-utils 2020-09-16 21:51:24 -07:00
auxten
a9c1acc044 Fix staticchecks ST1005,S1002,S1008,S1039 in pkg/kubelet 2020-09-07 10:53:43 +08:00
Davanum Srinivas
442a69c3bd switch over k/k to use klog v2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2020-05-16 07:54:27 -04:00
Kubernetes Prow Robot
ef672c1c2d Merge pull request #88678 from verult/slow-rxm-attach
Parallelize attach operations across different nodes for volumes that allow multi-attach
2020-03-06 13:17:21 -08:00
Cheng Xing
ef3d66b98b Parallelize attach operations across different nodes for volumes that allow multi-attach 2020-03-05 22:22:05 -08:00
Jan Safranek
2c1b743766 Promote block volume features to GA 2020-02-28 20:48:38 +01:00
Mike Danese
3aa59f7f30 generated: run refactor 2020-02-07 18:16:47 -08:00
Jordan Liggitt
cd1059e3c4 Revert "Merge pull request #87258 from verult/slow-rxm-attach"
This reverts commit 15c3f1b119, reversing
changes made to 52d7614a8c.
2020-01-29 14:58:32 -05:00
Cheng Xing
c6a03fa5be Parallelize attach operations across different nodes for volumes that allow multi-attach 2020-01-27 15:02:25 -08:00
Hemant Kumar
5feea93163 Rename MarkVolumeMountedOpts to MarkVolumeOpts
Also remove VolumeNotMounted state
2019-12-02 12:09:38 -05:00
Hemant Kumar
34a6007dfe Add code to mark volume as uncertain
Update bazel files
Add tests for volume mounts in uncertain state
2019-12-02 12:01:40 -05:00
Hemant Kumar
a795f3de88 Add code for introducing uncertain state of mounts
Add a comment about volumestate
2019-12-02 11:57:40 -05:00
Travis Rhoden
0c5c3d8bb9 Remove pkg/util/mount (moved out of tree)
This patch removes pkg/util/mount completely, and replaces it with the
mount package now located at k8s.io/utils/mount. The code found at
k8s.io/utils/mount was moved there from pkg/util/mount, so the code is
identical, just no longer in-tree to k/k.
2019-11-15 08:29:12 -07:00
Yecheng Fu
36a54399a6 support local volume with block source reconstruction 2019-10-28 10:34:57 +08:00