Commit Graph

551 Commits

Author SHA1 Message Date
Shyam Jeedigunta
8ff1f05f7c Increase verbosity of frequently printed logline in scheduler_binder 2018-03-08 19:25:01 +01:00
wackxu
b3ba80b223 update bazel 2018-02-27 20:23:36 +08:00
wackxu
f737ad62ed update import 2018-02-27 20:23:35 +08:00
Lei Wang
2e0abfa29f Fix grammar and log issue in volume cache code 2018-02-23 17:46:53 +08:00
NickrenREN
dad0fa07b7 rename StorageProtection to StorageObjectInUseProtection 2018-02-21 10:48:56 +08:00
Kubernetes Submit Queue
6ba46963f8
Merge pull request #59391 from msau42/topology-beta
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move volume scheduling and local storage to beta

**What this PR does / why we need it**:
* Move the feature gates and APIs for volume scheduling and local storage to beta
* Update tests to use the beta fields
@kubernetes/sig-storage-pr-reviews 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #59390

**Special notes for your reviewer**:

**Release note**:

```release-note
ACTION REQUIRED: VolumeScheduling and LocalPersistentVolume features are beta and enabled by default.  The PersistentVolume NodeAffinity alpha annotation is deprecated and will be removed in a future release.
```
2018-02-20 13:26:07 -08:00
Michelle Au
c7884550b2 Add VolumeNodeAffinity to PersistentVolumeSpec 2018-02-16 17:54:10 -08:00
Jeff Grafton
ef56a8d6bb Autogenerated: hack/update-bazel.sh 2018-02-16 13:43:01 -08:00
Michelle Au
5271edd9e2 Index PVs by StorageClass in assume cache 2018-02-15 17:12:32 -08:00
mlmhl
6f83110e77 add number measurement for bound/unbound pv/pvc 2018-02-14 17:31:14 +08:00
Kubernetes Submit Queue
2368f70959
Merge pull request #59523 from gnufied/revert-57872-volume_metric_bound_pvc
Automatic merge from submit-queue (batch tested with PRs 57445, 59523). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Revert "add number measurement for bound/unbound pv/pvc"

Reverts kubernetes/kubernetes#57872

Fixes : https://github.com/kubernetes/kubernetes/issues/59517
2018-02-13 21:29:29 -08:00
Di Xu
48388fec7e fix all the typos across the project 2018-02-11 11:04:14 +08:00
Hemant Kumar
f94a81ef19
Revert "add number measurement for bound/unbound pv/pvc" 2018-02-07 21:31:01 -05:00
Kubernetes Submit Queue
5cecc6ec68
Merge pull request #59350 from jsafrane/recycler-wait
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Do not recycle volumes that are used by pods

**What this PR does / why we need it**:
Recycler should wait until all pods that use a volume are finished.

Consider this scenario:

1. User creates a PVC that's bound to a NFS PV.
2. User creates a pod that uses the PVC
3. User deletes the PVC.

Now the PV gets `Released` (the PVC does not exists) and recycled, however the PV is still mounted to a running pod. PVC protection won't help us, because it puts finalizers on PVC that is under user's control and user can remove it.

This PR checks that there is no pod that uses a PV before it recycles it.

**Release note**:

```release-note
NONE
```

/sig storage
2018-02-07 10:01:32 -08:00
Kubernetes Submit Queue
e740fe68c5
Merge pull request #58264 from WanLinghao/log_fix
Automatic merge from submit-queue (batch tested with PRs 59441, 58264, 59287, 59396, 59439). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

	fix some log param error

this patch fix some log parameter mistakes.



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note 
/release-note-none
```
2018-02-06 20:27:36 -08:00
Kubernetes Submit Queue
4bd22b5467
Merge pull request #58415 from gnufied/fix-volume-resize-messages
Automatic merge from submit-queue (batch tested with PRs 52942, 58415). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Improve messaging on volume expansion

- we now provide clear message to user what to do when cloudprovider resizing is finished
  and file system resizing is needed.
- add a event when resizing is successful
- Use PATCH both in controller-manager and kubelet for updating PVC status
- Remove code duplication between controller-manager and kubelet for updating PVC status
- Only remove conditions that are managed by resize controller



```release-note
Improve messages user gets during and after volume resizing is done.
```
2018-02-06 07:55:32 -08:00
Kubernetes Submit Queue
997fe31941
Merge pull request #57872 from mlmhl/volume_metric_bound_pvc
Automatic merge from submit-queue (batch tested with PRs 58317, 58687, 57872, 59063, 59328). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add number measurement for bound/unbound pv/pvc

**What this PR does / why we need it**:

Implement number measurement for bound/unbound pv/pvc defined in the [Metrics Spec](https://docs.google.com/document/d/1Fh0T60T_y888LsRwC51CQHO75b2IZ3A34ZQS71s_F0g/edit#heading=h.bwzmc2tktae)

ref feature: [ kubernetes/features#496](https://github.com/kubernetes/features/issues/496)

**Release note**:

```release-note
Intended for post-1.9
```
2018-02-05 18:43:37 -08:00
Jan Safranek
53efb0fff7 Don't recycle PVs that are used by a pod 2018-02-05 16:41:39 +01:00
Jan Safranek
c96c0495f4 Pass pod informer to PV controller 2018-02-05 15:40:25 +01:00
Kubernetes Submit Queue
6f89c4742a
Merge pull request #59185 from jamiefang/fix-typo
Automatic merge from submit-queue (batch tested with PRs 55606, 59185, 58763, 59072, 59251). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix a typo in pkg/controller/volume/persistentvolume/pv_controller.go

**What this PR does / why we need it**:
Fix typo (a -> an)

**Special notes for your reviewer**:
none

**Release note**:

```release-note
none
```
2018-02-02 18:26:35 -08:00
Jamie Fang
8b9bfd5e5d
Fix typo (a -> an) 2018-02-01 16:29:45 +08:00
linyouchong
e657703346 fix spelling error in comment and log 2018-02-01 16:02:39 +08:00
NickrenREN
3fee293607 Add PV protection controller 2018-01-31 20:18:54 +08:00
NickrenREN
b99580ba3f existing PV controller changes 2018-01-31 20:02:01 +08:00
Hemant Kumar
1fa8cbc5e4 Improve messaging on resize
- we now provide clear message to user what to do when cloudprovider resizing is finished
  and file system resizing is needed.
- add a event when resizing is successful.
- Use Patch for updating PVCs in both kubelet and controller-manager
- Extract updating pvc util function in one place.
- Only update resize conditions on progress
2018-01-29 15:07:51 -05:00
Kubernetes Submit Queue
59bc6d0ed6
Merge pull request #58862 from wackxu/vlm
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Mini fix about typo

```release-note
NONE
```
2018-01-29 07:29:33 -08:00
wackxu
cd48a6745a mini fix about typo 2018-01-26 15:42:57 +08:00
WanLinghao
70ef581ecc fix some log param error
modified:   pkg/cloudprovider/providers/vsphere/vsphere_util.go
	modified:   pkg/controller/certificates/cleaner/cleaner.go
	modified:   pkg/controller/volume/pvcprotection/pvc_protection_controller.go
	modified:   pkg/volume/azure_dd/azure_mounter.go
2018-01-26 12:16:00 +08:00
Kubernetes Submit Queue
7de1a8e0f5
Merge pull request #56288 from jsafrane/multiattach-pods
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add list of pods that use a volume to multiattach events

So users knows what pods are blocking a volume and can realize their error.

**Release note**:

```release-note
NONE
```

UX:
* User can get one of following events, depending what other pod(s) are already using a volume and in which namespace they are:
```
Multi-Attach error for volume"volume-name" Volume is already exclusively attached to one node and can't be attached to another
Multi-Attach error for volume "volume-name" Volume is already used by pod(s) pod3 and 1 pod(s) in different namespaces
```

* controller-manager gets always full logs:
  * When the node where is the volume attached is known:
        ```
        Multi-Attach error for volume "volume-name" (UniqueName: "fake-plugin/volume-name") from node "node1" Volume is already used by pods ns2/pod2, ns1/pod3 on node node2, node3
        ```

  * When the node where is the volume attached is not known:
        ```
        Multi-Attach error for volume "volume-name" (UniqueName: "fake-plugin/volume-name") from node "node1" Volume is already exclusively attached to node node2 and can't be attached to another
        ```

/kind bug
/sig storage
/assign @gnufied
2018-01-25 05:31:34 -08:00
Jan Safranek
e46c886bf3 Add list of pods that use a volume to multiattach events
So users knows what pods are blocking a volume and can realize their error.
2018-01-24 13:22:03 +01:00
Jan Safranek
691fec5e34 Don't bind PVs and PVCs with different access modes.
PVC pre-bound to a PV can bind to the PV only if it has correct access mode.
Report an event if it does not and keep the PVC Pending.
2018-01-24 09:48:38 +01:00
Kubernetes Submit Queue
2f4cca73af
Merge pull request #57680 from hzxuzhonghu/volume-expand
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

process pvc watch deletion event miss in expand-controller

**What this PR does / why we need it**:

volume expand controller should also process exception case when watch deletion event missed as most controllers do.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-01-23 22:33:34 -08:00
Jan Safranek
24400c24ae Clean up error messages for pre-bound PVCs.
When a PVC explicitly requests specific PV and the PV does not match,
we should tell the user what exactly does not match.

From:
Volume's size is smaller than requested or volume's class does not match with claim

To:
Cannot bind to requested volume "<volume name>": %s
where %s is one of:
- requested PV is too small
- storageClasseNames do not match
- incompatible volumeMode
- error checking volumeMode: api defaulting for volumeMode failed (this should not ever happen)
2018-01-19 15:43:45 +01:00
mlmhl
925013f0c6 add number measurement for bound/unbound pv/pvc 2018-01-11 10:07:20 +08:00
Kubernetes Submit Queue
2f39e8a045
Merge pull request #57724 from mlmhl/ad_controller_populateDesiredStateOfWorld
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix populateDesiredStateOfWorld bug for attach/detach controller

**What this PR does / why we need it**:

The `attachDetachController.populateDesiredStateOfWorld` mehod will fetch all pods from the informer cache and add each pods to `DesiredStateOfWorld` by the [addPod method](da9a4d5da9/pkg/controller/volume/attachdetach/attach_detach_controller.go (L338)).
However, the element type of pod list returned by `podLister.List` is already `*v1.Pod`, so we needn't to take the address any more.

**Release note**:

```release-note
NONE
```

/sig storage
/kind bug
2018-01-10 05:12:27 -08:00
Kubernetes Submit Queue
6f3e1dabe4
Merge pull request #57501 from linyouchong/linyouchong-20171221
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix incorrect comment
2018-01-09 11:34:27 -08:00
mlmhl
0e6ac1df76 fix populateDesiredStateOfWorld bug for attach/detach controller 2018-01-07 17:45:42 +08:00
Kubernetes Submit Queue
d61eff6c19
Merge pull request #57676 from hzxuzhonghu/volume-resize-map-opt
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

optimize volumeResizeMap lock

**What this PR does / why we need it**:

volumeResizeMap implements VolumeResizeMap interface, but has several useless lock section and one too large critical section.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-01-05 10:40:45 -08:00
Kubernetes Submit Queue
27d2ffb32f
Merge pull request #49856 from dixudx/polish_UpdateNodeStatus
Automatic merge from submit-queue (batch tested with PRs 49856, 56257, 57027, 57695, 57432). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Change to pkg/util/node.UpdateNodeStatus

**What this PR does / why we need it**:

> // TODO: Change to pkg/util/node.UpdateNodeStatus.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
/cc @brendandburns @dchen1107 @lavalamp 

**Release note**:

```release-note
None
```
2018-01-02 13:15:42 -08:00
hzxuzhonghu
e6567108bd process pvc watch deletion event miss in expand-controller 2017-12-28 15:39:00 +08:00
hzxuzhonghu
bfa462a8c0 optimize volumeResizeMap lock 2017-12-28 14:44:36 +08:00
Jeff Grafton
efee0704c6 Autogenerate BUILD files 2017-12-23 13:12:11 -08:00
linyouchong
4acc23b409 fix incorrect comment 2017-12-21 23:36:20 +08:00
Kubernetes Submit Queue
a73aee954f
Merge pull request #56872 from davidz627/logImprovement
Automatic merge from submit-queue (batch tested with PRs 56375, 56872, 57053, 57165, 57218). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Improved event generation in volume mount,attach, and extend operations

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57217

refactored mount, attach, extend operation's so that all failures generate events and event generation is more consistent.

```release-note
Improved event generation in volume mount, attach, and extend operations
```
2017-12-17 05:33:41 -08:00
Kubernetes Submit Queue
3abbd6fb1f
Merge pull request #56298 from pospispa/566-improvements-suggested-by-thockin-during-review-of-PR55824
Automatic merge from submit-queue (batch tested with PRs 56401, 56506, 56551, 56298, 56581). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Addressing Comments from Code Review

**What this PR does / why we need it**: addressing comments from code review: https://github.com/kubernetes/kubernetes/pull/55824#pullrequestreview-78597250

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: N/A


**Special notes for your reviewer**:
@thockin @jsafrane @msau42 PTAL

**Release note**:

```release-note
NONE
```
2017-12-15 20:29:36 -08:00
Kubernetes Submit Queue
e87d8511b1
Merge pull request #49608 from guangxuli/fix_nodestatus_update_prints
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

The printing level for node updated failed info should be used WARNING type

**What this PR does / why we need it**:

when I add the new node to the currnent kubernetes cluster, the kube-controller-manager always outputs the error msg( `Failed to set statusUpdateNeeded to needed true because nodeName="openpass-215"  does not exist)` and (`Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true because nodeName="openpass-215"  does not exist`). 

The results of invocation `updateNodeStatusUpdateNeeded` have two context, normal and abnormal.
The normal context is adding a new node to the cluster, the abnormal context is really exist some errors.

The printing level is confusing because it awlays outputs the error level msg.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

none

/cc @saad-ali
2017-12-14 14:58:11 -08:00
David Zhu
e3f8f64c17 refactored mount, attach, resize operation's so that all failures generate events and event generation is more consistent.
refactored operation generator and operation executor to use more general generated functions for operations, completions, and events.
2017-12-14 11:09:12 -08:00
Kubernetes Submit Queue
876259435a
Merge pull request #55991 from supereagle/core-client-with-version
Automatic merge from submit-queue (batch tested with PRs 54604, 55781, 55806, 55935, 55991). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use core client with explicit version

**What this PR does / why we need it**:
Core client without explicit version has been deprecated, change them to the one with explicit version.

**Which issue(s) this PR fixes**:
Fixes partially #55993

**Special notes for your reviewer**:
/cc @kubernetes/sig-api-machinery-pr-reviews
/cc @caesarxuchao @k82cn @sttts @kevin-wangzefeng 

**Release note**:
```release-note
NONE
```
2017-12-13 22:26:01 -08:00
Vladimir Vivien
7405159558 VolumeHost.GetNodeName method added for CSI fix 2017-12-02 05:54:54 -05:00
Michelle Au
c26debecef Return no volume match if prebound PV node affinity doesn't match node 2017-11-29 17:29:58 -08:00
pospispa
e1312f2c00 Addressing Comments from Code Review
Addressing comments from code review (https://github.com/kubernetes/kubernetes/pull/55824#pullrequestreview-78597250) in order to simplify the code.
2017-11-29 15:50:23 +01:00
supereagle
032416c75d use core client with explicit version
fix more usage of deprecated core client
2017-11-25 08:14:10 +08:00
jsafrane
4ad4ee3153 Added PVC Protection Controller
This controller removes protection finalizer from PVCs that are being
deleted and are not referenced by any pod.
2017-11-23 11:46:34 +01:00
Kubernetes Submit Queue
6a889ec37f
Merge pull request #55039 from msau42/local-binding-4
Automatic merge from submit-queue (batch tested with PRs 51321, 55969, 55039, 56183, 55976). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Topology aware volume scheduler and PV controller changes

**What this PR does / why we need it**:
Scheduler and PV controller changes to support volume topology aware scheduling, as specified in kubernetes/community#1168

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #54435

**Special notes for your reviewer**:
* I've split the PR into logical commits to make it easier to review
* The remaining TODOs I plan to address next release unless you think it needs to be done now

**Release note**:
```release-note
Adds alpha support for volume scheduling, which allows the scheduler to make PersistentVolume binding decisions while respecting the Pod's scheduling requirements.  Dynamic provisioning is not supported with this feature yet.

Action required for existing users of the LocalPersistentVolumes alpha feature:
* The VolumeScheduling feature gate also has to be enabled on kube-scheduler and kube-controller-manager.
* The NoVolumeNodeConflict predicate has been removed.  For non-default schedulers, update your scheduler policy.
* The CheckVolumeBinding predicate has to be enabled in non-default schedulers.
```

@kubernetes/sig-storage-pr-reviews @kubernetes/sig-scheduling-pr-reviews
2017-11-22 11:59:55 -08:00
Michelle Au
a84e5b9613 update build files 2017-11-21 23:19:44 -08:00
Michelle Au
fa6b62fa63 Library for scheduler volume binding 2017-11-21 21:42:36 -08:00
Michelle Au
b49a1ce1a4 Cache for pod bindings 2017-11-21 21:42:36 -08:00
Michelle Au
10800e68ac Add assume cache for PVs 2017-11-21 21:42:35 -08:00
Michelle Au
58823a75a4 In findMatchingVolumes:
- Delay binding until Pod is scheduled
- Check for PV node affinity and access modes
- Exclude volumes from being matching
2017-11-21 21:42:35 -08:00
Kubernetes Submit Queue
db7bb9e2ed
Merge pull request #51494 from mtanino/pr/BlockVolumesSupport-volumemanager
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Block volumes Support: CRI, volumemanager and operationexecutor changes

**What this PR does / why we need it**:
This PR contains following items to enable block volumes support feature.
- container runtime interface change
- volumemanager changes
- operationexecuto changes


**Which issue this PR fixes**:
Based on this proposal (kubernetes/community#805) and this feature issue: kubernetes/features#351

**Special notes for your reviewer**:

There are another PRs related to this functionality.
(#50457) API Change
(#53385) VolumeMode PV-PVC Binding change
(#51494) Container runtime interface change, volumemanager changes, operationexecutor changes
(#55112) Block volume: Command line printer update
Plugins
(#51493) Block volumes Support: FC plugin update
(#54752) Block volumes Support: iSCSI plugin update

**Release note**:
```
Adds alpha support for block volume, which allows the users to attach raw block volume to their pod without filesystem on top of the volume.
```
/cc @msau42 @liggitt @jsafrane @saad-ali @erinboyd @screeley44
2017-11-21 19:43:27 -08:00
Michelle Au
3d0b510e25 Refactor bindVolumeToClaim() into 1) forming new PV object 2) making API and cache update 2017-11-21 19:11:36 -08:00
Michelle Au
3211b8b0c4 Refactor PV selection into a common call for scheduler and PV controller 2017-11-21 19:11:36 -08:00
Scott Creeley
74fcd97c84 fixing issue of feature gate not being turned off properly 2017-11-21 15:19:18 -05:00
Scott Creeley
86742a2c92 VolumeMode binding logic update 2017-11-20 15:18:19 -05:00
Mitsuhiro Tanino
9329dd5b91 generated code for Volumemanager change 2017-11-20 14:10:34 -05:00
mtanino
8903e8cd85 BlockVolumesSupport: CRI, VolumeManager and OperationExecutor changes
This patch contains following changes.
- container runtime changes for adding block devices
- volumemanager changes
- operationexecutor changes
2017-11-20 14:10:26 -05:00
Kubernetes Submit Queue
1e9204ba44
Merge pull request #55144 from houjun41544/20171106
Automatic merge from submit-queue (batch tested with PRs 54987, 55221, 54099, 55144, 54215). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Should not sync the volume when  claim.Spec.VolumeName is null 

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```
2017-11-10 14:51:30 -08:00
Dr. Stefan Schimanski
bec617f3cc Update generated files 2017-11-09 12:14:08 +01:00
Dr. Stefan Schimanski
012b085ac8 pkg/apis/core: mechanical import fixes in dependencies 2017-11-09 12:14:08 +01:00
houjun
f49634719c Should return when claim.Spec.VolumeName is null 2017-11-09 10:47:04 +08:00
Di Xu
13a355c837 refactor method to pkg/util/node 2017-11-06 09:51:09 +08:00
Kubernetes Submit Queue
443908193d
Merge pull request #54652 from msau42/update-owners
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update volume OWNERS to reflect active sig-storage reviewers

**What this PR does / why we need it**:
Update sig-storage reviewers to add new members and remove those that don't have as much time to review storage PRs.   Approvers are unchanged.

**Special notes for your reviewer**:
For all those that have been removed, please approve.  If you want to remain as a reviewer, let me know and I will add you back.

**Release note**:
NONE
2017-11-01 16:30:32 -07:00
guangxuli
cb73ab2b07 The printing level for node updated failed info should be used WARNING type
just use Warning instead of Warningf
2017-11-01 16:54:49 +08:00
Kevin
4c8539cece use core client with explicit version globally 2017-10-27 15:48:32 +08:00
Michelle Au
7d1e279d3b Update volume OWNERS to reflect active sig-storage reviewers 2017-10-26 13:26:33 -07:00
Kubernetes Submit Queue
912e6f86e4 Merge pull request #51402 from hzxuzhonghu/event-broadcaster
Automatic merge from submit-queue (batch tested with PRs 51402, 54283). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add event broadcaster logging for all contoller managers

**What this PR does / why we need it**:
add event broadcaster logging for controller -manager, and it help debug/trace resource status directly.
 
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #51400

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-10-20 08:06:14 -07:00
Hemant Kumar
e3f0c8bb2d Fixes spam from node status updates
The same error is logged in 2 places which is unncessary.
2017-10-19 09:37:07 -04:00
hzxuzhonghu
96b48d4386 add event broadcaster logging for all contoller managers 2017-10-19 09:18:43 +08:00
Kubernetes Submit Queue
1d8f1e268f Merge pull request #47699 from supereagle/fix-typos
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix typos: remove duplicated word in comments

**What this PR does / why we need it**: Remove the duplicated word `the` in comments

**Which issue this PR fixes** : fixes #

**Special notes for your reviewer**:

```release-note
NONE
```
2017-10-17 02:35:52 -07:00
Jeff Grafton
aee5f457db update BUILD files 2017-10-15 18:18:13 -07:00
Hemant Kumar
68d417d7d8 Fix possibly flake in multiattach unit test
It is possible that by the time we check for multiattach
error on node, the reconciler loop may not have processed second
volume and hence we are going to retry for multiattach error
on node before giving up and marking the test as failed.
2017-10-12 16:27:54 -04:00
Hemant Kumar
414c3104ca Make sure we use rwlocks not just RLock 2017-10-10 17:52:55 -04:00
Kubernetes Submit Queue
2aa0704c0e Merge pull request #51754 from huzhengchuan/fix/event_name
Automatic merge from submit-queue (batch tested with PRs 51754, 53261, 53450). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Rename the name of event source in controller-manager

**What this PR does / why we need it**: Make the event source more obvious 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #51753

**Special notes for your reviewer**:

**Release note**:

```
NONE
```
2017-10-04 13:13:15 -07:00
Hemant Kumar
67d4c40849 Fix spam of multiattach errors in event logs
We should be careful while generating multiattach errors.
We seem to be generating too many of them because old code
had minor bug.
2017-10-03 15:45:06 -04:00
Jan Safranek
c3c2ed8b60 Send VolumeMismatch event to PVC
PV controller should send events to PVCs and not PVs when a PVC requests PV
that's either too small or has mismatching StorageClass.

Regular users can't see events on PVs so `kubectl describe pvc` is the only
way how to tell user that something is wrong.
2017-10-02 12:21:42 +02:00
Kubernetes Submit Queue
bfaaa77263 Merge pull request #53098 from gnufied/add-volume-expand-comments
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add documentation comments for volume expand controller

These comments help clear out some of the design choices made
in code.

cc @kubernetes/sig-storage-pr-reviews @NickrenREN
2017-09-28 10:12:20 -07:00
Kubernetes Submit Queue
5a721f5a02 Merge pull request #53065 from msau42/add-reviewers
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add more reviewers for volume components

**Release note**:

NONE
2017-09-26 23:52:02 -07:00
Kubernetes Submit Queue
1cffa70c0d Merge pull request #52221 from gnufied/fix-detach-delay-mount-node
Automatic merge from submit-queue (batch tested with PRs 52990, 53064, 52686, 52221, 53069). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Always populate volume status from node

Fixes https://github.com/kubernetes/kubernetes/issues/52036 

As discussed offline with @jingxu97 

/sig storage
2017-09-26 23:12:30 -07:00
Hemant Kumar
0ad846cb18 Add documentation comments for volume expand controller
These comments help clear out some of the design choices made
in code.
2017-09-26 22:25:23 -04:00
Michelle Au
e6687ad5c6 Add more reviewers for volume components 2017-09-26 10:24:21 -07:00
Jeff Grafton
02fb4200dc Use buildozer to delete licenses() rules 2017-09-21 15:53:22 -07:00
Jeff Grafton
532bd482df Use buildozer to remove deprecated automanaged tags 2017-09-21 15:53:22 -07:00
supereagle
87c29a08e1 fix typos: remove duplicated word in comments 2017-09-16 14:38:10 +08:00
Kubernetes Submit Queue
1c55faf0bb Merge pull request #51387 from alrs/fix-storageos-swallowed-err
Automatic merge from submit-queue

Fix swallowed errors in various volume packages

**What this PR does / why we need it**: Fixes swallowed errors in various volume packages.

**Release note**:
```release-note NONE
```
2017-09-13 11:10:24 -07:00
Hemant Kumar
8edae9b3fc Always populate volume status from node 2017-09-12 09:03:42 -04:00
Jan Safranek
e054dd297a Fix panic in expand controller when checking PVs
Unbound PVs have their Spec.ClaimRef = nil, so we should not dereference it blindly.

In addition, increase AddPVCUpdate test coverage to 100%
2017-09-06 13:27:10 +02:00
zhengchuan hu
2b8891a59f rename the name of eventsource in controller-manager 2017-09-04 23:30:59 +08:00
Hemant Kumar
cd2a68473a Implement controller for resizing volumes 2017-09-04 09:02:34 +02:00
David Zhu
a60965337c Fixed integer overflow when matching PVPVC claims. Added tests to guard this behavior. 2017-09-01 10:10:13 -07:00
Kubernetes Submit Queue
583c4a442a Merge pull request #51228 from wongma7/mount-options-sc
Automatic merge from submit-queue

Add storageClass.mountOptions and use it in all applicable plugins

split off from https://github.com/kubernetes/kubernetes/pull/50919 and still dependent on it. cc @gnufied


issue: https://github.com/kubernetes/features/issues/168

```release-note
Add mount options field to StorageClass. The options listed there are automatically added to PVs provisioned using the class.
```
2017-08-29 23:48:32 -07:00
Kubernetes Submit Queue
b4a2c09ede Merge pull request #50738 from sttts/sttts-deepcopy-calls-controllers
Automatic merge from submit-queue (batch tested with PRs 49961, 50005, 50738, 51045, 49927)

controllers: simplify deepcopy calls
2017-08-29 21:43:35 -07:00
Dr. Stefan Schimanski
1d053c4f7c controllers: simplify deepcopy calls 2017-08-29 19:21:24 +02:00
Matthew Wong
5e772b8e4b Add storageClass.mountOptions and use it in all applicable plugins 2017-08-29 11:37:36 -04:00
Jan Safranek
195d5c1546 Fix handling of APIserver errors when saving provisioned PVs.
When API server crashes *after* saving a provisioned PV and before sending
200 OK, the controller tries to save the PV again. In this case, it gets
AlreadyExists error, which should be interpreted as success and not as error.

Especially, a volume that corresponds to the PV should not be deleted in the
underlying storage.
2017-08-29 09:47:32 +02:00
Kubernetes Submit Queue
520eab779e Merge pull request #51356 from wongma7/pv-cap-resize
Automatic merge from submit-queue (batch tested with PRs 51441, 51356, 51460)

Don't update pvc.status.capacity if pvc is already Bound

As discussed here https://github.com/kubernetes/community/pull/657#discussion_r128008128, in order for `pvc.status.Capacity < pv.Spec.Capcity` to be the mechanism for volume filesystem* resize, the pv controller should stop updating pvc.status.Capacity every resync period.

/assign @jsafrane
/sig storage
```release-note
NONE
```
2017-08-28 06:41:25 -07:00
Kubernetes Submit Queue
2a2f499455 Merge pull request #50036 from wongma7/metrics
Automatic merge from submit-queue

Add volume operation metrics to operation executor and PV controller

This PR implements the proposal for high level volume metrics https://github.com/kubernetes/community/pull/809

**Special notes for your reviewer**:

~Differences from proposal:~ all resolved

~"verify_volume" is now "verify_volumes_are_attached" + "verify_volumes_are_attached_per_node" + "verify_controller_attached_volume." Which of them do we want?~

~There is no "mount_device" metric because the MountVolume operation combines MountDevice and mount (plugin.Setup). Do we want to extract the mount_device metric or is it okay to keep mountvolume as one? For attachable volumes, MountDevice is the actual mount and Setup is a bindmount + setvolumeownership. For unattachable, mountDevice does not occur and Setup is an actual mount + setvolumeownership.~

~PV controller metrics I did not implement following the proposal at all. I did not change goroutinemap nor scheduleOperation. Because provisionClaimOperation does not return an error, so it's impossible for the caller to know if there is actually a failure worth reporting. So I manually create a new metric inside the function according to some conditions.~

@gnufied 

I have tested the operationexecutor metrics but not provision & delete. Sample: 
![screen shot 2017-08-02 at 15 01 08](https://user-images.githubusercontent.com/13111288/28889980-a7093526-7793-11e7-9aa9-ad7158be76fa.png)


**Release note**:

```release-note
Add error count and time-taken metrics for storage operations such as mount and attach, per-volume-plugin.
```
2017-08-28 04:20:49 -07:00
Lars Lehtonen
7fb3dbabdd
Merge branch 'fix-attachdetach-swallowed-error' into fix-storageos-swallowed-err 2017-08-26 14:50:46 -07:00
mtanino
5ff9dc0b3b WaitForAttach refactoring for iSCSI attacher/detacher
This change is prerequisite for implementing iSCSI attacher
and detacher.

In order to use chap authentication at iSCSI plugin after
implementing attacher and detacher, secret is needed at
AttachDisk() which is called from WaitForAttach().
To obtain secret, pod information is required, but
WaitForAttach() doesn't pass pod information inside.

This patch adds 'pod' as an argument of WaitForAttach()
and adds changes to drivers who implements WaitForAttach().

Fixes #48953
2017-08-26 17:21:34 -04:00
Matthew Wong
19ebaf2870 Don't update pvc.status.capacity if pvc is already Bound 2017-08-25 15:23:25 -04:00
Cheng Xing
396c3c7c6f Adding dynamic Flexvolume plugin discovery capability, using filesystem watch. 2017-08-25 11:42:32 -07:00
Kubernetes Submit Queue
b75d423979 Merge pull request #51066 from vmware/MultiAttachVolumeIssueVsphere
Automatic merge from submit-queue

Allow attach of volumes to multiple nodes for vSphere

This is a fix for issue #50944 which doesn't allow a volume to be attached to a new node after the node is powered off where the volume was previously attached.

Current behaviour:
One of the cluster worker nodes was powered off in vCenter.
Pods running on this node have been rescheduled on different nodes but got stuck in ContainerCreating. It failed to attach the volume on the new node with error "Multi-Attach error for volume pvc-xxx, Volume is already exclusively attached to one node and can't be attached to another" and hence the application running in the pod has no data available because the volume is not attached to the new node. Since the volume is still attached to powered off node, any attempt to attach the volume on the new node failed with error "Multi-Attach error". It's stuck for 6 minutes until attach/detach controller forcefully tried to detach the volume on the powered off node. After the end of 6 minutes when volume is detached on powered off node, the volume is now successfully attached on the new node and application has now the data available.

What is expected to happen:
I would want the attach/detach controller to go ahead with the attach of the volume on new node where the pod got provisioned instead of waiting for the volume to be detached on the powered off node. It is ok to eventually delete the volume on the powered off node after 6 minutes. This way the application downtime is low and pods are up as soon as possible.

The current fix ignore, vSphere volumes/persistent volume to check for multi-attach scenario in attach/detach controller.

@jingxu97 @saad-ali : Can you please take a look at it.

@tusharnt @divyenpatel @rohitjogvmw @luomiao 

```release-note
Allow attach of volumes to multiple nodes for vSphere
```
2017-08-23 14:32:31 -07:00
Matthew Wong
3ed34183d0 Add volume operation metrics to operation executor and PV controller 2017-08-23 14:27:47 -04:00
Lars Lehtonen
3893e28b66
Fix swallowed error in attachdetach tests 2017-08-22 23:35:54 -07:00
Kubernetes Submit Queue
70632276bb Merge pull request #50806 from verult/VolumeNotYetAttached
Automatic merge from submit-queue (batch tested with PRs 50806, 48789, 49922, 49935, 50438)

On AttachDetachController node status update, do not retry when node …

…doesn't exist but keep the node entry in cache.



**What this PR does / why we need it**: An alternative fix for https://github.com/kubernetes/kubernetes/issues/42438 which also fixes #50721.

Instead of removing the node entry entirely from the node status update cache (which prevents the node from ever being updated even when it recovers), here the node status updater does nothing, so that there won't be an update retry until the node is re-added, where the cache entry is set to true.

Will cherry pick to prior versions after this is merged.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #50721 

**Release Note**:
``` release-note
On AttachDetachController node status update, do not retry when node doesn't exist but keep the node entry in cache.
```

/assign @jingxu97 
/cc @saad-ali 
/sig storage
/release-note
2017-08-22 19:45:27 -07:00
Kubernetes Submit Queue
198e83588b Merge pull request #46458 from jsafrane/mount-prep
Automatic merge from submit-queue (batch tested with PRs 46458, 50934, 50766, 50970, 47698)

Prepare VolumeHost for running mount tools in containers

This is the first part of implementation of https://github.com/kubernetes/features/issues/278 - running mount utilities in containers.

It updates `VolumeHost` interface:

*  `GetMounter()` now requires volume plugin name, as it is going to return different mounter to different volume plugings, because mount utilities for these plugins can be on different places.
* New `GetExec()` method that should volume plugins use to execute any utilities. This new `Exec` interface will execute them on proper place.
* `SafeFormatAndMount` is updated to the new `Exec` interface.

This is just a preparation, `GetExec` right now leads to simple `os.Exec` and mount utilities are executed on the same place as before. Also, the volume plugins will be updated in subsequent PRs (split into separate PRs, some plugins required lot of changes).

```release-note
NONE
```

@kubernetes/sig-storage-pr-reviews 
@rootfs @gnufied
2017-08-21 18:11:16 -07:00
Balu Dontu
cfdff1ae46 Multi-Attach volume fix for vSphere 2017-08-21 18:06:29 -07:00
Cheng Xing
1234d2f500 On AttachDetachController node status update, do not retry when node doesn't exist but keep the node entry in cache 2017-08-16 15:42:15 -07:00
Matthew Wong
0356a840ff Add ReclaimPolicy field to StorageClass 2017-08-14 13:34:32 -04:00
Jan Safranek
bc0e170d9c Add pluginName to VolumeHost.GetMouter
Different plugins can get different mounter, depending where the mount
utilities are.
2017-08-14 12:16:26 +02:00
Jan Safranek
282404cbc9 Add Exec interface to VolumeHost
This exec should be used by volume plugins to execute mount utilities.
It will eventually execute things in mount containers.
2017-08-14 12:16:25 +02:00
Jeff Grafton
a7f49c906d Use buildozer to delete licenses() rules except under third_party/ 2017-08-11 09:32:39 -07:00
Jeff Grafton
33276f06be Use buildozer to remove deprecated automanaged tags 2017-08-11 09:31:50 -07:00
Kenneth Owens
5941f7b69f Merge pull request #49016 from jsafrane/pv-controller-no-resync
PV controller: resync informers manually
merging to unblock the submit queue
2017-08-01 16:39:40 -07:00
Hemant Kumar
f4e792ed42 Log attach detach controller skipping pods at higher priority
This will help us in tracking down problems related to pods
not getting added to desired state of world because of events
arriving out of order or some other problem related to that.
2017-07-28 13:23:28 -04:00
supereagle
adc0eef43e remove duplicated import and wrong alias name of api package 2017-07-25 10:04:25 +08:00
Dr. Stefan Schimanski
5925a0a1df Move pkg/api/v1/ref -> client-go/tools/reference 2017-07-18 22:45:43 +02:00
Jacob Simpson
b565f53822 update-bazel.sh 2017-07-17 15:06:08 -07:00
Chao Xu
9d489c8504 manual changes 2017-07-17 15:05:38 -07:00
Jacob Simpson
a765b8cfca Migrate api.Scheme to scheme.Scheme 2017-07-17 15:05:38 -07:00
Jacob Simpson
29c1b81d4c Scripted migration from clientset_generated to client-go. 2017-07-17 15:05:37 -07:00
Jan Safranek
0eface85e4 PV controller: resync informers manually
We want relatively short resync period of PV/PVCs and at the same time we
don't want to force such short resync to all shared informer consumers.
Therefore we need to make our own periodic resync.
2017-07-17 13:39:08 +02:00
Alexander Block
61275ad8d4 Fix flaky test Test_Run_OneVolumeAttachAndDetachMultipleNodesWithReadWriteMany
Only relying on the NewAttacher/Detacher call counts is not enough as they
happen in parallel to the testing/verification code and thus the actual
attaching/detaching may not be done yet, resulting in flaky test results.

Fixes #46244
2017-07-11 18:21:50 +02:00
Kubernetes Submit Queue
c662e1d7d8 Merge pull request #46949 from xingzhou/typo
Automatic merge from submit-queue

Fixed a comment typo

Typo fix

Fixed #48414 

**Release note**:
```
None
```
2017-07-03 11:33:36 -07:00
Kubernetes Submit Queue
d19a2841e3 Merge pull request #47645 from jsafrane/integration-test-speedup
Automatic merge from submit-queue (batch tested with PRs 48139, 48042, 47645, 48054, 48003)

Speed up attach/detach controller integration tests

Internal attach/detach controller timers should be configurable and tests should use much shorter values.

`reconcilerSyncDuration` is deliberately left out of `TimerConfig` because it's the only one that's not a constant one, it's configurable by user.

Fixes #47129 

Before:
```
--- PASS: TestPodDeletionWithDswp (63.21s)
--- PASS: TestPodUpdateWithWithADC (13.68s)
--- PASS: TestPodUpdateWithKeepTerminatedPodVolumes (13.55s)
--- PASS: TestPodAddedByDswp (183.01s)
--- PASS: TestPersistentVolumeRecycler (12.55s)
--- PASS: TestPersistentVolumeDeleter (12.54s)
--- PASS: TestPersistentVolumeBindRace (3.51s)
--- PASS: TestPersistentVolumeClaimLabelSelector (12.50s)
--- PASS: TestPersistentVolumeClaimLabelSelectorMatchExpressions (12.54s)
--- PASS: TestPersistentVolumeMultiPVs (3.05s)
--- PASS: TestPersistentVolumeMultiPVsPVCs (4.36s)
--- PASS: TestPersistentVolumeControllerStartup (7.29s)
--- PASS: TestPersistentVolumeProvisionMultiPVCs (5.02s)
--- PASS: TestPersistentVolumeMultiPVsDiffAccessModes (12.48s)
ok  	k8s.io/kubernetes/test/integration/volume	359.727s
```

After:
```
--- PASS: TestPodDeletionWithDswp (3.71s)
--- PASS: TestPodUpdateWithWithADC (3.63s)
--- PASS: TestPodUpdateWithKeepTerminatedPodVolumes (3.70s)
--- PASS: TestPodAddedByDswp (5.68s)
--- PASS: TestPersistentVolumeRecycler (12.54s)
--- PASS: TestPersistentVolumeDeleter (12.55s)
--- PASS: TestPersistentVolumeBindRace (3.55s)
--- PASS: TestPersistentVolumeClaimLabelSelector (12.50s)
--- PASS: TestPersistentVolumeClaimLabelSelectorMatchExpressions (12.52s)
--- PASS: TestPersistentVolumeMultiPVs (3.98s)
--- PASS: TestPersistentVolumeMultiPVsPVCs (3.85s)
--- PASS: TestPersistentVolumeControllerStartup (7.18s)
--- PASS: TestPersistentVolumeProvisionMultiPVCs (5.23s)
--- PASS: TestPersistentVolumeMultiPVsDiffAccessModes (12.48s)
ok  	k8s.io/kubernetes/test/integration/volume	103.267s
```

PV controller tests are the slowest ones now.

@kubernetes/sig-storage-pr-reviews 
/assign @gnufied 

```release-note
NONE
```
2017-06-27 14:08:17 -07:00
Kubernetes Submit Queue
18362beb0d Merge pull request #42254 from justinsb/volumes_dont_leak_nodestatusupdateneeded
Automatic merge from submit-queue

volumes: SetNodeStatusUpdateNeeded on error

If an error happened during the UpdateNodeStatuses loop, there were some
code paths where we would not call SetNodeStatusUpdateNeeded, leaking
the state.  Add it to all paths by adding a function.

Part of #40583

```release-note
NONE
```
2017-06-22 21:43:04 -07:00
Chao Xu
60604f8818 run hack/update-all 2017-06-22 11:31:03 -07:00
Chao Xu
f2d3220a11 run root-rewrite-import-client-go-api-types 2017-06-22 11:30:59 -07:00
Chao Xu
cde4772928 run ./root-rewrite-all-other-apis.sh, then run make all, pkg/... compiles 2017-06-22 11:30:52 -07:00
Chao Xu
f4989a45a5 run root-rewrite-v1-..., compile 2017-06-22 10:25:57 -07:00
Kubernetes Submit Queue
d0a2beb1e7 Merge pull request #42249 from justinsb/volumes_logging
Automatic merge from submit-queue (batch tested with PRs 42252, 42251, 42249, 47512, 47887)

volumes: Add logging when removing node fails

Part of #40583

```release-note
NONE
```
2017-06-21 22:13:30 -07:00
Kubernetes Submit Queue
b795ec7de0 Merge pull request #42251 from justinsb/simplify_append
Automatic merge from submit-queue (batch tested with PRs 42252, 42251, 42249, 47512, 47887)

volumes: simplify append-to-slice code

Minor simplification - can append to empty/nil slice.

Part of #40583

```release-note
NONE
```
2017-06-21 22:13:27 -07:00
Kubernetes Submit Queue
bebe346d5f Merge pull request #42252 from justinsb/volumes_raise_loglevels
Automatic merge from submit-queue (batch tested with PRs 42252, 42251, 42249, 47512, 47887)

volumes: promote some logs from info -> warning

Part of #40583

```release-note
NONE
```
2017-06-21 22:13:24 -07:00
Kubernetes Submit Queue
2df2247a82 Merge pull request #42250 from justinsb/volumes_getnodeandvolume_comment
Automatic merge from submit-queue

volumes: add comment on getNodeAndVolume

Add comments on getNodeAndVolume to explain the code - it is a little
subtle, and it confused me on first reading.

Part of #40583

```release-note
NONE
```
2017-06-20 15:07:47 -07:00
Jan Safranek
b28790a63b Speed up attach/detach controller integration tests
Internal attach/detach controller timers should be configurable and tests
should use much shorter values.

reconcilerSyncDuration is deliberately left out of TimerConfig because it's
the only one that's not a constant one, it's configurable by user.
2017-06-16 12:15:04 +02:00
Matthew Wong
5e788a6a67 Don't provision for PVCs with AccessModes unsupported by plugin 2017-06-12 12:56:41 -04:00
Xing Zhou
750d0d8730 Fixed a comment typo 2017-06-05 10:47:59 +08:00
Kubernetes Submit Queue
0f7ddbddef Merge pull request #46490 from NickrenREN/controller-volume-event
Automatic merge from submit-queue

Extract volume relevant events reason

Extract volume relevant events reason and make them const


**Release note**:

```release-note
NONE
```
2017-06-01 07:11:05 -07:00
Justin Santa Barbara
d420531f95 volumes: SetNodeStatusUpdateNeeded on error
If an error happened during the UpdateNodeStatuses loop, there were some
code paths where we would not call SetNodeStatusUpdateNeeded, leaking
the state.  Add it to all paths by adding a function.

Part of #40583
2017-06-01 00:32:20 -04:00
NickrenREN
e62bdf82da Extract volume relevant events reason
Extract volume relevant events reason and make them const
2017-05-31 22:11:36 +08:00
Shyam Jeedigunta
4425864707 Migrate kubelet configmap management logic to an interface 2017-05-31 10:39:36 +02:00
Kubernetes Submit Queue
0aad9d30e3 Merge pull request #44897 from msau42/local-storage-plugin
Automatic merge from submit-queue (batch tested with PRs 46076, 43879, 44897, 46556, 46654)

Local storage plugin

**What this PR does / why we need it**:
Volume plugin implementation for local persistent volumes.  Scheduler predicate will direct already-bound PVCs to the node that the local PV is at.  PVC binding still happens independently.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 
Part of #43640

**Release note**:

```
Alpha feature: Local volume plugin allows local directories to be created and consumed as a Persistent Volume.  These volumes have node affinity and pods will only be scheduled to the node that the volume is at.
```
2017-05-30 23:20:02 -07:00
Kubernetes Submit Queue
c77b74e328 Merge pull request #46486 from NickrenREN/pv-provisioner-check
Automatic merge from submit-queue

Optimize provisioner plugin result check logic

If err is not returned by findProvisionablePlugin(...), storageClass is certainly not nil


**Release note**:

```release-note
NONE
```
2017-05-29 05:31:38 -07:00
Kubernetes Submit Queue
c34b359bd7 Merge pull request #45923 from verult/cxing/NodeStatusUpdaterFix
Automatic merge from submit-queue (batch tested with PRs 46383, 45645, 45923, 44884, 46294)

Node status updater now deletes the node entry in attach updates...

… when node is missing in NodeInformer cache.

- Added RemoveNodeFromAttachUpdates as part of node status updater operations.



**What this PR does / why we need it**: Fixes issue of unnecessary node status updates when node is deleted.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #42438

**Special notes for your reviewer**: Unit tested added, but a more comprehensive test involving the attach detach controller requires certain testing functionality that is currently absent, and will require larger effort. Will be added at a later time.

There is an edge case caused by the following steps:
1) A node is deleted and restarted. The node exists, but is not yet recognized by Kubernetes.
2) A pod requiring a volume attach with nodeName specifically set to this node.

This would make the pod stuck in ContainerCreating state. This is low-pri since it's a specific edge case that can be avoided.

**Release note**:

```release-note
NONE
```
2017-05-26 12:58:02 -07:00
NickrenREN
51e2336476 Optimize provisioner plugin result check logic
If err is not returned by findProvisionablePlugin(...), storageClass is certainly not nil
2017-05-26 13:57:40 +08:00
Cheng Xing
f9dc2d5ca3 Node status updater now deletes the node entry in attach updates when node is missing in NodeInformer cache. Fixes #42438.
- Added RemoveNodeFromAttachUpdates as part of node status updater operations.
2017-05-24 18:31:47 -07:00
NickrenREN
add091b1fb fix regression in UX experience for double attach volume
send event when volume is not allowed to multi-attach
2017-05-25 09:27:24 +08:00
Justin Santa Barbara
35be997c2f volumes: promote some logs from info -> warning
Part of #40583
2017-05-23 22:36:42 -04:00
Michelle Au
6ade5461ad Add GetNodeLabels to VolumeHost interface 2017-05-22 14:44:06 -07:00
Alexander Block
06baeb33b2 Don't try to attach volumes which are already attached to other nodes 2017-05-18 06:56:30 +02:00
Kubernetes Submit Queue
6dbe853e29 Merge pull request #45544 from ianchakeres/reconciler-err-cleanup
Automatic merge from submit-queue (batch tested with PRs 45990, 45544, 45745, 45742, 45678)

Refactor reconciler volume log and error messages

**What this PR does / why we need it**:
Utilizes volume-specific error and log messages introduced in #44969, inside files that also log volume information. 

Specifically: 

- pkg/kubelet/volumemanager/reconciler/reconciler.go, 
- pkg/controller/volume/attachdetach/reconciler/reconciler.go, and
- pkg/kubelet/volumemanager/populator/desired_state_of_world_populator.go


**Which issue this PR fixes** : fixes #40905

**Special notes for your reviewer**:

**Release note**:

```release-note
```
NONE
2017-05-17 18:40:51 -07:00
Kubernetes Submit Queue
521d7d1ac5 Merge pull request #42472 from timchenxiaoyu/requesttypo
Automatic merge from submit-queue

fix request typo
2017-05-15 15:57:57 -07:00
Ian Chakeres
b1315f4491 Refactor reconciler volume log and error messages 2017-05-11 22:33:17 -07:00
Hemant Kumar
951a36aac7 Add Keepterminatedpodvolumes as a annotation on node
and lets make sure that controller respects it
and doesn't detaches mounted volumes.
2017-05-11 22:31:14 -04:00
Hemant Kumar
9a1a9cbe08 detach the volume when pod is terminated
Make sure volume is detached when pod is terminated because
of any reason and not deleted from api server.
2017-05-11 22:18:22 -04:00
NickrenREN
0861688237 add and clear err message in RemoveVolumeFromReportAsAttached 2017-05-08 09:37:21 +08:00
Jan Safranek
9d0c47f1db Use storage.v1 instead of v1beta1
storage.v1beta1 was used to work around GKE which does not expose v1. Now that
GKE is updated, we can switch everything to v1.
2017-04-25 10:13:38 +02:00
Tomas Smetana
3064fe4d39 Fix issue #44757: Flaky Test_AttachDetachControllerRecovery 2017-04-21 12:43:54 +02:00
Kubernetes Submit Queue
db53e80445 Merge pull request #39732 from tsmetana/issue_34242
Automatic merge from submit-queue (batch tested with PRs 44722, 44704, 44681, 44494, 39732)

Fix issue #34242: Attach/detach should recover from a crash

When the attach/detach controller crashes and a pod with attached PV is deleted afterwards the controller will never detach the pod's attached volumes. To prevent this the controller should try to recover the state from the nodes status and figure out which volumes to detach. This requires some changes in the volume providers too: the only information available from the nodes is the volume name and the device path. The controller needs to find the correct volume plugin and reconstruct the volume spec just from the name. This required a small change also in the volume plugin interface.

Fixes Issue #34242.
cc: @jsafrane @jingxu97
2017-04-20 16:01:04 -07:00
Tomas Smetana
852c44ae59 Fix issue #34242: Attach/detach should recover from a crash
When the attach/detach controller crashes and a pod with attached PV is deleted
afterwards the controller will never detach the pod's attached volumes. To
prevent this the controller should try to recover the state from the nodes
status.
2017-04-20 13:04:50 +02:00
NickrenREN
eca490bbdd Remove claimClass check and upgradeVolumeFrom1_2() function 2017-04-19 19:12:32 +08:00
NickrenREN
5cafb9042b find and add active pods for dswp
loops through the list of active pods and ensures that each one exists in the desired state of the world cache
2017-04-18 11:21:37 +08:00
Kubernetes Submit Queue
08ea5387f2 Merge pull request #44566 from wongma7/adc-wait
Automatic merge from submit-queue (batch tested with PRs 44469, 44566, 44467, 44526)

WaitForCacheSync before running attachdetach controller

@gnufied you wrote the test and @ncdc the TODO comment. Let's just run the pv and pvc informers, we do not care about them in this test. But we want to be able to stop the pod Informer at will, hence not just using informers.Start, is my understanding.
```release-note
NONE
```
2017-04-17 20:06:58 -07:00
Chao Xu
4f9591b1de move pkg/api/v1/ref.go and pkg/api/v1/resource.go to subpackages. move some functions in resource.go to pkg/api/v1/node and pkg/api/v1/pod 2017-04-17 11:38:11 -07:00
Matthew Wong
e1ce33d944 WaitForCacheSync before running attachdetach controller 2017-04-17 14:02:33 -04:00
Chao Xu
d4850b6c2b move pkg/api/v1/helpers.go to subpackage 2017-04-14 14:25:11 -07:00
Mike Danese
a05c3c0efd autogenerated 2017-04-14 10:40:57 -07:00
Andy Goldstein
e63fcf708d Make controller Run methods consistent
- startup/shutdown logging
- wait for cache sync logging
- defer utilruntime.HandleCrash()
- wait for stop channel before exiting
2017-04-14 07:27:45 -04:00
NickrenREN
e0ef5bfd40 Exit from NewController() for PersistentVolumeController when InitPlugins() failed just like NewAttachDetachController() does 2017-04-12 13:43:09 +08:00
Kubernetes Submit Queue
e60cc6ee3d Merge pull request #44090 from NickrenREN/remove-alpha-pv
Automatic merge from submit-queue

Remove alphaProvisioner in PVController and AlphaStorageClassAnnotation

remove alpha annotation and alphaProvisioner 

**Release note**:

```release-note
NONE
```
2017-04-11 20:41:40 -07:00
Kubernetes Submit Queue
50b104b8e2 Merge pull request #42313 from timchenxiaoyu/completelytypo
Automatic merge from submit-queue

fix completely typo
2017-04-10 05:57:06 -07:00
Kubernetes Submit Queue
3572e5ca86 Merge pull request #43679 from xingzhou/kube-42121
Automatic merge from submit-queue

Improve event msg for PV controller when using external provisioner

Improve event msg for PV controller when using external provisioner

**Which issue this PR fixes** *:
Fixed part of #42121

**Special notes for your reviewer**:
@jsafrane, as many of our users are confused by the original message, can we fix the message first and then consider how to control the count of the events?
2017-04-10 05:07:52 -07:00
NickrenREN
fa7bd44966 Remove alphaProvisioner in PVController and AlphaStorageClassAnnotation 2017-04-10 17:09:40 +08:00
Kubernetes Submit Queue
00743a4f2b Merge pull request #42629 from NickrenREN/pv-index
Automatic merge from submit-queue (batch tested with PRs 41775, 39678, 42629, 42524, 43028)

matchPredicate does not fit findByClaim()

matchPredicate has two args which are type of PV,and is not used in function findByClaim(),remove it


**Release note**:
```release-note
NONE
```
2017-04-07 17:44:16 -07:00
NickrenREN
67a55a4b02 matchPredicate does not fit findByClaim()
matchPredicate has two args which are type of PV,and is not used in function findByClaim(),remove it
2017-04-07 10:15:42 +08:00
Xing Zhou
5b29afb1ad Improve event msg for PV controller when using external provisioner
Improve event msg for PV controller when using external provisioner
2017-04-05 08:49:48 +08:00
Kubernetes Submit Queue
46d4c621a8 Merge pull request #42992 from NickrenREN/syncUnboundClaim
Automatic merge from submit-queue (batch tested with PRs 43453, 42992)

make sure that the volume satisfies the requirements of the claim before binding

check if the volume requested by the claim satisfies the requirements of the claim before binding when
syncUnboundClaim and claim.Spec.VolumeName is not set, although the volume is asked by user


**Release note**:
```release-note
NONE
```
2017-04-04 04:27:19 -07:00
Kubernetes Submit Queue
05c046f6d3 Merge pull request #43810 from gnufied/add-gnufied-vol-controller
Automatic merge from submit-queue

Adding gnufied as reviewer for volume controller

I have helped review several PRs and made new
PRs to this area.

cc @childsb @saad-ali
2017-04-03 12:46:25 -07:00
NickrenREN
f922af5138 make sure that the volume satisfies the requirements of the claim before binding
check if the volume requested by the claim satisfies the requirements of the claim before binding when
syncUnboundClaim and claim.Spec.VolumeName is not set
2017-04-04 01:03:37 +08:00
Kubernetes Submit Queue
50763cb6be Merge pull request #43627 from pospispa/make-constants-public-so-that-they-can-be-used-in-an-external-provisioner
Automatic merge from submit-queue

Make Constants Public so that They Can Be Used in an Ext. Provisioner

Out-of-tree external provisioners have the same purpose as in-tree provisioners. As external provisioners work with PV and PVC datastructures it's an advantage to import certain Kubernetes packages instead of copy-pasting the Kubernetes code.

That's why the constants are made public so that they can be used in an external provisioner.

@jsafrane  @kubernetes/sig-storage-pr-reviews 

```
NONE
```
2017-04-03 06:43:03 -07:00
Kubernetes Submit Queue
4fa902a915 Merge pull request #43933 from childsb/add_approver
Automatic merge from submit-queue

update pkg/controller/volume/OWNER to add appropriate approvers for both volume controllers

Update pkg/controller/volume approvers so that the attach/detach and binding controllers have approvers.
2017-04-01 11:37:15 -07:00
Kubernetes Submit Queue
fff5fae0a0 Merge pull request #43289 from tsmetana/adc-race-fix
Automatic merge from submit-queue

Attach/detach controller: fix potential race in constructor

**What this PR does / why we need it**:
There is a potential race condition in the Attach/detach controller: The "constructor" first installs informer event handlers and then creates and initializes the other data structures. However there is no guarantee an event cannot arrive before the data structures required by the event handlers are ready. This may result in nil pointer derefernces and potential crashes (e.g. the nodeAdd method calls adc.actualStateOfWorld.SetNodeStatusUpdateNeeded even though the actualStateOfWorld might be still nil).

It should be enough just to move the event handlers installation at the end of the constructor function.

**Release note**:

```release-note
NONE
```
2017-03-31 17:30:36 -07:00
childsb
308a3a8c45 Update OWNERS with approvers to cover both volume controllers. 2017-03-31 18:26:42 -05:00
Hemant Kumar
39ec7c7400 Adding myself as reviewer for volume controller
I have helped review several PRs and made new
PRs to this area.
2017-03-29 10:42:44 -04:00
Kubernetes Submit Queue
81d9e7f68a Merge pull request #42721 from NickrenREN/pv-provisionClaimOperation
Automatic merge from submit-queue

fix createProvisionedPV result err judgenment bug

When ctrl.kubeClient.Core().PersistentVolumes().Create(volume) returns no err, and storeVolumeUpdate() fails, we save PV sucessfully ,but here err is not nil,we should not run the codes next in block if err != nil {} to delete the storage asset.
same in the deletion retries below
change the err names to make it clear
**Release note**:
```release-note
NONE
```
@jsafrane @saad-ali  PTAL. Thanks
2017-03-26 23:30:34 -07:00
NickrenREN
0ab9f6fe55 fix createProvisionedPV result err judgenment bug
When ctrl.kubeClient.Core().PersistentVolumes().Create(volume) returns no err, but storeVolumeUpdate() failed, we save PV sucessfully ,but here err is not nil,
we should not run the codes next in block if err != nil {}
same in the deletion retries below
2017-03-25 10:48:16 +08:00
NickrenREN
5d1c7c77b4 Fix PVC Annotations
annDynamicallyProvisioned is added to PV, PVC has annStorageProvisioner
2017-03-25 10:42:04 +08:00
Kubernetes Submit Queue
d585eb3035 Merge pull request #42631 from NickrenREN/pv-isVolumeBoundToClaim-1
Automatic merge from submit-queue (batch tested with PRs 42522, 42545, 42556, 42006, 42631)

optimize the binding logic of bindClaimToVolume

extract var shouldSetBoundByController and do not need to judge volumename twice
**Release note**:
```release-note
NONE
```
2017-03-24 15:10:35 -07:00
Kubernetes Submit Queue
d2175bab9a Merge pull request #42522 from NickrenREN/pv-err-print
Automatic merge from submit-queue

print err message when update store failed

```release-note
NONE
```
2017-03-24 15:06:49 -07:00
Kubernetes Submit Queue
6ae0199eb0 Merge pull request #42402 from jorenhehe/pv-typo
Automatic merge from submit-queue

fix pv_controller typos

```release-note
NONE
```
2017-03-24 10:25:55 -07:00
pospispa
1fdd163dd1 Make Some Constants Public so that They Can Be Used in an External Provisioner
Out-of-tree external provisioners have the same purpose as in-tree provisioners. As external provisioners work with PV and PVC datastructures it's an advantage to import certain Kubernetes packages instead of copy-pasting the Kubernetes code.

That's why the constants are made public so that they can be used in an external provisioner.
2017-03-24 17:09:51 +01:00