Commit Graph

260 Commits

Author SHA1 Message Date
Yecheng Fu
1e27a28dca Ignore non-avaiable volumes in findMatchingVolume: update tests
- Use VolumeAvaiable instead of empty or pending phase in tests
- Add a test case to verify findMatchingVolume will not choose
  non-avaiable PVs if it's not pre-bind
- Add a test case to verify syncClaim will not choose non-avaibalbe PVs
  if it's not pre-bind
2018-09-19 13:13:31 +08:00
Yecheng Fu
b3cb1bdc4a Ignore non-available volumes in findMatchingVolume to reduce chance of
unnecessary binding failures.
2018-09-14 20:27:44 +08:00
Michelle Au
8fcbcafc11 Use rwlock for caches 2018-09-04 16:47:43 -07:00
Michelle Au
37d46a1e3f Volume scheduling library changes:
* FindPodVolumes
    * Prebound PVCs are treated like unbound immediate PVCs and will error
    * Always check for fully bound PVCs and cache bindings for not fully
      bound PVCs
* BindPodVolumes
    * Retry API updates for not fully bound PVCs even if the assume cache
      already marked it
    * Wait for PVCs to be fully bound after making the API updates
    * Error when detecting binding/provisioning failure conditions
2018-09-04 16:30:14 -07:00
Jan Safranek
7d673cb8f0 Pass new CSI API Client and informer to Volume Plugins 2018-08-31 12:25:59 -07:00
lichuqiang
4c43d626f2 related test update 2018-08-29 10:30:16 +08:00
lichuqiang
b4a57f6855 combine feature gate VolumeScheduling and DynamicProvisioningScheduling into one 2018-08-29 10:30:08 +08:00
Yecheng Fu
493502d0ae Double check PVC if not found in syncVolume.
If PV is bound by external PV binder (e.g. kube-scheduler), it's
possible on heavy load that corresponding PVC is not synced to
controller local cache yet.
2018-08-16 10:33:28 +08:00
Kubernetes Submit Queue
f4d8220df5
Merge pull request #65616 from cofyc/fix56163
Automatic merge from submit-queue (batch tested with PRs 65570, 65616). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Retry scheduling on StorageClass events

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56163

**Special notes for your reviewer**:

I have taken over #60006.
It's hard to test in e2e, because we cannot know reschedule of pod is triggered by which event (periodically service/node events will move pods to active queue too). ~~I'll add integration tests for this functionality after [this PR](https://github.com/kubernetes/kubernetes/pull/65296) get merged.~~ (already added)

**Release note**:

```release-note
NONE
```
2018-07-31 19:18:00 -07:00
Yecheng Fu
8f0373792f Retry scheduling on various events. 2018-07-20 09:54:34 +08:00
Fabio Bertinatto
a15cc29442 Add extra metrics for PV Controller
Specifically:

* Total provision time
* Total PV deletion time
* Number of times PV provisioning failed
* Number of times PV deletion failed
2018-07-19 15:36:37 +02:00
Fabio Bertinatto
97e63985dc Return error in provisionClaimOperation 2018-07-19 15:27:40 +02:00
Jonathan Pentecost
b4cab1be0e persistentvolume: fix spelling of storageClasseName
Fixed spelling of error in 'pv_controller', was 'storageClasseName', is
now 'storageClassName'.
2018-07-17 14:34:51 +01:00
Tim Allclair
b1012b2543
Remove unused io util writer & volume host GetWriter() 2018-07-09 14:09:48 -07:00
Kubernetes Submit Queue
8d73473ce8
Merge pull request #65774 from lichuqiang/add_reviewer
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add lichuqiang as reviewer of persistentvolume controller (for volume scheduling)

Now that I've been working on the storage topology-aware feature for quite a time. Really hope that I can help do some review.

```release-note
NONE
```

/assign @msau42
2018-07-03 11:31:12 -07:00
lichuqiang
67380c63aa Add lichuqiang as reviewer of persistentvolume controller (for volume scheduling) 2018-07-04 00:35:54 +08:00
Jeff Grafton
23ceebac22 Run hack/update-bazel.sh 2018-06-22 16:22:57 -07:00
Kubernetes Submit Queue
c615098a96
Merge pull request #63193 from lichuqiang/provision_0425
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Volume topology aware dynamic provisioning: work based on new API

**What this PR does / why we need it**:

The PR has been split to 3 parts:

Part1: https://github.com/kubernetes/kubernetes/pull/63232 for basic scheduler and PV controller plumbing
Part2: https://github.com/kubernetes/kubernetes/pull/63233 for API change

and the PR itself includes work based on the API change:

- Dynamic provisioning allowed topologies scheduler work
- Update provisioning interface to be aware of selected node and topology

**Which issue(s) this PR fixes** 
Feature: https://github.com/kubernetes/features/issues/561
Design: https://github.com/kubernetes/community/issues/2168

**Special notes for your reviewer**:
/sig storage
/sig scheduling
/assign @msau42 @jsafrane @saad-ali @bsalamat

@kubernetes/sig-storage-pr-reviews
@kubernetes/sig-scheduling-pr-reviews

**Release note**:

```release-note
Volume topology aware dynamic provisioning
```
2018-06-05 15:31:58 -07:00
lichuqiang
bccc8fe979 Provision interface change 2018-06-05 16:35:16 +08:00
lichuqiang
dffbd75f86 Dynamic provisioning allowed topologies scheduler work 2018-06-05 16:10:12 +08:00
Mike Danese
91feb345aa implement service account token projection 2018-06-04 17:22:08 -07:00
lichuqiang
446f36559e pv_controller change for provisioning 2018-05-24 17:12:38 +08:00
lichuqiang
95b530366a Add dynamic provisioning process 2018-05-24 17:12:38 +08:00
lichuqiang
91d403f384 cache update for dynamic provisioning 2018-05-24 17:12:12 +08:00
Michelle Au
2142737276 Only override objects from informer when version has increased.
Add more logging and tests to volume scheduler.
2018-05-17 16:46:20 -07:00
Michelle Au
6cf8a6606c Only count mounts that are from other pods 2018-04-19 15:40:51 -07:00
Kubernetes Submit Queue
b758341a04
Merge pull request #62646 from cofyc/reporteventtoapiserver
Automatic merge from submit-queue (batch tested with PRs 62694, 62569, 62646, 61633, 62433). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Report events to apiserver in local volume plugin.

**What this PR does / why we need it**:

See https://github.com/kubernetes/kubernetes/pull/62102#discussion_r179238429.

Report events to apiserver in local volume plugin.

- Add VolumeHost.GetEventRecorder() method
- Add related e2e tests

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #62248

**Special notes for your reviewer**:

Example output of `kubectl describe pods`:

```
Events:
  Type     Reason                 Age   From                Message
  ----     ------                 ----  ----                -------
  Normal   Scheduled              7s    default-scheduler   Successfully assigned e2e-tests-persistent-local-volumes-test-x4h5x/security-context-670da435-4174-11e8-9098-000c29bb0377 to 127.0.0.1
  Warning  AlreadyMountedVolume   7s    kubelet, 127.0.0.1  The requested fsGroup is 4321, but the volume local-pvfbb76 has GID 1234. The volume may not be shareable.
  Normal   SuccessfulMountVolume  7s    kubelet, 127.0.0.1  MountVolume.SetUp succeeded for volume "default-token-996xr"
  Normal   SuccessfulMountVolume  7s    kubelet, 127.0.0.1  MountVolume.SetUp succeeded for volume "local-pvfbb76"
  Normal   Pulled                 6s    kubelet, 127.0.0.1  Container image "k8s.gcr.io/busybox:1.24" already present on machine
  Normal   Created                6s    kubelet, 127.0.0.1  Created container
  Normal   Started                6s    kubelet, 127.0.0.1  Started container
```

**Release note**:

```release-note
NONE
```
2018-04-18 14:44:17 -07:00
Yecheng Fu
55ef18ad42 Report events to apiserver in local volume plugin.
- Add VolumeHost.GetEventRecorder() method
- Add related e2e tests
2018-04-18 11:19:36 +08:00
Kubernetes Submit Queue
8de8ca7bc8
Merge pull request #59947 from mkimuram/issue/59942
Automatic merge from submit-queue (batch tested with PRs 62448, 59317, 59947, 62418, 62352). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add binding error message for volumeMode:Block unsupported case

**What this PR does / why we need it**:
This PR adds a binding error message for volumeMode:Block
unsupported case.
With this message, users can understand why PV is not bound to
PVC on dynamic provisioning.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes: #59942

**Special notes for your reviewer**:
/cc @mtanino 
/sig storage

**Release note**:

```release-note
NONE
```
2018-04-17 16:31:21 -07:00
mkimuram
0ee71f0cc2 Add binding error message for volumeMode:Block unsupported case
This commit adds a binding error message for volumeMode:Block
unsupported case.
With this message, users can understand why PV is not bound to
PVC on both dynamic provisioning and manually creating volume.

This patch works as follows:

- In syncVolume, before adding the claim to claimQueue, check if
  there is a volumeMode mismatch, and if there is, record the
  event for both pv and pvc and skip adding to the queue.

fixes: #59942
2018-04-16 15:22:35 -04:00
Kubernetes Submit Queue
4d8e5d5d7e
Merge pull request #62483 from msau42/owners
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add msau42 to approvers for volume scheduling

**What this PR does / why we need it**:
Add me as an approver for the volume scheduling code

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-04-13 06:10:20 -07:00
Michelle Au
3b038bfebc Add msau42 to approvers for volume scheduling 2018-04-12 11:09:28 -07:00
WanLinghao
fd98d302c7 clean unused function
modified:   pkg/controller/volume/persistentvolume/scheduler_binder_test.go
2018-04-10 10:15:14 +08:00
Kubernetes Submit Queue
71f150422c
Merge pull request #62180 from msau42/binding-predicate
Automatic merge from submit-queue (batch tested with PRs 61918, 62180, 62198). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use provided node object in volume binding predicate

**What this PR does / why we need it**:
Autoscaler creates fake node objects, so we should use the provided node object instead of looking up the node from the informer.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #62178

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-04-06 11:56:07 -07:00
Michelle Au
8d1cd819ec Use provided node object in volume binding predicate 2018-04-05 14:35:55 -07:00
Kubernetes Submit Queue
b08ede9535
Merge pull request #61806 from hzxuzhonghu/LoadBalancerStatus-DeepCopy
Automatic merge from submit-queue (batch tested with PRs 61806, 61508, 62075, 62079, 62052). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

make use of auto generated LoadBalancerStatus.DeepCopy

Remove LoadBalancerStatusDeepCopy in helpers.go, and use the auto generated `LoadBalancerStatus.DeepCopy` in pkg/apis/core/zz_generated.deepcopy.go.

**Release note**:

```release-note
NONE
```
2018-04-03 17:24:21 -07:00
Mikhail Mazurskiy
468655b76a
Use typed events client directly 2018-04-01 18:57:29 +10:00
hzxuzhonghu
a5b121724d pv controller clean code 2018-03-27 20:00:45 +08:00
Shyam Jeedigunta
8ff1f05f7c Increase verbosity of frequently printed logline in scheduler_binder 2018-03-08 19:25:01 +01:00
wackxu
b3ba80b223 update bazel 2018-02-27 20:23:36 +08:00
wackxu
f737ad62ed update import 2018-02-27 20:23:35 +08:00
NickrenREN
dad0fa07b7 rename StorageProtection to StorageObjectInUseProtection 2018-02-21 10:48:56 +08:00
Kubernetes Submit Queue
6ba46963f8
Merge pull request #59391 from msau42/topology-beta
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move volume scheduling and local storage to beta

**What this PR does / why we need it**:
* Move the feature gates and APIs for volume scheduling and local storage to beta
* Update tests to use the beta fields
@kubernetes/sig-storage-pr-reviews 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #59390

**Special notes for your reviewer**:

**Release note**:

```release-note
ACTION REQUIRED: VolumeScheduling and LocalPersistentVolume features are beta and enabled by default.  The PersistentVolume NodeAffinity alpha annotation is deprecated and will be removed in a future release.
```
2018-02-20 13:26:07 -08:00
Michelle Au
c7884550b2 Add VolumeNodeAffinity to PersistentVolumeSpec 2018-02-16 17:54:10 -08:00
Jeff Grafton
ef56a8d6bb Autogenerated: hack/update-bazel.sh 2018-02-16 13:43:01 -08:00
Michelle Au
5271edd9e2 Index PVs by StorageClass in assume cache 2018-02-15 17:12:32 -08:00
mlmhl
6f83110e77 add number measurement for bound/unbound pv/pvc 2018-02-14 17:31:14 +08:00
Kubernetes Submit Queue
2368f70959
Merge pull request #59523 from gnufied/revert-57872-volume_metric_bound_pvc
Automatic merge from submit-queue (batch tested with PRs 57445, 59523). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Revert "add number measurement for bound/unbound pv/pvc"

Reverts kubernetes/kubernetes#57872

Fixes : https://github.com/kubernetes/kubernetes/issues/59517
2018-02-13 21:29:29 -08:00
Di Xu
48388fec7e fix all the typos across the project 2018-02-11 11:04:14 +08:00
Hemant Kumar
f94a81ef19
Revert "add number measurement for bound/unbound pv/pvc" 2018-02-07 21:31:01 -05:00