github/kubernetes
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8f11cc78a8
kubernetes/pkg/controller
History
Kubernetes Submit Queue 8f11cc78a8 Merge pull request #38339 from gnufied/backoff-on-volume-delete 
Automatic merge from submit-queue (batch tested with PRs 38377, 36365, 36648, 37691, 38339)

Exponential back off when volume delete fails

**What this PR does / why we need it**:

This PR implements ability in pv_controller to back off when deleting a volume fails from plugin API. 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

Partly fixes #38295 , but I think volume delete is most problematic thing happening in pv_controller without any sort of backoff. 

After this change the attempts of volume deletion look like:

```
controller : I1208 00:18:35.532061   16388 aws_util.go:55] Error deleting EBS Disk volume aws://us-east-1d/vol-abcdefg: VolumeInUse: Volume vol-abcdefg is currently attached to i-1234567
controller : I1208 00:20:50.578325   16388 aws_util.go:55] Error deleting EBS Disk volume aws://us-east-1d/vol-abcdefg: VolumeInUse: Volume vol-abcdefg is currently attached to i-1234567
controller : I1208 00:23:05.563488   16388 aws_util.go:55] Error deleting EBS Disk volume aws://us-east-1d/vol-abcdefg: VolumeInUse: Volume vol-abcdefg is currently attached to i-1234567
controller : I1208 00:25:20.599158   16388 aws_util.go:55] Error deleting EBS Disk volume aws://us-east-1d/vol-abcdefg: VolumeInUse: Volume vol-abcdefg is currently attached to i-1234567
controller : I1208 00:27:35.560009   16388 aws_util.go:55] Error deleting EBS Disk volume aws://us-east-1d/vol-abcdefg: VolumeInUse: Volume vol-abcdefg is currently attached to i-1234567
controller : I1208 00:29:50.594967   16388 aws_util.go:55] Error deleting EBS Disk volume aws://us-east-1d/vol-abcdefg: VolumeInUse: Volume vol-abcdefg is currently attached to i-1234567
controller : I1208 00:32:05.539168   16388 aws_util.go:55] Error deleting EBS Disk volume aws://us-east-1d/vol-abcdefg: VolumeInUse: Volume vol-abcdefg is currently attached to i-1234567
controller : I1208 00:34:20.581665   16388 aws_util.go:55] Error deleting EBS Disk volume aws://us-east-1d/vol-abcdefg: VolumeInUse: Volume vol-abcdefg is currently attached to i-1234567
```
2016-12-08 10:52:03 -08:00
..
certificates Change 'controller.go' filenames to more meaningfull ones 2016-12-05 09:12:22 +01:00
cronjob Change 'controller.go' filenames to more meaningfull ones 2016-12-05 09:12:22 +01:00
daemon refactor: update bazel, codec, and gofmt 2016-12-03 19:10:53 -05:00
deployment Merge pull request #36648 from kargakis/follow-up-to-perma-failed 2016-12-08 10:51:59 -08:00
disruption refactor: update bazel, codec, and gofmt 2016-12-03 19:10:53 -05:00
endpoint refactor: update bazel, codec, and gofmt 2016-12-03 19:10:53 -05:00
garbagecollector Add verb support to gc and namespace controllers 2016-12-05 12:36:05 +01:00
informers demonstrate separation of controller intializers 2016-12-05 10:24:45 -05:00
job refactor: update bazel, codec, and gofmt 2016-12-03 19:10:53 -05:00
namespace Update generated files 2016-12-05 12:42:31 +01:00
node Make NodeController test utils usable from outside 2016-12-05 10:56:06 +01:00
petset refactor: update bazel, codec, and gofmt 2016-12-03 19:10:53 -05:00
podautoscaler refactor: update bazel, codec, and gofmt 2016-12-03 19:10:53 -05:00
podgc gcOrphaned check via the API that the node doesn’t exist 2016-12-06 12:17:38 +01:00
replicaset controller: requeue replica sets for availability checks 2016-12-05 02:41:15 +01:00
replication controller: requeue replica sets for availability checks 2016-12-05 02:41:15 +01:00
resourcequota generated: refactor 2016-11-23 22:30:47 -06:00
route refactor: update bazel, codec, and gofmt 2016-12-03 19:10:53 -05:00
service Merge pull request #31321 from anguslees/lb-nodes 2016-12-05 20:16:53 -08:00
serviceaccount Update autogenerated files 2016-12-06 12:25:57 +01:00
volume Merge pull request #38339 from gnufied/backoff-on-volume-delete 2016-12-08 10:52:03 -08:00
.import-restrictions add import restrictions 2016-10-13 16:10:24 -07:00
BUILD refactor: generated 2016-12-03 19:10:46 -05:00
client_builder.go use the client builder to support using SAs 2016-11-28 15:02:22 -05:00
controller_ref_manager.go generated: refactor 2016-11-23 22:30:47 -06:00
controller_utils_test.go refactor: update bazel, codec, and gofmt 2016-12-03 19:10:53 -05:00
controller_utils.go refactor: update bazel, codec, and gofmt 2016-12-03 19:10:53 -05:00
doc.go Use Go canonical import paths 2016-07-16 13:48:21 -04:00
lookup_cache.go Remove "All rights reserved" from all the headers. 2016-06-29 17:47:36 -07:00
OWNERS Remove myself from a bunch of OWNERS files, as I am too overloaded 2016-05-11 13:34:51 -07:00