Commit Graph

11984 Commits

Author SHA1 Message Date
Jan Safranek
41adcc5496 Speed up binding of provisioned volumes
This fixes e2e test for provisioning - it expects that provisioned volumes
are bound quickly.

Majority of this patch is update of test framework needs to initialize the
controller appropriately.
2016-05-18 10:06:51 +02:00
Jan Safranek
c6f05c8056 provisioning: Add unit testso for provisioning errors. 2016-05-18 10:06:51 +02:00
Jan Safranek
c24b33793c unit test: Add possibility to inject kubeclient errors. 2016-05-18 10:06:51 +02:00
Jan Safranek
92dc159ab6 Delete provisioned volumes that are not needed.
We should delete volumes that are provisioned for a claim and the claim
gets bound to different volume during the provisioning.
2016-05-18 10:06:51 +02:00
Jan Safranek
9fb0f7a3fd provisioning: Unit tests 2016-05-18 10:06:51 +02:00
Jan Safranek
514d595881 provisioning: Implement provisioner 2016-05-18 10:06:51 +02:00
Jan Safranek
75b0e2ad63 provisioning: Refactor volume plugins.
NewPersistentVolumeTemplate() and Provision() are merged into one call.
2016-05-18 10:06:51 +02:00
Jan Safranek
dd7890c362 delete: Implement Deleter 2016-05-18 10:06:51 +02:00
Jan Safranek
22e68d4622 recycler: unit tests
- Add reclaim policy to newVolume() call.
- Implement reactor Volumes().Get().
- Implement mock volume plugin.
- Add recycler tests.
- Add a synchronization condition to controller.scheduleOperation
  - we need to pause the controller here, let the test to do some bad things
    to the controller and test error cases in recycleVolumeOperation.

Test framework gets more and more complicated... But this is the last piece,
I promise.
2016-05-18 10:06:24 +02:00
Jan Safranek
a08d826ca5 Make a separate functions to emit events and change status.
These two seem to be always used together.
2016-05-18 10:06:24 +02:00
Jan Safranek
1feb346830 recycler: implement recycler
Also update the old unit test to pass. New unit tests will be added in
subsequent commit.
2016-05-18 10:06:24 +02:00
Jan Safranek
56cae2dc20 unit test framework: Wait for all running operations to finish during all tests. 2016-05-18 10:06:24 +02:00
Jan Safranek
cf68370371 recycler: Maintain a list of long-running operations.
We need to keep list of running recyclers, deleters and provisioners in
memory in order not to start a new recycling/deleting/provisioning twice
for the same volume/claim.

This will be eventually replaced by GoRoutineMap from PR #24838.
2016-05-18 10:06:24 +02:00
Jan Safranek
4e47f69cba recycler: Implement volume host interfaces.
We need the controller to implement volume.VolumeHost interface to be able
to call recycle plugins.
2016-05-18 10:06:24 +02:00
Jan Safranek
a17f0d5949 Move release logic to standalone function. 2016-05-18 10:06:23 +02:00
Jan Safranek
7b73384fda Add controller method tests. 2016-05-18 10:05:14 +02:00
Jan Safranek
af295719f6 Add events. 2016-05-17 15:14:11 +02:00
Jan Safranek
61019b2401 Process deleted PVs
To speed up marking claims as "lost".
2016-05-17 15:14:10 +02:00
Jan Safranek
50b61ae168 Add "multi-sync" tests.
These test will call syncVolume/syncClaim until they reach consistent state.
2016-05-17 15:14:09 +02:00
Jan Safranek
f4f252e81c Implement syncVolume. 2016-05-17 15:14:08 +02:00
Jan Safranek
0be1512f1c Add ClaimLost phase. 2016-05-17 15:14:07 +02:00
Jan Safranek
5949b956f5 Implement syncClaim with bound claims. 2016-05-17 15:14:06 +02:00
Jan Safranek
eff6b50b93 Bind unbound claims in syncClaim. 2016-05-17 15:14:06 +02:00
Jan Safranek
e620bfc9cc Add unit test framework.
It's quite complicated one, see subsequent commits for usage.
2016-05-17 15:14:05 +02:00
Jan Safranek
a195802d3e Make standalone function to check for (pre-)bound volumes.
Note the semantic change, we now check for UID=""
2016-05-17 15:14:04 +02:00
Jan Safranek
20305f9235 Don't process events until fully initialized.
We do not want to process any volume / claim events until both PV and claim
caches are fully loaded.
2016-05-17 15:14:03 +02:00
Jan Safranek
71aa892a86 Implement volume controller skeleton.
This is a simple controller that watches changes of PersistentVolumes and
PersistentVolumeClaims.
2016-05-17 15:14:02 +02:00
Jan Safranek
b86e5923b2 Rename types.go to persistentvolume_index.go
With some changes:
- make some method private, nobody seems to use them.
- adapt to framework.NewIndexerInformer instead of using custom cache.
2016-05-17 15:14:01 +02:00
Jan Safranek
6fa527a460 Remove all three PersistentVolume controllers.
We will add new ones gradually in smaller chunks.
2016-05-10 17:57:54 +02:00
k8s-merge-robot
14b8f3350a Merge pull request #25008 from saad-ali/attachControllerSkeletonActual
Automatic merge from submit-queue

Introduce skeleton of new attach/detach controller

This PR introduces the skeleton of the new attach/detach controller for #20262
2016-05-10 04:04:29 -07:00
k8s-merge-robot
bbdbfc8940 Merge pull request #24787 from nikhiljindal/storageVersion
Automatic merge from submit-queue

Moving StorageFactory building logic to genericapiserver

Adding a DefaultStorageFactoryBuilder which builds the required StorageFactory.
This allows us to remove the duplicated code between `cmd/kube-apiserver` and `federation/cmd/federated-apiserver`

cc @deads2k @lavalamp @jianhuiz
2016-05-10 03:24:15 -07:00
k8s-merge-robot
5ca769e0a3 Merge pull request #25038 from mfojtik/json-line
Automatic merge from submit-queue

Display line number on JSON errors

Related issue: https://github.com/kubernetes/kubernetes/issues/12231

This PR will introduce line numbers for all JSON errors in the CLI:

(this is existing error reporting for YAML)
```console
$ kubectl create -f broken.yaml
yaml: line 8: mapping values are not allowed in this context
```

(this is error reporting proposed in this PR for JSON)
```console
$ kubectl create -f broken.json
json: line 35: invalid character '{' after object key
```

(and this is the current reporting:)
```console
$ kubectl create -f broken.json
invalid character '{' after object key
```
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()
2016-05-10 02:49:01 -07:00
nikhiljindal
2ffa3b4586 Moving StorageFactory building logic to genericapiserver 2016-05-10 00:57:11 -07:00
k8s-merge-robot
a57876b98c Merge pull request #25178 from zmerlynn/random_max_target_pools
Automatic merge from submit-queue

GCE: Allow node count to exceed GCE TargetPool maximums

```release-note
If the cluster node count exceeds the GCE TargetPool maximum (currently 1000),
randomly select which nodes are members of Kubernetes External Load Balancers.
```


[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

If we would exceeded the TargetPool API maximums, instead just
randomly select some subsection of the nodes to include in the TP
instead.
2016-05-09 23:18:15 -07:00
k8s-merge-robot
f9b8fd0c96 Merge pull request #25011 from zhouhaibing089/addclose
Automatic merge from submit-queue

followup to add http server close method

Fixes #25009, a follow up of https://github.com/kubernetes/kubernetes/pull/24595.
2016-05-09 22:32:02 -07:00
k8s-merge-robot
c4214f743f Merge pull request #24918 from Random-Liu/add-docker-operation-timeout
Automatic merge from submit-queue

Kubelet: Add docker operation timeout

For #23563.
Based on #24748, only the last 2 commits are new.

This PR:
1) Add timeout for all docker operations.
2) Add docker operation timeout metrics
3) Cleanup kubelet stats and add runtime operation error and timeout rate monitoring.
4) Monitor runtime operation error and timeout rate in kubelet perf.

@yujuhong 
/cc @gmarek Because of the metrics change.
/cc @kubernetes/sig-node
2016-05-09 21:51:52 -07:00
Zach Loafman
faf0c44429 GCE: Allow nodes to exceed target pool maximums
If we would exceeded the TargetPool API maximums, instead just
randomly select some subsection of the nodes to include in the TP
instead.
2016-05-09 21:44:58 -07:00
k8s-merge-robot
def7639457 Merge pull request #25245 from pmorie/kubelet/cadvisor
Automatic merge from submit-queue

Reduce kubelet LOC: extract cadvisor

Step 2 of #25028 

@yujuhong @kubernetes/sig-node
2016-05-09 21:09:42 -07:00
k8s-merge-robot
8a81000b71 Merge pull request #25331 from hongchaodeng/w
Automatic merge from submit-queue

etcd3/watcher: fix goroutine leak if ctx is canceled

### Problem
In reflector.go, it could probably call Stop() without retrieving all results
from ResultChan(). See [here](https://github.com/kubernetes/kubernetes/blob/master/pkg/client/cache/reflector.go#L369). A potential leak is that when an error has happened, it could block on resultChan,
and then cancelling context in Stop() wouldn't unblock it.

### What's this PR?
This fixes the problem by making it also select ctx.Done and cancel context afterwards if error happened.
2016-05-09 12:12:15 -07:00
saadali
214b4c28bc Skeleton of new attach detach controller 2016-05-09 11:34:11 -07:00
Hongchao Deng
97f4647dc3 etcd3/watcher: fix goroutine leak if ctx is canceled
In reflector.go, it could probably call Stop() without retrieving all results
from ResultChan().
A potential leak is that when an error has happened, it could block on resultChan,
and then cancelling context in Stop() wouldn't unblock it.
This fixes the problem by making it also select ctx.Done and cancel context
afterwards if error happened.
2016-05-09 09:06:11 -07:00
k8s-merge-robot
6596ae0c61 Merge pull request #24838 from saad-ali/attachControllerOpMap
Automatic merge from submit-queue

Add data structure for managing go routines by name

This PR introduces a data structure for managing go routines by name. It prevents the creation of new go routines if an existing go routine with the same name exists. This will enable parallelization of the designs in https://github.com/kubernetes/kubernetes/issues/20262 and https://github.com/kubernetes/kubernetes/issues/21931 with sufficient protection to prevent starting multiple operations on the same volume.
2016-05-09 06:31:22 -07:00
Wojciech Tyczynski
3c089bff41 Merge pull request #25344 from wojtek-t/fix_dynamic_client_with_protobufs
Fix dynamic client in protobuf configuration
2016-05-09 15:06:43 +02:00
k8s-merge-robot
545d56a63b Merge pull request #24810 from derekwaynecarr/sources_cleanup
Automatic merge from submit-queue

Clean-up sources ready tracking in kubelet

moved sources ready tracking behind an interface, made it thread-safe.
2016-05-09 05:48:09 -07:00
k8s-merge-robot
1a1229e206 Merge pull request #24795 from deads2k/use-all-attributes
Automatic merge from submit-queue

enable resource name and service account cases for impersonation

Adds the resource name check since that attribute was added for authorization.  Also adds a check against a separate resource for service accounts.  Allowing impersonation of service accounts to use a different resource check places control of impersonation with the same users to have the power to get the SA tokens directly.

@kubernetes/kube-iam 
@sgallagher FYI
2016-05-09 04:58:19 -07:00
Wojciech Tyczynski
ca5f18370a Fix dynamic client in protobuf configuration 2016-05-09 13:52:10 +02:00
k8s-merge-robot
2cf511b1f5 Merge pull request #24750 from derekwaynecarr/kubelet_eviction_flag_parsing
Automatic merge from submit-queue

Kubelet eviction flag parsers and tests

The first two commits are from https://github.com/kubernetes/kubernetes/pull/24559 that have achieved LGTM.  

The last commit is only part that is interesting, it adds the parsing logic to handle the flags, and reserves `pkg/kubelet/eviction` for eviction manager logic.
2016-05-09 04:15:04 -07:00
Wojciech Tyczynski
a7be41d4fe Merge pull request #25332 from wojtek-t/fix_protobuf_failures
Fix protobuf failures
2016-05-09 10:23:20 +02:00
Wojciech Tyczynski
1ea0d46bb3 Set headers in Watch() and Stream() requests 2016-05-09 09:33:13 +02:00
Wojciech Tyczynski
ac5d2cec44 Test if headers are set 2016-05-09 09:31:49 +02:00