Commit Graph

133 Commits

Author SHA1 Message Date
Jordan Liggitt
a0f8a36e48
Remove GC rate limiter metrics 2017-09-21 19:23:27 -04:00
David Eads
253b047d89 update GC controller to wait until controllers have been initialized once 2017-08-31 09:01:38 -04:00
Dan Mace
3d6d57a18f Improve GC discovery sync performance
Improve GC discovery sync performance by only syncing when discovered
resource diffs are detected. Before, the GC worker pool was shut down
and monitors resynced unconditionally every sync period, leading to
significant processing delays causing test flakes where otherwise
reasonable GC timeouts were being exceeded.

Related to https://github.com/kubernetes/kubernetes/issues/49966.
2017-08-09 09:16:05 -04:00
Chao Xu
3060e925f5 GC shouldn't send empty patch 2017-08-01 13:06:36 -07:00
Dan Mace
d08dfb92c7 Enable garbage collection of custom resources
Enhance the garbage collector to periodically refresh the resources it
monitors (via discovery) to enable custom resource definition GC.

This implementation caches Unstructured structs for any kinds not
covered by a shared informer. The existing meta-only codec only supports
compiled types; an improved codec which supports arbitrary types could
be introduced to optimize caching to store only metadata for all
non-informer types.
2017-07-28 10:00:10 -04:00
Kubernetes Submit Queue
89525c62fd Merge pull request #49516 from caesarxuchao/parallel-patch
Automatic merge from submit-queue (batch tested with PRs 48360, 48469, 49576, 49516, 49558)

Let garbage collector send orphaning patches in parallel

Garbage collector sends orphaning patches in parallel now.
2017-07-26 13:02:43 -07:00
deads2k
151d39682e add reflector metrics 2017-07-25 09:01:37 -04:00
Chao Xu
5e7d7c00a2 let garbage collector send orphaning patches in parallel 2017-07-24 13:56:39 -07:00
Jacob Simpson
29c1b81d4c Scripted migration from clientset_generated to client-go. 2017-07-17 15:05:37 -07:00
FengyunPan
154ab548f9 Cleanup useless metrics.go for garbagecollector
The metrics of garbagecollector are already published with the
workqueue metrics, so metrics.go is useless, let's cleanup it.
See https://github.com/kubernetes/client-go/blob/master/util/workqueue/metrics.go.
2017-07-06 09:51:39 +08:00
Chao Xu
229ae59e73 garbage collector controller propagates DeletePropagationForeground
policy if the object doesn't already have finalizers.
2017-06-26 09:53:03 -07:00
Kubernetes Submit Queue
3b05e19076 Merge pull request #47000 from caesarxuchao/fix-gc-orphan
Automatic merge from submit-queue (batch tested with PRs 47000, 47188, 47094, 47323, 47124)

GC should retry on patch error

Fixing https://github.com/kubernetes/kubernetes/issues/46998.

This is fixing a bug, so applying the 1.7 milestone.
2017-06-12 18:19:40 -07:00
Dr. Stefan Schimanski
30668b24fb client-go: GetOptions for dynamic client 2017-06-09 18:04:17 +02:00
Chao Xu
a0a2e95fc5 GC should retry on patch error 2017-06-05 18:28:46 -07:00
Andy Goldstein
d1a0384678 GC: allow ignored resources to be customized
Allow the list of resources the garbage collector controller should
ignore to be customizable, so downstream integrators can add their own
resources to the list, if necessary.
2017-05-23 12:05:09 -04:00
Andy Goldstein
2480f2ceb6 Use shared informers in gc controller if possible 2017-05-22 12:51:37 -04:00
Andy Goldstein
e63fcf708d Make controller Run methods consistent
- startup/shutdown logging
- wait for cache sync logging
- defer utilruntime.HandleCrash()
- wait for stop channel before exiting
2017-04-14 07:27:45 -04:00
Chao Xu
0605ba7a6d wait for garbagecollector to be synced in test 2017-03-14 16:19:33 -07:00
Chao Xu
d7aef0a338 Let GC print specific message for RESTMapping failure 2017-03-10 11:38:57 -08:00
Chao Xu
c3baf402f5 gc changes 2017-02-28 23:05:41 -08:00
Jordan Liggitt
e6dbe5f57e
Pass typed options to dynamic client 2017-02-22 08:53:57 -05:00
Wojciech Tyczynski
a21b08d00f Revert "Use watch param instead of deprecated /watch/ prefix" 2017-02-21 08:37:51 +01:00
Jordan Liggitt
308fdcd13f
Pass typed options to dynamic client 2017-02-19 22:12:55 -05:00
Jordan Liggitt
09b5d7279f
install authentication.k8s.io/v1, add tests 2017-02-07 01:44:56 -05:00
Jordan Liggitt
8706bc9617
install authorization.k8s.io/v1, add tests
Use specific v1beta1 Authorization client

Add client expansions
2017-02-06 18:16:51 -05:00
Dr. Stefan Schimanski
ef8bb4e7e7 Move pkg/api.FinanlizerOrphan into metav1 2017-02-02 15:17:34 +01:00
deads2k
2c1c0f3f72 move workqueue to client-go 2017-01-30 09:08:21 -05:00
deads2k
9488e2ba30 move testing/core to client-go 2017-01-26 13:54:40 -05:00
deads2k
2734f8f892 move dynamic and discovery clients 2017-01-26 08:37:06 -05:00
deads2k
b0b156b381 make tools/cache authoritative 2017-01-25 08:29:45 -05:00
Clayton Coleman
be6d2933df
refactor: Move *Options references to metav1 2017-01-24 13:41:51 -05:00
deads2k
5a8f075197 move authoritative client-go utils out of pkg 2017-01-24 08:59:18 -05:00
Clayton Coleman
469df12038
refactor: move ListOptions references to metav1 2017-01-23 17:52:46 -05:00
deads2k
c47717134b move utils used in restclient to client-go 2017-01-19 07:55:14 -05:00
Clayton Coleman
9a2a50cda7
refactor: use metav1.ObjectMeta in other types 2017-01-17 16:17:19 -05:00
Clayton Coleman
36acd90aba
Move APIs and core code to use metav1.ObjectMeta 2017-01-17 16:17:18 -05:00
deads2k
26c46971f2 move PatchType to apimachinery 2017-01-17 08:32:05 -05:00
deads2k
77b4d55982 mechanical 2017-01-16 09:35:12 -05:00
Klaus Ma
25fe1e0d82 Made cache.Controller to be interface. 2017-01-13 13:33:23 +08:00
deads2k
6a4d5cd7cc start the apimachinery repo 2017-01-11 09:09:48 -05:00
zhouhaibing089
b25873a4bc warning instead error when search kinds for resources 2017-01-08 09:47:43 +08:00
Clayton Coleman
42d410fdde
Switch to use pkg/apis/meta/v1/unstructured and the new interfaces
Avoid directly accessing an unstructured type if it is not required.
2016-12-10 18:05:28 -05:00
Clayton Coleman
c30862a488
Move OwnerReference to pkg/apis/meta/v1 and remove metatypes pkg
OwnerReference is common.
2016-12-10 18:05:28 -05:00
Dr. Stefan Schimanski
24e24fc7bb Add verb support to gc and namespace controllers 2016-12-05 12:36:05 +01:00
Dr. Stefan Schimanski
458d2b2fe4 Add verb support for discovery client 2016-12-05 12:36:05 +01:00
Clayton Coleman
3454a8d52c
refactor: update bazel, codec, and gofmt 2016-12-03 19:10:53 -05:00
Clayton Coleman
5df8cc39c9
refactor: generated 2016-12-03 19:10:46 -05:00
Clayton Coleman
35a6bfbcee
generated: refactor 2016-11-23 22:30:47 -06:00
Chao Xu
7eeb71f698 cmd/kube-controller-manager 2016-11-23 15:53:09 -08:00
Chao Xu
7249c9bd8a fix TestCreateWithNonExistentOwner
remove the use of gc.QueuesDrained
2016-09-26 16:51:56 -07:00
Clayton Coleman
97c35fcc67
Allow garbage collection to work against different API prefixes
The GC needs to build clients based only on Resource or Kind. Hoist the
restmapper out of the controller and the clientpool, support a new
ClientForGroupVersionKind and ClientForGroupVersionResource, and use the
appropriate one in both places.
2016-09-22 15:00:58 -04:00
Wojciech Tyczynski
27d75054b3 Avoid unnecessary API calls from GC 2016-09-18 07:09:11 +02:00
Kubernetes Submit Queue
0d9685b0b5 Merge pull request #32805 from caesarxuchao/more-gc-optimization
Automatic merge from submit-queue

Add the uid in a delete event to the absentOwnerCache

This is a small optimization to further reduce the traffic sent by the GC.

In #31167, GC caches the non-existent owners when it processes the dirtyQueue. As discovered in #32571, there is still small inefficiency, because there are multiple goroutines processing the dirtyQueue, many of them might send a GET to the apiserver before the cache gets populated.

This PR populates the cache when GC observes an object gets deleted, which happens before the processing of the dirtyQueue, so it avoids the simultaneous GET sent by the GC workers.

cc @lavalamp
2016-09-16 00:40:24 -07:00
Mike Danese
a765d59932 move informer and controller to pkg/client/cache
Signed-off-by: Mike Danese <mikedanese@google.com>
2016-09-15 12:50:08 -07:00
Chao Xu
21896dac4b add the uid in a delete event to the absentOwnerCache 2016-09-15 11:22:22 -07:00
deads2k
8fac64b43f add localSAR 2016-09-13 08:54:23 -04:00
Kubernetes Submit Queue
7a4d81ea43 Merge pull request #31271 from deads2k/self-sar
Automatic merge from submit-queue

add selfsubjectaccessreview API

Exposes the REST API for self subject access reviews.  This allows a user to see whether or not they can perform a particular action.

@kubernetes/sig-auth
2016-09-08 01:29:48 -07:00
Chao Xu
25966e1148 gc retries failed garbage collection 2016-09-02 14:17:42 -07:00
deads2k
97529e3687 add selfsubjectaccessreview API 2016-08-29 09:51:54 -04:00
Chao Xu
9ac91e5172 debugging gc 2016-08-26 15:58:33 -07:00
Chao Xu
f2d0f1e3f0 add absent owner cache 2016-08-23 17:21:15 -07:00
Chao Xu
10a4e912a8 gc never resync 2016-08-22 11:32:37 -07:00
Chao Xu
992afd9c45 let rate_limiter_helper use sync.Once 2016-08-19 20:32:28 -07:00
Chao Xu
c790773018 temporarlily stop register RateLimiter metrics in the garbage collector 2016-08-19 17:31:32 -07:00
Chao Xu
65d1dbe8d9 fix memory leak in gc 2016-08-18 21:54:44 -07:00
Kubernetes Submit Queue
035ec518af Merge pull request #28387 from caesarxuchao/gc-latency-measure
Automatic merge from submit-queue

[GarbageCollector] measure latency

First commit is #27600.

In e2e tests, I measure the average time an item spend in the eventQueue(~1.5 ms), dirtyQueue(~13ms), and orphanQueue(~37ms). There is no stress test in e2e yet, so the number may not be useful.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/28387)
<!-- Reviewable:end -->
2016-08-11 02:33:55 -07:00
Chao Xu
41572cb22d add metrics to gc 2016-08-10 12:45:06 -07:00
Kubernetes Submit Queue
01d6ac64b4 Merge pull request #29898 from matttproud/refactor/simplify/garbagecollector
Automatic merge from submit-queue

pkg/controller/garbagecollector: simplify mutexes.

pkg/controller/garbagecollector: simplified synchronization and made idiomatic.


Similar to #29598, we can rely on the zero-value construction behavior
to embed `sync.Mutex` into parent structs.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29898)
<!-- Reviewable:end -->
2016-08-09 21:39:26 -07:00
Chao Xu
4d2350632c only store typeMeta and objectMeta in the gc store 2016-08-08 17:23:13 -07:00
deads2k
32920b5617 add subjectaccessreviews resource 2016-08-05 11:20:56 -04:00
deads2k
60dd4a5d26 interesting changes to add tokenreviews endpoint to implement webhook 2016-08-03 08:37:45 -04:00
Matt T. Proud
1ca99119fd pkg/controller/garbagecollector: simplify mutexes.
Similar to #29598, we can rely on the zero-value construction behavior
to embed `sync.Mutex` into parent structs.
2016-08-02 08:23:27 +02:00
k8s-merge-robot
a4442250f3 Merge pull request #29550 from caesarxuchao/fix-28713
Automatic merge from submit-queue

[Garbage Collector] GC waits for all reflectors to be synced before starting workers

Fix 28713. See https://github.com/kubernetes/kubernetes/issues/28713#issuecomment-234826276 for details on the cause of the flake. The test has passed 100 times locally.
2016-08-01 18:41:12 -07:00
Chao Xu
d077db366a GC waits for all reflectors to be synced before starts workers 2016-07-25 14:03:30 -07:00
lixiaobing10051267
57eabf2b7e Garbagecollector.go 2016-07-21 22:31:08 +08:00
Wojciech Tyczynski
f71244975d Revert "[garbage collector] add e2e test" 2016-07-15 10:10:02 +02:00
Chao Xu
7a5b3c43a0 plumb --enable_garbage_collector from environment variable;
adding a simple e2e test
2016-07-08 10:27:16 -07:00
David McMahon
ef0c9f0c5b Remove "All rights reserved" from all the headers. 2016-06-29 17:47:36 -07:00
Chao Xu
d9f07925be let dynamic client handle non-registered ListOptions;
register ListOptions for apis/policy
2016-06-22 13:18:50 -07:00
Chao Xu
c15c10f312 fix a log line 2016-06-10 09:58:27 -07:00
Chao Xu
06f49f7ca7 Let the dynamic client take a customized parameter codec for List, Watch, and DeleteCollection.
Let the gc's ListWatcher use api.ParameterCodec. Fixes 25890.
2016-06-03 11:22:51 -07:00
Chao Xu
1665546d2d add finalizer logics to the API server and the garbage collector; handling DeleteOptions.OrphanDependents in the API server 2016-05-24 13:07:28 -07:00
Chao Xu
c73406bcfe the garbage collector controller 2016-05-15 16:04:19 -07:00