Commit Graph

91 Commits

Author SHA1 Message Date
ialidzhikov
aede3fbf40 pkg/controller: Replace deprecated func usage from the k8s.io/utils/pointer pkg 2022-11-23 17:40:23 +02:00
Kubernetes Prow Robot
0b79a791ed
Merge pull request #107180 from ardaguclu/remove-deprecated-serverresource
Remove deprecated discovery/ServerResources function
2022-03-25 23:45:21 -07:00
jlsong01
d66b3edd65 allocate a unique scheme for each test to fix concurrent usage issue 2022-02-02 15:22:59 +08:00
astraw99
a5a54754d5 refactor garbagecollection controllers to make forget impossible to forget 2022-01-09 15:19:10 +08:00
Arda Güçlü
ef39a89142 Remove deprecated discovery/ServerResources function
ServerResources function was deprecated and instead ServerGroupsAndResources
function is suggested.

This PR removes ServerResources function and move every place to use ServerGroupsAndResources.
2021-12-22 11:14:09 +03:00
sunzhaochang
f0e74bfb7f Fix unexpected end of TestDependentsRace 2021-12-10 20:07:37 +08:00
Mikhail Mazurskiy
de4598d0db
ResettableRESTMapper to make it possible to reset wrapped mappers 2021-11-06 10:44:02 +11:00
Manu Gupta
79a51090f9
fix: 81134: fix unsafe json for ReleaseControllerRevision (#104049)
* fix: 81134: fix unsafe json for ReleaseControllerRevision

1. Ensures that ReleaseControllerRevision returns a proper json by
marshalling an object into bytes. Otherwise, it returns an error.

2. Also, refactors the code to commonize the merge type
   GenerateDeleteOwnerRefStrategicMergeBytes that returns a byte and is
   used across ReleasePod, ReleaseControllerRevison
   ReleaseReplicaSet.

* Move GeneratePatchBytesForDelete to controller_ref_manager
2021-11-05 06:33:52 -07:00
Mike Dame
4960d0976a Wire contexts to Core controllers 2021-11-01 10:29:00 -04:00
Alexander Constantinescu
5b740f430e [queue] Implement ShutDownWithDrain allowing the queue to drain when shutting down
Signed-off-by: Alexander Constantinescu <aconstan@redhat.com>
2021-08-20 13:59:12 +02:00
Jordan Liggitt
1e71a28490 Clean up namespaced children of missing virtual parents with incorrectly cluster-scoped nodes 2021-01-14 11:29:31 -05:00
Jordan Liggitt
65d43faf3a Add unit test for child scope mismatch with missing parent 2021-01-14 10:45:56 -05:00
drfish
4a839fa6d3 Fix typos in garbagecollector_test.go 2020-12-29 23:20:18 +08:00
Jordan Liggitt
e491c3bc70 Add GC unit tests
Adds unit tests covering the problematic scenarios identified
around conflicting data in child owner references

                      Before   After
package level         51%      68%
garbagecollector.go   60%      75%
graph_builder.go      50%      81%
graph.go              50%      68%

Added/improved coverage of key functions that had lacking unit test coverage:

* attemptToDeleteWorker
* attemptToDeleteItem
* processGraphChanges (added coverage of all added code)
2020-11-17 10:49:32 -05:00
Jordan Liggitt
30eb6683e6 Avoid marking virtual nodes as observed when they haven't been
Virtual nodes can be added to the GC graph in order to represent objects
which have not been observed via an informer, but are referenced via ownerReferences.

These virtual nodes are requeued into attemptToDelete until they are observed via an informer,
or successfully verified absent via a live lookup. Previously, both of those code paths
called markObserved() to stop requeuing into attemptToDelete.

Because it is useful to know whether a particular node has been observed via
a real informer event, this commit does the following:

* adds a `virtual bool` attribute to graph events so we know which ones came from a real informer
* limits the markObserved() call to the code path where a real informer event is observed
* uses an alternative mechanism to stop requeueing into attemptToDelete when a virtual node is verified absent via a live lookup
2020-11-17 10:42:48 -05:00
Jordan Liggitt
445f20dbdb Switch GC absentOwnerCache to full reference
Before deleting an object based on absent owners, GC verifies absence of those owners with a live lookup.

The coordinates used to perform that live lookup are the ones specified in the ownerReference of the child.

In order to performantly delete multiple children from the same parent (e.g. 1000 pods from a replicaset),
a 404 response to a lookup is cached in absentOwnerCache.

Previously, the cache was a simple uid set. However, since children can disagree on the coordinates
that should be used to look up a given uid, the cache should record the exact coordinates verified absent.
This is a [apiVersion, kind, namespace, name, uid] tuple.
2020-11-17 10:42:48 -05:00
Jordan Liggitt
09bdf76b8a Plumb event recorder to garbage collector controller 2020-11-17 10:42:45 -05:00
cici37
a91a2cdad6 Move informer_factory to staging 2020-10-29 12:20:33 -07:00
wangkai1994
b3b95f5ab5 add defer 2020-07-07 10:20:19 +08:00
Lukasz Szaszkiewicz
50db32cf8c GC doesn't have to create monitors in the constructor 2020-06-10 14:59:45 +02:00
s-ito-ts
e6b9a7a309 Fix golint errors in pkg/controller/garbagecollector 2020-02-20 04:46:02 +00:00
Ted Yu
0d704f1ce2 Traverse OwnerReference maps more efficiently 2019-10-19 17:56:11 -07:00
Yassine TIJANI
7e4c3096fe move WaitForCacheSync to the sharedInformer package
Signed-off-by: Yassine TIJANI <ytijani@vmware.com>
2019-08-22 16:13:41 +01:00
David Xia
fabfd950b1
cleanup: fix some log and error capitalizations
Part of https://github.com/kubernetes/kubernetes/issues/15863
2019-07-20 18:26:16 -04:00
Clayton Coleman
637cc83341
Switch the garbage collector to use metadata client and protobuf 2019-07-11 12:18:51 -04:00
Yu Liao
05ebe91277 Switched to use dynamic shared informer for Garbage Collector. 2019-05-17 15:41:36 -07:00
Jordan Liggitt
8146fe47d8 Clean up test output 2019-03-26 23:15:39 -04:00
Dr. Stefan Schimanski
618050e35d client-go: extend discovery intf with ServerGroupsAndResources 2019-02-12 14:28:19 +01:00
Chao Xu
e2812ce312 Slightly modify the comment in the test 2019-01-24 10:42:30 -08:00
Wei Guo
44b2e8464b change the period of GarbageCollector.Sync to 200ms for TestGarbageCollectorSync 2019-01-10 17:51:53 +08:00
Kubernetes Submit Queue
ccb9590a3b
Merge pull request #63386 from roycaihw/gc-json-patch
Automatic merge from submit-queue (batch tested with PRs 63386, 64624, 62297, 64847). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Let the garbage collector use json merge patch when SMP is not supported

**What this PR does / why we need it**:
Let garbage collector fallback to use json merge patch when strategic merge patch returns 415. This enables orphan delete on custom resources. 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56348

**Special notes for your reviewer**:
This PR is developed based on https://github.com/kubernetes/kubernetes/pull/56595. Ref https://github.com/kubernetes/kubernetes/pull/56606 for more information. 

**Release note**:

```release-note
Orphan delete is now supported for custom resources
```

/sig api-machinery
2018-06-06 19:56:20 -07:00
Jordan Liggitt
7da3d65571
Avoid deadlock in gc resync if available resources change during sync 2018-06-01 21:16:51 -04:00
Chao Xu
29d72a7134 GC fallback to jsonmerge patch when SMP is not supported 2018-06-01 16:50:07 -07:00
David Eads
c5445d3c56 simplify api registration 2018-05-08 18:33:50 -04:00
David Eads
cf4f7aab65 update garbage collection to use the new dynamic client 2018-05-07 09:01:39 -04:00
David Eads
ef0d1ab819 remove incorrect static restmapper 2018-05-01 07:51:17 -04:00
David Eads
e2fc5cf259 remove versioning interface 2018-04-27 07:56:42 -04:00
David Eads
3632037e60 add easy to use dynamic client 2018-04-25 08:55:26 -04:00
jennybuckley
68e2a96016 Add unit test TestGarbageCollectorSync 2018-03-16 11:28:58 -07:00
Jordan Liggitt
dbfc43f723
Only attempt to construct GC informers for watchable resources 2017-11-15 02:26:58 -05:00
Dr. Stefan Schimanski
012b085ac8 pkg/apis/core: mechanical import fixes in dependencies 2017-11-09 12:14:08 +01:00
Dan Mace
c3dd82c30c Tolerate partial discovery in garbage collector
Allow the garbage collector to tolerate partial discovery failures. On a
partial failure, use whatever was discovered, log the failures, and
allow the resync logic to try again later.

Fixes #55022.
2017-11-07 16:54:49 -05:00
Dr. Stefan Schimanski
7773a30f67 pkg/api/legacyscheme: fixup imports 2017-10-18 17:23:55 +02:00
Clayton Coleman
8b571bb63b
Disable default paging in list watches
For 1.8 this will be off by default. In 1.9 it will be on by default.
Add tests and rename some fields to use the `chunking` terminology.

Note that the pager may be used for other things besides chunking.
2017-09-06 23:10:59 -04:00
Clayton Coleman
8952a0cb72
Server side implementation of paging for etcd3
Add a feature gate in the apiserver to control whether paging can be
used. Add controls to the storage factory that allow it to be disabled
per resource. Use a JSON encoded continuation token that can be
versioned. Create a 410 error if the continuation token is expired.

Adds GetContinue() to ListMeta.
2017-09-01 10:47:01 -04:00
David Eads
253b047d89 update GC controller to wait until controllers have been initialized once 2017-08-31 09:01:38 -04:00
Dan Mace
3d6d57a18f Improve GC discovery sync performance
Improve GC discovery sync performance by only syncing when discovered
resource diffs are detected. Before, the GC worker pool was shut down
and monitors resynced unconditionally every sync period, leading to
significant processing delays causing test flakes where otherwise
reasonable GC timeouts were being exceeded.

Related to https://github.com/kubernetes/kubernetes/issues/49966.
2017-08-09 09:16:05 -04:00
Dan Mace
d08dfb92c7 Enable garbage collection of custom resources
Enhance the garbage collector to periodically refresh the resources it
monitors (via discovery) to enable custom resource definition GC.

This implementation caches Unstructured structs for any kinds not
covered by a shared informer. The existing meta-only codec only supports
compiled types; an improved codec which supports arbitrary types could
be introduced to optimize caching to store only metadata for all
non-informer types.
2017-07-28 10:00:10 -04:00
Jacob Simpson
29c1b81d4c Scripted migration from clientset_generated to client-go. 2017-07-17 15:05:37 -07:00
FengyunPan
468b97a746 Update comment for garbagecollector 2017-06-29 21:42:47 +08:00