Commit Graph

126 Commits

Author SHA1 Message Date
Manu Gupta
79a51090f9
fix: 81134: fix unsafe json for ReleaseControllerRevision (#104049)
* fix: 81134: fix unsafe json for ReleaseControllerRevision

1. Ensures that ReleaseControllerRevision returns a proper json by
marshalling an object into bytes. Otherwise, it returns an error.

2. Also, refactors the code to commonize the merge type
   GenerateDeleteOwnerRefStrategicMergeBytes that returns a byte and is
   used across ReleasePod, ReleaseControllerRevison
   ReleaseReplicaSet.

* Move GeneratePatchBytesForDelete to controller_ref_manager
2021-11-05 06:33:52 -07:00
Mike Dame
4960d0976a Wire contexts to Core controllers 2021-11-01 10:29:00 -04:00
Jiahui Feng
8f5771d243 use common controller interface in KCM. 2021-08-25 13:29:03 -07:00
Jordan Liggitt
603a0b016e Log cluster-scoped owners referencing namespaced owners, avoid retrying lookups forever
If a cluster-scoped dependent references a namespace-scoped owner,
this is an invalid relationship, and the lookup will never succeed in attemptToDelete.

Short-circuit requeueing in attemptToDelete and log.
2020-11-17 10:49:30 -05:00
Jordan Liggitt
78317edb8b Short-circuit attemptToDelete loop for virtual nodes that are removed or observed
Virtual nodes are added to the attemptToDelete queue, and continue getting requeued
until they are successfully verified absent or are observed via informer.

In the meantime, if the real object associated with that UID is observed via informer,
or is observed to be deleted via informer, the graph node for that UID can be removed
or marked as observed. In that case, we should stop retrying to get the virtual node coordinates.
2020-11-17 10:46:00 -05:00
Jordan Liggitt
30eb6683e6 Avoid marking virtual nodes as observed when they haven't been
Virtual nodes can be added to the GC graph in order to represent objects
which have not been observed via an informer, but are referenced via ownerReferences.

These virtual nodes are requeued into attemptToDelete until they are observed via an informer,
or successfully verified absent via a live lookup. Previously, both of those code paths
called markObserved() to stop requeuing into attemptToDelete.

Because it is useful to know whether a particular node has been observed via
a real informer event, this commit does the following:

* adds a `virtual bool` attribute to graph events so we know which ones came from a real informer
* limits the markObserved() call to the code path where a real informer event is observed
* uses an alternative mechanism to stop requeueing into attemptToDelete when a virtual node is verified absent via a live lookup
2020-11-17 10:42:48 -05:00
Jordan Liggitt
445f20dbdb Switch GC absentOwnerCache to full reference
Before deleting an object based on absent owners, GC verifies absence of those owners with a live lookup.

The coordinates used to perform that live lookup are the ones specified in the ownerReference of the child.

In order to performantly delete multiple children from the same parent (e.g. 1000 pods from a replicaset),
a 404 response to a lookup is cached in absentOwnerCache.

Previously, the cache was a simple uid set. However, since children can disagree on the coordinates
that should be used to look up a given uid, the cache should record the exact coordinates verified absent.
This is a [apiVersion, kind, namespace, name, uid] tuple.
2020-11-17 10:42:48 -05:00
Jordan Liggitt
09bdf76b8a Plumb event recorder to garbage collector controller 2020-11-17 10:42:45 -05:00
cici37
a91a2cdad6 Move informer_factory to staging 2020-10-29 12:20:33 -07:00
Lukasz Szaszkiewicz
50db32cf8c GC doesn't have to create monitors in the constructor 2020-06-10 14:59:45 +02:00
tahsinrahman
78318c7a26 Migrate to log calls to klog.InfoS and klog.ErroS for pkg/controller 2020-06-05 13:23:01 +08:00
Davanum Srinivas
442a69c3bd
switch over k/k to use klog v2
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
2020-05-16 07:54:27 -04:00
Jordan Liggitt
04a72d5ef9 client-go metadata: update callers 2020-03-06 11:07:54 -05:00
s-ito-ts
e6b9a7a309 Fix golint errors in pkg/controller/garbagecollector 2020-02-20 04:46:02 +00:00
hwdef
38256df718 pkg/controller: fix staticcheck warning 2019-11-07 16:46:47 +08:00
Yassine TIJANI
7e4c3096fe move WaitForCacheSync to the sharedInformer package
Signed-off-by: Yassine TIJANI <ytijani@vmware.com>
2019-08-22 16:13:41 +01:00
Clayton Coleman
637cc83341
Switch the garbage collector to use metadata client and protobuf 2019-07-11 12:18:51 -04:00
Jordan Liggitt
6a7e7fc156 Fix TestStressingCascadingDeletion flake 2019-06-04 17:26:36 -04:00
Yu Liao
05ebe91277 Switched to use dynamic shared informer for Garbage Collector. 2019-05-17 15:41:36 -07:00
yue9944882
0a60d19ebd fixes reversed owner relation in error msg 2019-04-02 12:08:53 +08:00
Walter Fender
f192657380 Add gauge metric for master of leader election.
Fixes #71730
0 indicates standby, 1 indicates master, label indicates which lease.
Tweaked name and documentation
Factored in Mike Danese feedback.
Removed dependency on prometheus from client-go using adapter.
Centralized adapter import.
Fixed godeps
Fixed boilerplate.
Put in fixes for caesarxuchao
2018-12-27 09:40:33 -08:00
Davanum Srinivas
954996e231
Move from glog to klog
- Move from the old github.com/golang/glog to k8s.io/klog
- klog as explicit InitFlags() so we add them as necessary
- we update the other repositories that we vendor that made a similar
change from glog to klog
  * github.com/kubernetes/repo-infra
  * k8s.io/gengo/
  * k8s.io/kube-openapi/
  * github.com/google/cadvisor
- Entirely remove all references to glog
- Fix some tests by explicit InitFlags in their init() methods

Change-Id: I92db545ff36fcec83afe98f550c9e630098b3135
2018-11-10 07:50:31 -05:00
Russ Cox
2bd91dda64 kubernetes: fix printf format errors
These are all flagged by Go 1.11's
more accurate printf checking in go vet,
which runs as part of go test.

Lubomir I. Ivanov <neolit123@gmail.com>
applied ammend for:
  pkg/cloudprovider/provivers/vsphere/nodemanager.go
2018-07-11 00:10:15 +03:00
Kubernetes Submit Queue
ccb9590a3b
Merge pull request #63386 from roycaihw/gc-json-patch
Automatic merge from submit-queue (batch tested with PRs 63386, 64624, 62297, 64847). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Let the garbage collector use json merge patch when SMP is not supported

**What this PR does / why we need it**:
Let garbage collector fallback to use json merge patch when strategic merge patch returns 415. This enables orphan delete on custom resources. 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56348

**Special notes for your reviewer**:
This PR is developed based on https://github.com/kubernetes/kubernetes/pull/56595. Ref https://github.com/kubernetes/kubernetes/pull/56606 for more information. 

**Release note**:

```release-note
Orphan delete is now supported for custom resources
```

/sig api-machinery
2018-06-06 19:56:20 -07:00
Jordan Liggitt
7da3d65571
Avoid deadlock in gc resync if available resources change during sync 2018-06-01 21:16:51 -04:00
Chao Xu
29d72a7134 GC fallback to jsonmerge patch when SMP is not supported 2018-06-01 16:50:07 -07:00
David Eads
fd044d152e fix dynamic client name 2018-05-11 13:12:09 -04:00
David Eads
cf4f7aab65 update garbage collection to use the new dynamic client 2018-05-07 09:01:39 -04:00
David Eads
3632037e60 add easy to use dynamic client 2018-04-25 08:55:26 -04:00
jennybuckley
455c6fb049 Prevent garbage collector from attempting to sync with 0 resources 2018-03-16 11:44:09 -07:00
jennybuckley
68e2a96016 Add unit test TestGarbageCollectorSync 2018-03-16 11:28:58 -07:00
Di Xu
48388fec7e fix all the typos across the project 2018-02-11 11:04:14 +08:00
Jordan Liggitt
df60789a7e
Requeue unobserved nodes in attemptToDelete 2017-12-21 14:00:07 -05:00
Kubernetes Submit Queue
f588bb02f3
Merge pull request #56150 from php-coder/fix_format_string
Automatic merge from submit-queue (batch tested with PRs 57211, 56150, 56368, 56271, 55957). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

pkg/controller/garbagecollector/garbagecollector.go: fix string format

**What this PR does / why we need it**:
This PR fixes broken formatting in the warning message by using appropriate function:
> W1121 13:13:39.359283   19160 garbagecollector.go:601] failed to discover preferred resources: %vGet https://127.0.0.1:37983/api: dial tcp 127.0.0.1:37983: getsockopt: connection refused

**Special notes for your reviewer**:
This change was introduced in #55259

**Release note**:
```release-note
NONE
```

PTAL @ironcladlou 
CC @simo5
2017-12-15 14:00:41 -08:00
Jordan Liggitt
34c3a254d8
Process cluster-scoped owners correctly 2017-12-14 22:53:59 -05:00
Dan Mace
a62d07ce2a Add a GC deadlock note 2017-11-27 16:50:29 -05:00
Dan Mace
eeeabce831 Add more GC sync logging 2017-11-27 16:47:37 -05:00
Dan Mace
9b2886df29 Ensure sync failures are correctly retried
Only track the last synced resources when all preceding steps have
completed to ensure that failures will be correctly retried.
2017-11-27 16:29:18 -05:00
Dan Mace
d89f58fcc8 Fix GC sync race condition
Remove faulty diff detection logic from GC sync which leads to a race
condition: If the GC's discovery client is returning a fully up to date
view of server resources during the very first GC sync, the sync
function will never sync monitors or reset the REST mapper unless
discovery changes again. This causes REST mapping to fail for any custom
types already present in discovery.
2017-11-27 16:11:47 -05:00
David Eads
19578eb1a1 log errors while trying to GC resources 2017-11-22 10:40:03 -05:00
Slava Semushin
2b86881ab3 pkg/controller/garbagecollector/garbagecollector.go: fix string format. 2017-11-21 15:24:34 +01:00
Jordan Liggitt
dbfc43f723
Only attempt to construct GC informers for watchable resources 2017-11-15 02:26:58 -05:00
Dan Mace
c3dd82c30c Tolerate partial discovery in garbage collector
Allow the garbage collector to tolerate partial discovery failures. On a
partial failure, use whatever was discovered, log the failures, and
allow the resync logic to try again later.

Fixes #55022.
2017-11-07 16:54:49 -05:00
Jordan Liggitt
a0f8a36e48
Remove GC rate limiter metrics 2017-09-21 19:23:27 -04:00
David Eads
253b047d89 update GC controller to wait until controllers have been initialized once 2017-08-31 09:01:38 -04:00
Dan Mace
3d6d57a18f Improve GC discovery sync performance
Improve GC discovery sync performance by only syncing when discovered
resource diffs are detected. Before, the GC worker pool was shut down
and monitors resynced unconditionally every sync period, leading to
significant processing delays causing test flakes where otherwise
reasonable GC timeouts were being exceeded.

Related to https://github.com/kubernetes/kubernetes/issues/49966.
2017-08-09 09:16:05 -04:00
Chao Xu
3060e925f5 GC shouldn't send empty patch 2017-08-01 13:06:36 -07:00
Dan Mace
d08dfb92c7 Enable garbage collection of custom resources
Enhance the garbage collector to periodically refresh the resources it
monitors (via discovery) to enable custom resource definition GC.

This implementation caches Unstructured structs for any kinds not
covered by a shared informer. The existing meta-only codec only supports
compiled types; an improved codec which supports arbitrary types could
be introduced to optimize caching to store only metadata for all
non-informer types.
2017-07-28 10:00:10 -04:00
Kubernetes Submit Queue
89525c62fd Merge pull request #49516 from caesarxuchao/parallel-patch
Automatic merge from submit-queue (batch tested with PRs 48360, 48469, 49576, 49516, 49558)

Let garbage collector send orphaning patches in parallel

Garbage collector sends orphaning patches in parallel now.
2017-07-26 13:02:43 -07:00
deads2k
151d39682e add reflector metrics 2017-07-25 09:01:37 -04:00