Automatic merge from submit-queue
Handle secret namespace and data properly in federated secret controller
This PR fixes what was missed in #30669 review. It uses both namespace and secret name for keying and propagates secret data to underlying clusters.
cc: @quinton-hoole @kshafiee @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
federation: updating cluster name validation to allow DNS labels only
Forked from https://github.com/kubernetes/kubernetes/pull/28921
Summarizing the discussion from that PR:
* Right now, we allow name of the Cluster resource in federation/v1beta1 group version to be a subdomain (group of DNS labels separated by dots). This prevents us from using the cluster name in our dns search paths, since there is a restriction of 6 domain labels there.
* Restricting cluster name to DNS label will give us the flexibility to be able to do that in the future, if we want to. Though we do not have a concrete use case right now, the possibility is attractive.
* There is not a strong argument in favor of allowing subdomains as cluster names right now. If in future, there is one then we can get more permissive but its better to start with a stricter model.
Note that we are breaking a beta API, but it should be fine since we do not expect anyone to be using subdomain as cluster name. Have added release-note-action-required label
```release-note
Action required: federation-only: Please update your cluster name to be a valid DNS label.
Updating federation.v1beta1.Cluster API to disallow subdomains as valid cluster names. Only DNS labels are allowed as valid cluster names now.
```
Automatic merge from submit-queue
Support for preexisting replicas and estimated capacity in federated replicaset controller
With this PR the planer will be able to:
* Keep already existing replicas in their current clusters if rebalance = false and min/max boundaries are met.
* Limit the number of replicas in a cluster to the level that was measured by the count of running and unschedulable pods. And provide an estimate how much more pods would be nice to put in a cluster so that if they are scheduled we will be closer to the desired layout or to schedule the desired number of replicas at all.
cc: @quinton-hoole @jianhuiz @wojtek-t @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Move UTs that block on apiserver to integration tests.
In validating etcd.v3client we had uncovered that a change in the behavior of the client https://github.com/coreos/etcd/issues/6162 , caused a number of unit tests to fail. These test failures were due to the fact that the unit tests were trying to standup a apiserver even though there was no etcd backend stood up.
This PR simply shuffles those tests to integration tests, which is where they should be.
/cc @kubernetes/sig-scalability @wojtek-t @hongchaodeng @xiang90
Automatic merge from submit-queue
Fix deadlock possibility in federated informer
On cluster add subinformer locks and tries to add cluster to federated informer. When someone checks if everything is in sync federated informer is locked and then subinformer is inspected what apparently requires a lock. With really bad timing this can create a deadlock.
This PR ensures that there is always at most 1 lock taken in federated informer.
cc: @quinton-hoole @kubernetes/sig-cluster-federation
Fixes: #30855
Automatic merge from submit-queue
Federated namespace controller
Implemented based on federation common libs.
Depends on #30126.
cc: @quinton-hoole @wojtek-t @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Update dnsprovider multi-type support test to test for an A-record and an AAAA-record.
Having an A-record and a CNAME-record in the tests led to a confusion
that dns providers support such configurations. This change avoids
that confusion by putting only compatible records for the same domain
name in the tests.
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Fixing the error in registering /v1 api
Temporary fix for https://github.com/kubernetes/kubernetes/issues/30635.
Am working on a proper fix, but its much bigger.
Would be good to unblock devs in the meantime.
cc @lavalamp @kubernetes/sig-api-machinery
Having an A-record and a CNAME-record in the tests led to a confusion
that dns providers support such configurations. This change avoids
that confusion by putting only compatible records for the same domain
name in the tests.
Automatic merge from submit-queue
SetClientFactory in FederatedInformer
For unit tests.
cc: @quinton-hoole @wojtek-t @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
Move new etcd storage (low level storage) into cacher
In an effort for #29888, we are pushing forward this:
What?
- It changes creating etcd storage.Interface impl into creating config
- In creating cacher storage (StorageWithCacher), it passes config created above and new etcd storage inside.
Why?
- We want to expose the information of (etcd) kv client to cacher. Cacher storage uses this information to talk to remote storage.
Automatic merge from submit-queue
A build and deploy script to manage cluster federation lifecycle and a Makefile wrapper to drive that process.
Also includes a sample config file to describe clusters.
The build script implements the following things:
1. Generates the required configs.
2. Builds the hyperkube binary and the corresponding docker image.
3. Pushes the image to a specified repository.
4. Pulls the federation installer docker images.
5. Builds the Kubernetes clusters described the config.json file.
6. Pushes the federation components to one of the Kubernetes clusters
built in the previous step.
7. Also turns down the federation components and the Kubernetes
clusters.
**NOTE**: Installer images are right now being pulled from my public repository of docker images. I am working on pushing them to our release repository.
```release-note
Cluster Federation components can now be built and deployed using the make command. Please see federation/README.md for details.
```
cc @kubernetes/sig-cluster-federation @colhom
Fixes: Issue #26655
Also, wrap the script around a Makefile. And also provide a sample
config file to describe clusters.
The build script implements the following things:
1. Generates the required configs.
2. Builds the hyperkube binary and the corresponding docker image.
3. Pushes the image to a specified repository.
4. Pulls the federation installer docker images.
5. Builds the Kubernetes clusters described the config.json file.
6. Pushes the federation components to one of the Kubernetes clusters
built in the previous step.
7. Also turns down the federation components and the Kubernetes
clusters.
Automatic merge from submit-queue
Simplify canonical element term in deepcopy
Replace the old functional canonical element term in deepcopy registration with direct struct instantiation.
The old way was an artifact of non-uniform pointer/non-pointer types in the signature of deepcopy function. Since we changed that to always be a pointer, we can simplify the code.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30341)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Federation - common libs - a set of handy handlers for informers
A common scenario for using informer handlers would be to put the object key in a queue
for global reconciliation.
cc: @quinton-hoole @wojtek-t @kubernetes/sig-cluster-federation
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/30353)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Cut the client repo, staging it in the main repo
Tracking issue: #28559
ref: https://github.com/kubernetes/kubernetes/pull/25978#issuecomment-232710174
This PR implements the plan a few of us came up with last week for cutting client into its own repo:
1. creating "_staging" (name is tentative) directory in the main repo, using a script to copy the client and its dependencies to this directory
2. periodically publishing the contents of this staging client to k8s.io/client-go repo
3. converting k8s components in the main repo to use the staged client. They should import the staged client as if the client were vendored. (i.e., the import line should be `import "k8s.io/client-go/<pacakge name>`). This requirement is to ease step 4.
4. In the future, removing the staging area, and vendoring the real client-go repo.
The advantage of having the staging area is that we can continuously run integration/e2e tests with the latest client repo and the latest main repo, without waiting for the client repo to be vendored back into the main repo. This staging area will exist until our test matrix is vendoring both the client and the server.
In the above plan, the tricky part is step 3. This PR achieves it by creating a symlink under ./vendor, pointing to the staging area, so packages in the main repo can refer to the client repo as if it's vendored. To prevent the godep tool from messing up the staging area, we export the staged client to GOPATH in hack/godep-save.sh so godep will think the client packages are local and won't attempt to manage ./vendor/k8s.io/client-go.
This is a POC. We'll rearrange the directory layout of the client before merge.
@thockin @lavalamp @bgrant0607 @kubernetes/sig-api-machinery
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29147)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Remove default etcd validation in generic apiserver
Moving verification of `--etcd-servers` to the concrete apiserver instead of checking during defaulting in generic apiserver.
The context for this change is that heapster (will be another apiserver) doesn't need to have etcd underneath.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.kubernetes.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.kubernetes.io/reviews/kubernetes/kubernetes/29993)
<!-- Reviewable:end -->
Automatic merge from submit-queue
Run goimport for the whole repo
While removing GOMAXPROC and running goimports, I noticed quite a lot of other files also needed a goimport format. Didn't commit `*.generated.go`, `*.deepcopy.go` or files in `vendor`
This is more for testing if it builds.
The only strange thing here is the gopkg.in/gcfg.v1 => github.com/scalingdata/gcfg replace.
cc @jfrazelle @thockin
Automatic merge from submit-queue
Federation - common libs - cluster lifecycle handlers in FederatedInformer
Adds cluster lifecycle handlers to FederatedInformer to execute some actions when a cluster becomes available or unavailable.
ref: #29347
cc: @nikhiljindal @wojtek-t @kubernetes/sig-cluster-federation
Automatic merge from submit-queue
federation: Adding secret API
Adding secret API to federation-apiserver and updating the federation client to include secrets
Automatic merge from submit-queue
Federation - common libs - FederatedInformer
Fixes#29383
Will add more tests after the first pass of the review.
ref: #29347
cc: @nikhiljindal @wojtek-t
Automatic merge from submit-queue
Delete useless code
The correct code is `cluster, ok := obj.(*v1beta1.Cluster)`, so the above `cluster := obj.(*v1beta1.Cluster)` is useless.
Automatic merge from submit-queue
[Federation] Downsize the release binary distribution.
There are two things that this PR implements:
1. It removes `federation-apiserver` and `federation-controller-manager` from binaries and docker_wrapped_binaries target lists.
2. Build the docker image for `hyperkube` on-the-fly while pushing the federation images.
```release-note
Federation binaries and their corresponding docker images - `federation-apiserver` and `federation-controller-manager` are now folded in to the `hyperkube` binary. If you were using one of these binaries or docker images, please switch to using the `hyperkube` version. Please refer to the federation manifests - `federation/manifests/federation-apiserver.yaml` and `federation/manifests/federation-controller-manager-deployment.yaml` for examples.
```
cc @kubernetes/sig-cluster-federation @colhom
Fixes Issue #28633
