Automatic merge from submit-queue (batch tested with PRs 41104, 41245, 40722, 41439, 41502)
Bump the minimum kubeadm control plane version to v1.6.0-alpha.2
**What this PR does / why we need it**:
There went in quite a lot of useful features into v1.6.0-alpha.2 that kubeadm will use.
This bump the minimum limit so we can depend on those features.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@mikedanese @errordeveloper @pires @dmmcquay @dgoodwin
Automatic merge from submit-queue (batch tested with PRs 41104, 41245, 40722, 41439, 41502)
add sample fuzzing tests
Make fuzzing tests as simple as possible from both the API installer and the scheme, so its easy to add for api groups and so that I can build a scheme and then make sure I got it right.
@kubernetes/sig-api-machinery-pr-reviews @sttts @mikedanese
Some imports dont exist yet (or so it seems) in client-go (examples
being:
- "k8s.io/kubernetes/pkg/api/validation"
- "k8s.io/kubernetes/pkg/util/initsystem"
- "k8s.io/kubernetes/pkg/util/node"
one change in kubelet to import to client-go
Automatic merge from submit-queue
Allow multipe DNS servers as comma-seperated argument for kubelet --dns
This PR explores how kubectls "--dns" could be extended to specify multiple DNS servers for in-cluster PODs. Testing on the local libvirt-coreos cluster shows that multiple DNS server are injected without issues.
Specifying multiple DNS servers increases resilience against
- Packet drops
- Single server failure
I am debugging services that do 50+ DNS requests for a single incoming interactive request, thus highly increase the chance of a slowdown (+5s) due to a single packet drop. Switching to two DNS servers will reduce the impact of the issues (roughly +1s on glibc, 0s on musl, error-rate goes down to error-rate^2).
Note that there is no need to change any runtime related code as far as I know. In the case of "default" dns the /etc/resolv.conf is parsed and multiple DNS server are send to the backend anyway. This only adds the same capability for the clusterFirst case.
I've heard from @thockin that multiple DNS entries are somehow considered. I've no idea what was considered, though. This is what I would like to see for our production use, though.
```release-note
NONE
```
Automatic merge from submit-queue
Switch resourcequota controller to shared informers
Originally part of #40097
I have had some issues with this change in the past, when I updated `pkg/quota` to use the new informers while `pkg/controller/resourcequota` remained on the old informers. In this PR, both are switched to using the new informers. The issues in the past were lots of flakey test failures in the ResourceQuota e2es, where it would randomly fail to see deletions and handle replenishment. I am hoping that now that everything here is consistently using the new informers, there won't be any more of these flakes, but it's something to keep an eye out for.
I also think `pkg/controller/resourcequota` could be cleaned up. I don't think there's really any need for `replenishment_controller.go` any more since it's no longer running individual controllers per kind to replenish. It instead just uses the shared informer and adds event handlers to it. But maybe we do that in a follow up.
cc @derekwaynecarr @smarterclayton @wojtek-t @deads2k @sttts @liggitt @timothysc @kubernetes/sig-scalability-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41134, 41410, 40177, 41049, 41313)
apiserver: further cleanup of apiserver storage plumbing
- move kubeapiserver`s `RESTOptionsFactory` back to EtcdOptions by adding a `AddWithStorageFactoryTo`
- factor out storage backend `Config` construction from EtcdOptions
- move all `StorageFactory` related code into server/storage subpackage.
In short: remove my stomach ache about `kubeapiserver.RESTOptionsFactory`.
approved based on #40363
Automatic merge from submit-queue (batch tested with PRs 41360, 41423, 41430, 40647, 41352)
move kubeadm api group testing to kubeadm package
I think this is sufficient to at least preserve round trip testing.
Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)
move kube-aggregator to staging
Straight move of kube-aggregator to staging. The scripts are the interesting bits.
@sttts @lavalamp
Automatic merge from submit-queue
Make EnableCRI default to true
This change makes kubelet to use the CRI implementation by default,
unless the users opt out explicitly by using --enable-cri=false.
For the rkt integration, the --enable-cri flag will have no effect
since rktnetes does not use CRI.
Also, mark the original --experimental-cri flag hidden and deprecated,
so that we can remove it in the next release. If both flags are specified,
the --enable-cri flag overrides the --experimental-cri flag.
Automatic merge from submit-queue
kubeadm: add dmmcquay as a reviewer
**What this PR does / why we need it**: add dmmcquay (myself) as a reviewer to kubeadm.
**Special notes for your reviewer**: /cc @luxas
**Release note**:
```release-note
NONE
```
This change makes kubelet to use the CRI implementation by default,
unless the users opt out explicitly by using --enable-cri=false.
For the rkt integration, the --enable-cri flag will have no effect
since rktnetes does not use CRI.
Also, mark the original --experimental-cri flag hidden and deprecated,
so that we can remove it in the next release.
Automatic merge from submit-queue (batch tested with PRs 41216, 41362, 41275, 41277, 41412)
kubeadm: test-cmd: join{config, discovery}
**What this PR does / why we need it**: Adding test-cmds for kubeadm join {config, discovery}. Will followup with more test-cmds for other flags as soon as validation for the flags works.
Adding tests is a WIP from #34136
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41216, 41362, 41275, 41277, 41412)
kubeadm: test-cmd: init{config, version, api-port}
**What this PR does / why we need it**: Adding test-cmds for kubeadm init {config, version, api-port}. Will followup with more test-cmds for other flags as soon as validation for the flags works.
Adding tests is a WIP from #34136
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41216, 41362, 41275, 41277, 41412)
kubeadm: added unit tests for discovery pkg
**What this PR does / why we need it**: added tests to discovery pkg and raised coverage from ~25% to ~71%.
Adding unit tests is a WIP from #34136
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
kubeadm: test-cmd: token{delete}
**What this PR does / why we need it**: Adding test-cmds for kubeadm ex token delete. Will followup with more test-cmds for other flags as soon as validation for the flags works.
Adding tests is a WIP from #34136
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41382, 41407, 41409, 41296, 39636)
Update the DNS spec from upstream
**What this PR does / why we need it**:
Updates kubeadm to use the latest DNS spec.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
@bowei @MrHohn @thockin In the future, kubedns changes should be applied to this kubeadm file as well
**Release note**:
```release-note
NONE
```
@pires @errordeveloper @dmmcquay @mikedanese
Automatic merge from submit-queue (batch tested with PRs 41337, 41375, 41363, 41034, 41350)
kubeadm: adding tests for util/tokens.go
**What this PR does / why we need it**: added tests to util pkg and raised coverage from ~48% to ~67%. Will get better coverage once migration to client-go is complete. Included a fix for a logic error in tokens.go found through writing tests
Adding unit tests is a WIP from #34136
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Remove all aggregator dependencies on k8s.io/kubernetes
This removes the last links to k8s.io/kubernetes for the aggregator.
@sttts last commit.
Automatic merge from submit-queue
Change default attach_detach_controller reconciler sync period to 1 minute
When default reconciler sync period is set to 5 second, we often see
rateLimit issue for a large cluster. This PR is changing the period to 1
minute to mitigate this problem.
Make this period longer means that there might be some period of time
that the cached information in master's attach_detach_controller is out
of date. The node might use this information to mount to the wrong
device. For GCE PD, since device path is uniquely associated with volume
id, so mount operation will just fail because of this outdated
information. For AWS, before kubelet might mount to the wrong volume
because device path could be reused immediately once it is available.
But after PR #38818, device path will only be reused after all device
paths have been explored. That means it is very unlikely that kubelet will
mount to a wrong volume that is using the old device path that had been
assigned to the same node.
**Release note**:
```release-note
We change the default attach_detach_controller sync period to 1 minute to reduce the query frequency through cloud provider to check whether volumes are attached or not.
```
Automatic merge from submit-queue (batch tested with PRs 41342, 41257, 41295, 41367, 41230)
kubeadm: unit tests for apis/kubeadm/validation
**What this PR does / why we need it**: added tests to apis/kubeadm/validation pkg and raised coverage from ~21% to 100%.
Adding unit tests is a WIP from #34136
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41342, 41257)
kubeadm: fix to avoid panic if token not provided
**What this PR does / why we need it**: Prior to this, kubeadm would panic if no token was provided running `kubeadm ex token delete`. This does a check to verify an arg has been passed and prints out a more reasonable message if it is not provided.
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41115, 41212, 41346, 41340, 41172)
Add informers to client-go
Adds informers and listers to client-go so that you can build a controller on it.
@sttts @caesarxuchao @kubernetes/sig-api-machinery-misc
When default reconciler sync period is set to 5 second, we often see
rateLimit issue for a large cluster. This PR is change the period to 1
minute to mitigate this problem.
Make this period longer means that there might be some period of time
that the cached information in master's attach_detach_controller is out
of date. The node might use this information to mount to the wrong
device. For GCE PD, since device path is uniquely associated with volume
id, so mount operation will just fail because of this outdated
information. For AWS, before kubelet might mount to the wrong volume
because device path could be reused immediately once it is available.
But after PR #38818, device path will only be reused after all device
paths have been explored. That means it is very unlikely that kubelet will
mount to a wrong volume that is using the old device path that had been
assigned to the same node.
Automatic merge from submit-queue
copy pkg/util/logs to apiserver
This is a copy, not a move. API servers need to be able to init the logs, but so do clients. It would be weird to have the client-side commands depending on the server side logs utilities.
I updated all the server side references, but left the client-side ones.
@sttts @kubernetes/sig-api-machinery-pr-reviews acceptable?
Automatic merge from submit-queue (batch tested with PRs 41137, 41268)
Allow the CertificateController to use any Signer implementation.
**What this PR does / why we need it**:
This will allow developers to create `CertificateController`s with arbitrary `Signer`s, instead of forcing the use of `CFSSLSigner`. It matches the behavior of allowing an arbitrary `AutoApprover` to be passed in the constructor.
**Release note**:
```release-note
NONE
```
CC @mikedanese
Automatic merge from submit-queue (batch tested with PRs 41259, 41260)
kubeadm: changed manifest files to yaml
**What this PR does / why we need it**: Static Pods are currently stored as .json files in /etc/kubernetes/manifests. This PR instead writes them as YAML, as requested by the SIG.
**Which issue this PR fixes**: fixes #https://github.com/kubernetes/kubeadm/issues/153
**Special notes for your reviewer**: /cc @luxas
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 38252, 41122, 36101, 41017, 41264)
Fix cluster-cidr flag
**What this PR does / why we need it**:
Fixes the kube-proxy daemonset config when using the `pod-network-cidr flag`. The return value of `getClusterCIDR` should be prefixed with a `-`.
**Special notes for your reviewer**:
None
@luxas
Automatic merge from submit-queue (batch tested with PRs 38252, 41122, 36101, 41017, 41264)
Add alternative names for the server binaries to hyperkube
**What this PR does / why we need it**:
Right now one can't swap a server image to the hyperkube image without touching the `command` field in the yaml spec, and that's daunting and leading to extra and unnecessary logic for example in kubeadm.
This makes the hyperkube image directly swappable, so now `/usr/local/bin/kube-*` is a portable first argument (or simply `kube-*` if there's a shell).
**Special notes for your reviewer**:
**Release note**:
```release-note
Align the hyperkube image to support running binaries at /usr/local/bin/ like the other server images
```
@jessfraz @thockin @ixdy
Automatic merge from submit-queue (batch tested with PRs 38252, 41122, 36101, 41017, 41264)
BootstrapSigner and TokenCleaner controllers
This is part of https://github.com/kubernetes/features/issues/130 and is an implementation of https://github.com/kubernetes/community/pull/189.
Work that needs to be done yet in this PR:
* [ ] ~~e2e tests~~ Will come in new PR.
* [x] flag to disable this by default
```release-note
Native support for token based bootstrap flow. This includes signing a well known ConfigMap in the `kube-public` namespace and cleaning out expired tokens.
```
@kubernetes/sig-cluster-lifecycle @dgoodwin @roberthbailey @mikedanese
Automatic merge from submit-queue (batch tested with PRs 41223, 40892, 41220, 41207, 41242)
kubeadm: preflight should only warn about unresolvable hostnames
**What this PR does / why we need it**:
This is quite often the case on AWS, and we really don't care if
the hostname is resolvable or not. It's not an easy requirement
to ask user to fix, and there is no functional penalty at the
Kubernetes level, also it's possible that users fixes their host
resolution eventually, we don't have to make them do so.
**Special notes for your reviewer**: @dmmcquay @luxas PTAL 👍
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 39418, 41175, 40355, 41114, 32325)
TaintController
```release-note
This PR adds a manager to NodeController that is responsible for removing Pods from Nodes tainted with NoExecute Taints. This feature is beta (as the rest of taints) and enabled by default. It's gated by controller-manager enable-taint-manager flag.
```
Automatic merge from submit-queue (batch tested with PRs 39418, 41175, 40355, 41114, 32325)
ResyncPeriod Comment
ResyncPeriod Comment:
// ResyncPeriod returns a function which generates a duration each time it is
// invoked; this is so that multiple controllers don't get into lock-step and all
// hammer the apiserver with list requests simultaneously.
Automatic merge from submit-queue (batch tested with PRs 41112, 41201, 41058, 40650, 40926)
verify: Use macOS compatible copying method
**What this PR does / why we need it**:
Similar to the fix in #34944, this fixes issues in the `make verify` tests, by using a copy method that is compatible with macOS and the bsd version of `cp`.
Before fix:
```
Verifying hack/make-rules/../../hack/verify-codegen.sh
cp: illegal option -- T
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file target_file
cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file ... target_directory
FAILED hack/make-rules/../../hack/verify-codegen.sh 0s
```
After fix:
```
Verifying hack/make-rules/../../hack/verify-codegen.sh
Building client-gen
Building lister-gen
Building informer-gen
diffing cmd/kube-aggregator/hack/../pkg against freshly generated codegen
cmd/kube-aggregator/hack/../pkg up to date.
+++ [0128 10:06:48] Building the toolchain targets:
k8s.io/kubernetes/hack/cmd/teststale
k8s.io/kubernetes/vendor/github.com/jteeuwen/go-bindata/go-bindata
+++ [0128 10:06:48] Generating bindata:
test/e2e/generated/gobindata_util.go
/opt/gopath/src/k8s.io/kubernetes /opt/gopath/src/k8s.io/kubernetes/test/e2e/generated
/opt/gopath/src/k8s.io/kubernetes/test/e2e/generated
+++ [0128 10:06:49] Building go targets for darwin/amd64:
cmd/libs/go2idl/client-gen
cmd/libs/go2idl/lister-gen
cmd/libs/go2idl/informer-gen
Building client-gen
Building lister-gen
Building informer-gen
SUCCESS hack/make-rules/../../hack/verify-codegen.sh 59s
```
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41112, 41201, 41058, 40650, 40926)
Promote TokenReview to v1
Peer to https://github.com/kubernetes/kubernetes/pull/40709
We have multiple features that depend on this API:
- [webhook authentication](https://kubernetes.io/docs/admin/authentication/#webhook-token-authentication)
- [kubelet delegated authentication](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authentication)
- add-on API server delegated authentication
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating the token
This PR promotes the existing v1beta1 API to v1 with no changes
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authentication.k8s.io API group was promoted to v1
```
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
HPA v2 (API Changes)
**Release note**:
```release-note
Introduces an new alpha version of the Horizontal Pod Autoscaler including expanded support for specifying metrics.
```
Implements the API changes for kubernetes/features#117.
This implements #34754, which is the new design for the Horizontal Pod Autoscaler. It includes improved support for custom metrics (and/or arbitrary metrics) as well as expanded support for resource metrics. The new HPA object is introduces in the API group "autoscaling/v1alpha1".
Note that the improved custom metric support currently is limited to per pod metrics from Heapster -- attempting to use the new "object metrics" will simply result in an error. This will change once #34586 is merged and implemented.
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
kubeadm: added tests for preflight checks
**What this PR does / why we need it**: There hadn't been much care to add more unit tests as more preflight checks were added. I added tests that increased coverage from ~9% to ~71%
Adding unit tests is a WIP from https://github.com/kubernetes/kubernetes/issues/34136
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40796, 40878, 36033, 40838, 41210)
Implement TTL controller and use the ttl annotation attached to node in secret manager
For every secret attached to a pod as volume, Kubelet is trying to refresh it every sync period. Currently Kubelet has a ttl-cache of secrets of its pods and the ttl is set to 1 minute. That means that in large clusters we are targetting (5k nodes, 30pods/node), given that each pod has a secret associated with ServiceAccount from its namespaces, and with large enough number of namespaces (where on each node (almost) every pod is from a different namespace), that resource in ~30 GETs to refresh all secrets every minute from one node, which gives ~2500QPS for GET secrets to apiserver.
Apiserver cannot keep up with it very easily.
Desired solution would be to watch for secret changes, but because of security we don't want a node watching for all secrets, and it is not possible for now to watch only for secrets attached to pods from my node.
So as a temporary solution, we are introducing an annotation that would be a suggestion for kubelet for the TTL of secrets in the cache and a very simple controller that would be setting this annotation based on the cluster size (the large cluster is, the bigger ttl is).
That workaround mean that only very local changes are needed in Kubelet, we are creating a well separated very simple controller, and once watching "my secrets" will be possible it will be easy to remove it and switch to that. And it will allow us to reach scalability goals.
@dchen1107 @thockin @liggitt
This is quite often the case on AWS, and we really don't care if
the hostname is resolvable or not. It's not an easy problem
to ask user to fix, and there is no functional penalty at the
Kubernetes level, also it's possible that users fixes their host
resolution eventually, we don't have to make them do so.
Automatic merge from submit-queue (batch tested with PRs 40917, 41181, 41123, 36592, 41183)
fix ca cert in kubeadm
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
Automatic merge from submit-queue (batch tested with PRs 41074, 41147, 40854, 41167, 40045)
kubeadm: adding integration tests for init
**What this PR does / why we need it**: integration tests for kubeadm init focused on valid and invalid discovery tokens
**Special notes for your reviewer**: /cc @luxas @pires
This was taken from Pires's work in https://github.com/kubernetes/kubernetes/pull/40008 . This is just the testing aspect of it
As it stands, these tests will never complete. The reason being is once it passes with a valid discovery token, it will wait until `[apiclient] Created API client, waiting for the control plane to become ready ` which is an infinite poll (https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/master/apiclient.go#L71).
This is a WIP
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41037, 40118, 40959, 41084, 41092)
Switch CSR controller to use shared informer
Switch the CSR controller to use a shared informer. Originally part of #40097 but I'm splitting that up into multiple PRs.
I have added a test to try to ensure we don't mutate the cache. It could use some fleshing out for additional coverage but it gets the initial job done, I think.
cc @mikedanese @deads2k @liggitt @sttts @kubernetes/sig-scalability-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
add k8s.io/sample-apiserver to demonstrate how to build an aggregated API server
builds on https://github.com/kubernetes/kubernetes/pull/41093
This creates a sample API server is a separate staging repo to guarantee no cheating with `k8s.io/kubernetes` dependencies. The sample is run during integration tests (simple tests on it so far) to ensure that it continues to run.
@sttts @kubernetes/sig-api-machinery-misc ptal
@pwittrock @pmorie @kris-nova an aggregated API server example that will stay up to date.
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
Remove deprecated kubelet flags that look safe to remove
Removes:
```
--config
--auth-path
--resource-container
--system-container
```
which have all been marked deprecated since at least 1.4 and look safe to remove.
```release-note
The deprecated flags --config, --auth-path, --resource-container, and --system-container were removed.
```
Automatic merge from submit-queue (batch tested with PRs 40175, 41107, 41111, 40893, 40919)
kubeadm: skip integration tests if kubeadm-cmd-skip flag passed
Will skip integration tests for token generation if it can't find a file by the given --kubeadm-path or default value.
**What this PR does / why we need it**: Tests would fail if just running `go test` in the dir because it expects to have more values. This won't change the behavior of `make test-cmd` which gets run here:
https://github.com/kubernetes/kubernetes/blob/master/Makefile#L258
**Which issue this PR fixes**: fixes#40155
**Special notes for your reviewer**: /cc @pires @pipejakob @liggitt
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 38796, 40823, 40756, 41083, 41105)
Add more options to the RecommendedOptions struct.
Builds on https://github.com/kubernetes/kubernetes/pull/41028
Adds `AuditOptions` to the `RecommendedOptions`
@sttts @kubernetes/sig-api-machinery-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 41061, 40888, 40664, 41020, 41085)
move --runtime-config to kubeapiserver
`--runtime-config` is only useful if you have a lot of API groups in one server. If you have a single API group in your server (the vast majority of aggregated API servers), then the flag is unneeded and relatively complex. This moves it to closer to point of use.
@sttts
Automatic merge from submit-queue (batch tested with PRs 41061, 40888, 40664, 41020, 41085)
kubeadm: Small cleanup and fixes, validate the service subnet
**What this PR does / why we need it**:
- Validate the minimum subnet cidr so there are always 10 available addresses
- Remove an old proxy arg function, add clustercidr to the proxy manifest and automatically calculate the dns ip
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@errordeveloper @pires @mikedanese @dmmcquay @dgoodwin
Automatic merge from submit-queue (batch tested with PRs 41061, 40888, 40664, 41020, 41085)
kubeadm: added tests cert/pkiutil pkg
raised coverage from ~37% to ~77%
**What this PR does / why we need it**: added tests to cert/pkiutil pkg and raised coverage from ~37% to ~77%
Adding unit tests is a WIP from #34136
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 41103, 41042, 41097, 40946, 40770)
Use Clientset interface in KubeletDeps
**What this PR does / why we need it**:
This replaces the Clientset struct with the equivalent interface for the KubeClient injected via KubeletDeps. This is useful for testing and for accessing the Node and Pod status event stream without an API server.
**Special notes for your reviewer**:
Follow up to #4907
**Release note**:
`NONE`
Automatic merge from submit-queue (batch tested with PRs 41064, 41090, 41068, 41073, 40882)
kubeadm: updated tests in app/util pkg
**What this PR does / why we need it**: Tests did not follow the standard of being table driven. Updated to make future changes (Adding/deleting tests) easier as well as more readable
Adding unit tests is a WIP from #34136
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
After today's SIG meeting, it was discussed how to proceed with these
types of test-cmd tests. They will live in kubeamd/test/cmd and will
provide a flag that will allow you to skip them (--kubeadm-cmd-skip) and
by default will fail if kubeadm binary is not present
Automatic merge from submit-queue (batch tested with PRs 40345, 38183, 40236, 40861, 40900)
refactor approver and signer interfaces to be consisten w.r.t. apiserver interaction
This makes it so that only the controller loop talks to the
API server directly. The signatures for Sign and Approve also
become more consistent, while allowing the Signer to report
conditions (which it wasn't able to do before).
Automatic merge from submit-queue (batch tested with PRs 41023, 41031, 40947)
apiserver command line options lead to config
Logically command line options lead to config, not the other way around. We're clean enough now we can actually do the inversion.
WIP because I have some test cycles to fix, but this is all the meat.
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 41023, 41031, 40947)
scrub aggregator names to eliminate discovery
Cleanup old uses of `discovery`. Also removes the legacy functionality.
@kubernetes/sig-api-machinery-misc @sttts
Automatic merge from submit-queue (batch tested with PRs 40971, 41027, 40709, 40903, 39369)
Promote SubjectAccessReview to v1
We have multiple features that depend on this API:
SubjectAccessReview
- [webhook authorization](https://kubernetes.io/docs/admin/authorization/#webhook-mode)
- [kubelet delegated authorization](https://kubernetes.io/docs/admin/kubelet-authentication-authorization/#kubelet-authorization)
- add-on API server delegated authorization
The API has been in use since 1.3 in beta status (v1beta1) with negligible changes:
- Added a status field for reporting errors evaluating access
- A typo was discovered in the SubjectAccessReviewSpec Groups field name
This PR promotes the existing v1beta1 API to v1, with the only change being the typo fix to the groups field. (fixes https://github.com/kubernetes/kubernetes/issues/32709)
Because the API does not persist data (it is a query/response-style API), there are no data migration concerns.
This positions us to promote the features that depend on this API to stable in 1.7
cc @kubernetes/sig-auth-api-reviews @kubernetes/sig-auth-misc
```release-note
The authorization.k8s.io API group was promoted to v1
```
Automatic merge from submit-queue (batch tested with PRs 40385, 40786, 40999, 41026, 40996)
kubeadm: change os.Remove to os.RemoveAll
**What this PR does / why we need it**: This would not correctly clean out the tmp dir, since os.Remove doesn't recursively remove dirs, which could exist in the tmp dir.
Unit tests are a WIP from #34136
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Replace hand-written informers with generated ones
Replace existing uses of hand-written informers with generated ones.
Follow-up commits will switch the use of one-off informers to shared
informers.
This is a precursor to #40097. That PR will switch one-off informers to shared informers for the majority of the code base (but not quite all of it...).
NOTE: this does create a second set of shared informers in the kube-controller-manager. This will be resolved back down to a single factory once #40097 is reviewed and merged.
There are a couple of places where I expanded the # of caches we wait for in the calls to `WaitForCacheSync` - please pay attention to those. I also added in a commented-out wait in the attach/detach controller. If @kubernetes/sig-storage-pr-reviews is ok with enabling the waiting, I'll do it (I'll just need to tweak an integration test slightly).
@deads2k @sttts @smarterclayton @liggitt @soltysh @timothysc @lavalamp @wojtek-t @gmarek @sjenning @derekwaynecarr @kubernetes/sig-scalability-pr-reviews
Automatic merge from submit-queue (batch tested with PRs 39681, 39321, 41018, 40883)
kubeadm: app/util/template.go has unit tests
**What this PR does / why we need it**: There was a TODO tag about adding unit tests, but unit tests have been added and the coverage for that file is ~90%. Tag should be removed.
Adding unit tests is a WIP from #34136
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Kubeadm discovery remove error passing
**What this PR does / why we need it**: In the app/discovery there is some confusion about the passing of error values created in the discovery/token, discovery/https/ and discovery/file pkgs. Since they always return `nil` , it was very confusing in discovery/flags.go why to propagate them up as if there was a chance for them to return a value other than `nil`. This change makes it much more clear what is being passed.
I noticed this as I was making a sweep through trying to add more unit tests and it was very confusing to read the code.
**Which issue this PR fixes** : fixes #https://github.com/kubernetes/kubeadm/issues/141
**Special notes for your reviewer**: /cc @luxas @pires
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
kubeadm: Remove the kubernetes.io/cluster-service label from the Deployment templates
**What this PR does / why we need it**:
As discussed on Slack, these labels have no function when not using the addon-manager, so it's best to remove them to avoid confusion.
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@thockin @mikedanese @pires @MrHohn @bowei @dmmcquay @deads2k @philips
This makes it so that only the controller loop talks to the
API server directly. The signatures for Sign and Approve also
become more consistent, while allowing the Signer to report
conditions (which it wasn't able to do before).
Automatic merge from submit-queue (batch tested with PRs 35782, 35831, 39279, 40853, 40867)
remove unnecessarily duplication since types collapsed
We collapsed duplicate types into client-go, so we get to clean this up.
@sttts as promised.
Automatic merge from submit-queue (batch tested with PRs 35782, 35831, 39279, 40853, 40867)
genericapiserver: cut off more dependencies – episode 7
Follow-up of https://github.com/kubernetes/kubernetes/pull/40822
approved based on #40363
Automatic merge from submit-queue
Removed HPA objects from extensions api group
fix#29778
``` release-note
HorizontalPodAutoscaler is no longer supported in extensions/v1beta1 version. Use autoscaling/v1 instead.
```
cc @kubernetes/autoscaling
Automatic merge from submit-queue
kubeadm: fix reset error logging.
**What this PR does / why we need it**: while investigating https://github.com/kubernetes/kubeadm/issues/142 I realized error logging was misleading. This PR is meant to fix it.
**Special notes for your reviewer**: /cc @luxas
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Move post processing/backward compatibility of openapi out of generic package
Post processing step was put into generic package and was the same for both k8s api server and federation api server. They have different backward compatibility list of types. This PR move that step out of generic package and put it in each server's file.
Automatic merge from submit-queue
kubeadm: Refactoring the apiconfig and addons phases
**What this PR does / why we need it**:
First commit: Fix the defaulting for AuthorizationMode - **Ready for review**
Second commit: Refactoring the apiconfig and addons phases in kubeadm - **work in progress, broken at the moment**
**Special notes for your reviewer**:
Please take a look at the first commit now. You can also see the direction I'm going with the addons and apiconfig phases.
For example, I'm using Go templates instead of creating native Go structs for kube-dns and kube-proxy now. The question is if I should do it for the RBAC rules as well, it might make sense there as well.
Converting the dns addon to a yaml spec makes it 100x easier to maintain when changes are made to the "upstream" DNS spec, and also more swappable. We could for instance have a configuration param for it for those who want a custom file, etc.
**Release note**:
```release-note
NONE
```
@mikedanese @pires @lukemarsden @errordeveloper @dgoodwin @liggitt @andrewrynhard
Automatic merge from submit-queue (batch tested with PRs 40574, 40806, 40308, 40771, 39440)
kubeadm: kube-proxy needs to know the pod subnet CIDR
**What this PR does / why we need it**: `kube-proxy` 1.5 has a new flag `cluster-cidr` that isn't specified by `kubeadm`, thus resulting in bug https://github.com/kubernetes/kubeadm/issues/102.
**Which issue this PR fixes**: fixes https://github.com/kubernetes/kubeadm/issues/102
**Special notes for your reviewer**:
/cc @luxas @dmmcquay
Automatic merge from submit-queue (batch tested with PRs 40574, 40806, 40308, 40771, 39440)
kubeadm: preflight check for incorrect FQDN
**What this PR does / why we need it**: There are a variety of system configuration errors (such as cloud-init bugs when deploying on AWS) which can cause hostname and uname -n to be wrong for a given host. This will cause kubeadm setup to fail in interesting and hard-to-figure-out ways (it doesn't fail until you start trying to set up DNS on the master, for example).
This PR adds a preflight check to test whether or not the server can reach itself using that name. This does not catch the case that the FQDN belongs to a different but valid server, but it would catch some of the cases.
**Which issue this PR fixes** : fixes https://github.com/kubernetes/kubeadm/issues/135
**Special notes for your reviewer**: /cc @luxas
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40574, 40806, 40308, 40771, 39440)
move api server no kube dep packages
Simple moves, see commit titles. I did not move the source of the generated swagger, I simply change the destination of the script. I'm ok building a little debt to complete the move.
@sttts
Depending on an exact cluster setup multiple dns may make sense.
Comma-seperated lists of DNS server are quite common as DNS servers
are always plain IPs.
Automatic merge from submit-queue
kubeadm: Move some code from apiclient.go to the dedicated apiconfig phase
**What this PR does / why we need it**:
Add constants and somewhat refactor the RBAC code as well + some cleanup.
I'm planning to rewrite the code in `setupmaster.go` later, but this PR has only the move of the code for easier reviewing.
**Special notes for your reviewer**:
This is broken out from: https://github.com/kubernetes/kubernetes/pull/40556
**Release note**:
```release-note
NONE
```
@mikedanese @pires @dmmcquay @dgoodwin
Automatic merge from submit-queue
Use full package path for definition name in OpenAPI spec
We were using short package name (last part of package name) plus type name for OpenAPI spec definition name. That can result in duplicate names and make the spec invalid. To be sure we will always have unique names, we are going to use full package name as definition name. Also "x-kubernetes-tag" custom field is added to definitions to list Group/Version/Kind for the definitions that has it. This will help clients to discover definitions easier.
Lastly, we've added a reference from old definition names to the new ones to keep backward compatibilities. The list of old definitions will not be updated.
**Release note**:
- Rename OpenAPI definition names to type's full package names to prevent duplicates
- Create OpenAPI extension "x-kubernetes-group-version-kind" for definitions to store Group/Version/Kind
- Deprecate old definition names and create a reference to the new definitions. Old definitions will be removed in the next release.
Automatic merge from submit-queue
error strings should not end with punctuation
**What this PR does / why we need it**:
Delete the end punctuation of error strings
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
https://github.com/golang/go/wiki/CodeReviewComments#error-strings
**Release note**:
```release-note
```
Signed-off-by: yupeng <yu.peng36@zte.com.cn>
Automatic merge from submit-queue
remove unneeded storage options
Cleanup of some storage serialization options that only kube and federation api server (and maybe not even that one) need.
You may have called it a snip, but this moves the options out of generic entirely.
Automatic merge from submit-queue (batch tested with PRs 40638, 40742, 40710, 40718, 40763)
move client/record
An attempt at moving client/record to client-go. It's proving very stubborn and needs a lot manual intervention and near as I can tell, no one actually gets any benefit from the sink and source complexity it adds.
@sttts @caesarchaoxu
Automatic merge from submit-queue
add deads2k to some cmd packages
Selectively adding myself to cmd packages for commands that I've authored or significantly touched.
Automatic merge from submit-queue (batch tested with PRs 40707, 40698)
kubeadm: Default to control plane v1.6.0-alpha.1 and using RBAC
Also use constants for authz modes
**What this PR does / why we need it**:
Defaults to v1.6.0-alpha.1 (will be cut later today) because the certificates API group has been upgraded to beta, so `kubeadm join` at HEAD doesn't work on a `v1.5` cluster anyway.
By defaulting to v1.6.0-alpha.1, we can focus totally on v1.6 for kubeadm at HEAD, we don't support other versions in the upcoming v1.6 kubeadm release because of the alpha -> beta upgrades.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
@mikedanese @pires
Automatic merge from submit-queue (batch tested with PRs 40392, 39242, 40579, 40628, 40713)
fixed bug #36988 -- kubeadm join crashes when using multiple API endpoints
**What this PR does / why we need it**:
Simple bug fix for #36988 (kubeadm crashes when trying to join nodes to a master with multiple API endpoints) -- stopChan was getting closed multiple times, once per endpoint. Moved the close into the once.Do(...)
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
fixes#36988
**Special notes for your reviewer**:
this is my first kubernetes commit, just getting familiar with the process, so any/all feedback is appreciated
**Release note**:
Automatic merge from submit-queue (batch tested with PRs 38443, 40145, 40701, 40682)
Move kubeadm etcd SELinux options from container to pod.
**What this PR does / why we need it**:
Works around a bug that surfaces in Docker 1.12+ related to the pause
container's namespace and selinux labels being transferred to the etcd
container when it runs.
At present it appears that applying selinux options to a container may
be broken, or perhaps shouldn't be supported at all. Moving these to the
pod causes all containers (including pause) to run with the correct
labels.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
Related to and partial fix for https://github.com/kubernetes/kubeadm/issues/107
This is one of several selinux related fixes in flight for upcoming releases, and newer versions of Docker. To successfully run kubeadm with selinux enforcing right now would like require a recent container-selinux build as uncovered in https://github.com/kubernetes/kubeadm/issues/107, a bugfix for the format labels in #40179, and finally this fix.
**Release note**:
```release-note
Fixed an SELinux issue in kubeadm on Docker 1.12+ by moving etcd SELinux options from container to pod.
```
Automatic merge from submit-queue (batch tested with PRs 40691, 40551, 40683, 40700, 40702)
Clean up generated comment for lister/informers
The generated comment including the command line means the addition of a new group/version touches all listers/informers, not just the ones for the new group/version
@ncdc PTAL
Automatic merge from submit-queue (batch tested with PRs 40691, 40551, 40683, 40700, 40702)
Enable protobufs as storage format by default
**Release note**:
```release-note
Change default storage format to protobufs. With this PR apiserver will be writing objects serialized as protobufs to etcd. If the apiserver was upgraded in the existing clusters, until an object will be written, it will still be stored as JSON. Apiserver can deal with some data being in json and some in protobuf format as of 1.4 release.
```
@kubernetes/sig-api-machinery-misc @kubernetes/sig-api-machinery-pr-reviews
Works around a bug that surfaces in Docker 1.12+ related to the pause
container's namespace and selinux labels being transferred to the etcd
container when it runs.
At present it appears that applying selinux options to a container may
be broken, or perhaps shouldn't be supported at all. Moving these to the
pod causes all containers (including pause) to run with the correct
labels.
Automatic merge from submit-queue
Remove proxy-mode annotation from kube-proxy
**What this PR does / why we need it**:
This removes the net.experimental.kubernetes.io/proxy-mode and net.beta.kubernetes.io/proxy-mode annotations from kube-proxy.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#40582
**Special notes for your reviewer**:
**Release note**:
```release-note
Remove outdated net.experimental.kubernetes.io/proxy-mode and net.beta.kubernetes.io/proxy-mode annotations from kube-proxy.
```
Automatic merge from submit-queue (batch tested with PRs 40132, 39302, 40194, 40619, 40601)
fix error message in TestGetAddonImage
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
**What this PR does / why we need it**:
1. fix incorrect error message in TestGetAddonImage
2. rename some variables to make them more reasonable
Automatic merge from submit-queue (batch tested with PRs 40132, 39302, 40194, 40619, 40601)
fix nil check and typos
**What this PR does / why we need it**:
1. nil error should be checked before defer statement.
2. fix some typos.
Signed-off-by: bruceauyeung ouyang.qinhua@zte.com.cn
Automatic merge from submit-queue
add test tokens for TestValidateToken
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
**What this PR does / why we need it**:
1. add invalid tokens which token ID and token secret contain special characters such as `*`
2. add valid tokens which token ID and token secret contain mixed lowercase characters and numeric numbers
Automatic merge from submit-queue
fix incorrect parameter passing
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
**What this PR does / why we need it**:
1. fix incorrect parameter passing when creating error
2. fix ineffectual assignment to err variable.
Automatic merge from submit-queue (batch tested with PRs 40543, 39999)
remove pkg/feature dependency from genericapiserver.
@sttts I think you just forgot this when you were snipping the rest of the link. If I understood the previous pull correctly, this ought to ensure proper registration of the bits we need, right?
Automatic merge from submit-queue (batch tested with PRs 40239, 40397, 40449, 40448, 40360)
move the discovery and dynamic clients
Moved the dynamic client, discovery client, testing/core, and testing/cache to `client-go`. Dependencies on api groups we don't have generated clients for have dropped out, so federation, kubeadm, and imagepolicy.
@caesarxuchao @sttts
approved based on https://github.com/kubernetes/kubernetes/issues/40363
Automatic merge from submit-queue (batch tested with PRs 39538, 40188, 40357, 38214, 40195)
genericapiserver: cut off more dependencies – episode 2
Compare commit subjects.
approved based on #40363
Automatic merge from submit-queue
Wait for the self-hosted control plane during kubeadm init.
**What this PR does / why we need it**:
Currently we are completing kubeadm init while the scheduler and
controller manager are likely still not yet up. In some cases if they
will fail, they won't ever come up.
Instead wait until each pod enters running state before exiting kubeadm.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 40428, 40176)
Cleaup Affinity post conversion from annotations to fields
**What this PR does / why we need it**:
Cleans up leftover work from the conversion of affinity from annotations to fields.
fixes#40016
related #25319
**Special notes for your reviewer**:
There are some TODO items left for @luxas or @errordeveloper b/c they were trying to use affinity in a way that is not possible.
**Release note**:
```release-note
NONE
```
/cc @kubernetes/sig-scheduling-misc @rrati
Automatic merge from submit-queue
update kubectl approvers
All the other owners files have names (not groups). Updated for individual users for approvers so `/approve` works.
@kubernetes/sig-cli-api-reviews
Automatic merge from submit-queue
move client/cache and client/discovery to client-go
mechanical changes to move those packages. Had to create a `k8s.io/kubernetes/pkg/client/tests` package for tests that were blacklisted from client-go. We can rewrite these tests later and move them, but for now they'll still run at least.
@caesarxuchao @sttts
Currently we are completing kubeadm init while the scheduler and
controller manager are likely still not yet up. In some cases if they
will fail, they won't ever come up.
Instead wait until each pod enters running state before exiting kubeadm.
Automatic merge from submit-queue (batch tested with PRs 40303, 40316)
kubeadm: Fix up the apiserver kubelet client cert
**What this PR does / why we need it**:
I didn't notice that the apiserver serving cert was _only_ a server cert, not both a server and client cert in https://github.com/kubernetes/kubernetes/pull/40292.
This PR fixes that misconception and generates a dedicated client cert for apiserver -> kubelet communication and harmonizes the `ca.crt` path to `/etc/kubernetes/pki/ca.crt` on both master and node.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
```console
$ systemctl cat kubelet
# /lib/systemd/system/kubelet.service
[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=http://kubernetes.io/docs/
[Service]
ExecStart=/usr/bin/kubelet
Restart=always
StartLimitInterval=0
RestartSec=10
[Install]
WantedBy=multi-user.target
# /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--kubeconfig=/etc/kubernetes/kubelet.conf **--authorization-mode=Webhook --client-ca-file=/etc/kubernetes/pki/ca.crt** --require-kubeconfig=true"
Environment="KUBELET_SYSTEM_PODS_ARGS=--pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true"
Environment="KUBELET_NETWORK_ARGS=--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin"
Environment="KUBELET_DNS_ARGS=--cluster-dns=10.96.0.10 --cluster-domain=cluster.local"
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_EXTRA_ARGS
$ curl -k https://localhost:10250/containerLogs/kube-system/weave-net-arm64-4t2xv/weave
Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy)
$ curl --cert /etc/kubernetes/pki/apiserver-kubelet-client.crt --key /etc/kubernetes/pki/apiserver-kubelet-client.key -k https://localhost:10250/containerLogs/kube-system/weave-net-arm64-4t2xv/weave
[Works!!]
```
**Release note**:
```release-note
NONE
```
@liggitt @deads2k
Automatic merge from submit-queue
OWNERS: Update latest OWNERS files
These files have been created lately, so we don't have much information
about them anyway, so let's just:
- Remove assignees and make them approvers
- Copy approves as reviewers
Automatic merge from submit-queue (batch tested with PRs 40251, 40171)
Mark --docker-exec-handler deprecated
We plan to drop support for the nsenter exec handler in the future. Marking this flag as deprecated to warn the users.
These files have been created lately, so we don't have much information
about them anyway, so let's just:
- Remove assignees and make them approvers
- Copy approves as reviewers
Automatic merge from submit-queue
make client-go tools/auth and tools/clientcmd authoritative
These moved easily. Pure mechanical. I'll have a couple snip pulls for some of the more complicated packages. `pkg/util/homedir` was a transitive that moved with. It was only used in `kubectl` and you'd need it to be consistent with `clientcmd` loading order.
@sttts
Automatic merge from submit-queue (batch tested with PRs 38445, 40292)
kubeadm: Secure apiserver -> kubelet communication and set storage backend to etcd3
**What this PR does / why we need it**:
Switch storage backend to etcd3
Writes ca.crt down to disk so we can set `--client-ca-file=/etc/kubernetes/ca.crt` for the kubelet.
Adds --kubelet-client-{certificate,key} to the apiserver args and make it allowed to access the kubelets
In some cases the `path` dependency is switched to `filepath`
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
fixes https://github.com/kubernetes/kubeadm/issues/118
fixes https://github.com/kubernetes/kubeadm/issues/129
**Special notes for your reviewer**:
This PR is should make it possible to secure the apiserver -> kubelet communication.
**Release note**:
```release-note
NONE
```
@pires @mikedanese @andrewrynhard @liggitt @deads2k
Automatic merge from submit-queue (batch tested with PRs 39199, 37273, 29183, 39638, 40199)
Refactor/improve the kubeadm generation of certificates
**What this PR does / why we need it**:
Continues to refactor/improve kubeadm towards beta.
**Special notes for your reviewer**:
Modify the certs that are generated; generate on demand (if not exist) and only four files instead of eight previously. Basically implements what has been discussed so far in https://github.com/kubernetes/kubeadm/pull/100
**Release note**:
```release-note
NONE
```
cc @mikedanese @pires @lukemarsden @errordeveloper @dgoodwin @roberthbailey
Automatic merge from submit-queue (batch tested with PRs 40187, 40231, 40152)
Update client/server cert generation utilities
Limit generated cert usage to client or server use.
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
Cleanup temp dirs
So funny story my /tmp ran out of space running the unit tests so I am cleaning up all the temp dirs we create.
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubeadm: add optional self-hosted deployment
**What this PR does / why we need it**: add an optional self-hosted deployment type to `kubeadm`, for master components only, namely `apiserver`, `controller-manager` and `scheduler`.
**Which issue this PR fixes**: closes#38407
**Special notes for your reviewer**: /cc @aaronlevy @luxas @dgoodwin
**Release note**:
```release-note
kubeadm: add optional self-hosted deployment for apiserver, controller-manager and scheduler.
```
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
move apimachinery/announced to apimachinery
This also rewires the announce package to allow installation into multiple groups via multiple calls to `install.Install`. At some point, we'll want to unwire the hardcoded, "install to this scheme" call.
@lavalamp something we've wanted for a while and need for genericapiserver
@seh you've asked related questions
@sttts ptal. First commit is interesting, second commit is a straight move.
Automatic merge from submit-queue (batch tested with PRs 37228, 40146, 40075, 38789, 40189)
kubelet: storage: teardown terminated pod volumes
This is a continuation of the work done in https://github.com/kubernetes/kubernetes/pull/36779
There really is no reason to keep volumes for terminated pods attached on the node. This PR extends the removal of volumes on the node from memory-backed (the current policy) to all volumes.
@pmorie raised a concern an impact debugging volume related issues if terminated pod volumes are removed. To address this issue, the PR adds a `--keep-terminated-pod-volumes` flag the kubelet and sets it for `hack/local-up-cluster.sh`.
For consideration in 1.6.
Fixes#35406
@derekwaynecarr @vishh @dashpole
```release-note
kubelet tears down pod volumes on pod termination rather than pod deletion
```
Automatic merge from submit-queue (batch tested with PRs 36693, 40154, 40170, 39033)
make client-go authoritative for pkg/client/restclient
Moves client/restclient to client-go and a util/certs, util/testing as transitives.
Automatic merge from submit-queue (batch tested with PRs 39772, 39831, 39481, 40167, 40149)
Only enable RBAC if the k8s version is above 1.6.0-alpha.0
...and move the defaulting of the api type into a specific file
**What this PR does / why we need it**: RBAC should only be enabled when k8s version is v1.6.0-alpha.0+, where https://github.com/kubernetes/kubernetes/pull/39625 is merged
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
```
$ sudo ./kubeadm init
[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.
[init] Using Kubernetes version: v1.5.2
[init] Using Authorization mode: AlwaysAllow
[init] A token has not been provided, generating one
[preflight] Running pre-flight checks
[preflight] WARNING: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Starting the kubelet service
[certificates] Generated Certificate Authority key and certificate.
[certificates] Generated API Server key and certificate
[certificates] Generated Service Account signing keys
[certificates] Created keys and certificates in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[apiclient] Created API client, waiting for the control plane to become ready
[apiclient] All control plane components are healthy after 39.517396 seconds
[apiclient] Waiting for at least one node to register and become ready
[apiclient] First node is ready after 6.521587 seconds
[apiclient] Creating a test deployment
[apiclient] Test deployment succeeded
[token-discovery] Using token: 6c3430:2889e46ce3f11610
[token-discovery] Created the kube-discovery deployment, waiting for it to become ready
[token-discovery] kube-discovery is ready after 3.502025 seconds
[addons] Created essential addon: kube-proxy
[addons] Created essential addon: kube-dns
Your Kubernetes master has initialized successfully!
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
http://kubernetes.io/docs/admin/addons/
You can now join any number of machines by running the following on each node:
kubeadm join --discovery token://6c3430:2889e46ce3f11610@192.168.200.101:9898
```
```
$ cat fetch-ci-images.sh
#!/bin/bash
CI_COMMIT=${CI_COMMIT:-latest}
ARCH=${ARCH:-amd64}
OLD_REGISTRY=${OLD_REGISTRY:-gcr.io/google_containers}
NEW_REGISTRY=${NEW_REGISTRY:-${OLD_REGISTRY}}
if [[ ${CI_COMMIT} == "latest" ]]; then
CI_COMMIT=$(curl -sSL https://storage.googleapis.com/kubernetes-release-dev/ci-cross/latest.txt)
fi
CI_COMMIT_TAG=$(echo ${CI_COMMIT} | sed "s/+/-/")
echo "Using commit: ${CI_COMMIT}"
echo "Tagging to registry: ${NEW_REGISTRY}"
images=(
kube-apiserver
kube-scheduler
kube-controller-manager
kube-proxy
)
for image in ${images[@]}; do
curl -sSL https://storage.googleapis.com/kubernetes-release-dev/ci-cross/${CI_COMMIT}/bin/linux/${ARCH}/${image}.tar | docker load
image_tag=$(curl -sSL https://storage.googleapis.com/kubernetes-release-dev/ci-cross/${CI_COMMIT}/bin/linux/${ARCH}/${image}.md5)
docker tag ${OLD_REGISTRY}/${image}:${image_tag} ${NEW_REGISTRY}/${image}-${ARCH}:${CI_COMMIT_TAG}
docker rmi ${OLD_REGISTRY}/${image}:${image_tag}
echo "Image created: ${NEW_REGISTRY}/${image}-${ARCH}:${CI_COMMIT_TAG}"
done
cat <<EOF
Now you can run:
kubeadm init --use-kubernetes-version ${CI_COMMIT_TAG}
EOF
$ ./fetch-ci-images.sh
Using commit: v1.6.0-alpha.0.3549+5e4625cad72e5b
Tagging to registry: gcr.io/google_containers
c366cffde3c9: Loading layer 1.312 MB/1.312 MB
1911ea24d99d: Loading layer 1.024 kB/1.024 kB
3af1ee6d3b81: Loading layer 132.9 MB/132.9 MB
Untagged: gcr.io/google_containers/kube-apiserver:8562727e1eabe97a95bc2e52992db195
Image created: gcr.io/google_containers/kube-apiserver-amd64:v1.6.0-alpha.0.3549-5e4625cad72e5b
c366cffde3c9: Loading layer 1.312 MB/1.312 MB
1911ea24d99d: Loading layer 1.024 kB/1.024 kB
86bf938b66e2: Loading layer 54.88 MB/54.88 MB
Untagged: gcr.io/google_containers/kube-scheduler:5319acbc670b5050f119e6f6e3c9b7e3
Image created: gcr.io/google_containers/kube-scheduler-amd64:v1.6.0-alpha.0.3549-5e4625cad72e5b
c366cffde3c9: Loading layer 1.312 MB/1.312 MB
1911ea24d99d: Loading layer 1.024 kB/1.024 kB
6743a986e1ed: Loading layer 104.9 MB/104.9 MB
Untagged: gcr.io/google_containers/kube-controller-manager:6698cef6338d549160d58fd8381f3d07
Image created: gcr.io/google_containers/kube-controller-manager-amd64:v1.6.0-alpha.0.3549-5e4625cad72e5b
bf028f50c346: Loading layer 128.9 MB/128.9 MB
a99e621f3f37: Loading layer 1.024 kB/1.024 kB
76d4fdcd51a9: Loading layer 5.076 MB/5.076 MB
ee455b365369: Loading layer 42.52 MB/42.52 MB
Untagged: gcr.io/google_containers/kube-proxy:3914060ffe3225ae29fe88cb960826f9
Image created: gcr.io/google_containers/kube-proxy-amd64:v1.6.0-alpha.0.3549-5e4625cad72e5b
Now you can run:
kubeadm init --use-kubernetes-version v1.6.0-alpha.0.3549-5e4625cad72e5b
```
```
$ sudo ./kubeadm init --use-kubernetes-version v1.6.0-alpha.0.3549-5e4625cad72e5b
[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.
[init] Using Kubernetes version: v1.6.0-alpha.0.3549-5e4625cad72e5b
[init] Using Authorization mode: RBAC
[init] A token has not been provided, generating one
[preflight] Running pre-flight checks
[preflight] WARNING: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Starting the kubelet service
[certificates] Generated Certificate Authority key and certificate.
[certificates] Generated API Server key and certificate
[certificates] Generated Service Account signing keys
[certificates] Created keys and certificates in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[apiclient] Created API client, waiting for the control plane to become ready
[apiclient] All control plane components are healthy after 31.252686 seconds
[apiclient] Waiting for at least one node to register and become ready
[apiclient] First node is ready after 9.064842 seconds
[apiclient] Creating a test deployment
[apiclient] Test deployment succeeded
[apiconfig] Created kubelet-bootstrap RBAC rules
[apiconfig] Created kube-dns RBAC rules
[apiconfig] Created kube-proxy RBAC rules
[token-discovery] Using token: 99b957:ccb1fbc703a0db8a
[token-discovery] Created the kube-discovery deployment, waiting for it to become ready
[token-discovery] kube-discovery is ready after 3.502064 seconds
[addons] Created essential addon: kube-proxy
[addons] Created essential addon: kube-dns
Your Kubernetes master has initialized successfully!
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
http://kubernetes.io/docs/admin/addons/
You can now join any number of machines by running the following on each node:
kubeadm join --discovery token://99b957:ccb1fbc703a0db8a@192.168.200.101:9898
```
**Release note**:
```release-note
NONE
```
@pires @mikedanese @andrewrynhard
Automatic merge from submit-queue
promote certificates api to beta
Mostly posting to see what breaks but also this API is ready to be promoted.
```release-note
Promote certificates.k8s.io to beta and enable it by default. Users using the alpha certificates API should delete v1alpha1 CSRs from the API before upgrading and recreate them as v1beta1 CSR after upgrading.
```
@kubernetes/api-approvers @jcbsmpsn @pipejakob
Automatic merge from submit-queue
make client-go more authoritative
Builds on https://github.com/kubernetes/kubernetes/pull/40103
This moves a few more support package to client-go for origination.
1. restclient/watch - nodep
1. util/flowcontrol - used interface
1. util/integer, util/clock - used in controllers and in support of util/flowcontrol
Automatic merge from submit-queue
Add authorization mode to kubeadm
This PR adds an option in `kubeadm` to allow a user to specify an [authorization plugin](https://kubernetes.io/docs/admin/authorization/). It defaults to RBAC.
Automatic merge from submit-queue
Curating Owners: cmd/kubelet
cc @yujuhong @dchen1107 @vishh
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the future in
the **reviewers** section. You probably do NOT need to modify the **approvers**
section. Names asre sorted by relevance, using some secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue (batch tested with PRs 39625, 39842)
Add RBAC v1beta1
Add `rbac.authorization.k8s.io/v1beta1`. This scrubs `v1alpha1` to remove cruft, then add `v1beta1`. We'll update other bits of infrastructure to code to `v1beta1` as a separate step.
```release-note
The `attributeRestrictions` field has been removed from the PolicyRule type in the rbac.authorization.k8s.io/v1alpha1 API. The field was not used by the RBAC authorizer.
```
@kubernetes/sig-auth-misc @liggitt @erictune
Automatic merge from submit-queue (batch tested with PRs 40038, 40041, 39036)
move admission to genericapiserver
I disconnected the initialization that was type specific for later assessment.
@sttts
Automatic merge from submit-queue (batch tested with PRs 40008, 40005, 40018)
genericapiserver: move pkg/auth/handlers into filters
Move authn filters to the other api related filters.
Automatic merge from submit-queue
kubeadm: init must validate or generate token before anything else.
**What this PR does / why we need it**: `kubeadm init` must validate or generate a token before anything else. Otherwise, if token validation or generation fail, one will need to run `kubeadm reset && systemctl restart kubelet` before re-running `kubeadm init`.
**Which issue this PR fixes**: fixeskubernetes/kubeadm#112
**Special notes for your reviewer**: /cc @luxas
Tested manually.
### With no token
```
$ sudo ./kubeadm init
[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.
[preflight] Running pre-flight checks
[init] Using Kubernetes version: v1.5.2
[token-discovery] A token has not been provided, generating one
[certificates] Generated Certificate Authority key and certificate.
[certificates] Generated API Server key and certificate
[certificates] Generated Service Account signing keys
[certificates] Created keys and certificates in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[apiclient] Created API client, waiting for the control plane to become ready
[apiclient] All control plane components are healthy after 7.762803 seconds
[apiclient] Waiting for at least one node to register and become ready
[apiclient] First node is ready after 1.003148 seconds
[apiclient] Creating a test deployment
[apiclient] Test deployment succeeded
[token-discovery] Using token: 8321b6:a535ba541af7623c
[token-discovery] Created the kube-discovery deployment, waiting for it to become ready
[token-discovery] kube-discovery is ready after 1.003423 seconds
[addons] Created essential addon: kube-proxy
[addons] Created essential addon: kube-dns
Your Kubernetes master has initialized successfully!
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
http://kubernetes.io/docs/admin/addons/
You can now join any number of machines by running the following on each node:
kubeadm join --discovery token://8321b6:a535ba541af7623c@10.142.0.6:9898
```
### With invalid token
```
$ sudo ./kubeadm init --discovery token://12345:12345
[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.
[preflight] Running pre-flight checks
[init] Using Kubernetes version: v1.5.2
[token-discovery] A token has been provided, validating [&{ID:12345 Secret:12345 Addresses:[]}]
token ["12345:12345"] was not of form ["^([a-z0-9]{6})\\:([a-z0-9]{16})$"]
```
### With valid token
```
$ sudo ./kubeadm ex token generate
cd540e:c0e0318e2f4a63b1
$ sudo ./kubeadm init --discovery token://cd540e:c0e0318e2f4a63b1
[kubeadm] WARNING: kubeadm is in alpha, please do not use it for production clusters.
[preflight] Running pre-flight checks
[init] Using Kubernetes version: v1.5.2
[token-discovery] A token has been provided, validating [&{ID:cd540e Secret:c0e0318e2f4a63b1 Addresses:[]}]
[certificates] Generated Certificate Authority key and certificate.
[certificates] Generated API Server key and certificate
[certificates] Generated Service Account signing keys
[certificates] Created keys and certificates in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[apiclient] Created API client, waiting for the control plane to become ready
[apiclient] All control plane components are healthy after 13.513305 seconds
[apiclient] Waiting for at least one node to register and become ready
[apiclient] First node is ready after 0.502656 seconds
[apiclient] Creating a test deployment
[apiclient] Test deployment succeeded
[token-discovery] Using token: cd540e:c0e0318e2f4a63b1
[token-discovery] Created the kube-discovery deployment, waiting for it to become ready
[token-discovery] kube-discovery is ready after 2.002457 seconds
[addons] Created essential addon: kube-proxy
[addons] Created essential addon: kube-dns
Your Kubernetes master has initialized successfully!
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
http://kubernetes.io/docs/admin/addons/
You can now join any number of machines by running the following on each node:
kubeadm join --discovery token://cd540e:c0e0318e2f4a63b1@10.142.0.6:9898
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Made tracing of calls and container lifecycle steps in FakeDockerClient optional
Fixes#39717
Slightly refactored the FakeDockerClient code and made tracing optional (but enabled by default).
@yujuhong @Random-Liu
Automatic merge from submit-queue
log cfgzErr if err happened
We need to log err info when err info returned by initConfigz(),no matter what the result of utilconfig.DefaultFeatureGate.DynamicKubeletConfig() is and
whether s.RunOnce is true or not.
We should log the initKubeletConfigSync() err info too.
Automatic merge from submit-queue
Remove packages which are now apimachinery
Removes all the content from the packages that were moved to `apimachinery`. This will force all vendoring projects to figure out what's wrong. I had to leave many empty marker packages behind to have verify-godep succeed on vendoring heapster.
@sttts straight deletes and simple adds
Automatic merge from submit-queue (batch tested with PRs 34763, 38706, 39939, 40020)
Use Statefulset instead in e2e and controller
Quick fix ref: #35534
We should finish the issue to meet v1.6 milestone.
Automatic merge from submit-queue (batch tested with PRs 34763, 38706, 39939, 40020)
prevent anonymous auth and allow all
https://github.com/kubernetes/kubernetes/pull/38696 for master
@kubernetes/sig-auth
```release-note
Anonymous authentication is now automatically disabled if the API server is started with the AlwaysAllow authorizer.
```
Automatic merge from submit-queue
Move PatchType to apimachinery/pkg/types
Fixes https://github.com/kubernetes/kubernetes/issues/39970
`PatchType` is shared by the client and server, they have to agree, and its critical for our API to function.
@smarterclayton @kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 39911, 40002, 39969, 40012, 40009)
kubeadm: upgrade kube-dns to 1.11.0.
**What this PR does / why we need it**: See kubernetes/dns#25
**Which issue this PR fixes**: fixeskubernetes/kubeadm#121
**Special notes for your reviewer**: /cc @luxas
I know this is not the template solution you are looking for but seems to me it's important enough to do this now because of the issues it fixes.
Tested manually and it works.
`NONE`
Automatic merge from submit-queue
[kubeadm] resetting cluster should check whether docker service is active
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
**What this PR does / why we need it**:
if not, `kubeadm reset` will fail to remove kubernetes-managed containers
Automatic merge from submit-queue
Added validation for API server's 'apiserver-count' flag.
Added validation for API server's 'apiserver-count' flag. The value of this flag should be a positive number, otherwise, will cause error while reconciling endpoints in MasterCountEndpointsReconciler.
Fixed#38143
Automatic merge from submit-queue
move name generation to generic api server storage helpers
Move name generation to the genericapiserver since only the server needs to know about it.
@kubernetes/sig-api-machinery-misc @sttts
Automatic merge from submit-queue
kubeadm: must lower-case token portion used in DNS label.
**What this PR does / why we need it**: In Kubernetes, DNS labels must be lower-case. `kubeadm` doesn't care when creating certain objects through the API. This PR fixes that erroneous behavior.
**Which issue this PR fixes**: fixes https://github.com/kubernetes/kubeadm/issues/104
**Special notes for your reviewer**: /cc @luxas @mikedanese @dgoodwin
Automatic merge from submit-queue (batch tested with PRs 38592, 39949, 39946, 39882)
move api/errors to apimachinery
`pkg/api/errors` is a set of helpers around `meta/v1.Status` that help to create and interpret various apiserver errors. Things like `.NewNotFound` and `IsNotFound` pairings. This pull moves it into apimachinery for use by the clients and servers.
@smarterclayton @lavalamp First commit is the move plus minor fitting. Second commit is straight replace and generation.
Automatic merge from submit-queue (batch tested with PRs 38592, 39949, 39946, 39882)
genericapiserver: cut off pkg/apis/extensions and pkg/storage dependencies
Move BuildDefaultStorageFactory to kubeapiserver.
Automatic merge from submit-queue (batch tested with PRs 39947, 39936, 39902, 39859, 39915)
don't lie about starting the controllers in the controller manager
We print started even if it didn't start.
We need to log err info when err info returned by initConfigz(),no matter what the result of utilconfig.DefaultFeatureGate.DynamicKubeletConfig() is and
whether s.RunOnce is true or not.
We should log the initKubeletConfigSync() err info too.
Automatic merge from submit-queue
replace global registry in apimachinery with global registry in k8s.io/kubernetes
We'd like to remove all globals, but our immediate problem is that a shared registry between k8s.io/kubernetes and k8s.io/client-go doesn't work. Since client-go makes a copy, we can actually keep a global registry with other globals in pkg/api for now.
@kubernetes/sig-api-machinery-misc @lavalamp @smarterclayton @sttts
Automatic merge from submit-queue
Curating Owners: cmd/kube-apiserver
cc @lavalamp @smarterclayton @krousey @nikhiljindal
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the future in
the **reviewers** section. You probably do NOT need to modify the **approvers**
section. Names asre sorted by relevance, using some secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue (batch tested with PRs 39661, 39740, 39801, 39468, 39743)
add --controllers to controller manager
Adds a `--controllers` flag to the `kube-controller-manager` to indicate which controllers are enabled and disabled. From the help:
```
--controllers stringSlice A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'.
All controllers: certificatesigningrequests, cronjob, daemonset, deployment, disruption, endpoint, garbagecollector, horizontalpodautoscaling, job, namespace, podgc, replicaset, replicationcontroller, resourcequota, serviceaccount, statefuleset
```
Automatic merge from submit-queue (batch tested with PRs 39475, 38666, 39327, 38396, 39613)
Add checking the nil value
**What this PR does / why we need it**:
Add checking the nil value.
Thanks.
**Special notes for your reviewer**:
Automatic merge from submit-queue (batch tested with PRs 39495, 39547)
kubeadm: updated preflight types to avoid stutter
Small change to kubeadm preflight pkg to remove stutter from preflight types PreFlightError and PreFlightCheck (now names Error and Checker).
**Release note**:
`NONE`
Automatic merge from submit-queue (batch tested with PRs 39684, 39577, 38989, 39534, 39702)
kubeadm: refactored token discovery.
**What this PR does / why we need it**: refactored `kubeadm` token discovery to follow the discovery interface and use new TLS bootstrap mechanism.
/cc @luxas @mikedanese
Automatic merge from submit-queue (batch tested with PRs 39673, 39536, 39617, 39540, 39686)
kubeadm: updated DNS deployment.
**What this PR does / why we need it**: Updates KubeDNS deployment to match upstream.
**Special notes for your reviewer**: It was tested manually by bootstrapping a new cluster, running a busybox container and making sure one could `nslookup` from within the container to find `kubernetes` and other services.
Automatic merge from submit-queue (batch tested with PRs 38212, 38792, 39641, 36390, 39005)
Generate a kubelet CA and kube-apiserver cert-pair for kubelet auth.
cc @cjcullen
Automatic merge from submit-queue (batch tested with PRs 39695, 37054, 39627, 39546, 39615)
fixed error log that would cause two log lines to run into each other…
**What this PR does / why we need it**:
Fixed error log that would cause two log lines to run into each other. Logs running into each other can be reproduced by running local-up-cluster.sh on osx.
**Release note**:
```NONE
```
Automatic merge from submit-queue (batch tested with PRs 39695, 37054, 39627, 39546, 39615)
remove repeating const declaration
**What this PR does / why we need it**:
remove repeating const declaration , and avoid const declaration in loop
Thanks.
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue (batch tested with PRs 39628, 39551, 38746, 38352, 39607)
Increasing times on reconciling volumes fixing impact to AWS.
#**What this PR does / why we need it**:
We are currently blocked by API timeouts with PV volumes. See https://github.com/kubernetes/kubernetes/issues/39526. This is a workaround, not a fix.
**Special notes for your reviewer**:
A second PR will be dropped with CLI cobra options in it, but we are starting with increasing the reconciliation periods. I am dropping this without major testing and will test on our AWS account. Will be marked WIP until I run smoke tests.
**Release note**:
```release-note
Provide kubernetes-controller-manager flags to control volume attach/detach reconciler sync. The duration of the syncs can be controlled, and the syncs can be shut off as well.
```
Automatic merge from submit-queue
switch webhook to clientgo
Switches the delegating authentication and authorization webhooks to use client-go. The ripples go out aways, but I kept it as contained as I could.
@sttts
Automatic merge from submit-queue (batch tested with PRs 37845, 39439, 39514, 39457, 38866)
Move watch/versioned to pkg/apis/meta/v1/watch.go
Watch is a part of the server API
Automatic merge from submit-queue
fix glog message typo about init deserialization cache and watch cache
**What this PR does / why we need it**:
fix typo `Initalizing` to `Initializing`
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
Automatic merge from submit-queue (batch tested with PRs 39466, 39490, 39527)
bump gengo to latest
bumping gengo to limit surprises while working on https://github.com/kubernetes/kubernetes/pull/39475
@kubernetes/sig-api-machinery-misc
Automatic merge from submit-queue (batch tested with PRs 39466, 39490, 39527)
Generate OpenAPI definition for inlined types
Currently OpenAPI definition generator ignores any type's member with an empty json name tag. However, most (if not all) of these types also have "inline" json tag. That means we should inline their members into parent type's OpenAPI definition instead of ignoring them. This resulted in many types missing common parameters such as "Name". It look something serious to me for OpenAPI spec and I suggest we merge and cherry-pick this into 1.5 release branch as soon as possible.
Automatic merge from submit-queue
Implemented file and HTTPS based discovery for kubeadm
**What this PR does / why we need it**:
This PR implements both file and HTTPS based discovery for `kubeadm`.
**Which issue this PR fixes**:
fixes https://github.com/kubernetes/kubeadm/issues/93
fixes https://github.com/kubernetes/kubeadm/issues/94
**Special notes for your reviewer**:
I'd like to add some tests but 'm sure `kubeconfig` loading is already covered by other tests in `clientcmd` package.
/cc @luxas @pipejakob
Automatic merge from submit-queue
kubeadm token discovery URL may not have valid input for url.Parse
**What this PR does / why we need it**:
**Which issue this PR fixes**: fixes https://github.com/kubernetes/kubeadm/issues/95
**Special notes for your reviewer**:
/cc @dgoodwin @luxas @mikedanese this is a WIP! Need a couple things:
* decide token separator, `.` or `:`
* define error handling in `token.go::37`
Automatic merge from submit-queue
kubeadm: change etcd data dir to match host path.
**What this PR does / why we need it**: Trivial change that adopts a well-known path for etcd data, following the `hostPath` defined.
**Which issue this PR fixes**: Fixes https://github.com/kubernetes/kubeadm/issues/80
**Special notes for your reviewer**:
/cc @luxas @kad
Automatic merge from submit-queue (batch tested with PRs 36229, 39450)
Bugfix on kubeadm join with non-token discovery.
Without this change, using a non-token discovery mechanism during a join triggers a nil pointer panic (instead of an error message about not being implemented yet).
CC @mikedanese
**Release note**:
```release-note
NONE
```
Without this change, using a non-token discovery mechanism during a join
triggers a nil pointer panic (instead of an error message about not
being implemented yet).
Automatic merge from submit-queue
Start moving genericapiserver to staging
This moves `pkg/auth/user` to `staging/k8s.io/genericapiserver/pkg/authentication/user`. I'll open a separate pull into the upstream gengo to support using `import-boss` on vendored folders to support staging.
After we agree this is the correct approach and see everything build, I'll start moving other packages over which don't have k8s.io/kubernetes deps.
@kubernetes/sig-api-machinery-misc @lavalamp
@sttts @caesarxuchao ptal
Automatic merge from submit-queue
make discovery static when extensions/thirdpartyresources is not enabled
this should be a bug fix, if `extensions/thirdpartyresources` is enabled, the result of `Discovery().ServerPreferredNamespacedResources` will be dynamic then, so we are making the `discoverResourcesFn` static only when the `extensions/thirdpartyresources` is not enabled.
Automatic merge from submit-queue (batch tested with PRs 39075, 39350, 39353)
Move pkg/api.{Context,RequestContextMapper} into pkg/genericapiserver/api/request
**Based on #39350**
Automatic merge from submit-queue (batch tested with PRs 39351, 39322)
Remove the deprecated --reconcile-cidr flag from kubelet
**What this PR does / why we need it**:
Removes the `--reconcile-cidr` flag that has been deprecated since v1.5
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
The `--reconcile-cidr` kubelet flag was removed since it had been deprecated since v1.5
```
@thockin
Automatic merge from submit-queue (batch tested with PRs 39150, 38615)
Add work queues to PV controller
PV controller should not use Controller.Requeue, as as it is not available in
shared informers. We need to implement our own work queues instead, where we
can enqueue volumes/claims as we want.
PV controller should not use Controller.Requeue, as as it is not available in
shared informers. We need to implement our own work queues instead where we
can enqueue volumes/claims as we want.
Automatic merge from submit-queue (batch tested with PRs 37959, 36221)
Recycle Pod Template Check
The kube-controller-manager has two command line arguments (--pv-recycler-pod-template-filepath-hostpath and --pv-recycler-pod-template-filepath-nfs) that specify a recycle pod template. The recycle pod template may not contain the volume that shall be recycled.
A check is added to make sure that the recycle pod template contains at least a volume.
cc: @jsafrane
Automatic merge from submit-queue
Remove all MAINTAINER statements in the codebase as they are deprecated
**What this PR does / why we need it**:
ref: https://github.com/docker/docker/pull/25466
**Release note**:
```release-note
Remove all MAINTAINER statements in Dockerfiles in the codebase as they are deprecated by docker
```
@ixdy @thockin (who else should be notified?)
Automatic merge from submit-queue
Begin paths for internationalization in kubectl
This is just the first step, purposely simple so we can get the interface correct.
@kubernetes/sig-cli @deads2k
Automatic merge from submit-queue
Fix cloud-config name in test case
**What this PR does / why we need it**: fixes default cloud-config name in test cases for reset.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: related to kubernetes/kubeadm#75
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
use bytes.Buffer instead of append for error string concat
**What this PR does / why we need it**:
1. in my benchmark test, `bytes.Buffer` takes much less time ( about 1:1000 ) than string append( `+=` ).
>BenchmarkAppendConcat-4 100000 151438 ns/op 578181 B/op 2 allocs/op
BenchmarkBufferSprintf-4 3000000 487 ns/op 65 B/op 3 allocs/op
BenchmarkBufferConcat-4 5000000 271 ns/op 47 B/op 1 allocs/op
the benchmark codes is here https://play.golang.org/p/LS52zGuwZN
2. in our `RunInitMasterChecks`, `RunJoinNodeChecks` there are lots of preflight checks. they may result in a huge error message. so `bytes.Buffer` can bring considerable performance enhancement in the worst of conditions.
beyond that, this PR
1. fix an exported struct comment,
1. and use `found = append( found, errs...)` instead of for loop for simplicity.
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
Automatic merge from submit-queue (batch tested with PRs 39093, 34273)
start breaking up controller manager into two pieces
This PR addresses: https://github.com/kubernetes/features/issues/88
This commit starts breaking the controller manager into two pieces, namely,
1. cloudprovider dependent piece
2. coudprovider agnostic piece
the controller manager has the following control loops -
- nodeController
- volumeController
- routeController
- serviceController
- replicationController
- endpointController
- resourceQuotaController
- namespaceController
- deploymentController
etc..
among the above controller loops,
- nodeController
- volumeController
- routeController
- serviceController
are cloud provider dependent. As kubernetes has evolved tremendously, it has become difficult
for different cloudproviders (currently 8), to make changes and iterate quickly. Moreover, the
cloudproviders are constrained by the kubernetes build/release lifecycle. This commit is the first
step in moving towards a kubernetes code base where cloud providers specific code will move out of
the core repository, and will be maintained by the cloud providers themselves.
I have added a new cloud provider called "external", which signals the controller-manager that
cloud provider specific loops are being run by another controller. I have added these changes in such
a way that the existing cloud providers are not affected. This change is completely backwards compatible, and does not require any changes to the way kubernetes is run today.
Finally, along with the controller-manager, the kubelet also has cloud-provider specific code, and that will be addressed in a different commit/issue.
@alena1108 @ibuildthecloud @thockin @dchen1107
**Special notes for your reviewer**:
@thockin - Im making this **WIP** PR to ensure that I don't stray too far from everyone's view of how we should make this change. As you can see, only one controller, namely `nodecontroller` can be disabled with the `--cloudprovider=external` flag at the moment. I'm working on cleaning up the `rancher-controller-manger` that I wrote to test this.
Secondly, I'd like to use this PR to address cloudprovider specific code in kubelet and api-server.
**Kubelet**
Kubelet uses provider specific code for node registration and for checking node-status. I thought of two ways to divide the kubelet:
- We could start a cloud provider specific kubelet on each host as a part of kubernetes, and this cloud-specific-kubelet does node registration and node-status checks.
- Create a kubelet plugin for each provider, which will be started by kubelet as a long running service. This plugin can be packaged as a binary.
I'm leaning towards the first option. That way, kubelet does not have to manage another process, and we can offload the process management of the cloud-provider-specific-kubelet to something like systemd.
@dchen1107 @thockin what do you think?
**Kube-apiserver**
Kube-apiserver uses provider specific code for distributing ssh keys to all the nodes of a cluster. Do you have any suggestions about how to address this?
**Release note**:
``` release-note
```
Automatic merge from submit-queue
kubeadm: Default to using token discovery.
Recent changes to support multiple methods for discovery meant that
"kubeadm init" no longer was sufficient and users would need to add
"--discovery token://" to achieve the same results.
Instead lets assume discovery if the user does not specify anything else
to maintain parity and the brevity of our original instructions.
**Release note**:
```release-note
NONE
```
CC @mikedanese @luxas
Automatic merge from submit-queue
Fix skipping - protobuf fields
**What this PR does / why we need it**: fixes the protobuf generator to skip fields with a protobuf tag of `"-"`
Match changes in https://github.com/kubernetes/gengo/pull/19
I couldn't get godeps to work to vendor this change in from gengo, so I made the same edits manually in cmd/libs/go2idl. A task for another day...
@smarterclayton
Automatic merge from submit-queue (batch tested with PRs 39006, 39078, 37188, 39118)
Cleanup dns
* Remove hostname endpoints annotation (was beta feature)
* Remove references to non-client-go API
* Replaces references to internal kubernetes API with client-go.
Recent changes to support multiple methods for discovery meant that
"kubeadm init" no longer was sufficient and users would need to add
"--discovery token://" to achieve the same results.
Instead lets assume discovery if the user does not specify anything else
to maintain parity and the brevity of our original instructions.
The kube-controller-manager has two command line arguments (--pv-recycler-pod-template-filepath-hostpath and --pv-recycler-pod-template-filepath-nfs) that specify a recycle pod template. The recycle pod template may not contain the volume that shall be recycled.
A check is added to make sure that the recycle pod template contains at least a volume.
Automatic merge from submit-queue
remove unneeded authenticator dependencies from genericapiserver
Refactors the authenticator options to remove unneeded dependencies.
@sttts
Automatic merge from submit-queue
Implement kubeadm bootstrap token management
Creates bootstrap tokens as secrets per the specification in #30707
_WARNING_: These are not currently hooked up to the discovery service or the token it creates.
Still TODO:
- [x] delete tokens
- [x] merge with #35144 and adopt it's testing approach
- [x] determine if we want wholesale json output & templating like kubectl (we do not have an API object with the data we want here) may require a bit of plumbing.
- [x] allow specifying a token duration on the CLI
- [x] allow configuring the default token duration
- [x] hook up the initial token created during init
Sample output:
```
(root@centos1 ~) $ kubeadm token create
Running pre-flight checks
<cmd/token> Token secret created: f6dc69.c43e491752c4a0fd
(root@centos1 ~) $ kubeadm token create
Running pre-flight checks
<cmd/token> Token secret created: 8fad2f.e7b78c8a5f7c7b9a
(root@centos1 ~) $ kubeadm token list
Running pre-flight checks
ID TOKEN EXPIRATION
44d805 44d805.a4e78b6cf6435e33 23h
4f65bb 4f65bb.d006a3c7a0e428c9 23h
6a086e 6a086e.2ff99f0823236b5b 23h
8fad2f 8fad2f.e7b78c8a5f7c7b9a 23h
f6dc69 f6dc69.c43e491752c4a0fd 23h
f81653 f81653.9ab82a2926c7e985 23h
```
Adds kubeadm subcommands to create, list, and delete bootstrap tokens.
Tokens can be created with a TTL duration, or 0 for tokens that will not
expire. The create command can also be used to specify your own token
(for use when bootstrapping masters and nodes in parallel), or update an
existing token's secret or ttl.
Marked "ex" for experimental for now as the boostrap controllers are not
yet hooked up in core.
- move rest handler into handlers package
- move errors into handlers/errors package
- move negotation code into handlers/negotation package
- move http response writer helpers into handlers/responsewriter package
- split up pkg/apiserver/apiserver.go
Note: this is only an intermediate step towards a proper genericapiserver
package hierarchy.
Automatic merge from submit-queue
conversion-gen: add --skip-unsafe flag
We should expose the SkipUnsafe option, for legacy compatability, so
that conversion-go can be used in other projects, and for platforms
where unsafe is not available.
Make unsafe code generation the default though, and have the help text
hint that the resulting code is sub-optimal.
Automatic merge from submit-queue
path.Clean paths in GlobalEnvParams and remove unnecessary path.Join
**What this PR does / why we need it**:
1. clean all paths in `GlobalEnvParams`
1. remove unnecessary path.Join call in `pki.go`
2. fix some typos and comment errors
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
Automatic merge from submit-queue
use in-cluster kubeconfig for genericapiserver
Allow the use of the in-cluster config to communicate with the core API server for delegated authn/authz for an addon API server.
@kubernetes/sig-api-machinery @sttts
Automatic merge from submit-queue
genericapiserver: turn APIContainer.SecretRoutes into a real ServeMux
The secret routes `Mux` is actually a `http.ServeMux` and we are type-casting to it. For downstream we want to wrap it into a restful container which also needs a real `http.ServeMux`.
Automatic merge from submit-queue (batch tested with PRs 38154, 38502)
Rename "release_1_5" clientset to just "clientset"
We used to keep multiple releases in the main repo. Now that [client-go](https://github.com/kubernetes/client-go) does the versioning, there is no need to keep releases in the main repo. This PR renames the "release_1_5" clientset to just "clientset", clientset development will be done in this directory.
@kubernetes/sig-api-machinery @deads2k
```release-note
The main repository does not keep multiple releases of clientsets anymore. Please find previous releases at https://github.com/kubernetes/client-go
```
Automatic merge from submit-queue
genericapiserver: unify swagger and openapi in config
- make swagger config customizable
- remove superfluous `Config.Enable*` flags for OpenAPI and Swagger.
This is necessary for downstream projects to tweak the swagger spec.
Automatic merge from submit-queue
fix connection upgrades through kuberentes-discovery
The initial upgrade through the proxy doesn't use the passed transport to handle the communication to the remote side. Since we need auth proxy headers, this broke the upgrade for exec.
This sets those headers once if its an upgrade request (the transport stomps them if called anyway, so it won't shadow.).
@sttts I think this is the last required piece. Then we start wiring in for e2e.
Automatic merge from submit-queue (batch tested with PRs 38315, 38624, 38572, 38544)
kubernetes-discovery proxy
The last commit adds an authenticating (but not authorizing) proxy to `kubernetes-discovery`. The other commits are in the submit queue already.
@sttts @cjcullen This is it. After this we can start e2e wiring.
Automatic merge from submit-queue
Curating Owners: cmd/kubeadm
cc @jbeda @mikedanese @luxas @errordeveloper
In an effort to expand the existing pool of reviewers and establish a
two-tiered review process (first someone lgtms and then someone
experienced in the project approves), we are adding new reviewers to
existing owners files.
If You Care About the Process:
------------------------------
We did this by algorithmically figuring out who’s contributed code to
the project and in what directories. Unfortunately, that doesn’t work
well: people that have made mechanical code changes (e.g change the
copyright header across all directories) end up as reviewers in lots of
places.
Instead of using pure commit data, we generated an excessively large
list of reviewers and pruned based on all time commit data, recent
commit data and review data (number of PRs commented on).
At this point we have a decent list of reviewers, but it needs one last
pass for fine tuning.
Also, see https://github.com/kubernetes/contrib/issues/1389.
TLDR:
-----
As an owner of a sig/directory and a leader of the project, here’s what
we need from you:
1. Use PR https://github.com/kubernetes/kubernetes/pull/35715 as an example.
2. The pull-request is made editable, please edit the `OWNERS` file to
remove the names of people that shouldn't be reviewing code in the future in
the **reviewers** section. You probably do NOT need to modify the **approvers**
section. Names asre sorted by relevance, using some secret statistics.
3. Notify me if you want some OWNERS file to be removed. Being an
approver or reviewer of a parent directory makes you a reviewer/approver
of the subdirectories too, so not all OWNERS files may be necessary.
4. Please use ALIAS if you want to use the same list of people over and
over again (don't hesitate to ask me for help, or use the pull-request
above as an example)
Automatic merge from submit-queue
Remove json serialization annotations from internal types
fixes#3933
Internal types should never be serialized, and including json serialization tags on them makes it possible to accidentally do that without realizing it.
fixes in this PR:
* types
* [x] remove json tags from internal types
* [x] fix references from serialized types to internal ObjectMeta
* generation
* [x] remove generated json codecs for internal types (they should never be used)
* kubectl
* [x] fix `apply` to operate on versioned object
* [x] fix sorting by field to operate on versioned object
* [x] fix `--record` to build annotation patch using versioned object
* hpa
* [x] fix unmarshaling to internal CustomMetricTargetList in validation
* thirdpartyresources
* [x] fix encoding API responses using internal ObjectMeta
* tests
* [x] fix tests to use versioned objects when checking encoded content
* [x] fix tests passing internal objects to generic printers
follow ups (will open tracking issues or additional PRs):
- [ ] remove json tags from internal kubeconfig types (`kubectl config set` pathfinding needs to work against external type)
- [ ] HPA should version CustomMetricTargetList serialization in annotations
- [ ] revisit how TPR resthandlers encoding objects
- [ ] audit and add tests for printer use (human-readable printer requires internal versions, generic printers require external versions)
- [ ] add static analysis tests preventing new internal types from adding tags
- [ ] add static analysis tests requiring json tags on external types (and enforcing lower-case first letter)
- [ ] add more tests for `kubectl get` exercising known and unknown types with all output options
Automatic merge from submit-queue (batch tested with PRs 37270, 38309, 37568, 34554)
kubeadm UX improvements for the v1.5 stable release
This PR targets the next stable kubeadm release.
It's work in progress, but please comment on it and review, since there are many changes.
I tried to group the commits logically, so you can review them separately.
Q: Why this large PR? Why not many small?
A: Because of the Submit Queue and the time it takes.
PTAL @kubernetes/sig-cluster-lifecycle
_Edit:_ This work was splitted up in three PRs in total
Automatic merge from submit-queue (batch tested with PRs 38354, 38371)
Add GetOptions parameter to Get() calls in client library
Ref #37473
This PR is super mechanical - the non trivial commits are:
- Update client generator
- Register GetOptions in batch/v2alpha1 group
Automatic merge from submit-queue
trimspace on user specified env params
**What this PR does / why we need it**:
before this PR, for example, if user `export KUBE_REPO_PREFIX=" "`, kubeadm will get an absolutely invalid repo_prefix.
this PR trims all leading and trailing white spaces from user specified env params.
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
Fix boilerplates, comments in the code and make the output of kubeadm more user-friendly
Start using HostPKIPath and KubernetesDir everywhere in the code, so they can be changed for real
More robust kubeadm reset code now.
Removed old glog-things from app.Run()
Renamed /etc/kubernetes/cloud-config.json to /etc/kubernetes/cloud-config since it shouldn't be a json file
Simplification of the code
Less verbose output from master/pki.go
Cleaned up dead code
Start a small logging/output framework:
- fmt.Println("[the-stage-here] Capital first letter of this message. Tell the user what the current state is")
- fmt.Printf("[the-stage-here] Capital first letter. Maybe a [%v] in the end if an error should be displayed. Always ends with \n")
- fmt.Errorf("Never starts with []. Includes a short error message plus the underlying error in [%v]. Never ends with \n")
Automatic merge from submit-queue (batch tested with PRs 38278, 37770)
Refactor REST storage to use generic defaults
This removes the repetition in the REST storage builders by moving the logic to `restoptions.ApplyOptions`. `registry.StorageWithCacher`/`generic.StorageDecorator` no longer assume that they can build the `keyFunc` for arbitrary objects. `restoptions.ApplyOptions` uses the `registry.Store`'s `KeyFunc` for its call to `generic.StorageDecorator`.
```release-note
Cluster federation servers have changed the location in etcd where federated services are stored, so existing federated services must be deleted and recreated. Before upgrading, export all federated services from the federation server and delete the services. After upgrading the cluster, recreate the federated services from the exported data.
```
We should expose the SkipUnsafe option, for legacy compatability, so
that conversion-go can be used in other projects, and for platforms
where unsafe is not available.
Make unsafe code generation the default though, and have the help text
hint that the resulting code is sub-optimal.
Automatic merge from submit-queue (batch tested with PRs 38432, 36887, 38415)
Update client status generator to not use json tags
fixes dependence on json tags in internal versions and drives the generation of UpdateStatus based on type comments
caught a type we were missing an UpdateStatus for
Automatic merge from submit-queue (batch tested with PRs 38432, 36887, 38415)
Add --image-pull-stuck-timeout option to kubelet
In this PR, add --image-pull-stuck-time option to specify the stuck timeout for pulling image.
When docker extracts image layer, there is no progress. The progress will exceed 1m if the layer is big or system is busy. It happend in our cluster, so I add above option to specify the timeout.
Related error log:
<pre>
[... kube_docker_client.go:29] Cancel pulling image "our_registry/demo/test" because of no progress for 1m0s, latest progress "c914ad57d670": Extracting [==================>] 513.5 MB/513.5MB"
[... docker_manager.go:2254] container start failed: ErrImagePull: net/http: request canceled
</pre>
Automatic merge from submit-queue (batch tested with PRs 36736, 35956, 35655, 37713, 38316)
Tool to generate yaml representation of the kubectl help text
**What this PR does / why we need it**:
It generates a yaml representation of the kubectl help so we can generate higher fidelity docs.
Automatic merge from submit-queue (batch tested with PRs 36310, 37349, 38319, 38402, 38338)
add summarizing discovery controller and handlers
Requires https://github.com/kubernetes/kubernetes/pull/38304 .
This adds discovery support to the new `kubernetes-discovery` that runs based on a controller wired up to the `APIService.apiregistration.k8s.io`. It also adds in plumbing for `local-up-cluster.sh` to register the "normal" kube resources.
@kubernetes/sig-api-machinery
@sttts
Automatic merge from submit-queue
kubedns: use initial resource listing as ready signal
Fix#35140.
Set up the ready signal after the first resource listing finished for both endpoints and services instead of listen on kubernetes service.
@bprashanth @bowei @thockin
**Release note**:
```
```
Automatic merge from submit-queue (batch tested with PRs 35939, 38381, 37825, 38306, 38110)
split easy controllers from giant start func
continues work from https://github.com/kubernetes/kubernetes/pull/37976 to separate controller initialization.
Automatic merge from submit-queue
type HttpProxyCheck should be HTTPProxyCheck
**What this PR does / why we need it**:
Change HttpProxyCheck to HTTPProxyCheck
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
golint
**Release note**:
```release-note
```
Signed-off-by: yupeng <yu.peng36@zte.com.cn>
Automatic merge from submit-queue (batch tested with PRs 38318, 38258)
kernel memcg notification enabled via experimental flag
Kubelet integrates with kernel memcg notification API if and only if enabled via experimental flag.
Automatic merge from submit-queue (batch tested with PRs 36543, 38189, 38289, 38291, 36724)
add authentication/authorization to kubernetes-discovery
Wires authentication and authorization into `kubernetes-discovery` and re-enables the `local-up-cluster.sh` along with proper permission granting for RBAC cases.
@sttts @liggitt
Automatic merge from submit-queue (batch tested with PRs 38294, 37009, 36778, 38130, 37835)
Add a cloudprovider validator flag to kubeadm and update the DNS spec
Broken out from: https://github.com/kubernetes/kubernetes/pull/37568
This PR creates a flag for `cloud-provider` that validates the value before `RunInit()` is run, which makes it now act as a "real" flag
Then it removes the `k8s.io/kubernetes/pkg/cloudprovider` dependency, which makes the binary ~40MB smaller! That's _really_ worth it!
In the second commit, the DNS addon is updated to the latest version: https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/skydns-rc.yaml.base
@kubernetes/sig-cluster-lifecycle
Automatic merge from submit-queue
add a configuration for kubelet to register as a node with taints
and deprecate --register-schedulable
ref #28687#29178
cc @dchen1107 @davidopp @roberthbailey
Automatic merge from submit-queue (batch tested with PRs 37032, 38119, 38186, 38200, 38139)
Detect long-running requests from parsed request info
Follow up to https://github.com/kubernetes/kubernetes/pull/36064
Uses parsed request info to more tightly match verbs and subresources
Removes regex-based long-running request path matching (which is easily fooled)
```release-note
The --long-running-request-regexp flag to kube-apiserver is deprecated and will be removed in a future release. Long-running requests are now detected based on specific verbs (watch, proxy) or subresources (proxy, portforward, log, exec, attach).
```
Automatic merge from submit-queue (batch tested with PRs 38194, 37594, 38123, 37831, 37084)
[kubeadm] Fail fast if there already is a node in the cluster with that name
kubeadm Fail fast if there is another node with the same name already in the cluster.
Fixes#36255
cc @kubernetes/sig-cluster-federation
Automatic merge from submit-queue (batch tested with PRs 38194, 37594, 38123, 37831, 37084)
remove unnecessary fields from genericapiserver config
Cleans up some unnecessary fields in the genericapiserver config.
Automatic merge from submit-queue (batch tested with PRs 36990, 37494, 38152, 37561, 38136)
join client CA bundles
Last commit grabs client CA bundles from disparate parts of the auth config and makes a pool that contains all of them.
I suspect a rebase broke this because of ordering. I'll keep these separate to make it easier for me to debug.
Automatic merge from submit-queue (batch tested with PRs 36990, 37494, 38152, 37561, 38136)
api federation types
First commit adds types that can back the kubernetes-discovery server with an `kubectl` compatible way of adding federated servers. Second commit is just generated code.
After we have types, I'd like to start splitting `kubernetes-discovery` into a "legacy" mode which will support what we have today and a "normal" mode which will provide an API federation server like this: https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/federated-api-servers.md that includes both discovery and proxy in a single server. Something like this: https://github.com/openshift/kube-aggregator .
@kubernetes/sig-api-machinery @nikhiljindal
Automatic merge from submit-queue (batch tested with PRs 38111, 38121)
remove rbac super user
Cleaning up cruft and duplicated capabilities as we transition from RBAC alpha to beta. In 1.5, we added a secured loopback connection based on the `system:masters` group name. `system:masters` have full power in the API, so the RBAC super user is superfluous.
The flag will stay in place so that the process can still launch, but it will be disconnected.
@kubernetes/sig-auth
Automatic merge from submit-queue (batch tested with PRs 36352, 36538, 37976, 36374)
demonstrate separation of controller intializers
Currently, controllers are all initialized in a monster method that make it difficult to individually pick out whether there are side-effects, difficult to group related controllers for selective enablement, and impossible to determine if there are hidden dependencies.
This pull demonstrates how we can break apart the monolith and start start the process of grouping and naming controllers for selective enablement. In addition, the use of a map will help expose dependency ordering amongst these controllers and the separate methods will make it a lot harder to have side effects.
This also moves us closer to being able to author reflective unit tests that help ensure that basic RBAC bootstrap roles are at least present, even if they aren't correct.
@nikhiljindal since you were looking at the federation controller manager
@sttts since we're looking at trying out RBAC on these.
Automatic merge from submit-queue (batch tested with PRs 37366, 36373)
kubeadm: Let apiserver and controller-manager host-mount /etc/pki when required
#<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
This PR checks if /etc/pki is present on the host machine and adds a host-mount to the apiserver and controller-manager manifest if required.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#36150
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
Fix incompatible host mounts for SSL certificates when deploying on CentOS with kubeadm
```
This removes all dependencies on Config during cert generation, only operating
on ServerRunOptions. This way we get rid of the repeated call of Config.Complete
and cleanly stratify the GenericApiServer bootstrapping.
Automatic merge from submit-queue (batch tested with PRs 35300, 36709, 37643, 37813, 37697)
Revert "[kubeadm] use iteration instead of recursion in function"
Reverts kubernetes/kubernetes#36625
Removing the recursive call means that `n` is never updated, so you never succeed in the update, and you've creating an infinite loop.
Also, this entire bit of functionality should be a patch and you won't have to worry about conflicts.
@luxas
Automatic merge from submit-queue (batch tested with PRs 35300, 36709, 37643, 37813, 37697)
Add generated informers
Add informer-gen and the informers it generates. We'll do follow-up PRs to convert everything currently using the hand-written informers to the generated ones.
TODO:
- [x] switch to `GroupVersionResource`
- [x] finish godoc
@deads2k @caesarxuchao @sttts @liggitt
Automatic merge from submit-queue (batch tested with PRs 37094, 37663, 37442, 37808, 37826)
fix if condition question in kubelet run() function
Here variable err returned by function NewForConfig(&eventClientConfig) if CreateAPIServerClientConfig() function runs correctly . And we should not print "invalid kubeconfig" info.
Should we use else instead of if.
Automatic merge from submit-queue
Remove ExportOptions from api/internal and use unversioned
Should only have one internal object in use
Part of #37530
Automatic merge from submit-queue (batch tested with PRs 37945, 37498, 37391, 37209, 37169)
Refactor certificate controller to make approval an interface
@mikedanese
Automatic merge from submit-queue (batch tested with PRs 37945, 37498, 37391, 37209, 37169)
Warn if firewalld service is enabled.
Fixes https://github.com/kubernetes/kubeadm/issues/21
Output will be:
```
(root@centos1 ~) $ kubeadm init
Running pre-flight checks
WARNING: firewalld is active, please ensure ports [6443 9898 10250] are open
```
I went with the port list from @errordeveloper 's ansible playbook here but it's possible there should be others listed.
Automatic merge from submit-queue (batch tested with PRs 36263, 36755, 37357, 37222, 37524)
kubeadm: Skip etcd related preflight checks and reset actions for external etcd
**What this PR does / why we need it**:
Skip etcd related preflight checks and reset actions for external etcd
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes # https://github.com/kubernetes/kubeadm/issues/69#issuecomment-262988388
**Special notes for your reviewer**:
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
kubeadm: Skip etcd related preflight checks and reset actions for external etcd
```
Automatic merge from submit-queue (batch tested with PRs 36263, 36755, 37357, 37222, 37524)
kubeadm: Implement support for symbolic labels in --use-kubernetes-version
**What this PR does / why we need it**: This patch adds "stable", "latest" and other labels as valid versions in "kubeadm init --use-kubernetes-version" flag.
Now, defaults can be pointing to "stable" and users will always get
latest available stable build of Kubernetes via kubeadm.
There is no need anymore to hardcode version string inside kubeadm
binary.
It is also possible to use labels like "latest" or point to exact
branch: "stable-1.4"
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
- kubeadm init --use-kubernetes-version now understands "stable","latest", "stable-1.4" and other labels as valid version. It will fetch actual build number from release servers.
```
Automatic merge from submit-queue
add failure check on umount when kubeadm reset, and on service stop
**What this PR does / why we need it**:
before this PR, `umount` will exit with code `123` if `grep` does not match anything
`xargs` has an option:
>-r, --no-run-if-empty
If the standard input does not contain any nonblanks, do not run the command. Normally, the command is run once even if
there is no input. This option is a GNU extension.
1. this PR add `-r` option to `xargs` , so `umount` will not execute and exit with code `0` correctly while `grep` does not match anything.
2. this PR add failure check on umount. for example, if the directory to be umount is busy, a error message will be printed:
>failed to unmount directories in /var/lib/kubelet, umount: /var/lib/kubelet/foo/bar: target is busy
(In some cases useful info about processes that
use the device is found by lsof(8) or fuser(1).)
3. add failure check on kubelet service stop.
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
Automatic merge from submit-queue
Kubeadm unit tests pkg node
Added unit tests for the kubeadm/app/node package testing functionality of bootstrap.go, csr.go, and discovery.go.
This PR is part of the ongoing effort to add tests (#35025)
/cc @pires @jbeda
Automatic merge from submit-queue
change unconfined_t to spc_t
**What this PR does / why we need it**:
When installing kube via kubeadm on a system w/ selinux enabled, it's necessary to disable selinux in order for the etcd and kube-discovery containers to run.
The kube etcd and discovery pods are currently set to unconfined_t in order to avoid disabling selinux, but the correct type for an unconfined container is spc_t. For more information, see http://danwalsh.livejournal.com/2016/10/03/.
updated pkg "k8s.io/kubernetes/pkg/client/clientset_generated/internalclientset" to "k8s.io/kubernetes/pkg/client/clientset_generated/release_1_5" because the type that bootstrap.go:126 checkAPIEndpoint requires changed as well as *internalversion.CertificatesClient in csr_test.go:69
Automatic merge from submit-queue
Add the system verification test to the kubeadm preflight checks
And refactor the system verification test to accept to write to a specific writer in order to customize the output
This PR is targeting v1.5, PTAL
cc @Random-Liu @dchen1107 @kubernetes/sig-cluster-lifecycle
Automatic merge from submit-queue
[kubeadm] use iteration instead of recursion in function
**What this PR does / why we need it**:
before this PR, function `attemptToUpdateMasterRoleLabelsAndTaints` recursively call itself. there are some defeats in it:
1. potential stack overflow.
2. unnecessary extra `json.Marshal` calls.
3. unnecessary extra `client.Nodes().List` calls.
this PR rewrite `attemptToUpdateMasterRoleLabelsAndTaints` function, use iterate instead of recursion.
so these 3 defeats the metioned above are gone.
Signed-off-by: bruceauyeung <ouyang.qinhua@zte.com.cn>
Automatic merge from submit-queue
kubeadm: changed tests to be table driven
Small change to migrate tests in kubeadm/app/util/tokens_test.go to be table driven which should make adding more tests in the future easier and also have them match how other tests are being written.
This PR is part of the ongoing effort to add tests (#35025)
Automatic merge from submit-queue
move parts of the mega generic run struct out
This splits the main `ServerRunOptions` into composeable pieces that are bindable separately and adds easy paths for composing servers to run delegating authentication and authorization.
@sttts @ncdc alright, I think this is as far as I need to go to make the composing servers reasonable to write. I'll try leaving it here
Automatic merge from submit-queue
controller manager refactors
The controller manager needs some significant cleanup. This starts us down the patch by respecting parameters like `stopCh`, simplifying discovery checks, removing unnecessary parameters, preventing unncessary fatals, and using our client builder.
@sttts @ncdc
here variable err returned by function NewForConfig(&eventClientConfig) if CreateAPIServerClientConfig() function is executed correctly. We should use else instead of if.
Or put block (if err != nil) to block (if err == nil) above
Automatic merge from submit-queue
support customize repository prefix of image through environment KUBE…
## Problem
kubeadm does not support customize repository prefix of image. this prevent us from using our own image repository to deploy k8s.
## Fix
make ```gcr.io/google_containers/ ``` be configurable.
ADD environment variable KUBE_REPO_PREFIX
Signed-off-by: yaoyao.xyy <yaoyao.xyy@alibaba-inc.com>
Hardcoded known stable version will be returned if user
didn't request specific version and kubeadm for some reason
not able to fetch latest stable information from release servers.
For now, fallback version is v1.4.6
Now, defaults can be pointing to "stable" and users will always get
latest available stable build of Kubernetes via kubeadm.
There is no need anymore to hardcode version string inside kubeadm
binary.
It is also possible to use labels like "latest" or point to exact
branch: "stable-1.4"
Automatic merge from submit-queue
Update kubeadm etcd to 3.0.13 in order to switch to the etcd3 storage format
ref: https://github.com/kubernetes/kubernetes/issues/35723
I think we should switch as soon as possible, but run it in etcd2 mode until the full etcd3 mode is stable
@kubernetes/sig-cluster-lifecycle @wojtek-t @xiang90 @lavalamp
In future we might try to verify ports are actually exposed in firewalld
policy, but this can be quite complex. Instead lets just warn the user
if we see firewalld is running.
The kube etcd and discovery pods are set to unconfined_t in
order to avoid disabling selinux, but the correct type for
an unconfined container is spc_t. For more information, see
http://danwalsh.livejournal.com/2016/10/03/.
Automatic merge from submit-queue
Check sysfs Type instead of Device
Some distribution uses "none" device to mount "sysfs" type, so kube-proxy needs to list mount points and check "types" instead of "device".
fixes#37183
```release-note
Change sysfs mountpoint tests from Device to Type to be able to find sysfs mountpoint on "none" device
```
Automatic merge from submit-queue
Fields with omitempty tag should still be considered as optional
We've added an "+optional" tag while ago for optional fields. Before that OpenAPI spec generated assumed all fields with "omitempty" in their json tags are optional. This should be still the case (as well as +optional tag) until these two things happen:
- We update all documentation asking developers to use +optional (My bad, I should have added this after the +optional PR)
- We fix swagger 1.2 spec generator to use +optional instead of omitempty.
Fixes#37149
Automatic merge from submit-queue
OpenAPI Bugfix: []byte should be treated as integer array
data field of v1.Secret is a map of string to byte array. Generated spec should generate a map of string to (type="string", format="byte" that means map of base64 string) however current code converts it to an array of integer that is wrong.
fixes#37126
- Adds command line flags --config-map, --config-map-ns.
- Fixes 36194 (https://github.com/kubernetes/kubernetes/issues/36194)
- Update kube-dns yamls
- Update bazel (hack/update-bazel.sh)
- Update known command line flags
- Temporarily reference new kube-dns image (this will be fixed with
a separate commit when the DNS image is created)
Automatic merge from submit-queue
[kubelet] rename --cgroups-per-qos to --experimental-cgroups-per-qos
This reflects the true nature of "cgroups per qos" feature.
```release-note
* Rename `--cgroups-per-qos` to `--experimental-cgroups-per-qos` in Kubelet
```
Automatic merge from submit-queue
Kubelet: Fix the description of MaxContainers kubelet flag.
Found this during code review.
The default number has been changed to `-1` and `1`. 82c488bd6e/pkg/apis/componentconfig/v1alpha1/defaults.go (L279-L285)
@yujuhong
/cc @saad-ali This PR fixed incorrect doc.
Automatic merge from submit-queue
Add Windows support to kube-proxy
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->
**What this PR does / why we need it**:
This is the first stab at supporting kube-proxy (userspace mode) on Windows
**Which issue this PR fixes** :
fixes#30278
**Special notes for your reviewer**:
The MVP uses `netsh portproxy` to redirect traffic from `ServiceIP:ServicePort` to a `LocalIP:LocalPort`.
For the next version we are expecting to have guidance from Microsoft Container Networking team.
**Limitations**:
Current implementation does not support DNS queries over UDP as `netsh portproxy` currently only supports TCP. We are working with Microsoft to remediate this.
cc: @brendandburns @dcbw
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
```
Automatic merge from submit-queue
Support persistent volume usage for kubernetes running on Photon Controller platform
**What this PR does / why we need it:**
Enable the persistent volume usage for kubernetes running on Photon platform.
Photon Controller: https://vmware.github.io/photon-controller/
_Only the first commit include the real code change.
The following commits are for third-party vendor dependency and auto-generated code/docs updating._
Two components are added:
pkg/cloudprovider/providers/photon: support Photon Controller as cloud provider
pkg/volume/photon_pd: support Photon persistent disk as volume source for persistent volume
Usage introduction:
a. Photon Controller is supported as cloud provider.
When choosing to use photon controller as a cloud provider, "--cloud-provider=photon --cloud-config=[path_to_config_file]" is required for kubelet/kube-controller-manager/kube-apiserver. The config file of Photon Controller should follow the following usage:
```
[Global]
target = http://[photon_controller_endpoint_IP]
ignoreCertificate = true
tenant = [tenant_name]
project = [project_name]
overrideIP = true
```
b. Photon persistent disk is supported as volume source/persistent volume source.
yaml usage:
```
volumes:
- name: photon-storage-1
photonPersistentDisk:
pdID: "643ed4e2-3fcc-482b-96d0-12ff6cab2a69"
```
pdID is the persistent disk ID from Photon Controller.
c. Enable Photon Controller as volume provisioner.
yaml usage:
```
kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
name: gold_sc
provisioner: kubernetes.io/photon-pd
parameters:
flavor: persistent-disk-gold
```
The flavor "persistent-disk-gold" needs to be created by Photon platform admin before hand.
Automatic merge from submit-queue
Make GCI nodes mount non tmpfs, ext* & bind mounts using an external mounter
This PR downloads the stage1 & gci-mounter ACIs as part of cluster bring up instead of downloading them dynamically from gcr.io, which was the cause for #36206.
I have also optimized the containerized mounter to pre-load the mounter image once to avoid fetch latency while using it.
Original PR which got reverted: https://github.com/kubernetes/kubernetes/pull/35821
```release-note
GCI nodes use an external mounter script to mount NFS & GlusterFS storage volumes
```
@mtaufen Node e2e is not re-enabled in this PR.
cc @jingxu97
