Added configurable etcd initial-cluster-state to kube-up script. This
allows creation of multi-master cluster from scratch. This is a
cherry-pick of #41320 from 1.5 branch.
Automatic merge from submit-queue (batch tested with PRs 41121, 40048, 40502, 41136, 40759)
Remove deprecated kubelet flags that look safe to remove
Removes:
```
--config
--auth-path
--resource-container
--system-container
```
which have all been marked deprecated since at least 1.4 and look safe to remove.
```release-note
The deprecated flags --config, --auth-path, --resource-container, and --system-container were removed.
```
Automatic merge from submit-queue (batch tested with PRs 40971, 41027, 40709, 40903, 39369)
Bump GCI to gci-beta-56-9000-80-0
cc/ @Random-Liu @adityakali
Changelogs since gci-dev-56-8977-0-0 (currently used in Kubernetes):
- "net.ipv4.conf.eth0.forwarding" and "net.ipv4.ip_forward" may get reset to 0
- Track CVE-2016-9962 in Docker in GCI
- Linux kernel CVE-2016-7097
- Linux kernel CVE-2015-8964
- Linux kernel CVE-2016-6828
- Linux kernel CVE-2016-7917
- Linux kernel CVE-2016-7042
- Linux kernel CVE-2016-9793
- Linux kernel CVE-2016-7039 and CVE-2016-8666
- Linux kernel CVE-2016-8655
- Toolbox: allow docker image to be loaded from local tarball
- Update compute-image-package in GCI
- Change the product name on /etc/os-release (to COS)
- Remove 'dogfood' from HWID_OVERRIDE in /etc/lsb-release
- Include Google NVME extensions to optimize LocalSSD performance.
- /proc/<pid>/io missing on GCI (enables process stats accounting)
- Enable BLK_DEV_THROTTLING
cc/ @roberthbailey @fabioy for GKE cluster update
Automatic merge from submit-queue (batch tested with PRs 38739, 40480, 40495, 40172, 40393)
Use existing ABAC policy file when upgrading GCE cluster
When upgrading, continue loading an existing ABAC policy file so that existing system components continue working as-is
```
When upgrading an existing 1.5 GCE cluster using `cluster/gce/upgrade.sh`, an existing ABAC policy file located at /etc/srv/kubernetes/abac-authz-policy.jsonl (the default location in 1.5) will enable the ABAC authorizer in addition to the RBAC authorizer. To switch an upgraded 1.5 cluster completely to RBAC, ensure the control plane components and your superuser have been granted sufficient RBAC permissions, move the legacy ABAC policy file to a backup location, and restart the apiserver.
```
Automatic merge from submit-queue (batch tested with PRs 39260, 40216, 40213, 40325, 40333)
Fixed propagation of kube master certs during master replication.
Fixed propagation of kube-master-certs during master replication.
Automatic merge from submit-queue (batch tested with PRs 40299, 40311)
cluster: update default rkt version to 1.23.0
This updates cluster configurations to current stable rkt version.
These files have been created lately, so we don't have much information
about them anyway, so let's just:
- Remove assignees and make them approvers
- Copy approves as reviewers
Automatic merge from submit-queue
Build release tars using bazel
**What this PR does / why we need it**: builds equivalents of the various kubernetes release tarballs, solely using bazel.
For example, you can now do
```console
$ make bazel-release
$ hack/e2e.go -v -up -test -down
```
**Special notes for your reviewer**: this is currently dependent on 3b29803eb5, which I have yet to turn into a pull request, since I'm still trying to figure out if this is the best approach.
Basically, the issue comes up with the way we generate the various server docker image tarfiles and load them on nodes:
* we `md5sum` the binary being encapsulated (e.g. kube-proxy) and save that to `$binary.docker_tag` in the server tarball
* we then build the docker image and tag using that md5sum (e.g. `gcr.io/google_containers/kube-proxy:$MD5SUM`)
* we `docker save` this image, which embeds the full tag in the `$binary.tar` file.
* on cluster startup, we `docker load` these tarballs, which are loaded with the tag that we'd created at build time. the nodes then use the `$binary.docker_tag` file to find the right image.
With the current bazel `docker_build` rule, the tag isn't saved in the docker image tar, so the node is unable to find the image after `docker load`ing it.
My changes to the rule save the tag in the docker image tar, though I don't know if there are subtle issues with it. (Maybe we want to only tag when `--stamp` is given?)
Also, the docker images produced by bazel have the timestamp set to the unix epoch, which is not great for debugging. Might be another thing to change with a `--stamp`.
Long story short, we probably need to follow up with bazel folks on the best way to solve this problem.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 36467, 36528, 39568, 40094, 39042)
Bump GCE to container-vm-v20170117
Base image update only, no kubelet or Docker updates.
```release-note
Update GCE ContainerVM deployment to container-vm-v20170117 to pick up CVE fixes in base image.
```
Automatic merge from submit-queue
Enable lazy initialization of ext3/ext4 filesystems
**What this PR does / why we need it**: It enables lazy inode table and journal initialization in ext3 and ext4.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#30752, fixes#30240
**Release note**:
```release-note
Enable lazy inode table and journal initialization for ext3 and ext4
```
**Special notes for your reviewer**:
This PR removes the extended options to mkfs.ext3/mkfs.ext4, so that the defaults (enabled) for lazy initialization are used.
These extended options come from a script that was historically located at */usr/share/google/safe_format_and_mount* and later ported to GO so this dependency to the script could be removed. After some search, I found the original script here: https://github.com/GoogleCloudPlatform/compute-image-packages/blob/legacy/google-startup-scripts/usr/share/google/safe_format_and_mount
Checking the history of this script, I found the commit [Disable lazy init of inode table and journal.](4d7346f7f5). This one introduces the extended flags with this description:
```
Now that discard with guaranteed zeroing is supported by PD,
initializing them is really fast and prevents perf from being affected
when the filesystem is first mounted.
```
The problem is, that this is not true for all cloud providers and all disk types, e.g. Azure and AWS. I only tested with magnetic disks on Azure and AWS, so maybe it's different for SSDs on these cloud providers. The result is that this performance optimization dramatically increases the time needed to format a disk in such cases.
When mkfs.ext4 is told to not lazily initialize the inode tables and the check for guaranteed zeroing on discard fails, it falls back to a very naive implementation that simply loops and writes zeroed buffers to the disk. Performance on this highly depends on free memory and also uses up all this free memory for write caching, reducing performance of everything else in the system.
As of https://github.com/kubernetes/kubernetes/issues/30752, there is also something inside kubelet that somehow degrades performance of all this. It's however not exactly known what it is but I'd assume it has something to do with cgroups throttling IO or memory.
I checked the kernel code for lazy inode table initialization. The nice thing is, that the kernel also does the guaranteed zeroing on discard check. If it is guaranteed, the kernel uses discard for the lazy initialization, which should finish in a just few seconds. If it is not guaranteed, it falls back to using *bio*s, which does not require the use of the write cache. The result is, that free memory is not required and not touched, thus performance is maxed and the system does not suffer.
As the original reason for disabling lazy init was a performance optimization and the kernel already does this optimization by default (and in a much better way), I'd suggest to completely remove these flags and rely on the kernel to do it in the best way.
Automatic merge from submit-queue (batch tested with PRs 39803, 39698, 39537, 39478)
include bootstrap admin in super-user group, ensure tokens file is correct on upgrades
Fixes https://github.com/kubernetes/kubernetes/issues/39532
Possible issues with cluster bring-up scripts:
- [x] known_tokens.csv and basic_auth.csv is not rewritten if the file already exists
* new users (like the controller manager) are not available on upgrade
* changed users (like the kubelet username change) are not reflected
* group additions (like the addition of admin to the superuser group) don't take effect on upgrade
* this PR updates the token and basicauth files line-by-line to preserve user additions, but also ensure new data is persisted
- [x] existing 1.5 clusters may depend on more permissive ABAC permissions (or customized ABAC policies). This PR adds an option to enable existing ABAC policy files for clusters that are upgrading
Follow-ups:
- [ ] both scripts are loading e2e role-bindings, which only be loaded in e2e tests, not in normal kube-up scenarios
- [ ] when upgrading, set the option to use existing ABAC policy files
- [ ] update bootstrap superuser client certs to add superuser group? ("We also have a certificate that "used to be" a super-user. On GCE, it has CN "kubecfg", on GKE it's "client"")
- [ ] define (but do not load by default) a relaxed set of RBAC roles/rolebindings matching legacy ABAC, and document how to load that for new clusters that do not want to isolate user permissions
This works around a flake I saw which had the same root cause as
https://github.com/coreos/rkt/issues/3513.
This will potentially help reduce the impact of such future problems as
well.
Automatic merge from submit-queue (batch tested with PRs 39694, 39383, 39651, 39691, 39497)
Bump container-linux and gci timeout for docker health check
The command `docker ps` can take longer time to respond under heavy load or
when encountering some known issues. In these cases, the containers are running
fine, so aggressive health check could cause serious disruption. Bump the
timeout to 60s to be consistent with the debian-based containerVM.
This addresses #38588
Automatic merge from submit-queue (batch tested with PRs 38212, 38792, 39641, 36390, 39005)
Generate a kubelet CA and kube-apiserver cert-pair for kubelet auth.
cc @cjcullen
The command `docker ps` can take longer time to respond under heavy load or
when encountering some known issues. In these cases, the containers are running
fine, so aggressive health check could cause serious disruption. Bump the
timeout to 60s to be consistent with the debian-based containerVM.
Automatic merge from submit-queue
cluster/cl: move abac to rbac
See #39092
We based off of GCI in the brief time where it was using abac.
fixes#39395
cc @yifan-gu
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue
Remove all MAINTAINER statements in the codebase as they are deprecated
**What this PR does / why we need it**:
ref: https://github.com/docker/docker/pull/25466
**Release note**:
```release-note
Remove all MAINTAINER statements in Dockerfiles in the codebase as they are deprecated by docker
```
@ixdy @thockin (who else should be notified?)
This update includes significant refactoring. It moves almost all of the
logic into bash scripts, modeled after the `gci` cluster scripts.
The primary differences between the two are the following:
1. Use of the `/opt/kubernetes` directory over `/home/kubernetes`
2. Support for rkt as a runtime
3. No use of logrotate
4. No use of `/etc/default/`
5. No logic related to noexec mounts or gci-specific firewall-stuff
Automatic merge from submit-queue (batch tested with PRs 38906, 38808)
change the version in the yaml file
change the version in heapster-controller.yaml with image version
Automatic merge from submit-queue
Use the cluster name in the names of the firewall rules that allow cluster-internal traffic to disambiguate the rules belonging to different clusters.
Also dropping the network name from these firewall rule names.
Network name was used to disambiguate firewall rules in a given network.
However, since two clusters cannot share a name in a GCE project, this
sufficiently disambiguates the firewall rule names. A potential confusion
arises when someone tries to create a firewall rule with the same name
in a different network, but that's also an indication that they shouldn't
be doing that.
@jszczepkowski due to PR #33094
@ixdy for test-infra
cc @kubernetes/sig-federation @nikhiljindal
Network name was used to disambiguate firewall rules in a given network.
However, since two clusters cannot share a name in a GCE project, this
sufficiently disambiguates the firewall rule names. A potential confusion
arises when someone tries to create a firewall rule with the same name
in a different network, but that's also an indication that they shouldn't
be doing that.
Automatic merge from submit-queue
Allow GCI_VERSION to come from env
This is to facilitate GCI tip vs. K8s tip testing; we need to
dynamically set the version of GCI to stay current with their
latest canary (latest of the "gci-base" prefixed images).
This is to facilitate GCI tip vs. K8s tip testing; we need to
dynamically set the version of GCI to stay current with their
latest canary (latest of the "gci-base" prefixed images).
Automatic merge from submit-queue
Keeps addon manager yamls in sync
From #38437.
We should have kept all addon manager YAML files in sync. This does not fix the release scripts issue, but we should still have this.
@mikedanese @ixdy
Automatic merge from submit-queue (batch tested with PRs 38058, 38523)
Renames kube-dns configure files from skydns* to kubedns*
`skydns-` prefix and `-rc` suffix are confusing and misleading. Renaming it to `kubedns` in existing yaml files and scripts.
@bowei @thockin
Automatic merge from submit-queue (batch tested with PRs 38294, 37009, 36778, 38130, 37835)
Only configure basic auth on gci if KUBE_USER and KUBE_PASSWORD are specified.
This should not change the existing flow when KUBE_USER/KUBE_PASSWORD are specified.
It makes not specifying those a valid option that means "don't turn on basic auth".
I only did it for cluster/gce/gci for now, but others should be somewhat similar.
Automatic merge from submit-queue
add a configuration for kubelet to register as a node with taints
and deprecate --register-schedulable
ref #28687#29178
cc @dchen1107 @davidopp @roberthbailey
Automatic merge from submit-queue (batch tested with PRs 38049, 37823, 38000, 36646)
Fixes kubedns logging level
We should have bumped up the verbose level to v=2 for `kubedns` after cutting the last release, as the TODO indicates.
@bowei @thockin
Automatic merge from submit-queue (batch tested with PRs 37692, 37785, 37647, 37941, 37856)
Use unified gcp fluentd image for gci and cvm
Follow-up of https://github.com/kubernetes/kubernetes/pull/37681
Actually unify the pod specs for CVM and GCI, to simplify the configuration
CC @piosz
Automatic merge from submit-queue
Exit with error if <version number or publication> is not the final parameter.
getopts stops parsing flags after a non-flag, non-arg-to-a-flag parameter.
This commit adds an error message if any parameters are passed after the
first non-flag, non-arg-to-a-flag parameter in the arg list.
getopts stops parsing flags after a non-flag, non-arg-to-a-flag parameter.
This commit adds an error message if any parameters are passed after the
first non-flag, non-arg-to-a-flag parameter in the arg list.
Automatic merge from submit-queue
Don't update gcloud in cluster/*/util.sh
**What this PR does / why we need it**:
Removes automatic gcloud update commands from `cluster/gce/util.sh`, `cluster/gke/util.sh`. Setting env `KUBE_PROMPT_FOR_UPDATE=y` will update required components, otherwise it will only verify that required components are present and at a minimum required version.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#35834
**Special notes for your reviewer**:
Inline python is nasty but I *really* don't want to do version comparison in bash. Open to other suggestions for verifying required version of gcloud components. cc @kubernetes/sig-cluster-lifecycle, @kubernetes/sig-testing
**Release note**:
<!-- Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access)
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`.
-->
```release-note
`kube-up.sh`/`kube-down.sh` no longer force update gcloud for provider=gce|gke.
```
Automatic merge from submit-queue
Set strategy spec for kube-dns to support zero downtime rolling update
From #37728 and coreos/kube-aws#111.
Set `maxUnavailable` to 0 to prevent DNS service outage during update when the replica number is only 1.
Also keeps all kube-dns yaml files in sync.
@bowei @thockin
Automatic merge from submit-queue
Fix the equality checks for numeric values in cluster/gce/util.sh.
**What this PR does / why we need it**: This PR fixes an error in the gce shell scripts that results in inconsistent/incorrect behavior.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#37385
**Special notes for your reviewer**: This needs to be backported to 1.5 and 1.4.
@jszczepkowski
Automatic merge from submit-queue
Set Dashboard UI version to v1.5.0-beta1
There will be one more such PR coming for 1.5 release. In one week.
Setting release note to none. Will set notes for final version PR.
Github release info:
https://github.com/kubernetes/dashboard/releases/tag/v1.5.0-beta1
Automatic merge from submit-queue
Deploy a default StorageClass instance on AWS and GCE
This needs a newer kubectl in kube-addons-manager container. It's quite tricky to test as I cannot push new container image to gcr.io and I must copy the newer container manually.
cc @kubernetes/sig-storage
**Release note**:
```release-note
Kubernetes now installs a default StorageClass object when deployed on AWS, GCE and
OpenStack with kube-up.sh scripts. This StorageClass will automatically provision
a PeristentVolume in corresponding cloud for a PersistentVolumeClaim that cannot be
satisfied by any existing matching PersistentVolume in Kubernetes.
To override this default provisioning, administrators must manually delete this default StorageClass.
```
