- Move from the old github.com/golang/glog to k8s.io/klog
- klog as explicit InitFlags() so we add them as necessary
- we update the other repositories that we vendor that made a similar
change from glog to klog
* github.com/kubernetes/repo-infra
* k8s.io/gengo/
* k8s.io/kube-openapi/
* github.com/google/cadvisor
- Entirely remove all references to glog
- Fix some tests by explicit InitFlags in their init() methods
Change-Id: I92db545ff36fcec83afe98f550c9e630098b3135
Automatic merge from submit-queue (batch tested with PRs 67493, 67617, 67582, 67337). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
updates kibana to 6.3.2
**What this PR does / why we need it**:
* updates kibana to 6.3.2
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
needed as es was updated to 6.3.2 too in https://github.com/kubernetes/kubernetes/pull/67484
**Release note**:
```release-note
updates kibana to 6.3.2
```
Automatic merge from submit-queue (batch tested with PRs 64142, 64426, 62910, 63942, 64548). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update fluentd-elasticsearch addon
**What this PR does / why we need it**:
Elastic now provides a fully opensource version for their prebuild
docker images (elasticsearch, kibana and so on). To avoid running into
licensing conflicts for this addon example, we should rather use these
images instead of the premium ones (were we also have to disable premium
features manually right now).
This PR updates:
- fluentd from <=1.1.0 to ~>1.1.3
- elasticsearch and kibana from 5.6.2 to 6.2.4
- fluentd-elasticsearch-plugin from 2.4.1 to 2.9.1
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
https://github.com/fluent/fluentd/issues/1894
**Special notes for your reviewer**:
**Release note**:
```release-note
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove rescheduler and corresponding tests from master
**What this PR does / why we need it**:
This is to remove rescheduler from master branch as we are promoting priority and preemption to beta.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Part of #57471
**Special notes for your reviewer**:
/cc @bsalamat @aveshagarwal
**Release note**:
```release-note
Remove rescheduler from master.
```
Automatic merge from submit-queue (batch tested with PRs 57954, 62166). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fluentd-elasticsearch addon: allow graceful shutdown in fluentd-es image.
This PR contains an optimisation to the fluentd-elasticsearch addon. Restarting fluentd pods took unnecessarily long because the start script in the image did not support propagating signals to the actual fluentd process. This patch fixes this behaviour.
```release-note
NONE
```
* elastic now provides a fully opensource version for their prebuild
docker images (elasticsearch, kibana and so on). To avoid running into
licensing conflicts for this addon example, we should rather use these
images instead of the premium ones (were we also have to disable premium
features manually right now)
* remove disable flags for xpack, since *-oss images do not include this anymore
* bump elasticsearch and kibana version from 5.6.4 to 6.2.4
* use oss version from elastic as baseimg for kibana and elasticsearch
* bump fluentd version to ~>1.1.3
* bump gem 'fluent-plugin-elasticsearch' to '~>2.9.1' to allow usage of elasticsearch 6.x
* bump fluentd-es-image to v2.1.0
* fix elasticserach run.sh to align with new elasticsearch upstream container structure
I find it useful to have the kernel logs available for searching, for example that's the only place you can see processes killed when they hit memory limits.
This mapping is no longer needed since fluentd-gcp v2.0.16, in which it
started using a container image based on Debian Stretch, in which the
systemd libraries already include support for all the supported
compression algorithms.
The /run.sh in the image no longer accesses /host/lib anyways, so let's
stop mapping it here.
Related changes:
- fluentd-gcp on GoogleCloudPlatform/k8s-stackdriver#101
- fluentd-es on GoogleCloudPlatform/google-fluentd#80
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Upload container runtime log to sd/es.
I've verified this in my environment. My stackdriver has an extra `container-runtime` entry for node log, and it collects container runtime daemon log correctly.
@yujuhong @feiskyer @crassirostris @piosz
@kubernetes/sig-node-pr-reviews @kubernetes/sig-instrumentation-pr-reviews
Signed-off-by: Lantao Liu <lantaol@google.com>
**Release note**:
```release-note
Container runtime daemon (e.g. dockerd) logs in GCE cluster will be uploaded to stackdriver and elasticsearch with tag `container-runtime`
```
Automatic merge from submit-queue (batch tested with PRs 59767, 56454, 59237, 59730, 55479). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Change critical pods’ template to use priority
**What this PR does / why we need it**:
Change critical pods’ template to use priority
Thanks.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
ref #57471
**Special notes for your reviewer**:
**Release note**:
```release-note
```
This is the 2nd attempt. The previous was reverted while we figured out
the regional mirrors (oops).
New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest. To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today). For now the staging is an alias to
gcr.io/google_containers (the legacy URL).
When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.
We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it. Nice and
visible, easy to keep track of.
Automatic merge from submit-queue (batch tested with PRs 58263, 58425, 58419). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
apps api is now stable, use it
**What this PR does / why we need it**: as of kubernetes 1.9, the apps api is stable. we should use v1 instead of v1beta2
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes # none
**Special notes for your reviewer**:
**Release note**:
```
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55998, 55400). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update of elasticsearch kibana version
**What this PR does / why we need it**:
Updated elasticsearch and kibana version to version 5.6.4
This was motivated by @crassirostris in https://github.com/kubernetes/kubernetes/pull/54215#issuecomment-343003874
**Release note**:
```release-note
[fluentd-elasticsearch addon] Elasticsearch and Kibana are updated to version 5.6.4
```
Automatic merge from submit-queue (batch tested with PRs 54602, 54877, 55243, 55509, 55128). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
PodSecurityPolicies for addons
**What this PR does / why we need it**:
1. Colocate addon PodSecurityPolicy config with the addons (in a `podsecuritypolicies` subdirectory).
2. Add policies for addons that are currently missing policies (not in the default GCE suite)
3. Remove HostPath SSL certs from several heapster deployments, so that heapster doesn't require a special PSP
**Which issue(s) this PR fixes**:
#43538
**Release note**:
```release-note
- Add PodSecurityPolicies for cluster addons
- Remove SSL cert HostPath volumes from heapster addons
```
Automatic merge from submit-queue (batch tested with PRs 54460, 55258, 54858, 55506, 55510). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix CRI fluentd config.
This should fix the cri-containerd stackdriver test failure:
```
Cluster level logging implemented by Stackdriver should ingest logs
```
I copied the pattern from a comment previously. However, it doesn't actually work properly. `\b` only matches word boundary, and seems to match the boundary of previous word in our case.
That's why we get the log with a leading space:
```
Nov 10 18:39:11.661: INFO: Unexpected error occurred: log entry ingested incorrectly, got --> <--I0101 00:00:00.000000 1 main.go:1] Text, want Text
```
@kubernetes/sig-node-bugs @kubernetes/sig-instrumentation-bugs
Signed-off-by: Lantao Liu <lantaol@google.com>
```release-note
none
```
Automatic merge from submit-queue (batch tested with PRs 54987, 55221, 54099, 55144, 54215). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
extracted elasticsearch-logging service name as environment variable
**What this PR does / why we need it**:
Deploying the cluster-addon fluentd-elasticsearch with customized resource definitions can cause elasticsearch discovery to fail because the service name `elasticsearch-logging` is hard-coded in cluster/addons/fluentd-elasticsearch/es-image/elasticsearch_logging_discovery.go
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
-> none yet
**Special notes for your reviewer**:
The name of the environment variable is ELASTICSEARCH_SERVICE_NAME. When non is given the fallback service-name fallback is `elasticsearch-logging`
```release-note
[fluentd-elasticsearch addon] Elasticsearch service name can be overridden via env variable ELASTICSEARCH_SERVICE_NAME
```
Automatic merge from submit-queue (batch tested with PRs 48532, 50054, 50082)
Refactored the fluentd-es addon
Refactor fluentd-elasticsearch addon:
- Decrease the number of files by moving RBAC-related objects in the same files where they're used
- Move the fluentd configuration out of the image
- Don't use PVC to avoid leaking resources in e2e tests
- Fluentd now ingest docker and kubelet logs that are written to journald
- Disable X-Pack, because it's not free
Fixes https://github.com/kubernetes/kubernetes/issues/41462
Fixes https://github.com/kubernetes/kubernetes/issues/49816
Fixes https://github.com/kubernetes/kubernetes/issues/48973
Fixes https://github.com/kubernetes/kubernetes/issues/49450
@aknuds1 @coffeepac Could you please take a look?
```release-note
Fluentd DaemonSet in the fluentd-elasticsearch addon is configured via ConfigMap and includes journald plugin
Elasticsearch StatefulSet in the fluentd-elasticsearch addon uses local storage instead of PVC by default
```
Automatic merge from submit-queue (batch tested with PRs 50103, 49677, 49449, 43586, 48969)
Run kazel on the entire tree
**What this PR does / why we need it**: part of #47558: auto-generate `BUILD` files on the entire tree, since this is what `gazelle` does, and it'll make subsequent reviews easier if less is changing.
**Release note**:
```release-note
NONE
```
/assign
/release-note-none
Automatic merge from submit-queue (batch tested with PRs 48365, 49902, 49808, 48722, 47045)
Upgrade fluentd-elasticsearch addon to Elasticsearch/Kibana 5.5
This is a patch to upgrade the fluentd-elasticsearch addon to Elasticsearch/Kibana 5.5. Please provide feedback!
```release-notes
* Upgrade Elasticsearch/Kibana to 5.5.1 in fluentd-elasticsearch addon
* Switch to basing our image of Elasticsearch in fluentd-elasticsearch addon off the official one
* Switch to the official image of Kibana in fluentd-elasticsearch addon
* Use StatefulSet for Elasticsearch instead of ReplicationController, with persistent volume claims
* Require authenticating towards Elasticsearch, as Elasticsearch 5.5 by default requires basic authentication
```
Automatic merge from submit-queue (batch tested with PRs 47860, 47170)
Make fluentd log to stdio instead of a dedicated file
Lower verbosity also, to reduce volume of system logs exported to the backend.
Fix https://github.com/kubernetes/kubernetes/issues/43772
/cc @piosz
Automatic merge from submit-queue (batch tested with PRs 46151, 47602, 47507, 46203, 47471)
Add RBAC support to fluentd-elasticsearch cluster addon
**What this PR does / why we need it**:
Adds rbac support to the fluentd-elasticsearch addon
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#46023
**Special notes for your reviewer**:
**Release note**:
```release-note
Add RBAC support to fluentd-elasticsearch cluster addon
```
Automatic merge from submit-queue (batch tested with PRs 46151, 47602, 47507, 46203, 47471)
es discovery support args apiserver-host and kubeconfig
Now discovery elasticsearch through kubernetes client,but now does not support specifying the apiserver-host or kubeconfig create client.
Automatic merge from submit-queue (batch tested with PRs 41830, 42630)
Arrange for elasticsearch to shutdown cleanly
Kubernetes initiates "graceful shutdown" by sending SIGTERM to pid 1, which
is exactly what elasticsearch is expecting (good!)
The way the existing startup scripts worked however, this signal arrived at
the shell wrapper, not elasticsearch, and the shell wrapper exited,
killing the container immediately (bad!)
Before this change:
```
1 ? Ss 0:00 /bin/sh -c /run.sh
6 ? S 0:00 /bin/bash /run.sh
13 ? S 0:00 \_ /bin/su -c /elasticsearch/bin/elasticsearch elasticsearch
14 ? Ss 0:00 \_ sh -c /elasticsearch/bin/elasticsearch
15 ? Sl 19:18 \_ /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java ... org.elasticsearch.bootstrap.Elasticsearch start
```
After this change:
```
1 ? Ssl 0:29 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java ... org.elasticsearch.bootstrap.Elasticsearch start
```
Kubernetes initiates "graceful shutdown" by sending SIGTERM to pid 1.
The way the existing startup scripts worked, this signal arrived at
the shell wrapper, not elasticsearch, and the shell wrapper exited,
killing the container immediately.
Before this change:
1 ? Ss 0:00 /bin/sh -c /run.sh
6 ? S 0:00 /bin/bash /run.sh
13 ? S 0:00 \_ /bin/su -c /elasticsearch/bin/elasticsearch elasticsearch
14 ? Ss 0:00 \_ sh -c /elasticsearch/bin/elasticsearch
15 ? Sl 19:18 \_ /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java ... org.elasticsearch.bootstrap.Elasticsearch start
After this change:
1 ? Ssl 0:29 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java ... org.elasticsearch.bootstrap.Elasticsearch start
