Commit Graph

236 Commits

Author SHA1 Message Date
Kubernetes Submit Queue
e2d6362c09
Merge pull request #67691 from loburm/security_fixes
Automatic merge from submit-queue (batch tested with PRs 67691, 68147). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

Bump versions of components with latest security patches.

**What this PR does / why we need it**:
Upgrade versions of monitoring components used on GCP, to include latest security patches.

**Release note**:
```release-note
[fluentd-gcp-scaler addon] Bump fluentd-gcp-scaler to 0.4 to pick up security fixes.
[prometheus-to-sd addon] Bump prometheus-to-sd to 0.3.1 to pick up security fixes, bug fixes and new features.
[event-exporter addon] Bump event-exporter to 0.2.3 to pick up security fixes.
```
2018-09-05 09:49:31 -07:00
Kubernetes Submit Queue
888546c325
Merge pull request #68029 from neolit123/fluentd-owners
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

cluster/addons: add labels to fluentd owner files

**What this PR does / why we need it**:
this PR adds SIG labels to fluentd OWNER files:
- cluster/addons/fluentd-elasticsearch/OWNERS
- cluster/addons/fluentd-gcp/OWNERS

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
let me know if the labels need adjustment.

**Release note**:

```release-note
NONE
```

/assign @roberthbailey @mikedanese 
/cc @timothysc 
/sig gcp
/sig instrumentation
/kind cleanup
2018-09-02 12:51:38 -07:00
Arnold Szederjesi
fcdef3ffcc Put fluentd back to host network 2018-08-30 10:44:04 +02:00
Lubomir I. Ivanov
aefb5b3c0e cluser/addons: add labels to fluentd owner files 2018-08-30 00:38:08 +03:00
Marian Lobur
ffa934a939 Bump versions of components with latest security patches. 2018-08-22 11:27:36 +02:00
liangwei
5ea138f4e9 remove rescheduler 2018-08-22 11:49:14 +08:00
Karol Wychowaniec
d5b32d8830 Fix parameter for fluentd-gcp-scaler 2018-08-16 16:18:51 +02:00
Bryan Moyles
32c2bfadfd A large set of improvements to the Stackdriver components.
Metadata Agent Improvements
Bump metadata agent version to 0.2-0.0.21-1.
Expand the metadata agent's access to all API groups.
Remove metadata agent config maps in favor of command line flags.
Update the metadata agent's liveness probe to a new /healthz handler.

Logging Agent Improvements
Bump logging agent version to 0.2-1.5.33-1-k8s-1.
Appropriately set log severity for k8s_container.
Fix detect exceptions plugin to analyze message field instead of log field.
Fix detect exceptions plugin to analyze streams based on local resource id.
Disable the metadata agent for monitored resource construction in logging.
Disable timestamp adjustment in logs to optimize performance.
Reduce logging agent buffer chunk limit to 512k to optimize performance.
2018-08-06 11:26:35 -04:00
Marian Lobur
3b8dfb38bb Bump version of event-exporter. 2018-07-13 13:20:58 +02:00
Robert Jacob
8f340c6c6a Use correct field for exception detection. 2018-06-22 12:58:41 +02:00
Daniel Kłobuszewski
7773f8f5eb Increase fluentd-gcp grace termination period to 1min
By default, all pods have 30s for graceful termination. This gives fluentd additional 30s to export logs when the node is shutting down.
2018-06-14 10:44:13 +02:00
RaviSantosh Gudimetla
872addf9e3
Revert "Remove rescheduler and corresponding tests from master" 2018-05-31 22:18:49 -04:00
ravisantoshgudimetla
aeccffc339 Phase out rescheduler in favor of priority and preemption 2018-05-29 19:52:06 -04:00
Kubernetes Submit Queue
f105ae3e6d
Merge pull request #63918 from cezarygerard/sd-event-exporter
Automatic merge from submit-queue (batch tested with PRs 63569, 63918, 63980, 63295, 63989). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

New event exporter config with support for new stackdriver resources

New event exporter, with support for use new and old stackdriver resource model.

This should also be cherry-picked to release-1.10 branch, as all  fluentd-gcp components support new and stackdriver resource model.

```release-note
Update event-exporter to version v0.2.0  that supports old (gke_container/gce_instance) and new (k8s_container/k8s_node/k8s_pod) stackdriver resources.
```
2018-05-18 09:54:16 -07:00
Cezary Zawadka
d611aeac80 new event exporter config with support for new stackdriver resource types 2018-05-18 10:37:47 +02:00
Zhen Wang
6351e25203 Use runtime/default as default seccomp profile for unprivileged PodSecurityPolicy 2018-05-15 09:39:37 -07:00
Kubernetes Submit Queue
b617748f7b
Merge pull request #62905 from serathius/event-exporter-region
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

[fluentd-gcp addon] Pass region in seperate field

This PR makes location passed to event-exporter based on `MULTIZONE` env.

Fixes https://github.com/kubernetes/kubernetes/issues/62399
```release-note
NONE
```
/cc @loburm
2018-05-11 06:00:44 -07:00
Marek Siarkowicz
f351b00a99 [fluentd-gcp addon] Pass region in seperate field 2018-05-11 09:50:07 +02:00
Bryan Moyles
a0a7686e38 Use the logging agent's node name as the metadata agent URL. 2018-05-02 10:12:35 +02:00
Slava Semushin
044bf2e415 Update addon manifests to use policy/v1beta1 and grant permissions in policy API group. 2018-04-17 14:56:55 +02:00
Bryan Moyles
19f14ad8e2 Increase CPU limit to 1000 millicores to support 100kb/s throughput. 2018-04-11 18:08:53 -04:00
Kubernetes Submit Queue
1efd5f9456
Merge pull request #62198 from thockin/gcr-vanity
Automatic merge from submit-queue (batch tested with PRs 61918, 62180, 62198). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Pass 2: k8s GCR vanity URL

Also push out the old URL deprecation since we have not started the community transition yet and there are some instances of it still floating about.

```release-note
NONE
```
2018-04-06 11:56:10 -07:00
Kubernetes Submit Queue
4009cb3b8b
Merge pull request #62076 from qingling128/master
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add support to ingest log entries to Stackdriver against new "k8s_container" and "k8s_node" resources.

**What this PR does / why we need it**:

**Which issue(s) this PR fixes**
Fluentd 0.14 has some memory leak issues that caused the e2e tests to be flaky. Downgrading to v0.12.

**Special notes for your reviewer**:
We never released any previous version with Fluentd v0.14. Only upgraded it very recently. So this downgrading is not visible to users.

**Release note**:
```release-note
Add support to ingest log entries to Stackdriver against new "k8s_container" and "k8s_node" resources.
```
2018-04-06 09:51:32 -07:00
Tim Hockin
89ceb7ef46 Pass 2: k8s GCR vanity URL 2018-04-06 08:14:58 -07:00
Ling Huang
cbec62ada4 Add support to ingest log entries to Stackdriver against new "k8s_container" and "k8s_node" resources. 2018-04-06 08:47:19 -04:00
Mikhail Vyatskov
300fe8f179 Remove crassirostris from owners and reviewers
Signed-off-by: Mikhail Vyatskov <crassirostris@yandex.com>
2018-04-04 18:36:44 +02:00
Mik Vyatskov
d6cef02a9d
Revert "Enable partial success in fluentd-gcp" 2018-03-29 11:48:01 +02:00
Kubernetes Submit Queue
70463ec4e2
Merge pull request #61773 from crassirostris/fluentd-partial-success
Automatic merge from submit-queue (batch tested with PRs 60465, 61773, 61371, 61146). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Enable partial success in fluentd-gcp

Enable partial success in fluentd-gcp. This will allow to reduce amount of lost data in case of invalid (e.g. too big) entries: instead of dropping the whole request, only failed entries will be dropped.

```release-note
[fluentd-gcp addon] Partial success option is enabled in fluentd.
```

/assign @x13n 
/cc @bmoyles0117
2018-03-28 01:34:48 -07:00
Kubernetes Submit Queue
943f8e8231
Merge pull request #60465 from frapposelli/GH-55416
Automatic merge from submit-queue (batch tested with PRs 60465, 61773, 61371, 61146). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Adding resource constraints for fluentd-gcp

**What this PR does / why we need it**:

Adds resource constraints to `fluentd-gcp`. Values mostly lifted from `fluentd-es`, cpu cap set to a sensible value after reviewing various threads.

**Which issue(s) this PR fixes**
Fixes #55416

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-03-28 01:34:45 -07:00
Kubernetes Submit Queue
cc859a8624
Merge pull request #61727 from crassirostris/update-event-exporter
Automatic merge from submit-queue (batch tested with PRs 61452, 61727, 61462, 61692, 61738). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update event-exporter image

This is a follow-up of https://github.com/GoogleCloudPlatform/k8s-stackdriver/pull/126 to apply the latest patch to the base image of event-exporter.

```release-note
[fluentd-gcp addon] Update event-exporter image to have the latest base image.
```

/assign @x13n 

Could you please take a look?
2018-03-27 09:47:11 -07:00
Mik Vyatskov
c8773044ea Enable partial success in fluentd-gcp
Signed-off-by: Mik Vyatskov <vmik@google.com>
2018-03-27 15:51:16 +02:00
Mik Vyatskov
61fee91c7a Update event-exporter image
Signed-off-by: Mik Vyatskov <vmik@google.com>
2018-03-26 17:20:42 +02:00
Shyam Jeedigunta
123fa5c706 Revert "Increase fluentd rolling-upgrade maxUnavailable to large value"
This reverts commit 7dd6adc438.
2018-03-26 15:17:54 +02:00
Kubernetes Submit Queue
446e45a071
Merge pull request #61472 from shyamjvs/disable-fluentd-scaler
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Increase fluentd rolling-upgrade maxUnavailable to large value

~For testing wrt https://github.com/kubernetes/kubernetes/issues/61190#issuecomment-374899752~
Fixes issue https://github.com/kubernetes/kubernetes/issues/61190 wrt slow rolling-upgrade

/cc @x13n @wojtek-t 
/sig instrumentation
/kind bug
/priority critical-urgent

```release-note
NONE
```
2018-03-25 10:40:14 -07:00
Kubernetes Submit Queue
053a12aee9
Merge pull request #60107 from wangzhen127/cos-audit-placeholder
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update GCP fluentd configmap for COS audit logging on GKE node

**What this PR does / why we need it**:
This PR adds a placeholder in fluentd configmap for COS audit logging on GKE node.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
NONE

**Release note**:

```release-note
NONE
```
2018-03-25 00:51:52 -07:00
Zhen Wang
d5c2cdcbbb Update GCP fluentd configmap for GKE node journal logging 2018-03-22 12:04:11 -07:00
Daniel Kłobuszewski
5503b3e94f
Add @x13n to fluentd-gcp OWNERS 2018-03-22 15:26:27 +01:00
Shyam Jeedigunta
7dd6adc438 Increase fluentd rolling-upgrade maxUnavailable to large value 2018-03-22 12:33:42 +01:00
Kubernetes Submit Queue
ca06cc43f7
Merge pull request #61269 from x13n/fluentd-gcp-scaler
Automatic merge from submit-queue (batch tested with PRs 60722, 61269). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bump fluentd-gcp-scaler version

**What this PR does / why we need it**:
This version fixes a bug in which scaler was setting resources for all containers in the pod, not only fluentd-gcp one.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #60763

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-03-16 04:22:59 -07:00
Kubernetes Submit Queue
7e063329f3
Merge pull request #60722 from filbranden/fluentd1
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove mapping to /host/lib from fluentd-gcp container.

**What this PR does / why we need it**:

This mapping is no longer needed since fluentd-gcp v2.0.16, in which it started using a container image based on Debian Stretch, in which the systemd libraries already include support for all the supported
compression algorithms.

The `/run.sh` in the image no longer accesses `/host/lib` anyways, so let's stop mapping it here.

Related changes:
- fluentd-gcp on GoogleCloudPlatform/k8s-stackdriver#101
- fluentd-es on GoogleCloudPlatform/google-fluentd#80

/assign @timstclair 
/cc @crassirostris @bmoyles0117 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
N/A

**Special notes for your reviewer**:
N/A

**Release note**:

```release-note
NONE
```
2018-03-16 03:38:28 -07:00
Daniel Kłobuszewski
e430520d1e Bump fluentd-gcp-scaler version
Fixes #60763

This version fixes a bug in which scaler was setting resources for all containers in the pod, not only fluentd-gcp one.
2018-03-16 10:11:56 +01:00
Daniel Kłobuszewski
3d9e9e9663 Bump fluentd-gcp-scaler version
Fixes #61190.

This version verifies on its own whether resources should be updated or not, instead of relying on `kubectl set resources`.
2018-03-15 15:15:26 +01:00
Kubernetes Submit Queue
f850b42474
Merge pull request #60951 from bmoyles0117/fix-fluentd-liveness-probe
Automatic merge from submit-queue (batch tested with PRs 60737, 60739, 61080, 60968, 60951). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Find most recent modified date for fluentd buffers recursively.

Fixes #60762

**What this PR does / why we need it**:
Due to updates in Fluent v0.14, the buffers directory modified date is no
longer updated when files inside the directory are changed. Therefore we
must find the most recent modified date recursively to fix liveness probe.

**Release note**:
```release-note
NONE
```
2018-03-13 12:27:07 -07:00
Bryan Moyles
a844523c20 Find most recent modified date for fluentd buffers recursively.
Due to updates in Fluent v0.14, the buffers directory modified date is no
longer updated when files inside the directory are changed. Therefore we
must find the most recent modified date recursively to fix liveness probe.
2018-03-12 15:28:55 -04:00
Bryan Moyles
c05504b736 Use grpc to improve the CPU utilization of the logging agent. 2018-03-09 10:09:30 -05:00
Fabio Rapposelli
8458e6fbfa
align cpu/mem for fluentd-gcp to fluentd-es plus cpu cap 2018-03-05 15:13:09 +01:00
Marek Siarkowicz
288dbd03e5 [fluentd-gcp addon] Fix passing location to event exporter 2018-03-05 15:05:35 +01:00
Filipe Brandenburger
cea4c98508 Remove mapping to /host/lib from fluentd-gcp container.
This mapping is no longer needed since fluentd-gcp v2.0.16, in which it
started using a container image based on Debian Stretch, in which the
systemd libraries already include support for all the supported
compression algorithms.

The /run.sh in the image no longer accesses /host/lib anyways, so let's
stop mapping it here.

Related changes:
- fluentd-gcp on GoogleCloudPlatform/k8s-stackdriver#101
- fluentd-es on GoogleCloudPlatform/google-fluentd#80
2018-03-02 10:20:08 -08:00
Bryan Moyles
84a86cffce Update to use Stackdriver Agent image.
Prometheus is enabled by default.
2018-02-26 14:05:33 -05:00
Kubernetes Submit Queue
a85f7d9fff
Merge pull request #58090 from serathius/pass-location-to-event-exporter
Automatic merge from submit-queue (batch tested with PRs 60054, 60202, 60219, 58090, 60275). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Pass location parameter to event exporter.

**What this PR does / why we need it**:
This PR makes event-exporter export cluster location together with events.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2018-02-23 23:15:43 -08:00