Commit Graph

1241 Commits

Author SHA1 Message Date
Lantao Liu
faa581c5cb Add node e2e test for log rotation. 2018-02-23 01:42:35 +00:00
Lantao Liu
313e8717f6 Generated code 2018-02-23 01:42:35 +00:00
Kubernetes Submit Queue
270148d7d9 Merge pull request #58684 from hzxuzhonghu/default-enabled-admission
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

set default enabled admission plugins by official document

**What this PR does / why we need it**:

https://kubernetes.io/docs/admin/admission-controllers/#is-there-a-recommended-set-of-admission-controllers-to-use

recommend  running the following set of admission controllers 
```
If you previously had not set the `--admission-control` flag, your cluster behavior may change (to be more standard).  See [https://kubernetes.io/docs/admin/admission-controllers/] for explanation of admission control.
```

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
Set default enabled admission plugins `NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota`
```
2018-02-22 05:24:44 -08:00
Kubernetes Submit Queue
714b19ee75 Merge pull request #57583 from MorrisLaw/bugfix/logf-newline
Automatic merge from submit-queue (batch tested with PRs 60158, 60156, 58111, 57583, 60055). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bugfix/logf newline

**What this PR does / why we need it**:
Removes all redundant new lines being passed into the `Logf()` function. This involved going through code in both `test/e2e` and `test/e2e_node`, finding the newline redundancies in calls to `Logf()` and removing them.

**Which issue(s) this PR fixes**:
Fixes [#57102](https://github.com/kubernetes/kubernetes/issues/57102)

**Release note**:

```release-note
NONE
```
2018-02-21 22:10:34 -08:00
hzxuzhonghu
27f3fd2d79 set default enabled admission plugins by official document 2018-02-22 11:02:02 +08:00
Kubernetes Submit Queue
687c651dfd Merge pull request #59884 from mikedanese/remove-deprecated-proxy
Automatic merge from submit-queue (batch tested with PRs 58716, 59977, 59316, 59884, 60117). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

remove deprecated /proxy paths

These were deprecated in v1.2.
ref https://github.com/kubernetes/kubernetes/issues/59885
```release-note
kube-apiserver: the root /proxy paths have been removed (deprecated since v1.2). Use the /proxy subresources on objects that support HTTP proxying.
```

@kubernetes/sig-api-machinery-api-reviews
2018-02-21 15:40:45 -08:00
vikaschoudhary16
e64517cd74 Migrate deviceplugin api from v1alpha to v1beta1 2018-02-21 01:26:20 -05:00
vikaschoudhary16
defcab81d5 Invoke PreStart RPC call before container start, if desired by plugin
Signed-off-by: vikaschoudhary16 <vichoudh@redhat.com>
2018-02-21 01:25:24 -05:00
Mike Danese
7b4722964d remove deprecated /proxy paths
These were depercated in v1.2.
2018-02-20 14:42:19 -08:00
Kubernetes Submit Queue
96ec318718 Merge pull request #59842 from ixdy/update-rules_go-02-2018
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

 Update bazelbuild/rules_go, kubernetes/repo-infra, and gazelle dependencies

**What this PR does / why we need it**: updates our bazelbuild/rules_go dependency in order to bump everything to go1.9.4. I'm separating this effort into two separate PRs, since updating rules_go requires a large cleanup, removing an attribute from most build rules.

**Release note**:

```release-note
NONE
```
2018-02-19 22:23:05 -08:00
Jeremy L. Morris
e724886ad5 Removed newlines from e2e log statements. 2018-02-17 22:25:38 -05:00
David Ashpole
960856f4e8 collect metrics on the /kubepods cgroup on-demand 2018-02-17 12:32:40 -08:00
Kubernetes Submit Queue
1e5a58416b Merge pull request #59989 from mtaufen/fix-e2e-node-tests
Automatic merge from submit-queue (batch tested with PRs 59927, 59989, 59950). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix e2e node setKubeletConfiguration helper

The helper should have been using `apiequality.Semantic.DeepEqual`,
instead of `reflect.DeepEqual`. Previously, nil vs empty containers
were treated as not equal, but they should be considered equal for
objects managed by Kubernetes API machinery, like KubeletConfiguration.

This should fix the failing eviction tests.

```release-note
NONE
```
2018-02-16 17:42:33 -08:00
Kubernetes Submit Queue
270ed995f4 Merge pull request #59841 from dashpole/metrics_after_reclaim
Automatic merge from submit-queue (batch tested with PRs 59683, 59964, 59841, 59936, 59686). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Reevaluate eviction thresholds after reclaim functions

**What this PR does / why we need it**:
When the node comes under `DiskPressure` due to inodes or disk space, the eviction manager runs garbage collection functions to clean up dead containers and unused images.
Currently, we use the strategy of trying to measure the disk space and inodes freed by garbage collection.  However, as #46789 and #56573 point out, there are gaps in the implementation that can cause extra evictions even when they are not required.  Furthermore, for nodes which frequently cycle through images, it results in a large number of evictions, as running out of inodes always causes an eviction.

This PR changes this strategy to call the garbage collection functions and ignore the results.  Then, it triggers another collection of node-level metrics, and sees if the node is still under DiskPressure.
This way, we can simply observe the decrease in disk or inode usage, rather than trying to measure how much is freed.

**Which issue(s) this PR fixes**:
Fixes #46789
Fixes #56573
Related PR #56575

**Special notes for your reviewer**:
This will look cleaner after #57802  removes arguments from [makeSignalObservations](https://github.com/kubernetes/kubernetes/pull/57802/files#diff-9e5246d8c78d50ce4ba440f98663f3e9R719).

**Release note**:
```release-note
NONE
```

/sig node
/kind bug
/priority important-soon
cc @kubernetes/sig-node-pr-reviews
2018-02-16 16:31:33 -08:00
Michael Taufen
26cc4ff55c Fix e2e node setKubeletConfiguration helper
The helper should have been using `apiequality.Semantic.DeepEqual`,
instead of `reflect.DeepEqual`. Previously, nil vs empty containers
were treated as not equal, but they should be considered equal for
objects managed by Kubernetes API machinery, like KubeletConfiguration.

This should fix the failing eviction tests.
2018-02-16 14:53:27 -08:00
Jeff Grafton
ef56a8d6bb Autogenerated: hack/update-bazel.sh 2018-02-16 13:43:01 -08:00
Kubernetes Submit Queue
eac5bc0035 Merge pull request #57136 from k82cn/k8s_54313
Automatic merge from submit-queue (batch tested with PRs 57136, 59920). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Updated PID pressure node condition.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
part of #54313 

**Release note**:

```release-note
Updated PID pressure node condition
```
2018-02-16 10:35:33 -08:00
David Ashpole
e0830d0b71 reevaluate eviction thresholds after reclaim functions 2018-02-16 08:35:24 -08:00
Kubernetes Submit Queue
c03edcc58e Merge pull request #53833 from mtaufen/kubeletconfig-to-beta
Automatic merge from submit-queue (batch tested with PRs 59353, 59905, 53833). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Graduate kubeletconfig API group to beta

Regarding https://github.com/kubernetes/features/issues/281, this PR moves the kubeletconfig API group to beta. 

After #53088, the KubeletConfiguration type should not contain any deprecated or experimental fields, and we should not have to remove any more fields from the type before graduating it to beta. 

We need the community to double check for two things, however:
1. Are there any fields currently in the KubeletConfiguration type that you were going to mark deprecated this quarter, but haven't yet?
2. Are there any fields currently in the KubeletConfiguration type that are experimental or alpha, but were not explicitly denoted as such?

Please comment on this PR if you can answer "yes" to either of those two questions. Please cc anyone with a stake in the kubeletconfig API, so we get as much coverage as possible.

/cc @thockin @dchen1107 @Random-Liu @yujuhong @dashpole @tallclair @vishh @abw @freehan @dnardo @bowei @MrHohn @luxas @liggitt @ncdc @derekwaynecarr @mikedanese 

@kubernetes/sig-network-pr-reviews, @kubernetes/sig-node-pr-reviews 

```release-note
action required: The `kubeletconfig` API group has graduated from alpha to beta, and the name has changed to `kubelet.config.k8s.io`. Please use `kubelet.config.k8s.io/v1beta1`, as `kubeletconfig/v1alpha1` is no longer available. 
```

**TODO:**
- [x] Move experimental/non-gated-alpha/soon-to-be-deprecated fields to `KubeletFlags`
  - [x] #53088
  - [x] #54154
  - [x] #54160
  - [x] #55562
  - [x] #55983
  - [x] #57851
- [x] Lift embedded structure out of strings
  - [x] #53025
  - [x] #54643
  - [x] #54823
  - [x] #55254
- [x] Resolve relative paths against the location config files are loaded from
  - [x] #55648 
- [x] Rename to `kubelet.config.k8s.io`
- [x] Comments
  - [x] Make sure existing comments at least read sensibly.
  - [x] Note default values in comments on the versioned struct.
  - [x] Remove any reference to default values in comments on the internal struct.
- [x] Most fields should be `+optional` and `omitempty`. Add where necessary. ~Where omitted, explicitly comment.~ Edit: We should not distinguish between nil and empty, see below items.
- [x] Ensure defaults are specified via `pkg/kubelet/apis/kubelet.config.k8s.io/v1beta1/defaults.go`, not `cmd/kubelet/app/options/options.go`.
  - [x] #57770
- [x] Ensure kubeadm does not persist v1alpha1 KubeletConfiguration objects (or feature-gates this functionality)
- [x] Don't make a distinction between empty and nil, because of #43203.
  - [x] #59515
  - [x] #59681
- [x] Take the opportunity to fix insecure Kubelet defaults @tallclair 
  - [x] #59666
- [x] Remove CAdvisorPort from KubeletConfiguration wrt #56523.
  - [x] #59580
- [x] Hide `ConfigTrialDuration` until we're more sure what to do with it.
   - [x] #59628
- [x] Fix `// default: x` comments after rebasing on recent changes.
2018-02-15 11:06:40 -08:00
Michael Taufen
d8cc440dd6 Rename ConfigOK to KubeletConfigOk
This is a more accurate name for the condition, as it describes the
status of the Kubelet's configuration.

Also cleans up capitalization of internal names.
2018-02-14 19:36:52 -08:00
Michael Taufen
9ebaf5e7d2 Move the kubeletconfig v1alpha1 API to beta, rename to kubelet.config.k8s.io 2018-02-14 17:30:22 -08:00
Kubernetes Submit Queue
24be88a8c5 Merge pull request #59511 from derekwaynecarr/hugepage-e2e
Automatic merge from submit-queue (batch tested with PRs 59653, 58812, 59582, 59665, 59511). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add node e2e tests to verify HugePages feature

**What this PR does / why we need it**:
Add node e2e tests to verify HugePages feature.

**Special notes for your reviewer**:
Test follows same pattern as pod container manager tests.

**Release note**:
```release-note
NONE
```
2018-02-13 11:12:54 -08:00
Kubernetes Submit Queue
9de5839944 Merge pull request #59681 from mtaufen/kc-empty-eviction-hard
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Ignore 0% and 100% eviction thresholds

Primarily, this gives a way to explicitly disable eviction, which is
necessary to use omitempty on EvictionHard.
See: https://github.com/kubernetes/kubernetes/pull/53833#discussion_r166672137

As justification for this approach, neither 0% nor 100% make sense as
eviction thresholds; in the "less-than" case, you can't have less than
0% of a resource and 100% perpetually evicts; in the
"greater-than" case (assuming we ever add a resource with this
semantic), the reasoning is the reverse (not more than 100%, 0%
perpetually evicts).

```release-note
Eviction thresholds set to 0% or 100% are now ignored.
```
2018-02-13 09:48:11 -08:00
Michael Taufen
21dbbe14f2 Ignore 0% and 100% eviction thresholds
Primarily, this gives a way to explicitly disable eviction, which is
necessary to use omitempty on EvictionHard.
See: https://github.com/kubernetes/kubernetes/pull/53833#discussion_r166672137

As justification for this approach, neither 0% nor 100% make sense as
eviction thresholds; in the "less-than" case, you can't have less than
0% of a resource and 100% perpetually evicts; in the
"greater-than" case (assuming we ever add a resource with this
semantic), the reasoning is the reverse (not more than 100%, 0%
perpetually evicts).
2018-02-12 14:13:00 -08:00
Kubernetes Submit Queue
317853c90c Merge pull request #59464 from dixudx/fix_all_typos
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix all the typos across the project

**What this PR does / why we need it**:
There are lots of typos across the project. We should avoid small PRs on fixing those annoying typos, which is time-consuming and low efficient.

This PR does fix all the typos across the project currently. And with #59463, typos could be avoided when a new PR gets merged.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
/sig testing
/area test-infra
/sig release
/cc @ixdy 
/assign @fejta 

**Release note**:

```release-note
None
```
2018-02-10 22:12:45 -08:00
Di Xu
48388fec7e fix all the typos across the project 2018-02-11 11:04:14 +08:00
Michael Taufen
9c8eab96d0 Bury KubeletConfiguration.ConfigTrialDuration for now 2018-02-08 21:41:21 -08:00
Derek Carr
6e0a52e7ff Add node e2e to verify hugepages feature 2018-02-08 10:37:04 -05:00
Kubernetes Submit Queue
fb340a4695 Merge pull request #57824 from thockin/gcr-vanity
Automatic merge from submit-queue (batch tested with PRs 57824, 58806, 59410, 59280). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

2nd try at using a vanity GCR name

The 2nd commit here is the changes relative to the reverted PR.  Please focus review attention on that.

This is the 2nd attempt.  The previous try (#57573) was reverted while we
figured out the regional mirrors (oops).
    
New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest.  To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today).  For now the staging is an alias to
gcr.io/google_containers (the legacy URL).
    
When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.
    
We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it.  Nice and
visible, easy to keep track of.

xref https://github.com/kubernetes/release/issues/281

TL;DR:
  *  The new `staging-k8s.gcr.io` is where we push images.  It is literally an alias to `gcr.io/google_containers` (the existing repo) and is hosted in the US.
  * The contents of `staging-k8s.gcr.io` are automatically synced to `{asia,eu,us)-k8s.gcr.io`.
  * The new `k8s.gcr.io` will be a read-only alias to whichever regional repo is closest to you.
  * In the future, images will be promoted from `staging` to regional "prod" more explicitly and auditably.

 ```release-note
Use "k8s.gcr.io" for pulling container images rather than "gcr.io/google_containers".  Images are already synced, so this should not impact anyone materially.
    
Documentation and tools should all convert to the new name. Users should take note of this in case they see this new name in the system.
```
2018-02-08 03:29:32 -08:00
Tim Hockin
3586986416 Switch to k8s.gcr.io vanity domain
This is the 2nd attempt.  The previous was reverted while we figured out
the regional mirrors (oops).

New plan: k8s.gcr.io is a read-only facade that auto-detects your source
region (us, eu, or asia for now) and pulls from the closest.  To publish
an image, push k8s-staging.gcr.io and it will be synced to the regionals
automatically (similar to today).  For now the staging is an alias to
gcr.io/google_containers (the legacy URL).

When we move off of google-owned projects (working on it), then we just
do a one-time sync, and change the google-internal config, and nobody
outside should notice.

We can, in parallel, change the auto-sync into a manual sync - send a PR
to "promote" something from staging, and a bot activates it.  Nice and
visible, easy to keep track of.
2018-02-07 21:14:19 -08:00
Kubernetes Submit Queue
cf7073a831 Merge pull request #58973 from verb/cri-enum
Automatic merge from submit-queue (batch tested with PRs 59276, 51042, 58973, 59377, 59472). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update Container Runtime Interface to use enumerated namespace modes

**What this PR does / why we need it**: This updates the CRI as described in the [Shared PID Namespace](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/pod-pid-namespace.md#container-runtime-interface-changes) proposal. This change to the alpha API is not backwards compatible: implementations of the CRI will need to update to the new API version.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
WIP #1615

**Special notes for your reviewer**:
/assign @yujuhong 

**Release note**:

```release-note
[action-required] The Container Runtime Interface (CRI) version has increased from v1alpha1 to v1alpha2. Runtimes implementing the CRI will need to update to the new version, which configures container namespaces using an enumeration rather than booleans.
```
2018-02-07 12:00:47 -08:00
Lee Verberne
e10042d22f Increment CRI version from v1alpha1 to v1alpha2
This also incorporates the version string into the package name so
that incompatibile versions will fail to connect.

Arbitrary choices:
- The proto3 package name is runtime.v1alpha2. The proto compiler
  normally translates this to a go package of "runtime_v1alpha2", but
  I renamed it to "v1alpha2" for consistency with existing packages.
- kubelet/apis/cri is used as "internalapi". I left it alone and put the
  public "runtimeapi" in kubelet/apis/cri/runtime.
2018-02-07 09:06:26 +01:00
hangaoshuai
5fac1dd6c9 remove a todo which is out of date 2018-02-07 09:31:15 +08:00
Kubernetes Submit Queue
7fb445c92d Merge pull request #59094 from jianglingxia/jlx-013114
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

reopen #58913 Fix TODO move GetPauseImageNameForHostArch func

**What this PR does / why we need it**:
reopen #58913 Fix TODO move GetPauseImageNameForHostArch func,because of I squash to a single commit wrong,so recommit one,and close the #58913 
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
/assign @liggitt 
**Release note**:

```release-note
NONE
```
2018-02-03 22:36:09 -08:00
Kubernetes Submit Queue
7e33ffba10 Merge pull request #58728 from dashpole/cadvisor_testing
Automatic merge from submit-queue (batch tested with PRs 57683, 59116, 58728, 59140, 58976). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use node-e2e framework for testing cadvisor

**What this PR does / why we need it**:
With cadvisor checked out in your gopath, we can now run cadvisor integration tests: `make test-e2e-node TEST_SUITE=cadvisor`.  
This has a number of advantages:
 * we can use the same images to test both, configured the same way.
 * we will now get cadvisor logs from the integration test.
 * we can now use the familiar node-e2e arguments to specify images to test with cadvisor
 * no more managing snowflake VMs for cadvisor.

**Special notes for your reviewer**:
cadvisor doesnt currently produce junit* files, so I removed that as a requirement.
This wont actually work until https://github.com/google/cadvisor/pull/1868 is merged as well.

Related issue:
https://github.com/kubernetes/test-infra/issues/190

**Release note**:
```release-note
NONE
```

/assign @Random-Liu 
/sig node
/priority important-soon
/kind cleanup
2018-02-01 07:04:40 -08:00
Kubernetes Submit Queue
f96ac05774 Merge pull request #59062 from mtaufen/fix-pod-pids-limit
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix PodPidsLimit and ConfigTrialDuration on internal KubeletConfig type

They should both follow the convention of not being a pointer on the internal type. 

This required adding a conversion function between `int64` and `*int64`. A side effect is this removes a warning in the generated code for the apps API group.

@dims

```release-note
NONE
```
2018-02-01 01:45:55 -08:00
David Ashpole
17e8d8c040 use node-e2e framework for testing cadvisor 2018-01-31 10:14:54 -08:00
jianglingxia
76e90061a2 reopen #58913 Fix TODO move GetPauseImageNameForHostArch func 2018-01-31 15:06:32 +08:00
Kubernetes Submit Queue
84408378f9 Merge pull request #58174 from filbranden/ipcs1
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fixes for HostIPC tests to work when Docker has SELinux support enabled.

**What this PR does / why we need it**:

Fixes for HostIPC tests to work when Docker has SELinux support enabled.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:

N/A

**Special notes for your reviewer**:

The core of the matter is to use `ipcs` from util-linux rather than the one from busybox. The typical SELinux policy has enough to allow Docker containers (running under svirt_lxc_net_t SELinux type) to access IPC information by reading the contents of the files under /proc/sysvipc/, but not by using the shmctl etc. syscalls.

The `ipcs` implementation in busybox will use `shmctl(0, SHM_INFO, ...)` to detect whether it can read IPC info (see source code [here](https://git.busybox.net/busybox/tree/util-linux/ipcs.c?h=1_28_0#n138)), while the one in util-linux will prefer to read from the /proc files directly if they are available (see source code [here](https://github.com/karelzak/util-linux/blob/v2.27.1/sys-utils/ipcutils.c#L108)).

It turns out the SELinux policy doesn't allow the shmctl syscalls in an unprivileged container, while access to it through the /proc interface is fine. (One could argue this is a bug in the SELinux policy, but getting it fixed on stable OSs is hard, and it's not that hard for us to test it with an util-linux `ipcs`, so I propose we do so.)

This PR also contains a refactor of the code setting IpcMode, since setting it in the "common options" function is misleading, as on containers other than the sandbox, it ends up always getting overwritten, so let's only set it to "host" in the Sandbox.

It also has a minor fix for the `ipcmk` call, since support for size suffix was only introduced in recent versions of it.

**Release note**:

```release-note
NONE
```
2018-01-30 17:18:52 -08:00
Michael Taufen
da41a6e793 Fix PodPidsLimit and ConfigTrialDuration on internal KubeletConfig type
They should both follow the convention of not being a pointer on the
internal type. This required adding a conversion function between
`int64` and `*int64`.

A side effect is this removes a warning in the generated code for the
apps API group.
2018-01-30 11:43:41 -08:00
Kubernetes Submit Queue
9fa96264f9 Merge pull request #58996 from Random-Liu/enable-feature-gate-by-default
Automatic merge from submit-queue (batch tested with PRs 57467, 58996). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Set generate-kubelet-config-file to true by default.

This should fix the flaky suite.
https://k8s-testgrid.appspot.com/sig-node-kubelet#kubelet-flaky-gce-e2e

@mtaufen /cc @kubernetes/sig-node-bugs 

Signed-off-by: Lantao Liu <lantaol@google.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
none
```
2018-01-29 19:03:35 -08:00
Kubernetes Submit Queue
fc0e07465f Merge pull request #57467 from dashpole/move_eviction_tests
Automatic merge from submit-queue (batch tested with PRs 57467, 58996). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove flaky label from Eviction tests

**What this PR does / why we need it**:
All eviction tests in the flaky suite are no longer flaky.  Remove the flaky label to move them from the flaky suite to the serial suite.
I removed the QoS-based memory eviction test since it does not reflect the current eviction strategy.

**Release note**:
```release-note
NONE
```
/assign @mtaufen @Random-Liu 
/sig node
/priority important-soon
/kind cleanup
2018-01-29 19:03:32 -08:00
Kubernetes Submit Queue
f519cba47f Merge pull request #58980 from Random-Liu/fix-qps-set
Automatic merge from submit-queue (batch tested with PRs 58899, 58980). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix setting qps in density test.

Current QPS setting code doesn't work. All the density tests with higher QPS are failing. https://k8s-testgrid.appspot.com/sig-node-kubelet#kubelet-benchmark-gce-e2e

We should use existing helper function `tempSetCurrentKubeletConfig` to set QPS.

@kubernetes/sig-node-bugs @mtaufen 

**Release note**:

```release-note
none
```
2018-01-29 16:45:34 -08:00
Lantao Liu
e7531ca6c8 Set generate-kubelet-config-file to true by default.
Signed-off-by: Lantao Liu <lantaol@google.com>
2018-01-30 00:20:35 +00:00
Kubernetes Submit Queue
85e435d35a Merge pull request #58777 from filbranden/nnp1
Automatic merge from submit-queue (batch tested with PRs 58777, 58978, 58977, 58775). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Skip NoNewPrivileges test when SELinux is enabled

**What this PR does / why we need it**:

A bug in the SELinux policy prevented NoNewPrivileges from working on Docker with SELinux support enabled.

The problem has been fixed upstream (see projectatomic/container-selinux#45)

But hasn't been backported yet (a fix might come in RHEL 7.5)

For now, let's skip the NoNewPrivileges test when SELinux support is enabled in Docker.

Tested:

- Before this commit, the test fails:
```
    $ make test-e2e-node REMOTE=true FOCUS="allow privilege escalation"
    (on a host with SELinux enabled)

    • [SLOW TEST:22.798 seconds] (passed)
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should allow privilege escalation when true

    • Failure [16.539 seconds]
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should not allow privilege escalation when false [It]

        wait for pod "alpine-nnp-false-aef03e47-0090-11e8-886f-42010af00009" to success
        Expected success, but got an error:
            <*errors.errorString | 0xc4204e26d0>: {
                s: "pod \"alpine-nnp-false-aef03e47-0090-11e8-886f-42010af00009\" failed with reason: \"\", message: \"\"",
            }
            pod "alpine-nnp-false-aef03e47-0090-11e8-886f-42010af00009" failed with reason: "", message: ""

    • [SLOW TEST:26.572 seconds] (passed)
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should allow privilege escalation when not explicitly set and uid != 0

    Ran 3 of 257 Specs in 45.364 seconds
    FAIL! -- 2 Passed | 1 Failed | 0 Pending | 254 Skipped

    Ginkgo ran 1 suite in 49.389123442s
    Test Suite Failed
```
- After this commit, the test is skipped:
```
    $ make test-e2e-node REMOTE=true FOCUS="allow privilege escalation"
    (on a host with SELinux enabled)

    S [SKIPPING] in Spec Setup (BeforeEach) [12.452 seconds]
    S [SKIPPING] in Spec Setup (BeforeEach) [16.298 seconds]
    S [SKIPPING] in Spec Setup (BeforeEach) [18.183 seconds]

    Ran 0 of 257 Specs in 39.174 seconds
    SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 257 Skipped

    Ginkgo ran 1 suite in 43.570630357s
    Test Suite Passed
```
- No changes when SELinux is disabled:
```
    $ make test-e2e-node REMOTE=true FOCUS="allow privilege escalation"
    (on a host with SELinux disabled)

    • [SLOW TEST:15.013 seconds]
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should not allow privilege escalation when false

    • [SLOW TEST:19.155 seconds]
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should allow privilege escalation when true

    • [SLOW TEST:21.087 seconds]
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should allow privilege escalation when not explicitly set and uid != 0

    Ran 3 of 259 Specs in 38.560 seconds
    SUCCESS! -- 3 Passed | 0 Failed | 0 Pending | 256 Skipped

    Ginkgo ran 1 suite in 41.937918928s
    Test Suite Passed
```




**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
N/A

**Special notes for your reviewer**:
N/A

**Release note**:

```release-note
NONE
```
2018-01-29 14:59:36 -08:00
Lantao Liu
3d51577327 Fix setting qps in density test. 2018-01-29 19:41:31 +00:00
Kubernetes Submit Queue
5792214647 Merge pull request #58760 from mtaufen/kc-remove-kubeletconfigfile-gate
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Removal of KubeletConfigFile feature gate: Step 1

This feature gate was redundant with the `--config` flag, which already
enables/disables loading Kubelet config from a file.

Since the gate guarded an alpha feature, removing it is not a violation
of our API guidelines.

Some stuff in `kubernetes/test-infra` currently sets the gate,
so removing will be a 3 step process:
1. This PR, which makes the gate a no-op.
2. Stop setting the gate in `kubernetes/test-infra`.
3. Completely remove the gate (this PR will get the release note).

```release-note
NONE
```
2018-01-26 14:35:25 -08:00
Filipe Brandenburger
46a83c2883 Use ipc-utils container in HostIPC tests.
This ensures the `ipcs` command from util-linux will be used, which
succeeds when Docker is running with SELinux enabled (while the one from
busybox fails.)

Tested: On a host with Docker running with SELinux enabled:

  $ make test-e2e-node REMOTE=true FOCUS="host IPC"

  • [SLOW TEST:17.272 seconds] (passed)
  [k8s.io] Security Context
    when creating a pod in the host IPC namespace
      should show the shared memory ID in the host IPC containers

  • [SLOW TEST:20.419 seconds] (passed)
  [k8s.io] Security Context
    when creating a pod in the host IPC namespace
      should not show the shared memory ID in the non-hostIPC containers

  Ran 2 of 257 Specs in 43.934 seconds
  SUCCESS! -- 2 Passed | 0 Failed | 0 Pending | 255 Skipped
2018-01-25 11:09:16 -08:00
Filipe Brandenburger
67869273a8 Don't assume ipcmk command supports size suffix.
Expand the use of "1M" to the corresponding number of bytes, since
support for size suffix was only added to `ipcmk` in util-linux 2.27
which is not yet available in some Linux distributions.

Tested by running `make test-e2e-node` against distributions with ipcmk
that supports and doesn't support the suffix syntax, all of them passed.
2018-01-25 11:09:16 -08:00
Filipe Brandenburger
6d30b026ba Skip NoNewPrivileges test when SELinux is enabled
A bug in the SELinux policy prevented NoNewPrivileges from working on
Docker with SELinux support enabled.

The problem has been fixed upstream:
https://github.com/projectatomic/container-selinux/issues/45

But hasn't been backported yet (a fix might come in RHEL 7.5)

For now, let's skip the NoNewPrivileges test when SELinux support is
enabled in Docker.

Tested:

- Before this commit, the test fails:

    $ make test-e2e-node REMOTE=true FOCUS="allow privilege escalation"
    (on a host with SELinux enabled)

    • [SLOW TEST:22.798 seconds] (passed)
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should allow privilege escalation when true

    • Failure [16.539 seconds]
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should not allow privilege escalation when false [It]

        wait for pod "alpine-nnp-false-aef03e47-0090-11e8-886f-42010af00009" to success
        Expected success, but got an error:
            <*errors.errorString | 0xc4204e26d0>: {
                s: "pod \"alpine-nnp-false-aef03e47-0090-11e8-886f-42010af00009\" failed with reason: \"\", message: \"\"",
            }
            pod "alpine-nnp-false-aef03e47-0090-11e8-886f-42010af00009" failed with reason: "", message: ""

    • [SLOW TEST:26.572 seconds] (passed)
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should allow privilege escalation when not explicitly set and uid != 0

    Ran 3 of 257 Specs in 45.364 seconds
    FAIL! -- 2 Passed | 1 Failed | 0 Pending | 254 Skipped

    Ginkgo ran 1 suite in 49.389123442s
    Test Suite Failed

- After this commit, the test is skipped:

    $ make test-e2e-node REMOTE=true FOCUS="allow privilege escalation"
    (on a host with SELinux enabled)

    S [SKIPPING] in Spec Setup (BeforeEach) [12.452 seconds]
    S [SKIPPING] in Spec Setup (BeforeEach) [16.298 seconds]
    S [SKIPPING] in Spec Setup (BeforeEach) [18.183 seconds]

    Ran 0 of 257 Specs in 39.174 seconds
    SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 257 Skipped

    Ginkgo ran 1 suite in 43.570630357s
    Test Suite Passed

- No changes when SELinux is disabled:

    $ make test-e2e-node REMOTE=true FOCUS="allow privilege escalation"
    (on a host with SELinux disabled)

    • [SLOW TEST:15.013 seconds]
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should not allow privilege escalation when false

    • [SLOW TEST:19.155 seconds]
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should allow privilege escalation when true

    • [SLOW TEST:21.087 seconds]
    [k8s.io] Security Context
      when creating containers with AllowPrivilegeEscalation
        should allow privilege escalation when not explicitly set and uid != 0

    Ran 3 of 259 Specs in 38.560 seconds
    SUCCESS! -- 3 Passed | 0 Failed | 0 Pending | 256 Skipped

    Ginkgo ran 1 suite in 41.937918928s
    Test Suite Passed
2018-01-25 09:11:22 -08:00