Commit Graph

1657 Commits

Author SHA1 Message Date
Kubernetes Submit Queue
009858f18d Merge pull request #48555 from redbaron/hostPath-and-subPath-symlink
Automatic merge from submit-queue

Fix subPath existence check to not follow symlink

**What this PR does / why we need it**:
Volume mounting logic introduced in #43775 and #45623 checks
for subPath existence before attempting to create a directory,
should subPath not be present.

This breaks if subPath is a dangling symlink, os.Stat returns
"do not exist" status, yet `os.MkdirAll` can't create directory
as symlink is present at the given path.

This patch makes existence check to use os.Lstat which works for
normal files/directories as well as doesn't not attempt to follow
symlink, therefore it's "do not exist" status is more reliable when
making a decision whether to create directory or not.

subPath symlinks can be dangling in situations where kubelet is
running in a container itself with access to docker socket, such
as CoreOS's kubelet-wrapper script

**Release note**:
```release-note
Fix pods failing to start when subPath is a dangling symlink from kubelet point of view, which can happen if it is running inside a container
```
2017-07-12 23:43:12 -07:00
Kubernetes Submit Queue
2b03c2e6df Merge pull request #47948 from allencloud/remove-unused-code-in-loadSystemLanguage
Automatic merge from submit-queue

fix system language judging bug in loadSystemLanguage

Signed-off-by: allencloud <allen.sun@daocloud.io>



**What this PR does / why we need it**:
This PR removes some unused code in loadSystemLanguage. Since in code `pieces := strings.Split(langStr, ".")`, even `langStr` is an empty string, `piece` is a slice with one element of empty string, so there is no chance that len(pieces) == 0.

According to these, I think it is OK to remove the unused code in loadSystemLanguage.

According to the discuss we had, finally we decided to use a more accurate way to change the code, using `if len(pieces) != 1` to make the decision. 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
NONE

**Special notes for your reviewer**:
NONE

**Release note**:

```release-note
NONE
```
2017-07-12 03:42:10 -07:00
Kubernetes Submit Queue
03360d7b65 Merge pull request #48402 from ianchakeres/local-storage-teardown-fix
Automatic merge from submit-queue

Local storage teardown fix

**What this PR does / why we need it**: Local storage uses bindmounts and the method IsLikelyNotMountPoint does not detect these as mountpoints. Therefore, local PVs are not properly unmounted when they are deleted.

**Which issue this PR fixes**: fixes #48331

**Special notes for your reviewer**:

You can use these e2e tests to reproduce the issue and validate the fix works appropriately https://github.com/kubernetes/kubernetes/pull/47999

The existing method IsLikelyNotMountPoint purposely does not check mountpoints reliability (4c5b22d4c6/pkg/util/mount/mount_linux.go (L161)), since the number of mountpoints can be large. 4c5b22d4c6/pkg/util/mount/mount.go (L46)

This implementation changes the behavior for local storage to detect mountpoints reliably, and avoids changing the behavior for any other callers to a UnmountPath.

**Release note**:

```
Fixes bind-mount teardown failure with non-mount point Local volumes (issue https://github.com/kubernetes/kubernetes/issues/48331).
```
2017-07-11 20:35:29 -07:00
Ian Chakeres
2b18d3b6f7 Fixes bind-mount teardown failure with non-mount point Local volumes
Added IsNotMountPoint method to mount utils (pkg/util/mount/mount.go)
Added UnmountMountPoint method to volume utils (pkg/volume/util/util.go)
Call UnmountMountPoint method from local storage (pkg/volume/local/local.go)
IsLikelyNotMountPoint behavior was not modified, so the logic/behavior for UnmountPath is not modified
2017-07-11 17:19:58 -04:00
Kubernetes Submit Queue
22550b62a6 Merge pull request #48514 from freehan/iptables-lock
Automatic merge from submit-queue (batch tested with PRs 47234, 48410, 48514, 48529, 48348)

expose error lock release failure from iptables util

ref: https://github.com/kubernetes/kubernetes/issues/48107
2017-07-07 23:53:36 -07:00
Kubernetes Submit Queue
6cd6d89d40 Merge pull request #48410 from xiangpengzhao/remove-empty-util-dir
Automatic merge from submit-queue (batch tested with PRs 47234, 48410, 48514, 48529, 48348)

Remove unused sub-pkgs in pkg/util

**What this PR does / why we need it**:
Remove no longer used sug-pkgs in pkg/util

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #48386

**Special notes for your reviewer**:
/cc @deads2k 

**Release note**:

```release-note
NONE
```
2017-07-07 23:53:34 -07:00
Maxim Ivanov
020ec43b56 Fix subPath existence check to not follow symlink
Volume mounting logic introduced in #43775 and #45623 checks
for subPath existence before attempting to create a directory,
should subPath not be present.

This breaks if subPath is a dangling symlink, os.Stat returns
"do not exist" status, yet `os.MkdirAll` can't create directory
as symlink is present at the given path.

This patch makes existence check to use os.Lstat which works for
normal files/directories as well as doesn't not attempt to follow
symlink, therefore it's "do not exist" status is more reliable when
making a decision whether to create directory or not.

subPath symlinks can be dangling in situations where kubelet is
running in a container itself with access to docker socket, such
as CoreOS's kubelet-wrapper script
2017-07-07 11:52:16 +01:00
Minhan Xia
8de419b19f expose lock release error from iptables util 2017-07-05 14:31:24 -07:00
ymqytw
ce561b2044 fix cross build for windows 2017-07-05 12:42:41 -07:00
xiangpengzhao
6cfbd9329a Fix lint errors of pkg/util/net/sets/ipnet.go 2017-07-04 12:04:24 +08:00
xiangpengzhao
016f1cfd8f Remove unused sub-pkgs in pkg/util 2017-07-04 12:04:06 +08:00
ymqytw
8dac9639e4 split util/slice 2017-06-30 23:04:18 -07:00
Kubernetes Submit Queue
ed8993e3df Merge pull request #47944 from allencloud/fix-CopyStrings-and-ShuffleStrings-when-slice-is-nil
Automatic merge from submit-queue (batch tested with PRs 48264, 48324, 48125, 47944, 47489)

fix CopyStrings and ShuffleStrings for slice when slice is nil

Signed-off-by: allencloud <allen.sun@daocloud.io>



**What this PR does / why we need it**:
This PR fixes two functions in util/slice.go, in which I think `CopyStrings` and `ShuffleStrings` miss one case. The case is input data is nil, in this case I think the data returned should be nil as well rather than a non-nil slice with 0 element.
In addition, I added some test code for this.
 
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
NONE, I did not raise a issue for this code. I ran into this when code learning.

**Special notes for your reviewer**:
NONE

**Release note**:

```release-note
NONE
```
2017-06-30 20:58:33 -07:00
ymqytw
f0ce897277 move term to kubectl/util 2017-06-30 15:00:24 -07:00
Kubernetes Submit Queue
903a4541ba Merge pull request #48298 from mengqiy/kubectl_crlf
Automatic merge from submit-queue (batch tested with PRs 48295, 48298, 47339, 44910, 48037)

move crlf to kubectl/util

move crlf from pkg/util/crlf to pkg/kubectl/util/crlf

Ref: https://github.com/kubernetes/kubernetes/issues/48209

```release-note
NONE
```
/assign @apelisse @monopole 

cc: @pwittrock
2017-06-30 14:34:26 -07:00
ymqytw
2510a47374 move crlf to kubectl/util 2017-06-29 15:48:41 -07:00
ymqytw
6660726ce6 eliminate kubectl dependency on k8s.io/kubernetes/pkg/util 2017-06-29 14:49:51 -07:00
allencloud
2e79f6c61a remove unused codes in loadSystemLanguage
Signed-off-by: allencloud <allen.sun@daocloud.io>
2017-06-29 12:18:11 +08:00
Dan Winship
127eb53d4d Fix the names of some iptables tests
The delete tests were copy+pasted from the create tests, and the names
not fully updated to match.
2017-06-27 09:26:10 -04:00
Dan Winship
3d48091ce8 Fix ebtables_test.go to actually get run, and to pass 2017-06-27 09:26:10 -04:00
Kubernetes Submit Queue
95be540bd4 Merge pull request #47242 from k82cn/make_golint_happy
Automatic merge from submit-queue

Updated comments for functions.
2017-06-26 20:02:40 -07:00
Jan Safranek
b73517d437 Fix lint errors
Removal of io.go revealed new lint errors in pkg/util/io
2017-06-26 16:05:35 +02:00
Jan Safranek
c2dc5b5bf1 Move LoadPodFromFile to volume utils
SavePodToFile is not used anywhere and LoadPodFromFile is used only by PV
recycler.
2017-06-26 16:02:43 +02:00
allencloud
f98bc7d454 fix CopyStrings and ShuffleStrings for slice when slice is nil
Signed-off-by: allencloud <allen.sun@daocloud.io>
2017-06-23 11:47:13 +08:00
Chao Xu
60604f8818 run hack/update-all 2017-06-22 11:31:03 -07:00
Chao Xu
f4989a45a5 run root-rewrite-v1-..., compile 2017-06-22 10:25:57 -07:00
zhangxiaoyu-zidif
f528bce5c6 Use reflect.DeepEqual to replace sliceEqual 2017-06-14 16:25:43 +08:00
Klaus Ma
3953f90102 Updated comments for functions. 2017-06-09 16:41:24 +08:00
William Chang
9ac960124f Add Traditional Chinese translation for kubectl 2017-06-08 15:24:04 +08:00
Kubernetes Submit Queue
69342bd1df Merge pull request #43005 from cmluciano/cml/consolidatesysctl
Automatic merge from submit-queue (batch tested with PRs 43005, 46660, 46385, 46991, 47103)

Consolidate sysctl commands for kubelet

**What this PR does / why we need it**:
These commands are important enough to be in the Kubelet itself.
By default, Ubuntu 14.04 and Debian Jessie have these set to 200 and
20000. Without this setting, nodes are limited in the number of
containers that they can start.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #26005 

**Special notes for your reviewer**:
I had a difficult time writing tests for this. It is trivial to create a fake sysctl for testing, but the Kubelet does not have any tests for the prior settings.
**Release note**:

```release-note
```
2017-06-07 13:30:54 -07:00
Kubernetes Submit Queue
443ed358a2 Merge pull request #46756 from girikuncoro/japanese-translate
Automatic merge from submit-queue (batch tested with PRs 46235, 44786, 46833, 46756, 46669)

Add Japanese translation for kubectl

**What this PR does / why we need it**:
I messed up the original PR(#45562) which was already been reviewed and approved. This PR provides first attempt to translate kubectl in Japanese (related to #40645 and #40591).

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
No issues

**Special notes for your reviewer**:
Should be reviewed by member of Japanese k8s community (I stayed in Japan for 4 years, but my language is not as good as native Japanese)
2017-06-07 00:27:51 -07:00
Giri Kuncoro
dec34b5197 Add Japanese translation for kubectl 2017-06-02 14:34:39 -07:00
deads2k
954eb3ceb9 move labels to components which own the APIs 2017-05-31 10:32:06 -04:00
Kubernetes Submit Queue
28996ac6c3 Merge pull request #43275 from lvjiangzhao/fix-typo-170317
Automatic merge from submit-queue (batch tested with PRs 43275, 45014, 46449, 46488, 46525)

Fix typo

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-05-30 07:49:00 -07:00
Wojciech Tyczynski
1242e8ca20 Minor improvement for memory allocations 2017-05-30 08:11:12 +02:00
Wojciech Tyczynski
46e9b522fd Make BoundedFrequencyRunner Run() really non-blocking 2017-05-25 16:12:16 +02:00
Kubernetes Submit Queue
ee671e64ee Merge pull request #46266 from thockin/proxy-periodic-runner-2
Automatic merge from submit-queue (batch tested with PRs 44774, 46266, 46248, 46403, 46430)

kube-proxy: ratelimit runs of iptables by sync-period flags

This bounds how frequently iptables can be synced.  It will be no more often than every 10 seconds and no less often than every 1 minute, by default.

@timothysc FYI

@dcbw @freehan FYI
2017-05-25 06:17:56 -07:00
Kubernetes Submit Queue
4234d79aca Merge pull request #45573 from shiywang/zh
Automatic merge from submit-queue (batch tested with PRs 45573, 46354, 46376, 46162, 46366)

Add Simplified Chinese translation for kubectl

What this PR does / why we need it:
This PR provides first attempt to translate kubectl in Simplified Chinese.

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
No issues

Special notes for your reviewer:
Although I'm a native speaker for Mandarin Chinese, but I think translation is a whole different knowledge which I'm not good at it, so this pr absolutely need to be polished.
@adohe @mengqiy @resouer @k82cn @caesarxuchao @wanghaoran1988 sorry I think there are so many folks who are good at Chinese I haven't mention, feel free to leave a comment on it : )
also cc @brendandburns
2017-05-25 03:13:57 -07:00
Tim Hockin
bbb80c252b Add bounded frequency runner
This lib manages runs of a function to have min and max frequencies.
2017-05-24 20:33:15 -07:00
Kubernetes Submit Queue
c586f36e55 Merge pull request #46209 from wojtek-t/remove_iptables_save
Automatic merge from submit-queue (batch tested with PRs 46022, 46055, 45308, 46209, 43590)

Remove Save() from iptables interface

This is what @thockin requested in one of the reviews.
2017-05-22 20:00:00 -07:00
Kubernetes Submit Queue
503a8421fb Merge pull request #38990 from mikedanese/go-genrule-sets
Automatic merge from submit-queue (batch tested with PRs 38990, 45781, 46225, 44899, 43663)

migrate set generation to go_genrule

Depends on https://github.com/kubernetes/release/pull/238
2017-05-22 19:06:58 -07:00
Wojciech Tyczynski
9e6de42745 Remove Save() from iptables interface 2017-05-22 13:23:49 +02:00
Kubernetes Submit Queue
3456d4d239 Merge pull request #46033 from wojtek-t/reduce_memory_allocations_in_kube_proxy
Automatic merge from submit-queue

Reduce memory allocations in kube proxy

Memory allocation (and Go GarbageCollection) seems to be one of the most expensive operations in kube-proxy (I've seen profiles where it was more than 50%).

The commits are mostly independent from each other and all of them are mostly about reusing already allocated memory.

This PR is reducing memory allocation by ~5x (results below from 100-node load test):

before:
```
(pprof) top
38.64GB of 39.11GB total (98.79%)
Dropped 249 nodes (cum <= 0.20GB)
Showing top 10 nodes out of 61 (cum >= 0.20GB)
      flat  flat%   sum%        cum   cum%
   15.10GB 38.62% 38.62%    15.10GB 38.62%  bytes.makeSlice
    9.48GB 24.25% 62.87%     9.48GB 24.25%  runtime.rawstringtmp
    8.30GB 21.21% 84.07%    32.47GB 83.02%  k8s.io/kubernetes/pkg/proxy/iptables.(*Proxier).syncProxyRules
    2.08GB  5.31% 89.38%     2.08GB  5.31%  fmt.(*fmt).padString
    1.90GB  4.86% 94.24%     3.82GB  9.77%  strings.Join
    0.67GB  1.72% 95.96%     0.67GB  1.72%  runtime.hashGrow
    0.36GB  0.92% 96.88%     0.36GB  0.92%  runtime.stringtoslicebyte
    0.31GB  0.79% 97.67%     0.62GB  1.58%  encoding/base32.(*Encoding).EncodeToString
    0.24GB  0.62% 98.29%     0.24GB  0.62%  strings.genSplit
    0.20GB   0.5% 98.79%     0.20GB   0.5%  runtime.convT2E
```

after:
```
7.94GB of 8.13GB total (97.75%)
Dropped 311 nodes (cum <= 0.04GB)
Showing top 10 nodes out of 65 (cum >= 0.11GB)
      flat  flat%   sum%        cum   cum%
    3.32GB 40.87% 40.87%     8.05GB 99.05%  k8s.io/kubernetes/pkg/proxy/iptables.(*Proxier).syncProxyRules
    2.85GB 35.09% 75.95%     2.85GB 35.09%  runtime.rawstringtmp
    0.60GB  7.41% 83.37%     0.60GB  7.41%  runtime.hashGrow
    0.31GB  3.76% 87.13%     0.31GB  3.76%  runtime.stringtoslicebyte
    0.28GB  3.43% 90.56%     0.55GB  6.80%  encoding/base32.(*Encoding).EncodeToString
    0.19GB  2.29% 92.85%     0.19GB  2.29%  strings.genSplit
    0.18GB  2.17% 95.03%     0.18GB  2.17%  runtime.convT2E
    0.10GB  1.28% 96.31%     0.71GB  8.71%  runtime.mapassign
    0.10GB  1.21% 97.51%     0.10GB  1.21%  syscall.ByteSliceFromString
    0.02GB  0.23% 97.75%     0.11GB  1.38%  syscall.SlicePtrFromStrings
```
2017-05-19 23:21:49 -07:00
Wojciech Tyczynski
a3da8d7300 Fix naming and comments in kube-proxy. 2017-05-19 21:34:05 +02:00
Wojciech Tyczynski
7d44f83441 Descrese logs verbosity for iptables 2017-05-19 20:44:26 +02:00
Wojciech Tyczynski
bcfae7e1ed Extend Iptables interface with SaveInto 2017-05-19 20:44:25 +02:00
Wojciech Tyczynski
028ac8034b Remove SaveAll from iptables interface 2017-05-19 20:44:25 +02:00
Shiyang Wang
95fb791b3f Add Simplified Chinese translation for kubectl
update

update
2017-05-19 16:18:24 +08:00
Paul Michali
65342a0000 IPv6 support for hexCIDR()
Includes these changes:
- Modified so that IPv6 CIDRs can be converted correctly.
- Added test cases for IPv6 addresses.
- Split UTs for hexCIDR() and asciiCIDR() so that masking can be tested.
- Add UTs for failure cases.

Note: Some code that calls hexCIDR() builds a CIDR from the pod IP string
and the concatenation of "/32". These should, in the future, use "128",
if/when the pod IP is IPv6. Not addressed as part of this commit.
2017-05-17 20:35:37 +00:00
Kubernetes Submit Queue
c5e9031d7d Merge pull request #45856 from shyamjvs/save-iptables
Automatic merge from submit-queue

Make fake iptables' Save operation more realistic

Ref https://github.com/kubernetes/kubernetes/pull/45622#issuecomment-301624384 (2nd point)

This would make fake IPtables actually return the iptable contents it stores.

cc @kubernetes/sig-scalability-misc @wojtek-t
2017-05-16 04:28:34 -07:00
Shyam Jeedigunta
da924fc337 Make fake iptables' Save operation more realistic 2017-05-16 11:58:44 +02:00
Dmitry Shulyak
c0a3e9ad13 Fix golint and godeps verification 2017-05-15 16:28:56 +03:00
Dmitry Shulyak
2612e0c78a Move client/unversioned/remotecommand to client-go
Module remotecommand originally part of kubernetes/pkg/client/unversioned was moved
to client-go/tools, and will be used as authoritative in kubectl, e2e and other places.

Module remotecommand relies on util/exec module which will be copied to client-go/pkg/util
2017-05-15 16:28:56 +03:00
Dan Williams
a4624a0e75 util/iptables: fix cross-build failures due to syscall.Flock()
Fixes: https://github.com/kubernetes/kubernetes/issues/45554
2017-05-14 23:37:52 -05:00
Cao Shufeng
a393e95018 Remove leaked tmp file in unit tests 2017-05-08 18:07:02 +08:00
Kubernetes Submit Queue
15f0468986 Merge pull request #44895 from dcbw/iptables-restore-manual-locking
Automatic merge from submit-queue

util/iptables: grab iptables locks if iptables-restore doesn't support --wait

When iptables-restore doesn't support --wait (which < 1.6.2 don't), it may
conflict with other iptables users on the system, like docker, because it
doesn't acquire the iptables lock before changing iptables rules. This causes
sporadic docker failures when starting containers.

To ensure those don't happen, essentially duplicate the iptables locking
logic inside util/iptables when we know iptables-restore doesn't support
the --wait option.

Unfortunately iptables uses two different locking mechanisms, one until
1.4.x (abstract socket based) and another from 1.6.x (/run/xtables.lock
flock() based).  We have to grab both locks, because we don't know what
version of iptables-restore exists since iptables-restore doesn't have
a --version option before 1.6.2.  Plus, distros (like RHEL) backport the
/run/xtables.lock patch to 1.4.x versions.

Related: https://github.com/kubernetes/kubernetes/pull/43575
See also: https://github.com/openshift/origin/pull/13845
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1417234

@kubernetes/rh-networking @kubernetes/sig-network-misc @eparis @knobunc @danwinship @thockin @freehan
2017-05-06 15:17:21 -07:00
Kubernetes Submit Queue
79a2e359cd Merge pull request #44024 from zhangxiaoyu-zidif/egnew
Automatic merge from submit-queue

fix the typos of e.g.

fix the typos of e.g.


**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-05-05 23:41:33 -07:00
Dan Williams
1067a22885 iptables: add OWNERS file 2017-05-05 23:34:18 -05:00
Kubernetes Submit Queue
17d33ea82e Merge pull request #44830 from NickrenREN/remove-NodeLegacyHostIP
Automatic merge from submit-queue

Remove deprecated NodeLegacyHostIP

**Release note**:
```release-note
Remove deprecated node address type `NodeLegacyHostIP`.
```

ref #44807
2017-05-05 15:38:58 -07:00
Christopher M. Luciano
bafabcbb44
Consolidate sysctl commands for kubelet
These commands are important enough to be in the Kubelet itself.
By default, Ubuntu 14.04 and Debian Jessie have these set to 200 and
20000. Without this setting, nodes are limited in the number of
containers that they can start.
2017-05-02 12:15:01 -07:00
Andy Goldstein
43cb024402 Add kube-proxy config file support
Add support for configuring kube-proxy via a config file instead of
command line flags.
2017-05-01 18:02:47 -04:00
Mike Danese
ff4a814cce migrate set generation to go genrule
and move ./types to staging
2017-05-01 11:11:14 -07:00
Clayton Coleman
49209b3394
Make timeouts in the Kubelet slightly offset to aid debugging
Several of these loops overlap, and when they are the reason a failure
is happening it is difficult to sort them out. Slighly misalign these
loops to make their impact obvious.
2017-04-28 12:00:28 -04:00
Dan Williams
c7677a4753 util/iptables: grab iptables locks if iptables-restore doesn't support --wait
When iptables-restore doesn't support --wait (which < 1.6.2 don't), it may
conflict with other iptables users on the system, like docker, because it
doesn't acquire the iptables lock before changing iptables rules. This causes
sporadic docker failures when starting containers.

To ensure those don't happen, essentially duplicate the iptables locking
logic inside util/iptables when we know iptables-restore doesn't support
the --wait option.

Unfortunately iptables uses two different locking mechanisms, one until
1.4.x (abstract socket based) and another from 1.6.x (/run/xtables.lock
flock() based).  We have to grab both locks, because we don't know what
version of iptables-restore exists since iptables-restore doesn't have
a --version option before 1.6.2.  Plus, distros (like RHEL) backport the
/run/xtables.lock patch to 1.4.x versions.

Related: https://github.com/kubernetes/kubernetes/pull/43575
See also: https://github.com/openshift/origin/pull/13845
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1417234
2017-04-25 14:26:04 -05:00
Mike Danese
e48a4f0af7 fix various bad tests 2017-04-25 11:23:33 -07:00
Kubernetes Submit Queue
9ac111c5f9 Merge pull request #44825 from rata/remove-unused-abstraction
Automatic merge from submit-queue (batch tested with PRs 40060, 44860, 44865, 44825, 44162)

Remove unused chmod/chown abstractions

**What this PR does / why we need it**: Simplifies the code

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-04-24 23:07:44 -07:00
Kubernetes Submit Queue
902d53d5e7 Merge pull request #43575 from dcbw/iptables-restore-lock
Automatic merge from submit-queue (batch tested with PRs 43575, 44672)

util/iptables: check for and use new iptables-restore 'wait' argument

iptables-restore did not previously perform any locking, meaning that
when callers (like kube-proxy) asked iptables-restore to write large
numbers of rules, the iptables-restore process might run in parallel
with other 'iptables' invocations in kubelet (hostports), docker,
and other software.  This causes errors like:

"CNI request failed with status 400: 'Failed to ensure that nat chain
POSTROUTING jumps to MASQUERADE: error checking rule: exit status 4:
iptables: Resource temporarily  unavailable."

or from Docker:

"Failed to allocate and map port 1095-1095: iptables failed:
iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 1095
-j DNAT --to-destination 10.1.0.2:1095 ! -i lbr0: iptables:
Resource temporarily unavailable.\n (exit status 4)"

iptables-restore "wait" functionality was added in iptables git
commit 999eaa241212d3952ddff39a99d0d55a74e3639e which
is not yet in a release.

See also https://bugzilla.redhat.com/show_bug.cgi?id=1417234

@eparis @knobunc @kubernetes/rh-networking @kubernetes/sig-network-misc @freehan @thockin @brendandburns
2017-04-24 10:33:58 -07:00
NickrenREN
7d00e5cfb6 remove deprecated NodeLegacyHostIP 2017-04-24 11:01:25 +08:00
Rodrigo Campos
2eb1583e4b Remove unused chmod/chown abstractions
These abstractions have not been used since I first hacked into
kubernetes, so let's just remove them and add them back if it's needed
later.
2017-04-23 20:25:23 -03:00
Dan Williams
4cd6d34a0b util/iptables: check for and use new iptables-restore 'wait' argument
iptables-restore did not previously perform any locking, meaning that
when callers (like kube-proxy) asked iptables-restore to write large
numbers of rules, the iptables-restore process might run in parallel
with other 'iptables' invocations in kubelet (hostports), docker,
and other software.  This causes errors like:

"CNI request failed with status 400: 'Failed to ensure that nat chain
POSTROUTING jumps to MASQUERADE: error checking rule: exit status 4:
iptables: Resource temporarily  unavailable."

or from Docker

"Failed to allocate and map port 1095-1095: iptables failed:
iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 1095
-j DNAT --to-destination 10.1.0.2:1095 ! -i lbr0: iptables:
Resource temporarily unavailable.\n (exit status 4)"

iptables-restore "wait" functionality was added in iptables git
commit 999eaa241212d3952ddff39a99d0d55a74e3639e but is NOT YET
in a released version of iptables.

See also https://bugzilla.redhat.com/show_bug.cgi?id=1417234
2017-04-20 13:38:44 -05:00
Kubernetes Submit Queue
409b0a6f5d Merge pull request #44222 from fabianofranz/better_generic_getters_and_describers
Automatic merge from submit-queue (batch tested with PRs 44222, 44614, 44292, 44638)

Smarter generic getters and describers

Makes printers and describers smarter for generic resources.

This traverses unstructured objects and prints their attributes for generic resources (TPR, federated API, etc) in `kubectl get` and `kubectl describe`. Makes use of the object's field names to come up with a best guess for describer labels and get headers, and field value types to understand how to better print it, indent, etc.

A nice intermediate solution while we don't have [get and describe extensions](https://github.com/kubernetes/community/pull/308).

Examples:

```
$ kubectl get serviceclasses
NAME                    KIND                                          BINDABLE   BROKER NAME   OSB GUID
user-provided-service   ServiceClass.v1alpha1.servicecatalog.k8s.io   false      ups-broker    4f6e6cf6-ffdd-425f-a2c7-3c9258ad2468
```

```
$ kubectl describe serviceclasses/user-provided-service
Name:		user-provided-service
Namespace:	
Labels:		<none>
Annotations:	FOO=BAR
		openshift.io/deployment.phase=test
OSB Metadata:	<nil>
Kind:		ServiceClass
Metadata:
  Self Link:		/apis/servicecatalog.k8s.io/v1alpha1/serviceclassesuser-provided-service
  UID:			1509bd96-1b05-11e7-98bd-0242ac110006
  Resource Version:	256
  Creation Timestamp:	2017-04-06T20:10:29Z
Broker Name:		ups-broker
Bindable:		false
Plan Updatable:		false
OSB GUID:		4f6e6cf6-ffdd-425f-a2c7-3c9258ad2468
API Version:		servicecatalog.k8s.io/v1alpha1
Plans:
  Name:		default
  OSB GUID:	86064792-7ea2-467b-af93-ac9694d96d52
  OSB Free:	true
  OSB Metadata:	<nil>
Events:		<none>
```

**Release note**:
```release-note
Improved output on 'kubectl get' and 'kubectl describe' for generic objects.
```
PTAL @pmorie @pwittrock @kubernetes/sig-cli-pr-reviews
2017-04-18 17:41:59 -07:00
Kubernetes Submit Queue
a1684fea80 Merge pull request #42085 from cblecker/gofmt-fix
Automatic merge from submit-queue (batch tested with PRs 40055, 42085, 44509, 44568, 43956)

Fix gofmt errors

**What this PR does / why we need it**:
There were some gofmt errors on master. Ran the following to fix:
```
hack/verify-gofmt.sh | grep ^diff | awk '{ print $2 }' | xargs gofmt -w -s
```

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: none

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-04-17 15:39:07 -07:00
Fabiano Franz
151770c8fd Smarter describer for generic resources 2017-04-17 15:42:47 -03:00
Kubernetes Submit Queue
32e927f4d8 Merge pull request #44506 from caesarxuchao/fix-cross-build
Automatic merge from submit-queue

fix cross-build

Fix https://github.com/kubernetes/kubernetes/pull/41543#issuecomment-294207870
2017-04-14 14:51:06 -07:00
Chao Xu
59396e6b13 fix cross-build 2017-04-14 11:45:11 -07:00
Mike Danese
a05c3c0efd autogenerated 2017-04-14 10:40:57 -07:00
Kubernetes Submit Queue
4653a9b280 Merge pull request #41543 from dshulyak/decouple_remotecommand
Automatic merge from submit-queue (batch tested with PRs 44406, 41543, 44071, 44374, 44299)

Decouple remotecommand

Refactored unversioned/remotecommand to decouple it from undesirable dependencies:

- term package now is not required, and functionality required to resize terminal size can be plugged in directly in kubectl
- in order to remove dependency on kubelet package - constants from kubelet/server/remotecommand were moved to separate util package (pkg/util/remotecommand)
- remotecommand_test.go moved to pkg/client/tests module
2017-04-13 19:52:05 -07:00
Dmitry Shulyak
f50480c714 Decouple remotecommand client from term/kubelet dependencies
In order to move client/unversioned/remotecommand to client-go as a followup
for this change we have to decouple it from tons of dependencies
2017-04-13 15:56:40 +03:00
Avesh Agarwal
0a9cdbb587 Auto-generated stuff. 2017-04-06 16:15:34 -04:00
Avesh Agarwal
af53794854 Pod toleration restriction plugin with taints and tolerations. 2017-04-06 16:15:34 -04:00
zhangxiaoyu-zidif
932ece5cfd e3d534b2c4 2017-04-04 14:16:34 +08:00
Christoph Blecker
6681835b0c
Fix gofmt errors 2017-03-28 17:12:04 -07:00
Kubernetes Submit Queue
1ed77078da Merge pull request #43378 from jsafrane/fix-partition-mount
Automatic merge from submit-queue (batch tested with PRs 43378, 43216, 43384, 43083, 43428)

Do not reformat devices with partitions

`lsblk` reports FSTYPE of devices with partition tables as empty string `""`,
which is indistinguishable from empty devices. We must look for dependent
devices (i.e. partitions) to see that the device is really empty and report
error otherwise.

The main point of this patch is to run `lsblk` without `"-n"`, i.e. print all
dependent devices and check it output.

Sample output:
```
  FirstSeen     LastSeen        Count   From                                    SubObjectPath   Type            Reason          Message
  ---------     --------        -----   ----                                    -------------   --------        ------          -------
  10s           10s             1       default-scheduler                                       Normal          Scheduled       Successfully assigned testpod to ip-172-18-11-149.ec2.internal
  2s            2s              1       kubelet, ip-172-18-11-149.ec2.internal                  Warning         FailedMount     MountVolume.MountDevice failed for volume "kubernetes.io/aws-ebs/vol-0fa9da8b91913b187" (spec.Name: "vol") pod "b74f68c5-0d6a-11e7-9233-0e11251010c0" (UID: "b74f68c5-0d6a-11e7-9233-0e11251010c0") with: failed to mount the volume as "ext4", it already contains unknown data, probably partitions. Mount error: mount failed: exit status 32
Mounting command: mount
Mounting arguments: /dev/xvdbb /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/vol-0fa9da8b91913b187 ext4 [defaults]
Output: mount: wrong fs type, bad option, bad superblock on /dev/xvdbb,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

```

Without this patch, the device would be reformatted and all data in the device partitions would be lost.

Fixes #13212

Release note:
```release-note
NONE
```

@kubernetes/sig-storage-pr-reviews
2017-03-25 21:22:22 -07:00
Kubernetes Submit Queue
d14854fd5c Merge pull request #37698 from jsafrane/remove-all-filesystems
Automatic merge from submit-queue (batch tested with PRs 41139, 41186, 38882, 37698, 42034)

Make kubelet never delete files on mounted filesystems

With bug #27653, kubelet could remove mounted volumes and delete user data.
The bug itself is fixed, however our trust in kubelet is significantly lower.
Let's add an extra version of RemoveAll that does not cross mount boundary
(rm -rf --one-file-system).

It calls lstat(path) three times for each removed directory - once in
RemoveAllOneFilesystem and twice in IsLikelyNotMountPoint, however this way
it's platform independent and the directory that is being removed by kubelet
should be almost empty.
2017-03-24 12:33:27 -07:00
Jan Safranek
4cf36b8b39 Do not reformat devices with partitions
lsblk reports FSTYPE of devices with partition tables as empty string "",
which is indistinguishable from empty devices. We must look for dependent
devices (i.e. partitions) to see that the device is really empty and report
error otherwise.

I checked that LVM, LUKS and MD RAID have their own FSTYPE in lsblk output,
so it should be only a partition table that has empty FSTYPE.

The main point of this patch is to run lsblk without "-n", i.e. print all
dependent devices and check if they're there.
2017-03-20 13:08:13 +01:00
lvjiangzhao
92fff6c9c4 fix typo 2017-03-17 13:42:12 +08:00
Yifan Gu
a489bd2674 pkg/util/flock: Fix the flock so it actually locks.
With this PR, the second call to `Acquire()` will block unless the lock is released (process exits).
Also removed the memory mutex in the previous code since we don't need `Release()` here so no need to save and protect the local fd.

Fix #42929.
2017-03-13 14:24:59 -07:00
Kubernetes Submit Queue
44a7be98fd Merge pull request #41618 from JiangtianLi/k8swin
Automatic merge from submit-queue (batch tested with PRs 42316, 41618, 42201, 42113, 42191)

Support unqualified and partially qualified domain name in DNS query in Windows kube-proxy

**What this PR does / why we need it**:
In Windows container networking, --dns-search is not currently supported on Windows Docker. Besides, even with --dns-suffix, inside Windows container DNS suffix is not appended to DNS query names. That makes unqualified domain name or partially qualified domain name in DNS query not able to resolve.

This PR provides a solution to resolve unqualified domain name or partially qualified domain name in DNS query for Windows container in Windows kube-proxy. It uses well-known Kubernetes DNS suffix as well host DNS suffix search list to append to the name in DNS query. DNS packet in kube-proxy UDP stream is modified as appropriate.

This PR affects the Windows kube-proxy only.

**Special notes for your reviewer**:
This PR is based on top of Anthony Howe's commit 48647fb, 0e37f0a and 7e2c71f which is already included in the PR 41487. Please only review commit b9dfb69.

**Release note**:

```release-note
Add DNS suffix search list support in Windows kube-proxy.
```
2017-03-01 07:48:16 -08:00
Kubernetes Submit Queue
81d01a84e0 Merge pull request #41944 from jingxu97/Feb/mounter
Automatic merge from submit-queue (batch tested with PRs 35094, 42095, 42059, 42143, 41944)

Use chroot for containerized mounts

This PR is to modify the containerized mounter script to use chroot
instead of rkt fly. This will avoid the problem of possible large number
of mounts caused by rkt containers if they are not cleaned up.
2017-02-28 09:20:21 -08:00
Jan Safranek
d7d039dba2 Make kubelet never delete files on mounted filesystems
With bug #27653, kubelet could remove mounted volumes and delete user data.
The bug itself is fixed, however our trust in kubelet is significantly lower.
Let's add an extra version of RemoveAll that does not cross mount boundary
(rm -rf --one-file-system).

It calls lstat(path) three times for each removed directory - once in
RemoveAllOneFilesystem and twice in IsLikelyNotMountPoint, however this way
it's platform independent and the directory that is being removed by kubelet
should be almost empty.
2017-02-28 14:32:07 +01:00
Jiangtian Li
b9dfb69dd7 Fix DNS suffix search list issue for Windows container and workaround in kube-proxy.
kube-proxy iterates over DNS suffix search list and appends to DNS query for client.
2017-02-27 19:25:46 -08:00
Anthony Howe
0e37f0a890 cleanup proxier 2017-02-27 16:00:49 -08:00
Minhan Xia
f006c8bcd3 teach kubenet to use annotation instead of pod object for traffic shaper 2017-02-27 10:11:09 -08:00
Kubernetes Submit Queue
bf984aa328 Merge pull request #42120 from kevin-wangzefeng/kubectl-taint-respect-noexecute
Automatic merge from submit-queue (batch tested with PRs 41116, 41804, 42104, 42111, 42120)

make kubectl taint command respect effect NoExecute

**What this PR does / why we need it**:
Part of feature forgiveness implementation, make kubectl taint command respect effect NoExecute.

**Which issue this PR fixes**: 
Related Issue: #1574
Related PR: #39469

**Special notes for your reviewer**:

**Release note**:

```release-note
make kubectl taint command respect effect NoExecute
```
2017-02-27 04:10:30 -08:00
Kubernetes Submit Queue
d1f5331102 Merge pull request #41804 from chakri-nelluri/flex
Automatic merge from submit-queue (batch tested with PRs 41116, 41804, 42104, 42111, 42120)

Add support for attacher/detacher interface in Flex volume

Add support for attacher/detacher interface in Flex volume
This change breaks backward compatibility and requires to be release noted.

```release-note
Flex volume plugin is updated to support attach/detach interfaces. It broke backward compatibility. Please update your drivers and implement the new callouts. 
```
2017-02-27 04:10:25 -08:00
Kevin
f4d3339612 make kubectl respect taint effect NoExecute 2017-02-26 22:41:57 +08:00
Kubernetes Submit Queue
258a5cb3f1 Merge pull request #40665 from brendandburns/i18n
Automatic merge from submit-queue (batch tested with PRs 40665, 41094, 41351, 41721, 41843)

Update i18n tools and process.

@fabianofranz @zen @kubernetes/sig-cli-pr-reviews 

This is an update to the translation process based on feedback from folks.

The main changes are:
   * `msgctx` is being removed from the files.
   * String wrapping and string extraction have been separated.
   * More tools from the `gettext` family of tools are being used
   * Extracted strings are being sorted for canonical ordering
   * A `.pot` template has been added.
2017-02-25 03:56:51 -08:00
Kubernetes Submit Queue
a426904009 Merge pull request #31515 from jsafrane/format-error
Automatic merge from submit-queue (batch tested with PRs 41714, 41510, 42052, 41918, 31515)

Show specific error when a volume is formatted by unexpected filesystem.

kubelet now detects that e.g. xfs volume is being mounted as ext3 because of
wrong volume.Spec.

Mount error is left in the error message to diagnose issues with mounting e.g.
'ext3' volume as 'ext4' - they are different filesystems, however kernel should
mount ext3 as ext4 without errors.

Example kubectl describe pod output:

```
  FirstSeen     LastSeen        Count   From                                    SubobjectPath   Type            Reason          Message
  41s           3s              7       {kubelet ip-172-18-3-82.ec2.internal}                   Warning         FailedMount     MountVolume.MountDevice failed for volume "kubernetes.io/aws-ebs/aws://us-east-1d/vol-ba79c81d" (spec.Name: "pvc-ce175cbb-6b82-11e6-9fe4-0e885cca73d3") pod "3d19cb64-6b83-11e6-9fe4-0e885cca73d3" (UID: "3d19cb64-6b83-11e6-9fe4-0e885cca73d3") with: failed to mount the volume as "ext4", it's already formatted with "xfs". Mount error: mount failed: exit status 32
Mounting arguments: /dev/xvdba /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1d/vol-ba79c81d ext4 [defaults]
Output: mount: wrong fs type, bad option, bad superblock on /dev/xvdba,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
```
2017-02-25 02:17:57 -08:00
Chakravarthy Nelluri
0d2af70e95 Add support for attacher/detacher interface in Flex volume 2017-02-24 20:18:06 -05:00
Jing Xu
ac22416835 Use chroot for containerized mounts
This PR is to modify the containerized mounter script to use chroot
instead of rkt fly. This will avoid the problem of possible large number
of mounts caused by rkt containers if they are not cleaned up.
2017-02-24 13:46:26 -08:00