Commit Graph

1657 Commits

Author SHA1 Message Date
Shyam Jeedigunta
da924fc337 Make fake iptables' Save operation more realistic 2017-05-16 11:58:44 +02:00
Dmitry Shulyak
c0a3e9ad13 Fix golint and godeps verification 2017-05-15 16:28:56 +03:00
Dmitry Shulyak
2612e0c78a Move client/unversioned/remotecommand to client-go
Module remotecommand originally part of kubernetes/pkg/client/unversioned was moved
to client-go/tools, and will be used as authoritative in kubectl, e2e and other places.

Module remotecommand relies on util/exec module which will be copied to client-go/pkg/util
2017-05-15 16:28:56 +03:00
Dan Williams
a4624a0e75 util/iptables: fix cross-build failures due to syscall.Flock()
Fixes: https://github.com/kubernetes/kubernetes/issues/45554
2017-05-14 23:37:52 -05:00
Cao Shufeng
a393e95018 Remove leaked tmp file in unit tests 2017-05-08 18:07:02 +08:00
Kubernetes Submit Queue
15f0468986 Merge pull request #44895 from dcbw/iptables-restore-manual-locking
Automatic merge from submit-queue

util/iptables: grab iptables locks if iptables-restore doesn't support --wait

When iptables-restore doesn't support --wait (which < 1.6.2 don't), it may
conflict with other iptables users on the system, like docker, because it
doesn't acquire the iptables lock before changing iptables rules. This causes
sporadic docker failures when starting containers.

To ensure those don't happen, essentially duplicate the iptables locking
logic inside util/iptables when we know iptables-restore doesn't support
the --wait option.

Unfortunately iptables uses two different locking mechanisms, one until
1.4.x (abstract socket based) and another from 1.6.x (/run/xtables.lock
flock() based).  We have to grab both locks, because we don't know what
version of iptables-restore exists since iptables-restore doesn't have
a --version option before 1.6.2.  Plus, distros (like RHEL) backport the
/run/xtables.lock patch to 1.4.x versions.

Related: https://github.com/kubernetes/kubernetes/pull/43575
See also: https://github.com/openshift/origin/pull/13845
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1417234

@kubernetes/rh-networking @kubernetes/sig-network-misc @eparis @knobunc @danwinship @thockin @freehan
2017-05-06 15:17:21 -07:00
Kubernetes Submit Queue
79a2e359cd Merge pull request #44024 from zhangxiaoyu-zidif/egnew
Automatic merge from submit-queue

fix the typos of e.g.

fix the typos of e.g.


**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-05-05 23:41:33 -07:00
Dan Williams
1067a22885 iptables: add OWNERS file 2017-05-05 23:34:18 -05:00
Kubernetes Submit Queue
17d33ea82e Merge pull request #44830 from NickrenREN/remove-NodeLegacyHostIP
Automatic merge from submit-queue

Remove deprecated NodeLegacyHostIP

**Release note**:
```release-note
Remove deprecated node address type `NodeLegacyHostIP`.
```

ref #44807
2017-05-05 15:38:58 -07:00
Christopher M. Luciano
bafabcbb44
Consolidate sysctl commands for kubelet
These commands are important enough to be in the Kubelet itself.
By default, Ubuntu 14.04 and Debian Jessie have these set to 200 and
20000. Without this setting, nodes are limited in the number of
containers that they can start.
2017-05-02 12:15:01 -07:00
Andy Goldstein
43cb024402 Add kube-proxy config file support
Add support for configuring kube-proxy via a config file instead of
command line flags.
2017-05-01 18:02:47 -04:00
Mike Danese
ff4a814cce migrate set generation to go genrule
and move ./types to staging
2017-05-01 11:11:14 -07:00
Clayton Coleman
49209b3394
Make timeouts in the Kubelet slightly offset to aid debugging
Several of these loops overlap, and when they are the reason a failure
is happening it is difficult to sort them out. Slighly misalign these
loops to make their impact obvious.
2017-04-28 12:00:28 -04:00
Dan Williams
c7677a4753 util/iptables: grab iptables locks if iptables-restore doesn't support --wait
When iptables-restore doesn't support --wait (which < 1.6.2 don't), it may
conflict with other iptables users on the system, like docker, because it
doesn't acquire the iptables lock before changing iptables rules. This causes
sporadic docker failures when starting containers.

To ensure those don't happen, essentially duplicate the iptables locking
logic inside util/iptables when we know iptables-restore doesn't support
the --wait option.

Unfortunately iptables uses two different locking mechanisms, one until
1.4.x (abstract socket based) and another from 1.6.x (/run/xtables.lock
flock() based).  We have to grab both locks, because we don't know what
version of iptables-restore exists since iptables-restore doesn't have
a --version option before 1.6.2.  Plus, distros (like RHEL) backport the
/run/xtables.lock patch to 1.4.x versions.

Related: https://github.com/kubernetes/kubernetes/pull/43575
See also: https://github.com/openshift/origin/pull/13845
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1417234
2017-04-25 14:26:04 -05:00
Mike Danese
e48a4f0af7 fix various bad tests 2017-04-25 11:23:33 -07:00
Kubernetes Submit Queue
9ac111c5f9 Merge pull request #44825 from rata/remove-unused-abstraction
Automatic merge from submit-queue (batch tested with PRs 40060, 44860, 44865, 44825, 44162)

Remove unused chmod/chown abstractions

**What this PR does / why we need it**: Simplifies the code

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-04-24 23:07:44 -07:00
Kubernetes Submit Queue
902d53d5e7 Merge pull request #43575 from dcbw/iptables-restore-lock
Automatic merge from submit-queue (batch tested with PRs 43575, 44672)

util/iptables: check for and use new iptables-restore 'wait' argument

iptables-restore did not previously perform any locking, meaning that
when callers (like kube-proxy) asked iptables-restore to write large
numbers of rules, the iptables-restore process might run in parallel
with other 'iptables' invocations in kubelet (hostports), docker,
and other software.  This causes errors like:

"CNI request failed with status 400: 'Failed to ensure that nat chain
POSTROUTING jumps to MASQUERADE: error checking rule: exit status 4:
iptables: Resource temporarily  unavailable."

or from Docker:

"Failed to allocate and map port 1095-1095: iptables failed:
iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 1095
-j DNAT --to-destination 10.1.0.2:1095 ! -i lbr0: iptables:
Resource temporarily unavailable.\n (exit status 4)"

iptables-restore "wait" functionality was added in iptables git
commit 999eaa241212d3952ddff39a99d0d55a74e3639e which
is not yet in a release.

See also https://bugzilla.redhat.com/show_bug.cgi?id=1417234

@eparis @knobunc @kubernetes/rh-networking @kubernetes/sig-network-misc @freehan @thockin @brendandburns
2017-04-24 10:33:58 -07:00
NickrenREN
7d00e5cfb6 remove deprecated NodeLegacyHostIP 2017-04-24 11:01:25 +08:00
Rodrigo Campos
2eb1583e4b Remove unused chmod/chown abstractions
These abstractions have not been used since I first hacked into
kubernetes, so let's just remove them and add them back if it's needed
later.
2017-04-23 20:25:23 -03:00
Dan Williams
4cd6d34a0b util/iptables: check for and use new iptables-restore 'wait' argument
iptables-restore did not previously perform any locking, meaning that
when callers (like kube-proxy) asked iptables-restore to write large
numbers of rules, the iptables-restore process might run in parallel
with other 'iptables' invocations in kubelet (hostports), docker,
and other software.  This causes errors like:

"CNI request failed with status 400: 'Failed to ensure that nat chain
POSTROUTING jumps to MASQUERADE: error checking rule: exit status 4:
iptables: Resource temporarily  unavailable."

or from Docker

"Failed to allocate and map port 1095-1095: iptables failed:
iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 1095
-j DNAT --to-destination 10.1.0.2:1095 ! -i lbr0: iptables:
Resource temporarily unavailable.\n (exit status 4)"

iptables-restore "wait" functionality was added in iptables git
commit 999eaa241212d3952ddff39a99d0d55a74e3639e but is NOT YET
in a released version of iptables.

See also https://bugzilla.redhat.com/show_bug.cgi?id=1417234
2017-04-20 13:38:44 -05:00
Kubernetes Submit Queue
409b0a6f5d Merge pull request #44222 from fabianofranz/better_generic_getters_and_describers
Automatic merge from submit-queue (batch tested with PRs 44222, 44614, 44292, 44638)

Smarter generic getters and describers

Makes printers and describers smarter for generic resources.

This traverses unstructured objects and prints their attributes for generic resources (TPR, federated API, etc) in `kubectl get` and `kubectl describe`. Makes use of the object's field names to come up with a best guess for describer labels and get headers, and field value types to understand how to better print it, indent, etc.

A nice intermediate solution while we don't have [get and describe extensions](https://github.com/kubernetes/community/pull/308).

Examples:

```
$ kubectl get serviceclasses
NAME                    KIND                                          BINDABLE   BROKER NAME   OSB GUID
user-provided-service   ServiceClass.v1alpha1.servicecatalog.k8s.io   false      ups-broker    4f6e6cf6-ffdd-425f-a2c7-3c9258ad2468
```

```
$ kubectl describe serviceclasses/user-provided-service
Name:		user-provided-service
Namespace:	
Labels:		<none>
Annotations:	FOO=BAR
		openshift.io/deployment.phase=test
OSB Metadata:	<nil>
Kind:		ServiceClass
Metadata:
  Self Link:		/apis/servicecatalog.k8s.io/v1alpha1/serviceclassesuser-provided-service
  UID:			1509bd96-1b05-11e7-98bd-0242ac110006
  Resource Version:	256
  Creation Timestamp:	2017-04-06T20:10:29Z
Broker Name:		ups-broker
Bindable:		false
Plan Updatable:		false
OSB GUID:		4f6e6cf6-ffdd-425f-a2c7-3c9258ad2468
API Version:		servicecatalog.k8s.io/v1alpha1
Plans:
  Name:		default
  OSB GUID:	86064792-7ea2-467b-af93-ac9694d96d52
  OSB Free:	true
  OSB Metadata:	<nil>
Events:		<none>
```

**Release note**:
```release-note
Improved output on 'kubectl get' and 'kubectl describe' for generic objects.
```
PTAL @pmorie @pwittrock @kubernetes/sig-cli-pr-reviews
2017-04-18 17:41:59 -07:00
Kubernetes Submit Queue
a1684fea80 Merge pull request #42085 from cblecker/gofmt-fix
Automatic merge from submit-queue (batch tested with PRs 40055, 42085, 44509, 44568, 43956)

Fix gofmt errors

**What this PR does / why we need it**:
There were some gofmt errors on master. Ran the following to fix:
```
hack/verify-gofmt.sh | grep ^diff | awk '{ print $2 }' | xargs gofmt -w -s
```

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: none

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-04-17 15:39:07 -07:00
Fabiano Franz
151770c8fd Smarter describer for generic resources 2017-04-17 15:42:47 -03:00
Kubernetes Submit Queue
32e927f4d8 Merge pull request #44506 from caesarxuchao/fix-cross-build
Automatic merge from submit-queue

fix cross-build

Fix https://github.com/kubernetes/kubernetes/pull/41543#issuecomment-294207870
2017-04-14 14:51:06 -07:00
Chao Xu
59396e6b13 fix cross-build 2017-04-14 11:45:11 -07:00
Mike Danese
a05c3c0efd autogenerated 2017-04-14 10:40:57 -07:00
Kubernetes Submit Queue
4653a9b280 Merge pull request #41543 from dshulyak/decouple_remotecommand
Automatic merge from submit-queue (batch tested with PRs 44406, 41543, 44071, 44374, 44299)

Decouple remotecommand

Refactored unversioned/remotecommand to decouple it from undesirable dependencies:

- term package now is not required, and functionality required to resize terminal size can be plugged in directly in kubectl
- in order to remove dependency on kubelet package - constants from kubelet/server/remotecommand were moved to separate util package (pkg/util/remotecommand)
- remotecommand_test.go moved to pkg/client/tests module
2017-04-13 19:52:05 -07:00
Dmitry Shulyak
f50480c714 Decouple remotecommand client from term/kubelet dependencies
In order to move client/unversioned/remotecommand to client-go as a followup
for this change we have to decouple it from tons of dependencies
2017-04-13 15:56:40 +03:00
Avesh Agarwal
0a9cdbb587 Auto-generated stuff. 2017-04-06 16:15:34 -04:00
Avesh Agarwal
af53794854 Pod toleration restriction plugin with taints and tolerations. 2017-04-06 16:15:34 -04:00
zhangxiaoyu-zidif
932ece5cfd e3d534b2c4 2017-04-04 14:16:34 +08:00
Christoph Blecker
6681835b0c
Fix gofmt errors 2017-03-28 17:12:04 -07:00
Kubernetes Submit Queue
1ed77078da Merge pull request #43378 from jsafrane/fix-partition-mount
Automatic merge from submit-queue (batch tested with PRs 43378, 43216, 43384, 43083, 43428)

Do not reformat devices with partitions

`lsblk` reports FSTYPE of devices with partition tables as empty string `""`,
which is indistinguishable from empty devices. We must look for dependent
devices (i.e. partitions) to see that the device is really empty and report
error otherwise.

The main point of this patch is to run `lsblk` without `"-n"`, i.e. print all
dependent devices and check it output.

Sample output:
```
  FirstSeen     LastSeen        Count   From                                    SubObjectPath   Type            Reason          Message
  ---------     --------        -----   ----                                    -------------   --------        ------          -------
  10s           10s             1       default-scheduler                                       Normal          Scheduled       Successfully assigned testpod to ip-172-18-11-149.ec2.internal
  2s            2s              1       kubelet, ip-172-18-11-149.ec2.internal                  Warning         FailedMount     MountVolume.MountDevice failed for volume "kubernetes.io/aws-ebs/vol-0fa9da8b91913b187" (spec.Name: "vol") pod "b74f68c5-0d6a-11e7-9233-0e11251010c0" (UID: "b74f68c5-0d6a-11e7-9233-0e11251010c0") with: failed to mount the volume as "ext4", it already contains unknown data, probably partitions. Mount error: mount failed: exit status 32
Mounting command: mount
Mounting arguments: /dev/xvdbb /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/vol-0fa9da8b91913b187 ext4 [defaults]
Output: mount: wrong fs type, bad option, bad superblock on /dev/xvdbb,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

```

Without this patch, the device would be reformatted and all data in the device partitions would be lost.

Fixes #13212

Release note:
```release-note
NONE
```

@kubernetes/sig-storage-pr-reviews
2017-03-25 21:22:22 -07:00
Kubernetes Submit Queue
d14854fd5c Merge pull request #37698 from jsafrane/remove-all-filesystems
Automatic merge from submit-queue (batch tested with PRs 41139, 41186, 38882, 37698, 42034)

Make kubelet never delete files on mounted filesystems

With bug #27653, kubelet could remove mounted volumes and delete user data.
The bug itself is fixed, however our trust in kubelet is significantly lower.
Let's add an extra version of RemoveAll that does not cross mount boundary
(rm -rf --one-file-system).

It calls lstat(path) three times for each removed directory - once in
RemoveAllOneFilesystem and twice in IsLikelyNotMountPoint, however this way
it's platform independent and the directory that is being removed by kubelet
should be almost empty.
2017-03-24 12:33:27 -07:00
Jan Safranek
4cf36b8b39 Do not reformat devices with partitions
lsblk reports FSTYPE of devices with partition tables as empty string "",
which is indistinguishable from empty devices. We must look for dependent
devices (i.e. partitions) to see that the device is really empty and report
error otherwise.

I checked that LVM, LUKS and MD RAID have their own FSTYPE in lsblk output,
so it should be only a partition table that has empty FSTYPE.

The main point of this patch is to run lsblk without "-n", i.e. print all
dependent devices and check if they're there.
2017-03-20 13:08:13 +01:00
lvjiangzhao
92fff6c9c4 fix typo 2017-03-17 13:42:12 +08:00
Yifan Gu
a489bd2674 pkg/util/flock: Fix the flock so it actually locks.
With this PR, the second call to `Acquire()` will block unless the lock is released (process exits).
Also removed the memory mutex in the previous code since we don't need `Release()` here so no need to save and protect the local fd.

Fix #42929.
2017-03-13 14:24:59 -07:00
Kubernetes Submit Queue
44a7be98fd Merge pull request #41618 from JiangtianLi/k8swin
Automatic merge from submit-queue (batch tested with PRs 42316, 41618, 42201, 42113, 42191)

Support unqualified and partially qualified domain name in DNS query in Windows kube-proxy

**What this PR does / why we need it**:
In Windows container networking, --dns-search is not currently supported on Windows Docker. Besides, even with --dns-suffix, inside Windows container DNS suffix is not appended to DNS query names. That makes unqualified domain name or partially qualified domain name in DNS query not able to resolve.

This PR provides a solution to resolve unqualified domain name or partially qualified domain name in DNS query for Windows container in Windows kube-proxy. It uses well-known Kubernetes DNS suffix as well host DNS suffix search list to append to the name in DNS query. DNS packet in kube-proxy UDP stream is modified as appropriate.

This PR affects the Windows kube-proxy only.

**Special notes for your reviewer**:
This PR is based on top of Anthony Howe's commit 48647fb, 0e37f0a and 7e2c71f which is already included in the PR 41487. Please only review commit b9dfb69.

**Release note**:

```release-note
Add DNS suffix search list support in Windows kube-proxy.
```
2017-03-01 07:48:16 -08:00
Kubernetes Submit Queue
81d01a84e0 Merge pull request #41944 from jingxu97/Feb/mounter
Automatic merge from submit-queue (batch tested with PRs 35094, 42095, 42059, 42143, 41944)

Use chroot for containerized mounts

This PR is to modify the containerized mounter script to use chroot
instead of rkt fly. This will avoid the problem of possible large number
of mounts caused by rkt containers if they are not cleaned up.
2017-02-28 09:20:21 -08:00
Jan Safranek
d7d039dba2 Make kubelet never delete files on mounted filesystems
With bug #27653, kubelet could remove mounted volumes and delete user data.
The bug itself is fixed, however our trust in kubelet is significantly lower.
Let's add an extra version of RemoveAll that does not cross mount boundary
(rm -rf --one-file-system).

It calls lstat(path) three times for each removed directory - once in
RemoveAllOneFilesystem and twice in IsLikelyNotMountPoint, however this way
it's platform independent and the directory that is being removed by kubelet
should be almost empty.
2017-02-28 14:32:07 +01:00
Jiangtian Li
b9dfb69dd7 Fix DNS suffix search list issue for Windows container and workaround in kube-proxy.
kube-proxy iterates over DNS suffix search list and appends to DNS query for client.
2017-02-27 19:25:46 -08:00
Anthony Howe
0e37f0a890 cleanup proxier 2017-02-27 16:00:49 -08:00
Minhan Xia
f006c8bcd3 teach kubenet to use annotation instead of pod object for traffic shaper 2017-02-27 10:11:09 -08:00
Kubernetes Submit Queue
bf984aa328 Merge pull request #42120 from kevin-wangzefeng/kubectl-taint-respect-noexecute
Automatic merge from submit-queue (batch tested with PRs 41116, 41804, 42104, 42111, 42120)

make kubectl taint command respect effect NoExecute

**What this PR does / why we need it**:
Part of feature forgiveness implementation, make kubectl taint command respect effect NoExecute.

**Which issue this PR fixes**: 
Related Issue: #1574
Related PR: #39469

**Special notes for your reviewer**:

**Release note**:

```release-note
make kubectl taint command respect effect NoExecute
```
2017-02-27 04:10:30 -08:00
Kubernetes Submit Queue
d1f5331102 Merge pull request #41804 from chakri-nelluri/flex
Automatic merge from submit-queue (batch tested with PRs 41116, 41804, 42104, 42111, 42120)

Add support for attacher/detacher interface in Flex volume

Add support for attacher/detacher interface in Flex volume
This change breaks backward compatibility and requires to be release noted.

```release-note
Flex volume plugin is updated to support attach/detach interfaces. It broke backward compatibility. Please update your drivers and implement the new callouts. 
```
2017-02-27 04:10:25 -08:00
Kevin
f4d3339612 make kubectl respect taint effect NoExecute 2017-02-26 22:41:57 +08:00
Kubernetes Submit Queue
258a5cb3f1 Merge pull request #40665 from brendandburns/i18n
Automatic merge from submit-queue (batch tested with PRs 40665, 41094, 41351, 41721, 41843)

Update i18n tools and process.

@fabianofranz @zen @kubernetes/sig-cli-pr-reviews 

This is an update to the translation process based on feedback from folks.

The main changes are:
   * `msgctx` is being removed from the files.
   * String wrapping and string extraction have been separated.
   * More tools from the `gettext` family of tools are being used
   * Extracted strings are being sorted for canonical ordering
   * A `.pot` template has been added.
2017-02-25 03:56:51 -08:00
Kubernetes Submit Queue
a426904009 Merge pull request #31515 from jsafrane/format-error
Automatic merge from submit-queue (batch tested with PRs 41714, 41510, 42052, 41918, 31515)

Show specific error when a volume is formatted by unexpected filesystem.

kubelet now detects that e.g. xfs volume is being mounted as ext3 because of
wrong volume.Spec.

Mount error is left in the error message to diagnose issues with mounting e.g.
'ext3' volume as 'ext4' - they are different filesystems, however kernel should
mount ext3 as ext4 without errors.

Example kubectl describe pod output:

```
  FirstSeen     LastSeen        Count   From                                    SubobjectPath   Type            Reason          Message
  41s           3s              7       {kubelet ip-172-18-3-82.ec2.internal}                   Warning         FailedMount     MountVolume.MountDevice failed for volume "kubernetes.io/aws-ebs/aws://us-east-1d/vol-ba79c81d" (spec.Name: "pvc-ce175cbb-6b82-11e6-9fe4-0e885cca73d3") pod "3d19cb64-6b83-11e6-9fe4-0e885cca73d3" (UID: "3d19cb64-6b83-11e6-9fe4-0e885cca73d3") with: failed to mount the volume as "ext4", it's already formatted with "xfs". Mount error: mount failed: exit status 32
Mounting arguments: /dev/xvdba /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/aws/us-east-1d/vol-ba79c81d ext4 [defaults]
Output: mount: wrong fs type, bad option, bad superblock on /dev/xvdba,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
```
2017-02-25 02:17:57 -08:00
Chakravarthy Nelluri
0d2af70e95 Add support for attacher/detacher interface in Flex volume 2017-02-24 20:18:06 -05:00
Jing Xu
ac22416835 Use chroot for containerized mounts
This PR is to modify the containerized mounter script to use chroot
instead of rkt fly. This will avoid the problem of possible large number
of mounts caused by rkt containers if they are not cleaned up.
2017-02-24 13:46:26 -08:00
Brendan Burns
be26836f64 Update extraction script, sort messages, add .pot file. 2017-02-23 18:53:00 +00:00
Anthony Howe
48647fb9b5 add tcp or udp proxy for service addresses 2017-02-22 19:42:33 -08:00
Tim Hockin
c88b2d9bf4 Merge pull request #41029 from jingxu97/Feb/mountonwer
Add owner file in /pkg/util/mount package
2017-02-21 16:29:51 -06:00
Kubernetes Submit Queue
98962212a3 Merge pull request #41330 from feiskyer/fix
Automatic merge from submit-queue (batch tested with PRs 41706, 39063, 41330, 41739, 41576)

Fix regex match doc of procfs.PidOf

Fixes #41247.

cc @bboreham
2017-02-21 02:24:41 -08:00
Aaron Crickenberger
ddc8d8c400 fix FakeMounter to work with OSX
OSX 10.11.x has `/var` symlinked to `/private/var`, which was tripping
up logic in `mount.GetMountRefs`

This fixes unit tests for pkg/volume/fc and pkg/volume/iscsi
2017-02-15 16:14:33 -08:00
Harry Zhang
3bdc3f25ec Use fnv.New32a() in hash instead adler32 2017-02-15 14:03:54 +08:00
Pengfei Ni
55bfc8d7e6 Fix regex match doc of procfs.PidOf 2017-02-13 23:10:14 +08:00
Jan Safranek
c8df30973b Show specific error when a volume is formatted by unexpected filesystem.
kubelet now detects that e.g. xfs volume is being mounted as ext3 because of
wrong volume.Spec.

Mount error is left in the error message to diagnose issues with mounting e.g.
'ext3' volume as 'ext4' - they are different filesystems, however kernel should
mount ext3 as ext4 without errors.
2017-02-13 12:15:34 +01:00
Jing Xu
fb808c3417 Add owner file in /pkg/util/mount package
Add owner file in /pkg/util/mount
2017-02-06 10:51:26 -08:00
Kubernetes Submit Queue
5eef12700c Merge pull request #40645 from brendandburns/fr
Automatic merge from submit-queue

Add initial french translations for kubectl

Add initial French translations, mostly as an example of how to add a new language.

@fabianofranz @kubernetes/sig-cli-pr-reviews
2017-01-31 21:27:52 -08:00
Kubernetes Submit Queue
0c15857a1d Merge pull request #40145 from nebril/iptables-util-unit-tests
Automatic merge from submit-queue (batch tested with PRs 38443, 40145, 40701, 40682)

pkg/util/iptables missing unit tests
2017-01-30 20:59:42 -08:00
deads2k
c9a008dff3 move util/intstr to apimachinery 2017-01-30 12:46:59 -05:00
deads2k
2c1c0f3f72 move workqueue to client-go 2017-01-30 09:08:21 -05:00
Brendan Burns
45ade78eb3 Add initial french translations as an example. 2017-01-29 22:56:48 -08:00
Dr. Stefan Schimanski
44ea6b3f30 Update generated files 2017-01-29 21:41:45 +01:00
Dr. Stefan Schimanski
79adb99a13 pkg/api: move Semantic equality to k8s.io/apimachinery/pkg/api/equality 2017-01-29 21:41:45 +01:00
Dr. Stefan Schimanski
bc6fdd925d pkg/api/resource: move to apimachinery 2017-01-29 21:41:44 +01:00
Dr. Stefan Schimanski
a12c661773 pkg/util: move trace.go to k8s.io/apiserver/pkg/util/trace 2017-01-29 21:41:44 +01:00
Dr. Stefan Schimanski
674d78b2ab pkg/util: move httpstream to k8s.io/apimachinery 2017-01-27 15:10:42 -05:00
Dr. Stefan Schimanski
a0137e9b28 Update generated files 2017-01-25 19:49:45 +01:00
Dr. Stefan Schimanski
e57d7b1c60 third_party/forked/golang/netutil: move to k8s.io/apimachinery 2017-01-25 19:45:10 +01:00
Dr. Stefan Schimanski
d216ebef73 pkg/util: move proxy into k8s.io/apiserver 2017-01-25 19:45:10 +01:00
Dr. Stefan Schimanski
d7eb3b6870 pkg/util: move uuid and strategicpatch into k8s.io/apimachinery 2017-01-25 19:45:09 +01:00
Dr. Stefan Schimanski
8541304582 pkg/util: move CompileRegex helper into apiserver
This is a little helper, not a generic utility of general value.
2017-01-25 17:21:12 +01:00
Dr. Stefan Schimanski
3bd00afaf9 pkg/util: move Trie to k8s.io/apiserver 2017-01-25 17:21:12 +01:00
Dr. Stefan Schimanski
2b8e938128 Update generated files 2017-01-24 20:56:03 +01:00
Dr. Stefan Schimanski
82826ec273 pkg/util/flag: move to k8s.io/apiserver 2017-01-24 20:56:03 +01:00
Dr. Stefan Schimanski
a6b2ebb50c pkg/flag: make feature gate extensible and split between generic and kube 2017-01-24 20:56:03 +01:00
Dr. Stefan Schimanski
56d60cfae6 pkg/util: move flags from pkg/util/config to pkg/util/flags 2017-01-24 20:56:03 +01:00
deads2k
5a8f075197 move authoritative client-go utils out of pkg 2017-01-24 08:59:18 -05:00
Clayton Coleman
4f865efdb9
Create meta/internalversion for ListOptions
Move over only the conversions that are needed, create a new scheme that
is private to meta and only accessible via ParameterCodec. Move half of
pkg/util/labels/.readonly to pkg/apis/meta/v1/labels.go
2017-01-23 16:53:18 -05:00
Kubernetes Submit Queue
fce60637ae Merge pull request #39341 from smarterclayton/termination_log
Automatic merge from submit-queue

Improve TerminationMessagePath to be more flexible

* Support `terminationMessagePolicy: fallbackToLogsOnError` which allows pod authors to get useful information from containers as per kubernetes/community#154
* Set an upper bound on the size of the termination message path or log output to prevent callers from DoSing the master
* Add tests for running as root, non-root, and for the new terminationMessagePolicy cases.

I set the limit to 4096 bytes, but this may be too high for large pod containers.  Probably need to set an absolute bound, i.e. max message size allowed is 20k total, and we truncate if we're above that limit.

Fixes #31839, #23569

```release-note
A new field `terminationMessagePolicy` has been added to containers that allows a user to request `FallbackToLogsOnError`, which will read from the container's logs to populate the termination message if the user does not write to the termination message log file.  The termination message file is now properly readable for end users and has a maximum size (4k bytes) to prevent abuse.  Each pod may have up to 12k bytes of termination messages before the contents of each will be truncated.
```
2017-01-23 12:13:33 -08:00
Antoine Pelisse
62af7dd33d OWNERS: Update latest OWNERS files
These files have been created lately, so we don't have much information
about them anyway, so let's just:
- Remove assignees and make them approvers
- Copy approves as reviewers
2017-01-23 10:05:48 -08:00
Clayton Coleman
2bb2604f0b
Limit the size of the termination log and allow log input
Enforce the following limits:

12kb for total message length in container status
4kb for the termination message path file
2kb or 80 lines (whichever is shorter) from the log on error

Fallback to log output if the user requests it.
2017-01-23 12:26:38 -05:00
Clayton Coleman
e6d35b0362
generated: api 2017-01-23 12:26:35 -05:00
deads2k
335ef747d6 mechanical 2017-01-23 10:03:55 -05:00
deads2k
9c6a522882 mechanical results of client auth plugin 2017-01-20 08:05:18 -05:00
deads2k
2f51cc4ce4 move client auth plugins 2017-01-20 08:04:53 -05:00
deads2k
ee6752ef20 find and replace 2017-01-20 08:04:53 -05:00
deads2k
32a2c2d05c remove old restclient 2017-01-20 07:56:35 -05:00
Maciej Kwiek
2220c6dfbf pkg/util/iptables missing unit tests
Added tests for Save, SaveAll, Restore and RestoreAll
2017-01-20 10:50:45 +01:00
deads2k
c47717134b move utils used in restclient to client-go 2017-01-19 07:55:14 -05:00
Kubernetes Submit Queue
723fa08767 Merge pull request #39324 from wojtek-t/change_patch_api
Automatic merge from submit-queue

Prepare for using optimized conversion to/from map[string]interface{} in Patch operation

Ref #39017
2017-01-19 00:50:12 -08:00
Andrew Rynhard
f46bf42ba0 Add authorization mode to kubeadm 2017-01-18 14:02:53 -08:00
Jordan Liggitt
e49554501f
Use versioned Taint/Toleration/AllowPods objects when marshalling 2017-01-18 12:52:14 -05:00
Kubernetes Submit Queue
6dfe5c49f6 Merge pull request #38865 from vwfs/ext4_no_lazy_init
Automatic merge from submit-queue

Enable lazy initialization of ext3/ext4 filesystems

**What this PR does / why we need it**: It enables lazy inode table and journal initialization in ext3 and ext4.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #30752, fixes #30240

**Release note**:
```release-note
Enable lazy inode table and journal initialization for ext3 and ext4
```

**Special notes for your reviewer**:
This PR removes the extended options to mkfs.ext3/mkfs.ext4, so that the defaults (enabled) for lazy initialization are used.

These extended options come from a script that was historically located at */usr/share/google/safe_format_and_mount* and later ported to GO so this dependency to the script could be removed. After some search, I found the original script here: https://github.com/GoogleCloudPlatform/compute-image-packages/blob/legacy/google-startup-scripts/usr/share/google/safe_format_and_mount

Checking the history of this script, I found the commit [Disable lazy init of inode table and journal.](4d7346f7f5). This one introduces the extended flags with this description:
```
Now that discard with guaranteed zeroing is supported by PD,
initializing them is really fast and prevents perf from being affected
when the filesystem is first mounted.
```

The problem is, that this is not true for all cloud providers and all disk types, e.g. Azure and AWS. I only tested with magnetic disks on Azure and AWS, so maybe it's different for SSDs on these cloud providers. The result is that this performance optimization dramatically increases the time needed to format a disk in such cases.

When mkfs.ext4 is told to not lazily initialize the inode tables and the check for guaranteed zeroing on discard fails, it falls back to a very naive implementation that simply loops and writes zeroed buffers to the disk. Performance on this highly depends on free memory and also uses up all this free memory for write caching, reducing performance of everything else in the system. 

As of https://github.com/kubernetes/kubernetes/issues/30752, there is also something inside kubelet that somehow degrades performance of all this. It's however not exactly known what it is but I'd assume it has something to do with cgroups throttling IO or memory. 

I checked the kernel code for lazy inode table initialization. The nice thing is, that the kernel also does the guaranteed zeroing on discard check. If it is guaranteed, the kernel uses discard for the lazy initialization, which should finish in a just few seconds. If it is not guaranteed, it falls back to using *bio*s, which does not require the use of the write cache. The result is, that free memory is not required and not touched, thus performance is maxed and the system does not suffer.

As the original reason for disabling lazy init was a performance optimization and the kernel already does this optimization by default (and in a much better way), I'd suggest to completely remove these flags and rely on the kernel to do it in the best way.
2017-01-18 09:09:52 -08:00
Wojciech Tyczynski
d08abdb187 Allow for returning map[string]interface{} from patch. 2017-01-18 11:53:30 +01:00
Kubernetes Submit Queue
d357a72161 Merge pull request #40039 from timstclair/api-redirect
Automatic merge from submit-queue

Enable streaming proxy redirects by default (beta)

Prerequisite to moving CRI to Beta.

I'd like to enable this early in our 1.6 cycle to get plenty of test coverage before release.

@yujuhong @liggitt 

```release-note
Follow redirects for streaming requests (exec/attach/port-forward) in the apiserver by default (alpha -> beta).
```
2017-01-17 16:18:48 -08:00
Clayton Coleman
9a2a50cda7
refactor: use metav1.ObjectMeta in other types 2017-01-17 16:17:19 -05:00
Tim St. Clair
b42663bb14
Enable streaming proxy redirects by default (beta) 2017-01-17 12:56:03 -08:00
Kubernetes Submit Queue
e5fe366eaf Merge pull request #39894 from deads2k/client-03-remove-old
Automatic merge from submit-queue

Remove packages which are now apimachinery

Removes all the content from the packages that were moved to `apimachinery`.  This will force all vendoring projects to figure out what's wrong.  I had to leave many empty marker packages behind to have verify-godep succeed on vendoring heapster.

@sttts straight deletes and simple adds
2017-01-17 12:00:00 -08:00
deads2k
26c46971f2 move PatchType to apimachinery 2017-01-17 08:32:05 -05:00
deads2k
a6b655fb6a add back just enough empty packages to allow heapster cycles to succeed 2017-01-17 08:07:30 -05:00
deads2k
67fac70ea3 remove files moved to apimachinery 2017-01-17 07:46:34 -05:00
deads2k
8686d67c80 move pkg/util/rand 2017-01-16 16:04:03 -05:00
deads2k
7993e7c8aa move openapi types to pkg/openapi 2017-01-16 13:40:14 -05:00
Kubernetes Submit Queue
6defc30337 Merge pull request #39882 from deads2k/api-59-errors
Automatic merge from submit-queue (batch tested with PRs 38592, 39949, 39946, 39882)

move api/errors to apimachinery

`pkg/api/errors` is a set of helpers around `meta/v1.Status` that help to create and interpret various apiserver errors.  Things like `.NewNotFound` and `IsNotFound` pairings.  This pull moves it into apimachinery for use by the clients and servers.

@smarterclayton @lavalamp First commit is the move plus minor fitting.  Second commit is straight replace and generation.
2017-01-16 10:37:42 -08:00
deads2k
77b4d55982 mechanical 2017-01-16 09:35:12 -05:00
Dr. Stefan Schimanski
918868b115 genericapiserver: cut off certificates api dependency 2017-01-16 14:10:59 +01:00
Kubernetes Submit Queue
eb9f953496 Merge pull request #39876 from deads2k/generic-20-deps-03
Automatic merge from submit-queue

move more things to apiserver

```
pkg/genericapiserver/api/handlers/negotiation/ -> apiserver/pkg/handlers/negotiation
pkg/genericapiserver/api/metrics -> apiserver/pkg/metrics
pkg/genericapiserver/api/request -> apiserver/pkg/request
pkg/util/wsstream -> apiserver/pkg/util/wsstream
plugin/pkg/auth/authenticator/request/headerrequest -> apiserver/pkg/authentication/request/headerrequest
plugin/pkg/webhook -> apiserver/pkg/webhook
```

and mechanicals.

`k8s.io/kubernetes/pkg/genericapiserver/routes/data/swagger` needs to be sorted out.
2017-01-16 04:14:37 -08:00
Kubernetes Submit Queue
5723979b60 Merge pull request #39525 from kargakis/update-equality-helper
Automatic merge from submit-queue (batch tested with PRs 39807, 37505, 39844, 39525, 39109)

Update deployment equality helper

@mfojtik @janetkuo this is split out of https://github.com/kubernetes/kubernetes/pull/38714 to reduce the size of that PR, ptal
2017-01-13 13:40:45 -08:00
deads2k
31b6ba4e94 mechanicals 2017-01-13 16:33:09 -05:00
deads2k
81b073a5f5 move no k8s.io/kubernetes deps to apiserver 2017-01-13 16:26:58 -05:00
Kubernetes Submit Queue
212234ab3f Merge pull request #39807 from deads2k/client-02-client-go
Automatic merge from submit-queue

run staging client-go update

Chasing to see what real problems we have in staging-client-go.

@sttts you get similar results?
2017-01-13 13:21:19 -08:00
Kubernetes Submit Queue
a6fa5c2bfd Merge pull request #39814 from deads2k/api-58-multi-register
Automatic merge from submit-queue

replace global registry in apimachinery with global registry in k8s.io/kubernetes

We'd like to remove all globals, but our immediate problem is that a shared registry between k8s.io/kubernetes and k8s.io/client-go doesn't work.  Since client-go makes a copy, we can actually keep a global registry with other globals in pkg/api for now.

@kubernetes/sig-api-machinery-misc @lavalamp @smarterclayton @sttts
2017-01-13 12:37:02 -08:00
deads2k
633e9d98fc use apimachinery packages instead of client-go packages 2017-01-13 14:04:54 -05:00
deads2k
f1176d9c5c mechanical repercussions 2017-01-13 08:27:14 -05:00
Kubernetes Submit Queue
14362160ba Merge pull request #38665 from ymqytw/fix_list_of_primitives
Automatic merge from submit-queue (batch tested with PRs 39834, 38665)

Use parallel list for deleting items from a primitive list with merge strategy

Implemented parallel list for deleting items from a primitive list with merge strategy. Ref: [design doc](https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#list-of-primitives)

fixes #35163 and #32398

When using parallel list, we don't need to worry about version skew.
When an old APIServer gets a new patch like:
```yaml
metadata:
  $deleteFromPrimitiveList/finalizers:
  - b
  finalizers:
  - c
```
It won't fail and work as before, because the parallel list will be dropped during json decoding.

Remaining issue: There is no check when creating a set (primitive list with merge strategy). Duplicates may get in.
It happens in two cases:
1) Creation using POST
2) Creating a list that doesn't exist before using PATCH

Fixing the first case is the beyond the scope of this PR.
The second case can be fixed in this PR if we need that.

cc: @pwittrock @kubernetes/kubectl @kubernetes/sig-api-machinery 

```release-note
Fix issue around merging lists of primitives when using PATCH or kubectl apply.
```
2017-01-12 20:03:23 -08:00
ymqytw
9a8ef00888 add unit test 2017-01-12 15:01:38 -08:00
ymqytw
a3584f2c34 use parallel list for deleting items in a set 2017-01-12 15:01:16 -08:00
Kubernetes Submit Queue
ee49906c45 Merge pull request #39661 from NickrenREN/clientset-redundant-modify
Automatic merge from submit-queue

fix redundant alias clientset

remove redundant alias clientset
2017-01-12 13:29:16 -08:00
NickrenREN
ad1f0d30e0 delete ErrorTimeout() function and modify Umask() args
ErrorTimeout is never used
2017-01-12 11:05:30 +08:00
NickrenREN
a12dea14e0 fix redundant alias clientset 2017-01-12 10:21:05 +08:00
deads2k
c4fae4e690 mechanical repercussions 2017-01-11 15:20:36 -05:00
deads2k
5280c8d3ac moves of genericapiserver packages without dependencies 2017-01-11 15:06:38 -05:00
Michail Kargakis
6013186ac3 Update deployment equality helper 2017-01-11 18:34:12 +01:00
deads2k
6a4d5cd7cc start the apimachinery repo 2017-01-11 09:09:48 -05:00
Kubernetes Submit Queue
ebf1a533aa Merge pull request #38342 from ymqytw/make_SPatch_delete_all_duplicates
Automatic merge from submit-queue (batch tested with PRs 34488, 39511, 39619, 38342, 39491)

Make StrategicPatch delete all matching maps in a merging list

fixes #38332

```release-note
NONE
```

cc: @lavalamp @pwittrock
2017-01-10 16:07:16 -08:00
Kubernetes Submit Queue
7d2f798052 Merge pull request #39511 from zhouhaibing089/lru-time-ut
Automatic merge from submit-queue (batch tested with PRs 34488, 39511, 39619, 38342, 39491)

use fake clock in lruexpiration cache test

when the system clock is extremely slow(usually see in VMs), this [check](https://github.com/kubernetes/kubernetes/blob/master/pkg/util/cache/lruexpirecache.go#L74) might still return the value.  

```go
if c.clock.Now().After(e.(*cacheEntry).expireTime) {
	go c.remove(key)
	return nil, false
}
```

that means even we set the ttl to be 0 second, the after check might still be false(because the clock is too slow, and thus equals).

the change here helps to reduce flakes.
2017-01-10 16:07:12 -08:00
deads2k
e10d4cbd2e tolerate clock change in throttle testing 2017-01-09 14:03:09 -05:00
zhouhaibing089
f12a6c1618 use fake clock in lruexpiration cache test 2017-01-07 13:58:03 +08:00
ymqytw
03081a0003 make StrategicPatch delete all matching maps in a merging list 2017-01-06 15:52:00 -08:00
Jeff Grafton
20d221f75c Enable auto-generating sources rules 2017-01-05 14:14:13 -08:00
Kubernetes Submit Queue
13780bba82 Merge pull request #39433 from deads2k/generic-04-no-dep-01
Automatic merge from submit-queue

move no k8s.io/kubernetes dependencies round 01

Builds on the previous, adds unit testing the staging dir.

moves

 1. pkg/healthz -> apiserver/pkg/healthz
 2. pkg/util/flushwriter -> apiserver/pkg/util/flushwriter
 3. pkg/auth/authenticator -> apiserver/pkg/authentication/authenticator
 4. pkg/auth/authorizer -> apiserver/pkg/authorization/authorizer

@sttts
2017-01-05 10:18:54 -08:00
Kubernetes Submit Queue
4422ddecba Merge pull request #39432 from deads2k/generic-06-strings
Automatic merge from submit-queue

snip pkg/util/strings dependency

The `pkg/util/strings` package looks to be largely used by volumes, which are independent of the bits used by genericapiserver which aren't used by anyone else.  This moves the single function (used no where else) to its point of use.

@sttts
2017-01-05 08:32:38 -08:00
deads2k
b813e98a26 move no k8s.io/kubernetes dependencies round one 2017-01-05 11:09:42 -05:00
deads2k
37c0c14215 snip pkg/util/strings dependency 2017-01-05 09:42:42 -05:00
deads2k
1ebe759743 selectively move to client-go packages 2017-01-04 17:49:24 -05:00
Lucas Käldström
733393d800
Add a PrivateKeyFromFile method to certutil for parsing a private key from a file 2017-01-03 23:36:23 +02:00
Jeff Grafton
fae627dd65 Update generated for 2017 2017-01-01 23:11:09 -08:00
Mike Danese
161c391f44 autogenerated 2016-12-29 13:04:10 -08:00
Kubernetes Submit Queue
69ddd8eb27 Merge pull request #39247 from wojtek-t/optimize_controller_manager_memory
Automatic merge from submit-queue

Avoid unnecessary memory allocations

Low-hanging fruits in saving memory allocations. During our 5000-node kubemark runs I've see this:

ControllerManager:
- 40.17% k8s.io/kubernetes/pkg/util/system.IsMasterNode
- 19.04% k8s.io/kubernetes/pkg/controller.(*PodControllerRefManager).Classify

Scheduler:
- 42.74% k8s.io/kubernetes/plugin/pkg/scheduler/algrorithm/predicates.(*MaxPDVolumeCountChecker).filterVolumes

This PR is eliminating all of those.
2016-12-28 00:02:59 -08:00
Kubernetes Submit Queue
745928aa43 Merge pull request #39213 from k82cn/k8s_33936_master
Automatic merge from submit-queue (batch tested with PRs 38909, 39213)

Add TLS conf for Go1.7

fixes #33936 / #38596  in master
2016-12-27 16:14:12 -08:00
Wojciech Tyczynski
d1292a7397 Optimize memory allocations in controller manager 2016-12-27 16:11:11 +01:00
Wojciech Tyczynski
9a4828821d Retry connection reset by peer 2016-12-27 10:01:42 +01:00
Kubernetes Submit Queue
1ee7163308 Merge pull request #36802 from brendandburns/i18n
Automatic merge from submit-queue

Begin paths for internationalization in kubectl

This is just the first step, purposely simple so we can get the interface correct.

@kubernetes/sig-cli @deads2k
2016-12-24 13:09:12 -08:00
Klaus Ma
3bfe14b316 Add TLS conf for Go1.7 2016-12-24 15:36:45 +08:00
Brendan Burns
277306449b Add initial translation support. 2016-12-23 20:45:52 -08:00
Kubernetes Submit Queue
c200f27245 Merge pull request #38090 from xingzhou/kube-37654
Automatic merge from submit-queue (batch tested with PRs 38920, 38090)

Improve error message for name/label validation.

Instead of just providing regex in name/label validation error output, we need to add the naming rules of the name/label, which is more end-user readable.

Fixed #37654
2016-12-22 22:00:30 -08:00
Kubernetes Submit Queue
237be4b2be Merge pull request #38855 from gnufied/fix-variable-shadow-exp-backoff
Automatic merge from submit-queue (batch tested with PRs 36888, 38180, 38855, 38590)

Fix variable shadowing in exponential backoff when deleting volumes

While https://github.com/kubernetes/kubernetes/pull/38339 implemented exponential backoff on
volume deletion, that PR suffers from a minor bug when error thrown on volume deletion is anything other than `VolumeInUse` errors - in which case exponential backoff will not work.

This PR fixes that. This PR also makes unit tests more deterministic because exponential backoff changed the way operations are permitted.

CC @jsafrane @childsb @wongma7
2016-12-20 20:33:56 -08:00