Commit Graph

2717 Commits

Author SHA1 Message Date
Xingcai Zhang
70f47a7975 Update many misspelled word initializer 2017-10-15 22:09:26 +08:00
Robert Rati
0840e6d869 Added PreShutdownHooks to the apiserver 2017-10-15 07:27:01 -04:00
Kubernetes Submit Queue
a523084d88 Merge pull request #52762 from jinzhejz/jinzhejz_typo
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix typo in quantity.go

**What this PR does / why we need it**:
fix typo

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #N/A

**Release note**:
```release-note
None
```
2017-10-13 11:38:16 -07:00
Alexander Kanevskiy
ad3f9a81a3 NewProxierWithNoProxyCIDR: fix handling IPv6 URLs
Current logic of splitting hostname from URL does not work if URL
is for IPv6 address and does not explicitly specify port number.
Example: "https://[2001:db8::1]/".

Use standard library function to get hostname out of URL string.
2017-10-13 20:19:22 +03:00
Kubernetes Submit Queue
8cbc9d7e27 Merge pull request #53830 from juanvallejo/jvallejo/prevent-delta-fifo-store-panic
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

perform nil check before iterating over keys

**Release note**:
```release-note
NONE
```

Fixes panic due to nil pointer dereference 

Related downstream bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1377940
cc @smarterclayton
2017-10-13 09:43:13 -07:00
Kubernetes Submit Queue
f1d9962fec Merge pull request #53720 from shyamjvs/test-kubemark
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Optimize random string generator to avoid multiple locks & use bit-masking

Ref https://github.com/kubernetes/kubernetes/issues/53327

We recently started seeing a 50% decrease in scheduling throughput (for e.g in kubemark-500 scale job) and turns out https://github.com/kubernetes/kubernetes/pull/53135 introduced it.
The reason is [this call](2caae38d32/plugin/pkg/scheduler/algorithm/predicates/predicates.go (L272)) to create a random 32-length string.
From the code of the `rand` utility (which is being heavily used throughout the system for randomizing object names), I noticed following performance issues:
- to create an n-length string, we are making n calls to `rand.Intn()` each of which does a lock+unlock operation on the RNG.. while just 1 lock+unlock operation is enough for all
- we're choosing one character (from an alphabet of 27 chars) per each random integer.. while we can select 10 characters using a single int63 (by masking and bit-shifting) as 1 character uses just 5 bits of randomness
- the character set is defined as a global slice (mutable), so the compiler needs to fetch length of the slice on each invocation to `len()` (we're making n of those).. while we can just use a const string (immutable) which will make len directly available as a cached constant (yes, go does it!)

This PR is making the above fixes. I'll try to add some benchmarking to measure the difference (as @wojtek-t suggested).

/cc @kubernetes/sig-scalability-misc @kubernetes/sig-scheduling-bugs @kubernetes/sig-api-machinery-misc @wojtek-t @smarterclayton
2017-10-13 06:11:09 -07:00
Kubernetes Submit Queue
cc49b34d29 Merge pull request #53586 from sttts/sttts-storage-shutdown
Automatic merge from submit-queue (batch tested with PRs 53249, 53586). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

apiextensions-apiserver: stop cacher on CRD update

Potentially fixes #53485
2017-10-13 05:09:43 -07:00
Kubernetes Submit Queue
fd57b1c002 Merge pull request #53249 from hzxuzhonghu/envelop-encrypt
Automatic merge from submit-queue (batch tested with PRs 53249, 53586). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

rename encryptionconfig_test.go and remove unused filed in envelopeTransformer

**What this PR does / why we need it**:
useless field `cacheSize` and rename test file match original `config.go`.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
2017-10-13 05:09:40 -07:00
Shyam Jeedigunta
0185e55889 Add benchmark for random string generation utility 2017-10-13 13:22:10 +02:00
Dr. Stefan Schimanski
333f49f5d1 apiextensions: fix test loop for CRD validation 2017-10-13 12:17:02 +02:00
Dr. Stefan Schimanski
e23983189d apiextensions: keep CRD storage for updates outside of spec and accepted names 2017-10-13 12:14:09 +02:00
Kubernetes Submit Queue
7c75723867 Merge pull request #52383 from dixudx/quote_field_valid_string
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

quote valid strings for field validation

**What this PR does / why we need it**:
Some of the declared const vars are empty string `""`, such as

* [`NamespaceAll`, `NamespaceNone`](https://github.com/kubernetes/kubernetes/blob/master/pkg/api/types.go#L197-L200)
* [`HostPathUnset`](https://github.com/kubernetes/kubernetes/blob/master/pkg/api/types.go#L631)

When validating such supported fields, empty string will be printed without quotes, which is kind of ugly.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #53849

**Special notes for your reviewer**:
/assign @liggitt 

**Release note**:

```release-note
None
```
2017-10-12 20:25:10 -07:00
jianglingxia
bad417b91e unused para in quantity_test 2017-10-13 11:03:33 +08:00
Kubernetes Submit Queue
fe3b68ecc6 Merge pull request #53762 from smarterclayton/reflect_diff
Automatic merge from submit-queue (batch tested with PRs 53749, 53642, 53813, 53771, 53762). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Don't exit early in diff.ObjectReflectDiff on slices
2017-10-12 18:22:43 -07:00
Di Xu
7508afedd3 quote valid strings for field validation 2017-10-13 07:52:16 +08:00
Haowei Cai
eff8136442 Change name from sourceHost to sourceRequestHost. 2017-10-12 14:26:05 -07:00
Kubernetes Submit Queue
45fd545366 Merge pull request #53756 from ericchiang/webhook-timeout
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

generic webhook: set a default timeout for webhook requests

Add a 30 second timeout for all HTTP requests that the webhook sends
so they timeout instead of hanging forever.

closes https://github.com/kubernetes/kubernetes/issues/53698

cc @kubernetes/sig-api-machinery-pr-reviews 

```release-note
NONE
```
2017-10-12 13:52:08 -07:00
juanvallejo
1fcd410d4a perform nil check before iterating over keys 2017-10-12 16:25:31 -04:00
Shyam Jeedigunta
c279a53ca0 Optimize random string generator to avoid multiple locks & use bitmasking 2017-10-12 19:13:17 +02:00
Kubernetes Submit Queue
6901fc37d1 Merge pull request #52981 from CaoShuFeng/audit_v1beta2
Automatic merge from submit-queue (batch tested with PRs 53119, 53753, 53795, 52981). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add RequestReceivedTimestamp and StageTimestamp to audit event

fixes https://github.com/kubernetes/kubernetes/issues/52160

**Release note**:
```
Add RequestReceivedTimestamp and StageTimestamp with micro seconds to audit events.
```
2017-10-12 08:28:44 -07:00
zhengchuan hu
77dbe5995f generated when exec hack/update-all.sh 2017-10-12 23:19:31 +08:00
zhengchuan hu
f50e0e7c50 Fix incorrect link in api 2017-10-12 22:50:17 +08:00
xiangpengzhao
2a454dcea0 Mark etcd-quorum-read as deprecated. 2017-10-12 20:00:56 +08:00
Cao Shufeng
b69285af7f run hack/update-all.sh 2017-10-12 11:29:41 +08:00
Cao Shufeng
817bc6954c support micro time for advanced audit 2017-10-12 11:29:36 +08:00
Clayton Coleman
178baa9ac3 Don't exit early in diff.ObjectReflectDiff on slices
We were exiting at the first mismatched index, but we want to collect
the diffs for all children. Also, we were assuming diffs existed which
was not always true (and was due to other bugs that we weren't catching
before).
2017-10-11 22:53:25 -04:00
Eric Chiang
2de3ee5c48 generic webhook: set a default timeout for webhook requests
Add a 30 second timeout for all HTTP requests that the webhook sends
so they timeout instead of hanging forever.
2017-10-11 19:07:23 -07:00
Kubernetes Submit Queue
42d22aea9f Merge pull request #53717 from liggitt/quorum
Automatic merge from submit-queue (batch tested with PRs 51677, 53690, 53025, 53717, 53664). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Change --etcd-quorum-read default to true

The tested configurations for HA etcd use quorum reads. Defaulting this off causes potential correctness issues in controllers that do live lookups when processing their work queue. Given that, we should default this on.

Quorum reads are far more performant on etcd3 than they were on etcd2

xref
https://github.com/kubernetes/kubernetes/pull/53662#discussion_r143806500
https://github.com/kubernetes/kubernetes/issues/19902
https://github.com/kubernetes/kubernetes/issues/48865

```release-note
apiserver: --etcd-quorum-read now defaults to true, to ensure correct operation with HA etcd clusters
```
2017-10-11 09:35:32 -07:00
Kubernetes Submit Queue
df072ca97e Merge pull request #53025 from mtaufen/feature-gate-map
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Make feature gates loadable from a map[string]bool

Command line flag API remains the same. This allows ComponentConfig             
structures (e.g. KubeletConfiguration) to express the map structure             
behind feature gates in a natural way when written as JSON or YAML.             
                                                                                
For example:                                                                    
                                                                                
KubeletConfiguration Before:
```
apiVersion: kubeletconfig/v1alpha1
kind: KubeletConfiguration
featureGates: "DynamicKubeletConfig=true,Accelerators=true"
```

KubeletConfiguration After:
```
apiVersion: kubeletconfig/v1alpha1
kind: KubeletConfiguration
featureGates:
  DynamicKubeletConfig: true
  Accelerators: true
```

Fixes: #53024

```release-note
The Kubelet's feature gates are now specified as a map when provided via a JSON or YAML KubeletConfiguration, rather than as a string of key-value pairs.
```

/cc @mikedanese @jlowdermilk @smarterclayton
2017-10-11 09:05:33 -07:00
Dr. Stefan Schimanski
b125388767 apiextensions: create storage with accepted, not spec'ed names 2017-10-11 12:20:31 +02:00
Dr. Stefan Schimanski
c13d5ce36a apiextensions-apiserver: stop cacher on CRD update 2017-10-11 12:17:42 +02:00
Kubernetes Submit Queue
24cf4a8b5c Merge pull request #53441 from apelisse/completely-remove-swagger
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

kubectl: Remove swagger 1.2 entirely.

**What this PR does / why we need it**:
Remove dead code since nothing is using swagger 1.2 anymore. This doesn't change any feature, it's just removing unused code.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: Follow up on #44589

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2017-10-10 20:43:27 -07:00
Kubernetes Submit Queue
23cc4dc50a Merge pull request #52949 from smarterclayton/enable_paging
Automatic merge from submit-queue (batch tested with PRs 52354, 52949, 53551). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Enable API chunking and promote to beta for 1.9

All list watchers default to using chunking.  The server by default fills pages to avoid low cardinality filters from making excessive numbers of requests.  Fix an issue with continuation tokens where a `../` could be used if the feature was enabled.

```release-note
API chunking via the `limit` and `continue` request parameters is promoted to beta in this release.  Client libraries using the Informer or ListWatch types will automatically opt in to chunking.
```
2017-10-10 15:45:29 -07:00
Antoine Pelisse
d1ce36371e kubectl: Remove swagger 1.2 entirely. 2017-10-10 14:50:56 -07:00
Kubernetes Submit Queue
d6b18a96dd Merge pull request #53611 from p0lyn0mial/pass_scheme_to_admission_plugins
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

removes k8s.io/kubernetes/pkg/api dependency from the webhook plugin.

**What this PR does / why we need it**: removes `k8s.io/kubernetes/pkg/api` dependency from `webhook` plugin. The runtime.scheme can be injected to the webhook from the plugin initializer.

**Release note**:

```
NONE
```
2017-10-10 13:53:27 -07:00
Kubernetes Submit Queue
6154a9e16c Merge pull request #53477 from caesarxuchao/genenic-initializer
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move initializer plugin to the generic apiserver

* Moves `k8s.io/kuberentes/plugin/pkg/admission/initialization` to `k8s.io/apiserver/pkg/admission/plugin/initialization`
* Moves `k8s.io/kubernetes/pkg/kubeapiserver/admission/configuration` to `k8s.io/apiserver/pkg/admission/configuration`
* The initializer plugin used to depend on `k8s.io/kubernetes` because it does a type assertion of `api.Pod`. It tries to skip mirror pod. I converted that code to use the generic accessor pattern.
2017-10-10 09:46:00 -07:00
Michael Taufen
131b419596 Make feature gates loadable from a map[string]bool
Command line flag API remains the same. This allows ComponentConfig
structures (e.g. KubeletConfiguration) to express the map structure
behind feature gates in a natural way when written as JSON or YAML.

For example:

KubeletConfiguration Before:
```
apiVersion: kubeletconfig/v1alpha1
kind: KubeletConfiguration
featureGates: "DynamicKubeletConfig=true,Accelerators=true"
```

KubeletConfiguration After:
```
apiVersion: kubeletconfig/v1alpha1
kind: KubeletConfiguration
featureGates:
  DynamicKubeletConfig: true
  Accelerators: true
```
2017-10-10 09:37:51 -07:00
Kubernetes Submit Queue
73d1b38604 Merge pull request #53652 from sttts/sttts-generate-internal-groups.sh-flags
Automatic merge from submit-queue (batch tested with PRs 53525, 53652). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

code-generator: fix flag check in generate-internal-groups.sh
2017-10-10 08:31:44 -07:00
Kubernetes Submit Queue
aaf14d4619 Merge pull request #53525 from sttts/sttts-scheme-copier-romoval
Automatic merge from submit-queue (batch tested with PRs 53525, 53652). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

apimachinery: remove ObjectCopier interface(s)

The big commit is a mechanical, transitive removal of the copier interfaces in all structs and function calls.
2017-10-10 08:31:41 -07:00
Clayton Coleman
ce0dc76901 Avoid intermediate List allocations as items added to the list
Pick a reasonable middle ground between allocating larger chunks of
memory (2048 * ~500b for pod slices) and having many small allocations
as the list is resized by preallocating capacity based on the expected
list size. At worst, we'll allocate a 1M slice for pods and only add
a single pod to it (if the selector is very specific).
2017-10-10 09:11:45 -04:00
Clayton Coleman
6a76931e2c Use watch cache when rv=0 even when limit is set 2017-10-10 09:11:44 -04:00
Clayton Coleman
ac8808b792 Promote continuation token schema to v1
Change the filtering logic to require a leading path and clean that
afterwards.
2017-10-10 09:11:44 -04:00
Clayton Coleman
da7124e5e5 Fill partial pages on the server rather than forcing client to
The etcd3 storage now attempts to fill partial pages to prevent clients
having to make more round trips (latency from server to etcd is lower
than client to server). The server makes repeated requests to etcd of
the current page size, then uses the filter function to eliminate any
matches. After this change the apiserver will always return full pages,
but we leave the language in place that clients must tolerate it.

Reduces tail latency of large filtered lists, such as viewing pods
assigned to a node.
2017-10-10 09:11:44 -04:00
Clayton Coleman
113889e72d Enable API chunking and promote to beta for 1.9
All list watchers default to using chunking.
2017-10-10 09:11:41 -04:00
Dr. Stefan Schimanski
adc14ef935 code-generator: fix flag check in generate-internal-groups.sh 2017-10-10 14:29:48 +02:00
Cao Shufeng
a00b8153ca fix generate-groups.sh
This script should run when user pass arguements like this:
generate-groups.sh  deepcopy "k8s.io/coredump-detector/" "k8s.io/coredump-detector/apis/" "coredump:v1alpha1"
2017-10-10 16:10:39 +08:00
Jordan Liggitt
2400185f45 Change --etcd-quorum-read default to true 2017-10-10 00:16:33 -04:00
p0lyn0mial
fa96700b76 removes k8s.io/kubernetes/pkg/api dependency from the webhook plugin. 2017-10-09 22:25:03 +02:00
Kubernetes Submit Queue
bd9788f8f8 Merge pull request #53567 from maleck13/code-gen-nil-pointer-fix
Automatic merge from submit-queue (batch tested with PRs 53567, 53197, 52944, 49593). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Address code-gen issue 7 nil pointer when incorrect pkg name

**What this PR does / why we need it**:
When using the code-gen tool, if the comments in the ```doc.go``` file has a misspelled pkg name, it ends with a nil pointer. This can be difficult to debug. This PR fixes that by logging an error and continuing. 
https://github.com/kubernetes/code-generator/issues/7

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
I was not sure whether just exiting here would be correct behaviour as reading the code , I noticed in other places it continues if the pkg is nil.
Also perhaps I should use the ```glog.V(5).info```  ?

```release-note
`NONE`
```
2017-10-09 11:18:43 -07:00
Chao Xu
bbac32c299 generated 2017-10-09 10:58:12 -07:00