Commit Graph

20895 Commits

Author SHA1 Message Date
Dr. Stefan Schimanski
ad23081273 audit: fix deepcopy registration 2017-07-08 10:15:16 +02:00
Cao Shufeng
f41eb67798 remove extra WriteHeader function
The deleted two functions will be called later in the function
SerializeObject(). Not necessary to call them twice.
2017-07-08 14:07:01 +08:00
Cao Shufeng
bc94370e9c support json output for log backend of advanced audit 2017-07-08 11:25:28 +08:00
Chao Xu
65cedeeb43 update bazel 2017-07-07 19:04:26 -07:00
shiywang
9ac39849b9 add test resource carp and change name 2017-07-08 08:53:35 +08:00
Shiyang Wang
da4a875ef8 Move pkg/apimachinery/test to apimachinery 2017-07-08 08:48:38 +08:00
Chao Xu
24d5279c59 move leaderelection package to client-go 2017-07-07 17:01:04 -07:00
Clayton Coleman
12c7874c0d Prepare to introduce websockets for exec and portforward
Refactor the code in remotecommand to better represent the structure of
what is common between portforward and exec.
2017-07-07 18:22:51 -04:00
Clayton Coleman
edc12aafe2 Unify generic proxy code in apimachinery
Make the utility package truly generic, consolidate all generic proxy in
apimachinery.
2017-07-07 16:26:29 -04:00
Clayton Coleman
2e33a2f0bc Record 429 and timeout errors to prometheus
Allows gathering of load being shed.
2017-07-07 15:50:49 -04:00
deads2k
0801ded425 remove dead code 2017-07-07 09:12:29 -04:00
Paul Michali
78ae9a57df IPv6 support for getting IP from default route
This is part 2 of the effort to update ChoseHostInterface() to support IPv6
addresses (as part of issue 44848). This changeset includes:

- Supports finding IPv6 host addresses from default routes (but currently only
  provided with IPv4 default routes).
- getRoutes() filters for default routes.
- getFinalIP() checks that IP is in requested family. Uses IsGlobalUnicast(),
  instead of explicit tests for loopback, multicast, and link-local IPs.
- getIPFromInterace() checks for family requested.
- chooseHostInterfaceFromRoute()
    * Quickly exits, if no default routes.
    * Since only getting default routes, no check here.
    * Searches all default routes for IPv4 addresses, and then searches all
      default routes for IPv6 addresses (for backwards compatibility).
- More coverage in UTs (87.8% vs 62.6%).
- Better testing of error conditions/results.
- Tests for IPv6 IPs, throughout functions.
- Reduced duplicate testing for items tested at lower levels.

Commit has been rebased on top of PR46044.
2017-07-07 12:52:21 +00:00
Dr. Stefan Schimanski
b34464241d client-go: remove TPR example 2017-07-07 14:13:46 +02:00
Kubernetes Submit Queue
4e276d49b9 Merge pull request #46044 from pmichali/issue44848a
Automatic merge from submit-queue (batch tested with PRs 47435, 46044)

IPv6 support for getting node IP

As part of ChooseHostInterface(), it will call a function to try to get the
global IP for the host, by looking at all the system interfaces and select
the first IP that is not a loopback, link-local, or point-to-point IP.

This commit does the following:

- Allows IPv6 non-local IPs to be selected.
- IPv4 takes priority (checks all interfaces for IPv4 addresses and
  then checks all interfaces for IPv6), for backward compatibility.
- Adds UTs for code coverage (was no coverage of underlying function),
  increasing from 62% to 85%.
- Improved logging and reporting for error conditions.
- Minor renaming of functions and variables for readability.



**What this PR does / why we need it**:
This will be part of several PRs to add IPv6 support in apimachinery area for use by Kubernetes. It partially fixes the issue.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: xref #44848

**Special notes for your reviewer**:
The intent is to break up the PR 45116 into multiple PRs to take on this change a piece at a time.

**Release note**:

```release-noteNONE
```
2017-07-06 12:43:42 -07:00
Kubernetes Submit Queue
756a814042 Merge pull request #43674 from shiywang/fixjsonpath
Automatic merge from submit-queue (batch tested with PRs 48538, 43674)

fix JSONPath parser will not filter strings containing parentheses

Fixes https://github.com/kubernetes/client-go/issues/158
@mtaufen  @daizuozhuo @caesarxuchao
2017-07-06 11:21:28 -07:00
Kubernetes Submit Queue
f177389436 Merge pull request #48536 from CaoShuFeng/application/json
Automatic merge from submit-queue

Fix invalid Content-Type for 403 error

https://github.com/kubernetes/kubernetes/pull/47384 makes 403 errors return Status Object. However the Content-Type is still "text/plain"
This change fixes it.

Before this change:
kubectl get pods --as=tom
Error from server (Forbidden): {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \"\" is forbidden: User \"tom\" cannot list pods in the namespace \"default\".","reason":"Forbidden","details":{"kind":"pods"},"code":403} (get pods)

After this change:
$ kubectl get pods --as=tom
Error from server (Forbidden): pods "" is forbidden: User "tom" cannot list pods in the namespace "default".



**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```
NONE
```
2017-07-06 08:52:09 -07:00
Cao Shufeng
36e0a5ed14 Fix invalid Content-Type for 403 error
https://github.com/kubernetes/kubernetes/pull/47384 makes 403 errors
return Status Object. How the Content-Type is still "text/plain"
This change fix it.
2017-07-06 20:22:43 +08:00
Cao Shufeng
f21bc7bb9a Fix Audit-ID header key
Now http header key "Audit-ID" doesn't have effect, because golang
automaticly transforms "Audit-ID" into "Audit-Id". This change use
http.Header.Get() function to canonicalize "Audit-ID" to "Audit-Id".
2017-07-06 15:29:00 +08:00
Kubernetes Submit Queue
8b58346517 Merge pull request #48502 from ironcladlou/memcachediscovery
Automatic merge from submit-queue (batch tested with PRs 47700, 48464, 48502)

Add a refreshing discovery client

Introduce a discovery client (implementing `CachedDiscoveryInterface`) which caches discovery information in memory and which can be actively refreshed by the user.

This implementation fetches from discovery upon refresh and could later be improved to maintain updates from a watch.

Extracted from https://github.com/kubernetes/kubernetes/pull/47665 and https://github.com/kubernetes/kubernetes/pull/46000 to help reduce the scope of https://github.com/kubernetes/kubernetes/pull/48065.

```release-note
NONE
```
2017-07-05 13:27:46 -07:00
Kubernetes Submit Queue
d630c0b6b1 Merge pull request #48464 from wanghaoran1988/fix_err_type
Automatic merge from submit-queue (batch tested with PRs 47700, 48464, 48502)

fix error type

**What this PR does / why we need it**:
invalidToken should be errInvalideToken
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
No
**Release note**:
```
None
```
2017-07-05 13:27:45 -07:00
Kubernetes Submit Queue
1108738200 Merge pull request #48033 from smarterclayton/generic_printer
Automatic merge from submit-queue (batch tested with PRs 45467, 48091, 48033, 48498)

Refactor and simplify generic printer for unknown objects

The first two commits are part of other PRs

@kubernetes/sig-cli-pr-reviews part of the general refactoring for server side print
2017-07-05 12:37:33 -07:00
Dan Mace
2639b75d84 Refactor cached discovery client 2017-07-05 11:05:46 -04:00
Kubernetes Submit Queue
fdee1d5488 Merge pull request #48445 from deads2k/server-28-panichandler
Automatic merge from submit-queue (batch tested with PRs 47162, 48444, 48445)

make the panic handler first

Move the panic handler to the beginning of the chain so that panics will get information in the log beyond a stack.
2017-07-05 06:39:31 -07:00
Kubernetes Submit Queue
cfde9e5832 Merge pull request #48444 from p0lyn0mial/add_fisher_resource_to_wardle_group
Automatic merge from submit-queue (batch tested with PRs 47162, 48444, 48445)

Introducing a cluster-scoped resource in the wardle.k8s.io group.

**What this PR does / why we need it**:
This PR adds a cluster-scoped resource to the wardle.k8s.io group.
The cluster scoped resource has a field that indicates Flunder.Names that are disallowed.
The resource is going to be used by an admission plugin.
The admission plugin will list the cluster-scope resources and check against banned names.

**Special notes for your reviewer**:
Issue: #47868

**Release note**:

```
NONE
```
2017-07-05 06:39:29 -07:00
Kubernetes Submit Queue
c746680143 Merge pull request #47384 from shiywang/api403
Automatic merge from submit-queue (batch tested with PRs 48383, 47384)

Fix 401/403 apiserver errors do not return 'Status' objects

fixes https://github.com/kubernetes/kubernetes/issues/45970
2017-07-05 02:18:21 -07:00
Kubernetes Submit Queue
0ec36bdc8f Merge pull request #47043 from CaoShuFeng/validate_audit
Automatic merge from submit-queue

Add Validate() function for audit options

**Release note**:

```
NONE
```
Fixes: #47114
2017-07-04 08:48:20 -07:00
p0lyn0mial
113e9ba1d3 Introducing a cluster-scoped resource in the wardle.k8s.io group.
The cluster scoped resource has a field that indicates Flunder.Names that are disallowed.
The resource is going to be used by an admission plugin.
The admission plugin will list the cluster-scope resources and check against banned names.

Issue: #47868
2017-07-04 15:43:45 +02:00
Kubernetes Submit Queue
1033dae18e Merge pull request #46928 from CaoShuFeng/audit_unused_function
Automatic merge from submit-queue

remove unused function and variable from audit backend

auditStringSlice is not used here anymore.
sink variable is also not used.
**Release note**:

```
NONE
```
Fixes: #47114
2017-07-04 04:21:15 -07:00
Haoran Wang
45ec7d9f51 fix error type 2017-07-04 17:21:36 +08:00
Cao Shufeng
00e871a846 remove unused function and variable from audit backend 2017-07-04 16:14:10 +08:00
Shiyang Wang
3d6479f721 Fix 401/403 apiserver errors do not return 'Status' objects 2017-07-04 15:35:51 +08:00
Kubernetes Submit Queue
ea6e84365e Merge pull request #47206 from gyliu513/typo
Automatic merge from submit-queue

Follow up for https://github.com/kubernetes/kubernetes/pull/47003

**What this PR does / why we need it**:
This is addressing left comments for https://github.com/kubernetes/kubernetes/pull/47003#discussion_r120888982

/cc @aveshagarwal 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-07-04 00:25:22 -07:00
Kubernetes Submit Queue
4ae3b032f4 Merge pull request #48394 from smarterclayton/must_serialize_if_data_differs
Automatic merge from submit-queue (batch tested with PRs 48439, 48440, 48394)

GuaranteedUpdate must write if stored data is not canonical

An optimization added to the GuaranteedUpdate loop changed the
comparison of the current objects serialization against the stored data,
instead comparing to the in memory object, which defeated the mechanism
we use to migrate stored data (GET then PUT should update the version stored in etcd if the canonical serialization has changed)

This commit preserves that optimization but correctly verifies the in
memory serialization against the on disk serialization by fetching the
latest serialized data. Since most updates are not no-ops, this should
not regress the performance of the normal path.

Fixes #48393

```release-note
When performing a GET then PUT, the kube-apiserver must write the canonical representation of the object to etcd if the current value does not match. That allows external agents to migrate content in etcd from one API version to another, across different storage types, or across varying encryption levels. This fixes a bug introduced in 1.5 where we unintentionally stopped writing the newest data.
```
2017-07-03 13:05:30 -07:00
deads2k
9b43bd4a5b make the panic handler first 2017-07-03 14:34:03 -04:00
Kubernetes Submit Queue
74bde7f7ff Merge pull request #48354 from deads2k/gc-01-deletenever
Automatic merge from submit-queue (batch tested with PRs 47784, 47793, 48334, 48435, 48354)

allow a deletestrategy to opt-out of GC

Not all resources should be GC-able and we implemented an ignore list to handle this, but at the storage layer they could still set finalizers, they just hung in a stuck state forever.  This updates the strategy to allow a resource to indicate that they shouldn't be GCed.

@kubernetes/sig-api-machinery-misc
2017-07-03 10:41:56 -07:00
Kubernetes Submit Queue
cf9f110387 Merge pull request #48334 from CaoShuFeng/LevelMetadata
Automatic merge from submit-queue (batch tested with PRs 47784, 47793, 48334, 48435, 48354)

update events' ResponseStatus at Metadata level

ResponseStatus is populated in MetadataLevel, so we also update it in
MetadataLevel.

**Release note**:

```
NONE
```
2017-07-03 10:41:51 -07:00
Kubernetes Submit Queue
55261d7955 Merge pull request #47784 from CaoShuFeng/rule_validate
Automatic merge from submit-queue (batch tested with PRs 47784, 47793, 48334, 48435, 48354)

add validate for advanced audit policy

This change checks group name and non-resrouce URLs format for audit
policy.

**Release note**:

```
add validate for advanced audit policy, kube-apiserver will do a stricter validation and will break existing users with invalid configs.
```
2017-07-03 10:41:47 -07:00
deads2k
312fb1e1fa allow a deletestrategy to opt-out of GC 2017-07-03 11:27:45 -04:00
Cao Shufeng
b6abcacb38 update events' ResponseStatus at Metadata level
ResponseStatus is populated in MetadataLevel, so we also update it in
MetadataLevel.
2017-07-03 20:43:48 +08:00
Cao Shufeng
7437b88386 add validate for advanced audit policy
This change checks group name and non-resrouce URLs format for audit
policy.
2017-07-03 20:35:35 +08:00
Clayton Coleman
b851614adf GuaranteedUpdate must write if stored data is not canonical
An optimization added to the GuaranteedUpdate loop changed the
comparison of the current objects serialization against the stored data,
instead comparing to the in memory object, which defeated the mechanism
we use to migrate stored data.

This commit preserves that optimization but correctly verifies the in
memory serialization against the on disk serialization by fetching the
latest serialized data. Since most updates are not no-ops, this should
not regress the performance of the normal path.
2017-07-02 23:13:15 -04:00
Nikhita Raghunath
36271f985d Add Cleanup section to apiextensions client-go 2017-07-02 03:53:35 +05:30
Kubernetes Submit Queue
fbba67098b Merge pull request #48317 from CaoShuFeng/impersonation_group
Automatic merge from submit-queue

remove useless check from impersonation filter

When groupsSpecified is false, that means no other groups are added
rather than the service account groups. So this check doesn't make
any sense.

**Release note**:

```
NONE
```
2017-06-30 19:47:38 -07:00
Kubernetes Submit Queue
19724f0b98 Merge pull request #48151 from apelisse/accept-encoding-gzip
Automatic merge from submit-queue (batch tested with PRs 47918, 47964, 48151, 47881, 48299)

GZip openapi schema if accepted by client

**What this PR does / why we need it**: Uses gzip "Accept-Encoding" flag rather than specific path to download gzipped openapi schema.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #48375

**Special notes for your reviewer**:

**Release note**:

```release-note
```
2017-06-30 18:42:38 -07:00
Kubernetes Submit Queue
9c74026ffc Merge pull request #46803 from apelisse/new-download-openapi
Automatic merge from submit-queue (batch tested with PRs 43558, 48261, 42376, 46803, 47058)

OpenAPI downloads protobuf rather than Json

**What this PR does / why we need it**: 
The current implementation of the OpenAPI getter fetches the swagger in a Json format from the apiserver. The Json file is big (~1.7mb), which means that it takes a long time to download, and then a long time to parse. Because that is going to be needed on each `kubectl` run later, we want this to be as fast as possible.

The apiserver has been modified to be able to return a protobuf version of the swagger, which this patch intends to use.

Note that there is currently no piece of code that exists that allows us to go from the protobuf version of the file, back into Json and/or `spec.Swagger`. Because the protobuf is not very different (but significantly different enough that it can't be translated), I've updated the code to use `openapi_v2.Document` (the protobuf type) everywhere rather than `spec.Swagger`. The behavior should be identical though.

There are more changes that are coming in follow-up pull-requests: using the gzip version (also provided by the new apiserver) to even further reduce the size of the downloaded content, and use the HTTP Etag cache mechanism to completely get rid of recurrent fetch requests. I'm currently working on these two features.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: partly #38637

**Special notes for your reviewer**:

**Release note**:
```release-note
NONE
```
2017-06-30 16:28:48 -07:00
Clayton Coleman
3662184786 meta.EachListItem should support runtime.Unstructured
Allows callers to iterate over that construct.
2017-06-30 16:40:26 -04:00
Kubernetes Submit Queue
992002689e Merge pull request #47132 from pichouk/patch-1
Automatic merge from submit-queue

Fix a typo in deletion log of apiserver

**What this PR does / why we need it**:
I just fix a typo in a log message. Nothing more 😄 

**Which issue this PR fixes**
apiserver sometimes log this message "About do delete object from database". It seems that there is a typo for `to`.

```release-note
Fix a typo in apiserver log message
```
2017-06-30 05:30:17 -07:00
Cao Shufeng
0a1e24f31e remove useless check from impersonation filter
When groupsSpecified is false, that means no other groups are added
rather than the service account groups. So this check doesn't make
any sense.
2017-06-30 10:51:07 +08:00
Mikhail Mazurskiy
dc1ee493a2 Refactor unstructured converter 2017-06-30 09:35:14 +10:00
Aaron Crickenberger
8469b01333 TestLoopbackHostPort should accept IPv6 loopback host 2017-06-29 11:49:28 -07:00