Commit Graph

155 Commits

Author SHA1 Message Date
k8s-merge-robot
0c06f31cb8 Merge pull request #23340 from justinsb/fix_23339
Auto commit by PR queue bot
2016-03-29 05:04:27 -07:00
k8s-merge-robot
4e4ad61260 Merge pull request #23366 from goltermann/vet
Auto commit by PR queue bot
2016-03-24 21:50:56 -07:00
Chris Batey and James Ravn
be9ce30897 Change LoadBalancer methods to take api.Service
This is a better abstraction than passing in specific pieces of the
Service that each of the cloudproviders may or may not need. For
instance, many of the providers don't need a region, yet this is passed
in. Similarly many of the providers want a string IP for the load
balancer, but it passes in a converted net ip. Affinity is unused by
AWS. A provider change may also require adding a new parameter which has
an effect on all other cloud provider implementations.

Further, this will simplify adding provider specific load balancer
options, such as with labels or some other metadata. For example, we
could add labels for configuring the details of an AWS elastic load
balancer, such as idle timeout on connections, whether it is
internal or external, cross-zone load balancing, and so on.

Authors: @chbatey, @jsravn
2016-03-23 10:48:11 +00:00
goltermann
34d4eaea08 Fixing several (but not all) go vet errors. Most are around string formatting, or unreachable code. 2016-03-22 17:26:50 -07:00
Justin Santa Barbara
59013f5507 AWS: Fix problems with >2 security groups
The previous logic was incorrect; if we saw two untagged security groups
before seeing the first tagged security, we would incorrectly return an
error.

Fix #23339
2016-03-22 13:00:14 -04:00
k8s-merge-robot
8c02a46c4d Merge pull request #22486 from thockin/update-gcfg-dep
Auto commit by PR queue bot
2016-03-21 18:47:21 -07:00
k8s-merge-robot
0a28a38110 Merge pull request #22280 from justinsb/fix_error_message_formats
Auto commit by PR queue bot
2016-03-21 16:53:38 -07:00
Tim Hockin
a073c80e45 Use newer home for gcfg package
Switch from obsolete "github.com/scalingdata/gcfg" to "gopkg.in/gcfg.v1".
2016-03-16 08:42:08 -07:00
Jan Safranek
f270cb1b9b Allow 39 atached EBS devices.
AWS has soft support limit for 40 attached EBS devices. Assuming there is just
one root device, use the rest for persistent volumes.

The devices will have name /dev/xvdba - /dev/xvdcm, leaving /dev/sda - /dev/sdz
to the system.

Also, add better error handling and propagate error
"Too many EBS volumes attached to node XYZ" to a pod.
2016-03-15 17:28:59 +01:00
k8s-merge-robot
bf1bb5d309 Merge pull request #22899 from justinsb/aws_fix_e2e
Auto commit by PR queue bot
2016-03-14 17:37:13 -07:00
Justin Santa Barbara
3e12e8b7cc AWS e2e: don't try to build a full cloudprovider in e2e
We have previously tried building a full cloudprovider in e2e for AWS;
this wasn't the best idea, because e2e runs on a different machine than
normal operations, and often doesn't even run in AWS.  In turn, this
meant that the cloudprovider had to do extra work and have extra code,
which we would like to get rid of.  Indeed, I got rid of some code which
tolerated not running in AWS, and this broke e2e.
2016-03-12 06:14:45 -05:00
k8s-merge-robot
45064e19d1 Merge pull request #22793 from justinsb/fix_22792
Auto commit by PR queue bot
2016-03-11 20:37:25 -08:00
k8s-merge-robot
26e309c753 Merge pull request #22784 from justinsb/fix_17626
Auto commit by PR queue bot
2016-03-11 20:04:13 -08:00
k8s-merge-robot
de7193a095 Merge pull request #22788 from justinsb/fix_22786
Auto commit by PR queue bot
2016-03-11 15:13:06 -08:00
Justin Santa Barbara
e40595fa57 AWS volumes: Release disk from attaching map on error
If AWS gives us an actual error (vs just timing out), we know the disk
did not attach, and so we can remove it immediately from the attaching
map.
2016-03-11 11:40:39 -05:00
Justin Santa Barbara
16730aba96 AWS: Tag created EBS volumes with our cluster tag
Fix #22792
2016-03-10 08:13:50 -05:00
Justin Santa Barbara
79b2b7edef AWS EBS: Remove the attached volumes cache
There are known issues with the attached-volume state cache that just aren't
possible to fix with the current interface.

Replace it with a map of the active attach jobs (that was the original
requirement, to avoid a nasty race condition).

This costs us an extra DescribeInstance call on attach/detach, but that
seems worth it if it ends this class of bugs.

Fix #15073
2016-03-10 07:50:35 -05:00
Justin Santa Barbara
0921af4aca AWS: Don't pass empty filters to AWS requests
It gives an error: `The filter 'null' is invalid`

Instead of a zero-length filter list, provide a nil value.

Fix #22786
2016-03-10 07:22:31 -05:00
Justin Santa Barbara
7c82fe7389 AWS: Increase timeout deleting ELB; log remaining security groups
Either ELB is slow to delete (in which case the bumped timeout will
help), or the security groups are otherwise blocked (in which case
logging them will help us track this down).

Fix #17626
2016-03-10 06:57:13 -05:00
Justin Santa Barbara
cb818a01d0 AWS: Fix some error messages
Some error messages had incorrect spacing.  Prefer Warningf to Warning,
and fix some of those problems.
2016-03-08 06:29:29 -05:00
Justin Santa Barbara
02e79b9e52 AWS: If we have no subnets, bail out early
We know the ELB call will fail, so we error out early rather than
hitting the API.  Preserves rate limit quota, and also allows us to give
a more self-evident message.

Fix #21993
2016-03-06 09:41:29 -05:00
Justin Santa Barbara
5cf837452b AWS: Fix problems identifying subnets for internal ELBs
We tacitly supported this before, but we broke this with the
public-subnet detection.

Fix #22527
2016-03-06 09:41:29 -05:00
Justin Santa Barbara
43e6602c42 AWS: Fix test failure introduced by rebase 2016-03-05 08:11:30 -05:00
Justin Santa Barbara
cff564b1a6 AWS: Remove dead code and fix up comments 2016-03-05 08:09:40 -05:00
Justin Santa Barbara
f8e6098e4d AWS: Update tests for refactoring 2016-03-05 08:09:40 -05:00
Justin Santa Barbara
af9efa02b4 AWS: Remove getSelfAWSInstance, use field directly
Now that we always populate the local instance, we don't need a getter.
2016-03-05 08:09:40 -05:00
Justin Santa Barbara
8c492c7536 AWS: Don't store the AZ on the cloud
Now we have Ubernetes-Lite, an AWSCloud can span multiple AZs.
2016-03-05 08:09:40 -05:00
Justin Santa Barbara
ddb5072a54 AWS: Don't pretend getSelfAWSInstance can return an error
It can't any more; this simplifies calling code.
2016-03-05 08:09:40 -05:00
Justin Santa Barbara
40d0afbb1b AWS: Capture VPC ID into AWSCloud, avoiding requeries
By storing the VPC ID on AWSCloud, we avoid the need to requery or to
pass it around.
2016-03-05 08:09:40 -05:00
Justin Santa Barbara
00b666f853 AWS: Rename getInfo -> describeInstance/describeVolume
Makes it clearer that we are making an AWS API call
2016-03-05 08:09:40 -05:00
Justin Santa Barbara
efa68a3590 AWS: Build awsInstance as part of cloud provider creation
We need getSelfAWSInstance to be working anyway; we might as well build
it early, and then we can use its methods to extract e.g. the VPC ID
2016-03-05 08:09:40 -05:00
Justin Santa Barbara
0375fa057f AWS: Refactor newAWSInstance
Now that we can't build an awsInstance from metadata, because of the
PrivateDnsName issue, we might as well simplify the arguments.

Create a 'placeholder' method though - newAWSInstanceFromMetadata - that
documents the desire to use metadata, shows how we would get it, but
links to the bug which explains why we can't use it.
2016-03-05 08:09:40 -05:00
k8s-merge-robot
a80f6a7ea6 Merge pull request #21905 from justinsb/aws_wrap_security_group_error
Auto commit by PR queue bot
2016-03-05 01:25:38 -08:00
k8s-merge-robot
264f5786ea Merge pull request #21987 from justinsb/fixx_21895
Auto commit by PR queue bot
2016-03-04 17:07:57 -08:00
k8s-merge-robot
40778f3d2c Merge pull request #22194 from chbatey/testify-for-aws-provider
Auto commit by PR queue bot
2016-03-04 02:29:32 -08:00
Justin Santa Barbara
62e34da125 AWS: Remove extra load balancer security group ingress rules
The ingress CIDRs are going to be dynamic, and in general we don't want
to leave old ingress rules around.

Fix #21895
2016-03-03 18:55:49 -05:00
k8s-merge-robot
de72b6be1b Merge pull request #21907 from justinsb/load_balancer_source_ranges
Auto commit by PR queue bot
2016-03-03 14:10:47 -08:00
Justin Santa Barbara
cb92133dfa LB Source Ranges: Move validation into API layer
Had to move other things around too to avoid a weird api ->
cloudprovider dependency.

Also adding fixes per code reviews.

(This is a squash of the previously approved commits)
2016-03-03 10:27:17 -05:00
k8s-merge-robot
d81d823ca5 Merge pull request #22393 from eparis/blunderbuss
Auto commit by PR queue bot
2016-03-02 18:51:56 -08:00
Eric Paris
5e5a823294 Move blunderbuss assignees into tree 2016-03-02 20:46:32 -05:00
Christopher Batey
aebd4c95e1 Use testify for AWS provider
This has two main advantages:

* The use of the mock package to verify API calls against the aws SDK
* Nicer error messages for asserts without having to use if statements
2016-03-01 14:32:45 +00:00
k8s-merge-robot
6e6550a105 Merge pull request #21989 from justinsb/fix_21986
Auto commit by PR queue bot
2016-03-01 03:51:43 -08:00
Justin Santa Barbara
49e1149227 AWS: Add support for load balancer source ranges
This refactors #21431 to pull a lot of the code into cloudprovider so it
can be reused by AWS.

It also changes the name of the annotation to be non-GCE specific:
service.beta.kubernetes.io/load-balancer-source-ranges

Fix #21651
2016-02-29 19:32:08 -05:00
James Ravn
f568b6511a Handle aws implicit and shared routing tables
Fix the AWS subnet lookup that checks if a subnet is public, which was
missing a few cases:

- Subnets without explicit routing tables, which use the main VPC
  routing table.
- Routing tables not tagged with KubernetesCluster. The filter for this
  is now removed.
2016-02-25 22:52:26 +00:00
Justin Santa Barbara
1cdfc9ad84 AWS: Find the correct security group by looking at tags
Like everything else AWS, we differentiate between k8s-owned security
groups and k8s-not-owned security groups using tags.

When we are setting up the ingress rule for ELBs, pick the security
group that is tagged over any others.

We continue to tolerate a single security group being untagged, but
having multiple security groups without tagging is now an error, as it
leads to undefined behaviour.

We also log at startup if the cluster tag is not defined.

Fix #21986
2016-02-25 11:20:58 -05:00
k8s-merge-robot
2a58c0062d Merge pull request #17913 from jtblin/jtblin/17912-pick-public-subnets
Auto commit by PR queue bot
2016-02-24 23:48:15 -08:00
Justin Santa Barbara
e50ae40301 AWS: Wrap AWS error when failing to create security group ingress
All AWS errors should be wrapped in a user-friendly error before
returning.
2016-02-24 14:13:44 -05:00
k8s-merge-robot
9c1d8bf99d Merge pull request #21399 from sky-uk/disable-ingress-sg
Auto commit by PR queue bot
2016-02-24 00:05:47 -08:00
James Ravn and Yoseph Samuel
9f62e81be5 Disable aws node security group ingress creation
Add aws cloud config:

    [global]
    disableSecurityGroupIngress = true

The aws provider creates an inbound rule per load balancer on the node
security group. However, this can quickly run into the AWS security
group rule limit of 50.

This disables the automatic ingress creation. It requires that the user
has setup a rule that allows inbound traffic on kubelet ports from the
local VPC subnet (so load balancers can access it). E.g.  `10.82.0.0/16
30000-32000`.

Limits: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Appendix_Limits.html#vpc-limits-security-groups

Authors: @jsravn, @balooo
2016-02-23 15:24:50 +00:00
Chris Batey, James Ravn and Yoseph Samuel
087ff78cf9 Only find running aws hosts by nodename
When finding instance by node name in AWS, only retrieve running
instances.  Otherwise terminated, old nodes can show up with the same
tag when rebuilding nodes in the cluster.

Another improvement made is to filter instances by the node names
provided, rather than selecting all instances and filtering in code.

Authors: @jsravn, @chbatey, @balooo
2016-02-23 14:47:16 +00:00