Automatic merge from submit-queue (batch tested with PRs 55217, 54260). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Unit tests for Azure service session affinity
**What this PR does / why we need it**: We added session affinity support in the Azure load balancer in commit 8b50b83067. This PR adds unit tests for this behaviour.
**Which issue this PR fixes**: None
**Special notes for your reviewer**: None
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 55233, 55927, 55903, 54867, 55940). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix azure disk storage account init issue
**What this PR does / why we need it**:
There are two issues for the original azure disk storage account initialaztion code:
1) wrong controller-master detection, see issue #54570, #55776
2) should not initialize two storage account even if it's not necessary, see issue #50883
This PR would fix the above two issues:
For 1: remove the controller-master process binding
For 2: remove the storage account initialization process, just create on demand
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#54570Fixes#55776Fixes#50883
**Special notes for your reviewer**:
@rootfs @karataliu
**Release note**:
```
fix azure disk storage account init issue
```
/sig azure
Automatic merge from submit-queue (batch tested with PRs 55594, 47849, 54692, 55478, 54133). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Added service annotation to set Azure DNS label for public IP
**What this PR does / why we need it**: Added a feature to set the DNS label for public IPs in the Azure cloud.
For example:
```
apiVersion: v1
kind: Service
metadata:
annotations:
service.alpha.kubernetes.io/label-name: myservice
...
```
Will resolve myservice.westus.cloudapp.azure.com to the service's IP.
**Which issue this PR fixes**: fixes#44775
**Special notes for your reviewer**: Note that this is defining a new annotation, so feel free to point out if there is a preferred convention or anything else that needs to be done.
**Release note**:
```release-note
New service annotation "service.beta.kubernetes.io/azure-dns-label-name" to set Azure DNS label name for public IP
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Restrict Azure NSG rules to allow external access only to load balancer IP
**What this PR does / why we need it**: On Azure, we create NSG (Network Security Group) rules on the vnet to allow external clients to access services exposed as type LoadBalancer. At the moment, these rules have a destination of `Any`, which means that they will permit requests on the opened port to any IP within the vnet. This PR restricts the security rules so that they admit external access only to the load balancer IP.
**Which issue this PR fixes**: None in upstream - reported as https://github.com/Azure/acs-engine/issues/1619
**Special notes for your reviewer**: None
**Release note**:
```release-note
Azure NSG rules for services exposed via external load balancer
now limit the destination IP address to the relevant front end load
balancer IP.
```
Fix#51406
If cloud providers(like aws, gce etc...) implement ExternalID()
and support getting instance by ProviderID , they also implement
InstanceExistsByProviderID().
Automatic merge from submit-queue (batch tested with PRs 53694, 53919). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix controller manager crash issue on a manually created k8s cluster
**What this PR does / why we need it**:
fix controller manager crash issue on a manually created k8s cluster, it's due to availability set nil issue in azure loadbalancer
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
In the testing of a manually created k8s cluster, I found controller manager on master would crash in current scenario:
1. Use acs-engine to set up k8s 1.7.7 cluster (it's with an availability set)
2. Manually add a node to the k8s cluster (without an availibity set in this VM)
3. Set up a service and schedule the pod onto this newly added node
4. controller manager would crash on master because although this k8s cluster has an availability set, the newly added node's `machine.AvailabilitySet` is nil which would cause controller manager crash
**Special notes for your reviewer**:
@brendanburns @karataliu @JiangtianLi
**Release note**:
```
fix controller manager crash issue on a manually created k8s cluster
```
/sig azure
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix format specifiers in Azure cloud provider
**What this PR does / why we need it**: Fixes invalid/mismatched format specifiers in Azure cloud provider logging statements (`glog...Infof(...)`) that would cause information to be lost in logging output, as flagged by `go vet`.
**Which issue this PR fixes**: None
**Special notes for your reviewer**: None
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 53444, 52067, 53571, 53182). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Retry when checking Azure storage account readiness
**What this PR does / why we need it**: When the Azure cloud provider ensures that a default storage container exists, if the storage account exists but is still provisioning, it exits without retrying. This is a bug as the code is wrapped in a backoff policy but never signals the policy to retry. This PR fixes this behaviour by returning values which allow the backoff policy to operate.
**Which issue this PR fixes**: fixes#53052
**Special notes for your reviewer**: Not sure how to test this - I have done a deployment using acs-engine and it seems to work but I am not sure of the best way to exercise the failure path.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 52355, 52537, 52551, 52403, 50673). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Azure - Use cloud environment to instantiate storage client
**What this PR does / why we need it**:
Since 1.7 and managed disk for azure, blob storage on Azure cloud other than the default public one is broken, because kubernetes expect blob ressources URI to end with `.blob.core.windows.net ` (ignoring storageEndpointSuffix).
This include the chinese Cloud, for which storageEndpointSuffix is `blob.core.chinacloudapi.cn` for example.
See : https://github.com/Azure/azure-storage-go/blob/master/client.go#L194
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 50294, 50422, 51757, 52379, 52014). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>..
Azure cloud provider: expose services on non-default subnets
**What this PR does / why we need it**: The Azure cloud provider allows users to specify that a service should be exposed on an internal load balancer instead of the default external load balancer. However, in a VNet environment, such services are currently always exposed on the master subnet. Where there are multiple subnets in the VNet, it's desirable to be able to expose an internal service on any subnet. This PR allows this via a new annotation, `service.beta.kubernetes.io/azure-load-balancer-internal-subnet`.
**Which issue this PR fixes**: fixes https://github.com/Azure/acs-engine/issues/1296 (no corresponding issue has been raised in the k8s core repo)
**Special notes for your reviewer**: None
**Release note**:
```release-note
A new service annotation has been added for services of type LoadBalancer on Azure,
to specify the subnet on which the service's front end IP should be provisioned. The
annotation is service.beta.kubernetes.io/azure-load-balancer-internal-subnet and its
value is the subnet name (not the subnet ARM ID). If omitted, the default is the
master subnet. It is ignored if the service is not on Azure, if the type is not
LoadBalancer, or if the load balancer is not internal.
```
Automatic merge from submit-queue
Fix splitProviderID for Azure
**What this PR does / why we need it**:
#46940 add 'splitProviderID' for Azure to get node name from provider, but it captures the resource id instead of node name.
Functions such as NodeAddresses are accepting node names:
84d9778f22/pkg/cloudprovider/providers/azure/azure_instances.go (L32)
With current implementation, it takes in a resource ID, and will result in following error
```
E0830 04:15:09.877143 10427 azure_instances.go:63] error: az.NodeAddresses, az.getIPForMachine(/subscriptions/{id}/resourceGroups/{id}/providers/Microsoft.Compute/virtualMachines/k8s-master-0), err=instance not found
```
This fix makes is return node names instead.
**Which issue this PR fixes**
**Special notes for your reviewer**:
**Release note**:
`NONE`
@brendandburns @realfake @wlan0
Automatic merge from submit-queue (batch tested with PRs 52047, 52063, 51528)
implementation of GetZoneByProviderID and GetZoneByNodeName for azure
This is part of the #50926 effort
cc @luxas
**Release note**:
```release-note
None
```
Automatic merge from submit-queue (batch tested with PRs 51174, 51363, 51087, 51382, 51388)
Add InstanceExistsByProviderID to cloud provider interface for CCM
**What this PR does / why we need it**:
Currently, [`MonitorNode()`](02b520f0a4/pkg/controller/cloud/nodecontroller.go (L240)) in the node controller checks with the CCM if a node still exists by calling `ExternalID(nodeName)`. `ExternalID` is supposed to return the provider id of a node which is not supported on every cloud. This means that any clouds who cannot infer the provider id by the node name from a remote location will never remove nodes that no longer exist.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes#50985
**Special notes for your reviewer**:
We'll want to create a subsequent issue to track the implementation of these two new methods in the cloud providers.
**Release note**:
```release-note
Adds `InstanceExists` and `InstanceExistsByProviderID` to cloud provider interface for the cloud controller manager
```
/cc @wlan0 @thockin @andrewsykim @luxas @jhorwit2
/area cloudprovider
/sig cluster-lifecycle
