Automatic merge from submit-queue (batch tested with PRs 65024, 65287, 65345, 64693, 64941). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
complete ipvs proxier ut
**What this PR does / why we need it**:
Complete ipvs proxier ut:
1. Finish loadbalancer type service test.
2. Add external ip local case.
3. Add UDP type nodeport test.
4. Add TestCleanupLeftovers test.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61390
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
update ipvs readme file
**What this PR does / why we need it**:
Since some iptables rules that ipvs proxier installed have changed, we need update ipvs readme file.
**Special notes for your reviewer**:
1. fix some error, like nodePort realization.
2. Update iptables rules.
3. Add externalIP
4. Add ipset sets introduction
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Do not need to set SupportIPVSProxyMode after 1.10.
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
`SupportIPVSProxyMode` is `true` by default since 1.10, there is no need to set it explicitly any more.
/area ipvs
/cc @m1093782566
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix bug excludeCIDRs was not assign in func NewProxier
**What this PR does / why we need it**:
fix bug excludeCIDRs was not assign in func NewProxier
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Init ipvsInterface only when ipvs modules are present
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61074
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Update ipvs docs --- check the prerequisite
**What this PR does / why we need it**:
As #63850 have been merged, the `Prerequisite` should be updated.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 64034, 64072, 64146, 64059, 64161). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix session affinity for LoadBalancer service with ESIPP
**What this PR does / why we need it**:
fix session affinity for LoadBalancer service with ESIPP
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63351
**Special notes for your reviewer**:
In cases that loadbalancer type service with externaltrafficpolicy=local and session-affinity specified, traffic to loadbalancer should only route to backends that in the same node with kube-proxy.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Enable checking whether ipvs modules are built-in or not
**What this PR does / why we need it**:
Enable checking whether ipvs modules are built-in or not.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#63801
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63787, 62003). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix localport open with --nodeport-addresses specified
**What this PR does / why we need it**:
Fix localport open with --nodeport-addresses specified.
**Which issue(s) this PR fixes**:
Fixes#61953
**Special notes for your reviewer**:
@ephur
**Release note**:
```release-note
Services can listen on same host ports on different interfaces with --nodeport-addresses specified
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Fix problem that ipvs can't work with hostPort
**What this PR does / why we need it**:
Make ipvs proxy mode can work with pods that have hostPort.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61938#60688 and #60305 are related too.
**Special notes for your reviewer**:
IPVS proxier will create dummy device named `kube-ipvs0`, which will maintain all ipvs virtual service address. That means all ipvs maintained clusterIP/externalIP/ingress will be treat as local address.
Then if we have a pod with hostPort, cni will attach this rule to `PREROUTING` chain:
```
KUBE-HOSTPORTS all -- 0.0.0.0/0 0.0.0.0/0 /* kube hostport portals */ ADDRTYPE match dst-type LOCAL
```
so if a service have same port with pod's hostport, then this service can't be access.
In this pr, we added `ACCESS` rule for traffic that aim to ipvs virtual service, to prevent those traffic from be blocked by other rules.
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 63129, 63066, 60009, 63136, 63086). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
ipvs proxy bind externalIP and lb ingress.IP to dummy interface
**What this PR does / why we need it**:
IPVS proxy bind externalIP and lb ingress.IP to dummy interface.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#59976
**Special notes for your reviewer**:
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Add --ipvs-exclude-cidrs flag to kube-proxy.
**What this PR does / why we need it**:
Add a flag to kube-proxy called --ipvs-exclude-cidrs. This flag allows a user to specify a list of CIDR ranges that should not be included in the cleanup of IPVS rules.
Fixes: #59507
**Release note**:
```
Use --ipvs-exclude-cidrs to specify a list of CIDR's which the IPVS proxier should not touch when cleaning up IPVS rules.
```
/assign @m1093782566
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
fix externaltrafficpolicy=local related ipvs ci case
**What this PR does / why we need it**:
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#61328
**Special notes for your reviewer**:
To realize externalTrafficPolicy=local, but do not affect traffic inside the cluster.
If thie pr got merged, the iptables rules of ipvs proxy mode ESIPP should be like(for loadbalance case):
```
Chain KUBE-FIRE-WALL (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-LB-INGRESS-LOCAL dst,dst
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0 /* mark MASQ for external traffic policy not local */
Chain KUBE-MARK-DROP (0 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x8000
Chain KUBE-MARK-MASQ (3 references)
target prot opt source destination
MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODE-PORT (1 references)
target prot opt source destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-NODE-PORT-LOCAL-TCP dst
KUBE-MARK-MASQ all -- 0.0.0.0/0 0.0.0.0/0 /* mark MASQ for external traffic policy not local */
Chain KUBE-POSTROUTING (0 references)
target prot opt source destination
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000
MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-LOOP-BACK dst,dst,src
Chain KUBE-SERVICES (2 references)
target prot opt source destination
KUBE-MARK-MASQ all -- !10.64.0.0/14 0.0.0.0/0 match-set KUBE-CLUSTER-IP dst,dst
KUBE-FIRE-WALL all -- 0.0.0.0/0 0.0.0.0/0 match-set KUBE-LOAD-BALANCER-MASQ dst,dst
KUBE-NODE-PORT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp match-set KUBE-NODE-PORT-TCP dst
```
**Release note**:
```release-note
NONE
```
Automatic merge from submit-queue (batch tested with PRs 53689, 56880, 55856, 59289, 60249). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Harden kube-proxy for unmatched IP versions
**What this PR does / why we need it**:
This PR makes kube-proxy omits & logs & emits event for unmatched IP versions configuration (IPv6 address in IPv4 mode or IPv4 address in IPv6 mode).
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes#57219
**Special notes for your reviewer**:
**Release note**:
```release-note
Fix the issue in kube-proxy iptables/ipvs mode to properly handle incorrect IP version.
```
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
Remove old featureGate flag in Readme
**What this PR does / why we need it**:
This PR corrects the Readme for ipvs mode: https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/README.md. `kubeadm init` doesn't support the `--feature-gates=SupportIPVSProxyMode=true` flag anymore since IPVS is in beta and not alpha anymore.
```
sudo kubeadm init --help
Run this command in order to set up the Kubernetes master.
Usage:
kubeadm init [flags]
Flags:
--apiserver-advertise-address string The IP address the API Server will advertise it's listening on. Specify '0.0.0.0' to use the address of the default network interface.
--apiserver-bind-port int32 Port for the API Server to bind to. (default 6443)
--apiserver-cert-extra-sans stringSlice Optional extra Subject Alternative Names (SANs) to use for the API Server serving certificate. Can be both IP addresses and DNS names.
--cert-dir string The path where to save and store the certificates. (default "/etc/kubernetes/pki")
--config string Path to kubeadm config file. WARNING: Usage of a configuration file is experimental.
--cri-socket string Specify the CRI socket to connect to. (default "/var/run/dockershim.sock")
--dry-run Don't apply any changes; just output what would be done.
--feature-gates string A set of key=value pairs that describe feature gates for various features. Options are:
CoreDNS=true|false (ALPHA - default=false)
DynamicKubeletConfig=true|false (ALPHA - default=false)
SelfHosting=true|false (ALPHA - default=false)
StoreCertsInSecrets=true|false (ALPHA - default=false)
--ignore-preflight-errors stringSlice A list of checks whose errors will be shown as warnings. Example: 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks.
--kubernetes-version string Choose a specific Kubernetes version for the control plane. (default "stable-1.9")
--node-name string Specify the node name.
--pod-network-cidr string Specify range of IP addresses for the pod network. If set, the control plane will automatically allocate CIDRs for every node.
--service-cidr string Use alternative range of IP address for service VIPs. (default "10.96.0.0/12")
--service-dns-domain string Use alternative domain for services, e.g. "myorg.internal". (default "cluster.local")
--skip-token-print Skip printing of the default bootstrap token generated by 'kubeadm init'.
--token string The token to use for establishing bidirectional trust between nodes and masters.
--token-ttl duration The duration before the bootstrap token is automatically deleted. If set to '0', the token will never expire. (default 24h0m0s)
```
**Special notes for your reviewer**:
Used kubeadm version:
```bash
sudo kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.3", GitCommit:"d2835416544f298c919e2ead3be3d0864b52323b", GitTreeState:"clean", BuildDate:"2018-02-07T11:55:20Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
```
**Release note**:
```release-note
NONE
```
