e6a2c07902
* Use delegated plugin call to simplify cni-bridge-cni * Add README.md for cni-bridge-cni Signed-off-by: Wei Fu <fuweid89@gmail.com>
160 lines
3.6 KiB
Markdown
160 lines
3.6 KiB
Markdown
## cni-bridge-f(ail)p(oint)
|
|
|
|
### Overview
|
|
|
|
The `cni-bridge-fp` is a CNI plugin which delegates interface-creating function
|
|
to [CNI bridge plugin][1] and allows user to inject failpoint before delegation.
|
|
|
|
Since the CNI plugin is invoked by binary call from CRI and it is short-lived,
|
|
the failpoint need to be configured by a JSON file, which can be persisted.
|
|
There is an example about failpoint description.
|
|
|
|
```json
|
|
{
|
|
"cmdAdd": "1*error(you-shall-not-pass!)->1*panic(again)",
|
|
"cmdDel": "1*error(try-again)",
|
|
"cmdCheck": "10*off"
|
|
}
|
|
```
|
|
|
|
* `cmdAdd` (string, optional): The failpoint for `ADD` command.
|
|
* `cmdDel` (string, optional): The failpoint for `DEL` command.
|
|
* `cmdCheck` (string, optional): The failpoint for `CHECK` command.
|
|
|
|
Since the `cmdXXX` can be multiple failpoints, each CNI binary call will update
|
|
the current state to make sure the order of execution is expected.
|
|
|
|
And the failpoint injection is enabled by pod's annotation. Currently, the key
|
|
of customized CNI capabilities in containerd can only be `io.kubernetes.cri.pod-annotations`
|
|
and containerd will pass pod's annotations to CNI under the that object. The
|
|
user can use the `failpoint.cni.containerd.io/confpath` annotation to enable
|
|
failpoint for the pod.
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Pod
|
|
metadata:
|
|
name: nginx
|
|
annotations:
|
|
failpoint.cni.containerd.io/confpath: "/tmp/pod-failpoints.json"
|
|
spec:
|
|
containers:
|
|
- name: nginx
|
|
image: nginx:1.14.2
|
|
ports:
|
|
- containerPort: 80
|
|
```
|
|
|
|
### Example
|
|
|
|
Let's use the following json as failpoint description.
|
|
|
|
```bash
|
|
$ cat <<EOF | tee /tmp/cni-failpoint.json
|
|
{
|
|
"cmdAdd": "1*error(try-again)",
|
|
"cmdDel": "2*error(oops)",
|
|
"cmdCheck": "1*off->1*panic(sorry)"
|
|
}
|
|
EOF
|
|
```
|
|
|
|
And use `ip netns` to create persisted net namespace named by `failpoint`.
|
|
|
|
```bash
|
|
$ sudo ip netns add failpoint
|
|
```
|
|
|
|
And then setup the following bash script for demo.
|
|
|
|
```bash
|
|
$ cat <<EOFDEMO | tee /tmp/cni-failpoint-demo-helper.sh
|
|
#!/usr/bin/env bash
|
|
|
|
export CNI_CONTAINERID=failpoint-testing
|
|
export CNI_NETNS=/run/netns/failpoint
|
|
export CNI_IFNAME=fpeni0
|
|
export CNI_PATH=/opt/cni/bin/
|
|
|
|
cat <<EOF | /opt/cni/bin/cni-bridge-fp
|
|
{
|
|
"cniVersion": "0.3.0",
|
|
"name": "containerd-net-fp",
|
|
"type": "cni-bridge-fp",
|
|
"bridge": "fp-cni0",
|
|
"isGateway": true,
|
|
"ipMasq": true,
|
|
"promiscMode": true,
|
|
"ipam": {
|
|
"type": "host-local",
|
|
"ranges": [
|
|
[{
|
|
"subnet": "10.88.0.0/16"
|
|
}],
|
|
[{
|
|
"subnet": "2001:4860:4860::/64"
|
|
}]
|
|
],
|
|
"routes": [
|
|
{ "dst": "0.0.0.0/0" },
|
|
{ "dst": "::/0" }
|
|
]
|
|
},
|
|
"runtimeConfig": {
|
|
"io.kubernetes.cri.pod-annotations": {
|
|
"failpoint.cni.containerd.io/confpath": "/tmp/cni-failpoint.json"
|
|
}
|
|
}
|
|
}
|
|
EOF
|
|
|
|
EOFDEMO
|
|
```
|
|
|
|
Let's try to setup CNI and we should get a error `try-again`.
|
|
|
|
```bash
|
|
$ sudo CNI_COMMAND=ADD bash /tmp/cni-failpoint-demo-helper.sh
|
|
{
|
|
"code": 999,
|
|
"msg": "try-again"
|
|
}
|
|
|
|
# there is no failpoint for ADD command.
|
|
$ cat /tmp/cni-failpoint.json | jq .
|
|
{
|
|
"cmdAdd": "0*error(try-again)",
|
|
"cmdDel": "2*error(oops)",
|
|
"cmdCheck": "1*off->1*panic(sorry)"
|
|
}
|
|
```
|
|
|
|
We should setup CNI successfully after retry. When we teardown the interface,
|
|
there should be two failpoints.
|
|
|
|
```bash
|
|
$ sudo CNI_COMMAND=ADD bash /tmp/cni-failpoint-demo-helper.sh
|
|
...
|
|
|
|
$ sudo CNI_COMMAND=DEL bash /tmp/cni-failpoint-demo-helper.sh
|
|
{
|
|
"code": 999,
|
|
"msg": "oops"
|
|
}
|
|
|
|
$ sudo CNI_COMMAND=DEL bash /tmp/cni-failpoint-demo-helper.sh
|
|
{
|
|
"code": 999,
|
|
"msg": "oops"
|
|
}
|
|
|
|
$ cat /tmp/cni-failpoint.json | jq .
|
|
{
|
|
"cmdAdd": "0*error(try-again)",
|
|
"cmdDel": "0*error(oops)",
|
|
"cmdCheck": "1*off->1*panic(sorry)"
|
|
}
|
|
```
|
|
|
|
[1]: <https://www.cni.dev/plugins/current/main/bridge/>
|