137 lines
7.1 KiB
Markdown
137 lines
7.1 KiB
Markdown
## Introduction
|
|
|
|
The Kubernetes iSCSI implementation can connect to iSCSI devices via open-iscsi and multipathd on Linux.
|
|
Currently supported features are
|
|
* Connecting to one portal
|
|
* Mounting a device directly or via multipathd
|
|
* Formatting and partitioning any new device connected
|
|
* CHAP authentication
|
|
|
|
## Prerequisites
|
|
|
|
This example expects there to be a working iSCSI target to connect to.
|
|
If there isn't one in place then it is possible to setup a software version on Linux by following these guides
|
|
|
|
* [Setup a iSCSI target on Fedora](http://www.server-world.info/en/note?os=Fedora_21&p=iscsi)
|
|
* [Install the iSCSI initiator on Fedora](http://www.server-world.info/en/note?os=Fedora_21&p=iscsi&f=2)
|
|
* [Install multipathd for mpio support if required](http://www.linuxstories.eu/2014/07/how-to-setup-dm-multipath-on-rhel.html)
|
|
|
|
|
|
## Creating the pod with iSCSI persistent storage
|
|
|
|
Once you have configured the iSCSI initiator, you can create a pod based on the example *iscsi.yaml*. In the pod YAML, you need to provide *targetPortal* (the iSCSI target's **IP** address and *port* if not the default port 3260), target's *iqn*, *lun*, and the type of the filesystem that has been created on the lun, and *readOnly* boolean. No initiator information is required. If you have more than one target portals for a single IQN, you can mention other portal IPs in *portals* field.
|
|
|
|
If you want to use an iSCSI offload card or other open-iscsi transports besides tcp, setup an iSCSI interface and provide *iscsiInterface* in the pod YAML. The default name for an iscsi iface (open-iscsi parameter iface.iscsi\_ifacename) is in the format transport\_name.hwaddress when generated by iscsiadm. See [open-iscsi](http://www.open-iscsi.org/docs/README) or [openstack](http://docs.openstack.org/kilo/config-reference/content/iscsi-iface-config.html) for detailed configuration information.
|
|
|
|
**Note:** If you have followed the instructions in the links above you
|
|
may have partitioned the device, the iSCSI volume plugin does not
|
|
currently support partitions so format the device as one partition or leave the device raw and Kubernetes will partition and format it one first mount.
|
|
|
|
### CHAP Authentication
|
|
|
|
To enable one-way or two-way CHAP authentication for discovery or session, following these steps.
|
|
|
|
* Set `chapAuthDiscovery` to `true` for discovery authentication.
|
|
* Set `chapAuthSession` to `true` for session authentication.
|
|
* Create a CHAP secret and set `secretRef` to reference the CHAP secret.
|
|
|
|
|
|
Example can be found at [iscsi-chap.yaml](iscsi-chap.yaml)
|
|
|
|
### CHAP Secret
|
|
|
|
As illustrated in [chap-secret.yaml](chap-secret.yaml), the secret must have type `kubernetes.io/iscsi-chap` and consists of the following keys:
|
|
|
|
```yaml
|
|
---
|
|
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: chap-secret
|
|
type: "kubernetes.io/iscsi-chap"
|
|
data:
|
|
discovery.sendtargets.auth.username:
|
|
discovery.sendtargets.auth.password:
|
|
discovery.sendtargets.auth.username_in:
|
|
discovery.sendtargets.auth.password_in:
|
|
node.session.auth.username:
|
|
node.session.auth.password:
|
|
node.session.auth.username_in:
|
|
node.session.auth.password_in:
|
|
```
|
|
|
|
These keys map to those used by Open-iSCSI initiator. Detailed documents on these keys can be found at [Open-iSCSI](https://github.com/open-iscsi/open-iscsi/blob/master/etc/iscsid.conf)
|
|
|
|
#### Create CHAP secret before creating iSCSI volumes and Pods
|
|
|
|
```console
|
|
# kubectl create -f examples/volumes/iscsi/chap-iscsi.yaml
|
|
```
|
|
|
|
|
|
|
|
Once the pod config is created, run it on the Kubernetes master:
|
|
|
|
```console
|
|
kubectl create -f ./your_new_pod.yaml
|
|
```
|
|
|
|
Here is the example pod created and expected output:
|
|
|
|
```console
|
|
# kubectl create -f examples/volumes/iscsi/iscsi.yaml
|
|
# kubectl get pods
|
|
NAME READY STATUS RESTARTS AGE
|
|
iscsipd 2/2 RUNNING 0 2m
|
|
```
|
|
|
|
On the Kubernetes node, verify the mount output
|
|
|
|
For a non mpio device the output should look like the following
|
|
|
|
```console
|
|
# mount |grep kub
|
|
/dev/sdb on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.15:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
|
|
/dev/sdb on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw type ext4 (ro,relatime,data=ordered)
|
|
/dev/sdc on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.16:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
|
|
/dev/sdc on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw type ext4 (rw,relatime,data=ordered)
|
|
/dev/sdd on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.17:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
|
|
/dev/sdd on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw type ext4 (rw,relatime,data=ordered)
|
|
```
|
|
|
|
And for a node with mpio enabled the expected output would be similar to the following
|
|
|
|
```console
|
|
# mount |grep kub
|
|
/dev/mapper/mpatha on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.15:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
|
|
/dev/mapper/mpatha on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-ro type ext4 (ro,relatime,data=ordered)
|
|
/dev/mapper/mpathb on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.16:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
|
|
/dev/mapper/mpathb on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw type ext4 (rw,relatime,data=ordered)
|
|
/dev/mapper/mpathc on /var/lib/kubelet/plugins/kubernetes.io/iscsi/10.0.2.17:3260-iqn.2001-04.com.example:storage.kube.sys1.xyz-lun-0 type ext4 (rw,relatime,data=ordered)
|
|
/dev/mapper/mpathb on /var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw type ext4 (rw,relatime,data=ordered)
|
|
```
|
|
|
|
|
|
If you ssh to that machine, you can run `docker ps` to see the actual pod.
|
|
|
|
```console
|
|
# docker ps
|
|
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
|
|
3b8a772515d2 kubernetes/pause "/pause" 6 minutes ago Up 6 minutes k8s_iscsipd-rw.ed58ec4e_iscsipd_default_f527ca5b-6d87-11e5-aa7e-080027ff6387_d25592c5
|
|
```
|
|
|
|
Run *docker inspect* and verify the container mounted the host directory into the their */mnt/iscsipd* directory.
|
|
|
|
```console
|
|
# docker inspect --format '{{ range .Mounts }}{{ if eq .Destination "/mnt/iscsipd" }}{{ .Source }}{{ end }}{{ end }}' f855336407f4
|
|
/var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-ro
|
|
|
|
# docker inspect --format '{{ range .Mounts }}{{ if eq .Destination "/mnt/iscsipd" }}{{ .Source }}{{ end }}{{ end }}' 3b8a772515d2
|
|
/var/lib/kubelet/pods/f527ca5b-6d87-11e5-aa7e-080027ff6387/volumes/kubernetes.io~iscsi/iscsipd-rw
|
|
```
|
|
|
|
|
|
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
|
|
[]()
|
|
<!-- END MUNGE: GENERATED_ANALYTICS -->
|