Merge containerd/cri into containerd/containerd

Signed-off-by: Derek McGowan <derek@mcg.dev>
2020-10-07 10:58:39 -07:00
parent d852786705 0820015314
commit e7a350176a
272 changed files with 35258 additions and 0 deletions
--- a/contrib/ansible/README.md
+++ b/contrib/ansible/README.md
@@ -0,0 +1,122 @@
+# Kubernetes Cluster with Containerd
+<p align="center">
+<img src="https://kubernetes.io/images/favicon.png" width="50" height="50">
+<img src="https://containerd.io/img/logos/icon/black/containerd-icon-black.png" width="50" >
+</p>
+
+
+This document provides the steps to bring up a Kubernetes cluster using ansible and kubeadm tools.
+
+### Prerequisites:
+- **OS**: Ubuntu 16.04 (will be updated with additional distros after testing)
+- **Python**: 2.7+
+- **Ansible**: 2.4+
+
+## Step 0:
+-  Install Ansible on the host where you will provision the cluster. This host may be one of the nodes you plan to include in your cluster. Installation instructions for Ansible are found [here](http://docs.ansible.com/ansible/latest/intro_installation.html).
+-  Create a hosts file and include the IP addresses of the hosts that need to be provisioned by Ansible.
+```console
+$ cat hosts
+172.31.7.230
+172.31.13.159
+172.31.1.227
+```
+-  Setup passwordless SSH access from the host where you are running Ansible to all the hosts in the hosts file. The instructions can be found in [here](http://www.linuxproblem.org/art_9.html)
+
+## Step 1:
+At this point, the ansible playbook should be able to ssh into the machines in the hosts file.
+```console
+git clone https://github.com/containerd/cri
+cd ./cri/contrib/ansible
+ansible-playbook -i hosts cri-containerd.yaml
+```
+A typical cloud login might have a username and private key file, in which case the following can be used:
+```console
+ansible-playbook -i hosts -u <username> --private-key <example.pem> cri-containerd.yaml
+ ```
+For more options ansible config file (/etc/ansible/ansible.cfg) can be used to set defaults. Please refer to [Ansible options](http://docs.ansible.com/ansible/latest/intro_configuration.html) for advanced ansible configurations.
+
+At the end of this step, you will have the required software installed in the hosts to bringup a kubernetes cluster.
+```console
+PLAY RECAP ***************************************************************************************************************************************************************
+172.31.1.227               : ok=21   changed=7    unreachable=0    failed=0
+172.31.13.159              : ok=21   changed=7    unreachable=0    failed=0
+172.31.7.230               : ok=21   changed=7    unreachable=0    failed=0
+```
+
+## Step 2:
+Use [kubeadm](https://kubernetes.io/docs/setup/independent/install-kubeadm/) to bring up a Kubernetes Cluster. Depending on what third-party provider you choose, you might have to set the ```--pod-network-cidr``` to something provider-specific.
+Initialize the cluster from one of the nodes (Note: This node will be the master node):
+```console
+$sudo kubeadm init --skip-preflight-checks
+[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
+[init] Using Kubernetes version: v1.7.6
+[init] Using Authorization modes: [Node RBAC]
+[preflight] Skipping pre-flight checks
+[kubeadm] WARNING: starting in 1.8, tokens expire after 24 hours by default (if you require a non-expiring token use --token-ttl 0)
+[certificates] Generated CA certificate and key.
+[certificates] Generated API server certificate and key.
+[certificates] API Server serving cert is signed for DNS names [abhi-k8-ubuntu-1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 172.31.7.230]
+[certificates] Generated API server kubelet client certificate and key.
+[certificates] Generated service account token signing key and public key.
+[certificates] Generated front-proxy CA certificate and key.
+[certificates] Generated front-proxy client certificate and key.
+[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
+[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
+[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
+[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
+[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
+[apiclient] Created API client, waiting for the control plane to become ready
+[apiclient] All control plane components are healthy after 42.002391 seconds
+[token] Using token: 43a25d.420ff2e06336e4c1
+[apiconfig] Created RBAC rules
+[addons] Applied essential addon: kube-proxy
+[addons] Applied essential addon: kube-dns
+
+Your Kubernetes master has initialized successfully!
+
+To start using your cluster, you need to run (as a regular user):
+
+  mkdir -p $HOME/.kube
+  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
+  sudo chown $(id -u):$(id -g) $HOME/.kube/config
+
+You should now deploy a pod network to the cluster.
+Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
+  http://kubernetes.io/docs/admin/addons/
+
+You can now join any number of machines by running the following on each node
+as root:
+
+  kubeadm join --token 43a25d.420ff2e06336e4c1 172.31.7.230:6443
+
+```
+## Step 3:
+Use kubeadm join to add each of the remaining nodes to your cluster. (Note: Uses token that was generated during cluster init.)
+```console
+$sudo kubeadm join --token 43a25d.420ff2e06336e4c1 172.31.7.230:6443 --skip-preflight-checks
+[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
+[preflight] Skipping pre-flight checks
+[discovery] Trying to connect to API Server "172.31.7.230:6443"
+[discovery] Created cluster-info discovery client, requesting info from "https://172.31.7.230:6443"
+[discovery] Cluster info signature and contents are valid, will use API Server "https://172.31.7.230:6443"
+[discovery] Successfully established connection with API Server "172.31.7.230:6443"
+[bootstrap] Detected server version: v1.7.6
+[bootstrap] The server supports the Certificates API (certificates.k8s.io/v1beta1)
+[csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request
+[csr] Received signed certificate from the API server, generating KubeConfig...
+[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
+
+Node join complete:
+* Certificate signing request sent to master and response
+  received.
+* Kubelet informed of new secure connection details.
+
+Run 'kubectl get nodes' on the master to see this machine join.
+```
+At the end of Step 3 you should have a kubernetes cluster up and running and ready for deployment.
+
+## Step 4:
+Please follow the instructions [here](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network) to deploy CNI network plugins and start a demo app.
+
+We are constantly striving to improve the installer. Please feel free to open issues and provide suggestions to make the installer fast and easy to use. We are open to receiving help in validating and improving the installer on different distros.
--- a/contrib/ansible/cri-containerd.yaml
+++ b/contrib/ansible/cri-containerd.yaml
@@ -0,0 +1,66 @@
+---
+- hosts: all
+  become: true
+  tasks:
+    - include_vars: vars/vars.yaml # Contains tasks variables for installer
+    - include_tasks: tasks/bootstrap_ubuntu.yaml # Contains tasks bootstrap components for ubuntu systems
+      when: ansible_distribution == "Ubuntu"
+    - include_tasks: tasks/bootstrap_centos.yaml # Contains tasks bootstrap components for centos systems
+      when: ansible_distribution == "CentOS"
+    - include_tasks: tasks/k8s.yaml # Contains tasks kubernetes component installation
+    - include_tasks: tasks/binaries.yaml # Contains tasks for pulling containerd components
+
+    - name: "Create a directory for containerd config"
+      file: path=/etc/containerd state=directory
+
+    - name: "Start Containerd"
+      systemd: name=containerd daemon_reload=yes state=started enabled=yes
+
+    - name: "Load br_netfilter kernel module"
+      modprobe:
+        name: br_netfilter
+        state: present
+
+    - name: "Set bridge-nf-call-iptables"
+      sysctl:
+        name: net.bridge.bridge-nf-call-iptables
+        value: 1
+
+    - name: "Set ip_forward"
+      sysctl:
+        name: net.ipv4.ip_forward
+        value: 1
+
+    - name: "Check kubelet args in kubelet config (Ubuntu)"
+      shell: grep "^Environment=\"KUBELET_EXTRA_ARGS=" /etc/systemd/system/kubelet.service.d/10-kubeadm.conf || true
+      register: check_args
+      when: ansible_distribution == "Ubuntu"
+
+    - name: "Add runtime args in kubelet conf (Ubuntu)"
+      lineinfile:
+        dest: "/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"
+        line: "Environment=\"KUBELET_EXTRA_ARGS= --runtime-cgroups=/system.slice/containerd.service --container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock\""
+        insertafter: '\[Service\]'
+      when: ansible_distribution == "Ubuntu" and check_args.stdout == ""
+
+    - name: "Check kubelet args in kubelet config (CentOS)"
+      shell: grep "^Environment=\"KUBELET_EXTRA_ARGS=" /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf || true
+      register: check_args
+      when: ansible_distribution == "CentOS"
+
+    - name: "Add runtime args in kubelet conf (CentOS)"
+      lineinfile:
+        dest: "/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf"
+        line: "Environment=\"KUBELET_EXTRA_ARGS= --runtime-cgroups=/system.slice/containerd.service --container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock\""
+        insertafter: '\[Service\]'
+      when: ansible_distribution == "CentOS" and check_args.stdout == ""
+
+    - name: "Start Kubelet"
+      systemd: name=kubelet daemon_reload=yes state=started enabled=yes
+
+    # TODO This needs to be removed once we have consistent concurrent pull results
+    - name: "Pre-pull pause container image"
+      shell: |
+        /usr/local/bin/ctr pull k8s.gcr.io/pause:3.2
+        /usr/local/bin/crictl --runtime-endpoint unix:///run/containerd/containerd.sock \
+        pull k8s.gcr.io/pause:3.2
--- a/contrib/ansible/tasks/binaries.yaml
+++ b/contrib/ansible/tasks/binaries.yaml
@@ -0,0 +1,12 @@
+---
+- name: "Get Containerd"
+  unarchive:
+    src: "https://storage.googleapis.com/cri-containerd-release/cri-containerd-{{ containerd_release_version }}.linux-amd64.tar.gz"
+    dest: "/"
+    remote_src: yes
+
+- name: "Create a directory for cni binary"
+  file: path={{ cni_bin_dir }} state=directory
+
+- name: "Create a directory for cni config files"
+  file: path={{ cni_conf_dir }} state=directory
--- a/contrib/ansible/tasks/bootstrap_centos.yaml
+++ b/contrib/ansible/tasks/bootstrap_centos.yaml
@@ -0,0 +1,12 @@
+---
+- name: "Install required packages on CentOS "
+  yum:
+    name: "{{ item }}"
+    state: latest
+  with_items:
+    - unzip
+    - tar
+    - btrfs-progs
+    - libseccomp
+    - util-linux
+    - libselinux-python
--- a/contrib/ansible/tasks/bootstrap_ubuntu.yaml
+++ b/contrib/ansible/tasks/bootstrap_ubuntu.yaml
@@ -0,0 +1,12 @@
+---
+- name: "Install required packages on Ubuntu"
+  package:
+    name: "{{ item }}"
+    state: latest
+  with_items:
+      - unzip
+      - tar
+      - apt-transport-https
+      - btrfs-tools
+      - libseccomp2
+      - util-linux
--- a/contrib/ansible/tasks/k8s.yaml
+++ b/contrib/ansible/tasks/k8s.yaml
@@ -0,0 +1,52 @@
+---
+- name: "Add gpg key (Ubuntu)"
+  apt_key:
+    url: https://packages.cloud.google.com/apt/doc/apt-key.gpg
+    state: present
+  when: ansible_distribution == "Ubuntu"
+
+- name: "Add kubernetes source list (Ubuntu)"
+  apt_repository:
+    repo: "deb http://apt.kubernetes.io/ kubernetes-{{ ansible_distribution_release }} main"
+    state: present
+    filename: "kubernetes"
+  when: ansible_distribution == "Ubuntu"
+
+- name: "Update the repository cache (Ubuntu)"
+  apt: 
+    update_cache: yes
+  when: ansible_distribution == "Ubuntu"
+
+- name: "Add Kubernetes repository and install gpg key (CentOS)"
+  yum_repository:
+    name: kubernetes
+    description: Kubernetes repository
+    baseurl: https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
+    gpgcheck: yes
+    enabled: yes
+    repo_gpgcheck: yes
+    gpgkey: 
+      - https://packages.cloud.google.com/yum/doc/yum-key.gpg 
+      - https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
+  when: ansible_distribution == "CentOS"
+
+- name: "Disable SELinux (CentOS)"
+  selinux:
+    state: disabled
+  when: ansible_distribution == "CentOS"
+
+- name: "Install kubelet,kubeadm,kubectl (CentOS)"
+  yum: state=present name={{ item }}
+  with_items:
+    - kubelet
+    - kubeadm
+    - kubectl
+  when: ansible_distribution == "CentOS"
+
+- name: "Install kubelet, kubeadm, kubectl (Ubuntu)"
+  apt: name={{item}} state=installed
+  with_items:
+    - kubelet
+    - kubeadm
+    - kubectl
+  when: ansible_distribution == "Ubuntu"
--- a/contrib/ansible/vars/vars.yaml
+++ b/contrib/ansible/vars/vars.yaml
@@ -0,0 +1,4 @@
+---
+containerd_release_version: 1.3.0
+cni_bin_dir: /opt/cni/bin/
+cni_conf_dir: /etc/cni/net.d/
--- a/contrib/linuxkit/README.md
+++ b/contrib/linuxkit/README.md
@@ -0,0 +1,5 @@
+# LinuxKit Kubernetes project
+
+The LinuxKit [`projects/kubernetes`](https://github.com/linuxkit/linuxkit/tree/master/projects/kubernetes) subdirectory contains a project to build master and worker node virtual machines. When built with `KUBE_RUNTIME=cri-containerd` then these images will use `cri-containerd` as their execution backend.
+
+See the [project README](https://github.com/linuxkit/linuxkit/blob/master/projects/kubernetes/README.md).
--- a/contrib/systemd-units/containerd.service
+++ b/contrib/systemd-units/containerd.service
@@ -0,0 +1,22 @@
+[Unit]
+Description=containerd container runtime
+Documentation=https://containerd.io
+After=network.target
+
+[Service]
+ExecStartPre=/sbin/modprobe overlay
+ExecStart=/usr/local/bin/containerd
+Restart=always
+RestartSec=5
+Delegate=yes
+KillMode=process
+OOMScoreAdjust=-999
+LimitNOFILE=1048576
+# Having non-zero Limit*s causes performance problems due to accounting overhead
+# in the kernel. We recommend using cgroups to do container-local accounting.
+LimitNPROC=infinity
+LimitCORE=infinity
+TasksMax=infinity
+
+[Install]
+WantedBy=multi-user.target