Containerd basic Commands and Usage

Containerd basic Commands and Usage
Containerd basic Commands and Usage


In Kubernetes 1.24 and newer versions, opting for containerd as the runtime is advisable. Let’s explore the differences in directory structure between containerd and Docker.

What is containerd

Containerd is a container runtime that manages the complete container lifecycle, including image transfer, storage, execution, and network attachments. It supports the Open Container Initiative (OCI) runtime specification and is designed to be embedded into larger systems. Containerd is used by Kubernetes and other container orchestration tools for managing containers. It is open-source and part of the Cloud Native Computing Foundation (CNCF).

containerd

Basic Containerd filesystem locations

  • When containerd pulls an image, it is stored in /var/lib/containerd/io.containerd.content.v1.content/blob/sha256.
  • When containerd is the runtime, the standard output (stdout) log file is stored in /var/log/containers.
  • For containerd as the runtime, all running files (merger layer) are stored in /run/containerd/io.containerd.runtime.v2.task/k8s.io/.
  • In containerd as the runtime, both lower and upper layers exist in /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/.
  • In containerd as the runtime, empty mounts of the container are stored in /var/lib/kubelet/pods/****/volumes/kubernetes.io~empty-dir/.

Containerd Installation

There are two installation methods. One involves installing Docker, as Docker itself includes containerd installation, requiring only minimal configuration adjustments. The other method is to directly download and use the binary package provided by the official GitHub, and we opt for the latter.

Download the containerd tar package and extract it to /usr/local.

tar Cxzvf /usr/local containerd-1.6.2-linux-amd64.tar.gz

Configure containerd:

containerd config default > /etc/containerd/config.toml

# Configure containerd to use systemd as cgroup
systemd_cgroup = true

# Adjust directories as needed
# 'root' stores persistent data including Snapshots, Content, Metadata, and various plugin data. Each plugin has its own directory, and containerd itself does not store any data. All its functionalities come from loaded plugins.
root = "/var/lib/containerd"

# 'state' stores temporary runtime data, including sockets, PID, mount points, runtime status, and plugin data that doesn't need persistence.
state = "/run/containerd"

If you prefer using systemd to manage containerd, create the /usr/lib/systemd/system/containerd.service file:

Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target

[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerd

Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
LimitNPROC=655350
LimitCORE=655350
LimitNOFILE=655350
TasksMax=infinity
OOMScoreAdjust=-999

[Install]
WantedBy=multi-user.target

After creating the service file, reload and enable auto-start at boot:

systemctl daemon-reload
systemctl enable --now containerd
systemctl restart containerd

Install runc

To install runc, which is a dependency for containerd, we need to download the latest version from the official GitHub.

Download the runc tar package:

# Execute this command to install runc.
install -m 755 runc.amd64 /usr/local/sbin/runc

This command sets the permissions and installs runc to /usr/local/sbin.

Install CNI

containerd lacks built-in network-related functionalities and requires the integration of a CNI (Container Network Interface) plugin. Otherwise, attempting to create a container will result in a CNI error.

To address this, download the CNI plugin tar package

$ mkdir -p /opt/cni/bin
$ tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.2.0.tgz
./
./macvlan
./static
./vlan
./portmap
./host-local
./vrf
./bridge
./tuning
./firewall
./host-device
./sbr
./loopback
./dhcp
./ptp
./ipvlan
./bandwidth

# restart container
systemctl start containerd
systemctl status containerd

Management Tools

Here’s the information about management tools:

  • ctr: This is a command line client provided as part of the containerd project. It’s primarily used for debugging purposes. You can check its help information using ctr --help.
  • nerdctl: This tool is more user-friendly and compatible with Docker commands. It’s suitable for standalone nodes and personal use, providing an experience similar to Docker with minimal learning costs. You can replace Docker commands with nerdctl commands. More information can be found on the nerdctl official GitHub page.
  • crictl: This command line client utilizes the Kubernetes CRI-compatible container runtime interface to manage containers and Pods primarily in Kubernetes environments. It’s primarily used for debugging purposes. You can find more information on the crictl official GitHub page.

Although these tools have their own characteristics, they may not be particularly easy to use. For example, when using nerdctl, mapping a host port to a container may not be immediately reflected in netstat -tnlp output on the host, despite the mapping being successful.

Here’s a comparison overview of the tools:

ToolCommunityAPITargetHelp Information
ctrcontainerdNativeDebugging onlyctr --help
nerdctlcontainerd (non-core)NativeGeneral-purposenerdctl official GitHub
crictlKubernetes SIG-nodeCRIDebugging onlycrictl official GitHub

Comparing tools and commands, crictl is more recommended in Kubernetes scenarios. Here’s a comparison of commands:

OrderDockerCrictl (recommended)Ctr
View container listdocker pscrictl psctr -n k8s.io c ls
View container detailsdocker inspectcrictl inspectctr -n k8s.io c info
View container logsdocker logscrictl logsnone
Execute commands within the containerdocker execcrictl execnone
Mount containerdocker attachcrictl attachnone
Container resource usagedocker statscrictl statsnone
Create containerdocker createcrictl createctr -n k8s.io c create
Start containerdocker startcrictl startctr -n k8s.io run
Stop containerdocker stopcrictl stopnone
Delete containerdocker rmcrictl rmctr -n k8s.io c del
View image listdocker imagescrictl imagesctr -n k8s.io i ls
View image detailsdocker inspectcrictl inspectinone
Pull imagedocker pullcrictl pullctr -n k8s.io i pull
Push imagedocker pushnonectr -n k8s.io i push
Delete imagedocker rmicrictl rmictr -n k8s.io i rm
View Pod listnonecrictl podsnone
View Pod detailsnonecrictl inspectpnone
Start Podnonecrictl runpnone
Stop Podnonecrictl stopnone


To install a single-node Kubernetes cluster and test it, follow these steps:

  1. First, configure the yum source:
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
  1. Install kubelet, kubectl, and kubeadm:
yum install -y kubelet-1.24.11 kubeadm-1.24.11 kubectl-1.24.11 --disableexcludes=kubernet
  1. Initialize a single-node Kubernetes cluster:
kubeadm init --apiserver-advertise-address=192.168.22.226 --image-repository registry


To install a single-node Kubernetes cluster and test it, follow these steps:

  1. First, configure the yum source:
cat > /etc/yum.repos.d/kubernetes.repo << EOF
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
  1. Install kubelet, kubectl, and kubeadm:
yum install -y kubelet-1.24.11 kubeadm-1.24.11 kubectl-1.24.11 --disableexcludes=kubernetes
  1. Initialize a single-node Kubernetes cluster:
kubeadm init --apiserver-advertise-address=192.168.22.226 --image-repository registry.aliyuncs.com/google_containers --control-plane-endpoint=192.168.22.226 --kubernetes-version v1.24.11 --service-cidr=10.1.0.0/16 --pod-network-cidr=10.244.0.0/16 --v=5
  1. View all pods using nerdctl:
$ nerdctl -n k8s.io ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
01228b6dd482 registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 13 minutes ago Up k8s://kube-system/coredns-74586cf9b6-p42vf
2bdd9d045757 registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 13 minutes ago Up k8s://kube-system/kube-scheduler-dev01
37cbd0677e1a registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 13 minutes ago Up k8s://kube-system/etcd-dev01
40ca90ad1c4f registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 13 minutes ago Up k8s://kube-system/kube-controller-manager-dev01
451f34d736f1 registry.aliyuncs.com/google_containers/coredns:v1.8.6 "/coredns -conf /etc…" 13 minutes ago Up k8s://kube-system/coredns-74586cf9b6-jwt87/coredns
5152d7631cb2 registry.aliyuncs.com/google_containers/coredns:v1.8.6 "/coredns -conf /etc…" 13 minutes ago Up k8s://kube-system/coredns-74586cf9b6-p42vf/coredns
6040f26266b9 registry.aliyuncs.com/google_containers/kube-controller-manager:v1.24.11 "kube-controller-man…" 13 minutes ago Up k8s://kube-system/kube-controller-manager-dev01/kube-controller-manager
73f778808fab registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 13 minutes ago Up k8s://kube-system/kube-apiserver-dev01
9fa31ae29788 registry.aliyuncs.com/google_containers/etcd:3.5.6-0 "etcd --advertise-cl…" 13 minutes ago Up k8s://kube-system/etcd-dev01/etcd
a06a11848bfd registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 13 minutes ago Up k8s://kube-system/coredns-74586cf9b6-jwt87
b30c34841895 registry.aliyuncs.com/google_containers/kube-apiserver:v1.24.11 "kube-apiserver --ad…" 13 minutes ago Up k8s://kube-system/kube-apiserver-dev01/kube-apiserver
b82098336ac6 registry.aliyuncs.com/google_containers/kube-proxy:v1.24.11 "/usr/local/bin/kube…" 13 minutes ago Up k8s://kube-system/kube-proxy-j264w/kube-proxy
ba0cfb9ec381 registry.aliyuncs.com/google_containers/pause:3.6 "/pause" 13 minutes ago Up k8s://kube-system/kube-proxy-j264w
df4c61aabb97 registry.aliyuncs.com/google_containers/kube-scheduler:v1.24.11 "kube-scheduler --au…" 13 minutes ago Up k8s://kube-system/kube-scheduler-dev01/kube-scheduler
  1. Alternatively, view all pods using crictl:
$ crictl ps

CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
5152d7631cb25 a4ca41631cc7a 11 minutes ago Running coredns 0 01228b6dd4828 coredns-74586cf9b6-p42vf
451f34d736f13 a4ca41631cc7a 11 minutes ago Running coredns 0 a06a11848bfd8 coredns-74586cf9b6-jwt87
b82098336ac6c d6ccf30c8566e 11 minutes ago Running kube-proxy 0 ba0cfb9ec381a kube-proxy-j264w
b30c348418959 0849bf5f3ef4e 11 minutes ago Running kube-apiserver 0 73f778808fab6 kube-apiserver-dev01
9fa31ae29788a fce326961ae2d 11 minutes ago Running etcd 0 37cbd0677e1a1 etcd-dev01
df4c61aabb976 b1ef1dbb09ca5 11 minutes ago Running kube-scheduler 0 2bdd9d045757a kube-scheduler-dev01
6040f26266b93 4b0f3887b66e7 11 minutes ago Running kube-controller-manager 0 40ca90ad1c4f8 kube-controller-manager-dev01

Directory Verification

To create a deployment and verify the storage path of containerd data on the host, follow these steps:

  • Create the deployment YAML:
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "2"
labels:
app: sample
name: sample
namespace: default
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: sample
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: sample
spec:
containers:
- image: nginx:alpine
imagePullPolicy: IfNotPresent
name: nginx
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/log/app
name: tmp-vol
volumes:
- emptyDir: {}
name: tmp-vol
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
terminationGracePeriodSeconds: 30
  • Viewing the pod UID:
uid:baf1e1a6-cf0e-474c-912d-e32a0d424c25
  • Entering the pod container and creating a file:
$ cd /var/log/app/
$ echo hello >> app.log
  • Entering the directory corresponding to the UID in the kubelet pods directory of the host machine:
$ cd /var/lib/kubelet/pods/baf1e1a6-cf0e-474c-912d-e32a0d424c25/volumes/kubernetes.io~empty-dir/tmp-vol/
$ cat app.log
hello
  • Creating a file within the /var/log/nginx directory of the container using touch test.txt.

Next, using crictl ps to check the container ID of the pod and entering the corresponding directory:

/run/containerd/io.containerd.runtime.v2.task/k8s.io/20220e35cc63208ac772bffed478d34a5bb54b0ffbb9a447d169f5b7415b3063/rootfs

This is equivalent to entering the / directory of the container, then entering /var/log/nginx. You can verify that test.txt also appears on the host machine, confirming the content we created above.

Conclusion

This time, I’ve simply documented some basic usage of containerd. As this article focuses on basic installation and exploration, our understanding of containerd remains at this foundational level. To uncover more of its wonderful capabilities, we must continue using and learning about it in the future.

Author

  • Mohamed BEN HASSINE

    Mohamed BEN HASSINE is a Hands-On Cloud Solution Architect based out of France. he has been working on Java, Web , API and Cloud technologies for over 12 years and still going strong for learning new things. Actually , he plays the role of Cloud / Application Architect in Paris ,while he is designing cloud native solutions and APIs ( REST , gRPC). using cutting edge technologies ( GCP / Kubernetes / APIGEE / Java / Python )

0 Shares:
You May Also Like
Scaling Kubernetes (GKE / AKS / EKS )
Read More

Scaling Kubernetes by Examples

Table of Contents Hide Scaling a Kubernetes DeploymentUse CaseSolutionImplementing Horizontal Pod AutoscalingUse CaseSolutionAutomating Cluster Scaling in GKEUse CaseSolutionDiscussionDynamically…