COMP3000 Operating Systems F23: Tutorial 9

By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible.

General Instructions (same for all tutorials)

Click on Expand to view.

Tutorials are graded based on participation and effort (so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers), but you should still turn in your work. Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the Tasks/Questions section.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA (in person or by responding to a poll if online). Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.

Background

Docker containers

Docker is one of the OS-level virtualization technologies that allows the creation of isolated virtual execution environments, called containers, sharing the same OS kernel and certain runtime services/libraries with other containers. The workload running inside a container will feel as if it had its own everything, OS kernel, root file system, etc. This is possible thanks to the way Docker containers are designed and implemented. The Linux kernel does not directly support the construction of containers (the userspace libraries do) but its various building blocks play an important role, among which we will cover namespaces in this tutorial.

Docker images

A Docker image is a file containing a root file system as well as configuration parameters. It is very similar to other disk image formats in this sense. One of the main differences is that it employs a layered file system to maximize storage efficiency and allow flexible reuse of existing files.

Dockerfile

A Dockerfile is a text file and serves as instructions to build a Docker image. Unlike storing and distributing binary data directly in images, using Dockerfiles can save space and rebuild the Docker image by always downloading the latest version of software.

Docker Registry

The Registry is a server-side application containing and managing repositories of Docker images. We use Docker Hub (among other choices by different providers) to pull images from in this tutorial.

Namespaces

Namespaces allow for multiple “spaces” where identical names can coexist. For example, the same process ID (PID) may refer to completely different processes. This is like how we can distinguish Tom (PID) in Ottawa (namespace) from Tom in Toronto (namespace). Namespaces are enforced by the OS kernel. They are the fundamental building block of containers.

A few namespaces we will cover here: Mount (mnt), Network (net), User ID (uid) and Process ID (pid).

You can use the command lsns to see the namespaces the current process is in. A namespace ID is actually an inode.

Also, the command nsenter allows you to run programs with namespaces of another process. Use the -t option to specify the target PID. For instance, sudo nsenter -t 2772 -m ls / will list the root file system of the mount namespace of process 2772.

Tasks/Questions

The purpose of the following questions and tasks is to help you understand how containers work, which is an important artifact on top of and based on modern operating systems. The focus is on the OS building blocks – how it is made possible by the OS. Docker is just a typical and popular example to facilitate understanding.

You need to sudo for all the docker commands below.

Install Docker with sudo apt install docker.io if it’s not already installed.

Use docker images and docker ps (similar to the purpose of ps) to see what images you have and whether any containers are running. Run your first container with docker run hello-world. Now, check the images and docker ps again, what do you see and why?
Look at the hello-world image above and you may wonder how the tiny-sized image forms a full container. Find out where the files are located (hint: using the command docker inspect and looking for paths starting with /var/lib/docker). What is the size of the found file? Run the file directly. What do you think is the way the tiny-sized image forms the full container?
docker ps -a will show something different. Next, let’s try an image of a bit more complexity that will remain running. Try the centos (a Linux distribution) image: docker run -d centos sleep infinity (man docker-run for the meaning of the options).
Start a new shell: docker exec -it <CONTAINER ID> /bin/bash (you can find out the container ID with docker ps -a. Note that you can specify just the first few digits of the ID as long as it’s distinctive). Explore the file system therein.
Next, we will examine the namespaces.
Compare the mount namespaces (different views of the file system structure) between the centos container and the host, using ls -l /proc/self/ns/mnt. Are they the same and if not, what are they (format: mnt:[????])? In particular, take a look at the root mount point (“/”) inside and outside the container (mount | grep "on / "). You can always use two sessions (one in the container and one in the host) or tmux as you feel convenient.
With the lsns command, you can get an overview of all namespaces. Comparing the centos container and the host, which namespaces are the same (shared) and which are different?
Inside the container, use the command id to see what user you’re logged in as (uid, gid and group). Then ps -eo uid,gid,pid,command | grep sleep inside the container and outside the container respectively. Are their outputs the same?
Using the pid of “sleep” you have got outside the container above, find its pid namespace (sudo ls -l /proc/<pid>/ns/pid, format pid:[????]). Is it consistent with the lsns output inside the container?
From step 6, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (sudo nsenter -t <pid> -n ip addr).
Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).
Now, make your own Docker container. Read the current Dockerfile of hello-world. Change it in a way that it is no longer from scratch but from hello-world . Here we can avoid defining csimpleshell as the default command as it works with a terminal for interaction (then it will inherit CMD from hello-world). Build a new image called csimpleshell by doing the following:
mkdir anyname

copy your csimipleshell and the Dockerfile of hello-world into this directory

cd anyname

make necessary changes to the Dockerfile

docker build -t csimpleshell .
Run the new csimpleshell container (e.g., docker run -it csimpleshell /csimpleshell). Depending on how you compiled csimpleshell, check whether the container runs or if it does whether csimpleshell works as before. Try compiling csimpleshell statically (if it was dynamic) and repeat the steps.
Edit the Dockerfile again and make it from “centos” instead of “hello-world”. Remove any reference to hello. Rebuild and run it. How is csimpleshell now? Why is there such a difference?