COMP3000 Operating Systems W22: Tutorial 9 - Revision history

Lianyingzhao at 20:35, 29 March 2022

2022-03-29T20:35:28Z

← Older revision		Revision as of 20:35, 29 March 2022
Line 1:		Line 1:
	By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible.		By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible.

	Tutorials are graded based on participation and effort ('''so no need to try to have the “correct” answers — what matters is the process'''), but you should still turn in your work. Even if you have no idea about certain tasks or disagree about something, still make sure to document your confusions/opinions that reflect your thinking about that task. Submit your answers on Brightspace as a single text file named "<username>-comp3000-t8.txt" (where username is your MyCarletonOne username). The first four lines of this file should be "COMP 3000 Tutorial 8", your name, student number, and the date of submission.		Tutorials are graded based on participation and effort ('''so no need to try to have the “correct” answers — what matters is the process'''), but you should still turn in your work. Even if you have no idea about certain tasks or disagree about something, still make sure to document your confusions/opinions that reflect your thinking about that task. Submit your answers on Brightspace as a single text file named "<username>-comp3000-t9.txt" (where username is your MyCarletonOne username). The first four lines of this file should be "COMP 3000 Tutorial 9", your name, student number, and the date of submission.

	The deadline is usually four days after the tutorial date (see the actual due date and '''time''' on the submission entry). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.		The deadline is usually four days after the tutorial date (see the actual due date and '''time''' on the submission entry). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

Lianyingzhao at 19:39, 26 March 2022

2022-03-26T19:39:58Z

← Older revision		Revision as of 19:39, 26 March 2022
Line 50:		Line 50:
	# From step 6, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (<tt>sudo nsenter -t </tt><pid><tt> -n ip addr</tt>).		# From step 6, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (<tt>sudo nsenter -t </tt><pid><tt> -n ip addr</tt>).
	#: Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).		#: Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).
	# Now, make your own Docker container. Read the current [https://raw.githubusercontent.com/docker-library/hello-world/7ecae6978055d2fb6960e2a29d24a2af612e2716/amd64/hello-world/Dockerfile Dockerfile of hello-world]. Change it in a way that it is no longer from scratch but ''from <tt>hello-world </tt>''~~, and runs~~ <tt>/csimpleshell</tt> ~~after~~ <tt>/hello</tt>. Build a new image called <tt>csimpleshell</tt> by doing the following:		# Now, make your own Docker container. Read the current [https://raw.githubusercontent.com/docker-library/hello-world/7ecae6978055d2fb6960e2a29d24a2af612e2716/amd64/hello-world/Dockerfile Dockerfile of hello-world]. Change it in a way that it is no longer from scratch but ''from <tt>hello-world </tt>''. Here we can avoid defining <tt>csimpleshell</tt> as the default command as it works with a terminal for interaction (then it will inherit <tt>CMD</tt> from <tt>hello-world</tt>). Build a new image called <tt>csimpleshell</tt> by doing the following:
	#: <tt>mkdir anyname</tt>		#: <tt>mkdir anyname</tt>
	#: copy your <tt>csimipleshell</tt> and the Dockerfile of <tt>hello-world</tt> into this directory		#: copy your <tt>csimipleshell</tt> and the Dockerfile of <tt>hello-world</tt> into this directory
Line 56:		Line 56:
	#: make necessary changes to the Dockerfile		#: make necessary changes to the Dockerfile
	#: <tt>docker build -t csimpleshell .</tt>		#: <tt>docker build -t csimpleshell .</tt>
	# Run the new csimpleshell container (e.g., <tt>docker run -it csimpleshell</tt>). Depending on how you compiled csimpleshell, check whether the container runs or if it does whether csimpleshell works as before. Try compiling csimpleshell statically (if it was dynamic) and repeat the steps.		# Run the new csimpleshell container (e.g., <tt>docker run -it csimpleshell /csimpleshell</tt>). Depending on <tt>how you compiled csimpleshell</tt>, check whether the container runs or if it does whether csimpleshell works as before. Try compiling csimpleshell statically (if it was dynamic) and repeat the steps.
	# Edit the Dockerfile again and make it from “centos” instead of “hello-world”. Remove any reference to hello. Rebuild and run it. How is csimpleshell now? <u>Why</u> is there such a difference?		# Edit the Dockerfile again and make it from “centos” instead of “hello-world”. Remove any reference to hello. Rebuild and run it. How is csimpleshell now? <u>Why</u> is there such a difference?

Lianyingzhao at 16:01, 25 March 2022

2022-03-25T16:01:50Z

← Older revision		Revision as of 16:01, 25 March 2022
Line 50:		Line 50:
	# From step 6, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (<tt>sudo nsenter -t </tt><pid><tt> -n ip addr</tt>).		# From step 6, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (<tt>sudo nsenter -t </tt><pid><tt> -n ip addr</tt>).
	#: Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).		#: Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).
	# Now, make your own Docker container. Read the current Dockerfile of hello-world. Change it in a way that it is no longer from scratch but ''from <tt>hello-world </tt>'', and runs <tt>/csimpleshell</tt> after <tt>/hello</tt>. Build a new image called <tt>csimpleshell</tt> by doing the following:		# Now, make your own Docker container. Read the current [https://raw.githubusercontent.com/docker-library/hello-world/7ecae6978055d2fb6960e2a29d24a2af612e2716/amd64/hello-world/Dockerfile Dockerfile of hello-world]. Change it in a way that it is no longer from scratch but ''from <tt>hello-world </tt>'', and runs <tt>/csimpleshell</tt> after <tt>/hello</tt>. Build a new image called <tt>csimpleshell</tt> by doing the following:
	#: <tt>mkdir anyname</tt>		#: <tt>mkdir anyname</tt>
	#: copy your <tt>csimipleshell</tt> and the Dockerfile of <tt>hello-world</tt> into this directory		#: copy your <tt>csimipleshell</tt> and the Dockerfile of <tt>hello-world</tt> into this directory

Lianyingzhao at 16:00, 25 March 2022

2022-03-25T16:00:44Z

← Older revision		Revision as of 16:00, 25 March 2022
Line 48:		Line 48:
	# Inside the container, use the command <tt>id</tt> to see what user you’re logged in as (uid, gid and group). Then <tt>ps -eo uid,gid,pid,command \| grep sleep</tt> inside the container and outside the container respectively. Are their outputs the same?		# Inside the container, use the command <tt>id</tt> to see what user you’re logged in as (uid, gid and group). Then <tt>ps -eo uid,gid,pid,command \| grep sleep</tt> inside the container and outside the container respectively. Are their outputs the same?
	# Using the pid of “sleep” you have got outside the container above, find its pid namespace (<tt>sudo ls -l /proc/<pid>/ns/pid</tt>, format <tt>pid:[????]</tt>). Is it consistent with the <tt>lsns</tt> output inside the container?		# Using the pid of “sleep” you have got outside the container above, find its pid namespace (<tt>sudo ls -l /proc/<pid>/ns/pid</tt>, format <tt>pid:[????]</tt>). Is it consistent with the <tt>lsns</tt> output inside the container?
	# From step 7, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (<tt>sudo nsenter -t </tt><pid><tt> -n ip addr</tt>).		# From step 6, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (<tt>sudo nsenter -t </tt><pid><tt> -n ip addr</tt>).
	#: Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).		#: Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).
	# Now, make your own Docker container. Read the current Dockerfile of hello-world. Change it in a way that it is no longer from scratch but ''from <tt>hello-world </tt>'', and runs <tt>/csimpleshell</tt> after <tt>/hello</tt>. Build a new image called <tt>csimpleshell</tt> by doing the following:		# Now, make your own Docker container. Read the current Dockerfile of hello-world. Change it in a way that it is no longer from scratch but ''from <tt>hello-world </tt>'', and runs <tt>/csimpleshell</tt> after <tt>/hello</tt>. Build a new image called <tt>csimpleshell</tt> by doing the following:

Lianyingzhao: Created page with "By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible. Tutorials are graded based on participation and effort ('''so no need to try to have the “correct” answers — what matters is the process'''), but you should still turn in your work. Even if you have no..."

2022-03-25T15:49:21Z

Created page with "By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible. Tutorials are graded based on participation and effort ('''so no need to try to have the “correct” answers — what matters is the process'''), but you should still turn in your work. Even if you have no..."

New page

By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible.

Tutorials are graded based on participation and effort ('''so no need to try to have the “correct” answers — what matters is the process'''), but you should still turn in your work. Even if you have no idea about certain tasks or disagree about something, still make sure to document your confusions/opinions that reflect your thinking about that task. Submit your answers on Brightspace as a single text file named "<username>-comp3000-t8.txt" (where username is your MyCarletonOne username). The first four lines of this file should be "COMP 3000 Tutorial 8", your name, student number, and the date of submission.

The deadline is usually four days after the tutorial date (see the actual due date and '''time''' on the submission entry). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA online (by responding to the poll in the Teams channel tutorials-public or the private channel). Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

==Background==

===Docker containers===
Docker is one of the OS-level virtualization technologies that allows the creation of isolated virtual execution environments, called containers, sharing the same OS kernel and certain runtime services/libraries with other containers. The workload running inside a container will feel as if it had its own everything, OS kernel, root file system, etc. This is possible thanks to the way Docker containers are designed and implemented. The Linux kernel does not directly support the construction of containers (the userspace libraries do) but its various building blocks play an important role, among which we will cover namespaces in this tutorial.

===Docker images===
A Docker image is a file containing a root file system as well as configuration parameters. It is very similar to other disk image formats in this sense. One of the main differences is that it employs a layered file system to maximize storage efficiency and allow flexible reuse of existing files.

===Dockerfile===
A Dockerfile is a text file and serves as instructions to build a Docker image. Unlike storing and distributing binary data directly in images, using Dockerfiles can save space and rebuild the Docker image by always downloading the latest version of software.

===Docker Registry===
The Registry is a server-side application containing and managing repositories of Docker images. We use Docker Hub (among other choices by different providers) to pull images from in this tutorial.

===Namespaces===
Namespaces allow for multiple “spaces” where identical names can coexist. For example, the same process ID (PID) may refer to completely different processes. This is like how we can distinguish Tom (PID) in Ottawa (namespace) from Tom in Toronto (namespace). Namespaces are enforced by the OS kernel. They are the fundamental building block of containers.

A few namespaces we will cover here: Mount (<tt>mnt</tt>), Network (<tt>net</tt>), User ID (<tt>uid</tt>) and Process ID (<tt>pid</tt>).

You can use the command <tt>lsns</tt> to see the namespaces the current process is in. A namespace ID is actually an '''inode'''.

Also, the command <tt>nsenter</tt> allows you to run programs with namespaces of another process. Use the <tt>-t</tt> option to specify the target PID. For instance, <tt>sudo nsenter -t 2772 -m ls /</tt> will list the root file system of the mount namespace of process 2772.

==Tasks/Questions==
The purpose of the following questions and tasks is to help you understand how containers work, which is an important artifact on top of and based on modern operating systems. The focus is on the OS building blocks – how it is made possible by the OS. Docker is just a typical and popular example to facilitate understanding.

'''You need to <tt>sudo</tt> for all the <tt>docker</tt> commands below'''.

Install Docker with <tt>sudo apt install docker.io</tt> if it’s not already installed.

# Use <tt>docker images</tt> and <tt>docker ps</tt> (similar to the purpose of <tt>ps</tt>) to see what images you have and whether any containers are running. Run your first container with <tt>docker run hello-world</tt>. Now, check the images and <tt>docker ps</tt> again, what do you see and why?
# Look at the hello-world image above and you may wonder how the tiny-sized image forms a full container. Find out where the files are located (hint: using the command <tt>docker inspect</tt> and looking for paths starting with <tt>/var/lib/docker</tt>). What is the size of the found file? Run the file directly. What do you think is the way the tiny-sized image forms the full container?
# <tt>docker ps -a</tt> will show something different. Next, let’s try an image of a bit more complexity that will remain running. Try the centos (a Linux distribution) image: <tt>docker run -d centos sleep infinity</tt> (<tt>man docker-run</tt> for the meaning of the options).
# Start a new shell: <tt>docker exec -it</tt> <CONTAINER ID> <tt>/bin/bash</tt> (you can find out the container ID with <tt>docker ps -a</tt>. Note that you can specify just the first few digits of the ID as long as it’s distinctive). Explore the file system therein.
#; Next, we will examine the namespaces.
# Compare the mount namespaces (different views of the file system structure) between the centos container and the host, using <tt>ls -l /proc/self/ns/mnt</tt>. Are they the same and if not, what are they (format: <tt>mnt:[????]</tt>)? In particular, take a look at the root mount point (“<tt>/</tt>”) inside and outside the container (<tt>mount | grep "on / "</tt>). You can always use two sessions (one in the container and one in the host) or <tt>tmux</tt> as you feel convenient.
# With the <tt>lsns</tt> command, you can get an overview of all namespaces. Comparing the centos container and the host, which namespaces are the same (shared) and which are different?
# Inside the container, use the command <tt>id</tt> to see what user you’re logged in as (uid, gid and group). Then <tt>ps -eo uid,gid,pid,command | grep sleep</tt> inside the container and outside the container respectively. Are their outputs the same?
# Using the pid of “sleep” you have got outside the container above, find its pid namespace (<tt>sudo ls -l /proc/<pid>/ns/pid</tt>, format <tt>pid:[????]</tt>). Is it consistent with the <tt>lsns</tt> output inside the container?
# From step 7, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (<tt>sudo nsenter -t </tt><pid><tt> -n ip addr</tt>).
#: Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).
# Now, make your own Docker container. Read the current Dockerfile of hello-world. Change it in a way that it is no longer from scratch but ''from <tt>hello-world </tt>'', and runs <tt>/csimpleshell</tt> after <tt>/hello</tt>. Build a new image called <tt>csimpleshell</tt> by doing the following:
#: <tt>mkdir anyname</tt>
#: copy your <tt>csimipleshell</tt> and the Dockerfile of <tt>hello-world</tt> into this directory
#: <tt>cd anyname</tt>
#: make necessary changes to the Dockerfile
#: <tt>docker build -t csimpleshell .</tt>
# Run the new csimpleshell container (e.g., <tt>docker run -it csimpleshell</tt>). Depending on how you compiled csimpleshell, check whether the container runs or if it does whether csimpleshell works as before. Try compiling csimpleshell statically (if it was dynamic) and repeat the steps.
# Edit the Dockerfile again and make it from “centos” instead of “hello-world”. Remove any reference to hello. Rebuild and run it. How is csimpleshell now? Why is there such a difference?