Soma-notes - User contributions [en]

COMP3000 Operating Systems 2023F Tutorials

2023-11-30T17:26:42Z

Abdou: /* Tutorial Sections */

This page contains Tutorial instructions for COMP3000 A, Fall 2023.

The main page for the course can be found [https://people.scs.carleton.ca/~abdou/COMP3000_F2023.html here].

==Information about the Tutorials==
Tutorials are in person. You need to be physically in the room to finish up the tutorial instructions on your laptop and get checked-out by one of the two present TAs. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

Please stick to the tutorial session you are registered in. This avoids overcrowding.

Make well use of these tutorials to help you better understand the course material. The plan is to have 9 tutorial sessions this term, each would be worth 2.22% of your total grade. We might revisit this as the term progresses.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

== Getting Access to a Linux Machine ==

For the tutorials, you need to get access to a Linux machine. It is recommended to use an SCS Openstack instance (see instructions below). Alternatively, if you prefer to work directly on your laptop and avoid Openstack, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova this VirtualBox image] (courtesy of Prof. Zhao from Winter 2023). You will need access to a Linux system for the entire semester, ideally the same one.

=== Setting up an Openstack VM instance for the first time ===
We will use Openstack for the tutorials of this course. Follow these instructions for setting up an Openstack VM instance: [[COMP3000 Operating Systems F23: Openstack Instructions | instructions here]]. You only need to do this once for the whole term. Please use the same instance for both: the tutorials and the assignments. ''No need to create two instances''.

=== Connecting to the VM ===
You can find instructions with various ways to connect to your Openstack instance [[COMP3000 Operating Systems F23: Connecting to SCS Openstack | here]].

=== Downloading files from the VM ===
To download files from your Openstack VM instance, you can find instructions [[COMP3000 Operating Systems F23: Downloading files from your Openstack VM | here]].

=== Backing up the VM ===

To backup your VM, the image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account's home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

Reminder: ''do '''not''' take snapshots of your instance!'' Note that you cannot take snapshots of your VM anyway (it will keep trying and never succeed, and you'll create work for the tech staff who will have to undo what you did), so please don't try.

==Tutorial Sections==

* COMP 3000 A1: Tue: 08:35 - 09:55 at Loeb 720 (LA720). [Ethan and Sneha (T1-T9)]
* COMP 3000 A2: Fri: 11:35 - 12:55 at Loeb 720 (LA720). [Ethan and Ali (T1-T3), Ethan and Nareen (T4-T6), Ali and Nareen (T7-T9)]
* COMP 3000 A3: Thu: 08:35 - 09:55 at St. Patrick's 303 (SP303). [Ali and Nareen (T1-T9)]

{| class="wikitable"
|+ Tutorial Schedules
|-
! Tutorial !! Section !! Date !! Time !! Room !! TAs
|-
| rowspan="3" | [[COMP3000 Operating Systems 2023F Tutorials: Tutorial 1 | Tutorial 1]] || A1 || Sep 26 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Sep 28 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Sep 29 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 2 | Tutorial 2]] || A1 || Oct 3 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 5 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 6 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 3 | Tutorial 3]] || A1 || Oct 10 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 12 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 13 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 4 | Tutorial 4]] || A1 || Oct 31 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 2 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 3 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 5 | Tutorial 5]] || A1 || Nov 7 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 9 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 10 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 6 | Tutorial 6]] || A1 || Nov 14 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 16 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 17 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 7 | Tutorial 7]] || A1 || Nov 21 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 23 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 24 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 8 | Tutorial 8]] || A1 || Nov 28 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 30 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 1 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 9 | Tutorial 9]] || A1 || Dec 5 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Dec 7 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 8 || 1:05 pm-2:25 pm || MC2000 || Ali and Nareen
|-
|}

COMP3000 Operating Systems F23: Tutorial 9

2023-10-25T02:11:57Z

Abdou:

By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible.

==Background==

===Docker containers===
Docker is one of the OS-level virtualization technologies that allows the creation of isolated virtual execution environments, called containers, sharing the same OS kernel and certain runtime services/libraries with other containers. The workload running inside a container will feel as if it had its own everything, OS kernel, root file system, etc. This is possible thanks to the way Docker containers are designed and implemented. The Linux kernel does not directly support the construction of containers (the userspace libraries do) but its various building blocks play an important role, among which we will cover namespaces in this tutorial.

===Docker images===
A Docker image is a file containing a root file system as well as configuration parameters. It is very similar to other disk image formats in this sense. One of the main differences is that it employs a layered file system to maximize storage efficiency and allow flexible reuse of existing files.

===Dockerfile===
A Dockerfile is a text file and serves as instructions to build a Docker image. Unlike storing and distributing binary data directly in images, using Dockerfiles can save space and rebuild the Docker image by always downloading the latest version of software.

===Docker Registry===
The Registry is a server-side application containing and managing repositories of Docker images. We use Docker Hub (among other choices by different providers) to pull images from in this tutorial.

===Namespaces===
Namespaces allow for multiple “spaces” where identical names can coexist. For example, the same process ID (PID) may refer to completely different processes. This is like how we can distinguish Tom (PID) in Ottawa (namespace) from Tom in Toronto (namespace). Namespaces are enforced by the OS kernel. They are the fundamental building block of containers.

A few namespaces we will cover here: Mount (<code>mnt</code>), Network (<code>net</code>), User ID (<code>uid</code>) and Process ID (<code>pid</code>).

You can use the command <code>lsns</code> to see the namespaces the current process is in. A namespace ID is actually an '''inode'''.

Also, the command <code>nsenter</code> allows you to run programs with namespaces of another process. Use the <code>-t</code> option to specify the target PID. For instance, <code>sudo nsenter -t 2772 -m ls /</code> will list the root file system of the mount namespace of process 2772.

==Tasks/Questions==
The purpose of the following questions and tasks is to help you understand how containers work, which is an important artifact on top of and based on modern operating systems. The focus is on the OS building blocks – how it is made possible by the OS. Docker is just a typical and popular example to facilitate understanding.

'''You need to <code>sudo</code> for all the <code>docker</code> commands below'''.

Install Docker with <code>sudo apt install docker.io</code> if it’s not already installed.

# Use <code>docker images</code> and <code>docker ps</code> (similar to the purpose of <code>ps</code>) to see what images you have and whether any containers are running. Run your first container with <code>docker run hello-world</code>. Now, check the images and <code>docker ps</code> again, what do you see and why?
# Look at the hello-world image above and you may wonder how the tiny-sized image forms a full container. Find out where the files are located (hint: using the command <code>docker inspect</code> and looking for paths starting with <code>/var/lib/docker</code>). What is the size of the found file? Run the file directly. What do you think is the way the tiny-sized image forms the full container?
# <code>docker ps -a</code> will show something different. Next, let’s try an image of a bit more complexity that will remain running. Try the centos (a Linux distribution) image: <code>docker run -d centos sleep infinity</code> (<code>man docker-run</code> for the meaning of the options).
# Start a new shell: <code>docker exec -it</code> <CONTAINER ID> <code>/bin/bash</code> (you can find out the container ID with <code>docker ps -a</code>. Note that you can specify just the first few digits of the ID as long as it’s distinctive). Explore the file system therein.
#; Next, we will examine the namespaces.
# Compare the mount namespaces (different views of the file system structure) between the centos container and the host, using <code>ls -l /proc/self/ns/mnt</code>. Are they the same and if not, what are they (format: <code>mnt:[????]</code>)? In particular, take a look at the root mount point (“<code>/</code>”) inside and outside the container (<code>mount | grep "on / "</code>). You can always use two sessions (one in the container and one in the host) or <code>tmux</code> as you feel convenient.
# With the <code>lsns</code> command, you can get an overview of all namespaces. Comparing the centos container and the host, which namespaces are the same (shared) and which are different?
# Inside the container, use the command <code>id</code> to see what user you’re logged in as (uid, gid and group). Then <code>ps -eo uid,gid,pid,command | grep sleep</code> inside the container and outside the container respectively. Are their outputs the same?
# Using the pid of “sleep” you have got outside the container above, find its pid namespace (<code>sudo ls -l /proc/<pid>/ns/pid</code>, format <code>pid:[????]</code>). Is it consistent with the <code>lsns</code> output inside the container?
# From step 6, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (<code>sudo nsenter -t </code><pid><code> -n ip addr</code>).
#: Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).
# Now, make your own Docker container. Read the current [https://raw.githubusercontent.com/docker-library/hello-world/7ecae6978055d2fb6960e2a29d24a2af612e2716/amd64/hello-world/Dockerfile Dockerfile of hello-world]. Change it in a way that it is no longer from scratch but ''from <code>hello-world </code>''. Here we can avoid defining <code>csimpleshell</code> as the default command as it works with a terminal for interaction (then it will inherit <code>CMD</code> from <code>hello-world</code>). Build a new image called <code>csimpleshell</code> by doing the following:
#: <code>mkdir anyname</code>
#: copy your <code>csimipleshell</code> and the Dockerfile of <code>hello-world</code> into this directory
#: <code>cd anyname</code>
#: make necessary changes to the Dockerfile
#: <code>docker build -t csimpleshell .</code>
# Run the new csimpleshell container (e.g., <code>docker run -it csimpleshell /csimpleshell</code>). Depending on <code>how you compiled csimpleshell</code>, check whether the container runs or if it does whether csimpleshell works as before. Try compiling csimpleshell statically (if it was dynamic) and repeat the steps.
# Edit the Dockerfile again and make it from “centos” instead of “hello-world”. Remove any reference to hello. Rebuild and run it. How is csimpleshell now? Why is there such a difference?

COMP3000 Operating Systems F23: Tutorial 9

2023-10-25T02:09:06Z

Abdou: /* General Instructions (same for all tutorials) */

By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible.

==Background==

===Docker containers===
Docker is one of the OS-level virtualization technologies that allows the creation of isolated virtual execution environments, called containers, sharing the same OS kernel and certain runtime services/libraries with other containers. The workload running inside a container will feel as if it had its own everything, OS kernel, root file system, etc. This is possible thanks to the way Docker containers are designed and implemented. The Linux kernel does not directly support the construction of containers (the userspace libraries do) but its various building blocks play an important role, among which we will cover namespaces in this tutorial.

===Docker images===
A Docker image is a file containing a root file system as well as configuration parameters. It is very similar to other disk image formats in this sense. One of the main differences is that it employs a layered file system to maximize storage efficiency and allow flexible reuse of existing files.

===Dockerfile===
A Dockerfile is a text file and serves as instructions to build a Docker image. Unlike storing and distributing binary data directly in images, using Dockerfiles can save space and rebuild the Docker image by always downloading the latest version of software.

===Docker Registry===
The Registry is a server-side application containing and managing repositories of Docker images. We use Docker Hub (among other choices by different providers) to pull images from in this tutorial.

===Namespaces===
Namespaces allow for multiple “spaces” where identical names can coexist. For example, the same process ID (PID) may refer to completely different processes. This is like how we can distinguish Tom (PID) in Ottawa (namespace) from Tom in Toronto (namespace). Namespaces are enforced by the OS kernel. They are the fundamental building block of containers.

A few namespaces we will cover here: Mount (<tt>mnt</tt>), Network (<tt>net</tt>), User ID (<tt>uid</tt>) and Process ID (<tt>pid</tt>).

You can use the command <tt>lsns</tt> to see the namespaces the current process is in. A namespace ID is actually an '''inode'''.

Also, the command <tt>nsenter</tt> allows you to run programs with namespaces of another process. Use the <tt>-t</tt> option to specify the target PID. For instance, <tt>sudo nsenter -t 2772 -m ls /</tt> will list the root file system of the mount namespace of process 2772.

==Tasks/Questions==
The purpose of the following questions and tasks is to help you understand how containers work, which is an important artifact on top of and based on modern operating systems. The focus is on the OS building blocks – how it is made possible by the OS. Docker is just a typical and popular example to facilitate understanding.

'''You need to <tt>sudo</tt> for all the <tt>docker</tt> commands below'''.

Install Docker with <tt>sudo apt install docker.io</tt> if it’s not already installed.

# Use <tt>docker images</tt> and <tt>docker ps</tt> (similar to the purpose of <tt>ps</tt>) to see what images you have and whether any containers are running. Run your first container with <tt>docker run hello-world</tt>. Now, check the images and <tt>docker ps</tt> again, what do you see and why?
# Look at the hello-world image above and you may wonder how the tiny-sized image forms a full container. Find out where the files are located (hint: using the command <tt>docker inspect</tt> and looking for paths starting with <tt>/var/lib/docker</tt>). What is the size of the found file? Run the file directly. What do you think is the way the tiny-sized image forms the full container?
# <tt>docker ps -a</tt> will show something different. Next, let’s try an image of a bit more complexity that will remain running. Try the centos (a Linux distribution) image: <tt>docker run -d centos sleep infinity</tt> (<tt>man docker-run</tt> for the meaning of the options).
# Start a new shell: <tt>docker exec -it</tt> <CONTAINER ID> <tt>/bin/bash</tt> (you can find out the container ID with <tt>docker ps -a</tt>. Note that you can specify just the first few digits of the ID as long as it’s distinctive). Explore the file system therein.
#; Next, we will examine the namespaces.
# Compare the mount namespaces (different views of the file system structure) between the centos container and the host, using <tt>ls -l /proc/self/ns/mnt</tt>. Are they the same and if not, what are they (format: <tt>mnt:[????]</tt>)? In particular, take a look at the root mount point (“<tt>/</tt>”) inside and outside the container (<tt>mount | grep "on / "</tt>). You can always use two sessions (one in the container and one in the host) or <tt>tmux</tt> as you feel convenient.
# With the <tt>lsns</tt> command, you can get an overview of all namespaces. Comparing the centos container and the host, which namespaces are the same (shared) and which are different?
# Inside the container, use the command <tt>id</tt> to see what user you’re logged in as (uid, gid and group). Then <tt>ps -eo uid,gid,pid,command | grep sleep</tt> inside the container and outside the container respectively. Are their outputs the same?
# Using the pid of “sleep” you have got outside the container above, find its pid namespace (<tt>sudo ls -l /proc/<pid>/ns/pid</tt>, format <tt>pid:[????]</tt>). Is it consistent with the <tt>lsns</tt> output inside the container?
# From step 6, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (<tt>sudo nsenter -t </tt><pid><tt> -n ip addr</tt>).
#: Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).
# Now, make your own Docker container. Read the current [https://raw.githubusercontent.com/docker-library/hello-world/7ecae6978055d2fb6960e2a29d24a2af612e2716/amd64/hello-world/Dockerfile Dockerfile of hello-world]. Change it in a way that it is no longer from scratch but ''from <tt>hello-world </tt>''. Here we can avoid defining <tt>csimpleshell</tt> as the default command as it works with a terminal for interaction (then it will inherit <tt>CMD</tt> from <tt>hello-world</tt>). Build a new image called <tt>csimpleshell</tt> by doing the following:
#: <tt>mkdir anyname</tt>
#: copy your <tt>csimipleshell</tt> and the Dockerfile of <tt>hello-world</tt> into this directory
#: <tt>cd anyname</tt>
#: make necessary changes to the Dockerfile
#: <tt>docker build -t csimpleshell .</tt>
# Run the new csimpleshell container (e.g., <tt>docker run -it csimpleshell /csimpleshell</tt>). Depending on <tt>how you compiled csimpleshell</tt>, check whether the container runs or if it does whether csimpleshell works as before. Try compiling csimpleshell statically (if it was dynamic) and repeat the steps.
# Edit the Dockerfile again and make it from “centos” instead of “hello-world”. Remove any reference to hello. Rebuild and run it. How is csimpleshell now? Why is there such a difference?

COMP3000 Operating Systems F23: Tutorial 9

2023-10-25T02:09:00Z

Abdou: Created page with "By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible. ==General Instructions (same for all tutorials)== <div class="mw-collapsible mw-collapsed"> Click on Expand to view. <div class="mw-collapsible-content"> Tutorials are graded based on participation and effo..."

By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''), but you should still turn in your work. Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA (in person or by responding to a poll if online). Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Background==

===Docker containers===
Docker is one of the OS-level virtualization technologies that allows the creation of isolated virtual execution environments, called containers, sharing the same OS kernel and certain runtime services/libraries with other containers. The workload running inside a container will feel as if it had its own everything, OS kernel, root file system, etc. This is possible thanks to the way Docker containers are designed and implemented. The Linux kernel does not directly support the construction of containers (the userspace libraries do) but its various building blocks play an important role, among which we will cover namespaces in this tutorial.

===Docker images===
A Docker image is a file containing a root file system as well as configuration parameters. It is very similar to other disk image formats in this sense. One of the main differences is that it employs a layered file system to maximize storage efficiency and allow flexible reuse of existing files.

===Dockerfile===
A Dockerfile is a text file and serves as instructions to build a Docker image. Unlike storing and distributing binary data directly in images, using Dockerfiles can save space and rebuild the Docker image by always downloading the latest version of software.

===Docker Registry===
The Registry is a server-side application containing and managing repositories of Docker images. We use Docker Hub (among other choices by different providers) to pull images from in this tutorial.

===Namespaces===
Namespaces allow for multiple “spaces” where identical names can coexist. For example, the same process ID (PID) may refer to completely different processes. This is like how we can distinguish Tom (PID) in Ottawa (namespace) from Tom in Toronto (namespace). Namespaces are enforced by the OS kernel. They are the fundamental building block of containers.

A few namespaces we will cover here: Mount (<tt>mnt</tt>), Network (<tt>net</tt>), User ID (<tt>uid</tt>) and Process ID (<tt>pid</tt>).

You can use the command <tt>lsns</tt> to see the namespaces the current process is in. A namespace ID is actually an '''inode'''.

Also, the command <tt>nsenter</tt> allows you to run programs with namespaces of another process. Use the <tt>-t</tt> option to specify the target PID. For instance, <tt>sudo nsenter -t 2772 -m ls /</tt> will list the root file system of the mount namespace of process 2772.

==Tasks/Questions==
The purpose of the following questions and tasks is to help you understand how containers work, which is an important artifact on top of and based on modern operating systems. The focus is on the OS building blocks – how it is made possible by the OS. Docker is just a typical and popular example to facilitate understanding.

'''You need to <tt>sudo</tt> for all the <tt>docker</tt> commands below'''.

Install Docker with <tt>sudo apt install docker.io</tt> if it’s not already installed.

# Use <tt>docker images</tt> and <tt>docker ps</tt> (similar to the purpose of <tt>ps</tt>) to see what images you have and whether any containers are running. Run your first container with <tt>docker run hello-world</tt>. Now, check the images and <tt>docker ps</tt> again, what do you see and why?
# Look at the hello-world image above and you may wonder how the tiny-sized image forms a full container. Find out where the files are located (hint: using the command <tt>docker inspect</tt> and looking for paths starting with <tt>/var/lib/docker</tt>). What is the size of the found file? Run the file directly. What do you think is the way the tiny-sized image forms the full container?
# <tt>docker ps -a</tt> will show something different. Next, let’s try an image of a bit more complexity that will remain running. Try the centos (a Linux distribution) image: <tt>docker run -d centos sleep infinity</tt> (<tt>man docker-run</tt> for the meaning of the options).
# Start a new shell: <tt>docker exec -it</tt> <CONTAINER ID> <tt>/bin/bash</tt> (you can find out the container ID with <tt>docker ps -a</tt>. Note that you can specify just the first few digits of the ID as long as it’s distinctive). Explore the file system therein.
#; Next, we will examine the namespaces.
# Compare the mount namespaces (different views of the file system structure) between the centos container and the host, using <tt>ls -l /proc/self/ns/mnt</tt>. Are they the same and if not, what are they (format: <tt>mnt:[????]</tt>)? In particular, take a look at the root mount point (“<tt>/</tt>”) inside and outside the container (<tt>mount | grep "on / "</tt>). You can always use two sessions (one in the container and one in the host) or <tt>tmux</tt> as you feel convenient.
# With the <tt>lsns</tt> command, you can get an overview of all namespaces. Comparing the centos container and the host, which namespaces are the same (shared) and which are different?
# Inside the container, use the command <tt>id</tt> to see what user you’re logged in as (uid, gid and group). Then <tt>ps -eo uid,gid,pid,command | grep sleep</tt> inside the container and outside the container respectively. Are their outputs the same?
# Using the pid of “sleep” you have got outside the container above, find its pid namespace (<tt>sudo ls -l /proc/<pid>/ns/pid</tt>, format <tt>pid:[????]</tt>). Is it consistent with the <tt>lsns</tt> output inside the container?
# From step 6, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (<tt>sudo nsenter -t </tt><pid><tt> -n ip addr</tt>).
#: Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).
# Now, make your own Docker container. Read the current [https://raw.githubusercontent.com/docker-library/hello-world/7ecae6978055d2fb6960e2a29d24a2af612e2716/amd64/hello-world/Dockerfile Dockerfile of hello-world]. Change it in a way that it is no longer from scratch but ''from <tt>hello-world </tt>''. Here we can avoid defining <tt>csimpleshell</tt> as the default command as it works with a terminal for interaction (then it will inherit <tt>CMD</tt> from <tt>hello-world</tt>). Build a new image called <tt>csimpleshell</tt> by doing the following:
#: <tt>mkdir anyname</tt>
#: copy your <tt>csimipleshell</tt> and the Dockerfile of <tt>hello-world</tt> into this directory
#: <tt>cd anyname</tt>
#: make necessary changes to the Dockerfile
#: <tt>docker build -t csimpleshell .</tt>
# Run the new csimpleshell container (e.g., <tt>docker run -it csimpleshell /csimpleshell</tt>). Depending on <tt>how you compiled csimpleshell</tt>, check whether the container runs or if it does whether csimpleshell works as before. Try compiling csimpleshell statically (if it was dynamic) and repeat the steps.
# Edit the Dockerfile again and make it from “centos” instead of “hello-world”. Remove any reference to hello. Rebuild and run it. How is csimpleshell now? Why is there such a difference?

COMP3000 Operating Systems 2023F Tutorials

2023-10-25T02:08:53Z

Abdou: /* Tutorial Sections */

This page contains Tutorial instructions for COMP3000 A, Fall 2023.

The main page for the course can be found [https://people.scs.carleton.ca/~abdou/COMP3000_F2023.html here].

==Information about the Tutorials==
Tutorials are in person. You need to be physically in the room to finish up the tutorial instructions on your laptop and get checked-out by one of the two present TAs. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

Please stick to the tutorial session you are registered in. This avoids overcrowding.

Make well use of these tutorials to help you better understand the course material. The plan is to have 9 tutorial sessions this term, each would be worth 2.22% of your total grade. We might revisit this as the term progresses.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

== Getting Access to a Linux Machine ==

For the tutorials, you need to get access to a Linux machine. It is recommended to use an SCS Openstack instance (see instructions below). Alternatively, if you prefer to work directly on your laptop and avoid Openstack, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova this VirtualBox image] (courtesy of Prof. Zhao from Winter 2023). You will need access to a Linux system for the entire semester, ideally the same one.

=== Setting up an Openstack VM instance for the first time ===
We will use Openstack for the tutorials of this course. Follow these instructions for setting up an Openstack VM instance: [[COMP3000 Operating Systems F23: Openstack Instructions | instructions here]]. You only need to do this once for the whole term. Please use the same instance for both: the tutorials and the assignments. ''No need to create two instances''.

=== Connecting to the VM ===
You can find instructions with various ways to connect to your Openstack instance [[COMP3000 Operating Systems F23: Connecting to SCS Openstack | here]].

=== Downloading files from the VM ===
To download files from your Openstack VM instance, you can find instructions [[COMP3000 Operating Systems F23: Downloading files from your Openstack VM | here]].

=== Backing up the VM ===

To backup your VM, the image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account's home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

Reminder: ''do '''not''' take snapshots of your instance!'' Note that you cannot take snapshots of your VM anyway (it will keep trying and never succeed, and you'll create work for the tech staff who will have to undo what you did), so please don't try.

==Tutorial Sections==

* COMP 3000 A1: Tue: 08:35 - 09:55 at Loeb 720 (LA720). [Ethan and Sneha (T1-T9)]
* COMP 3000 A2: Fri: 11:35 - 12:55 at Loeb 720 (LA720). [Ethan and Ali (T1-T3), Ethan and Nareen (T4-T6), Ali and Nareen (T7-T9)]
* COMP 3000 A3: Thu: 08:35 - 09:55 at St. Patrick's 303 (SP303). [Ali and Nareen (T1-T9)]

{| class="wikitable"
|+ Tutorial Schedules
|-
! Tutorial !! Section !! Date !! Time !! Room !! TAs
|-
| rowspan="3" | [[COMP3000 Operating Systems 2023F Tutorials: Tutorial 1 | Tutorial 1]] || A1 || Sep 26 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Sep 28 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Sep 29 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 2 | Tutorial 2]] || A1 || Oct 3 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 5 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 6 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 3 | Tutorial 3]] || A1 || Oct 10 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 12 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 13 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 4 | Tutorial 4]] || A1 || Oct 31 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 2 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 3 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 5 | Tutorial 5]] || A1 || Nov 7 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 9 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 10 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 6 | Tutorial 6]] || A1 || Nov 14 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 16 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 17 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 7 | Tutorial 7]] || A1 || Nov 21 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 23 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 24 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 8 | Tutorial 8]] || A1 || Nov 28 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 30 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 1 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 9 | Tutorial 9]] || A1 || Dec 5 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Dec 7 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 8 || 11:35 am-12:55 pm || MC2000 || Ali and Nareen
|-
|}

COMP3000 Operating Systems 2023F Tutorials

2023-10-25T02:08:30Z

Abdou: /* Tutorial Sections */

This page contains Tutorial instructions for COMP3000 A, Fall 2023.

The main page for the course can be found [https://people.scs.carleton.ca/~abdou/COMP3000_F2023.html here].

==Information about the Tutorials==
Tutorials are in person. You need to be physically in the room to finish up the tutorial instructions on your laptop and get checked-out by one of the two present TAs. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

Please stick to the tutorial session you are registered in. This avoids overcrowding.

Make well use of these tutorials to help you better understand the course material. The plan is to have 9 tutorial sessions this term, each would be worth 2.22% of your total grade. We might revisit this as the term progresses.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

== Getting Access to a Linux Machine ==

For the tutorials, you need to get access to a Linux machine. It is recommended to use an SCS Openstack instance (see instructions below). Alternatively, if you prefer to work directly on your laptop and avoid Openstack, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova this VirtualBox image] (courtesy of Prof. Zhao from Winter 2023). You will need access to a Linux system for the entire semester, ideally the same one.

=== Setting up an Openstack VM instance for the first time ===
We will use Openstack for the tutorials of this course. Follow these instructions for setting up an Openstack VM instance: [[COMP3000 Operating Systems F23: Openstack Instructions | instructions here]]. You only need to do this once for the whole term. Please use the same instance for both: the tutorials and the assignments. ''No need to create two instances''.

=== Connecting to the VM ===
You can find instructions with various ways to connect to your Openstack instance [[COMP3000 Operating Systems F23: Connecting to SCS Openstack | here]].

=== Downloading files from the VM ===
To download files from your Openstack VM instance, you can find instructions [[COMP3000 Operating Systems F23: Downloading files from your Openstack VM | here]].

=== Backing up the VM ===

To backup your VM, the image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account's home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

Reminder: ''do '''not''' take snapshots of your instance!'' Note that you cannot take snapshots of your VM anyway (it will keep trying and never succeed, and you'll create work for the tech staff who will have to undo what you did), so please don't try.

==Tutorial Sections==

* COMP 3000 A1: Tue: 08:35 - 09:55 at Loeb 720 (LA720). [Ethan and Sneha (T1-T9)]
* COMP 3000 A2: Fri: 11:35 - 12:55 at Loeb 720 (LA720). [Ethan and Ali (T1-T3), Ethan and Nareen (T4-T6), Ali and Nareen (T7-T9)]
* COMP 3000 A3: Thu: 08:35 - 09:55 at St. Patrick's 303 (SP303). [Ali and Nareen (T1-T9)]

{| class="wikitable"
|+ Tutorial Schedules
|-
! Tutorial !! Section !! Date !! Time !! Room !! TAs
|-
| rowspan="3" | [[COMP3000 Operating Systems 2023F Tutorials: Tutorial 1 | Tutorial 1]] || A1 || Sep 26 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Sep 28 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Sep 29 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 2 | Tutorial 2]] || A1 || Oct 3 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 5 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 6 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 3 | Tutorial 3]] || A1 || Oct 10 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 12 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 13 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 4 | Tutorial 4]] || A1 || Oct 31 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 2 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 3 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 5 | Tutorial 5]] || A1 || Nov 7 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 9 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 10 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 6 | Tutorial 6]] || A1 || Nov 14 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 16 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 17 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 7 | Tutorial 7]] || A1 || Nov 21 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 23 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 24 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 8 | Tutorial 8]] || A1 || Nov 28 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 30 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 1 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems W23: Tutorial 9 | Tutorial 9]] || A1 || Dec 5 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Dec 7 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 8 || 11:35 am-12:55 pm || MC2000 || Ali and Nareen
|-
|}

COMP3000 Operating Systems F23: Tutorial 8

2023-10-25T02:07:37Z

Abdou:

In this tutorial, you’ll be learning about how virtual addresses are mapped to physical addresses (the address translation) and continue to use kernel modules to extract information that only the kernel has access to. In particular, the kernel module performs a 5-level page table walk to find out the physical address corresponding to a userspace virtual address. In addition to what was discussed in the class, You can also read [https://en.wikipedia.org/wiki/Intel_5-level_paging more about 5-level paging], if interested.

==Introduction==
WARNING: The commands and programs in this tutorial are potentially extremely dangerous and may result in crashes or loss of data. Additionally, questions may not work as expected on a machine other than the course VM. For that reason, you are strongly encouraged to do this tutorial on the provided OpenStack virtual machine.

To get started, we will first examine the source code for <code>3000physicalview.c</code> and <code>3000memview2.c</code>, both of which are in [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut8/3000physicalview.tar.gz 3000physicalview.tar.gz].

==Tasks part A: Getting Started==
Compile <code>3000physicalview</code> and <code>3000memview2</code> using the provided <code>Makefile</code> (i.e., by running <code>make</code>).

Insert <code>3000physicalview</code> by running <code>make insert</code>. Confirm that the module is inserted using <code>lsmod</code>.
# Examine the call to <code>copy_from_user()</code> and <code>copy_to_user()</code> on lines 120 and 132 of <code>3000physicalview.c</code> (as also discussed in the lecture). Consider the following:
#* How are these functions different from <code>get_user()</code>/<code>put_user()</code> that we have seen in the previous tutorial?
#* Why are these functions necessary? Couldn't we just access the userspace address directly? What could happen if we did?
# <code>3000physicalview</code> exposes its API to userspace in the form of an <code>ioctl(2)</code> call. Consider the following:
#* What is an ioctl? How is it different from a read or write system call? Hint: check <code>man 2 ioctl</code>.
#* How does <code>3000physicalview</code> implement its ioctl? What arguments does it take?
#* How does <code>3000memview2</code> call the ioctl? What arguments does it pass to the ioctl?
# Which function does the virtual-to-physical translation? What type is <code>current->mm</code>? How does it give you the address of the <code>pgd</code>? Recall the page table walk explained in the lecture and see how it’s reflected in this function (writing it down is optional).
# Once you’ve got the page frame number (pfn), how is the physical address (phys) calculated? Describe in words.

==Tasks part B: Examining Physical Memory Mappings==
# With <code>3000physicalview</code> inserted, run <code>3000memview2</code> and examine the output. Note that it presents virtual memory addresses on the left, and physical addresses on the right. Are these mappings consistent with what you expected (e.g., in terms of patterns)?
# Compare <code>3000memview2</code> with <code>3000memview</code> from Tutorial 2. What is similar about their code, and what is different? How similar is their output?
# Do you notice a pattern in the virtual addresses of <code>buf[i]</code>? Is this same pattern present in the physical addresses? Why or why not?
# Run <code>3000memview2</code> a few more times and consider the following:
#* Are the virtual addresses the same or different between runs? How about physical addresses?
#* Some physical addresses don't seem to be changing between runs. Which ones? Why do you think this might be the case?
# Force the kernel to drop the virtual memory cache using <code>sync && echo 3 | sudo tee /proc/sys/vm/drop_caches</code>. Run <code>3000memview2</code> one more time and note that the physical addresses that stayed the same previously have now changed. What do you think just happened?

==Tasks part C: Using <code>bpftrace</code> to Monitor <code>3000physicalview</code> and <code>3000memview2</code>==
Now recall what you tried with eBPF in Tutorial 7 (where you applied the fix if applicable). We can also do something similar to watch the interaction between <code>3000physicalview</code> and <code>3000memview2</code>.
Note: No in-depth understanding of eBPF is expected in this course. You only need to understand what is involved and discussed. Feel free to read more if interested.

<ol>
<li>Use the following "one-liner" (as it is just a one-line string within the single quotation marks) in a new terminal session first and in the orginal one run <code>./3000memview2</code> as above:



<code>sudo bpftrace -e 'tracepoint:syscalls:sys_enter_ioctl { printf("%s: fd=%d; cmd=%d; arg=%ld \n", comm, args->fd, args->cmd, args->arg); }' | grep 3000memview2</code>



What does this one-liner do? As always, you can check <code>man bpftrace</code> if needed.</li>
<li>(Optional in the submission but you should do it) It seems the command above does not show useful information as the third argument is actually a pointer. So, to be able to show the two arguments <code>virt</code> and <code>phys</code> defined in <code>3000physicalview.h</code>, you will need to create a *.bt file.
Refer to <code>/usr/sbin/*.bt</code> and convert the one-liner above into something like <code>snoop3000physicalview.bt</code>, so that when it runs it prints the <code>virt</code> passed in and the <code>phys</code> that is returned.
The code will be posted and explained.
 Note: if you choose to directly include the header file you need to create a copy and remove the <code>MODULE_*</code> macros. Otherwise, you can simply include the struct definition.</li>
</ol>

COMP3000 Operating Systems F23: Tutorial 8

2023-10-25T01:56:05Z

Abdou:

In this tutorial, you’ll be learning about how virtual addresses are mapped to physical addresses (the address translation) and continue to use kernel modules to extract information that only the kernel has access to. In particular, the kernel module performs a 5-level page table walk to find out the physical address corresponding to a userspace virtual address. In addition to what was discussed in the class, You can also read [https://en.wikipedia.org/wiki/Intel_5-level_paging more about 5-level paging], if interested.

==Introduction==
WARNING: The commands and programs in this tutorial are potentially extremely dangerous and may result in crashes or loss of data. Additionally, questions may not work as expected on a machine other than the course VM. For that reason, you are strongly encouraged to do this tutorial on the provided OpenStack virtual machine.

To get started, we will first examine the source code for <tt>3000physicalview.c</tt> and <tt>3000memview2.c</tt>, both of which are in [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut8/3000physicalview.tar.gz 3000physicalview.tar.gz].

==Tasks part A: Getting Started==
Compile <tt>3000physicalview</tt> and <tt>3000memview2</tt> using the provided <tt>Makefile</tt> (i.e., by running <tt>make</tt>).

Insert <tt>3000physicalview</tt> by running <tt>make insert</tt>. Confirm that the module is inserted using <tt>lsmod</tt>.
# Examine the call to <tt>copy_from_user()</tt> and <tt>copy_to_user()</tt> on lines 120 and 132 of <tt>3000physicalview.c</tt> (as also discussed in the lecture). Consider the following:
#* How are these functions different from <tt>get_user()</tt>/<tt>put_user()</tt> that we have seen in the previous tutorial?
#* Why are these functions necessary? Couldn't we just access the userspace address directly? What could happen if we did?
# <tt>3000physicalview</tt> exposes its API to userspace in the form of an <tt>ioctl(2)</tt> call. Consider the following:
#* What is an ioctl? How is it different from a read or write system call? Hint: check <tt>man 2 ioctl</tt>.
#* How does <tt>3000physicalview</tt> implement its ioctl? What arguments does it take?
#* How does <tt>3000memview2</tt> call the ioctl? What arguments does it pass to the ioctl?
# Which function does the virtual-to-physical translation? What type is <tt>current->mm</tt>? How does it give you the address of the <tt>pgd</tt>? Recall the page table walk explained in the lecture and see how it’s reflected in this function (writing it down is optional).
# Once you’ve got the page frame number (pfn), how is the physical address (phys) calculated? Describe in words.

==Tasks part B: Examining Physical Memory Mappings==
# With <tt>3000physicalview</tt> inserted, run <tt>3000memview2</tt> and examine the output. Note that it presents virtual memory addresses on the left, and physical addresses on the right. Are these mappings consistent with what you expected (e.g., in terms of patterns)?
# Compare <tt>3000memview2</tt> with <tt>3000memview</tt> from Tutorial 2. What is similar about their code, and what is different? How similar is their output?
# Do you notice a pattern in the virtual addresses of <tt>buf[i]</tt>? Is this same pattern present in the physical addresses? Why or why not?
# Run <tt>3000memview2</tt> a few more times and consider the following:
#* Are the virtual addresses the same or different between runs? How about physical addresses?
#* Some physical addresses don't seem to be changing between runs. Which ones? Why do you think this might be the case?
# Force the kernel to drop the virtual memory cache using <tt>sync && echo 3 | sudo tee /proc/sys/vm/drop_caches</tt>. Run <tt>3000memview2</tt> one more time and note that the physical addresses that stayed the same previously have now changed. What do you think just happened?

==Tasks part C: Using <tt>bpftrace</tt> to Monitor <tt>3000physicalview</tt> and <tt>3000memview2</tt>==
Now recall what you tried with eBPF in Tutorial 7 (where you applied the fix if applicable). We can also do something similar to watch the interaction between <tt>3000physicalview</tt> and <tt>3000memview2</tt>.
Note: No in-depth understanding of eBPF is expected in this course. You only need to understand what is involved and discussed. Feel free to read more if interested.

<ol>
<li>Use the following "one-liner" (as it is just a one-line string within the single quotation marks) in a new terminal session first and in the orginal one run <tt>./3000memview2</tt> as above:



<tt>sudo bpftrace -e 'tracepoint:syscalls:sys_enter_ioctl { printf("%s: fd=%d; cmd=%d; arg=%ld \n", comm, args->fd, args->cmd, args->arg); }' | grep 3000memview2</tt>



What does this one-liner do? As always, you can check <tt>man bpftrace</tt> if needed.</li>
<li>(Optional in the submission but you should do it) It seems the command above does not show useful information as the third argument is actually a pointer. So, to be able to show the two arguments <tt>virt</tt> and <tt>phys</tt> defined in <tt>3000physicalview.h</tt>, you will need to create a *.bt file.
Refer to <tt>/usr/sbin/*.bt</tt> and convert the one-liner above into something like <tt>snoop3000physicalview.bt</tt>, so that when it runs it prints the <tt>virt</tt> passed in and the <tt>phys</tt> that is returned.
The code will be posted and explained.
 Note: if you choose to directly include the header file you need to create a copy and remove the <tt>MODULE_*</tt> macros. Otherwise, you can simply include the struct definition.</li>
</ol>

COMP3000 Operating Systems F23: Tutorial 8

2023-10-25T01:54:09Z

Abdou: /* General Instructions (same for all tutorials) */

In this tutorial, you’ll be learning about how virtual addresses are mapped to physical addresses (the address translation) and continue to use kernel modules to extract information that only the kernel has access to. In particular, the kernel module performs a 5-level page table walk to find out the physical address corresponding to a userspace virtual address. In addition to what was discussed in the class, You can also read [https://en.wikipedia.org/wiki/Intel_5-level_paging more about 5-level paging], if interested.

==Introduction==
WARNING: The commands and programs in this tutorial are potentially extremely dangerous and may result in crashes or loss of data. Additionally, questions may not work as expected on a machine other than the course VM. For that reason, you are strongly encouraged to do this tutorial on the provided OpenStack virtual machine.

To get started, we will first examine the source code for <tt>3000physicalview.c</tt> and <tt>3000memview2.c</tt>, both of which are in [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut8/3000physicalview.tar.gz 3000physicalview.tar.gz].

==Tasks part A: Getting Started==
Compile <tt>3000physicalview</tt> and <tt>3000memview2</tt> using the provided <tt>Makefile</tt> (i.e., by running <tt>make</tt>).

Insert <tt>3000physicalview</tt> by running <tt>make insert</tt>. Confirm that the module is inserted using <tt>lsmod</tt>.
# Examine the call to <tt>copy_from_user()</tt> and <tt>copy_to_user()</tt> on lines 120 and 132 of <tt>3000physicalview.c</tt> (as also discussed in the lecture). Consider the following:
#* How are these functions different from <tt>get_user()</tt>/<tt>put_user()</tt> that we have seen in the previous tutorial?
#* Why are these functions necessary? Couldn't we just access the userspace address directly? What could happen if we did?
# <tt>3000physicalview</tt> exposes its API to userspace in the form of an <tt>ioctl(2)</tt> call. Consider the following:
#* What is an ioctl? How is it different from a read or write system call? Hint: check <tt>man 2 ioctl</tt>.
#* How does <tt>3000physicalview</tt> implement its ioctl? What arguments does it take?
#* How does <tt>3000memview2</tt> call the ioctl? What arguments does it pass to the ioctl?
# Which function does the virtual-to-physical translation? What type is <tt>current->mm</tt>? How does it give you the address of the <tt>pgd</tt>? Recall the page table walk explained in the lecture and see how it’s reflected in this function (writing it down is optional).
# Once you’ve got the page frame number (pfn), how is the physical address (phys) calculated? Describe in words.

==Tasks part B: Examining Physical Memory Mappings==
# With <tt>3000physicalview</tt> inserted, run <tt>3000memview2</tt> and examine the output. Note that it presents virtual memory addresses on the left, and physical addresses on the right. Are these mappings consistent with what you expected (e.g., in terms of patterns)?
# Compare <tt>3000memview2</tt> with <tt>3000memview</tt> from Tutorial 2. What is similar about their code, and what is different? How similar is their output?
# Do you notice a pattern in the virtual addresses of <tt>buf[i]</tt>? Is this same pattern present in the physical addresses? Why or why not?
# Run <tt>3000memview2</tt> a few more times and consider the following:
#* Are the virtual addresses the same or different between runs? How about physical addresses?
#* Some physical addresses don't seem to be changing between runs. Which ones? Why do you think this might be the case?
# Force the kernel to drop the virtual memory cache using <tt>sync && echo 3 | sudo tee /proc/sys/vm/drop_caches</tt>. Run <tt>3000memview2</tt> one more time and note that the physical addresses that stayed the same previously have now changed. What do you think just happened?

==Tasks part C: Using <tt>bpftrace</tt> to Monitor <tt>3000physicalview</tt> and <tt>3000memview2</tt>==
Now recall what you tried with eBPF in Tutorial 7 (where you applied the fix if applicable). We can also do something similar to watch the interaction between <tt>3000physicalview</tt> and <tt>3000memview2</tt>.
Note: No in-depth understanding of eBPF is expected in this course. You only need to understand what is involved and discussed. Feel free to read more if interested.

<ol>
<li>Use the following "one-liner" (as it is just a one-line string within the single quotation marks) in a new terminal session first and in the orginal one run <tt>./3000memview2</tt> as above:



<tt>sudo bpftrace -e 'tracepoint:syscalls:sys_enter_ioctl { printf("%s: fd=%d; cmd=%d; arg=%ld \n", comm, args->fd, args->cmd, args->arg); }' | grep 3000memview2</tt>



What does this one-liner do? As always, you can check <tt>man bpftrace</tt> if needed.</li>
<li>(Optional in the submission but you should do it) It seems the command above does not show useful information as the third argument is actually a pointer. So, to be able to show the two arguments <tt>virt</tt> and <tt>phys</tt> defined in <tt>3000physicalview.h</tt>, you will need to create a *.bt file.
Refer to <tt>/usr/sbin/*.bt</tt> and convert the one-liner above into something like <tt>snoop3000physicalview.bt</tt>, so that when it runs it prints the <tt>virt</tt> passed in and the <tt>phys</tt> that is returned.
The code will be posted and explained.
 Note: if you choose to directly include the header file you need to create a copy and remove the <tt>MODULE_*</tt> macros. Otherwise, you can simply include the struct definition.</li>
</ol>

COMP3000 Operating Systems F23: Tutorial 8

2023-10-25T01:50:57Z

Abdou: Created page with "In this tutorial, you’ll be learning about how virtual addresses are mapped to physical addresses (the address translation) and continue to use kernel modules to extract information that only the kernel has access to. In particular, the kernel module performs a 5-level page table walk to find out the physical address corresponding to a userspace virtual address. In addition to what was discussed in the class, You can also read [https://en.wikipedia.org/wiki/Intel_5-lev..."

In this tutorial, you’ll be learning about how virtual addresses are mapped to physical addresses (the address translation) and continue to use kernel modules to extract information that only the kernel has access to. In particular, the kernel module performs a 5-level page table walk to find out the physical address corresponding to a userspace virtual address. In addition to what was discussed in the class, You can also read [https://en.wikipedia.org/wiki/Intel_5-level_paging more about 5-level paging], if interested.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''), but you should still turn in your work. Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA (in person or by responding to a poll if online). Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Introduction==
WARNING: The commands and programs in this tutorial are potentially extremely dangerous and may result in crashes or loss of data. Additionally, questions may not work as expected on a machine other than the course VM. For that reason, you are strongly encouraged to do this tutorial on the provided OpenStack virtual machine.

To get started, we will first examine the source code for <tt>3000physicalview.c</tt> and <tt>3000memview2.c</tt>, both of which are in [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut8/3000physicalview.tar.gz 3000physicalview.tar.gz].

==Tasks part A: Getting Started==
Compile <tt>3000physicalview</tt> and <tt>3000memview2</tt> using the provided <tt>Makefile</tt> (i.e., by running <tt>make</tt>).

Insert <tt>3000physicalview</tt> by running <tt>make insert</tt>. Confirm that the module is inserted using <tt>lsmod</tt>.
# Examine the call to <tt>copy_from_user()</tt> and <tt>copy_to_user()</tt> on lines 120 and 132 of <tt>3000physicalview.c</tt> (as also discussed in the lecture). Consider the following:
#* How are these functions different from <tt>get_user()</tt>/<tt>put_user()</tt> that we have seen in the previous tutorial?
#* Why are these functions necessary? Couldn't we just access the userspace address directly? What could happen if we did?
# <tt>3000physicalview</tt> exposes its API to userspace in the form of an <tt>ioctl(2)</tt> call. Consider the following:
#* What is an ioctl? How is it different from a read or write system call? Hint: check <tt>man 2 ioctl</tt>.
#* How does <tt>3000physicalview</tt> implement its ioctl? What arguments does it take?
#* How does <tt>3000memview2</tt> call the ioctl? What arguments does it pass to the ioctl?
# Which function does the virtual-to-physical translation? What type is <tt>current->mm</tt>? How does it give you the address of the <tt>pgd</tt>? Recall the page table walk explained in the lecture and see how it’s reflected in this function (writing it down is optional).
# Once you’ve got the page frame number (pfn), how is the physical address (phys) calculated? Describe in words.

==Tasks part B: Examining Physical Memory Mappings==
# With <tt>3000physicalview</tt> inserted, run <tt>3000memview2</tt> and examine the output. Note that it presents virtual memory addresses on the left, and physical addresses on the right. Are these mappings consistent with what you expected (e.g., in terms of patterns)?
# Compare <tt>3000memview2</tt> with <tt>3000memview</tt> from Tutorial 2. What is similar about their code, and what is different? How similar is their output?
# Do you notice a pattern in the virtual addresses of <tt>buf[i]</tt>? Is this same pattern present in the physical addresses? Why or why not?
# Run <tt>3000memview2</tt> a few more times and consider the following:
#* Are the virtual addresses the same or different between runs? How about physical addresses?
#* Some physical addresses don't seem to be changing between runs. Which ones? Why do you think this might be the case?
# Force the kernel to drop the virtual memory cache using <tt>sync && echo 3 | sudo tee /proc/sys/vm/drop_caches</tt>. Run <tt>3000memview2</tt> one more time and note that the physical addresses that stayed the same previously have now changed. What do you think just happened?

==Tasks part C: Using <tt>bpftrace</tt> to Monitor <tt>3000physicalview</tt> and <tt>3000memview2</tt>==
Now recall what you tried with eBPF in Tutorial 7 (where you applied the fix if applicable). We can also do something similar to watch the interaction between <tt>3000physicalview</tt> and <tt>3000memview2</tt>.
Note: No in-depth understanding of eBPF is expected in this course. You only need to understand what is involved and discussed. Feel free to read more if interested.

<ol>
<li>Use the following "one-liner" (as it is just a one-line string within the single quotation marks) in a new terminal session first and in the orginal one run <tt>./3000memview2</tt> as above:



<tt>sudo bpftrace -e 'tracepoint:syscalls:sys_enter_ioctl { printf("%s: fd=%d; cmd=%d; arg=%ld \n", comm, args->fd, args->cmd, args->arg); }' | grep 3000memview2</tt>



What does this one-liner do? As always, you can check <tt>man bpftrace</tt> if needed.</li>
<li>(Optional in the submission but you should do it) It seems the command above does not show useful information as the third argument is actually a pointer. So, to be able to show the two arguments <tt>virt</tt> and <tt>phys</tt> defined in <tt>3000physicalview.h</tt>, you will need to create a *.bt file.
Refer to <tt>/usr/sbin/*.bt</tt> and convert the one-liner above into something like <tt>snoop3000physicalview.bt</tt>, so that when it runs it prints the <tt>virt</tt> passed in and the <tt>phys</tt> that is returned.
The code will be posted and explained.
 Note: if you choose to directly include the header file you need to create a copy and remove the <tt>MODULE_*</tt> macros. Otherwise, you can simply include the struct definition.</li>
</ol>

COMP3000 Operating Systems 2023F Tutorials

2023-10-25T01:50:50Z

Abdou: /* Tutorial Sections */

This page contains Tutorial instructions for COMP3000 A, Fall 2023.

The main page for the course can be found [https://people.scs.carleton.ca/~abdou/COMP3000_F2023.html here].

==Information about the Tutorials==
Tutorials are in person. You need to be physically in the room to finish up the tutorial instructions on your laptop and get checked-out by one of the two present TAs. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

Please stick to the tutorial session you are registered in. This avoids overcrowding.

Make well use of these tutorials to help you better understand the course material. The plan is to have 9 tutorial sessions this term, each would be worth 2.22% of your total grade. We might revisit this as the term progresses.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

== Getting Access to a Linux Machine ==

For the tutorials, you need to get access to a Linux machine. It is recommended to use an SCS Openstack instance (see instructions below). Alternatively, if you prefer to work directly on your laptop and avoid Openstack, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova this VirtualBox image] (courtesy of Prof. Zhao from Winter 2023). You will need access to a Linux system for the entire semester, ideally the same one.

=== Setting up an Openstack VM instance for the first time ===
We will use Openstack for the tutorials of this course. Follow these instructions for setting up an Openstack VM instance: [[COMP3000 Operating Systems F23: Openstack Instructions | instructions here]]. You only need to do this once for the whole term. Please use the same instance for both: the tutorials and the assignments. ''No need to create two instances''.

=== Connecting to the VM ===
You can find instructions with various ways to connect to your Openstack instance [[COMP3000 Operating Systems F23: Connecting to SCS Openstack | here]].

=== Downloading files from the VM ===
To download files from your Openstack VM instance, you can find instructions [[COMP3000 Operating Systems F23: Downloading files from your Openstack VM | here]].

=== Backing up the VM ===

To backup your VM, the image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account's home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

Reminder: ''do '''not''' take snapshots of your instance!'' Note that you cannot take snapshots of your VM anyway (it will keep trying and never succeed, and you'll create work for the tech staff who will have to undo what you did), so please don't try.

==Tutorial Sections==

* COMP 3000 A1: Tue: 08:35 - 09:55 at Loeb 720 (LA720). [Ethan and Sneha (T1-T9)]
* COMP 3000 A2: Fri: 11:35 - 12:55 at Loeb 720 (LA720). [Ethan and Ali (T1-T3), Ethan and Nareen (T4-T6), Ali and Nareen (T7-T9)]
* COMP 3000 A3: Thu: 08:35 - 09:55 at St. Patrick's 303 (SP303). [Ali and Nareen (T1-T9)]

{| class="wikitable"
|+ Tutorial Schedules
|-
! Tutorial !! Section !! Date !! Time !! Room !! TAs
|-
| rowspan="3" | [[COMP3000 Operating Systems 2023F Tutorials: Tutorial 1 | Tutorial 1]] || A1 || Sep 26 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Sep 28 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Sep 29 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 2 | Tutorial 2]] || A1 || Oct 3 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 5 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 6 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 3 | Tutorial 3]] || A1 || Oct 10 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 12 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 13 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 4 | Tutorial 4]] || A1 || Oct 31 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 2 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 3 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 5 | Tutorial 5]] || A1 || Nov 7 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 9 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 10 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 6 | Tutorial 6]] || A1 || Nov 14 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 16 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 17 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 7 | Tutorial 7]] || A1 || Nov 21 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 23 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 24 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 8 | Tutorial 8]] || A1 || Nov 28 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 30 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 1 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | Tutorial 9 || A1 || Dec 5 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Dec 7 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 8 || 11:35 am-12:55 pm || MC2000 || Ali and Nareen
|-
|}

COMP3000 Operating Systems 2023F Tutorials

2023-10-25T01:50:23Z

Abdou: /* Tutorial Sections */

This page contains Tutorial instructions for COMP3000 A, Fall 2023.

The main page for the course can be found [https://people.scs.carleton.ca/~abdou/COMP3000_F2023.html here].

==Information about the Tutorials==
Tutorials are in person. You need to be physically in the room to finish up the tutorial instructions on your laptop and get checked-out by one of the two present TAs. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

Please stick to the tutorial session you are registered in. This avoids overcrowding.

Make well use of these tutorials to help you better understand the course material. The plan is to have 9 tutorial sessions this term, each would be worth 2.22% of your total grade. We might revisit this as the term progresses.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

== Getting Access to a Linux Machine ==

For the tutorials, you need to get access to a Linux machine. It is recommended to use an SCS Openstack instance (see instructions below). Alternatively, if you prefer to work directly on your laptop and avoid Openstack, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova this VirtualBox image] (courtesy of Prof. Zhao from Winter 2023). You will need access to a Linux system for the entire semester, ideally the same one.

=== Setting up an Openstack VM instance for the first time ===
We will use Openstack for the tutorials of this course. Follow these instructions for setting up an Openstack VM instance: [[COMP3000 Operating Systems F23: Openstack Instructions | instructions here]]. You only need to do this once for the whole term. Please use the same instance for both: the tutorials and the assignments. ''No need to create two instances''.

=== Connecting to the VM ===
You can find instructions with various ways to connect to your Openstack instance [[COMP3000 Operating Systems F23: Connecting to SCS Openstack | here]].

=== Downloading files from the VM ===
To download files from your Openstack VM instance, you can find instructions [[COMP3000 Operating Systems F23: Downloading files from your Openstack VM | here]].

=== Backing up the VM ===

To backup your VM, the image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account's home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

Reminder: ''do '''not''' take snapshots of your instance!'' Note that you cannot take snapshots of your VM anyway (it will keep trying and never succeed, and you'll create work for the tech staff who will have to undo what you did), so please don't try.

==Tutorial Sections==

* COMP 3000 A1: Tue: 08:35 - 09:55 at Loeb 720 (LA720). [Ethan and Sneha (T1-T9)]
* COMP 3000 A2: Fri: 11:35 - 12:55 at Loeb 720 (LA720). [Ethan and Ali (T1-T3), Ethan and Nareen (T4-T6), Ali and Nareen (T7-T9)]
* COMP 3000 A3: Thu: 08:35 - 09:55 at St. Patrick's 303 (SP303). [Ali and Nareen (T1-T9)]

{| class="wikitable"
|+ Tutorial Schedules
|-
! Tutorial !! Section !! Date !! Time !! Room !! TAs
|-
| rowspan="3" | [[COMP3000 Operating Systems 2023F Tutorials: Tutorial 1 | Tutorial 1]] || A1 || Sep 26 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Sep 28 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Sep 29 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 2 | Tutorial 2]] || A1 || Oct 3 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 5 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 6 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 3 | Tutorial 3]] || A1 || Oct 10 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 12 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 13 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 4 | Tutorial 4]] || A1 || Oct 31 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 2 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 3 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 5 | Tutorial 5]] || A1 || Nov 7 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 9 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 10 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 6 | Tutorial 6]] || A1 || Nov 14 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 16 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 17 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 7 | Tutorial 7]] || A1 || Nov 21 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 23 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 24 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems W23: Tutorial 8 | Tutorial 8]] || A1 || Nov 28 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 30 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 1 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | Tutorial 9 || A1 || Dec 5 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Dec 7 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 8 || 11:35 am-12:55 pm || MC2000 || Ali and Nareen
|-
|}

COMP3000 Operating Systems F23: Tutorial 7

2023-10-25T01:44:44Z

Abdou: /* Using bpftrace to monitor kernel module events */

In this tutorial you'll be learning about special files and Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface.

==Important Tips==
* There is a chance your VM will crash or your ssh session will become unresponsive when messing with the kernel. To recover, you can reboot your VM from the OpenStack web console.
* '''Definitely do not under any circumstances attempt this on your own system. This is your last warning.'''
* Remember to recompile and insert your module after making changes.
* Work incrementally and be prepared for your system to crash.
* It might be wise to code on your own computer and use scp or sshfs to transfer the files to your VM.
* https://elixir.bootlin.com/linux/latest/source is a great resource for learning about kernel functions and data structures.

Note: although it's highly recommended that you use the openstack instance, if you cannot for a reason, or if you encounter other issues, check out [[COMP3000 Operating Systems F23: Tutorial 7 Not Using SCS OpenStack | these instructions]] to see if they help.

==Special Files==
Before you start, review what you have learned about special files and think about the following questions (no need to answer them): 1) What does a special file represent and what backs it up? 2) Can you have multiple special files that are somehow "the same"? What does it mean to "copy" a special file? 3) How are named pipes different from unnamed pipes and what are their similarities? Take this opportunity to deepen your understanding of special files to prepare you for working with device drivers that support such special files.

===Tasks part A: Understanding special files===
# Try the following commands as a non-privileged user. What does each do? How do the files f1-f3 compare? How do they compare to <code>/dev/urandom</code>? Remember you can get output from <code>/dev/urandom</code> using <code>cat</code> or <code>dd</code>. If you use <code>cat</code>, make sure to pipe it to <code>less</code>!
#* <code>cp /dev/urandom f1</code> (hit Ctrl+C right away to avoid a huge file)
#* <code>sudo cp -a /dev/urandom f2</code>
#* <code>sudo mknod f3 c 1 9</code>
#: Note: If you see any command running for more than a few seconds, hit Ctrl+C right away, check the produced file (often it’s very large) and delete it if not needed.
# Make named pipes using <code>mknod</code> and <code>mkfifo</code>. Use them to simulate <code>ls | wc</code> using just the '>' and '<' operators.
# Use <code>mknod</code> to make a copy of your current terminal's tty - <code>/dev/tty</code> (Hint: consider how you did something similar with <code>/dev/urandom</code> above). You can name it <code>mytty</code>. Examine its characteristics using <code>stty --file=mytty</code>. Do the same for the original tty.
# Run <code>stty --help</code> to see what you can do with stty. Try disabling local echo. How does the shell behave with echo disabled? How can you restore echo without logging out and back in? Recall we did this in the class.

==Kernel Modules==
In this part of the tutorial you will be building and installing kernel modules. You will need root access to install kernel modules.

It is '''highly recommended''' that you use a comp3000 openstack instance for the exercises below for two reasons. First, you may have difficulties compiling kernel modules on other systems. Second, these operations are potentially dangerous, and mistakes could '''destroy all data''' on the Linux system. Consider yourself warned!

Before you start, review what you have learned about kernel modules and device drivers, and think about the following questions (no need to answer them): 1) How does the source of kernel modules differ from C programs? 2) How does building kernel modules differ from building userspace C programs? 3) How does invoking the file operations (e.g., read/write) of a special file trigger corresponding functions in a device driver?

You will learn how you can access and manipulate kernel data structures and call kernel functions in a kernel module. You will also understand how processes are represented by <code>task_struct</code>'s in the Linux kernel, what kind of information is stored in a task struct, and how to access the <code>task_struct</code> of a process.

===Tasks part B: A simple kernel module===
Download the source for this [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/simple.tar.gz simple module], unpack, and build it by typing "<code>make</code>". You can safely ignore the warning of "Skipping BTF generation...".
# Install the module using "<code>sudo insmod simple.ko</code>". The hello message is recorded in the kernel logs. How do you view the kernel logs? How many ways are there to view them?
# Check to see that the module has been loaded. How do you do this?
# Remove the module from the kernel. What did you do?

===Tasks part C: A character device kernel module (driver)===
Download the source for [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/ones.tar.gz ones], a kernel module implementing a character device that outputs an unbounded string of "1"'s. Build, compile, and run it as before.
# What kernel messages does the module generate? Does it create any new files (other than <code>/dev/ones</code>)? If so, where? (Hint: search in <code>/sys</code>)
# What happens when you "cat" the device <code>/dev/ones</code>? How can you limit the output in more than one way?
# Add more <code>printk()</code>’s (where you see needed) to find out at which point which functions in ones.c are called and for which purpose.

===Tasks part D: Getting process information from a module===
Download the source [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/newgetpid.tar.gz newgetpid.c]. Build and run it as before. Hints for the questions below:

https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h,
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/current.h,
https://elixir.bootlin.com/linux/latest/source/include/linux/cred.h
# What type is "<code>current</code>"? How can you figure this out?
# Modify newgetpid.c so that it creates a device file <code>/dev/describe</code> rather than <code>/dev/newgetpid</code>.
# Make <code>/dev/describe</code> output the calling process's parent ID (ppid), user ID (uid), group ID (gid), effective user ID (euid), and effective group ID (egid).

==Using <code>bpftrace</code> to monitor kernel module events==
Note: the following is optional but you are encouraged to do it, as it can help you understand eBPF better (see how simple but still powerful an eBPF program can be).

If you see an error like "<code>Could not resolve symbol</code>" when using <code>bpftrace</code>, you need to apply a fix by installing the dbgsym package of <code>bpftrace</code>: Just <code>wget</code> [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/install-bpftrace-dbgsym this file], and run it with

sudo apt update
source install-bpftrace-dbgsym

Download this tiny code [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/kmsnoop.bt kmsnoop.bt] and make it executable with <code>chmod 755 kmsnoop.bt</code>.

Now, you can open a separate SSH session and run it with <code>sudo ./kmsnoop.bt</code>. It will wait for events. When you do any "insmod" or "rmmod" in the other session, you will see it outputs a line of message.

If you want to avoid opening another SSH window, you can use tmux (a terminal multiplexer) by just typing tmux and hit enter.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

How could this save you from doing any <code>printk</code>'s in your kernel module code?
You can also <code>sudo bpftrace -l | grep</code> for your needed kernel functions (e.g.,<code>printk</code>) to hook to.

COMP3000 Operating Systems F23: Tutorial 7

2023-10-25T01:43:35Z

Abdou:

In this tutorial you'll be learning about special files and Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface.

==Important Tips==
* There is a chance your VM will crash or your ssh session will become unresponsive when messing with the kernel. To recover, you can reboot your VM from the OpenStack web console.
* '''Definitely do not under any circumstances attempt this on your own system. This is your last warning.'''
* Remember to recompile and insert your module after making changes.
* Work incrementally and be prepared for your system to crash.
* It might be wise to code on your own computer and use scp or sshfs to transfer the files to your VM.
* https://elixir.bootlin.com/linux/latest/source is a great resource for learning about kernel functions and data structures.

Note: although it's highly recommended that you use the openstack instance, if you cannot for a reason, or if you encounter other issues, check out [[COMP3000 Operating Systems F23: Tutorial 7 Not Using SCS OpenStack | these instructions]] to see if they help.

==Special Files==
Before you start, review what you have learned about special files and think about the following questions (no need to answer them): 1) What does a special file represent and what backs it up? 2) Can you have multiple special files that are somehow "the same"? What does it mean to "copy" a special file? 3) How are named pipes different from unnamed pipes and what are their similarities? Take this opportunity to deepen your understanding of special files to prepare you for working with device drivers that support such special files.

===Tasks part A: Understanding special files===
# Try the following commands as a non-privileged user. What does each do? How do the files f1-f3 compare? How do they compare to <code>/dev/urandom</code>? Remember you can get output from <code>/dev/urandom</code> using <code>cat</code> or <code>dd</code>. If you use <code>cat</code>, make sure to pipe it to <code>less</code>!
#* <code>cp /dev/urandom f1</code> (hit Ctrl+C right away to avoid a huge file)
#* <code>sudo cp -a /dev/urandom f2</code>
#* <code>sudo mknod f3 c 1 9</code>
#: Note: If you see any command running for more than a few seconds, hit Ctrl+C right away, check the produced file (often it’s very large) and delete it if not needed.
# Make named pipes using <code>mknod</code> and <code>mkfifo</code>. Use them to simulate <code>ls | wc</code> using just the '>' and '<' operators.
# Use <code>mknod</code> to make a copy of your current terminal's tty - <code>/dev/tty</code> (Hint: consider how you did something similar with <code>/dev/urandom</code> above). You can name it <code>mytty</code>. Examine its characteristics using <code>stty --file=mytty</code>. Do the same for the original tty.
# Run <code>stty --help</code> to see what you can do with stty. Try disabling local echo. How does the shell behave with echo disabled? How can you restore echo without logging out and back in? Recall we did this in the class.

==Kernel Modules==
In this part of the tutorial you will be building and installing kernel modules. You will need root access to install kernel modules.

It is '''highly recommended''' that you use a comp3000 openstack instance for the exercises below for two reasons. First, you may have difficulties compiling kernel modules on other systems. Second, these operations are potentially dangerous, and mistakes could '''destroy all data''' on the Linux system. Consider yourself warned!

Before you start, review what you have learned about kernel modules and device drivers, and think about the following questions (no need to answer them): 1) How does the source of kernel modules differ from C programs? 2) How does building kernel modules differ from building userspace C programs? 3) How does invoking the file operations (e.g., read/write) of a special file trigger corresponding functions in a device driver?

You will learn how you can access and manipulate kernel data structures and call kernel functions in a kernel module. You will also understand how processes are represented by <code>task_struct</code>'s in the Linux kernel, what kind of information is stored in a task struct, and how to access the <code>task_struct</code> of a process.

===Tasks part B: A simple kernel module===
Download the source for this [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/simple.tar.gz simple module], unpack, and build it by typing "<code>make</code>". You can safely ignore the warning of "Skipping BTF generation...".
# Install the module using "<code>sudo insmod simple.ko</code>". The hello message is recorded in the kernel logs. How do you view the kernel logs? How many ways are there to view them?
# Check to see that the module has been loaded. How do you do this?
# Remove the module from the kernel. What did you do?

===Tasks part C: A character device kernel module (driver)===
Download the source for [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/ones.tar.gz ones], a kernel module implementing a character device that outputs an unbounded string of "1"'s. Build, compile, and run it as before.
# What kernel messages does the module generate? Does it create any new files (other than <code>/dev/ones</code>)? If so, where? (Hint: search in <code>/sys</code>)
# What happens when you "cat" the device <code>/dev/ones</code>? How can you limit the output in more than one way?
# Add more <code>printk()</code>’s (where you see needed) to find out at which point which functions in ones.c are called and for which purpose.

===Tasks part D: Getting process information from a module===
Download the source [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/newgetpid.tar.gz newgetpid.c]. Build and run it as before. Hints for the questions below:

https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h,
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/current.h,
https://elixir.bootlin.com/linux/latest/source/include/linux/cred.h
# What type is "<code>current</code>"? How can you figure this out?
# Modify newgetpid.c so that it creates a device file <code>/dev/describe</code> rather than <code>/dev/newgetpid</code>.
# Make <code>/dev/describe</code> output the calling process's parent ID (ppid), user ID (uid), group ID (gid), effective user ID (euid), and effective group ID (egid).

==Using <code>bpftrace</code> to monitor kernel module events==
Note: you are not required to write about your experience for this part in the submitted work, but you should still do it, which can help you understand eBPF better (see how simple but still powerful an eBPF program can be).

If you see an error like "<code>Could not resolve symbol</code>" when using <code>bpftrace</code>, you need to apply a fix by installing the dbgsym package of <code>bpftrace</code>: Just <code>wget</code> [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/install-bpftrace-dbgsym this file], and run it with

sudo apt update
source install-bpftrace-dbgsym

Download this tiny code [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/kmsnoop.bt kmsnoop.bt] and make it executable with <code>chmod 755 kmsnoop.bt</code>.

Now, you can open a separate SSH session and run it with <code>sudo ./kmsnoop.bt</code>. It will wait for events. When you do any "insmod" or "rmmod" in the other session, you will see it outputs a line of message.

If you want to avoid opening another SSH window, you can use tmux (a terminal multiplexer) by just typing tmux and hit enter.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

How could this save you from doing any <code>printk</code>'s in your kernel module code?
You can also <code>sudo bpftrace -l | grep</code> for your needed kernel functions (e.g.,<code>printk</code>) to hook to.

COMP3000 Operating Systems F23: Tutorial 7

2023-10-25T01:41:07Z

Abdou: /* Using bpftrace to monitor kernel module events */

In this tutorial you'll be learning about special files and Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface.

==Important Tips==
* There is a chance your VM will crash or your ssh session will become unresponsive when messing with the kernel. To recover, you can reboot your VM from the OpenStack web console.
* '''Definitely do not under any circumstances attempt this on your own system. This is your last warning.'''
* Remember to recompile and insert your module after making changes.
* Work incrementally and be prepared for your system to crash.
* It might be wise to code on your own computer and use scp or sshfs to transfer the files to your VM.
* https://elixir.bootlin.com/linux/latest/source is a great resource for learning about kernel functions and data structures.

Note: although it's highly recommended that you use the openstack instance, if you cannot for a reason, or if you encounter other issues, check out [[COMP3000 Operating Systems F23: Tutorial 7 Not Using SCS OpenStack | these instructions]] to see if they help.

==Special Files==
Before you start, review what you have learned about special files and think about the following questions (no need to answer them): 1) What does a special file represent and what backs it up? 2) Can you have multiple special files that are somehow "the same"? What does it mean to "copy" a special file? 3) How are named pipes different from unnamed pipes and what are their similarities? Take this opportunity to deepen your understanding of special files to prepare you for working with device drivers that support such special files.

===Tasks part A: Understanding special files===
# Try the following commands as a non-privileged user. What does each do? How do the files f1-f3 compare? How do they compare to <tt>/dev/urandom</tt>? Remember you can get output from <tt>/dev/urandom</tt> using <tt>cat</tt> or <tt>dd</tt>. If you use <tt>cat</tt>, make sure to pipe it to <tt>less</tt>!
#* <tt>cp /dev/urandom f1</tt> (hit Ctrl+C right away to avoid a huge file)
#* <tt>sudo cp -a /dev/urandom f2</tt>
#* <tt>sudo mknod f3 c 1 9</tt>
#: Note: If you see any command running for more than a few seconds, hit Ctrl+C right away, check the produced file (often it’s very large) and delete it if not needed.
# Make named pipes using <tt>mknod</tt> and <tt>mkfifo</tt>. Use them to simulate <tt>ls | wc</tt> using just the '>' and '<' operators.
# Use <tt>mknod</tt> to make a copy of your current terminal's tty - <tt>/dev/tty</tt> (Hint: consider how you did something similar with <tt>/dev/urandom</tt> above). You can name it <tt>mytty</tt>. Examine its characteristics using <tt>stty --file=mytty</tt>. Do the same for the original tty.
# Run <tt>stty --help</tt> to see what you can do with stty. Try disabling local echo. How does the shell behave with echo disabled? How can you restore echo without logging out and back in? Recall we did this in the class.

==Kernel Modules==
In this part of the tutorial you will be building and installing kernel modules. You will need root access to install kernel modules.

It is '''highly recommended''' that you use a comp3000 openstack instance for the exercises below for two reasons. First, you may have difficulties compiling kernel modules on other systems. Second, these operations are potentially dangerous, and mistakes could '''destroy all data''' on the Linux system. Consider yourself warned!

Before you start, review what you have learned about kernel modules and device drivers, and think about the following questions (no need to answer them): 1) How does the source of kernel modules differ from C programs? 2) How does building kernel modules differ from building userspace C programs? 3) How does invoking the file operations (e.g., read/write) of a special file trigger corresponding functions in a device driver?

You will learn how you can access and manipulate kernel data structures and call kernel functions in a kernel module. You will also understand how processes are represented by <tt>task_struct</tt>'s in the Linux kernel, what kind of information is stored in a task struct, and how to access the <tt>task_struct</tt> of a process.

===Tasks part B: A simple kernel module===
Download the source for this [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/simple.tar.gz simple module], unpack, and build it by typing "<tt>make</tt>". You can safely ignore the warning of "Skipping BTF generation...".
# Install the module using "<tt>sudo insmod simple.ko</tt>". The hello message is recorded in the kernel logs. How do you view the kernel logs? How many ways are there to view them?
# Check to see that the module has been loaded. How do you do this?
# Remove the module from the kernel. What did you do?

===Tasks part C: A character device kernel module (driver)===
Download the source for [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/ones.tar.gz ones], a kernel module implementing a character device that outputs an unbounded string of "1"'s. Build, compile, and run it as before.
# What kernel messages does the module generate? Does it create any new files (other than <tt>/dev/ones</tt>)? If so, where? (Hint: search in <tt>/sys</tt>)
# What happens when you "cat" the device <tt>/dev/ones</tt>? How can you limit the output in more than one way?
# Add more <tt>printk()</tt>’s (where you see needed) to find out at which point which functions in ones.c are called and for which purpose.

===Tasks part D: Getting process information from a module===
Download the source [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/newgetpid.tar.gz newgetpid.c]. Build and run it as before. Hints for the questions below:

https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h,
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/current.h,
https://elixir.bootlin.com/linux/latest/source/include/linux/cred.h
# What type is "<tt>current</tt>"? How can you figure this out?
# Modify newgetpid.c so that it creates a device file <tt>/dev/describe</tt> rather than <tt>/dev/newgetpid</tt>.
# Make <tt>/dev/describe</tt> output the calling process's parent ID (ppid), user ID (uid), group ID (gid), effective user ID (euid), and effective group ID (egid).

==Using <tt>bpftrace</tt> to monitor kernel module events==
Note: you are not required to write about your experience for this part in the submitted work, but you should still do it, which can help you understand eBPF better (see how simple but still powerful an eBPF program can be).

If you see an error like "<tt>Could not resolve symbol</tt>" when using <tt>bpftrace</tt>, you need to apply a fix by installing the dbgsym package of <tt>bpftrace</tt>: Just <tt>wget</tt> [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/install-bpftrace-dbgsym this file], and run it with

sudo apt update
source install-bpftrace-dbgsym

Download this tiny code [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/kmsnoop.bt kmsnoop.bt] and make it executable with <tt>chmod 755 kmsnoop.bt</tt>.

Now, you can open a separate SSH session and run it with <tt>sudo ./kmsnoop.bt</tt>. It will wait for events. When you do any "insmod" or "rmmod" in the other session, you will see it outputs a line of message.

If you want to avoid opening another SSH window, you can use tmux (a terminal multiplexer) by just typing tmux and hit enter.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

How could this save you from doing any <tt>printk</tt>'s in your kernel module code?
You can also <tt>sudo bpftrace -l | grep</tt> for your needed kernel functions (e.g.,<tt>printk</tt>) to hook to.

COMP3000 Operating Systems F23: Tutorial 7

2023-10-25T01:39:47Z

Abdou: /* Tasks part D: Getting process information from a module */

In this tutorial you'll be learning about special files and Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface.

==Important Tips==
* There is a chance your VM will crash or your ssh session will become unresponsive when messing with the kernel. To recover, you can reboot your VM from the OpenStack web console.
* '''Definitely do not under any circumstances attempt this on your own system. This is your last warning.'''
* Remember to recompile and insert your module after making changes.
* Work incrementally and be prepared for your system to crash.
* It might be wise to code on your own computer and use scp or sshfs to transfer the files to your VM.
* https://elixir.bootlin.com/linux/latest/source is a great resource for learning about kernel functions and data structures.

Note: although it's highly recommended that you use the openstack instance, if you cannot for a reason, or if you encounter other issues, check out [[COMP3000 Operating Systems F23: Tutorial 7 Not Using SCS OpenStack | these instructions]] to see if they help.

==Special Files==
Before you start, review what you have learned about special files and think about the following questions (no need to answer them): 1) What does a special file represent and what backs it up? 2) Can you have multiple special files that are somehow "the same"? What does it mean to "copy" a special file? 3) How are named pipes different from unnamed pipes and what are their similarities? Take this opportunity to deepen your understanding of special files to prepare you for working with device drivers that support such special files.

===Tasks part A: Understanding special files===
# Try the following commands as a non-privileged user. What does each do? How do the files f1-f3 compare? How do they compare to <tt>/dev/urandom</tt>? Remember you can get output from <tt>/dev/urandom</tt> using <tt>cat</tt> or <tt>dd</tt>. If you use <tt>cat</tt>, make sure to pipe it to <tt>less</tt>!
#* <tt>cp /dev/urandom f1</tt> (hit Ctrl+C right away to avoid a huge file)
#* <tt>sudo cp -a /dev/urandom f2</tt>
#* <tt>sudo mknod f3 c 1 9</tt>
#: Note: If you see any command running for more than a few seconds, hit Ctrl+C right away, check the produced file (often it’s very large) and delete it if not needed.
# Make named pipes using <tt>mknod</tt> and <tt>mkfifo</tt>. Use them to simulate <tt>ls | wc</tt> using just the '>' and '<' operators.
# Use <tt>mknod</tt> to make a copy of your current terminal's tty - <tt>/dev/tty</tt> (Hint: consider how you did something similar with <tt>/dev/urandom</tt> above). You can name it <tt>mytty</tt>. Examine its characteristics using <tt>stty --file=mytty</tt>. Do the same for the original tty.
# Run <tt>stty --help</tt> to see what you can do with stty. Try disabling local echo. How does the shell behave with echo disabled? How can you restore echo without logging out and back in? Recall we did this in the class.

==Kernel Modules==
In this part of the tutorial you will be building and installing kernel modules. You will need root access to install kernel modules.

It is '''highly recommended''' that you use a comp3000 openstack instance for the exercises below for two reasons. First, you may have difficulties compiling kernel modules on other systems. Second, these operations are potentially dangerous, and mistakes could '''destroy all data''' on the Linux system. Consider yourself warned!

Before you start, review what you have learned about kernel modules and device drivers, and think about the following questions (no need to answer them): 1) How does the source of kernel modules differ from C programs? 2) How does building kernel modules differ from building userspace C programs? 3) How does invoking the file operations (e.g., read/write) of a special file trigger corresponding functions in a device driver?

You will learn how you can access and manipulate kernel data structures and call kernel functions in a kernel module. You will also understand how processes are represented by <tt>task_struct</tt>'s in the Linux kernel, what kind of information is stored in a task struct, and how to access the <tt>task_struct</tt> of a process.

===Tasks part B: A simple kernel module===
Download the source for this [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/simple.tar.gz simple module], unpack, and build it by typing "<tt>make</tt>". You can safely ignore the warning of "Skipping BTF generation...".
# Install the module using "<tt>sudo insmod simple.ko</tt>". The hello message is recorded in the kernel logs. How do you view the kernel logs? How many ways are there to view them?
# Check to see that the module has been loaded. How do you do this?
# Remove the module from the kernel. What did you do?

===Tasks part C: A character device kernel module (driver)===
Download the source for [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/ones.tar.gz ones], a kernel module implementing a character device that outputs an unbounded string of "1"'s. Build, compile, and run it as before.
# What kernel messages does the module generate? Does it create any new files (other than <tt>/dev/ones</tt>)? If so, where? (Hint: search in <tt>/sys</tt>)
# What happens when you "cat" the device <tt>/dev/ones</tt>? How can you limit the output in more than one way?
# Add more <tt>printk()</tt>’s (where you see needed) to find out at which point which functions in ones.c are called and for which purpose.

===Tasks part D: Getting process information from a module===
Download the source [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/newgetpid.tar.gz newgetpid.c]. Build and run it as before. Hints for the questions below:

https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h,
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/current.h,
https://elixir.bootlin.com/linux/latest/source/include/linux/cred.h
# What type is "<tt>current</tt>"? How can you figure this out?
# Modify newgetpid.c so that it creates a device file <tt>/dev/describe</tt> rather than <tt>/dev/newgetpid</tt>.
# Make <tt>/dev/describe</tt> output the calling process's parent ID (ppid), user ID (uid), group ID (gid), effective user ID (euid), and effective group ID (egid).

==Using <tt>bpftrace</tt> to monitor kernel module events==
Note: you are not required to write about your experience for this part in the submitted work, but you should still do it, which can help you understand eBPF better (see how simple but still powerful an eBPF program can be).

If you see an error like "<tt>Could not resolve symbol</tt>" when using <tt>bpftrace</tt>, you need to apply a fix by installing the dbgsym package of <tt>bpftrace</tt>: Just <tt>wget</tt> [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/install-bpftrace-dbgsym this file], and run it with

sudo apt update
source install-bpftrace-dbgsym

Download this tiny code [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/kmsnoop.bt kmsnoop.bt] and make it executable with <tt>chmod 755 kmsnoop.bt</tt>.

Now, you can open a separate SSH session and run it with <tt>sudo ./kmsnoop.bt</tt>. It will wait for events. When you do any "insmod" or "rmmod" in the other session, you will see it outputs a line of message.

If you want to avoid opening another SSH window, you can use tmux (a terminal multiplexer) by just typing tmux and hit enter.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

How could this save you from doing any <tt>printk</tt>'s in your kernel module code?
You can also <tt>sudo bpftrace -l | grep</tt> for your needed kernel functions (e.g.,<tt>printk</tt>) to hook to.

COMP3000 Operating Systems F23: Tutorial 7

2023-10-25T01:38:48Z

Abdou: /* Tasks part C: A character device kernel module (driver) */

In this tutorial you'll be learning about special files and Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface.

==Important Tips==
* There is a chance your VM will crash or your ssh session will become unresponsive when messing with the kernel. To recover, you can reboot your VM from the OpenStack web console.
* '''Definitely do not under any circumstances attempt this on your own system. This is your last warning.'''
* Remember to recompile and insert your module after making changes.
* Work incrementally and be prepared for your system to crash.
* It might be wise to code on your own computer and use scp or sshfs to transfer the files to your VM.
* https://elixir.bootlin.com/linux/latest/source is a great resource for learning about kernel functions and data structures.

Note: although it's highly recommended that you use the openstack instance, if you cannot for a reason, or if you encounter other issues, check out [[COMP3000 Operating Systems F23: Tutorial 7 Not Using SCS OpenStack | these instructions]] to see if they help.

==Special Files==
Before you start, review what you have learned about special files and think about the following questions (no need to answer them): 1) What does a special file represent and what backs it up? 2) Can you have multiple special files that are somehow "the same"? What does it mean to "copy" a special file? 3) How are named pipes different from unnamed pipes and what are their similarities? Take this opportunity to deepen your understanding of special files to prepare you for working with device drivers that support such special files.

===Tasks part A: Understanding special files===
# Try the following commands as a non-privileged user. What does each do? How do the files f1-f3 compare? How do they compare to <tt>/dev/urandom</tt>? Remember you can get output from <tt>/dev/urandom</tt> using <tt>cat</tt> or <tt>dd</tt>. If you use <tt>cat</tt>, make sure to pipe it to <tt>less</tt>!
#* <tt>cp /dev/urandom f1</tt> (hit Ctrl+C right away to avoid a huge file)
#* <tt>sudo cp -a /dev/urandom f2</tt>
#* <tt>sudo mknod f3 c 1 9</tt>
#: Note: If you see any command running for more than a few seconds, hit Ctrl+C right away, check the produced file (often it’s very large) and delete it if not needed.
# Make named pipes using <tt>mknod</tt> and <tt>mkfifo</tt>. Use them to simulate <tt>ls | wc</tt> using just the '>' and '<' operators.
# Use <tt>mknod</tt> to make a copy of your current terminal's tty - <tt>/dev/tty</tt> (Hint: consider how you did something similar with <tt>/dev/urandom</tt> above). You can name it <tt>mytty</tt>. Examine its characteristics using <tt>stty --file=mytty</tt>. Do the same for the original tty.
# Run <tt>stty --help</tt> to see what you can do with stty. Try disabling local echo. How does the shell behave with echo disabled? How can you restore echo without logging out and back in? Recall we did this in the class.

==Kernel Modules==
In this part of the tutorial you will be building and installing kernel modules. You will need root access to install kernel modules.

It is '''highly recommended''' that you use a comp3000 openstack instance for the exercises below for two reasons. First, you may have difficulties compiling kernel modules on other systems. Second, these operations are potentially dangerous, and mistakes could '''destroy all data''' on the Linux system. Consider yourself warned!

Before you start, review what you have learned about kernel modules and device drivers, and think about the following questions (no need to answer them): 1) How does the source of kernel modules differ from C programs? 2) How does building kernel modules differ from building userspace C programs? 3) How does invoking the file operations (e.g., read/write) of a special file trigger corresponding functions in a device driver?

You will learn how you can access and manipulate kernel data structures and call kernel functions in a kernel module. You will also understand how processes are represented by <tt>task_struct</tt>'s in the Linux kernel, what kind of information is stored in a task struct, and how to access the <tt>task_struct</tt> of a process.

===Tasks part B: A simple kernel module===
Download the source for this [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/simple.tar.gz simple module], unpack, and build it by typing "<tt>make</tt>". You can safely ignore the warning of "Skipping BTF generation...".
# Install the module using "<tt>sudo insmod simple.ko</tt>". The hello message is recorded in the kernel logs. How do you view the kernel logs? How many ways are there to view them?
# Check to see that the module has been loaded. How do you do this?
# Remove the module from the kernel. What did you do?

===Tasks part C: A character device kernel module (driver)===
Download the source for [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/ones.tar.gz ones], a kernel module implementing a character device that outputs an unbounded string of "1"'s. Build, compile, and run it as before.
# What kernel messages does the module generate? Does it create any new files (other than <tt>/dev/ones</tt>)? If so, where? (Hint: search in <tt>/sys</tt>)
# What happens when you "cat" the device <tt>/dev/ones</tt>? How can you limit the output in more than one way?
# Add more <tt>printk()</tt>’s (where you see needed) to find out at which point which functions in ones.c are called and for which purpose.

===Tasks part D: Getting process information from a module===
Download the source [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/newgetpid.tar.gz newgetpid.c]. Build and run it as before. Hints for the questions below:

https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h,
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/current.h,
https://elixir.bootlin.com/linux/latest/source/include/linux/cred.h
# What type is "<tt>current</tt>"? How can you figure this out?
# Modify newgetpid.c so that it creates a device file <tt>/dev/describe</tt> rather than <tt>/dev/newgetpid</tt>.
# Make <tt>/dev/describe</tt> output the calling process's parent ID (ppid), user ID (uid), group ID (gid), effective user ID (euid), and effective group ID (egid).

==Using <tt>bpftrace</tt> to monitor kernel module events==
Note: you are not required to write about your experience for this part in the submitted work, but you should still do it, which can help you understand eBPF better (see how simple but still powerful an eBPF program can be).

If you see an error like "<tt>Could not resolve symbol</tt>" when using <tt>bpftrace</tt>, you need to apply a fix by installing the dbgsym package of <tt>bpftrace</tt>: Just <tt>wget</tt> [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/install-bpftrace-dbgsym this file], and run it with

sudo apt update
source install-bpftrace-dbgsym

Download this tiny code [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/kmsnoop.bt kmsnoop.bt] and make it executable with <tt>chmod 755 kmsnoop.bt</tt>.

Now, you can open a separate SSH session and run it with <tt>sudo ./kmsnoop.bt</tt>. It will wait for events. When you do any "insmod" or "rmmod" in the other session, you will see it outputs a line of message.

If you want to avoid opening another SSH window, you can use tmux (a terminal multiplexer) by just typing tmux and hit enter.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

How could this save you from doing any <tt>printk</tt>'s in your kernel module code?
You can also <tt>sudo bpftrace -l | grep</tt> for your needed kernel functions (e.g.,<tt>printk</tt>) to hook to.

COMP3000 Operating Systems F23: Tutorial 7

2023-10-25T01:37:48Z

Abdou: /* Tasks part B: A simple kernel module */

In this tutorial you'll be learning about special files and Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface.

==Important Tips==
* There is a chance your VM will crash or your ssh session will become unresponsive when messing with the kernel. To recover, you can reboot your VM from the OpenStack web console.
* '''Definitely do not under any circumstances attempt this on your own system. This is your last warning.'''
* Remember to recompile and insert your module after making changes.
* Work incrementally and be prepared for your system to crash.
* It might be wise to code on your own computer and use scp or sshfs to transfer the files to your VM.
* https://elixir.bootlin.com/linux/latest/source is a great resource for learning about kernel functions and data structures.

Note: although it's highly recommended that you use the openstack instance, if you cannot for a reason, or if you encounter other issues, check out [[COMP3000 Operating Systems F23: Tutorial 7 Not Using SCS OpenStack | these instructions]] to see if they help.

==Special Files==
Before you start, review what you have learned about special files and think about the following questions (no need to answer them): 1) What does a special file represent and what backs it up? 2) Can you have multiple special files that are somehow "the same"? What does it mean to "copy" a special file? 3) How are named pipes different from unnamed pipes and what are their similarities? Take this opportunity to deepen your understanding of special files to prepare you for working with device drivers that support such special files.

===Tasks part A: Understanding special files===
# Try the following commands as a non-privileged user. What does each do? How do the files f1-f3 compare? How do they compare to <tt>/dev/urandom</tt>? Remember you can get output from <tt>/dev/urandom</tt> using <tt>cat</tt> or <tt>dd</tt>. If you use <tt>cat</tt>, make sure to pipe it to <tt>less</tt>!
#* <tt>cp /dev/urandom f1</tt> (hit Ctrl+C right away to avoid a huge file)
#* <tt>sudo cp -a /dev/urandom f2</tt>
#* <tt>sudo mknod f3 c 1 9</tt>
#: Note: If you see any command running for more than a few seconds, hit Ctrl+C right away, check the produced file (often it’s very large) and delete it if not needed.
# Make named pipes using <tt>mknod</tt> and <tt>mkfifo</tt>. Use them to simulate <tt>ls | wc</tt> using just the '>' and '<' operators.
# Use <tt>mknod</tt> to make a copy of your current terminal's tty - <tt>/dev/tty</tt> (Hint: consider how you did something similar with <tt>/dev/urandom</tt> above). You can name it <tt>mytty</tt>. Examine its characteristics using <tt>stty --file=mytty</tt>. Do the same for the original tty.
# Run <tt>stty --help</tt> to see what you can do with stty. Try disabling local echo. How does the shell behave with echo disabled? How can you restore echo without logging out and back in? Recall we did this in the class.

==Kernel Modules==
In this part of the tutorial you will be building and installing kernel modules. You will need root access to install kernel modules.

It is '''highly recommended''' that you use a comp3000 openstack instance for the exercises below for two reasons. First, you may have difficulties compiling kernel modules on other systems. Second, these operations are potentially dangerous, and mistakes could '''destroy all data''' on the Linux system. Consider yourself warned!

Before you start, review what you have learned about kernel modules and device drivers, and think about the following questions (no need to answer them): 1) How does the source of kernel modules differ from C programs? 2) How does building kernel modules differ from building userspace C programs? 3) How does invoking the file operations (e.g., read/write) of a special file trigger corresponding functions in a device driver?

You will learn how you can access and manipulate kernel data structures and call kernel functions in a kernel module. You will also understand how processes are represented by <tt>task_struct</tt>'s in the Linux kernel, what kind of information is stored in a task struct, and how to access the <tt>task_struct</tt> of a process.

===Tasks part B: A simple kernel module===
Download the source for this [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut7/simple.tar.gz simple module], unpack, and build it by typing "<tt>make</tt>". You can safely ignore the warning of "Skipping BTF generation...".
# Install the module using "<tt>sudo insmod simple.ko</tt>". The hello message is recorded in the kernel logs. How do you view the kernel logs? How many ways are there to view them?
# Check to see that the module has been loaded. How do you do this?
# Remove the module from the kernel. What did you do?

===Tasks part C: A character device kernel module (driver)===
Download the source for [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/ones.tar.gz ones], a kernel module implementing a character device that outputs an unbounded string of "1"'s. Build, compile, and run it as before.
# What kernel messages does the module generate? Does it create any new files (other than <tt>/dev/ones</tt>)? If so, where? (Hint: search in <tt>/sys</tt>)
# What happens when you "cat" the device <tt>/dev/ones</tt>? How can you limit the output in more than one way?
# Add more <tt>printk()</tt>’s (where you see needed) to find out at which point which functions in ones.c are called and for which purpose.

===Tasks part D: Getting process information from a module===
Download the source [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/newgetpid.tar.gz newgetpid.c]. Build and run it as before. Hints for the questions below:

https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h,
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/current.h,
https://elixir.bootlin.com/linux/latest/source/include/linux/cred.h
# What type is "<tt>current</tt>"? How can you figure this out?
# Modify newgetpid.c so that it creates a device file <tt>/dev/describe</tt> rather than <tt>/dev/newgetpid</tt>.
# Make <tt>/dev/describe</tt> output the calling process's parent ID (ppid), user ID (uid), group ID (gid), effective user ID (euid), and effective group ID (egid).

==Using <tt>bpftrace</tt> to monitor kernel module events==
Note: you are not required to write about your experience for this part in the submitted work, but you should still do it, which can help you understand eBPF better (see how simple but still powerful an eBPF program can be).

If you see an error like "<tt>Could not resolve symbol</tt>" when using <tt>bpftrace</tt>, you need to apply a fix by installing the dbgsym package of <tt>bpftrace</tt>: Just <tt>wget</tt> [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/install-bpftrace-dbgsym this file], and run it with

sudo apt update
source install-bpftrace-dbgsym

Download this tiny code [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/kmsnoop.bt kmsnoop.bt] and make it executable with <tt>chmod 755 kmsnoop.bt</tt>.

Now, you can open a separate SSH session and run it with <tt>sudo ./kmsnoop.bt</tt>. It will wait for events. When you do any "insmod" or "rmmod" in the other session, you will see it outputs a line of message.

If you want to avoid opening another SSH window, you can use tmux (a terminal multiplexer) by just typing tmux and hit enter.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

How could this save you from doing any <tt>printk</tt>'s in your kernel module code?
You can also <tt>sudo bpftrace -l | grep</tt> for your needed kernel functions (e.g.,<tt>printk</tt>) to hook to.

COMP3000 Operating Systems F23: Tutorial 7 Not Using SCS OpenStack

2023-10-25T01:37:02Z

Abdou: Created page with "If you are not using our course VM for some reason, you might encounter problems and you can refer to the instructions below. Although they are by no means comprehensve to solve the problems, they can in many cases and serve as pointers about where to look. ==Modules fail to build== If you see errors (not just warnings) after running <tt>make</tt> and no <tt>.ko</tt> file is produced, you may have installed a version of Ubuntu that is too minimal. But you can fix it by i..."

If you are not using our course VM for some reason, you might encounter problems and you can refer to the instructions below. Although they are by no means comprehensve to solve the problems, they can in many cases and serve as pointers about where to look.
==Modules fail to build==
If you see errors (not just warnings) after running <tt>make</tt> and no <tt>.ko</tt> file is produced, you may have installed a version of Ubuntu that is too minimal. But you can fix it by installing the right packages. Do the following:
sudo apt update

sudo apt dist-upgrade

sudo apt install build-essential

sudo apt clean

If your build failed before doing this and again after, delete the downloaded code and unpack the zip file again.

==Disk full==
If you find your disk space filled up, you can use the following command to pinpoint the top 20 largest files (no need to understand the specifics unless you are curious):

sudo find / -type f -exec du -h {} 2>&- + | sort -rh | head -20

Or including directories:

sudo du -h / 2>&- | sort -rh | head -20

==Command 'bpftrace' not found==
You may be able to fix this by installing the <tt>bpftrace</tt> package (just the frontend and the examples):

sudo apt-get install bpftrace

Use its equivalent on other Linux distributions.

Note: it could be much worse than this, e.g., not having the right kernel compiled with eBPF support, for which there won't be a quick fix.
In such cases, you are suggested to move to the course VM or choose a different VM of your own.

COMP3000 Operating Systems F23: Tutorial 7

2023-10-25T01:36:54Z

Abdou: /* Important Tips */

In this tutorial you'll be learning about special files and Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface.

==Important Tips==
* There is a chance your VM will crash or your ssh session will become unresponsive when messing with the kernel. To recover, you can reboot your VM from the OpenStack web console.
* '''Definitely do not under any circumstances attempt this on your own system. This is your last warning.'''
* Remember to recompile and insert your module after making changes.
* Work incrementally and be prepared for your system to crash.
* It might be wise to code on your own computer and use scp or sshfs to transfer the files to your VM.
* https://elixir.bootlin.com/linux/latest/source is a great resource for learning about kernel functions and data structures.

Note: although it's highly recommended that you use the openstack instance, if you cannot for a reason, or if you encounter other issues, check out [[COMP3000 Operating Systems F23: Tutorial 7 Not Using SCS OpenStack | these instructions]] to see if they help.

==Special Files==
Before you start, review what you have learned about special files and think about the following questions (no need to answer them): 1) What does a special file represent and what backs it up? 2) Can you have multiple special files that are somehow "the same"? What does it mean to "copy" a special file? 3) How are named pipes different from unnamed pipes and what are their similarities? Take this opportunity to deepen your understanding of special files to prepare you for working with device drivers that support such special files.

===Tasks part A: Understanding special files===
# Try the following commands as a non-privileged user. What does each do? How do the files f1-f3 compare? How do they compare to <tt>/dev/urandom</tt>? Remember you can get output from <tt>/dev/urandom</tt> using <tt>cat</tt> or <tt>dd</tt>. If you use <tt>cat</tt>, make sure to pipe it to <tt>less</tt>!
#* <tt>cp /dev/urandom f1</tt> (hit Ctrl+C right away to avoid a huge file)
#* <tt>sudo cp -a /dev/urandom f2</tt>
#* <tt>sudo mknod f3 c 1 9</tt>
#: Note: If you see any command running for more than a few seconds, hit Ctrl+C right away, check the produced file (often it’s very large) and delete it if not needed.
# Make named pipes using <tt>mknod</tt> and <tt>mkfifo</tt>. Use them to simulate <tt>ls | wc</tt> using just the '>' and '<' operators.
# Use <tt>mknod</tt> to make a copy of your current terminal's tty - <tt>/dev/tty</tt> (Hint: consider how you did something similar with <tt>/dev/urandom</tt> above). You can name it <tt>mytty</tt>. Examine its characteristics using <tt>stty --file=mytty</tt>. Do the same for the original tty.
# Run <tt>stty --help</tt> to see what you can do with stty. Try disabling local echo. How does the shell behave with echo disabled? How can you restore echo without logging out and back in? Recall we did this in the class.

==Kernel Modules==
In this part of the tutorial you will be building and installing kernel modules. You will need root access to install kernel modules.

It is '''highly recommended''' that you use a comp3000 openstack instance for the exercises below for two reasons. First, you may have difficulties compiling kernel modules on other systems. Second, these operations are potentially dangerous, and mistakes could '''destroy all data''' on the Linux system. Consider yourself warned!

Before you start, review what you have learned about kernel modules and device drivers, and think about the following questions (no need to answer them): 1) How does the source of kernel modules differ from C programs? 2) How does building kernel modules differ from building userspace C programs? 3) How does invoking the file operations (e.g., read/write) of a special file trigger corresponding functions in a device driver?

You will learn how you can access and manipulate kernel data structures and call kernel functions in a kernel module. You will also understand how processes are represented by <tt>task_struct</tt>'s in the Linux kernel, what kind of information is stored in a task struct, and how to access the <tt>task_struct</tt> of a process.

===Tasks part B: A simple kernel module===
Download the source for this [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/simple.tar.gz simple module], unpack, and build it by typing "<tt>make</tt>". You can safely ignore the warning of "Skipping BTF generation...".
# Install the module using "<tt>sudo insmod simple.ko</tt>". The hello message is recorded in the kernel logs. How do you view the kernel logs? How many ways are there to view them?
# Check to see that the module has been loaded. How do you do this?
# Remove the module from the kernel. What did you do?

===Tasks part C: A character device kernel module (driver)===
Download the source for [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/ones.tar.gz ones], a kernel module implementing a character device that outputs an unbounded string of "1"'s. Build, compile, and run it as before.
# What kernel messages does the module generate? Does it create any new files (other than <tt>/dev/ones</tt>)? If so, where? (Hint: search in <tt>/sys</tt>)
# What happens when you "cat" the device <tt>/dev/ones</tt>? How can you limit the output in more than one way?
# Add more <tt>printk()</tt>’s (where you see needed) to find out at which point which functions in ones.c are called and for which purpose.

===Tasks part D: Getting process information from a module===
Download the source [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/newgetpid.tar.gz newgetpid.c]. Build and run it as before. Hints for the questions below:

https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h,
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/current.h,
https://elixir.bootlin.com/linux/latest/source/include/linux/cred.h
# What type is "<tt>current</tt>"? How can you figure this out?
# Modify newgetpid.c so that it creates a device file <tt>/dev/describe</tt> rather than <tt>/dev/newgetpid</tt>.
# Make <tt>/dev/describe</tt> output the calling process's parent ID (ppid), user ID (uid), group ID (gid), effective user ID (euid), and effective group ID (egid).

==Using <tt>bpftrace</tt> to monitor kernel module events==
Note: you are not required to write about your experience for this part in the submitted work, but you should still do it, which can help you understand eBPF better (see how simple but still powerful an eBPF program can be).

If you see an error like "<tt>Could not resolve symbol</tt>" when using <tt>bpftrace</tt>, you need to apply a fix by installing the dbgsym package of <tt>bpftrace</tt>: Just <tt>wget</tt> [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/install-bpftrace-dbgsym this file], and run it with

sudo apt update
source install-bpftrace-dbgsym

Download this tiny code [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/kmsnoop.bt kmsnoop.bt] and make it executable with <tt>chmod 755 kmsnoop.bt</tt>.

Now, you can open a separate SSH session and run it with <tt>sudo ./kmsnoop.bt</tt>. It will wait for events. When you do any "insmod" or "rmmod" in the other session, you will see it outputs a line of message.

If you want to avoid opening another SSH window, you can use tmux (a terminal multiplexer) by just typing tmux and hit enter.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

How could this save you from doing any <tt>printk</tt>'s in your kernel module code?
You can also <tt>sudo bpftrace -l | grep</tt> for your needed kernel functions (e.g.,<tt>printk</tt>) to hook to.

COMP3000 Operating Systems F23: Tutorial 7

2023-10-25T01:31:41Z

Abdou: /* Important Tips */

In this tutorial you'll be learning about special files and Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface.

==Important Tips==
* There is a chance your VM will crash or your ssh session will become unresponsive when messing with the kernel. To recover, you can reboot your VM from the OpenStack web console.
* '''Definitely do not under any circumstances attempt this on your own system. This is your last warning.'''
* Remember to recompile and insert your module after making changes.
* Work incrementally and be prepared for your system to crash.
* It might be wise to code on your own computer and use scp or sshfs to transfer the files to your VM.
* https://elixir.bootlin.com/linux/latest/source is a great resource for learning about kernel functions and data structures.

Note: although it's highly recommended that you use the openstack instance, if you cannot for a reason, or if you encounter other issues, check out [[COMP3000 Operating Systems W23: Tutorial 7 Not Using SCS OpenStack | these instructions]] to see if they help.

==Special Files==
Before you start, review what you have learned about special files and think about the following questions (no need to answer them): 1) What does a special file represent and what backs it up? 2) Can you have multiple special files that are somehow "the same"? What does it mean to "copy" a special file? 3) How are named pipes different from unnamed pipes and what are their similarities? Take this opportunity to deepen your understanding of special files to prepare you for working with device drivers that support such special files.

===Tasks part A: Understanding special files===
# Try the following commands as a non-privileged user. What does each do? How do the files f1-f3 compare? How do they compare to <tt>/dev/urandom</tt>? Remember you can get output from <tt>/dev/urandom</tt> using <tt>cat</tt> or <tt>dd</tt>. If you use <tt>cat</tt>, make sure to pipe it to <tt>less</tt>!
#* <tt>cp /dev/urandom f1</tt> (hit Ctrl+C right away to avoid a huge file)
#* <tt>sudo cp -a /dev/urandom f2</tt>
#* <tt>sudo mknod f3 c 1 9</tt>
#: Note: If you see any command running for more than a few seconds, hit Ctrl+C right away, check the produced file (often it’s very large) and delete it if not needed.
# Make named pipes using <tt>mknod</tt> and <tt>mkfifo</tt>. Use them to simulate <tt>ls | wc</tt> using just the '>' and '<' operators.
# Use <tt>mknod</tt> to make a copy of your current terminal's tty - <tt>/dev/tty</tt> (Hint: consider how you did something similar with <tt>/dev/urandom</tt> above). You can name it <tt>mytty</tt>. Examine its characteristics using <tt>stty --file=mytty</tt>. Do the same for the original tty.
# Run <tt>stty --help</tt> to see what you can do with stty. Try disabling local echo. How does the shell behave with echo disabled? How can you restore echo without logging out and back in? Recall we did this in the class.

==Kernel Modules==
In this part of the tutorial you will be building and installing kernel modules. You will need root access to install kernel modules.

It is '''highly recommended''' that you use a comp3000 openstack instance for the exercises below for two reasons. First, you may have difficulties compiling kernel modules on other systems. Second, these operations are potentially dangerous, and mistakes could '''destroy all data''' on the Linux system. Consider yourself warned!

Before you start, review what you have learned about kernel modules and device drivers, and think about the following questions (no need to answer them): 1) How does the source of kernel modules differ from C programs? 2) How does building kernel modules differ from building userspace C programs? 3) How does invoking the file operations (e.g., read/write) of a special file trigger corresponding functions in a device driver?

You will learn how you can access and manipulate kernel data structures and call kernel functions in a kernel module. You will also understand how processes are represented by <tt>task_struct</tt>'s in the Linux kernel, what kind of information is stored in a task struct, and how to access the <tt>task_struct</tt> of a process.

===Tasks part B: A simple kernel module===
Download the source for this [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/simple.tar.gz simple module], unpack, and build it by typing "<tt>make</tt>". You can safely ignore the warning of "Skipping BTF generation...".
# Install the module using "<tt>sudo insmod simple.ko</tt>". The hello message is recorded in the kernel logs. How do you view the kernel logs? How many ways are there to view them?
# Check to see that the module has been loaded. How do you do this?
# Remove the module from the kernel. What did you do?

===Tasks part C: A character device kernel module (driver)===
Download the source for [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/ones.tar.gz ones], a kernel module implementing a character device that outputs an unbounded string of "1"'s. Build, compile, and run it as before.
# What kernel messages does the module generate? Does it create any new files (other than <tt>/dev/ones</tt>)? If so, where? (Hint: search in <tt>/sys</tt>)
# What happens when you "cat" the device <tt>/dev/ones</tt>? How can you limit the output in more than one way?
# Add more <tt>printk()</tt>’s (where you see needed) to find out at which point which functions in ones.c are called and for which purpose.

===Tasks part D: Getting process information from a module===
Download the source [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/newgetpid.tar.gz newgetpid.c]. Build and run it as before. Hints for the questions below:

https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h,
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/current.h,
https://elixir.bootlin.com/linux/latest/source/include/linux/cred.h
# What type is "<tt>current</tt>"? How can you figure this out?
# Modify newgetpid.c so that it creates a device file <tt>/dev/describe</tt> rather than <tt>/dev/newgetpid</tt>.
# Make <tt>/dev/describe</tt> output the calling process's parent ID (ppid), user ID (uid), group ID (gid), effective user ID (euid), and effective group ID (egid).

==Using <tt>bpftrace</tt> to monitor kernel module events==
Note: you are not required to write about your experience for this part in the submitted work, but you should still do it, which can help you understand eBPF better (see how simple but still powerful an eBPF program can be).

If you see an error like "<tt>Could not resolve symbol</tt>" when using <tt>bpftrace</tt>, you need to apply a fix by installing the dbgsym package of <tt>bpftrace</tt>: Just <tt>wget</tt> [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/install-bpftrace-dbgsym this file], and run it with

sudo apt update
source install-bpftrace-dbgsym

Download this tiny code [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/kmsnoop.bt kmsnoop.bt] and make it executable with <tt>chmod 755 kmsnoop.bt</tt>.

Now, you can open a separate SSH session and run it with <tt>sudo ./kmsnoop.bt</tt>. It will wait for events. When you do any "insmod" or "rmmod" in the other session, you will see it outputs a line of message.

If you want to avoid opening another SSH window, you can use tmux (a terminal multiplexer) by just typing tmux and hit enter.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

How could this save you from doing any <tt>printk</tt>'s in your kernel module code?
You can also <tt>sudo bpftrace -l | grep</tt> for your needed kernel functions (e.g.,<tt>printk</tt>) to hook to.

COMP3000 Operating Systems F23: Tutorial 7

2023-10-25T01:31:13Z

Abdou: /* General Instructions (same for all tutorials) */

In this tutorial you'll be learning about special files and Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface.

==Important Tips==
* There is a chance your VM will crash or your ssh session will become unresponsive when messing with the kernel. To recover, you can reboot your VM from the OpenStack web console.
* Definitely do not under any circumstances attempt this on your own system. This is your last warning.
* Remember to recompile and insert your module after making changes.
* Work incrementally and be prepared for your system to crash.
* It might be wise to code on your own computer and use scp or sshfs to transfer the files to your VM.
* https://elixir.bootlin.com/linux/latest/source is a great resource for learning about kernel functions and data structures.

Note: although it's highly recommended that you use the openstack instance, if you cannot for a reason, or if you encounter other issues, check out [[COMP3000 Operating Systems W23: Tutorial 7 Not Using SCS OpenStack | these instructions]] to see if they help.

==Special Files==
Before you start, review what you have learned about special files and think about the following questions (no need to answer them): 1) What does a special file represent and what backs it up? 2) Can you have multiple special files that are somehow "the same"? What does it mean to "copy" a special file? 3) How are named pipes different from unnamed pipes and what are their similarities? Take this opportunity to deepen your understanding of special files to prepare you for working with device drivers that support such special files.

===Tasks part A: Understanding special files===
# Try the following commands as a non-privileged user. What does each do? How do the files f1-f3 compare? How do they compare to <tt>/dev/urandom</tt>? Remember you can get output from <tt>/dev/urandom</tt> using <tt>cat</tt> or <tt>dd</tt>. If you use <tt>cat</tt>, make sure to pipe it to <tt>less</tt>!
#* <tt>cp /dev/urandom f1</tt> (hit Ctrl+C right away to avoid a huge file)
#* <tt>sudo cp -a /dev/urandom f2</tt>
#* <tt>sudo mknod f3 c 1 9</tt>
#: Note: If you see any command running for more than a few seconds, hit Ctrl+C right away, check the produced file (often it’s very large) and delete it if not needed.
# Make named pipes using <tt>mknod</tt> and <tt>mkfifo</tt>. Use them to simulate <tt>ls | wc</tt> using just the '>' and '<' operators.
# Use <tt>mknod</tt> to make a copy of your current terminal's tty - <tt>/dev/tty</tt> (Hint: consider how you did something similar with <tt>/dev/urandom</tt> above). You can name it <tt>mytty</tt>. Examine its characteristics using <tt>stty --file=mytty</tt>. Do the same for the original tty.
# Run <tt>stty --help</tt> to see what you can do with stty. Try disabling local echo. How does the shell behave with echo disabled? How can you restore echo without logging out and back in? Recall we did this in the class.

==Kernel Modules==
In this part of the tutorial you will be building and installing kernel modules. You will need root access to install kernel modules.

It is '''highly recommended''' that you use a comp3000 openstack instance for the exercises below for two reasons. First, you may have difficulties compiling kernel modules on other systems. Second, these operations are potentially dangerous, and mistakes could '''destroy all data''' on the Linux system. Consider yourself warned!

Before you start, review what you have learned about kernel modules and device drivers, and think about the following questions (no need to answer them): 1) How does the source of kernel modules differ from C programs? 2) How does building kernel modules differ from building userspace C programs? 3) How does invoking the file operations (e.g., read/write) of a special file trigger corresponding functions in a device driver?

You will learn how you can access and manipulate kernel data structures and call kernel functions in a kernel module. You will also understand how processes are represented by <tt>task_struct</tt>'s in the Linux kernel, what kind of information is stored in a task struct, and how to access the <tt>task_struct</tt> of a process.

===Tasks part B: A simple kernel module===
Download the source for this [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/simple.tar.gz simple module], unpack, and build it by typing "<tt>make</tt>". You can safely ignore the warning of "Skipping BTF generation...".
# Install the module using "<tt>sudo insmod simple.ko</tt>". The hello message is recorded in the kernel logs. How do you view the kernel logs? How many ways are there to view them?
# Check to see that the module has been loaded. How do you do this?
# Remove the module from the kernel. What did you do?

===Tasks part C: A character device kernel module (driver)===
Download the source for [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/ones.tar.gz ones], a kernel module implementing a character device that outputs an unbounded string of "1"'s. Build, compile, and run it as before.
# What kernel messages does the module generate? Does it create any new files (other than <tt>/dev/ones</tt>)? If so, where? (Hint: search in <tt>/sys</tt>)
# What happens when you "cat" the device <tt>/dev/ones</tt>? How can you limit the output in more than one way?
# Add more <tt>printk()</tt>’s (where you see needed) to find out at which point which functions in ones.c are called and for which purpose.

===Tasks part D: Getting process information from a module===
Download the source [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/newgetpid.tar.gz newgetpid.c]. Build and run it as before. Hints for the questions below:

https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h,
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/current.h,
https://elixir.bootlin.com/linux/latest/source/include/linux/cred.h
# What type is "<tt>current</tt>"? How can you figure this out?
# Modify newgetpid.c so that it creates a device file <tt>/dev/describe</tt> rather than <tt>/dev/newgetpid</tt>.
# Make <tt>/dev/describe</tt> output the calling process's parent ID (ppid), user ID (uid), group ID (gid), effective user ID (euid), and effective group ID (egid).

==Using <tt>bpftrace</tt> to monitor kernel module events==
Note: you are not required to write about your experience for this part in the submitted work, but you should still do it, which can help you understand eBPF better (see how simple but still powerful an eBPF program can be).

If you see an error like "<tt>Could not resolve symbol</tt>" when using <tt>bpftrace</tt>, you need to apply a fix by installing the dbgsym package of <tt>bpftrace</tt>: Just <tt>wget</tt> [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/install-bpftrace-dbgsym this file], and run it with

sudo apt update
source install-bpftrace-dbgsym

Download this tiny code [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/kmsnoop.bt kmsnoop.bt] and make it executable with <tt>chmod 755 kmsnoop.bt</tt>.

Now, you can open a separate SSH session and run it with <tt>sudo ./kmsnoop.bt</tt>. It will wait for events. When you do any "insmod" or "rmmod" in the other session, you will see it outputs a line of message.

If you want to avoid opening another SSH window, you can use tmux (a terminal multiplexer) by just typing tmux and hit enter.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

How could this save you from doing any <tt>printk</tt>'s in your kernel module code?
You can also <tt>sudo bpftrace -l | grep</tt> for your needed kernel functions (e.g.,<tt>printk</tt>) to hook to.

COMP3000 Operating Systems F23: Tutorial 7

2023-10-25T01:30:50Z

Abdou: Created page with "In this tutorial you'll be learning about special files and Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface. ==General Instructions (same for all tutorials)== <div class="mw-collapsible mw-collapsed"> Click on Expand to view. <div class="mw-collapsible-content"> Tutorials are graded based on pa..."

In this tutorial you'll be learning about special files and Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''), but you should still turn in your work. Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA (in person or by responding to a poll if online). Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Important Tips==
* There is a chance your VM will crash or your ssh session will become unresponsive when messing with the kernel. To recover, you can reboot your VM from the OpenStack web console.
* Definitely do not under any circumstances attempt this on your own system. This is your last warning.
* Remember to recompile and insert your module after making changes.
* Work incrementally and be prepared for your system to crash.
* It might be wise to code on your own computer and use scp or sshfs to transfer the files to your VM.
* https://elixir.bootlin.com/linux/latest/source is a great resource for learning about kernel functions and data structures.

Note: although it's highly recommended that you use the openstack instance, if you cannot for a reason, or if you encounter other issues, check out [[COMP3000 Operating Systems W23: Tutorial 7 Not Using SCS OpenStack | these instructions]] to see if they help.

==Special Files==
Before you start, review what you have learned about special files and think about the following questions (no need to answer them): 1) What does a special file represent and what backs it up? 2) Can you have multiple special files that are somehow "the same"? What does it mean to "copy" a special file? 3) How are named pipes different from unnamed pipes and what are their similarities? Take this opportunity to deepen your understanding of special files to prepare you for working with device drivers that support such special files.

===Tasks part A: Understanding special files===
# Try the following commands as a non-privileged user. What does each do? How do the files f1-f3 compare? How do they compare to <tt>/dev/urandom</tt>? Remember you can get output from <tt>/dev/urandom</tt> using <tt>cat</tt> or <tt>dd</tt>. If you use <tt>cat</tt>, make sure to pipe it to <tt>less</tt>!
#* <tt>cp /dev/urandom f1</tt> (hit Ctrl+C right away to avoid a huge file)
#* <tt>sudo cp -a /dev/urandom f2</tt>
#* <tt>sudo mknod f3 c 1 9</tt>
#: Note: If you see any command running for more than a few seconds, hit Ctrl+C right away, check the produced file (often it’s very large) and delete it if not needed.
# Make named pipes using <tt>mknod</tt> and <tt>mkfifo</tt>. Use them to simulate <tt>ls | wc</tt> using just the '>' and '<' operators.
# Use <tt>mknod</tt> to make a copy of your current terminal's tty - <tt>/dev/tty</tt> (Hint: consider how you did something similar with <tt>/dev/urandom</tt> above). You can name it <tt>mytty</tt>. Examine its characteristics using <tt>stty --file=mytty</tt>. Do the same for the original tty.
# Run <tt>stty --help</tt> to see what you can do with stty. Try disabling local echo. How does the shell behave with echo disabled? How can you restore echo without logging out and back in? Recall we did this in the class.

==Kernel Modules==
In this part of the tutorial you will be building and installing kernel modules. You will need root access to install kernel modules.

It is '''highly recommended''' that you use a comp3000 openstack instance for the exercises below for two reasons. First, you may have difficulties compiling kernel modules on other systems. Second, these operations are potentially dangerous, and mistakes could '''destroy all data''' on the Linux system. Consider yourself warned!

Before you start, review what you have learned about kernel modules and device drivers, and think about the following questions (no need to answer them): 1) How does the source of kernel modules differ from C programs? 2) How does building kernel modules differ from building userspace C programs? 3) How does invoking the file operations (e.g., read/write) of a special file trigger corresponding functions in a device driver?

You will learn how you can access and manipulate kernel data structures and call kernel functions in a kernel module. You will also understand how processes are represented by <tt>task_struct</tt>'s in the Linux kernel, what kind of information is stored in a task struct, and how to access the <tt>task_struct</tt> of a process.

===Tasks part B: A simple kernel module===
Download the source for this [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/simple.tar.gz simple module], unpack, and build it by typing "<tt>make</tt>". You can safely ignore the warning of "Skipping BTF generation...".
# Install the module using "<tt>sudo insmod simple.ko</tt>". The hello message is recorded in the kernel logs. How do you view the kernel logs? How many ways are there to view them?
# Check to see that the module has been loaded. How do you do this?
# Remove the module from the kernel. What did you do?

===Tasks part C: A character device kernel module (driver)===
Download the source for [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/ones.tar.gz ones], a kernel module implementing a character device that outputs an unbounded string of "1"'s. Build, compile, and run it as before.
# What kernel messages does the module generate? Does it create any new files (other than <tt>/dev/ones</tt>)? If so, where? (Hint: search in <tt>/sys</tt>)
# What happens when you "cat" the device <tt>/dev/ones</tt>? How can you limit the output in more than one way?
# Add more <tt>printk()</tt>’s (where you see needed) to find out at which point which functions in ones.c are called and for which purpose.

===Tasks part D: Getting process information from a module===
Download the source [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/newgetpid.tar.gz newgetpid.c]. Build and run it as before. Hints for the questions below:

https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h,
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/current.h,
https://elixir.bootlin.com/linux/latest/source/include/linux/cred.h
# What type is "<tt>current</tt>"? How can you figure this out?
# Modify newgetpid.c so that it creates a device file <tt>/dev/describe</tt> rather than <tt>/dev/newgetpid</tt>.
# Make <tt>/dev/describe</tt> output the calling process's parent ID (ppid), user ID (uid), group ID (gid), effective user ID (euid), and effective group ID (egid).

==Using <tt>bpftrace</tt> to monitor kernel module events==
Note: you are not required to write about your experience for this part in the submitted work, but you should still do it, which can help you understand eBPF better (see how simple but still powerful an eBPF program can be).

If you see an error like "<tt>Could not resolve symbol</tt>" when using <tt>bpftrace</tt>, you need to apply a fix by installing the dbgsym package of <tt>bpftrace</tt>: Just <tt>wget</tt> [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/install-bpftrace-dbgsym this file], and run it with

sudo apt update
source install-bpftrace-dbgsym

Download this tiny code [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut7/kmsnoop.bt kmsnoop.bt] and make it executable with <tt>chmod 755 kmsnoop.bt</tt>.

Now, you can open a separate SSH session and run it with <tt>sudo ./kmsnoop.bt</tt>. It will wait for events. When you do any "insmod" or "rmmod" in the other session, you will see it outputs a line of message.

If you want to avoid opening another SSH window, you can use tmux (a terminal multiplexer) by just typing tmux and hit enter.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

How could this save you from doing any <tt>printk</tt>'s in your kernel module code?
You can also <tt>sudo bpftrace -l | grep</tt> for your needed kernel functions (e.g.,<tt>printk</tt>) to hook to.

COMP3000 Operating Systems 2023F Tutorials

2023-10-25T01:30:40Z

Abdou: /* Tutorial Sections */

This page contains Tutorial instructions for COMP3000 A, Fall 2023.

The main page for the course can be found [https://people.scs.carleton.ca/~abdou/COMP3000_F2023.html here].

==Information about the Tutorials==
Tutorials are in person. You need to be physically in the room to finish up the tutorial instructions on your laptop and get checked-out by one of the two present TAs. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

Please stick to the tutorial session you are registered in. This avoids overcrowding.

Make well use of these tutorials to help you better understand the course material. The plan is to have 9 tutorial sessions this term, each would be worth 2.22% of your total grade. We might revisit this as the term progresses.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

== Getting Access to a Linux Machine ==

For the tutorials, you need to get access to a Linux machine. It is recommended to use an SCS Openstack instance (see instructions below). Alternatively, if you prefer to work directly on your laptop and avoid Openstack, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova this VirtualBox image] (courtesy of Prof. Zhao from Winter 2023). You will need access to a Linux system for the entire semester, ideally the same one.

=== Setting up an Openstack VM instance for the first time ===
We will use Openstack for the tutorials of this course. Follow these instructions for setting up an Openstack VM instance: [[COMP3000 Operating Systems F23: Openstack Instructions | instructions here]]. You only need to do this once for the whole term. Please use the same instance for both: the tutorials and the assignments. ''No need to create two instances''.

=== Connecting to the VM ===
You can find instructions with various ways to connect to your Openstack instance [[COMP3000 Operating Systems F23: Connecting to SCS Openstack | here]].

=== Downloading files from the VM ===
To download files from your Openstack VM instance, you can find instructions [[COMP3000 Operating Systems F23: Downloading files from your Openstack VM | here]].

=== Backing up the VM ===

To backup your VM, the image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account's home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

Reminder: ''do '''not''' take snapshots of your instance!'' Note that you cannot take snapshots of your VM anyway (it will keep trying and never succeed, and you'll create work for the tech staff who will have to undo what you did), so please don't try.

==Tutorial Sections==

* COMP 3000 A1: Tue: 08:35 - 09:55 at Loeb 720 (LA720). [Ethan and Sneha (T1-T9)]
* COMP 3000 A2: Fri: 11:35 - 12:55 at Loeb 720 (LA720). [Ethan and Ali (T1-T3), Ethan and Nareen (T4-T6), Ali and Nareen (T7-T9)]
* COMP 3000 A3: Thu: 08:35 - 09:55 at St. Patrick's 303 (SP303). [Ali and Nareen (T1-T9)]

{| class="wikitable"
|+ Tutorial Schedules
|-
! Tutorial !! Section !! Date !! Time !! Room !! TAs
|-
| rowspan="3" | [[COMP3000 Operating Systems 2023F Tutorials: Tutorial 1 | Tutorial 1]] || A1 || Sep 26 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Sep 28 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Sep 29 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 2 | Tutorial 2]] || A1 || Oct 3 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 5 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 6 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 3 | Tutorial 3]] || A1 || Oct 10 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 12 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 13 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 4 | Tutorial 4]] || A1 || Oct 31 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 2 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 3 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 5 | Tutorial 5]] || A1 || Nov 7 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 9 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 10 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 6 | Tutorial 6]] || A1 || Nov 14 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 16 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 17 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 7 | Tutorial 7]] || A1 || Nov 21 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 23 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 24 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 8 || A1 || Nov 28 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 30 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 1 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | Tutorial 9 || A1 || Dec 5 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Dec 7 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 8 || 11:35 am-12:55 pm || MC2000 || Ali and Nareen
|-
|}

COMP3000 Operating Systems 2023F Tutorials

2023-10-25T01:30:11Z

Abdou: /* Tutorial Sections */

This page contains Tutorial instructions for COMP3000 A, Fall 2023.

The main page for the course can be found [https://people.scs.carleton.ca/~abdou/COMP3000_F2023.html here].

==Information about the Tutorials==
Tutorials are in person. You need to be physically in the room to finish up the tutorial instructions on your laptop and get checked-out by one of the two present TAs. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

Please stick to the tutorial session you are registered in. This avoids overcrowding.

Make well use of these tutorials to help you better understand the course material. The plan is to have 9 tutorial sessions this term, each would be worth 2.22% of your total grade. We might revisit this as the term progresses.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

== Getting Access to a Linux Machine ==

For the tutorials, you need to get access to a Linux machine. It is recommended to use an SCS Openstack instance (see instructions below). Alternatively, if you prefer to work directly on your laptop and avoid Openstack, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova this VirtualBox image] (courtesy of Prof. Zhao from Winter 2023). You will need access to a Linux system for the entire semester, ideally the same one.

=== Setting up an Openstack VM instance for the first time ===
We will use Openstack for the tutorials of this course. Follow these instructions for setting up an Openstack VM instance: [[COMP3000 Operating Systems F23: Openstack Instructions | instructions here]]. You only need to do this once for the whole term. Please use the same instance for both: the tutorials and the assignments. ''No need to create two instances''.

=== Connecting to the VM ===
You can find instructions with various ways to connect to your Openstack instance [[COMP3000 Operating Systems F23: Connecting to SCS Openstack | here]].

=== Downloading files from the VM ===
To download files from your Openstack VM instance, you can find instructions [[COMP3000 Operating Systems F23: Downloading files from your Openstack VM | here]].

=== Backing up the VM ===

To backup your VM, the image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account's home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

Reminder: ''do '''not''' take snapshots of your instance!'' Note that you cannot take snapshots of your VM anyway (it will keep trying and never succeed, and you'll create work for the tech staff who will have to undo what you did), so please don't try.

==Tutorial Sections==

* COMP 3000 A1: Tue: 08:35 - 09:55 at Loeb 720 (LA720). [Ethan and Sneha (T1-T9)]
* COMP 3000 A2: Fri: 11:35 - 12:55 at Loeb 720 (LA720). [Ethan and Ali (T1-T3), Ethan and Nareen (T4-T6), Ali and Nareen (T7-T9)]
* COMP 3000 A3: Thu: 08:35 - 09:55 at St. Patrick's 303 (SP303). [Ali and Nareen (T1-T9)]

{| class="wikitable"
|+ Tutorial Schedules
|-
! Tutorial !! Section !! Date !! Time !! Room !! TAs
|-
| rowspan="3" | [[COMP3000 Operating Systems 2023F Tutorials: Tutorial 1 | Tutorial 1]] || A1 || Sep 26 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Sep 28 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Sep 29 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 2 | Tutorial 2]] || A1 || Oct 3 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 5 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 6 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 3 | Tutorial 3]] || A1 || Oct 10 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 12 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 13 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 4 | Tutorial 4]] || A1 || Oct 31 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 2 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 3 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 5 | Tutorial 5]] || A1 || Nov 7 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 9 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 10 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 6 | Tutorial 6]] || A1 || Nov 14 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 16 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 17 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems W23: Tutorial 7 | Tutorial 7]] || A1 || Nov 21 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 23 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 24 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 8 || A1 || Nov 28 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 30 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 1 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | Tutorial 9 || A1 || Dec 5 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Dec 7 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 8 || 11:35 am-12:55 pm || MC2000 || Ali and Nareen
|-
|}

COMP3000 Operating Systems F23: Tutorial 4

2023-10-25T01:14:57Z

Abdou:

In this tutorial, you will learn about how user accounts and logging in work through exploring [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin.c]. You’ll also have a better understanding of the permission system and the shell/terminal environment.

==Logging in to a Linux system==
As mentioned in the lecture, in order to log in to a Linux-like system (including UNIX), the following steps must occur (potentially not in this order).
# The user must use a terminal to connect to the system, locally or remotely (e.g., initiated by <code>/usr/sbin/getty</code>)
# The user must authenticate themselves, proving an identify with authorized access to the system. By default, this is done through a username and password (e.g., by exec’ing <code>/usr/bin/login</code>).
# After authentication, login changes uid and gid to that of the new user.
# Login sets up other aspects of the user's context (mainly setting key environment variables).
# Login does an exec of the user's chosen/default shell.

[https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin] is a basic implementation of steps 3-5. (Steps 1 and 2 are accomplished by running 3000userlogin in a shell, as running an external command means the shell also creates a new process for it, assuming an authenticated session with a terminal.)

When 3000userlogin is properly compiled and set up, you can run:

./3000userlogin student

and you'll be logged in (again) as "student". This only works when you’re already logged in as student (see below).

You can add a user with the <code>adduser</code> command. For example, to create the user "someuser":

sudo adduser someuser

Note you'll have to answer several questions. Then you need to switch to someuser (<code>su someuser</code>).

If you just compile 3000userlogin normally, you won't be able to log in as anyone except the current user (be it student or someuser). To set up 3000userlogin properly, do the following:

sudo chown root:root 3000userlogin
sudo chmod u+s 3000userlogin

Feel free to read the <code>man</code> pages for both commands to learn more.

The <code>chown</code> command makes the binary owned by root, and the <code>chmod</code> command makes it setuid. Thus, when the program is exec'd it will have an effective user ID of root (euid=0). To facilitate the following tasks, you may also want to create different versions of 3000userlogin by naming them, e.g., 3000userlogin.setuid and 3000userlogin.orig, of your choice.

==Tasks/Questions==
# Compile and setup [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin] as described above. Create a new user account. Verify that you can use 3000userlogin to login as the new user without typing the password of the new user (without logging in beforehand). Think about why you are able to avoid typing the password and still get authenticated.
# Why is the syntax highlighting gone (coloring characters)? What simple change (one line) can you make to have 3000userlogin preserve the colors? (Hint: check the environment variables)
# What is returned as the user's password by <code>getpwnam()</code> (in terms of type)? Is this what you expected? Read the manual page as you find necessary.
# Compare the uid, gid, euid, egid when running 3000userlogin as a regular user (this means without the “chown” and “chmod” commands above), running it setuid root, and running it as root (i.e., running it from a root shell without the setuid bit set, e.g., “<code>sudo ./3000userlogin someuser</code>”). Also check to see what happens when you set the setgid bit.
# Why does 3000userlogin change its gid before changing its uid? What happens if you switch the order of these operations?
# Make 3000userlogin use the shell that is specified in the user's password entry (which is actually read from <code>/etc/passwd</code>). Check by setting someuser’s shell to a new shell and then see if that new shell runs when you run 3000userlogin. You can change a user's shell with the <code>chsh</code> command.
# Can you set 3000shell to be a user's default shell? What changes do you have to make for <code>chsh</code> to accept 3000shell? Does anything obvious break when running 3000shell this way, and how can you change 3000userlogin to fix it?
# Does a user's default shell have to be a regular shell? Could it instead be an arbitrary program? How do you know?
# Are the environment variables set by 3000userlogin the only environment variables that are set (visible) after you successfully login? How do you know?
# Note that 3000userlogin uses <code>environ</code> not <code>envp</code> (as an argument to <code>main()</code>) to access environment variables. Why not use <code>envp</code>? (Try changing the code to use <code>envp</code> and see what happens: worse or better?)

==Optional/advanced tasks==
# Prompt for the user's password before logging the user in (Step 2 above). To do this, you'll need to get the right password hash and then figure out how to hash the entered password to check if it matches. Recall the format of <code>/etc/shadow</code> and you may need to learn how to use the <code>openssl passwd</code> command.

COMP3000 Operating Systems F23: Tutorial 5

2023-10-25T01:12:45Z

Abdou:

In this tutorial you will be learning about files and filesystems by experimenting with and extending [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c], and creating and manipulating local filesystems. '''WARNING:''' Several of the commands here can lead to system corruption and data loss if not properly used. You have been warned. Please use a VM and make backups, when necessary.

==Files and inodes==
In UNIX/Linux filesystems, a filename does not directly refer to the contents of a file. Instead, a filename refers to an inode (as specified in a directory entry --- dentry, of its parent directory), and the inode then refers to the data. An inode is where file metadata is stored. File ownership, timestamps, and permissions are all stored in a file's inode. Regular files are just '''hard links''' connecting a file name to an inode.

In addition to regular files, we also have symbolic links, directories, block devices, character devices, and pipes. Each is its own type of inode.

Note that while you can find out a file's inode, you cannot go from an inode to a pathname or otherwise manipulate an inode directly from userspace - you always have to go through the pathname. The kernel, however, can access individual inodes directly (indeed it has to in order to get to the contents of a file when given a filename).

3000test.c uses stat() to give information on a given inode.

==Tasks/Questions (Part A)==
# Compile and run [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c]. It takes a filename as an argument and reports information on the file. Try giving it the following and see what it reports:
#* a regular text file that exists
#* a directory
#* a symbolic link
#* a device file (character or block)
# Change 3000test to use <code>lstat()</code> rather than stat. How does its behavior change?
# Modify 3000test so when it is given a symbolic link it reports the name of the target. Use <code>readlink(2)</code> (this notation describes how to access the man page, e.g., <code>man 2 readlink</code>).
# Are there files or directories that you cannot run 3000test on? Can you configure file/directory permissions so as to make something inaccessible to 3000test? Note that this is twofold: 1) whether it is completely inaccessible to 3000test ('''nothing can be displayed''', except the error msg); 2) whether it can be accessed for the search ('''no access to file content''').

==Creating, Mounting, and Unmounting Filesystem==
In UNIX/Linux systems we always have a single hierarchical namespace for files and directories. Pathnames that start with a <code>/</code> are absolute, starting at the top of the namespace, while ones that do not start with a / are relative to the current directory.

The files in this single namespace come from many sources, each of which can have very different semantics. These sources are known as '''filesystems'''.

When first started, a system must have at least one filesystem. This first filesystem is known as the root filesystem and has the contents of <code>/</code>. We can then add other filesystems to this initial set of files by '''mounting''' each filesystem on an existing directory.

On Ubuntu 22.04, the root filesystem is ext4 by default and many others are mounted on top of it. For example, a proc filesystem is mounted on <code>/proc</code>, a sysfs is mounted on <code>/sys</code>. If you run "mount" with no arguments, it lists all the currently mounted filesystems. Note that most of these are virtual filesystems, in that they do not correspond to any persistent storage (like a disk).

If you insert a USB stick on a laptop or desktop running Linux, it must be mounted before you can access its contents. The current convention is for it to be mounted in <code>/media/</code><mounting user><code>/</code><name of USB stick filesystem>. This usually happens automatically nowadays when the USB stick is plugged in.

To create an ext4 filesystem, you just run

mkfs.ext4 <writable block device> #Do not try it with any <code>/dev/*</code>

If you were to run this on the root filesystem of your current system, it would reformat your entire disk if it was allowed to complete. In all likelihood the system should prevent you from doing this as the root filesystem (and any mounted filesystem) gets special protections.

To mount a filesystem, you specify the filesystem to be mounted (by its type or by the device) and the mountpoint:

mount /dev/sdb1 /mnt #Do not try it unless you have that block device and want to do it

This would mount the first partition of the second disk on <code>/mnt</code>.

Filesystems can be stored on block devices - devices whose contents are accessed by specifying a block index/address. The other main type of device file, character devices, are used for devices where the data is accessed a single byte (character) at a time. Terminals, modems, printers – for example, will generally be represented as a character device.

Note that mount here can identify the filesystem on <code>/dev/sdb1</code> by looking at its superblock. The superblock is usually either in block 0 or 1 of a device. The superblock holds metadata of a filesystem, allowing it to be identified and specified. Without the superblock you cannot do anything with a filesystem. Because of its importance, there are backup superblocks in addition to the primary one. The blocks of a filesystem can be classified as either being superblocks, inode blocks, or data blocks.

Sometimes we want to play with a new filesystem but we don't have a physical device to format. If we have free space for a file in our current filesystem, however, we can put a filesystem in a file by using a loopback block device. A loopback block device is a block device where its data is stored in a regular file on another filesystem (rather than on a separate device). If you do filesystem commands with a regular file, it will transparently associate the file with an available loopback block device (e.g., <code>/dev/loop1</code>).

Note that inode numbers are filesystem-specific. Thus, the contents of a file are uniquely specified by its filesystem and inode. Only the kernel has to worry about this level of detail; in userspace, a file's full path contains all the necessary information.

Filesystems on persistent storage are always at risk of corruption. File system checker programs (<code>fsck</code>) can detect and repair filesystem errors.

==Tasks/Questions (Part B)==
# Run <code>dd if=/dev/zero of=foo bs=8192 count=32K</code> and use the <code>ls</code> command to check it. What is the logical size of the file foo? How much space does it consume on disk? (Hint: Look at the size option to ls). In comparison, create another file with <code>dd if=/dev/zero of=foo2 bs=8192 seek=31K count=1K</code>. Any observations?
# Run <code>mkfs.ext4 foo</code>. Does foo consume any more space or less? Do the same (<code>mkfs.ext4 foo2</code>) to the other file and answer the same question.
# Create any file in <code>/mnt</code> (e.g., <code>sudo touch test.txt</code>) and run <code>sudo mount foo /mnt</code>. Do you still see the file you just created?
# Run <code>df</code>. What device is mounted on <code>/mnt</code>? What is this device?
# Run <code>sudo umount /mnt</code>. What have gone away and what is back?
# Run <code>dd if=/dev/zero of=foo conv=notrunc count=10 bs=512</code>. How does the "<code>conv=notrunc</code>" change dd's behavior (versus the command in question 1)?
# Run <code>sudo mount foo /mnt</code>. What error do you get?
# What command can you run to make foo mountable again? What characteristic of the file system enables this command to work? (Hint: revisit the instructions of this tutorial if you have no idea)

COMP3000 Operating Systems F23: Tutorial 5

2023-10-25T01:12:35Z

Abdou:

COMP3000 Operating Systems F23: Tutorial 5

2023-10-25T01:12:05Z

Abdou: /* Tasks/Questions (Part A) */

In this tutorial you will be learning about files and filesystems by experimenting with and extending [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c], and creating and manipulating local filesystems. '''WARNING:''' Several of the commands here can lead to system corruption and data loss if not properly used. You have been warned. Please use a VM and make backups, when necessary.

'''Note: there are two parts of Tasks/Questions, Part A and Part B'''

==Files and inodes==
In UNIX/Linux filesystems, a filename does not directly refer to the contents of a file. Instead, a filename refers to an inode (as specified in a directory entry --- dentry, of its parent directory), and the inode then refers to the data. An inode is where file metadata is stored. File ownership, timestamps, and permissions are all stored in a file's inode. Regular files are just '''hard links''' connecting a file name to an inode.

In addition to regular files, we also have symbolic links, directories, block devices, character devices, and pipes. Each is its own type of inode.

Note that while you can find out a file's inode, you cannot go from an inode to a pathname or otherwise manipulate an inode directly from userspace - you always have to go through the pathname. The kernel, however, can access individual inodes directly (indeed it has to in order to get to the contents of a file when given a filename).

3000test.c uses stat() to give information on a given inode.

==Tasks/Questions (Part A)==
# Compile and run [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c]. It takes a filename as an argument and reports information on the file. Try giving it the following and see what it reports:
#* a regular text file that exists
#* a directory
#* a symbolic link
#* a device file (character or block)
# Change 3000test to use <code>lstat()</code> rather than stat. How does its behavior change?
# Modify 3000test so when it is given a symbolic link it reports the name of the target. Use <code>readlink(2)</code> (this notation describes how to access the man page, e.g., <code>man 2 readlink</code>).
# Are there files or directories that you cannot run 3000test on? Can you configure file/directory permissions so as to make something inaccessible to 3000test? Note that this is twofold: 1) whether it is completely inaccessible to 3000test ('''nothing can be displayed''', except the error msg); 2) whether it can be accessed for the search ('''no access to file content''').

==Creating, Mounting, and Unmounting Filesystem==
In UNIX/Linux systems we always have a single hierarchical namespace for files and directories. Pathnames that start with a <code>/</code> are absolute, starting at the top of the namespace, while ones that do not start with a / are relative to the current directory.

The files in this single namespace come from many sources, each of which can have very different semantics. These sources are known as '''filesystems'''.

When first started, a system must have at least one filesystem. This first filesystem is known as the root filesystem and has the contents of <code>/</code>. We can then add other filesystems to this initial set of files by '''mounting''' each filesystem on an existing directory.

On Ubuntu 22.04, the root filesystem is ext4 by default and many others are mounted on top of it. For example, a proc filesystem is mounted on <code>/proc</code>, a sysfs is mounted on <code>/sys</code>. If you run "mount" with no arguments, it lists all the currently mounted filesystems. Note that most of these are virtual filesystems, in that they do not correspond to any persistent storage (like a disk).

If you insert a USB stick on a laptop or desktop running Linux, it must be mounted before you can access its contents. The current convention is for it to be mounted in <code>/media/</code><mounting user><code>/</code><name of USB stick filesystem>. This usually happens automatically nowadays when the USB stick is plugged in.

To create an ext4 filesystem, you just run

mkfs.ext4 <writable block device> #Do not try it with any <code>/dev/*</code>

If you were to run this on the root filesystem of your current system, it would reformat your entire disk if it was allowed to complete. In all likelihood the system should prevent you from doing this as the root filesystem (and any mounted filesystem) gets special protections.

To mount a filesystem, you specify the filesystem to be mounted (by its type or by the device) and the mountpoint:

mount /dev/sdb1 /mnt #Do not try it unless you have that block device and want to do it

This would mount the first partition of the second disk on <code>/mnt</code>.

Filesystems can be stored on block devices - devices whose contents are accessed by specifying a block index/address. The other main type of device file, character devices, are used for devices where the data is accessed a single byte (character) at a time. Terminals, modems, printers – for example, will generally be represented as a character device.

Note that mount here can identify the filesystem on <code>/dev/sdb1</code> by looking at its superblock. The superblock is usually either in block 0 or 1 of a device. The superblock holds metadata of a filesystem, allowing it to be identified and specified. Without the superblock you cannot do anything with a filesystem. Because of its importance, there are backup superblocks in addition to the primary one. The blocks of a filesystem can be classified as either being superblocks, inode blocks, or data blocks.

Sometimes we want to play with a new filesystem but we don't have a physical device to format. If we have free space for a file in our current filesystem, however, we can put a filesystem in a file by using a loopback block device. A loopback block device is a block device where its data is stored in a regular file on another filesystem (rather than on a separate device). If you do filesystem commands with a regular file, it will transparently associate the file with an available loopback block device (e.g., <code>/dev/loop1</code>).

Note that inode numbers are filesystem-specific. Thus, the contents of a file are uniquely specified by its filesystem and inode. Only the kernel has to worry about this level of detail; in userspace, a file's full path contains all the necessary information.

Filesystems on persistent storage are always at risk of corruption. File system checker programs (<code>fsck</code>) can detect and repair filesystem errors.

==Tasks/Questions (Part B)==
# Run <code>dd if=/dev/zero of=foo bs=8192 count=32K</code> and use the <code>ls</code> command to check it. What is the logical size of the file foo? How much space does it consume on disk? (Hint: Look at the size option to ls). In comparison, create another file with <code>dd if=/dev/zero of=foo2 bs=8192 seek=31K count=1K</code>. Any observations?
# Run <code>mkfs.ext4 foo</code>. Does foo consume any more space or less? Do the same (<code>mkfs.ext4 foo2</code>) to the other file and answer the same question.
# Create any file in <code>/mnt</code> (e.g., <code>sudo touch test.txt</code>) and run <code>sudo mount foo /mnt</code>. Do you still see the file you just created?
# Run <code>df</code>. What device is mounted on <code>/mnt</code>? What is this device?
# Run <code>sudo umount /mnt</code>. What have gone away and what is back?
# Run <code>dd if=/dev/zero of=foo conv=notrunc count=10 bs=512</code>. How does the "<code>conv=notrunc</code>" change dd's behavior (versus the command in question 1)?
# Run <code>sudo mount foo /mnt</code>. What error do you get?
# What command can you run to make foo mountable again? What characteristic of the file system enables this command to work? (Hint: revisit the instructions of this tutorial if you have no idea)

COMP3000 Operating Systems F23: Tutorial 5

2023-10-25T01:11:36Z

Abdou: /* Tasks/Questions (Part B) */

In this tutorial you will be learning about files and filesystems by experimenting with and extending [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c], and creating and manipulating local filesystems. '''WARNING:''' Several of the commands here can lead to system corruption and data loss if not properly used. You have been warned. Please use a VM and make backups, when necessary.

'''Note: there are two parts of Tasks/Questions, Part A and Part B'''

==Files and inodes==
In UNIX/Linux filesystems, a filename does not directly refer to the contents of a file. Instead, a filename refers to an inode (as specified in a directory entry --- dentry, of its parent directory), and the inode then refers to the data. An inode is where file metadata is stored. File ownership, timestamps, and permissions are all stored in a file's inode. Regular files are just '''hard links''' connecting a file name to an inode.

In addition to regular files, we also have symbolic links, directories, block devices, character devices, and pipes. Each is its own type of inode.

Note that while you can find out a file's inode, you cannot go from an inode to a pathname or otherwise manipulate an inode directly from userspace - you always have to go through the pathname. The kernel, however, can access individual inodes directly (indeed it has to in order to get to the contents of a file when given a filename).

3000test.c uses stat() to give information on a given inode.

==Tasks/Questions (Part A)==
# Compile and run [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c]. It takes a filename as an argument and reports information on the file. Try giving it the following and see what it reports:
#* a regular text file that exists
#* a directory
#* a symbolic link
#* a device file (character or block)
# Change 3000test to use <tt>lstat()</tt> rather than stat. How does its behavior change?
# Modify 3000test so when it is given a symbolic link it reports the name of the target. Use <tt>readlink(2)</tt> (this notation describes how to access the man page, e.g., <tt>man 2 readlink</tt>).
# Are there files or directories that you cannot run 3000test on? Can you configure file/directory permissions so as to make something inaccessible to 3000test? Note that this is twofold: 1) whether it is completely inaccessible to 3000test ('''nothing can be displayed''', except the error msg); 2) whether it can be accessed for the search ('''no access to file content''').

==Creating, Mounting, and Unmounting Filesystem==
In UNIX/Linux systems we always have a single hierarchical namespace for files and directories. Pathnames that start with a <code>/</code> are absolute, starting at the top of the namespace, while ones that do not start with a / are relative to the current directory.

The files in this single namespace come from many sources, each of which can have very different semantics. These sources are known as '''filesystems'''.

When first started, a system must have at least one filesystem. This first filesystem is known as the root filesystem and has the contents of <code>/</code>. We can then add other filesystems to this initial set of files by '''mounting''' each filesystem on an existing directory.

On Ubuntu 22.04, the root filesystem is ext4 by default and many others are mounted on top of it. For example, a proc filesystem is mounted on <code>/proc</code>, a sysfs is mounted on <code>/sys</code>. If you run "mount" with no arguments, it lists all the currently mounted filesystems. Note that most of these are virtual filesystems, in that they do not correspond to any persistent storage (like a disk).

If you insert a USB stick on a laptop or desktop running Linux, it must be mounted before you can access its contents. The current convention is for it to be mounted in <code>/media/</code><mounting user><code>/</code><name of USB stick filesystem>. This usually happens automatically nowadays when the USB stick is plugged in.

To create an ext4 filesystem, you just run

mkfs.ext4 <writable block device> #Do not try it with any <code>/dev/*</code>

If you were to run this on the root filesystem of your current system, it would reformat your entire disk if it was allowed to complete. In all likelihood the system should prevent you from doing this as the root filesystem (and any mounted filesystem) gets special protections.

To mount a filesystem, you specify the filesystem to be mounted (by its type or by the device) and the mountpoint:

mount /dev/sdb1 /mnt #Do not try it unless you have that block device and want to do it

This would mount the first partition of the second disk on <code>/mnt</code>.

Filesystems can be stored on block devices - devices whose contents are accessed by specifying a block index/address. The other main type of device file, character devices, are used for devices where the data is accessed a single byte (character) at a time. Terminals, modems, printers – for example, will generally be represented as a character device.

Note that mount here can identify the filesystem on <code>/dev/sdb1</code> by looking at its superblock. The superblock is usually either in block 0 or 1 of a device. The superblock holds metadata of a filesystem, allowing it to be identified and specified. Without the superblock you cannot do anything with a filesystem. Because of its importance, there are backup superblocks in addition to the primary one. The blocks of a filesystem can be classified as either being superblocks, inode blocks, or data blocks.

Sometimes we want to play with a new filesystem but we don't have a physical device to format. If we have free space for a file in our current filesystem, however, we can put a filesystem in a file by using a loopback block device. A loopback block device is a block device where its data is stored in a regular file on another filesystem (rather than on a separate device). If you do filesystem commands with a regular file, it will transparently associate the file with an available loopback block device (e.g., <code>/dev/loop1</code>).

Note that inode numbers are filesystem-specific. Thus, the contents of a file are uniquely specified by its filesystem and inode. Only the kernel has to worry about this level of detail; in userspace, a file's full path contains all the necessary information.

Filesystems on persistent storage are always at risk of corruption. File system checker programs (<code>fsck</code>) can detect and repair filesystem errors.

==Tasks/Questions (Part B)==
# Run <code>dd if=/dev/zero of=foo bs=8192 count=32K</code> and use the <code>ls</code> command to check it. What is the logical size of the file foo? How much space does it consume on disk? (Hint: Look at the size option to ls). In comparison, create another file with <code>dd if=/dev/zero of=foo2 bs=8192 seek=31K count=1K</code>. Any observations?
# Run <code>mkfs.ext4 foo</code>. Does foo consume any more space or less? Do the same (<code>mkfs.ext4 foo2</code>) to the other file and answer the same question.
# Create any file in <code>/mnt</code> (e.g., <code>sudo touch test.txt</code>) and run <code>sudo mount foo /mnt</code>. Do you still see the file you just created?
# Run <code>df</code>. What device is mounted on <code>/mnt</code>? What is this device?
# Run <code>sudo umount /mnt</code>. What have gone away and what is back?
# Run <code>dd if=/dev/zero of=foo conv=notrunc count=10 bs=512</code>. How does the "<code>conv=notrunc</code>" change dd's behavior (versus the command in question 1)?
# Run <code>sudo mount foo /mnt</code>. What error do you get?
# What command can you run to make foo mountable again? What characteristic of the file system enables this command to work? (Hint: revisit the instructions of this tutorial if you have no idea)

COMP3000 Operating Systems F23: Tutorial 5

2023-10-25T01:10:50Z

Abdou: /* Creating, Mounting, and Unmounting Filesystem */

In this tutorial you will be learning about files and filesystems by experimenting with and extending [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c], and creating and manipulating local filesystems. '''WARNING:''' Several of the commands here can lead to system corruption and data loss if not properly used. You have been warned. Please use a VM and make backups, when necessary.

'''Note: there are two parts of Tasks/Questions, Part A and Part B'''

==Files and inodes==
In UNIX/Linux filesystems, a filename does not directly refer to the contents of a file. Instead, a filename refers to an inode (as specified in a directory entry --- dentry, of its parent directory), and the inode then refers to the data. An inode is where file metadata is stored. File ownership, timestamps, and permissions are all stored in a file's inode. Regular files are just '''hard links''' connecting a file name to an inode.

In addition to regular files, we also have symbolic links, directories, block devices, character devices, and pipes. Each is its own type of inode.

Note that while you can find out a file's inode, you cannot go from an inode to a pathname or otherwise manipulate an inode directly from userspace - you always have to go through the pathname. The kernel, however, can access individual inodes directly (indeed it has to in order to get to the contents of a file when given a filename).

3000test.c uses stat() to give information on a given inode.

==Tasks/Questions (Part A)==
# Compile and run [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c]. It takes a filename as an argument and reports information on the file. Try giving it the following and see what it reports:
#* a regular text file that exists
#* a directory
#* a symbolic link
#* a device file (character or block)
# Change 3000test to use <tt>lstat()</tt> rather than stat. How does its behavior change?
# Modify 3000test so when it is given a symbolic link it reports the name of the target. Use <tt>readlink(2)</tt> (this notation describes how to access the man page, e.g., <tt>man 2 readlink</tt>).
# Are there files or directories that you cannot run 3000test on? Can you configure file/directory permissions so as to make something inaccessible to 3000test? Note that this is twofold: 1) whether it is completely inaccessible to 3000test ('''nothing can be displayed''', except the error msg); 2) whether it can be accessed for the search ('''no access to file content''').

==Creating, Mounting, and Unmounting Filesystem==
In UNIX/Linux systems we always have a single hierarchical namespace for files and directories. Pathnames that start with a <code>/</code> are absolute, starting at the top of the namespace, while ones that do not start with a / are relative to the current directory.

The files in this single namespace come from many sources, each of which can have very different semantics. These sources are known as '''filesystems'''.

When first started, a system must have at least one filesystem. This first filesystem is known as the root filesystem and has the contents of <code>/</code>. We can then add other filesystems to this initial set of files by '''mounting''' each filesystem on an existing directory.

On Ubuntu 22.04, the root filesystem is ext4 by default and many others are mounted on top of it. For example, a proc filesystem is mounted on <code>/proc</code>, a sysfs is mounted on <code>/sys</code>. If you run "mount" with no arguments, it lists all the currently mounted filesystems. Note that most of these are virtual filesystems, in that they do not correspond to any persistent storage (like a disk).

If you insert a USB stick on a laptop or desktop running Linux, it must be mounted before you can access its contents. The current convention is for it to be mounted in <code>/media/</code><mounting user><code>/</code><name of USB stick filesystem>. This usually happens automatically nowadays when the USB stick is plugged in.

To create an ext4 filesystem, you just run

mkfs.ext4 <writable block device> #Do not try it with any <code>/dev/*</code>

If you were to run this on the root filesystem of your current system, it would reformat your entire disk if it was allowed to complete. In all likelihood the system should prevent you from doing this as the root filesystem (and any mounted filesystem) gets special protections.

To mount a filesystem, you specify the filesystem to be mounted (by its type or by the device) and the mountpoint:

mount /dev/sdb1 /mnt #Do not try it unless you have that block device and want to do it

This would mount the first partition of the second disk on <code>/mnt</code>.

Filesystems can be stored on block devices - devices whose contents are accessed by specifying a block index/address. The other main type of device file, character devices, are used for devices where the data is accessed a single byte (character) at a time. Terminals, modems, printers – for example, will generally be represented as a character device.

Note that mount here can identify the filesystem on <code>/dev/sdb1</code> by looking at its superblock. The superblock is usually either in block 0 or 1 of a device. The superblock holds metadata of a filesystem, allowing it to be identified and specified. Without the superblock you cannot do anything with a filesystem. Because of its importance, there are backup superblocks in addition to the primary one. The blocks of a filesystem can be classified as either being superblocks, inode blocks, or data blocks.

Sometimes we want to play with a new filesystem but we don't have a physical device to format. If we have free space for a file in our current filesystem, however, we can put a filesystem in a file by using a loopback block device. A loopback block device is a block device where its data is stored in a regular file on another filesystem (rather than on a separate device). If you do filesystem commands with a regular file, it will transparently associate the file with an available loopback block device (e.g., <code>/dev/loop1</code>).

Note that inode numbers are filesystem-specific. Thus, the contents of a file are uniquely specified by its filesystem and inode. Only the kernel has to worry about this level of detail; in userspace, a file's full path contains all the necessary information.

Filesystems on persistent storage are always at risk of corruption. File system checker programs (<code>fsck</code>) can detect and repair filesystem errors.

==Tasks/Questions (Part B)==
# Run <tt>dd if=/dev/zero of=foo bs=8192 count=32K</tt> and use the <tt>ls</tt> command to check it. What is the logical size of the file foo? How much space does it consume on disk? (Hint: Look at the size option to ls). In comparison, create another file with <tt>dd if=/dev/zero of=foo2 bs=8192 seek=31K count=1K</tt>. Any observations?
# Run <tt>mkfs.ext4 foo</tt>. Does foo consume any more space or less? Do the same (<tt>mkfs.ext4 foo2</tt>) to the other file and answer the same question.
# Create any file in <tt>/mnt</tt> (e.g., <tt>sudo touch test.txt</tt>) and run <tt>sudo mount foo /mnt</tt>. Do you still see the file you just created?
# Run <tt>df</tt>. What device is mounted on <tt>/mnt</tt>? What is this device?
# Run <tt>sudo umount /mnt</tt>. What have gone away and what is back?
# Run <tt>dd if=/dev/zero of=foo conv=notrunc count=10 bs=512</tt>. How does the "<tt>conv=notrunc</tt>" change dd's behavior (versus the command in question 1)?
# Run <tt>sudo mount foo /mnt</tt>. What error do you get?
# What command can you run to make foo mountable again? What characteristic of the file system enables this command to work? (Hint: revisit the instructions of this tutorial if you have no idea)

COMP3000 Operating Systems F23: Tutorial 6

2023-10-25T01:09:18Z

Abdou: /* C: Producer/Consumer with Shared Memory */

In this tutorial you will be learning about two implementations of the [https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem producer-consumer problem], a classic example of a concurrency problem. The [http://pages.cs.wisc.edu/~remzi/OSTEP/ class textbook] covers concurrency in great detail in Chapters 25-34, and the producer-consumer problem is covered in [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf Chapter 30 (Condition Variables)] and [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-sema.pdf Chapter 31 (Semaphores)]. While you can look at this part of the textbook, note that we will not be covering this material in the same level of detail, as should be clear from this tutorial.

==A: Getting Started==
Download [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut6/3000pc.zip 3000pc.zip], unpack, and run <code>make</code> to compile <code>3000pc-fifo</code> and <code>3000pc-rendezvous</code>.

Try to take this opportunity to read and understand the two programs line by line.

:: Note that these programs take 3 arguments each:
::* The number of events to process
::* The number of events to produce before the producer sleeps for 1 second
::* The number of events to consume before the consumer sleeps for 1 second

# Run both programs with the same arguments of your choice. Do this a few times with different arguments. Do they behave the same way? Do you notice any differences? (again, no “correct” answers but just play with them and document your observations)
# Repeat the above experiment, this time running each program under <code>strace</code> (with the <code>-f</code> flag to trace children too). Do you notice any difference in the system calls each program makes? Remember to ignore the lines before <code>main()</code> that might be distracting, e.g., start looking around and after the <code>clone()</code> system call.
# You can see there is "<code>#define QUEUESIZE 32</code>" in both programs. Are they actually used? Why (not)?

==B: Producer/Consumer with Pipes==
<ol style="list-style-type:decimal">
<li> Examine the source code of <code>3000pc-fifo</code>. Pay attention to the call to <code>pipe(pipefd)</code> on line 192 and make the connection with what has been discussed in the lecture (also look at the man page for <code>pipe(2)</code>). Explain the following:
<ol style="list-style-type:lower-alpha">
<li> How does the consumer receive words to consume? </li>
<li> How does the producer generate and send words to the consumer? </li>
<li> Why is the call to <code>srandom(time(NULL))</code> on line 169 necessary? </li>
</ol>
</li>
<li> Replace the call to <code>srandom(time(NULL))</code> on line 169 with <code>srandom(42)</code>. What differences do you notice in <code>3000pc-fifo</code>'s behavior? </li>
</ol>

==C: Producer/Consumer with Shared Memory==
<ol style="list-style-type:decimal">
<li> Examine the source code of <code>3000pc-rendezvous</code>. Explain the following:
<ol style="list-style-type:lower-alpha">
<li> What are a few ways that <code>3000pc-rendezvous</code> is different from <code>3000pc-fifo</code>? </li>
<li> What does the call to <code>mmap()</code> on line 347 do (compared to our previous uses of <code>mmap()</code>)? How did you figure this out? </li>
<li> How does the producer notify the consumer that the queue is no longer empty? </li>
<li> How does the consumer notify the producer that the queue is no longer full? </li>
</ol>
</li>
<li> What arguments can you provide to make the producer wait for the consumer? Hint: Check the size of the queue. </li>
<li> What arguments can you provide to make the consumer wait for the producer? (You can disregard the wait at the very beginning as the queue is initially empty) </li>
<li> Another student tells you that the difference between processes and threads is that processes never share memory, while threads do. Is this statement correct or incorrect? How can the behavior of <code>3000pc-rendezvous</code> help justify your answer? </li>
<li> Change the calls to <code>pthread_mutexattr_setpshared()</code> and <code>pthread_condattr_setpshared()</code> (lines 293 and 298) to take 0 instead of 1. How does the behavior or <code>3000pc-rendezvous</code> change? Does anything break? </li>
</ol>

COMP3000 Operating Systems F23: Tutorial 6

2023-10-25T01:08:23Z

Abdou: /* B: Producer/Consumer with Pipes */

In this tutorial you will be learning about two implementations of the [https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem producer-consumer problem], a classic example of a concurrency problem. The [http://pages.cs.wisc.edu/~remzi/OSTEP/ class textbook] covers concurrency in great detail in Chapters 25-34, and the producer-consumer problem is covered in [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf Chapter 30 (Condition Variables)] and [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-sema.pdf Chapter 31 (Semaphores)]. While you can look at this part of the textbook, note that we will not be covering this material in the same level of detail, as should be clear from this tutorial.

==A: Getting Started==
Download [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut6/3000pc.zip 3000pc.zip], unpack, and run <code>make</code> to compile <code>3000pc-fifo</code> and <code>3000pc-rendezvous</code>.

Try to take this opportunity to read and understand the two programs line by line.

:: Note that these programs take 3 arguments each:
::* The number of events to process
::* The number of events to produce before the producer sleeps for 1 second
::* The number of events to consume before the consumer sleeps for 1 second

# Run both programs with the same arguments of your choice. Do this a few times with different arguments. Do they behave the same way? Do you notice any differences? (again, no “correct” answers but just play with them and document your observations)
# Repeat the above experiment, this time running each program under <code>strace</code> (with the <code>-f</code> flag to trace children too). Do you notice any difference in the system calls each program makes? Remember to ignore the lines before <code>main()</code> that might be distracting, e.g., start looking around and after the <code>clone()</code> system call.
# You can see there is "<code>#define QUEUESIZE 32</code>" in both programs. Are they actually used? Why (not)?

==B: Producer/Consumer with Pipes==
<ol style="list-style-type:decimal">
<li> Examine the source code of <code>3000pc-fifo</code>. Pay attention to the call to <code>pipe(pipefd)</code> on line 192 and make the connection with what has been discussed in the lecture (also look at the man page for <code>pipe(2)</code>). Explain the following:
<ol style="list-style-type:lower-alpha">
<li> How does the consumer receive words to consume? </li>
<li> How does the producer generate and send words to the consumer? </li>
<li> Why is the call to <code>srandom(time(NULL))</code> on line 169 necessary? </li>
</ol>
</li>
<li> Replace the call to <code>srandom(time(NULL))</code> on line 169 with <code>srandom(42)</code>. What differences do you notice in <code>3000pc-fifo</code>'s behavior? </li>
</ol>

==C: Producer/Consumer with Shared Memory==
<ol style="list-style-type:decimal">
<li> Examine the source code of <tt>3000pc-rendezvous</tt>. Explain the following:
<ol style="list-style-type:lower-alpha">
<li> What are a few ways that <tt>3000pc-rendezvous</tt> is different from <tt>3000pc-fifo</tt>? </li>
<li> What does the call to <tt>mmap()</tt> on line 347 do (compared to our previous uses of <tt>mmap()</tt>)? How did you figure this out? </li>
<li> How does the producer notify the consumer that the queue is no longer empty? </li>
<li> How does the consumer notify the producer that the queue is no longer full? </li>
</ol>
</li>
<li> What arguments can you provide to make the producer wait for the consumer? Hint: Check the size of the queue. </li>
<li> What arguments can you provide to make the consumer wait for the producer? (You can disregard the wait at the very beginning as the queue is initially empty) </li>
<li> Another student tells you that the difference between processes and threads is that processes never share memory, while threads do. Is this statement correct or incorrect? How can the behavior of <tt>3000pc-rendezvous</tt> help justify your answer? </li>
<li> Change the calls to <tt>pthread_mutexattr_setpshared()</tt> and <tt>pthread_condattr_setpshared()</tt> (lines 293 and 298) to take 0 instead of 1. How does the behavior or <tt>3000pc-rendezvous</tt> change? Does anything break? </li>
</ol>

COMP3000 Operating Systems F23: Tutorial 6

2023-10-25T01:07:51Z

Abdou: /* A: Getting Started */

In this tutorial you will be learning about two implementations of the [https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem producer-consumer problem], a classic example of a concurrency problem. The [http://pages.cs.wisc.edu/~remzi/OSTEP/ class textbook] covers concurrency in great detail in Chapters 25-34, and the producer-consumer problem is covered in [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf Chapter 30 (Condition Variables)] and [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-sema.pdf Chapter 31 (Semaphores)]. While you can look at this part of the textbook, note that we will not be covering this material in the same level of detail, as should be clear from this tutorial.

==A: Getting Started==
Download [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut6/3000pc.zip 3000pc.zip], unpack, and run <code>make</code> to compile <code>3000pc-fifo</code> and <code>3000pc-rendezvous</code>.

Try to take this opportunity to read and understand the two programs line by line.

:: Note that these programs take 3 arguments each:
::* The number of events to process
::* The number of events to produce before the producer sleeps for 1 second
::* The number of events to consume before the consumer sleeps for 1 second

# Run both programs with the same arguments of your choice. Do this a few times with different arguments. Do they behave the same way? Do you notice any differences? (again, no “correct” answers but just play with them and document your observations)
# Repeat the above experiment, this time running each program under <code>strace</code> (with the <code>-f</code> flag to trace children too). Do you notice any difference in the system calls each program makes? Remember to ignore the lines before <code>main()</code> that might be distracting, e.g., start looking around and after the <code>clone()</code> system call.
# You can see there is "<code>#define QUEUESIZE 32</code>" in both programs. Are they actually used? Why (not)?

==B: Producer/Consumer with Pipes==
<ol style="list-style-type:decimal">
<li> Examine the source code of <tt>3000pc-fifo</tt>. Pay attention to the call to <tt>pipe(pipefd)</tt> on line 192 and make the connection with what has been discussed in the lecture (also look at the man page for <tt>pipe(2)</tt>). Explain the following:
<ol style="list-style-type:lower-alpha">
<li> How does the consumer receive words to consume? </li>
<li> How does the producer generate and send words to the consumer? </li>
<li> Why is the call to <tt>srandom(time(NULL))</tt> on line 169 necessary? </li>
</ol>
</li>
<li> Replace the call to <tt>srandom(time(NULL))</tt> on line 169 with <tt>srandom(42)</tt>. What differences do you notice in <tt>3000pc-fifo</tt>'s behavior? </li>
</ol>

==C: Producer/Consumer with Shared Memory==
<ol style="list-style-type:decimal">
<li> Examine the source code of <tt>3000pc-rendezvous</tt>. Explain the following:
<ol style="list-style-type:lower-alpha">
<li> What are a few ways that <tt>3000pc-rendezvous</tt> is different from <tt>3000pc-fifo</tt>? </li>
<li> What does the call to <tt>mmap()</tt> on line 347 do (compared to our previous uses of <tt>mmap()</tt>)? How did you figure this out? </li>
<li> How does the producer notify the consumer that the queue is no longer empty? </li>
<li> How does the consumer notify the producer that the queue is no longer full? </li>
</ol>
</li>
<li> What arguments can you provide to make the producer wait for the consumer? Hint: Check the size of the queue. </li>
<li> What arguments can you provide to make the consumer wait for the producer? (You can disregard the wait at the very beginning as the queue is initially empty) </li>
<li> Another student tells you that the difference between processes and threads is that processes never share memory, while threads do. Is this statement correct or incorrect? How can the behavior of <tt>3000pc-rendezvous</tt> help justify your answer? </li>
<li> Change the calls to <tt>pthread_mutexattr_setpshared()</tt> and <tt>pthread_condattr_setpshared()</tt> (lines 293 and 298) to take 0 instead of 1. How does the behavior or <tt>3000pc-rendezvous</tt> change? Does anything break? </li>
</ol>

COMP3000 Operating Systems F23: Tutorial 6

2023-10-25T01:06:17Z

Abdou: /* General Instructions (same for all tutorials) */

In this tutorial you will be learning about two implementations of the [https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem producer-consumer problem], a classic example of a concurrency problem. The [http://pages.cs.wisc.edu/~remzi/OSTEP/ class textbook] covers concurrency in great detail in Chapters 25-34, and the producer-consumer problem is covered in [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf Chapter 30 (Condition Variables)] and [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-sema.pdf Chapter 31 (Semaphores)]. While you can look at this part of the textbook, note that we will not be covering this material in the same level of detail, as should be clear from this tutorial.

==A: Getting Started==
Download [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut6/3000pc.zip 3000pc.zip], unpack, and run <tt>make</tt> to compile <tt>3000pc-fifo</tt> and <tt>3000pc-rendezvous</tt>.

Try to take this opportunity to read and understand the two programs line by line.

:: Note that these programs take 3 arguments each:
::* The number of events to process
::* The number of events to produce before the producer sleeps for 1 second
::* The number of events to consume before the consumer sleeps for 1 second

# Run both programs with the same arguments of your choice. Do this a few times with different arguments. Do they behave the same way? Do you notice any differences? (again, no “correct” answers but just play with them and document your observations)
# Repeat the above experiment, this time running each program under <tt>strace</tt> (with the <tt>-f</tt> flag to trace children too). Do you notice any difference in the system calls each program makes? Remember to ignore the lines before <tt>main()</tt> that might be distracting, e.g., start looking around and after the <tt>clone()</tt> system call.
# You can see there is "<tt>#define QUEUESIZE 32</tt>" in both programs. Are they actually used? Why (not)?

==B: Producer/Consumer with Pipes==
<ol style="list-style-type:decimal">
<li> Examine the source code of <tt>3000pc-fifo</tt>. Pay attention to the call to <tt>pipe(pipefd)</tt> on line 192 and make the connection with what has been discussed in the lecture (also look at the man page for <tt>pipe(2)</tt>). Explain the following:
<ol style="list-style-type:lower-alpha">
<li> How does the consumer receive words to consume? </li>
<li> How does the producer generate and send words to the consumer? </li>
<li> Why is the call to <tt>srandom(time(NULL))</tt> on line 169 necessary? </li>
</ol>
</li>
<li> Replace the call to <tt>srandom(time(NULL))</tt> on line 169 with <tt>srandom(42)</tt>. What differences do you notice in <tt>3000pc-fifo</tt>'s behavior? </li>
</ol>

==C: Producer/Consumer with Shared Memory==
<ol style="list-style-type:decimal">
<li> Examine the source code of <tt>3000pc-rendezvous</tt>. Explain the following:
<ol style="list-style-type:lower-alpha">
<li> What are a few ways that <tt>3000pc-rendezvous</tt> is different from <tt>3000pc-fifo</tt>? </li>
<li> What does the call to <tt>mmap()</tt> on line 347 do (compared to our previous uses of <tt>mmap()</tt>)? How did you figure this out? </li>
<li> How does the producer notify the consumer that the queue is no longer empty? </li>
<li> How does the consumer notify the producer that the queue is no longer full? </li>
</ol>
</li>
<li> What arguments can you provide to make the producer wait for the consumer? Hint: Check the size of the queue. </li>
<li> What arguments can you provide to make the consumer wait for the producer? (You can disregard the wait at the very beginning as the queue is initially empty) </li>
<li> Another student tells you that the difference between processes and threads is that processes never share memory, while threads do. Is this statement correct or incorrect? How can the behavior of <tt>3000pc-rendezvous</tt> help justify your answer? </li>
<li> Change the calls to <tt>pthread_mutexattr_setpshared()</tt> and <tt>pthread_condattr_setpshared()</tt> (lines 293 and 298) to take 0 instead of 1. How does the behavior or <tt>3000pc-rendezvous</tt> change? Does anything break? </li>
</ol>

COMP3000 Operating Systems F23: Tutorial 6

2023-10-25T01:05:40Z

Abdou: Created page with "In this tutorial you will be learning about two implementations of the [https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem producer-consumer problem], a classic example of a concurrency problem. The [http://pages.cs.wisc.edu/~remzi/OSTEP/ class textbook] covers concurrency in great detail in Chapters 25-34, and the producer-consumer problem is covered in [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf Chapter 30 (Condition Variables)] and [http://pages..."

In this tutorial you will be learning about two implementations of the [https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem producer-consumer problem], a classic example of a concurrency problem. The [http://pages.cs.wisc.edu/~remzi/OSTEP/ class textbook] covers concurrency in great detail in Chapters 25-34, and the producer-consumer problem is covered in [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf Chapter 30 (Condition Variables)] and [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-sema.pdf Chapter 31 (Semaphores)]. While you can look at this part of the textbook, note that we will not be covering this material in the same level of detail, as should be clear from this tutorial.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''), but you should still turn in your work. Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA (in person or by responding to a poll if online). Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==A: Getting Started==
Download [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut6/3000pc.zip 3000pc.zip], unpack, and run <tt>make</tt> to compile <tt>3000pc-fifo</tt> and <tt>3000pc-rendezvous</tt>.

Try to take this opportunity to read and understand the two programs line by line.

:: Note that these programs take 3 arguments each:
::* The number of events to process
::* The number of events to produce before the producer sleeps for 1 second
::* The number of events to consume before the consumer sleeps for 1 second

# Run both programs with the same arguments of your choice. Do this a few times with different arguments. Do they behave the same way? Do you notice any differences? (again, no “correct” answers but just play with them and document your observations)
# Repeat the above experiment, this time running each program under <tt>strace</tt> (with the <tt>-f</tt> flag to trace children too). Do you notice any difference in the system calls each program makes? Remember to ignore the lines before <tt>main()</tt> that might be distracting, e.g., start looking around and after the <tt>clone()</tt> system call.
# You can see there is "<tt>#define QUEUESIZE 32</tt>" in both programs. Are they actually used? Why (not)?

==B: Producer/Consumer with Pipes==
<ol style="list-style-type:decimal">
<li> Examine the source code of <tt>3000pc-fifo</tt>. Pay attention to the call to <tt>pipe(pipefd)</tt> on line 192 and make the connection with what has been discussed in the lecture (also look at the man page for <tt>pipe(2)</tt>). Explain the following:
<ol style="list-style-type:lower-alpha">
<li> How does the consumer receive words to consume? </li>
<li> How does the producer generate and send words to the consumer? </li>
<li> Why is the call to <tt>srandom(time(NULL))</tt> on line 169 necessary? </li>
</ol>
</li>
<li> Replace the call to <tt>srandom(time(NULL))</tt> on line 169 with <tt>srandom(42)</tt>. What differences do you notice in <tt>3000pc-fifo</tt>'s behavior? </li>
</ol>

==C: Producer/Consumer with Shared Memory==
<ol style="list-style-type:decimal">
<li> Examine the source code of <tt>3000pc-rendezvous</tt>. Explain the following:
<ol style="list-style-type:lower-alpha">
<li> What are a few ways that <tt>3000pc-rendezvous</tt> is different from <tt>3000pc-fifo</tt>? </li>
<li> What does the call to <tt>mmap()</tt> on line 347 do (compared to our previous uses of <tt>mmap()</tt>)? How did you figure this out? </li>
<li> How does the producer notify the consumer that the queue is no longer empty? </li>
<li> How does the consumer notify the producer that the queue is no longer full? </li>
</ol>
</li>
<li> What arguments can you provide to make the producer wait for the consumer? Hint: Check the size of the queue. </li>
<li> What arguments can you provide to make the consumer wait for the producer? (You can disregard the wait at the very beginning as the queue is initially empty) </li>
<li> Another student tells you that the difference between processes and threads is that processes never share memory, while threads do. Is this statement correct or incorrect? How can the behavior of <tt>3000pc-rendezvous</tt> help justify your answer? </li>
<li> Change the calls to <tt>pthread_mutexattr_setpshared()</tt> and <tt>pthread_condattr_setpshared()</tt> (lines 293 and 298) to take 0 instead of 1. How does the behavior or <tt>3000pc-rendezvous</tt> change? Does anything break? </li>
</ol>

COMP3000 Operating Systems 2023F Tutorials

2023-10-25T01:05:32Z

Abdou: /* Tutorial Sections */

This page contains Tutorial instructions for COMP3000 A, Fall 2023.

The main page for the course can be found [https://people.scs.carleton.ca/~abdou/COMP3000_F2023.html here].

==Information about the Tutorials==
Tutorials are in person. You need to be physically in the room to finish up the tutorial instructions on your laptop and get checked-out by one of the two present TAs. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

Please stick to the tutorial session you are registered in. This avoids overcrowding.

Make well use of these tutorials to help you better understand the course material. The plan is to have 9 tutorial sessions this term, each would be worth 2.22% of your total grade. We might revisit this as the term progresses.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

== Getting Access to a Linux Machine ==

For the tutorials, you need to get access to a Linux machine. It is recommended to use an SCS Openstack instance (see instructions below). Alternatively, if you prefer to work directly on your laptop and avoid Openstack, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova this VirtualBox image] (courtesy of Prof. Zhao from Winter 2023). You will need access to a Linux system for the entire semester, ideally the same one.

=== Setting up an Openstack VM instance for the first time ===
We will use Openstack for the tutorials of this course. Follow these instructions for setting up an Openstack VM instance: [[COMP3000 Operating Systems F23: Openstack Instructions | instructions here]]. You only need to do this once for the whole term. Please use the same instance for both: the tutorials and the assignments. ''No need to create two instances''.

=== Connecting to the VM ===
You can find instructions with various ways to connect to your Openstack instance [[COMP3000 Operating Systems F23: Connecting to SCS Openstack | here]].

=== Downloading files from the VM ===
To download files from your Openstack VM instance, you can find instructions [[COMP3000 Operating Systems F23: Downloading files from your Openstack VM | here]].

=== Backing up the VM ===

To backup your VM, the image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account's home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

Reminder: ''do '''not''' take snapshots of your instance!'' Note that you cannot take snapshots of your VM anyway (it will keep trying and never succeed, and you'll create work for the tech staff who will have to undo what you did), so please don't try.

==Tutorial Sections==

* COMP 3000 A1: Tue: 08:35 - 09:55 at Loeb 720 (LA720). [Ethan and Sneha (T1-T9)]
* COMP 3000 A2: Fri: 11:35 - 12:55 at Loeb 720 (LA720). [Ethan and Ali (T1-T3), Ethan and Nareen (T4-T6), Ali and Nareen (T7-T9)]
* COMP 3000 A3: Thu: 08:35 - 09:55 at St. Patrick's 303 (SP303). [Ali and Nareen (T1-T9)]

{| class="wikitable"
|+ Tutorial Schedules
|-
! Tutorial !! Section !! Date !! Time !! Room !! TAs
|-
| rowspan="3" | [[COMP3000 Operating Systems 2023F Tutorials: Tutorial 1 | Tutorial 1]] || A1 || Sep 26 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Sep 28 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Sep 29 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 2 | Tutorial 2]] || A1 || Oct 3 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 5 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 6 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 3 | Tutorial 3]] || A1 || Oct 10 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 12 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 13 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 4 | Tutorial 4]] || A1 || Oct 31 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 2 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 3 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 5 | Tutorial 5]] || A1 || Nov 7 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 9 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 10 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 6 | Tutorial 6]] || A1 || Nov 14 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 16 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 17 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 7 || A1 || Nov 21 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 23 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 24 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 8 || A1 || Nov 28 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 30 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 1 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | Tutorial 9 || A1 || Dec 5 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Dec 7 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 8 || 11:35 am-12:55 pm || MC2000 || Ali and Nareen
|-
|}

COMP3000 Operating Systems 2023F Tutorials

2023-10-25T01:04:53Z

Abdou: /* Tutorial Sections */

This page contains Tutorial instructions for COMP3000 A, Fall 2023.

The main page for the course can be found [https://people.scs.carleton.ca/~abdou/COMP3000_F2023.html here].

==Information about the Tutorials==
Tutorials are in person. You need to be physically in the room to finish up the tutorial instructions on your laptop and get checked-out by one of the two present TAs. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

Please stick to the tutorial session you are registered in. This avoids overcrowding.

Make well use of these tutorials to help you better understand the course material. The plan is to have 9 tutorial sessions this term, each would be worth 2.22% of your total grade. We might revisit this as the term progresses.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

== Getting Access to a Linux Machine ==

For the tutorials, you need to get access to a Linux machine. It is recommended to use an SCS Openstack instance (see instructions below). Alternatively, if you prefer to work directly on your laptop and avoid Openstack, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova this VirtualBox image] (courtesy of Prof. Zhao from Winter 2023). You will need access to a Linux system for the entire semester, ideally the same one.

=== Setting up an Openstack VM instance for the first time ===
We will use Openstack for the tutorials of this course. Follow these instructions for setting up an Openstack VM instance: [[COMP3000 Operating Systems F23: Openstack Instructions | instructions here]]. You only need to do this once for the whole term. Please use the same instance for both: the tutorials and the assignments. ''No need to create two instances''.

=== Connecting to the VM ===
You can find instructions with various ways to connect to your Openstack instance [[COMP3000 Operating Systems F23: Connecting to SCS Openstack | here]].

=== Downloading files from the VM ===
To download files from your Openstack VM instance, you can find instructions [[COMP3000 Operating Systems F23: Downloading files from your Openstack VM | here]].

=== Backing up the VM ===

To backup your VM, the image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account's home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

Reminder: ''do '''not''' take snapshots of your instance!'' Note that you cannot take snapshots of your VM anyway (it will keep trying and never succeed, and you'll create work for the tech staff who will have to undo what you did), so please don't try.

==Tutorial Sections==

* COMP 3000 A1: Tue: 08:35 - 09:55 at Loeb 720 (LA720). [Ethan and Sneha (T1-T9)]
* COMP 3000 A2: Fri: 11:35 - 12:55 at Loeb 720 (LA720). [Ethan and Ali (T1-T3), Ethan and Nareen (T4-T6), Ali and Nareen (T7-T9)]
* COMP 3000 A3: Thu: 08:35 - 09:55 at St. Patrick's 303 (SP303). [Ali and Nareen (T1-T9)]

{| class="wikitable"
|+ Tutorial Schedules
|-
! Tutorial !! Section !! Date !! Time !! Room !! TAs
|-
| rowspan="3" | [[COMP3000 Operating Systems 2023F Tutorials: Tutorial 1 | Tutorial 1]] || A1 || Sep 26 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Sep 28 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Sep 29 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 2 | Tutorial 2]] || A1 || Oct 3 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 5 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 6 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 3 | Tutorial 3]] || A1 || Oct 10 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 12 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 13 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 4 | Tutorial 4]] || A1 || Oct 31 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 2 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 3 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 5 | Tutorial 5]] || A1 || Nov 7 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 9 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 10 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems W23: Tutorial 6 | Tutorial 6]] || A1 || Nov 14 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 16 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 17 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 7 || A1 || Nov 21 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 23 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 24 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 8 || A1 || Nov 28 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 30 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 1 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | Tutorial 9 || A1 || Dec 5 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Dec 7 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 8 || 11:35 am-12:55 pm || MC2000 || Ali and Nareen
|-
|}

COMP3000 Operating Systems F23: Tutorial 5

2023-10-25T00:50:25Z

Abdou: /* Tasks/Questions (Part A) */

In this tutorial you will be learning about files and filesystems by experimenting with and extending [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c], and creating and manipulating local filesystems. '''WARNING:''' Several of the commands here can lead to system corruption and data loss if not properly used. You have been warned. Please use a VM and make backups, when necessary.

'''Note: there are two parts of Tasks/Questions, Part A and Part B'''

==Files and inodes==
In UNIX/Linux filesystems, a filename does not directly refer to the contents of a file. Instead, a filename refers to an inode (as specified in a directory entry --- dentry, of its parent directory), and the inode then refers to the data. An inode is where file metadata is stored. File ownership, timestamps, and permissions are all stored in a file's inode. Regular files are just '''hard links''' connecting a file name to an inode.

In addition to regular files, we also have symbolic links, directories, block devices, character devices, and pipes. Each is its own type of inode.

Note that while you can find out a file's inode, you cannot go from an inode to a pathname or otherwise manipulate an inode directly from userspace - you always have to go through the pathname. The kernel, however, can access individual inodes directly (indeed it has to in order to get to the contents of a file when given a filename).

3000test.c uses stat() to give information on a given inode.

==Tasks/Questions (Part A)==
# Compile and run [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c]. It takes a filename as an argument and reports information on the file. Try giving it the following and see what it reports:
#* a regular text file that exists
#* a directory
#* a symbolic link
#* a device file (character or block)
# Change 3000test to use <tt>lstat()</tt> rather than stat. How does its behavior change?
# Modify 3000test so when it is given a symbolic link it reports the name of the target. Use <tt>readlink(2)</tt> (this notation describes how to access the man page, e.g., <tt>man 2 readlink</tt>).
# Are there files or directories that you cannot run 3000test on? Can you configure file/directory permissions so as to make something inaccessible to 3000test? Note that this is twofold: 1) whether it is completely inaccessible to 3000test ('''nothing can be displayed''', except the error msg); 2) whether it can be accessed for the search ('''no access to file content''').

==Creating, Mounting, and Unmounting Filesystem==
In UNIX/Linux systems we always have a single hierarchical namespace for files and directories. Pathnames that start with a <tt>/</tt> are absolute, starting at the top of the namespace, while ones that do not start with a / are relative to the current directory.

The files in this single namespace come from many sources, each of which can have very different semantics. These sources are known as '''filesystems'''.

When first started, a system must have at least one filesystem. This first filesystem is known as the root filesystem and has the contents of <tt>/</tt>. We can then add other filesystems to this initial set of files by '''mounting''' each filesystem on an existing directory.

On Ubuntu 22.04, the root filesystem is ext4 by default and many others are mounted on top of it. For example, a proc filesystem is mounted on <tt>/proc</tt>, a sysfs is mounted on <tt>/sys</tt>. If you run "mount" with no arguments, it lists all the currently mounted filesystems. Note that most of these are virtual filesystems, in that they do not correspond to any persistent storage (like a disk).

If you insert a USB stick on a laptop or desktop running Linux, it must be mounted before you can access its contents. The current convention is for it to be mounted in <tt>/media/</tt><mounting user><tt>/</tt><name of USB stick filesystem>. This usually happens automatically nowadays when the USB stick is plugged in.

To create an ext4 filesystem, you just run

mkfs.ext4 <writable block device> #Do not try it with any <tt>/dev/*</tt>

If you were to run this on the root filesystem of your current system, it would reformat your entire disk if it was allowed to complete. In all likelihood the system should prevent you from doing this as the root filesystem (and any mounted filesystem) gets special protections.

To mount a filesystem, you specify the filesystem to be mounted (by its type or by the device) and the mountpoint:

mount /dev/sdb1 /mnt #Do not try it unless you have that block device and want to do it

This would mount the first partition of the second disk on <tt>/mnt</tt>.

Filesystems can be stored on block devices - devices whose contents are accessed by specifying a block index/address. The other main type of device file, character devices, are used for devices where the data is accessed a single byte (character) at a time. Terminals, modems, printers – for example, will generally be represented as a character device.

Note that mount here can identify the filesystem on <tt>/dev/sdb1</tt> by looking at its superblock. The superblock is usually either in block 0 or 1 of a device. The superblock holds metadata of a filesystem, allowing it to be identified and specified. Without the superblock you cannot do anything with a filesystem. Because of its importance, there are backup superblocks in addition to the primary one. The blocks of a filesystem can be classified as either being superblocks, inode blocks, or data blocks.

Sometimes we want to play with a new filesystem but we don't have a physical device to format. If we have free space for a file in our current filesystem, however, we can put a filesystem in a file by using a loopback block device. A loopback block device is a block device where its data is stored in a regular file on another filesystem (rather than on a separate device). If you do filesystem commands with a regular file, it will transparently associate the file with an available loopback block device (e.g., <tt>/dev/loop1</tt>).

Note that inode numbers are filesystem-specific. Thus, the contents of a file are uniquely specified by its filesystem and inode. Only the kernel has to worry about this level of detail; in userspace, a file's full path contains all the necessary information.

Filesystems on persistent storage are always at risk of corruption. File system checker programs (<tt>fsck</tt>) can detect and repair filesystem errors.

==Tasks/Questions (Part B)==
# Run <tt>dd if=/dev/zero of=foo bs=8192 count=32K</tt> and use the <tt>ls</tt> command to check it. What is the logical size of the file foo? How much space does it consume on disk? (Hint: Look at the size option to ls). In comparison, create another file with <tt>dd if=/dev/zero of=foo2 bs=8192 seek=31K count=1K</tt>. Any observations?
# Run <tt>mkfs.ext4 foo</tt>. Does foo consume any more space or less? Do the same (<tt>mkfs.ext4 foo2</tt>) to the other file and answer the same question.
# Create any file in <tt>/mnt</tt> (e.g., <tt>sudo touch test.txt</tt>) and run <tt>sudo mount foo /mnt</tt>. Do you still see the file you just created?
# Run <tt>df</tt>. What device is mounted on <tt>/mnt</tt>? What is this device?
# Run <tt>sudo umount /mnt</tt>. What have gone away and what is back?
# Run <tt>dd if=/dev/zero of=foo conv=notrunc count=10 bs=512</tt>. How does the "<tt>conv=notrunc</tt>" change dd's behavior (versus the command in question 1)?
# Run <tt>sudo mount foo /mnt</tt>. What error do you get?
# What command can you run to make foo mountable again? What characteristic of the file system enables this command to work? (Hint: revisit the instructions of this tutorial if you have no idea)

COMP3000 Operating Systems F23: Tutorial 5

2023-10-25T00:48:54Z

Abdou: /* General Instructions (same for all tutorials) */

In this tutorial you will be learning about files and filesystems by experimenting with and extending [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c], and creating and manipulating local filesystems. '''WARNING:''' Several of the commands here can lead to system corruption and data loss if not properly used. You have been warned. Please use a VM and make backups, when necessary.

'''Note: there are two parts of Tasks/Questions, Part A and Part B'''

==Files and inodes==
In UNIX/Linux filesystems, a filename does not directly refer to the contents of a file. Instead, a filename refers to an inode (as specified in a directory entry --- dentry, of its parent directory), and the inode then refers to the data. An inode is where file metadata is stored. File ownership, timestamps, and permissions are all stored in a file's inode. Regular files are just '''hard links''' connecting a file name to an inode.

In addition to regular files, we also have symbolic links, directories, block devices, character devices, and pipes. Each is its own type of inode.

Note that while you can find out a file's inode, you cannot go from an inode to a pathname or otherwise manipulate an inode directly from userspace - you always have to go through the pathname. The kernel, however, can access individual inodes directly (indeed it has to in order to get to the contents of a file when given a filename).

3000test.c uses stat() to give information on a given inode.

==Tasks/Questions (Part A)==
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut5/3000test.c 3000test.c]. It takes a filename as an argument and reports information on the file. Try giving it the following and see what it reports:
#* a regular text file that exists
#* a directory
#* a symbolic link
#* a device file (character or block)
# Change 3000test to use <tt>lstat()</tt> rather than stat. How does its behavior change?
# Modify 3000test so when it is given a symbolic link it reports the name of the target. Use <tt>readlink(2)</tt> (this notation describes how to access the man page, e.g., <tt>man 2 readlink</tt>).
# Are there files or directories that you cannot run 3000test on? Can you configure file/directory permissions so as to make something inaccessible to 3000test? Note that this is twofold: 1) whether it is completely inaccessible to 3000test ('''nothing can be displayed''', except the error msg); 2) whether it can be accessed for the search ('''no access to file content''').

==Creating, Mounting, and Unmounting Filesystem==
In UNIX/Linux systems we always have a single hierarchical namespace for files and directories. Pathnames that start with a <tt>/</tt> are absolute, starting at the top of the namespace, while ones that do not start with a / are relative to the current directory.

The files in this single namespace come from many sources, each of which can have very different semantics. These sources are known as '''filesystems'''.

When first started, a system must have at least one filesystem. This first filesystem is known as the root filesystem and has the contents of <tt>/</tt>. We can then add other filesystems to this initial set of files by '''mounting''' each filesystem on an existing directory.

On Ubuntu 22.04, the root filesystem is ext4 by default and many others are mounted on top of it. For example, a proc filesystem is mounted on <tt>/proc</tt>, a sysfs is mounted on <tt>/sys</tt>. If you run "mount" with no arguments, it lists all the currently mounted filesystems. Note that most of these are virtual filesystems, in that they do not correspond to any persistent storage (like a disk).

If you insert a USB stick on a laptop or desktop running Linux, it must be mounted before you can access its contents. The current convention is for it to be mounted in <tt>/media/</tt><mounting user><tt>/</tt><name of USB stick filesystem>. This usually happens automatically nowadays when the USB stick is plugged in.

To create an ext4 filesystem, you just run

mkfs.ext4 <writable block device> #Do not try it with any <tt>/dev/*</tt>

If you were to run this on the root filesystem of your current system, it would reformat your entire disk if it was allowed to complete. In all likelihood the system should prevent you from doing this as the root filesystem (and any mounted filesystem) gets special protections.

To mount a filesystem, you specify the filesystem to be mounted (by its type or by the device) and the mountpoint:

mount /dev/sdb1 /mnt #Do not try it unless you have that block device and want to do it

This would mount the first partition of the second disk on <tt>/mnt</tt>.

Filesystems can be stored on block devices - devices whose contents are accessed by specifying a block index/address. The other main type of device file, character devices, are used for devices where the data is accessed a single byte (character) at a time. Terminals, modems, printers – for example, will generally be represented as a character device.

Note that mount here can identify the filesystem on <tt>/dev/sdb1</tt> by looking at its superblock. The superblock is usually either in block 0 or 1 of a device. The superblock holds metadata of a filesystem, allowing it to be identified and specified. Without the superblock you cannot do anything with a filesystem. Because of its importance, there are backup superblocks in addition to the primary one. The blocks of a filesystem can be classified as either being superblocks, inode blocks, or data blocks.

Sometimes we want to play with a new filesystem but we don't have a physical device to format. If we have free space for a file in our current filesystem, however, we can put a filesystem in a file by using a loopback block device. A loopback block device is a block device where its data is stored in a regular file on another filesystem (rather than on a separate device). If you do filesystem commands with a regular file, it will transparently associate the file with an available loopback block device (e.g., <tt>/dev/loop1</tt>).

Note that inode numbers are filesystem-specific. Thus, the contents of a file are uniquely specified by its filesystem and inode. Only the kernel has to worry about this level of detail; in userspace, a file's full path contains all the necessary information.

Filesystems on persistent storage are always at risk of corruption. File system checker programs (<tt>fsck</tt>) can detect and repair filesystem errors.

==Tasks/Questions (Part B)==
# Run <tt>dd if=/dev/zero of=foo bs=8192 count=32K</tt> and use the <tt>ls</tt> command to check it. What is the logical size of the file foo? How much space does it consume on disk? (Hint: Look at the size option to ls). In comparison, create another file with <tt>dd if=/dev/zero of=foo2 bs=8192 seek=31K count=1K</tt>. Any observations?
# Run <tt>mkfs.ext4 foo</tt>. Does foo consume any more space or less? Do the same (<tt>mkfs.ext4 foo2</tt>) to the other file and answer the same question.
# Create any file in <tt>/mnt</tt> (e.g., <tt>sudo touch test.txt</tt>) and run <tt>sudo mount foo /mnt</tt>. Do you still see the file you just created?
# Run <tt>df</tt>. What device is mounted on <tt>/mnt</tt>? What is this device?
# Run <tt>sudo umount /mnt</tt>. What have gone away and what is back?
# Run <tt>dd if=/dev/zero of=foo conv=notrunc count=10 bs=512</tt>. How does the "<tt>conv=notrunc</tt>" change dd's behavior (versus the command in question 1)?
# Run <tt>sudo mount foo /mnt</tt>. What error do you get?
# What command can you run to make foo mountable again? What characteristic of the file system enables this command to work? (Hint: revisit the instructions of this tutorial if you have no idea)

COMP3000 Operating Systems F23: Tutorial 5

2023-10-25T00:48:24Z

Abdou:

In this tutorial you will be learning about files and filesystems by experimenting with and extending [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut5/3000test.c 3000test.c], and creating and manipulating local filesystems. '''WARNING:''' Several of the commands here can lead to system corruption and data loss if not properly used. You have been warned. Please use a VM and make backups, when necessary.

'''Note: there are two parts of Tasks/Questions, Part A and Part B'''

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''), but you should still turn in your work. Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA (in person or by responding to a poll if online). Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Files and inodes==
In UNIX/Linux filesystems, a filename does not directly refer to the contents of a file. Instead, a filename refers to an inode (as specified in a directory entry --- dentry, of its parent directory), and the inode then refers to the data. An inode is where file metadata is stored. File ownership, timestamps, and permissions are all stored in a file's inode. Regular files are just '''hard links''' connecting a file name to an inode.

In addition to regular files, we also have symbolic links, directories, block devices, character devices, and pipes. Each is its own type of inode.

Note that while you can find out a file's inode, you cannot go from an inode to a pathname or otherwise manipulate an inode directly from userspace - you always have to go through the pathname. The kernel, however, can access individual inodes directly (indeed it has to in order to get to the contents of a file when given a filename).

3000test.c uses stat() to give information on a given inode.

==Tasks/Questions (Part A)==
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut5/3000test.c 3000test.c]. It takes a filename as an argument and reports information on the file. Try giving it the following and see what it reports:
#* a regular text file that exists
#* a directory
#* a symbolic link
#* a device file (character or block)
# Change 3000test to use <tt>lstat()</tt> rather than stat. How does its behavior change?
# Modify 3000test so when it is given a symbolic link it reports the name of the target. Use <tt>readlink(2)</tt> (this notation describes how to access the man page, e.g., <tt>man 2 readlink</tt>).
# Are there files or directories that you cannot run 3000test on? Can you configure file/directory permissions so as to make something inaccessible to 3000test? Note that this is twofold: 1) whether it is completely inaccessible to 3000test ('''nothing can be displayed''', except the error msg); 2) whether it can be accessed for the search ('''no access to file content''').

==Creating, Mounting, and Unmounting Filesystem==
In UNIX/Linux systems we always have a single hierarchical namespace for files and directories. Pathnames that start with a <tt>/</tt> are absolute, starting at the top of the namespace, while ones that do not start with a / are relative to the current directory.

The files in this single namespace come from many sources, each of which can have very different semantics. These sources are known as '''filesystems'''.

When first started, a system must have at least one filesystem. This first filesystem is known as the root filesystem and has the contents of <tt>/</tt>. We can then add other filesystems to this initial set of files by '''mounting''' each filesystem on an existing directory.

On Ubuntu 22.04, the root filesystem is ext4 by default and many others are mounted on top of it. For example, a proc filesystem is mounted on <tt>/proc</tt>, a sysfs is mounted on <tt>/sys</tt>. If you run "mount" with no arguments, it lists all the currently mounted filesystems. Note that most of these are virtual filesystems, in that they do not correspond to any persistent storage (like a disk).

If you insert a USB stick on a laptop or desktop running Linux, it must be mounted before you can access its contents. The current convention is for it to be mounted in <tt>/media/</tt><mounting user><tt>/</tt><name of USB stick filesystem>. This usually happens automatically nowadays when the USB stick is plugged in.

To create an ext4 filesystem, you just run

mkfs.ext4 <writable block device> #Do not try it with any <tt>/dev/*</tt>

If you were to run this on the root filesystem of your current system, it would reformat your entire disk if it was allowed to complete. In all likelihood the system should prevent you from doing this as the root filesystem (and any mounted filesystem) gets special protections.

To mount a filesystem, you specify the filesystem to be mounted (by its type or by the device) and the mountpoint:

mount /dev/sdb1 /mnt #Do not try it unless you have that block device and want to do it

This would mount the first partition of the second disk on <tt>/mnt</tt>.

Filesystems can be stored on block devices - devices whose contents are accessed by specifying a block index/address. The other main type of device file, character devices, are used for devices where the data is accessed a single byte (character) at a time. Terminals, modems, printers – for example, will generally be represented as a character device.

Note that mount here can identify the filesystem on <tt>/dev/sdb1</tt> by looking at its superblock. The superblock is usually either in block 0 or 1 of a device. The superblock holds metadata of a filesystem, allowing it to be identified and specified. Without the superblock you cannot do anything with a filesystem. Because of its importance, there are backup superblocks in addition to the primary one. The blocks of a filesystem can be classified as either being superblocks, inode blocks, or data blocks.

Sometimes we want to play with a new filesystem but we don't have a physical device to format. If we have free space for a file in our current filesystem, however, we can put a filesystem in a file by using a loopback block device. A loopback block device is a block device where its data is stored in a regular file on another filesystem (rather than on a separate device). If you do filesystem commands with a regular file, it will transparently associate the file with an available loopback block device (e.g., <tt>/dev/loop1</tt>).

Note that inode numbers are filesystem-specific. Thus, the contents of a file are uniquely specified by its filesystem and inode. Only the kernel has to worry about this level of detail; in userspace, a file's full path contains all the necessary information.

Filesystems on persistent storage are always at risk of corruption. File system checker programs (<tt>fsck</tt>) can detect and repair filesystem errors.

==Tasks/Questions (Part B)==
# Run <tt>dd if=/dev/zero of=foo bs=8192 count=32K</tt> and use the <tt>ls</tt> command to check it. What is the logical size of the file foo? How much space does it consume on disk? (Hint: Look at the size option to ls). In comparison, create another file with <tt>dd if=/dev/zero of=foo2 bs=8192 seek=31K count=1K</tt>. Any observations?
# Run <tt>mkfs.ext4 foo</tt>. Does foo consume any more space or less? Do the same (<tt>mkfs.ext4 foo2</tt>) to the other file and answer the same question.
# Create any file in <tt>/mnt</tt> (e.g., <tt>sudo touch test.txt</tt>) and run <tt>sudo mount foo /mnt</tt>. Do you still see the file you just created?
# Run <tt>df</tt>. What device is mounted on <tt>/mnt</tt>? What is this device?
# Run <tt>sudo umount /mnt</tt>. What have gone away and what is back?
# Run <tt>dd if=/dev/zero of=foo conv=notrunc count=10 bs=512</tt>. How does the "<tt>conv=notrunc</tt>" change dd's behavior (versus the command in question 1)?
# Run <tt>sudo mount foo /mnt</tt>. What error do you get?
# What command can you run to make foo mountable again? What characteristic of the file system enables this command to work? (Hint: revisit the instructions of this tutorial if you have no idea)

COMP3000 Operating Systems F23: Tutorial 5

2023-10-25T00:47:25Z

Abdou: Created page with "In this tutorial you will be learning about files and filesystems by experimenting with and extending [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut5/3000test.c 3000test.c], and creating and manipulating local filesystems. '''WARNING:''' Several of the commands here can lead to system corruption and data loss if not properly used. You have been warned. Please use a VM and make backups, when necessary. '''Note: there are two parts of Tasks/Questions, Part..."

In this tutorial you will be learning about files and filesystems by experimenting with and extending [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut5/3000test.c 3000test.c], and creating and manipulating local filesystems. '''WARNING:''' Several of the commands here can lead to system corruption and data loss if not properly used. You have been warned. Please use a VM and make backups, when necessary.

'''Note: there are two parts of Tasks/Questions, Part A and Part B'''

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''), but you should still turn in your work. Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA (in person or by responding to a poll if online). Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Files and inodes==
In UNIX/Linux filesystems, a filename does not directly refer to the contents of a file. Instead, a filename refers to an inode (as specified in a directory entry --- dentry, of its parent directory), and the inode then refers to the data. An inode is where file metadata is stored. File ownership, timestamps, and permissions are all stored in a file's inode. Regular files are just '''hard links''' connecting a file name to an inode.

In addition to regular files, we also have symbolic links, directories, block devices, character devices, and pipes. Each is its own type of inode.

Note that while you can find out a file's inode, you cannot go from an inode to a pathname or otherwise manipulate an inode directly from userspace - you always have to go through the pathname. The kernel, however, can access individual inodes directly (indeed it has to in order to get to the contents of a file when given a filename).

3000test.c uses stat() to give information on a given inode.

==Tasks/Questions (Part A)==
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut5/3000test.c 3000test.c]. It takes a filename as an argument and reports information on the file. Try giving it the following and see what it reports:
#* a regular text file that exists
#* a directory
#* a symbolic link
#* a device file (character or block)
# Change 3000test to use <tt>lstat()</tt> rather than stat. How does its behavior change?
# Modify 3000test so when it is given a symbolic link it reports the name of the target. Use <tt>readlink(2)</tt> (this notation describes how to access the man page, e.g., <tt>man 2 readlink</tt>).
# Are there files or directories that you cannot run 3000test on? Can you configure file/directory permissions so as to make something inaccessible to 3000test? Note that this is twofold: 1) whether it is completely inaccessible to 3000test ('''nothing can be displayed''', except the error msg); 2) whether it can be accessed for the search ('''no access to file content''').

==Creating, Mounting, and Unmounting Filesystem==
In UNIX/Linux systems we always have a single hierarchical namespace for files and directories. Pathnames that start with a <tt>/</tt> are absolute, starting at the top of the namespace, while ones that do not start with a / are relative to the current directory.

The files in this single namespace come from many sources, each of which can have very different semantics. These sources are known as '''filesystems'''.

When first started, a system must have at least one filesystem. This first filesystem is known as the root filesystem and has the contents of <tt>/</tt>. We can then add other filesystems to this initial set of files by '''mounting''' each filesystem on an existing directory.

On Ubuntu 22.04, the root filesystem is ext4 by default and many others are mounted on top of it. For example, a proc filesystem is mounted on <tt>/proc</tt>, a sysfs is mounted on <tt>/sys</tt>. If you run "mount" with no arguments, it lists all the currently mounted filesystems. Note that most of these are virtual filesystems, in that they do not correspond to any persistent storage (like a disk).

If you insert a USB stick on a laptop or desktop running Linux, it must be mounted before you can access its contents. The current convention is for it to be mounted in <tt>/media/</tt><mounting user><tt>/</tt><name of USB stick filesystem>. This usually happens automatically nowadays when the USB stick is plugged in.

To create an ext4 filesystem, you just run

mkfs.ext4 <writable block device> #Do not try it with any <tt>/dev/*</tt>

If you were to run this on the root filesystem of your current system, it would reformat your entire disk if it was allowed to complete. In all likelihood the system should prevent you from doing this as the root filesystem (and any mounted filesystem) gets special protections.

To mount a filesystem, you specify the filesystem to be mounted (by its type or by the device) and the mountpoint:

mount /dev/sdb1 /mnt #Do not try it unless you have that block device and want to do it

This would mount the first partition of the second disk on <tt>/mnt</tt>.

Filesystems can be stored on block devices - devices whose contents are accessed by specifying a block index/address. The other main type of device file, character devices, are used for devices where the data is accessed a single byte (character) at a time. Terminals, modems, printers – for example, will generally be represented as a character device.

Note that mount here can identify the filesystem on <tt>/dev/sdb1</tt> by looking at its superblock. The superblock is usually either in block 0 or 1 of a device. The superblock holds metadata of a filesystem, allowing it to be identified and specified. Without the superblock you cannot do anything with a filesystem. Because of its importance, there are backup superblocks in addition to the primary one. The blocks of a filesystem can be classified as either being superblocks, inode blocks, or data blocks.

Sometimes we want to play with a new filesystem but we don't have a physical device to format. If we have free space for a file in our current filesystem, however, we can put a filesystem in a file by using a loopback block device. A loopback block device is a block device where its data is stored in a regular file on another filesystem (rather than on a separate device). If you do filesystem commands with a regular file, it will transparently associate the file with an available loopback block device (e.g., <tt>/dev/loop1</tt>).

Note that inode numbers are filesystem-specific. Thus, the contents of a file are uniquely specified by its filesystem and inode. Only the kernel has to worry about this level of detail; in userspace, a file's full path contains all the necessary information.

Filesystems on persistent storage are always at risk of corruption. File system checker programs (<tt>fsck</tt>) can detect and repair filesystem errors.

==Tasks/Questions (Part B)==
# Run <tt>dd if=/dev/zero of=foo bs=8192 count=32K</tt> and use the <tt>ls</tt> command to check it. What is the logical size of the file foo? How much space does it consume on disk? (Hint: Look at the size option to ls). In comparison, create another file with <tt>dd if=/dev/zero of=foo2 bs=8192 seek=31K count=1K</tt>. Any observations?
# Run <tt>mkfs.ext4 foo</tt>. Does foo consume any more space or less? Do the same (<tt>mkfs.ext4 foo2</tt>) to the other file and answer the same question.
# Create any file in <tt>/mnt</tt> (e.g., <tt>sudo touch test.txt</tt>) and run <tt>sudo mount foo /mnt</tt>. Do you still see the file you just created?
# Run <tt>df</tt>. What device is mounted on <tt>/mnt</tt>? What is this device?
# Run <tt>sudo umount /mnt</tt>. What have gone away and what is back?
# Run <tt>dd if=/dev/zero of=foo conv=notrunc count=10 bs=512</tt>. How does the "<tt>conv=notrunc</tt>" change dd's behavior (versus the command in question 1)?
# Run <tt>sudo mount foo /mnt</tt>. What error do you get?
# What command can you run to make foo mountable again? What characteristic of the file system enables this command to work? (Hint: revisit the instructions of this tutorial if you have no idea)

COMP3000 Operating Systems 2023F Tutorials

2023-10-25T00:47:19Z

Abdou: /* Tutorial Sections */

This page contains Tutorial instructions for COMP3000 A, Fall 2023.

The main page for the course can be found [https://people.scs.carleton.ca/~abdou/COMP3000_F2023.html here].

==Information about the Tutorials==
Tutorials are in person. You need to be physically in the room to finish up the tutorial instructions on your laptop and get checked-out by one of the two present TAs. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

Please stick to the tutorial session you are registered in. This avoids overcrowding.

Make well use of these tutorials to help you better understand the course material. The plan is to have 9 tutorial sessions this term, each would be worth 2.22% of your total grade. We might revisit this as the term progresses.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

== Getting Access to a Linux Machine ==

For the tutorials, you need to get access to a Linux machine. It is recommended to use an SCS Openstack instance (see instructions below). Alternatively, if you prefer to work directly on your laptop and avoid Openstack, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova this VirtualBox image] (courtesy of Prof. Zhao from Winter 2023). You will need access to a Linux system for the entire semester, ideally the same one.

=== Setting up an Openstack VM instance for the first time ===
We will use Openstack for the tutorials of this course. Follow these instructions for setting up an Openstack VM instance: [[COMP3000 Operating Systems F23: Openstack Instructions | instructions here]]. You only need to do this once for the whole term. Please use the same instance for both: the tutorials and the assignments. ''No need to create two instances''.

=== Connecting to the VM ===
You can find instructions with various ways to connect to your Openstack instance [[COMP3000 Operating Systems F23: Connecting to SCS Openstack | here]].

=== Downloading files from the VM ===
To download files from your Openstack VM instance, you can find instructions [[COMP3000 Operating Systems F23: Downloading files from your Openstack VM | here]].

=== Backing up the VM ===

To backup your VM, the image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account's home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

Reminder: ''do '''not''' take snapshots of your instance!'' Note that you cannot take snapshots of your VM anyway (it will keep trying and never succeed, and you'll create work for the tech staff who will have to undo what you did), so please don't try.

==Tutorial Sections==

* COMP 3000 A1: Tue: 08:35 - 09:55 at Loeb 720 (LA720). [Ethan and Sneha (T1-T9)]
* COMP 3000 A2: Fri: 11:35 - 12:55 at Loeb 720 (LA720). [Ethan and Ali (T1-T3), Ethan and Nareen (T4-T6), Ali and Nareen (T7-T9)]
* COMP 3000 A3: Thu: 08:35 - 09:55 at St. Patrick's 303 (SP303). [Ali and Nareen (T1-T9)]

{| class="wikitable"
|+ Tutorial Schedules
|-
! Tutorial !! Section !! Date !! Time !! Room !! TAs
|-
| rowspan="3" | [[COMP3000 Operating Systems 2023F Tutorials: Tutorial 1 | Tutorial 1]] || A1 || Sep 26 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Sep 28 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Sep 29 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 2 | Tutorial 2]] || A1 || Oct 3 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 5 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 6 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 3 | Tutorial 3]] || A1 || Oct 10 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 12 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 13 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 4 | Tutorial 4]] || A1 || Oct 31 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 2 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 3 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 5 | Tutorial 5]] || A1 || Nov 7 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 9 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 10 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 6 || A1 || Nov 14 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 16 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 17 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 7 || A1 || Nov 21 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 23 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 24 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 8 || A1 || Nov 28 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 30 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 1 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | Tutorial 9 || A1 || Dec 5 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Dec 7 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 8 || 11:35 am-12:55 pm || MC2000 || Ali and Nareen
|-
|}

COMP3000 Operating Systems 2023F Tutorials

2023-10-25T00:46:45Z

Abdou: /* Tutorial Sections */

This page contains Tutorial instructions for COMP3000 A, Fall 2023.

The main page for the course can be found [https://people.scs.carleton.ca/~abdou/COMP3000_F2023.html here].

==Information about the Tutorials==
Tutorials are in person. You need to be physically in the room to finish up the tutorial instructions on your laptop and get checked-out by one of the two present TAs. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

Please stick to the tutorial session you are registered in. This avoids overcrowding.

Make well use of these tutorials to help you better understand the course material. The plan is to have 9 tutorial sessions this term, each would be worth 2.22% of your total grade. We might revisit this as the term progresses.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

== Getting Access to a Linux Machine ==

For the tutorials, you need to get access to a Linux machine. It is recommended to use an SCS Openstack instance (see instructions below). Alternatively, if you prefer to work directly on your laptop and avoid Openstack, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova this VirtualBox image] (courtesy of Prof. Zhao from Winter 2023). You will need access to a Linux system for the entire semester, ideally the same one.

=== Setting up an Openstack VM instance for the first time ===
We will use Openstack for the tutorials of this course. Follow these instructions for setting up an Openstack VM instance: [[COMP3000 Operating Systems F23: Openstack Instructions | instructions here]]. You only need to do this once for the whole term. Please use the same instance for both: the tutorials and the assignments. ''No need to create two instances''.

=== Connecting to the VM ===
You can find instructions with various ways to connect to your Openstack instance [[COMP3000 Operating Systems F23: Connecting to SCS Openstack | here]].

=== Downloading files from the VM ===
To download files from your Openstack VM instance, you can find instructions [[COMP3000 Operating Systems F23: Downloading files from your Openstack VM | here]].

=== Backing up the VM ===

To backup your VM, the image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account's home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

Reminder: ''do '''not''' take snapshots of your instance!'' Note that you cannot take snapshots of your VM anyway (it will keep trying and never succeed, and you'll create work for the tech staff who will have to undo what you did), so please don't try.

==Tutorial Sections==

* COMP 3000 A1: Tue: 08:35 - 09:55 at Loeb 720 (LA720). [Ethan and Sneha (T1-T9)]
* COMP 3000 A2: Fri: 11:35 - 12:55 at Loeb 720 (LA720). [Ethan and Ali (T1-T3), Ethan and Nareen (T4-T6), Ali and Nareen (T7-T9)]
* COMP 3000 A3: Thu: 08:35 - 09:55 at St. Patrick's 303 (SP303). [Ali and Nareen (T1-T9)]

{| class="wikitable"
|+ Tutorial Schedules
|-
! Tutorial !! Section !! Date !! Time !! Room !! TAs
|-
| rowspan="3" | [[COMP3000 Operating Systems 2023F Tutorials: Tutorial 1 | Tutorial 1]] || A1 || Sep 26 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Sep 28 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Sep 29 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 2 | Tutorial 2]] || A1 || Oct 3 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 5 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 6 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 3 | Tutorial 3]] || A1 || Oct 10 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Oct 12 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Oct 13 || 11:35 am-12:55 pm || LA720 || Ethan and Ali
|-
| rowspan="3" | [[COMP3000 Operating Systems F23: Tutorial 4 | Tutorial 4]] || A1 || Oct 31 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 2 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 3 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | [[COMP3000 Operating Systems W23: Tutorial 5 | Tutorial 5]] || A1 || Nov 7 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 9 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 10 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 6 || A1 || Nov 14 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 16 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 17 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 7 || A1 || Nov 21 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 23 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Nov 24 || 11:35 am-12:55 pm || LA720 || Ethan and Nareen
|-
| rowspan="3" | Tutorial 8 || A1 || Nov 28 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Nov 30 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 1 || 11:35 am-12:55 pm || LA720 || Ali and Nareen
|-
| rowspan="3" | Tutorial 9 || A1 || Dec 5 || 8:35 am-9:55 am || LA720 || Ethan and Sneha
|-
| A3 || Dec 7 || 8:35 am-9:55 am || SP303 || Ali and Nareen
|-
| A2 || Dec 8 || 11:35 am-12:55 pm || MC2000 || Ali and Nareen
|-
|}

COMP3000 Operating Systems F23: Tutorial 4

2023-10-24T20:56:11Z

Abdou: /* Optional/advanced tasks */

In this tutorial, you will learn about how user accounts and logging in work through exploring [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin.c]. You’ll also have a better understanding of the permission system and the shell/terminal environment.

==Logging in to a Linux system==
As mentioned in the lecture, in order to log in to a Linux-like system (including UNIX), the following steps must occur (potentially not in this order).
# The user must use a terminal to connect to the system, locally or remotely (e.g., initiated by <tt>/usr/sbin/getty</tt>)
# The user must authenticate themselves, proving an identify with authorized access to the system. By default, this is done through a username and password (e.g., by exec’ing <tt>/usr/bin/login</tt>).
# After authentication, login changes uid and gid to that of the new user.
# Login sets up other aspects of the user's context (mainly setting key environment variables).
# Login does an exec of the user's chosen/default shell.

[https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin] is a basic implementation of steps 3-5. (Steps 1 and 2 are accomplished by running 3000userlogin in a shell, as running an external command means the shell also creates a new process for it, assuming an authenticated session with a terminal.)

When 3000userlogin is properly compiled and set up, you can run:

./3000userlogin student

and you'll be logged in (again) as "student". This only works when you’re already logged in as student (see below).

You can add a user with the <code>adduser</code> command. For example, to create the user "someuser":

sudo adduser someuser

Note you'll have to answer several questions. Then you need to switch to someuser (<code>su someuser</code>).

If you just compile 3000userlogin normally, you won't be able to log in as anyone except the current user (be it student or someuser). To set up 3000userlogin properly, do the following:

sudo chown root:root 3000userlogin
sudo chmod u+s 3000userlogin

Feel free to read the <code>man</code> pages for both commands to learn more.

The <code>chown</code> command makes the binary owned by root, and the <code>chmod</code> command makes it setuid. Thus, when the program is exec'd it will have an effective user ID of root (euid=0). To facilitate the following tasks, you may also want to create different versions of 3000userlogin by naming them, e.g., 3000userlogin.setuid and 3000userlogin.orig, of your choice.

==Tasks/Questions==
# Compile and setup [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin] as described above. Create a new user account. Verify that you can use 3000userlogin to login as the new user without typing the password of the new user (without logging in beforehand). Think about why you are able to avoid typing the password and still get authenticated.
# Why is the syntax highlighting gone (coloring characters)? What simple change (one line) can you make to have 3000userlogin preserve the colors? (Hint: check the environment variables)
# What is returned as the user's password by <tt>getpwnam()</tt> (in terms of type)? Is this what you expected? Read the manual page as you find necessary.
# Compare the uid, gid, euid, egid when running 3000userlogin as a regular user (this means without the “chown” and “chmod” commands above), running it setuid root, and running it as root (i.e., running it from a root shell without the setuid bit set, e.g., “<tt>sudo ./3000userlogin someuser</tt>”). Also check to see what happens when you set the setgid bit.
# Why does 3000userlogin change its gid before changing its uid? What happens if you switch the order of these operations?
# Make 3000userlogin use the shell that is specified in the user's password entry (which is actually read from <tt>/etc/passwd</tt>). Check by setting someuser’s shell to a new shell and then see if that new shell runs when you run 3000userlogin. You can change a user's shell with the <tt>chsh</tt> command.
# Can you set 3000shell to be a user's default shell? What changes do you have to make for <tt>chsh</tt> to accept 3000shell? Does anything obvious break when running 3000shell this way, and how can you change 3000userlogin to fix it?
# Does a user's default shell have to be a regular shell? Could it instead be an arbitrary program? How do you know?
# Are the environment variables set by 3000userlogin the only environment variables that are set (visible) after you successfully login? How do you know?
# Note that 3000userlogin uses <tt>environ</tt> not <tt>envp</tt> (as an argument to <tt>main()</tt>) to access environment variables. Why not use <tt>envp</tt>? (Try changing the code to use <tt>envp</tt> and see what happens: worse or better?)

==Optional/advanced tasks==
# Prompt for the user's password before logging the user in (Step 2 above). To do this, you'll need to get the right password hash and then figure out how to hash the entered password to check if it matches. Recall the format of <code>/etc/shadow</code> and you may need to learn how to use the <code>openssl passwd</code> command.

COMP3000 Operating Systems F23: Tutorial 4

2023-10-24T20:55:07Z

Abdou: /* Optional/advanced tasks */

In this tutorial, you will learn about how user accounts and logging in work through exploring [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin.c]. You’ll also have a better understanding of the permission system and the shell/terminal environment.

==Logging in to a Linux system==
As mentioned in the lecture, in order to log in to a Linux-like system (including UNIX), the following steps must occur (potentially not in this order).
# The user must use a terminal to connect to the system, locally or remotely (e.g., initiated by <tt>/usr/sbin/getty</tt>)
# The user must authenticate themselves, proving an identify with authorized access to the system. By default, this is done through a username and password (e.g., by exec’ing <tt>/usr/bin/login</tt>).
# After authentication, login changes uid and gid to that of the new user.
# Login sets up other aspects of the user's context (mainly setting key environment variables).
# Login does an exec of the user's chosen/default shell.

[https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin] is a basic implementation of steps 3-5. (Steps 1 and 2 are accomplished by running 3000userlogin in a shell, as running an external command means the shell also creates a new process for it, assuming an authenticated session with a terminal.)

When 3000userlogin is properly compiled and set up, you can run:

./3000userlogin student

and you'll be logged in (again) as "student". This only works when you’re already logged in as student (see below).

You can add a user with the <code>adduser</code> command. For example, to create the user "someuser":

sudo adduser someuser

Note you'll have to answer several questions. Then you need to switch to someuser (<code>su someuser</code>).

If you just compile 3000userlogin normally, you won't be able to log in as anyone except the current user (be it student or someuser). To set up 3000userlogin properly, do the following:

sudo chown root:root 3000userlogin
sudo chmod u+s 3000userlogin

Feel free to read the <code>man</code> pages for both commands to learn more.

The <code>chown</code> command makes the binary owned by root, and the <code>chmod</code> command makes it setuid. Thus, when the program is exec'd it will have an effective user ID of root (euid=0). To facilitate the following tasks, you may also want to create different versions of 3000userlogin by naming them, e.g., 3000userlogin.setuid and 3000userlogin.orig, of your choice.

==Tasks/Questions==
# Compile and setup [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin] as described above. Create a new user account. Verify that you can use 3000userlogin to login as the new user without typing the password of the new user (without logging in beforehand). Think about why you are able to avoid typing the password and still get authenticated.
# Why is the syntax highlighting gone (coloring characters)? What simple change (one line) can you make to have 3000userlogin preserve the colors? (Hint: check the environment variables)
# What is returned as the user's password by <tt>getpwnam()</tt> (in terms of type)? Is this what you expected? Read the manual page as you find necessary.
# Compare the uid, gid, euid, egid when running 3000userlogin as a regular user (this means without the “chown” and “chmod” commands above), running it setuid root, and running it as root (i.e., running it from a root shell without the setuid bit set, e.g., “<tt>sudo ./3000userlogin someuser</tt>”). Also check to see what happens when you set the setgid bit.
# Why does 3000userlogin change its gid before changing its uid? What happens if you switch the order of these operations?
# Make 3000userlogin use the shell that is specified in the user's password entry (which is actually read from <tt>/etc/passwd</tt>). Check by setting someuser’s shell to a new shell and then see if that new shell runs when you run 3000userlogin. You can change a user's shell with the <tt>chsh</tt> command.
# Can you set 3000shell to be a user's default shell? What changes do you have to make for <tt>chsh</tt> to accept 3000shell? Does anything obvious break when running 3000shell this way, and how can you change 3000userlogin to fix it?
# Does a user's default shell have to be a regular shell? Could it instead be an arbitrary program? How do you know?
# Are the environment variables set by 3000userlogin the only environment variables that are set (visible) after you successfully login? How do you know?
# Note that 3000userlogin uses <tt>environ</tt> not <tt>envp</tt> (as an argument to <tt>main()</tt>) to access environment variables. Why not use <tt>envp</tt>? (Try changing the code to use <tt>envp</tt> and see what happens: worse or better?)

==Optional/advanced tasks==
# Prompt for the user's password before logging the user in (Step 2 above). To do this, you'll need to get the right password hash and then figure out how to hash the entered password to check if it matches. Recall the format of <code>/etc/shadow</code> and you may need to learn how to use the <code>openssl passwd</code> command.

Note: you are '''not''' required to complete and submit work for this optional task. Doing it will enhance your understanding of how authentication works.

COMP3000 Operating Systems F23: Tutorial 4

2023-10-24T20:53:43Z

Abdou: /* Logging in to a Linux system */

In this tutorial, you will learn about how user accounts and logging in work through exploring [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin.c]. You’ll also have a better understanding of the permission system and the shell/terminal environment.

==Logging in to a Linux system==
As mentioned in the lecture, in order to log in to a Linux-like system (including UNIX), the following steps must occur (potentially not in this order).
# The user must use a terminal to connect to the system, locally or remotely (e.g., initiated by <tt>/usr/sbin/getty</tt>)
# The user must authenticate themselves, proving an identify with authorized access to the system. By default, this is done through a username and password (e.g., by exec’ing <tt>/usr/bin/login</tt>).
# After authentication, login changes uid and gid to that of the new user.
# Login sets up other aspects of the user's context (mainly setting key environment variables).
# Login does an exec of the user's chosen/default shell.

[https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin] is a basic implementation of steps 3-5. (Steps 1 and 2 are accomplished by running 3000userlogin in a shell, as running an external command means the shell also creates a new process for it, assuming an authenticated session with a terminal.)

When 3000userlogin is properly compiled and set up, you can run:

./3000userlogin student

and you'll be logged in (again) as "student". This only works when you’re already logged in as student (see below).

You can add a user with the <code>adduser</code> command. For example, to create the user "someuser":

sudo adduser someuser

Note you'll have to answer several questions. Then you need to switch to someuser (<code>su someuser</code>).

If you just compile 3000userlogin normally, you won't be able to log in as anyone except the current user (be it student or someuser). To set up 3000userlogin properly, do the following:

sudo chown root:root 3000userlogin
sudo chmod u+s 3000userlogin

Feel free to read the <code>man</code> pages for both commands to learn more.

The <code>chown</code> command makes the binary owned by root, and the <code>chmod</code> command makes it setuid. Thus, when the program is exec'd it will have an effective user ID of root (euid=0). To facilitate the following tasks, you may also want to create different versions of 3000userlogin by naming them, e.g., 3000userlogin.setuid and 3000userlogin.orig, of your choice.

==Tasks/Questions==
# Compile and setup [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin] as described above. Create a new user account. Verify that you can use 3000userlogin to login as the new user without typing the password of the new user (without logging in beforehand). Think about why you are able to avoid typing the password and still get authenticated.
# Why is the syntax highlighting gone (coloring characters)? What simple change (one line) can you make to have 3000userlogin preserve the colors? (Hint: check the environment variables)
# What is returned as the user's password by <tt>getpwnam()</tt> (in terms of type)? Is this what you expected? Read the manual page as you find necessary.
# Compare the uid, gid, euid, egid when running 3000userlogin as a regular user (this means without the “chown” and “chmod” commands above), running it setuid root, and running it as root (i.e., running it from a root shell without the setuid bit set, e.g., “<tt>sudo ./3000userlogin someuser</tt>”). Also check to see what happens when you set the setgid bit.
# Why does 3000userlogin change its gid before changing its uid? What happens if you switch the order of these operations?
# Make 3000userlogin use the shell that is specified in the user's password entry (which is actually read from <tt>/etc/passwd</tt>). Check by setting someuser’s shell to a new shell and then see if that new shell runs when you run 3000userlogin. You can change a user's shell with the <tt>chsh</tt> command.
# Can you set 3000shell to be a user's default shell? What changes do you have to make for <tt>chsh</tt> to accept 3000shell? Does anything obvious break when running 3000shell this way, and how can you change 3000userlogin to fix it?
# Does a user's default shell have to be a regular shell? Could it instead be an arbitrary program? How do you know?
# Are the environment variables set by 3000userlogin the only environment variables that are set (visible) after you successfully login? How do you know?
# Note that 3000userlogin uses <tt>environ</tt> not <tt>envp</tt> (as an argument to <tt>main()</tt>) to access environment variables. Why not use <tt>envp</tt>? (Try changing the code to use <tt>envp</tt> and see what happens: worse or better?)

==Optional/advanced tasks==
# Prompt for the user's password before logging the user in (Step 2 above). To do this, you'll need to get the right password hash and then figure out how to hash the entered password to check if it matches. Recall the format of <tt>/etc/shadow</tt> and you may need to learn how to use the <tt>openssl passwd</tt> command.

Note: you are '''not''' required to complete and submit work for this optional task. Doing it will enhance your understanding of how authentication works.

COMP3000 Operating Systems F23: Tutorial 4

2023-10-24T20:49:37Z

Abdou: /* Tasks/Questions */

In this tutorial, you will learn about how user accounts and logging in work through exploring [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin.c]. You’ll also have a better understanding of the permission system and the shell/terminal environment.

==Logging in to a Linux system==
As mentioned in the lecture, in order to log in to a Linux-like system (including UNIX), the following steps must occur (potentially not in this order).
# The user must use a terminal to connect to the system, locally or remotely (e.g., initiated by <tt>/usr/sbin/getty</tt>)
# The user must authenticate themselves, proving an identify with authorized access to the system. By default, this is done through a username and password (e.g., by exec’ing <tt>/usr/bin/login</tt>).
# After authentication, login changes uid and gid to that of the new user.
# Login sets up other aspects of the user's context (mainly setting key environment variables).
# Login does an exec of the user's chosen/default shell.

[https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin] is a basic implementation of steps 3-5. (Steps 1 and 2 are accomplished by running 3000userlogin in a shell, as running an external command means the shell also creates a new process for it, assuming an authenticated session with a terminal.)

When 3000userlogin is properly compiled and set up, you can run:

./3000userlogin student

and you'll be logged in (again) as "student". This only works when you’re already logged in as student (see below).

You can add a user with the <tt>adduser</tt> command. For example, to create the user "someuser":

sudo adduser someuser

Note you'll have to answer several questions. Then you need to switch to someuser (<tt>su someuser</tt>).

If you just compile 3000userlogin normally, you won't be able to log in as anyone except the current user (be it student or someuser). To set up 3000userlogin properly, do the following:

sudo chown root:root 3000userlogin
sudo chmod u+s 3000userlogin

Feel free to read the <tt>man</tt> pages for both commands to learn more.

The <tt>chown</tt> command makes the binary owned by root, and the <tt>chmod</tt> command makes it setuid. Thus, when the program is exec'd it will have an effective user ID of root (euid=0). To facilitate the following tasks, you may also want to create different versions of 3000userlogin by naming them, e.g., 3000userlogin.setuid and 3000userlogin.orig, of your choice.

==Tasks/Questions==
# Compile and setup [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin] as described above. Create a new user account. Verify that you can use 3000userlogin to login as the new user without typing the password of the new user (without logging in beforehand). Think about why you are able to avoid typing the password and still get authenticated.
# Why is the syntax highlighting gone (coloring characters)? What simple change (one line) can you make to have 3000userlogin preserve the colors? (Hint: check the environment variables)
# What is returned as the user's password by <tt>getpwnam()</tt> (in terms of type)? Is this what you expected? Read the manual page as you find necessary.
# Compare the uid, gid, euid, egid when running 3000userlogin as a regular user (this means without the “chown” and “chmod” commands above), running it setuid root, and running it as root (i.e., running it from a root shell without the setuid bit set, e.g., “<tt>sudo ./3000userlogin someuser</tt>”). Also check to see what happens when you set the setgid bit.
# Why does 3000userlogin change its gid before changing its uid? What happens if you switch the order of these operations?
# Make 3000userlogin use the shell that is specified in the user's password entry (which is actually read from <tt>/etc/passwd</tt>). Check by setting someuser’s shell to a new shell and then see if that new shell runs when you run 3000userlogin. You can change a user's shell with the <tt>chsh</tt> command.
# Can you set 3000shell to be a user's default shell? What changes do you have to make for <tt>chsh</tt> to accept 3000shell? Does anything obvious break when running 3000shell this way, and how can you change 3000userlogin to fix it?
# Does a user's default shell have to be a regular shell? Could it instead be an arbitrary program? How do you know?
# Are the environment variables set by 3000userlogin the only environment variables that are set (visible) after you successfully login? How do you know?
# Note that 3000userlogin uses <tt>environ</tt> not <tt>envp</tt> (as an argument to <tt>main()</tt>) to access environment variables. Why not use <tt>envp</tt>? (Try changing the code to use <tt>envp</tt> and see what happens: worse or better?)

==Optional/advanced tasks==
# Prompt for the user's password before logging the user in (Step 2 above). To do this, you'll need to get the right password hash and then figure out how to hash the entered password to check if it matches. Recall the format of <tt>/etc/shadow</tt> and you may need to learn how to use the <tt>openssl passwd</tt> command.

Note: you are '''not''' required to complete and submit work for this optional task. Doing it will enhance your understanding of how authentication works.

COMP3000 Operating Systems F23: Tutorial 4

2023-10-24T20:49:22Z

Abdou:

In this tutorial, you will learn about how user accounts and logging in work through exploring [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin.c]. You’ll also have a better understanding of the permission system and the shell/terminal environment.

==Logging in to a Linux system==
As mentioned in the lecture, in order to log in to a Linux-like system (including UNIX), the following steps must occur (potentially not in this order).
# The user must use a terminal to connect to the system, locally or remotely (e.g., initiated by <tt>/usr/sbin/getty</tt>)
# The user must authenticate themselves, proving an identify with authorized access to the system. By default, this is done through a username and password (e.g., by exec’ing <tt>/usr/bin/login</tt>).
# After authentication, login changes uid and gid to that of the new user.
# Login sets up other aspects of the user's context (mainly setting key environment variables).
# Login does an exec of the user's chosen/default shell.

[https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin] is a basic implementation of steps 3-5. (Steps 1 and 2 are accomplished by running 3000userlogin in a shell, as running an external command means the shell also creates a new process for it, assuming an authenticated session with a terminal.)

When 3000userlogin is properly compiled and set up, you can run:

./3000userlogin student

and you'll be logged in (again) as "student". This only works when you’re already logged in as student (see below).

You can add a user with the <tt>adduser</tt> command. For example, to create the user "someuser":

sudo adduser someuser

Note you'll have to answer several questions. Then you need to switch to someuser (<tt>su someuser</tt>).

If you just compile 3000userlogin normally, you won't be able to log in as anyone except the current user (be it student or someuser). To set up 3000userlogin properly, do the following:

sudo chown root:root 3000userlogin
sudo chmod u+s 3000userlogin

Feel free to read the <tt>man</tt> pages for both commands to learn more.

The <tt>chown</tt> command makes the binary owned by root, and the <tt>chmod</tt> command makes it setuid. Thus, when the program is exec'd it will have an effective user ID of root (euid=0). To facilitate the following tasks, you may also want to create different versions of 3000userlogin by naming them, e.g., 3000userlogin.setuid and 3000userlogin.orig, of your choice.

==Tasks/Questions==
# Compile and setup [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut4/3000userlogin.c 3000userlogin] as described above. Create a new user account. Verify that you can use 3000userlogin to login as the new user without typing the password of the new user (without logging in beforehand). Think about why you are able to avoid typing the password and still get authenticated.
# Why is the syntax highlighting gone (coloring characters)? What simple change (one line) can you make to have 3000userlogin preserve the colors? (Hint: check the environment variables)
# What is returned as the user's password by <tt>getpwnam()</tt> (in terms of type)? Is this what you expected? Read the manual page as you find necessary.
# Compare the uid, gid, euid, egid when running 3000userlogin as a regular user (this means without the “chown” and “chmod” commands above), running it setuid root, and running it as root (i.e., running it from a root shell without the setuid bit set, e.g., “<tt>sudo ./3000userlogin someuser</tt>”). Also check to see what happens when you set the setgid bit.
# Why does 3000userlogin change its gid before changing its uid? What happens if you switch the order of these operations?
# Make 3000userlogin use the shell that is specified in the user's password entry (which is actually read from <tt>/etc/passwd</tt>). Check by setting someuser’s shell to a new shell and then see if that new shell runs when you run 3000userlogin. You can change a user's shell with the <tt>chsh</tt> command.
# Can you set 3000shell to be a user's default shell? What changes do you have to make for <tt>chsh</tt> to accept 3000shell? Does anything obvious break when running 3000shell this way, and how can you change 3000userlogin to fix it?
# Does a user's default shell have to be a regular shell? Could it instead be an arbitrary program? How do you know?
# Are the environment variables set by 3000userlogin the only environment variables that are set (visible) after you successfully login? How do you know?
# Note that 3000userlogin uses <tt>environ</tt> not <tt>envp</tt> (as an argument to <tt>main()</tt>) to access environment variables. Why not use <tt>envp</tt>? (Try changing the code to use <tt>envp</tt> and see what happens: worse or better?)

==Optional/advanced tasks==
# Prompt for the user's password before logging the user in (Step 2 above). To do this, you'll need to get the right password hash and then figure out how to hash the entered password to check if it matches. Recall the format of <tt>/etc/shadow</tt> and you may need to learn how to use the <tt>openssl passwd</tt> command.

Note: you are '''not''' required to complete and submit work for this optional task. Doing it will enhance your understanding of how authentication works.

COMP3000 Operating Systems F23: Tutorial 4

2023-10-24T20:48:25Z

Abdou:

In this tutorial, you will learn about how user accounts and logging in work through exploring [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut4/3000userlogin.c 3000userlogin.c]. You’ll also have a better understanding of the permission system and the shell/terminal environment.

==Logging in to a Linux system==
As mentioned in the lecture, in order to log in to a Linux-like system (including UNIX), the following steps must occur (potentially not in this order).
# The user must use a terminal to connect to the system, locally or remotely (e.g., initiated by <tt>/usr/sbin/getty</tt>)
# The user must authenticate themselves, proving an identify with authorized access to the system. By default, this is done through a username and password (e.g., by exec’ing <tt>/usr/bin/login</tt>).
# After authentication, login changes uid and gid to that of the new user.
# Login sets up other aspects of the user's context (mainly setting key environment variables).
# Login does an exec of the user's chosen/default shell.

[https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut4/3000userlogin.c 3000userlogin] is a basic implementation of steps 3-5. (Steps 1 and 2 are accomplished by running 3000userlogin in a shell, as running an external command means the shell also creates a new process for it, assuming an authenticated session with a terminal.)

When 3000userlogin is properly compiled and set up, you can run:

./3000userlogin student

and you'll be logged in (again) as "student". This only works when you’re already logged in as student (see below).

You can add a user with the <tt>adduser</tt> command. For example, to create the user "someuser":

sudo adduser someuser

Note you'll have to answer several questions. Then you need to switch to someuser (<tt>su someuser</tt>).

If you just compile 3000userlogin normally, you won't be able to log in as anyone except the current user (be it student or someuser). To set up 3000userlogin properly, do the following:

sudo chown root:root 3000userlogin
sudo chmod u+s 3000userlogin

Feel free to read the <tt>man</tt> pages for both commands to learn more.

The <tt>chown</tt> command makes the binary owned by root, and the <tt>chmod</tt> command makes it setuid. Thus, when the program is exec'd it will have an effective user ID of root (euid=0). To facilitate the following tasks, you may also want to create different versions of 3000userlogin by naming them, e.g., 3000userlogin.setuid and 3000userlogin.orig, of your choice.

==Tasks/Questions==
# Compile and setup [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut4/3000userlogin.c 3000userlogin] as described above. Create a new user account. Verify that you can use 3000userlogin to login as the new user without typing the password of the new user (without logging in beforehand). Think about why you are able to avoid typing the password and still get authenticated.
# Why is the syntax highlighting gone (coloring characters)? What simple change (one line) can you make to have 3000userlogin preserve the colors? (Hint: check the environment variables)
# What is returned as the user's password by <tt>getpwnam()</tt> (in terms of type)? Is this what you expected? Read the manual page as you find necessary.
# Compare the uid, gid, euid, egid when running 3000userlogin as a regular user (this means without the “chown” and “chmod” commands above), running it setuid root, and running it as root (i.e., running it from a root shell without the setuid bit set, e.g., “<tt>sudo ./3000userlogin someuser</tt>”). Also check to see what happens when you set the setgid bit.
# Why does 3000userlogin change its gid before changing its uid? What happens if you switch the order of these operations?
# Make 3000userlogin use the shell that is specified in the user's password entry (which is actually read from <tt>/etc/passwd</tt>). Check by setting someuser’s shell to a new shell and then see if that new shell runs when you run 3000userlogin. You can change a user's shell with the <tt>chsh</tt> command.
# Can you set 3000shell to be a user's default shell? What changes do you have to make for <tt>chsh</tt> to accept 3000shell? Does anything obvious break when running 3000shell this way, and how can you change 3000userlogin to fix it?
# Does a user's default shell have to be a regular shell? Could it instead be an arbitrary program? How do you know?
# Are the environment variables set by 3000userlogin the only environment variables that are set (visible) after you successfully login? How do you know?
# Note that 3000userlogin uses <tt>environ</tt> not <tt>envp</tt> (as an argument to <tt>main()</tt>) to access environment variables. Why not use <tt>envp</tt>? (Try changing the code to use <tt>envp</tt> and see what happens: worse or better?)

==Optional/advanced tasks==
# Prompt for the user's password before logging the user in (Step 2 above). To do this, you'll need to get the right password hash and then figure out how to hash the entered password to check if it matches. Recall the format of <tt>/etc/shadow</tt> and you may need to learn how to use the <tt>openssl passwd</tt> command.

Note: you are '''not''' required to complete and submit work for this optional task. Doing it will enhance your understanding of how authentication works.