Soma-notes - User contributions [en]

COMP3000 Operating Systems W25: Tutorial 9

2025-03-15T15:17:16Z

Lianyingzhao: Created page with "By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible. ==General Instructions (same for all tutorials)== <div class="mw-collapsible mw-collapsed"> Click on Expand to view. <div class="mw-collapsible-content"> Tutorials are graded based on participation and effo..."

By the end of this tutorial, you will have a basic familiarity with the Docker platform and be able to run, operate, and customize containers, and create simple containers of your own. You will see how kernel building blocks like namespaces make containers possible.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Background==

===Docker containers===
Docker is one of the OS-level virtualization technologies that allows the creation of isolated virtual execution environments, called containers, sharing the same OS kernel and certain runtime services/libraries with other containers. The workload running inside a container will feel as if it had its own everything, OS kernel, root file system, etc. This is possible thanks to the way Docker containers are designed and implemented. The Linux kernel does not directly support the construction of containers (the userspace libraries do) but its various building blocks play an important role, among which we will cover namespaces in this tutorial.

===Docker images===
A Docker image is a file containing a root file system as well as configuration parameters. It is very similar to other disk image formats in this sense. One of the main differences is that it employs a layered file system to maximize storage efficiency and allow flexible reuse of existing files.

===Dockerfile===
A Dockerfile is a text file and serves as instructions to build a Docker image. Unlike storing and distributing binary data directly in images, using Dockerfiles can save space and rebuild the Docker image by always downloading the latest version of software.

===Docker Registry===
The Registry is a server-side application containing and managing repositories of Docker images. We use Docker Hub (among other choices by different providers) to pull images from in this tutorial.

===Namespaces===
Namespaces allow for multiple “spaces” where identical names can coexist. For example, the same process ID (PID) may refer to completely different processes. This is like how we can distinguish Tom (PID) in Ottawa (namespace) from Tom in Toronto (namespace). Namespaces are enforced by the OS kernel. They are the fundamental building block of containers.

A few namespaces we will cover here: Mount (<tt>mnt</tt>), Network (<tt>net</tt>), User ID (<tt>uid</tt>) and Process ID (<tt>pid</tt>).

You can use the command <tt>lsns</tt> to see the namespaces the current process is in. A namespace ID is actually an '''inode'''.

Also, the command <tt>nsenter</tt> allows you to run programs with namespaces of another process. Use the <tt>-t</tt> option to specify the target PID. For instance, <tt>sudo nsenter -t 2772 -m ls /</tt> will list the root file system of the mount namespace of process 2772.

==Tasks/Questions==
The purpose of the following questions and tasks is to help you understand how containers work, which is an important artifact on top of and based on modern operating systems. The focus is on the OS building blocks – how it is made possible by the OS. Docker is just a typical and popular example to facilitate understanding.

'''You need to <tt>sudo</tt> for all the <tt>docker</tt> commands below'''.

Install Docker with <tt>sudo apt install docker.io</tt> if it’s not already installed.

# Use <tt>docker images</tt> and <tt>docker ps</tt> (similar to the purpose of <tt>ps</tt>) to see what images you have and whether any containers are running. Run your first container with <tt>docker run hello-world</tt>. Now, check the images and <tt>docker ps</tt> again, what do you see and <u>why</u>?
# Look at the hello-world image above and you may wonder how the tiny-sized image forms a full container. Find out where the files are located (hint: using the command <tt>docker inspect</tt> and looking for paths starting with <tt>/var/lib/docker</tt>). What is the size of the found file? Run the file directly. <u>What do you think</u> is the way the tiny-sized image forms the full container?
# <tt>docker ps -a</tt> will show something different. Next, let’s try an image of a bit more complexity that will remain running. Try the centos (a Linux distribution) image: <tt>docker run -d centos sleep infinity</tt> (<tt>man docker-run</tt> for the meaning of the options).
# Start a new shell: <tt>docker exec -it</tt> <CONTAINER ID> <tt>/bin/bash</tt> (you can find out the container ID with <tt>docker ps -a</tt>. Note that you can specify just the first few digits of the ID as long as it’s distinctive). Explore the file system therein.
#; Next, we will examine the namespaces.
# Compare the mount namespaces (different views of the file system structure) between the centos container and the host, using <tt>ls -l /proc/self/ns/mnt</tt>. Are they the same and if not, what are they (format: <tt>mnt:[????]</tt>)? In particular, take a look at the root mount point (“<tt>/</tt>”) inside and outside the container (<tt>mount | grep "on / "</tt>). You can always use two sessions (one in the container and one in the host) or <tt>tmux</tt> as you feel convenient.
# With the <tt>lsns</tt> command, you can get an overview of all namespaces. Comparing the centos container and the host, <u>which namespaces are the same (shared) and which are different</u>?
# Inside the container, use the command <tt>id</tt> to see what user you’re logged in as (uid, gid and group). Then <tt>ps -eo uid,gid,pid,command | grep sleep</tt> inside the container and outside the container respectively. Are their outputs the same?
# Using the pid of “sleep” you have got outside the container above, find its pid namespace (<tt>sudo ls -l /proc/<pid>/ns/pid</tt>, format <tt>pid:[????]</tt>). Is it consistent with the <tt>lsns</tt> output inside the container?
# From step 6, you have the individual namespaces used by the “sleep” container (no need to go inside the container again). Let’s check the network namespace (<tt>sudo nsenter -t </tt><pid><tt> -n ip addr</tt>).
#: Note that uid remapping is not enabled by default. So the user (uid) inside the container will be the same outside (which does not consider security).
# Now, make your own Docker container. Read the current [https://raw.githubusercontent.com/docker-library/hello-world/7ecae6978055d2fb6960e2a29d24a2af612e2716/amd64/hello-world/Dockerfile Dockerfile of hello-world]. Change it in a way that it is no longer from scratch but ''from <tt>hello-world </tt>''. Here we can avoid defining <tt>csimpleshell</tt> as the default command as it works with a terminal for interaction (then it will inherit <tt>CMD</tt> from <tt>hello-world</tt>). Build a new image called <tt>csimpleshell</tt> by doing the following:
#: <tt>mkdir anyname</tt>
#: copy your <tt>csimipleshell</tt> and the Dockerfile of <tt>hello-world</tt> into this directory
#: <tt>cd anyname</tt>
#: make necessary changes to the Dockerfile
#: <tt>docker build -t csimpleshell .</tt>
# Run the new csimpleshell container (e.g., <tt>docker run -it csimpleshell /csimpleshell</tt>). Depending on <tt>how you compiled csimpleshell</tt>, check whether the container runs or if it does whether csimpleshell works as before. Try compiling csimpleshell statically (if it was dynamic) and repeat the steps.
# Edit the Dockerfile again and make it from “centos” instead of “hello-world”. Remove any reference to hello. Rebuild and run it. How is csimpleshell now? <u>Why</u> is there such a difference?

COMP3000 Operating Systems W25: Tutorial 8

2025-03-07T06:01:04Z

Lianyingzhao: Created page with "In this tutorial, you’ll be learning about how virtual addresses are mapped to physical addresses (the address translation) and continue to use kernel modules to extract information that only the kernel has access to. In particular, the kernel module performs a 5-level page table walk to find out the physical address corresponding to a userspace virtual address. In addition to what was discussed in the class, You can also read [https://en.wikipedia.org/wiki/Intel_5-lev..."

In this tutorial, you’ll be learning about how virtual addresses are mapped to physical addresses (the address translation) and continue to use kernel modules to extract information that only the kernel has access to. In particular, the kernel module performs a 5-level page table walk to find out the physical address corresponding to a userspace virtual address. In addition to what was discussed in the class, You can also read [https://en.wikipedia.org/wiki/Intel_5-level_paging more about 5-level paging], if interested.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Introduction==
WARNING: The commands and programs in this tutorial are potentially extremely dangerous and may result in crashes or loss of data. Additionally, questions may not work as expected on a machine other than the course VM. For that reason, you are strongly encouraged to do this tutorial on the provided OpenStack or Virtualbox virtual machine.

To get started, we will first examine the source code for <tt>3000physicalview.c</tt> and <tt>3000memview2.c</tt>, both of which are in [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut8/3000physicalview.tar.gz 3000physicalview.tar.gz].

==Tasks part A: Getting Started==
Compile <tt>3000physicalview</tt> and <tt>3000memview2</tt> using the provided <tt>Makefile</tt> (i.e., by running <tt>make</tt>).

Insert <tt>3000physicalview</tt> by running <tt>make insert</tt>. Confirm that the module is inserted using <tt>lsmod</tt>.
# Examine the call to <tt>copy_from_user()</tt> and <tt>copy_to_user()</tt> on lines 120 and 132 of <tt>3000physicalview.c</tt> (as also discussed in the lecture). Consider the following:
#* How are these functions different from <tt>get_user()</tt>/<tt>put_user()</tt> that we have seen in the previous tutorial?
#* Why are these functions necessary? Couldn't we just access the userspace address directly? What could happen if we did?
# <tt>3000physicalview</tt> exposes its API to userspace in the form of an <tt>ioctl(2)</tt> call. Consider the following:
#* What is an ioctl? How is it different from a read or write system call? Hint: check <tt>man 2 ioctl</tt>.
#* How does <tt>3000physicalview</tt> implement its ioctl? What arguments does it take?
#* How does <tt>3000memview2</tt> call the ioctl? What arguments does it pass to the ioctl?
# Which function does the virtual-to-physical translation? What type is <tt>current->mm</tt>? How does it give you the address of the <tt>pgd</tt>? Recall the page table walk explained in the lecture and see how it’s reflected in this function (writing it down is optional).
# Once you’ve got the page frame number (pfn), how is the physical address (phys) calculated? Describe in words.

==Tasks part B: Examining Physical Memory Mappings==
# With <tt>3000physicalview</tt> inserted, run <tt>3000memview2</tt> and examine the output. Note that it presents virtual memory addresses on the left, and physical addresses on the right. Are these mappings consistent with what you expected (e.g., in terms of patterns)?
# Compare <tt>3000memview2</tt> with <tt>3000memview</tt> from Tutorial 2. What is similar about their code, and what is different? How similar is their output?
# Do you notice a pattern in the virtual addresses of <tt>buf[i]</tt>? Is this same pattern present in the physical addresses? Why or why not?
# Run <tt>3000memview2</tt> a few more times and consider the following:
#* Are the virtual addresses the same or different between runs? How about physical addresses?
#* Some physical addresses don't seem to be changing between runs. Which ones? Why do you think this might be the case?
# Force the kernel to drop the virtual memory cache using <tt>sync && echo 3 | sudo tee /proc/sys/vm/drop_caches</tt>. Run <tt>3000memview2</tt> one more time and note that the physical addresses that stayed the same previously have now changed. What do you think just happened?

==Tasks part C: Using <tt>bpftrace</tt> to Monitor <tt>3000physicalview</tt> and <tt>3000memview2</tt>==
Now recall what you tried with eBPF in Tutorial 7. We can also do something similar to watch the interaction between <tt>3000physicalview</tt> and <tt>3000memview2</tt>.
Note: No in-depth understanding of eBPF is expected in this course. You only need to understand what is involved and discussed. Feel free to read more if interested.

<ol>
<li>Use the following "one-liner" (as it is just a one-line string within the single quotation marks) in a new terminal session first and in the orginal one run <tt>./3000memview2</tt> as above:

<p>

<tt>sudo bpftrace -e 'tracepoint:syscalls:sys_enter_ioctl { printf("%s: fd=%d; cmd=%d; arg=%ld \n", comm, args->fd, args->cmd, args->arg); }' | grep 3000memview2</tt>

</p>

What does this one-liner do? As always, you can check <tt>man bpftrace</tt> if needed.</li>
<li>(<b>Optional</b> in the submission but you should do it) It seems the command above does not show useful information as the third argument is actually a pointer. So, to be able to show the two arguments <tt>virt</tt> and <tt>phys</tt> defined in <tt>3000physicalview.h</tt>, you will need to create a *.bt file.
Refer to <tt>/usr/sbin/*.bt</tt> and convert the one-liner above into something like <tt>snoop3000physicalview.bt</tt>, so that when it runs it prints the <tt>virt</tt> passed in and the <tt>phys</tt> that is returned.
The code will be posted and explained.
<br> Note: if you choose to directly include the header file you need to create a copy and remove the <tt>MODULE_*</tt> macros. Otherwise, you can simply include the struct definition.</li>
</ol>

COMP3000 Operating Systems W25: Tutorial 7

2025-03-03T03:12:54Z

Lianyingzhao: Created page with "In this tutorial you'll be learning about Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface. You can also have a better understanding of special files by doing so. ==General Instructions (same for all tutorials)== <div class="mw-collapsible mw-collapsed"> Click on Expand to view. <div class="mw-..."

In this tutorial you'll be learning about Linux kernel modules, part of which are device drivers. You’ll create several kernel modules and see how to interact with device drivers using special files (e.g., character device) and their file operations as an interface. You can also have a better understanding of special files by doing so.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Important Tips==
* There is a chance your VM will crash or your ssh session will become unresponsive when messing with the kernel. To recover, you can reboot your VM from the OpenStack web console.
* Definitely do not under any circumstances attempt this on your own system. This is your last warning.
* Remember to recompile and insert your module after making changes.
* Work incrementally and be prepared for your system to crash.
* It might be wise to code on your own computer and use <tt>scp</tt> or <tt>sshfs</tt> to transfer the files to your VM.
* https://elixir.bootlin.com/linux/latest/source is a great resource for learning about kernel functions and data structures.

Note: although it's highly recommended that you use the openstack instance, if you cannot for a reason, or if you encounter other issues, check out [[COMP3000 Operating Systems W25: Tutorial 7 Not Using SCS OpenStack | these instructions]] to see if they help.

==Special Files==
Before you start, review what you have learned about special files and think about the following questions (no need to answer them): 1) What does a special file represent and what backs it up? 2) Can you have multiple special files that are somehow "the same"? What does it mean to "copy" a special file? 3) How are named pipes different from unnamed pipes and what are their similarities? Take this opportunity to deepen your understanding of special files to prepare you for working with device drivers that support such special files.

===Tasks part A: Understanding special files===
# Try the following commands as a non-privileged user. What does each do? How do the files f1-f3 compare? How do they compare to <tt>/dev/urandom</tt>? Remember you can get output from <tt>/dev/urandom</tt> using <tt>cat</tt> or <tt>dd</tt>. If you use <tt>cat</tt>, make sure to pipe it to <tt>less</tt>!
#* <tt>cp /dev/urandom f1</tt> (hit Ctrl+C right away to avoid a huge file)
#* <tt>sudo cp -a /dev/urandom f2</tt>
#* <tt>sudo mknod f3 c 1 9</tt>
#: Note: If you see any command running for more than a few seconds, hit Ctrl+C right away, check the produced file (often it’s very large) and delete it if not needed.
# Make named pipes using <tt>mknod</tt> and <tt>mkfifo</tt>. Use them to simulate <tt>ls | wc</tt> using just the '>' and '<' operators.
# Use <tt>mknod</tt> to make a copy of your current terminal's tty - <tt>/dev/tty</tt> (Hint: consider how you did something similar with <tt>/dev/urandom</tt> above). You can name it <tt>mytty</tt>. Examine its characteristics using <tt>stty --file=mytty</tt>. Do the same for the original tty.
# Run <tt>stty --help</tt> to see what you can do with stty. Try disabling local echo. How does the shell behave with echo disabled? How can you restore echo without logging out and back in? Recall we did this in the class.

==Kernel Modules==
In this part of the tutorial you will be building and installing kernel modules. You will need root access to install kernel modules.

It is '''highly recommended''' that you use a comp3000 openstack instance for the exercises below for two reasons. First, you may have difficulties compiling kernel modules on other systems. Second, these operations are potentially dangerous, and mistakes could '''destroy all data''' on the Linux system. Consider yourself warned!

Before you start, review what you have learned about kernel modules and device drivers, and think about the following questions (no need to answer them): 1) How does the source of kernel modules differ from C programs? 2) How does building kernel modules differ from building userspace C programs? 3) How does invoking the file operations (e.g., read/write) of a special file trigger corresponding functions in a device driver?

You will learn how you can access and manipulate kernel data structures and call kernel functions in a kernel module. You will also understand how processes are represented by <tt>task_struct</tt>'s in the Linux kernel, what kind of information is stored in a task struct, and how to access the <tt>task_struct</tt> of a process.

===Tasks part B: A simple kernel module===
Download the source for this [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut7/simple.tar.gz simple module], unpack, and build it by typing "<tt>make</tt>". You can safely ignore the warning of "Skipping BTF generation..." if encountered.
# Install the module using "<tt>sudo insmod simple.ko</tt>". The hello message is recorded in the kernel logs. How do you view the kernel logs? How many ways are there to view them?
# Check to see that the module has been loaded. How do you do this?
# Remove the module from the kernel. What did you do?

===Tasks part C: A character device kernel module (driver)===
Download the source for [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut7/ones.tar.gz ones], a kernel module implementing a character device that outputs an unbounded string of "1"'s. Build, compile, and run it as before.
# What kernel messages does the module generate? Does it create any new files (other than <tt>/dev/ones</tt>)? If so, where? (Hint: search in <tt>/sys</tt>)
# What happens when you "cat" the device <tt>/dev/ones</tt>? How can you limit the output in more than one way?
# Add more <tt>printk()</tt>’s (where you see needed) to find out at which point which functions in ones.c are called and for which purpose.

===Tasks part D: Getting process information from a module===
Download the source [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut7/newgetpid.tar.gz newgetpid.c]. Build and run it as before. Hints for the questions below:

https://elixir.bootlin.com/linux/latest/source/include/linux/sched.h,
https://elixir.bootlin.com/linux/latest/source/arch/x86/include/asm/current.h,
https://elixir.bootlin.com/linux/latest/source/include/linux/cred.h
# What type is "<tt>current</tt>"? <u>How</u> can you figure this out?
# Modify newgetpid.c so that it creates a device file <tt>/dev/describe</tt> rather than <tt>/dev/newgetpid</tt>.
# Make <tt>/dev/describe</tt> output the calling process's parent ID (ppid), user ID (uid), group ID (gid), effective user ID (euid), and effective group ID (egid).

==Using <tt>bpftrace</tt> to monitor kernel module events==
Note: you are not required to write about your experience for this part in the submitted work, but you should still do it, which can help you understand eBPF better (see how simple but still powerful an eBPF program can be).

Download this tiny code [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut7/kmsnoop.bt kmsnoop.bt] and make it executable with <tt>chmod 755 kmsnoop.bt</tt>.

Now, you can open a separate SSH session and run it with <tt>sudo ./kmsnoop.bt</tt>. It will wait for events. When you do any "insmod" or "rmmod" in the other session, you will see it outputs a line of message.

If you want to avoid opening another SSH window, you can use <tt>tmux</tt> (which we learned in Tutorial 1).

How could this save you from doing any <tt>printk</tt>'s in your kernel module code?
You can also <tt>sudo bpftrace -l | grep</tt> for your needed kernel functions (e.g.,<tt>printk</tt>) to hook to.

COMP3000 Operating Systems W25: Tutorial 7 Not Using SCS OpenStack

2025-03-03T02:34:41Z

Lianyingzhao: Created page with "If you are not using our course VM for some reason, you might encounter problems and you can refer to the instructions below. Although they are by no means comprehensive to solve the problems, they can in many cases and serve as pointers about where to look. ==Modules fail to build== If you see errors (not just warnings) after running <tt>make</tt> and no <tt>.ko</tt> file is produced, you may have installed a version of Ubuntu that is too minimal. But you can fix it by..."

If you are not using our course VM for some reason, you might encounter problems and you can refer to the instructions below. Although they are by no means comprehensive to solve the problems, they can in many cases and serve as pointers about where to look.

==Modules fail to build==
If you see errors (not just warnings) after running <tt>make</tt> and no <tt>.ko</tt> file is produced, you may have installed a version of Ubuntu that is too minimal. But you can fix it by installing the right packages. Do the following:
sudo apt update

sudo apt dist-upgrade

sudo apt install build-essential

sudo apt clean

If your build failed before doing this and again after, delete the downloaded code and unpack the zip file again.

==Disk full==
If you find your disk space filled up, you can use the following command to pinpoint the top 20 largest files (no need to understand the specifics unless you are curious):

sudo find / -type f -exec du -h {} 2>&- + | sort -rh | head -20

Or including directories:

sudo du -h / 2>&- | sort -rh | head -20

==Command 'bpftrace' not found==
You may be able to fix this by installing the <tt>bpftrace</tt> package (just the frontend and the examples):

sudo apt-get install bpftrace

Use its equivalent on other Linux distributions.

Note: it could be much worse than this, e.g., not having the right kernel compiled with eBPF support, for which there won't be a quick fix.
In such cases, you are suggested to move to the course VM or choose a different VM of your own.

If you see an error like "<tt>Could not resolve symbol</tt>" when using <tt>bpftrace</tt>, you need to apply a fix by installing the dbgsym package of <tt>bpftrace</tt>: Just <tt>wget</tt> [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut7/install-bpftrace-dbgsym this file], and run it with

sudo apt update
source install-bpftrace-dbgsym

COMP3000 Operating Systems W25: Tutorial 6

2025-02-21T03:25:54Z

Lianyingzhao: Created page with "In this tutorial you will be learning about two implementations of the [https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem producer-consumer problem], a classic example of a concurrency problem. The [http://pages.cs.wisc.edu/~remzi/OSTEP/ class textbook] covers concurrency in great detail in Chapters 25-34, and the producer-consumer problem is covered in [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf Chapter 30 (Condition Variables)] and [http://pages..."

In this tutorial you will be learning about two implementations of the [https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem producer-consumer problem], a classic example of a concurrency problem. The [http://pages.cs.wisc.edu/~remzi/OSTEP/ class textbook] covers concurrency in great detail in Chapters 25-34, and the producer-consumer problem is covered in [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf Chapter 30 (Condition Variables)] and [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-sema.pdf Chapter 31 (Semaphores)]. While you can look at this part of the textbook, note that we will not be covering this material in the same level of detail, as should be clear from this tutorial.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==A: Getting Started==
Download [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut6/3000pc.zip 3000pc.zip], unpack, and run <tt>make</tt> to compile <tt>3000pc-fifo</tt> and <tt>3000pc-rendezvous</tt>.

Try to take this opportunity to read and understand the two programs line by line.

:: Note that these programs take 3 arguments each:
::* The number of events to process
::* The number of events to produce before the producer sleeps for 1 second
::* The number of events to consume before the consumer sleeps for 1 second

# Run both programs with the same arguments of your choice. Do this a few times with different arguments. Do they behave the same way? Do you notice <u>any differences</u>? (again, no “correct” answers but just play with them and document your observations)
# Repeat the above experiment, this time running each program under <tt>strace</tt> (with the <tt>-f</tt> flag to trace children too). Do you notice any difference in the system calls each program makes? Remember to ignore the lines before <tt>main()</tt> that might be distracting, e.g., start looking around and after the <tt>clone()</tt> system call.
# You can see there is "<tt>#define QUEUESIZE 32</tt>" in both programs. Are they actually used? <u>Why (not)</u>?

==B: Producer/Consumer with Pipes==
<ol style="list-style-type:decimal">
<li> Examine the source code of <tt>3000pc-fifo</tt>. Pay attention to the call to <tt>pipe(pipefd)</tt> on line 192 and make the connection with what has been discussed in the lecture (also look at the man page for <tt>pipe(2)</tt>). Explain the following:
<ol style="list-style-type:lower-alpha">
<li> How does the consumer receive words to consume? </li>
<li> How does the producer generate and send words to the consumer? </li>
<li> Why is the call to <tt>srandom(time(NULL))</tt> on line 169 necessary? </li>
</ol>
</li>
<li> Replace the call to <tt>srandom(time(NULL))</tt> on line 169 with <tt>srandom(42)</tt>. What differences do you notice in <tt>3000pc-fifo</tt>'s behavior? </li>
</ol>

==C: Producer/Consumer with Shared Memory==
<ol style="list-style-type:decimal">
<li> Examine the source code of <tt>3000pc-rendezvous</tt>. Explain the following:
<ol style="list-style-type:lower-alpha">
<li> What are a few ways that <tt>3000pc-rendezvous</tt> is different from <tt>3000pc-fifo</tt>? </li>
<li> What does the call to <tt>mmap()</tt> on line 347 do (compared to our previous uses of <tt>mmap()</tt>)? <u>How</u> did you figure this out? </li>
<li> How does the producer notify the consumer that the queue is no longer empty? </li>
<li> How does the consumer notify the producer that the queue is no longer full? </li>
</ol>
</li>
<li> What arguments can you provide to make the producer wait for the consumer? Hint: Check the size of the queue. </li>
<li> What arguments can you provide to make the consumer wait for the producer? (You can disregard the wait at the very beginning as the queue is initially empty) </li>
<li> Another student tells you that the difference between processes and threads is that processes never share memory, while threads do. Is this statement correct or incorrect? How can the behavior of <tt>3000pc-rendezvous</tt> help justify your answer? </li>
<li> Change the calls to <tt>pthread_mutexattr_setpshared()</tt> and <tt>pthread_condattr_setpshared()</tt> (lines 293 and 298) to take 0 instead of 1. How does the behavior or <tt>3000pc-rendezvous</tt> change? Does anything break? </li>
</ol>

COMP3000 Operating Systems W25: Tutorial 5

2025-01-30T07:11:55Z

Lianyingzhao: Created page with "In this tutorial you will be learning about files and filesystems by experimenting with and extending [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut5/3000test.c 3000test.c], and creating and manipulating local filesystems. '''WARNING:''' Several of the commands here can lead to system corruption and data loss if not properly used. You have been warned. Please use a VM and make backups, when necessary. ==General Instructions (same for all tutorials)== <d..."

In this tutorial you will be learning about files and filesystems by experimenting with and extending [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut5/3000test.c 3000test.c], and creating and manipulating local filesystems. '''WARNING:''' Several of the commands here can lead to system corruption and data loss if not properly used. You have been warned. Please use a VM and make backups, when necessary.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Files and inodes==
In UNIX/Linux filesystems, a filename does not directly refer to the contents of a file. Instead, a filename refers to an inode (as specified in a directory entry --- dentry, of its parent directory), and the inode then refers to the data. An inode is where file metadata is stored. File ownership, timestamps, and permissions are all stored in a file's inode. Files listed by the <tt>ls</tt> command are just '''hard links''' connecting a file name to an inode.

In addition to regular files, we also have symbolic links, directories, block devices, character devices, and pipes. Each is its own type of inode.

Note that while you can find out a file's inode, you cannot go from an inode to a pathname or otherwise manipulate an inode directly from userspace - you always have to go through the pathname. The kernel, however, can access individual inodes directly (indeed it has to in order to get to the contents of a file when given a filename).

3000test.c uses <tt>stat()</tt> to give information on a given inode.

==Tasks/Questions (Part A)==
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut5/3000test.c 3000test.c]. It takes a filename as an argument and reports information on the file. Try giving it the following and see what it reports:
#* a regular text file that exists
#* a directory
#* a symbolic link
#* a device file (character or block)
# Change 3000test to use <tt>lstat()</tt> rather than stat. How does its behavior change?
# Modify 3000test so when it is given a symbolic link it reports the name of the target. Use <tt>readlink(2)</tt> (this notation describes how to access the man page, e.g., <tt>man 2 readlink</tt>).
# Are there files or directories that you cannot run 3000test on? Can you configure file/directory permissions so as to make something inaccessible to 3000test? Note that this is twofold: 1) whether it is completely inaccessible to 3000test ('''nothing can be displayed''', except the error msg); 2) whether it can be accessed for the search ('''no access to file content''').

==Creating, Mounting, and Unmounting Filesystem==
In UNIX/Linux systems we always have a single hierarchical namespace for files and directories. Pathnames that start with a <tt>/</tt> are absolute, starting at the top of the namespace, while ones that do not start with a / are relative to the current directory.

The files in this single namespace come from many sources, each of which can have very different semantics. These sources are known as '''filesystems'''.

When first started, a system must have at least one filesystem. This first filesystem is known as the root filesystem and has the contents of <tt>/</tt>. We can then add other filesystems to this initial set of files by '''mounting''' each filesystem on an existing directory.

On our current Ubuntu, the root filesystem is ext4 by default and many others are mounted on top of it. For example, a proc filesystem is mounted on <tt>/proc</tt>, a sysfs is mounted on <tt>/sys</tt>. If you run "mount" with no arguments, it lists all the currently mounted filesystems. Note that most of these are special (aka. virtual) filesystems, in that they do not correspond to any persistent storage (like a disk).

If you insert a USB stick on a laptop or desktop running Linux, it must be mounted before you can access its contents. The current convention is for it to be mounted in <tt>/media/</tt><mounting user><tt>/</tt><name of USB stick filesystem>. This usually happens automatically nowadays when the USB stick is plugged in.

To create an ext4 filesystem, you just run

mkfs.ext4 <writable block device> <span style="color:#ff0000">#Do not try it with any <tt>/dev/*</tt></span>

If you were to run this on the root filesystem of your current system, it would reformat your entire disk if it was allowed to complete. In all likelihood the system should prevent you from doing this as the root filesystem (and any mounted filesystem) gets special protections.

To mount a filesystem, you specify the filesystem to be mounted (by its type or by the device) and the mountpoint:

mount /dev/sdb1 /mnt <span style="color:#ff0000">#Do not try it unless you have that block device and want to do it</span>

This would mount the first partition of the second disk on <tt>/mnt</tt>.

Filesystems can be stored on block devices - devices whose contents are accessed by specifying a block index/address. The other main type of device file, character devices, are used for devices where the data is accessed a single byte (character) at a time. Terminals, modems, printers – for example, will generally be represented as a character device.

Note that mount here can identify the filesystem on <tt>/dev/sdb1</tt> by looking at its superblock. The superblock is usually either in block 0 or 1 of a device. The superblock holds metadata of a filesystem, allowing it to be identified and specified. Without the superblock you cannot do anything with a filesystem. Because of its importance, there are backup superblocks in addition to the primary one. The blocks of a filesystem can be classified as either being superblocks, inode blocks, or data blocks.

Sometimes we want to play with a new filesystem but we don't have a physical device to format. If we have free space for a file in our current filesystem, however, we can put a filesystem in a file by using a loopback block device. A loopback block device is a block device where its data is stored in a regular file on another filesystem (rather than on a separate device). If you do filesystem commands with a regular file, it will transparently associate the file with an available loopback block device (e.g., <tt>/dev/loop1</tt>).

Note that inode numbers are filesystem-specific. Thus, the contents of a file are uniquely specified by its filesystem and inode. Only the kernel has to worry about this level of detail; in userspace, a file's full path contains all the necessary information.

Filesystems on persistent storage are always at risk of corruption. File system checker programs (<tt>fsck</tt>) can detect and repair filesystem errors.

==Tasks/Questions (Part B)==
# Run <tt>dd if=/dev/zero of=foo bs=8192 count=32K</tt> and use the <tt>ls</tt> command to check it. What is the logical size of the file foo? How much space does it consume on disk? (Hint: Look at the size option to ls). In comparison, create another file with <tt>dd if=/dev/zero of=foo2 bs=8192 seek=31K count=1K</tt>. Any observations?
# Run <tt>mkfs.ext4 foo</tt>. Does foo consume any more space or less? Do the same (<tt>mkfs.ext4 foo2</tt>) to the other file and answer the same question.
# Create any file in <tt>/mnt</tt> (e.g., <tt>sudo touch test.txt</tt>) and run <tt>sudo mount foo /mnt</tt>. Do you still see the file you just created?
# Run <tt>df</tt>. What device is mounted on <tt>/mnt</tt>? What is this device?
# Run <tt>sudo umount /mnt</tt>. What have gone away and what is back?
# Run <tt>dd if=/dev/zero of=foo conv=notrunc count=10 bs=512</tt>. How does the "<tt>conv=notrunc</tt>" change dd's behavior (versus the command in question 1)?
# Run <tt>sudo mount foo /mnt</tt>. What error do you get?
# What command can you run to make foo mountable again? What characteristic of the file system enables this command to work? (Hint: revisit the instructions of this tutorial if you have no idea)

COMP3000 Operating Systems W25: Tutorial 4

2025-01-24T05:23:58Z

Lianyingzhao: /* Tasks/Questions */

In this tutorial, you will learn about how user accounts are managed and how the login process works through exploring [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut4/3000userlogin.c 3000userlogin.c]. You’ll also have a better understanding of the permission system and the shell/terminal environment.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Logging in to a Linux system==
As mentioned in class, in order to log in to a Linux-like system (including UNIX), the following steps must occur (potentially not in this order).
# The user must use a terminal to connect to the system, locally or remotely (e.g., initiated by <tt>/usr/sbin/getty</tt>)
# The user must authenticate themselves, proving an identify with authorized access to the system. By default, this is done through a username and password (e.g., by exec’ing <tt>/usr/bin/login</tt>).
# After authentication, login changes uid and gid to that of the new user.
# Login sets up other aspects of the user's context (mainly setting key environment variables).
# Login does an exec of the user's chosen/default shell.

[https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut4/3000userlogin.c 3000userlogin] is a basic implementation of steps 3-5. (Steps 1 and 2 are accomplished by running 3000userlogin in a shell, as running an external command means the shell also creates a new process for it, assuming an authenticated session with a terminal.)

When 3000userlogin is properly compiled and set up, you can run:

./3000userlogin student

and you'll be logged in (again) as "student". This only works when you’re already logged in as student (see below).

You can add a user with the <tt>adduser</tt> command. For example, to create the user "someuser":

sudo adduser someuser

Note you'll have to answer several questions. Then you need to switch to someuser (<tt>su someuser</tt>).

If you just compile 3000userlogin normally, you won't be able to log in as anyone except the current user (be it student or someuser). To set up 3000userlogin properly, do the following:

sudo chown root:root 3000userlogin
sudo chmod u+s 3000userlogin

Feel free to read the <tt>man</tt> pages for both commands to learn more.

The <tt>chown</tt> command makes the binary owned by root, and the <tt>chmod</tt> command makes it setuid. Thus, when the program is exec'd it will have an effective user ID of root (euid=0). To facilitate the following tasks, you may also want to create different versions of 3000userlogin by naming them, e.g., 3000userlogin.setuid and 3000userlogin.orig, of your choice.

==Tasks/Questions==
# Compile and setup [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut4/3000userlogin.c 3000userlogin] as described above. Create a new user account. Verify that you can use 3000userlogin to login as the new user without typing the password of the new user (without logging in beforehand). Think about <u>why</u> you are able to avoid typing the password and still get authenticated.
# <u>Why</u> is the syntax highlighting gone (coloring characters)? What simple change (one line) can you make to have 3000userlogin preserve the colors? (Hint: check the environment variables)
# What is returned as the user's password by <tt>getpwnam()</tt> (in terms of data type)? Is this what you expected? Read the manual page as you find necessary.
#: Which file is accessed during this process? <u>How can you find it out</u>? (if you end up looking at another file accessed, it is also worth the time.)
# Compare the uid, gid, euid, egid when running 3000userlogin as a regular user (this means without the “chown” and “chmod” commands above), running it setuid root, and running it as root (i.e., running it from a root shell without the setuid bit set, e.g., “<tt>sudo ./3000userlogin someuser</tt>”). Also check to see what happens when you set the setgid bit.
# <u>Why</u> does 3000userlogin change its gid before changing its uid? <u>What happens</u> if you switch the order of these operations?
# You may have noticed that there are multiple mentions of "bash" (on different lines if you search). What are their purposes, respectively?
# Make 3000userlogin use the shell that is specified in the user's password entry (which is actually read from <tt>/etc/passwd</tt>). Check by setting someuser’s shell to a new shell and then see if that new shell runs when you run 3000userlogin. You can change a user's shell with the <tt>chsh</tt> command.
# Can you set 3000shell to be a user's default shell? What changes do you have to make for <tt>chsh</tt> to accept 3000shell? Does anything obvious break when running 3000shell this way, and how can you change 3000userlogin to fix it?
# Does a user's default shell have to be a regular shell? Could it instead be an arbitrary program? <u>How do you know</u>?
#: <b>CAUTION:</b> do not try with the user student (or if you choose to give it a try, you must change it back before logging out or you could be locked out having to turn to IT staff or the instructor for help). The best way is to play with the default shell of someuser.
# Are the environment variables set by 3000userlogin the only environment variables that are set (visible) after you successfully login? <u>How do you know</u>?
# Further to Task 3 above, if the returned data is not what you expected, which other function (very similar to <tt>getpwnam()</tt>) can you use, so that you will be able to actually check the user's password?

COMP3000 Operating Systems W25: Tutorial 4

2025-01-24T05:23:30Z

Lianyingzhao: /* Tasks/Questions */

In this tutorial, you will learn about how user accounts are managed and how the login process works through exploring [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut4/3000userlogin.c 3000userlogin.c]. You’ll also have a better understanding of the permission system and the shell/terminal environment.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Logging in to a Linux system==
As mentioned in class, in order to log in to a Linux-like system (including UNIX), the following steps must occur (potentially not in this order).
# The user must use a terminal to connect to the system, locally or remotely (e.g., initiated by <tt>/usr/sbin/getty</tt>)
# The user must authenticate themselves, proving an identify with authorized access to the system. By default, this is done through a username and password (e.g., by exec’ing <tt>/usr/bin/login</tt>).
# After authentication, login changes uid and gid to that of the new user.
# Login sets up other aspects of the user's context (mainly setting key environment variables).
# Login does an exec of the user's chosen/default shell.

[https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut4/3000userlogin.c 3000userlogin] is a basic implementation of steps 3-5. (Steps 1 and 2 are accomplished by running 3000userlogin in a shell, as running an external command means the shell also creates a new process for it, assuming an authenticated session with a terminal.)

When 3000userlogin is properly compiled and set up, you can run:

./3000userlogin student

and you'll be logged in (again) as "student". This only works when you’re already logged in as student (see below).

You can add a user with the <tt>adduser</tt> command. For example, to create the user "someuser":

sudo adduser someuser

Note you'll have to answer several questions. Then you need to switch to someuser (<tt>su someuser</tt>).

If you just compile 3000userlogin normally, you won't be able to log in as anyone except the current user (be it student or someuser). To set up 3000userlogin properly, do the following:

sudo chown root:root 3000userlogin
sudo chmod u+s 3000userlogin

Feel free to read the <tt>man</tt> pages for both commands to learn more.

The <tt>chown</tt> command makes the binary owned by root, and the <tt>chmod</tt> command makes it setuid. Thus, when the program is exec'd it will have an effective user ID of root (euid=0). To facilitate the following tasks, you may also want to create different versions of 3000userlogin by naming them, e.g., 3000userlogin.setuid and 3000userlogin.orig, of your choice.

==Tasks/Questions==
# Compile and setup [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut4/3000userlogin.c 3000userlogin] as described above. Create a new user account. Verify that you can use 3000userlogin to login as the new user without typing the password of the new user (without logging in beforehand). Think about <u>why</u> you are able to avoid typing the password and still get authenticated.
# <u>Why</u> is the syntax highlighting gone (coloring characters)? What simple change (one line) can you make to have 3000userlogin preserve the colors? (Hint: check the environment variables)
# What is returned as the user's password by <tt>getpwnam()</tt> (in terms of data type)? Is this what you expected? Read the manual page as you find necessary.
#: Which file is accessed during this process? <u>How can you find it out</u>? (if you end up looking at another file accessed, it is also worth the time.)
# Compare the uid, gid, euid, egid when running 3000userlogin as a regular user (this means without the “chown” and “chmod” commands above), running it setuid root, and running it as root (i.e., running it from a root shell without the setuid bit set, e.g., “<tt>sudo ./3000userlogin someuser</tt>”). Also check to see what happens when you set the setgid bit.
# <u>Why</u> does 3000userlogin change its gid before changing its uid? <u>What happens</u> if you switch the order of these operations?
# You may have noticed that there are multiple mentions of "bash" (on different lines if you search). What are their purposes, respectively?
# Make 3000userlogin use the shell that is specified in the user's password entry (which is actually read from <tt>/etc/passwd</tt>). Check by setting someuser’s shell to a new shell and then see if that new shell runs when you run 3000userlogin. You can change a user's shell with the <tt>chsh</tt> command.
# Can you set 3000shell to be a user's default shell? What changes do you have to make for <tt>chsh</tt> to accept 3000shell? Does anything obvious break when running 3000shell this way, and how can you change 3000userlogin to fix it?
# Does a user's default shell have to be a regular shell? Could it instead be an arbitrary program? <u>How do you know</u>?
<b>CAUTION:</b> do not try with the user student (or if you choose to give it a try, you must change it back before logging out or you could be locked out having to turn to IT staff or the instructor for help). The best way is to play with the default shell of someuser.
# Are the environment variables set by 3000userlogin the only environment variables that are set (visible) after you successfully login? <u>How do you know</u>?
# Further to Task 3 above, if the returned data is not what you expected, which other function (very similar to <tt>getpwnam()</tt>) can you use, so that you will be able to actually check the user's password?

COMP3000 Operating Systems W25: Tutorial 4

2025-01-24T05:22:31Z

Lianyingzhao: Created page with "In this tutorial, you will learn about how user accounts are managed and how the login process works through exploring [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut4/3000userlogin.c 3000userlogin.c]. You’ll also have a better understanding of the permission system and the shell/terminal environment. ==General Instructions (same for all tutorials)== <div class="mw-collapsible mw-collapsed"> Click on Expand to view. <div class="mw-collapsible-content">..."

In this tutorial, you will learn about how user accounts are managed and how the login process works through exploring [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut4/3000userlogin.c 3000userlogin.c]. You’ll also have a better understanding of the permission system and the shell/terminal environment.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Logging in to a Linux system==
As mentioned in class, in order to log in to a Linux-like system (including UNIX), the following steps must occur (potentially not in this order).
# The user must use a terminal to connect to the system, locally or remotely (e.g., initiated by <tt>/usr/sbin/getty</tt>)
# The user must authenticate themselves, proving an identify with authorized access to the system. By default, this is done through a username and password (e.g., by exec’ing <tt>/usr/bin/login</tt>).
# After authentication, login changes uid and gid to that of the new user.
# Login sets up other aspects of the user's context (mainly setting key environment variables).
# Login does an exec of the user's chosen/default shell.

[https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut4/3000userlogin.c 3000userlogin] is a basic implementation of steps 3-5. (Steps 1 and 2 are accomplished by running 3000userlogin in a shell, as running an external command means the shell also creates a new process for it, assuming an authenticated session with a terminal.)

When 3000userlogin is properly compiled and set up, you can run:

./3000userlogin student

and you'll be logged in (again) as "student". This only works when you’re already logged in as student (see below).

You can add a user with the <tt>adduser</tt> command. For example, to create the user "someuser":

sudo adduser someuser

Note you'll have to answer several questions. Then you need to switch to someuser (<tt>su someuser</tt>).

If you just compile 3000userlogin normally, you won't be able to log in as anyone except the current user (be it student or someuser). To set up 3000userlogin properly, do the following:

sudo chown root:root 3000userlogin
sudo chmod u+s 3000userlogin

Feel free to read the <tt>man</tt> pages for both commands to learn more.

The <tt>chown</tt> command makes the binary owned by root, and the <tt>chmod</tt> command makes it setuid. Thus, when the program is exec'd it will have an effective user ID of root (euid=0). To facilitate the following tasks, you may also want to create different versions of 3000userlogin by naming them, e.g., 3000userlogin.setuid and 3000userlogin.orig, of your choice.

==Tasks/Questions==
# Compile and setup [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut4/3000userlogin.c 3000userlogin] as described above. Create a new user account. Verify that you can use 3000userlogin to login as the new user without typing the password of the new user (without logging in beforehand). Think about <u>why</u> you are able to avoid typing the password and still get authenticated.
# <u>Why</u> is the syntax highlighting gone (coloring characters)? What simple change (one line) can you make to have 3000userlogin preserve the colors? (Hint: check the environment variables)
# What is returned as the user's password by <tt>getpwnam()</tt> (in terms of data type)? Is this what you expected? Read the manual page as you find necessary.
#: Which file is accessed during this process? <u>How can you find it out</u>? (if you end up looking at another file accessed, it is also worth the time.)
# Compare the uid, gid, euid, egid when running 3000userlogin as a regular user (this means without the “chown” and “chmod” commands above), running it setuid root, and running it as root (i.e., running it from a root shell without the setuid bit set, e.g., “<tt>sudo ./3000userlogin someuser</tt>”). Also check to see what happens when you set the setgid bit.
# <u>Why</u> does 3000userlogin change its gid before changing its uid? <u>What happens</u> if you switch the order of these operations?
# You may have noticed that there are multiple mentions of "bash" (on different lines if you search). What are their purposes, respectively?
# Make 3000userlogin use the shell that is specified in the user's password entry (which is actually read from <tt>/etc/passwd</tt>). Check by setting someuser’s shell to a new shell and then see if that new shell runs when you run 3000userlogin. You can change a user's shell with the <tt>chsh</tt> command.
# Can you set 3000shell to be a user's default shell? What changes do you have to make for <tt>chsh</tt> to accept 3000shell? Does anything obvious break when running 3000shell this way, and how can you change 3000userlogin to fix it?
# Does a user's default shell have to be a regular shell? Could it instead be an arbitrary program? <u>How do you know</u>?
<b>CAUTION:</b> do not try with the user student (or if you choose to give it a try, you must change it back before logging out or you could be locked out having to turn to IT staff or the instructor for help). The best way is to plat with the default shell of someuser.
# Are the environment variables set by 3000userlogin the only environment variables that are set (visible) after you successfully login? <u>How do you know</u>?
# Further to Task 3 above, if the returned data is not what you expected, which other function (very similar to <tt>getpwnam()</tt>) can you use, so that you will be able to actually check the user's password?

COMP3000 Operating Systems W25: Tutorial 3

2025-01-18T06:36:22Z

Lianyingzhao: Created page with "In this tutorial, you will be experimenting with and extending [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut3/3000shell.c 3000shell.c], a proof-of-concept program to show you how a Linux shell works. Also, 3000shell's source code will be used as the basis for further modifications in upcoming assignments. ==General Instructions (same for all tutorials)== <div class="mw-collapsible mw-collapsed"> Click on Expand to view. <div class="mw-collapsible-conte..."

In this tutorial, you will be experimenting with and extending [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut3/3000shell.c 3000shell.c], a proof-of-concept program to show you how a Linux shell works. Also, 3000shell's source code will be used as the basis for further modifications in upcoming assignments.

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Background==
===Processes===
Each application running on a system is assigned a unique process identifier (PID). The <tt>ps</tt> command shows the process identifiers for running processes. Each process running on the system is kept separated from other processes by the operating system.

When you enter a command at a shell prompt, most of the time you are creating a new process which runs the program you specified.

===Controlling Processes===
On Linux, once the processes have been created, you can control them by sending them signals. From the terminal (e.g., command line), you send signals when you type certain key sequences in most shells: Control-C sends INT (interrupt), Control-Z sends STOP.

You can also send a signal to a process using the kill command:

kill -<signal> <process ID>

So, to stop process 4542, type:

kill -STOP 4542

By default, <tt>kill</tt> sends the <tt>TERM</tt> signal.

Each signal has both a string name (e.g., <tt>STOP</tt>) and a numeric form (e.g., 19 correspondingly). See the full list of signals with <tt>kill -l</tt>.

===Standard input/output, Shell and Terminal===
Recall that the shell is a command interpreter (program). In comparison, a terminal is a device used to enter data into and display/printing data from a computer. It used to be physical - [https://en.wikipedia.org/wiki/Teleprinter teletypes, or "TTY"'s], but today we mostly use virtual terminals, saving the need for the physical form but providing the same interface (recall virtualization). There is a special virtual terminal mechanism in Linux/UNIX referred to as pts -- pseudo teletypes, e.g., <tt>/dev/pts/1</tt> (they are just the slave side. Those who are curious can refer to <tt>man ptmx</tt>, but not needed in this tutorial).

The standard input (e.g., <tt>/dev/stdin</tt>), output (e.g., <tt>/dev/stdout</tt>) and error (<tt>/dev/stderr</tt>) are just symbolic endpoints in any programs (of which the shell is an example), that can be redirected anywhere, where output goes, input comes and error is reported. By default, they refer to file descriptors 0, 1, and 2 for the current process.

<u>So, in summary, you connect to a terminal to interact with a computer, by running a shell to receive your commands, which redirects standard in/out/err to the terminal.</u>

As you progress in this course, you will gradually find how such stdin/stdout/stderr can be useful, especially when combined with the pipeline/redirection. For instance:

ls | less #to paginate long output from ls

ps -ef | grep 3000shell #to look for the 3000shell process from the process list

gcc -O2 -o csimpleshell csimpleshell.c 2>err.txt #to redirect error messages to a file err.txt

Later in the term if you find your disk space filled up, you can use the following command to pinpoint the top 20 largest files (no need to understand the specifics unless you are curious):

sudo find / -type f -exec du -h {} 2>&- + | sort -rh | head -20

==Tasks/Questions==
The purpose of the following questions and tasks is to help you understand how 3000shell (a simple implementation of the shell, but still more complex than the previous <tt>csimpleshell</tt>) works. Eventually, you should have an understanding of every function and every line of the code. If you understand 3000shell, then you understand the basics of all Linux/UNIX shells. Your understanding of the code will be tested later, so use this opportunity to dive deep into the code.

You do not need to submit a separate file for source code changes. Just describe your changes or paste the code snippet (usually a few lines at the most).

# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut3/3000shell.c 3000shell.c].
# Compared to [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/csimpleshell.c csimpleshell] from Tutorial 1, what functionality improvements has 3000shell introduced? List at least two improvements (functional differences). You can mention whether you found them by reading the source code or trying both shells.
# Pay attention to lines 229-235. How can you run a program in the background? Note: “in the background” means the shell starting a program without waiting for it to finish. So, the shell returns immediately to be ready to accept the next command.
# Observe the behavior when running different programs in the background. What happens to the input and output of the program? Try this for different types of programs, such as <tt>ls</tt> and <tt>bc</tt>, or <tt>nano</tt> and <tt>top</tt>, or others of your choice. Write down your observations.
# You may have trouble interacting with the shell after running programs in the background. How can you recover from such a situation? Note: you can do it in a harsh way for this question. We will see a smarter way next.
# 3000shell implements a simple form of output redirection (i.e., the standard output mentioned earlier). What syntax should you use to redirect standard output to a file?
# Make the shell output "Ouch!" when you send it a <tt>SIGUSR1</tt> signal.
# If you delete line 324 (<tt>SA_RESTART</tt>), how does the behavior of 3000shell change?
# Replace the use of <tt>find_env()</tt> with <tt>getenv()</tt>. How do their interfaces differ?
# Make <tt>plist</tt> output the parent process id for each process, e.g., "5123 ls (5122)". Pay attention to the <tt>stat</tt> and <tt>status</tt> files in the per-process directories in <tt>/proc</tt>.
# If the shell gets stuck as a result of running a program in the background, how can you recover elegantly (resuming it) by sending just a signal (which signal)? <u>Why?</u>

COMP3000 Operating Systems W25: Downloading files from your Openstack VM

2025-01-16T04:08:01Z

Lianyingzhao:

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

You will need to transfer files between the Openstack VM and your own PC from time to time, for example, when submitting your tutorial/assignment work. As your laptop/desktop does not have a fixed IP the VM can connect to, it is always better to initiate the connection <b>from your PC to the VM</b>, which has a floating IP address associated. Here, you have a few options depending on your situation (assuming you have the Carleton VPN connected). Choose one that works for you or feel free to use other alternatives.

==Using <tt>scp</tt> (with Carleton VPN)==
Secure file copy uses the same protocol as the secure shell (SSH), similar to the SSH command.

scp <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

The part in blue is an example pathname of the file to submit on your VM (make sure it matches your situation) and don't forget '''the last part in red, which is the destination directory on your PC''' (e.g., if could simply be a dot "." representing your current directory, where you are typing this command).

Replace the <VMIP> with the floating IP address of your VM. This should work for both Windows and Linux-like systems.

==Using <tt>scp</tt> (without Carleton VPN)==

scp -o "ProxyJump <CarletonUserName>@access.scs.carleton.ca" <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

Or
scp -J <CarletonUserName>@access.scs.carleton.ca <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

You will first be promoted for your MC1 password set through the SCS account, then your VM's password for the student user. Replace <CarletonUserName> with your Carleton username.

==Using <tt>rsync</tt> (with Carleton VPN)==
You can also use the more secure and new command, <tt>rsync</tt> instead.

rsync -avhP <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

Replace the <VMIP> with the floating IP address of your VM. Likewise, also pay attention to the two locations (in red and blue).

==Using <tt>rsync</tt> (without Carleton VPN)==

rsync -avhP -e "ssh -A -J <CarletonUserName>@access.scs.carleton.ca" <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

COMP3000 Operating Systems W25: Downloading files from your Openstack VM

2025-01-16T04:06:49Z

Lianyingzhao: /* Using rsync */

COMP3000 Operating Systems W25: Downloading files from your Openstack VM

2025-01-16T04:05:49Z

Lianyingzhao: /* Using scp (without Carleton VPN) */

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

You will need to transfer files between the Openstack VM and your own PC from time to time, for example, when submitting your tutorial/assignment work. As your laptop/desktop does not have a fixed IP the VM can connect to, it is always better to initiate the connection <b>from your PC to the VM</b>, which has a floating IP address associated. Here, you have a few options depending on your situation (assuming you have the Carleton VPN connected). Choose one that works for you or feel free to use other alternatives.

==Using <tt>scp</tt> (with Carleton VPN)==
Secure file copy uses the same protocol as the secure shell (SSH), similar to the SSH command.

scp <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

The part in blue is an example pathname of the file to submit on your VM (make sure it matches your situation) and don't forget '''the last part in red, which is the destination directory on your PC''' (e.g., if could simply be a dot "." representing your current directory, where you are typing this command).

Replace the <VMIP> with the floating IP address of your VM. This should work for both Windows and Linux-like systems.

==Using <tt>scp</tt> (without Carleton VPN)==

scp -o "ProxyJump <CarletonUserName>@access.scs.carleton.ca" <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

Or
scp -J <CarletonUserName>@access.scs.carleton.ca <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

You will first be promoted for your MC1 password set through the SCS account, then your VM's password for the student user. Replace <CarletonUserName> with your Carleton username.

==Using <tt>rsync</tt>==
You can also use the more secure and new command, <tt>rsync</tt> instead.

rsync -avhP <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

Replace the <VMIP> with the floating IP address of your VM. Likewise, also pay attention to the two locations (in red and blue).

COMP3000 Operating Systems W25: Downloading files from your Openstack VM

2025-01-16T04:04:47Z

Lianyingzhao:

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

You will need to transfer files between the Openstack VM and your own PC from time to time, for example, when submitting your tutorial/assignment work. As your laptop/desktop does not have a fixed IP the VM can connect to, it is always better to initiate the connection <b>from your PC to the VM</b>, which has a floating IP address associated. Here, you have a few options depending on your situation (assuming you have the Carleton VPN connected). Choose one that works for you or feel free to use other alternatives.

==Using <tt>scp</tt> (with Carleton VPN)==
Secure file copy uses the same protocol as the secure shell (SSH), similar to the SSH command.

scp <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

The part in blue is an example pathname of the file to submit on your VM (make sure it matches your situation) and don't forget '''the last part in red, which is the destination directory on your PC''' (e.g., if could simply be a dot "." representing your current directory, where you are typing this command).

Replace the <VMIP> with the floating IP address of your VM. This should work for both Windows and Linux-like systems.

==Using <tt>scp</tt> (without Carleton VPN)==

scp -o "ProxyJump <CarletonUserName>@access.scs.carleton.ca" <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000">.</span>

Or
scp -J <CarletonUserName>@access.scs.carleton.ca <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

You will first be promoted for your MC1 password set through the SCS account, then your VM's password for the student user. Replace <CarletonUserName> with your Carleton username.

==Using <tt>rsync</tt>==
You can also use the more secure and new command, <tt>rsync</tt> instead.

rsync -avhP <span style="color:#0000ff"><VMUserName>@<VMIP>:/home/student/<CarletonUserName>-comp3000-assign1.tar.gz</span> <span style="color:#ff0000"><YourLocalPath></span>

Replace the <VMIP> with the floating IP address of your VM. Likewise, also pay attention to the two locations (in red and blue).

COMP3000 Operating Systems W25: Openstack Instructions

2025-01-16T03:51:22Z

Lianyingzhao:

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

You can create a VM instance on the [https://carleton.ca/scs/tech-support/scs-open-stack/ SCS openstack] cluster for the tutorials, by doing the following:

(obvious steps are skipped)

# [https://carleton.ca/its/help-centre/remote-access/ Connect to Carleton's VPN], if you are not on campus.
# Log in to [https://openstack-stein.scs.carleton.ca/ https://openstack-stein.scs.carleton.ca/]
# Most of you would need to refresh your SCS account to have the right entitlements for our course (or you may not be able to log in or not listed in the COMP3000-W25 project): https://newacct.scs.carleton.ca/scs_authentication/newacct-policy-form.php
# Switch to the COMP3000-W25 project (top-left corner), and launch an instance (button on the right).
#;[[File:select-project-w25-400px.jpg]]
# Make a choice for necessary items according to the screenshots attached.
#;[[File:boot-source.jpg|300px]] [[File:select-image-w25-580px.jpg]]
#; Flavor
#;[[File:select-flavor-w25-580px.jpg]]
# You must add "ping-ssh-egress" to the security groups, or you'll not be able to ping/ssh to your instance.
#; Security groups:
#;[[File:security-group.jpg|580px]]
# Assign a floating IP.
#;[[File:floating-ip.jpg|170px]]
#; NEVER attempt to create a snapshot
# Then the new IP address (134.117.XXX.XXX) to the right of 192.168.XXX.XXX will be the one (floating IP) you will use to connect to your instance.
# <u>Note down</u> your floating IP and your instance name for subsequent uses for the rest of the term.

COMP3000 Operating Systems 2025W

2025-01-16T03:50:48Z

Lianyingzhao:

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

==Course Outline and Student Hours==
Aside from the updated version in Brightspace, you can also find it [https://outline.scs.carleton.ca/media/2025/W/COMP3000AW2025/COMP-3000-A-W-2025.pdf here]. Note that only the Brightspace version will contain contact information of the TAs and student hours.

Follow the schedule to meet a TA in the corresponding room. Alternatively, you may also contact the instructor or TAs by email, on MS Teams or booking an appointment to meet online over Zoom.

'''Extended student hours.''' Every one or two weeks, a Q&A session will be held online
with the instructor. This is similar to regular student hours except that multiple students will
be admitted into the session (like a class), and the focus is on lecture topics, e.g., you can ask
the instructor to re-explain a concept. For ad hoc technical problems, e.g., how to set up the
environment for a tutorial, please attend a TA’s student hours. The meeting time varies as intended
and will be announced below and on Brightspace so that if it does not work for a student, it likely will the next time. Priority will be given to students in Section B (if there are too many participants).

==Tutorial Instructions==
Attending tutorials is required (there is a 0.5 participation mark each) and you need to submit the work by the deadline indicated on Brightspace (for the other 1.5 marks). What is submitted is not graded for correctness but effort showing your attempts and thinking. <u>To receive full marks, make sure to include in your submission how you arrived at the solution or other thoughts (if you figured it out) or any confusion/attempts or even disagreements (if you failed to)</u>. After the submission, you should continue to study it if there is still anything you do not understand well, for the assignments and exams.

===[[COMP3000 Operating Systems W25: Openstack Instructions | Instructions for setting up an Openstack instance]]===

===Various ways of [[COMP3000 Operating Systems W25: Connecting to SCS Openstack | connecting to SCS Openstack]]===

===[[COMP3000 Operating Systems W25: Downloading files from your Openstack VM | Downloading files from your Openstack VM]]===

===Tutorial 1===
Jan 9 and Jan 10, 2025: [[COMP3000 Operating Systems W25: Tutorial 1 | Instructions]]

===Tutorial 2===
Jan 16 and Jan 17, 2025: [[COMP3000 Operating Systems W25: Tutorial 2 | Instructions]]

===Tutorial 3===
Jan 23 and Jan 24, 2025: [[COMP3000 Operating Systems W25: Tutorial 3 | Instructions]]

===Tutorial 4===
Jan 30 and Jan 31, 2025: [[COMP3000 Operating Systems W25: Tutorial 4 | Instructions]]

===Tutorial 5===
Feb 6 and Feb 7, 2025: [[COMP3000 Operating Systems W25: Tutorial 5 | Instructions]]

===Tutorial 6===
Feb 27 and Feb 28, 2025: [[COMP3000 Operating Systems W25: Tutorial 6 | Instructions]]

===Tutorial 7===
Mar 6 and Mar 7, 2025: [[COMP3000 Operating Systems W25: Tutorial 7 | Instructions]]

===Tutorial 8===
Mar 13 and Mar 14, 2025: [[COMP3000 Operating Systems W25: Tutorial 8 | Instructions]]

===Tutorial 9===
Mar 20 and Mar 21, 2025: [[COMP3000 Operating Systems W25: Tutorial 9 | Instructions]]

==How to Do Well in This Course==
The goal here is to establish a <u>conceptual model</u> to understand how operating systems work to serve applications. It is not a course like math or data structures. '''To succeed, you need to be more hands-on and focus on figuring out how to find the answers (like learning to fish) instead of the answers alone (like being given a fish)'''. You can NOT expect to do well only by reading course material. You must work with a real OS, with the course material as guidelines. As you progress, you should be gradually able to imagine how various pieces of the OS fit together to function, which will be very useful in your everyday work/life with computers.

If you are asking a lot of "why"s, you are on the right track. Use all available resources (including the course material) as references, like a dictionary, to help answer your "why"s. For example, when you are asked about some information, based on your knowledge, think: 1) which part of the OS should have this information? 2) is it supposed to be available at the point in question? 3) then how do you retrieve that information? e.g., any tool/command/function call?
For sure, this would not be possible if you are just getting started, so a bit of confusion at the beginning is normal. As you move on, the conceptual model will become clearer as the basis for your thinking.

Mandatory textbook reading will be posted on Brightspace, and thus not all of the textbook will be covered. The exams and assignments are based on lectures, tutorials and designated textbook chapters.

The mini-quizzes are designed to ensure that you are following along by attending classes or listening to recorded lectures every week. So, they are straightforward, directly from class discussions (unlike the midterm and final exams).

COMP3000 Operating Systems 2025W

2025-01-16T03:50:10Z

Lianyingzhao:

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

==Course Outline and Student Hours==
Aside from the updated version in Brightspace, you can also find it [https://outline.scs.carleton.ca/media/2025/W/COMP3000AW2025/COMP-3000-A-W-2025.pdf here]. Note that only the Brightspace version will contain contact information of the TAs and student hours.

Follow the schedule to meet a TA in the corresponding room. Alternatively, you may also contact the instructor or TAs by email, on MS Teams or booking an appointment to meet online over Zoom.

'''Extended student hours.''' Every one or two weeks, a Q&A session will be held online
with the instructor. This is similar to regular student hours except that multiple students will
be admitted into the session (like a class), and the focus is on lecture topics, e.g., you can ask
the instructor to re-explain a concept. For ad hoc technical problems, e.g., how to set up the
environment for a tutorial, please attend a TA’s student hours. The meeting time varies as intended
and will be announced below and on Brightspace so that if it does not work for a student, it likely will the next time. Priority will be given to students in Section B (if there are too many participants).

==Tutorial Instructions==
Attending tutorials is required (there is a 0.5 participation mark each) and you need to submit the work by the deadline indicated on Brightspace (for the other 1.5 marks). What is submitted is not graded for correctness but effort showing your attempts and thinking. <u>To receive full marks, make sure to include in your submission how you arrived at the solution or other thoughts (if you figured it out) or any confusion/attempts or even disagreements (if you failed to)</u>. After the submission, you should continue to study it if there is still anything you do not understand well, for the assignments and exams.

===[[COMP3000 Operating Systems W25: Openstack Instructions | Instructions for setting up an Openstack instance]]===

Various ways of [[COMP3000 Operating Systems W25: Connecting to SCS Openstack | connecting to SCS Openstack]]===

===[[COMP3000 Operating Systems W25: Downloading files from your Openstack VM | Downloading files from your Openstack VM]]===

===Tutorial 1===
Jan 9 and Jan 10, 2025: [[COMP3000 Operating Systems W25: Tutorial 1 | Instructions]]

===Tutorial 2===
Jan 16 and Jan 17, 2025: [[COMP3000 Operating Systems W25: Tutorial 2 | Instructions]]

===Tutorial 3===
Jan 23 and Jan 24, 2025: [[COMP3000 Operating Systems W25: Tutorial 3 | Instructions]]

===Tutorial 4===
Jan 30 and Jan 31, 2025: [[COMP3000 Operating Systems W25: Tutorial 4 | Instructions]]

===Tutorial 5===
Feb 6 and Feb 7, 2025: [[COMP3000 Operating Systems W25: Tutorial 5 | Instructions]]

===Tutorial 6===
Feb 27 and Feb 28, 2025: [[COMP3000 Operating Systems W25: Tutorial 6 | Instructions]]

===Tutorial 7===
Mar 6 and Mar 7, 2025: [[COMP3000 Operating Systems W25: Tutorial 7 | Instructions]]

===Tutorial 8===
Mar 13 and Mar 14, 2025: [[COMP3000 Operating Systems W25: Tutorial 8 | Instructions]]

===Tutorial 9===
Mar 20 and Mar 21, 2025: [[COMP3000 Operating Systems W25: Tutorial 9 | Instructions]]

==How to Do Well in This Course==
The goal here is to establish a <u>conceptual model</u> to understand how operating systems work to serve applications. It is not a course like math or data structures. '''To succeed, you need to be more hands-on and focus on figuring out how to find the answers (like learning to fish) instead of the answers alone (like being given a fish)'''. You can NOT expect to do well only by reading course material. You must work with a real OS, with the course material as guidelines. As you progress, you should be gradually able to imagine how various pieces of the OS fit together to function, which will be very useful in your everyday work/life with computers.

If you are asking a lot of "why"s, you are on the right track. Use all available resources (including the course material) as references, like a dictionary, to help answer your "why"s. For example, when you are asked about some information, based on your knowledge, think: 1) which part of the OS should have this information? 2) is it supposed to be available at the point in question? 3) then how do you retrieve that information? e.g., any tool/command/function call?
For sure, this would not be possible if you are just getting started, so a bit of confusion at the beginning is normal. As you move on, the conceptual model will become clearer as the basis for your thinking.

Mandatory textbook reading will be posted on Brightspace, and thus not all of the textbook will be covered. The exams and assignments are based on lectures, tutorials and designated textbook chapters.

The mini-quizzes are designed to ensure that you are following along by attending classes or listening to recorded lectures every week. So, they are straightforward, directly from class discussions (unlike the midterm and final exams).

COMP3000 Operating Systems W25: Tutorial 1

2025-01-11T04:15:30Z

Lianyingzhao:

This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Getting Started==
For the tutorials, you need to get access to a Linux machine. We strongly suggest you use an SCS Openstack instance (see the instructions below). Alternatively, if you have resource available and prefer to work offline with your desktop/laptop, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2024F-2025W/COMP3000-W25.ova <b>this VirtualBox image</b>] (which is the same with the Openstack image, and follow [https://carleton.ca/scs/2019/creating-a-new-host-only-adapter-in-virtualbox/ the instructions] to set up the host-only network adapter, after updating to <b>the latest version of VirtualBox</b>). Note that as our VM is headless (i.e., no GUI desktop), you are supposed to only either use its local command line or connect to it via ssh. This is intended since you are learning to use an OS for which command-line access is the most native and close to its implementation for a better understanding, before you enjoy GUI in your future work. You will need access to a system for the entire semester, ideally the same one.

The concepts covered below are mostly part of standard UNIX/Linux tutorials. Feel free to consult one or more of them. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!

Feel free to discuss this tutorial on Teams.

===Openstack===
'''For emphasis: do not take snapshots!''' (see below)

You can create a VM on the SCS openstack by following [[COMP3000 Operating Systems W25: Openstack Instructions | '''these steps''']]. If you encouter any issue and would like more reading, SCS already has [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/ lots of documentation on openstack], including a [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/openstack-step-by-step-guide/ step-by-step guide].

Make sure of the key steps: ①Carleton VPN, ②refresh your SCS account if needed, ③the COMP3000-W25 project, ④the COMP3000-W25 '''snapshot''' image, ⑤adding the ping-ssh-egress security group, and ⑥associating a floating IP address.

The 192.168.X.X IP addresses are private (and cannot be accessed outside of '''the Openstack cluster'''), the 134.117.X.X floating IP addresses can be accessed from '''the Carleton network''' and will allow you to access the wider Internet.

You need to ssh to your VM instance, to do the tutorial work. Windows 10/11, Ubuntu and MacOS all have SSH clients available from their command lines. Just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. Other tools supporting SSH (e.g., PuTTY) also work.
Once you are prompted to log in, '''the default user is student, default password is student'''. You'll have to change your password once you are first logged in. (If you want to change your password later, use the <tt>passwd</tt> command.)



The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account’s home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed, and you'll make work for the tech staff who have to cancel what you did).'''

===Making use of Man pages===
The <tt>man</tt> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages. For example, if you look at files in an installed software package (e.g., <tt>dpkg -L ftp</tt> to list files in the ftp package) you can see there are a few files such as <tt>/usr/share/man/man1, 5, …</tt> They are the man page entries (sections).

The topics go beyond just software/command manuals, but also include conventions and abstract concepts (e.g., <tt>man syscalls</tt> and <tt>man man-pages</tt>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<tt>man 1 tee</tt>) and a system call (<tt>man 2 tee</tt>). A default will be used if the section is not specified.

For any commands mentioned in the tutorials, you can use <tt>man</tt> to find the usage. The advantage is the exact match with your current environment, compared to searching on the Internet.

Compared to the summary and brief information provided by the <tt>man</tt> command, you can also use the <tt>info</tt> command for a more detailed explanation for certain topics (just give it a try).

==Background==

===The Shell===
The shell or command line provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the operating system.

It works as an interpreter of the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the current default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of such commands (programmed in a scripting language). You will see shell scripts as we move on with this course.

===Permissions===
Your permission to access a file in Linux/Unix is determined by who you are logged in as. All files on the file system (including directories and other special files) have three different sets of permissions. The first set of permissions denotes the allowed file operations for '''the owner of the file''' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for '''a group of users''' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for '''everyone else''' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group. The set of permissions is threefold: read, write, and execute (rwx).

===Environment & Shell Variables===
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.

In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could write <tt>echo $HELLO</tt>).

If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:

X="Important Data"

just defines X as a shell variable. However, if you then type:

export X

X will be turned into an environment variable (no longer private), and so every subsequent program run by the shell will also get X. You can combine both in one line:

export X="Important Data"

This is the idiom for setting environment variables normally. You will be able to better understand the differnces between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).

One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:

export X = "Important Data"

as you now are giving export three arguments, not one.

To delete an environment variable, you can <tt>unset X</tt>.

===Downloading files using commands===
When working with the command line and you need to download a small number of files from a website (i.e., using the Web/HTTP protocol), you can choose one of the two below (use <tt>man</tt> to find out more if interested):

curl -o hello.c [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]
wget [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]

In later tutorials/assignments, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page or a PDF.

===Learning to work with a text console without GUI===
You can take advantage of the graphical user interface (GUI) as much as possible, except when learning to use an OS. The command line interface is closer to the internal implementation of the OS, e.g., the commands being programs you execute, the arguments being what the OS understands, as well as saving multiple layers of indirection such as the window and the desktop subsystems, and graphics rendering.

[https://github.com/tmux/tmux/wiki '''tmux'''] is a terminal multiplexer already installed in our VM (feel free to use your own alternatives). It avoids the need for opening multiple console windows or SSH sessions, which defeats the purpose of not using GUI. More importantly and usefully, tmux can help maintain your active sessions even if you accidentally get disconnected.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

To list and attach to an existing session: <tt>tmux ls</tt> and <tt>tmux attach</tt> (or <tt>tmux attach -t</tt> session-id). Use <tt>man tmux</tt> for more details.

===Compiling & Running Programs===
To compile a program, use <tt>gcc</tt>:

gcc -O2 hello.c -o hello

This compiles it with level 2 optimization and without debugging symbols, and the output program is hello.

To run, you have to specify where it is (the single dot <tt>.</tt> represents your current directory):

./hello

Remember you can always change directories using the <tt>cd</tt> command.

==Tasks/Questions==
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out mutiple ways for each (if you can).
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* Hardware information, e.g., RAM (remaining/total), CPU (model and number of cores), storage (what disks are connected and their capacity).
#* The name (binary path) of the current shell.
# Using the <tt>man</tt> command, find out what the following commands do: <tt>which</tt>, <tt>pwd</tt>, <tt>who</tt>, <tt>whoami</tt>, <tt>whereis</tt>, and <tt>whatis</tt>.
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <tt>cd</tt>) is internal or external?
#* For the program you just compiled above, what happens if you use <tt>hello</tt> to run it instead of <tt>./hello</tt> (and why does that happen)?
#* Figure out where at least three external commands reside on the system.
# Making your own commands: the <tt>PATH</tt> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Now, spend a few minutes to do a quick comparison: what are environment variables like in MS Windows? What are the two types? (hint: just search for "env" by clicking on the magnifier icon next to the Start menu of your Windows; you can also check its command line, if interested.)
# Going back to your Linux, look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# For those same program binaries, figure out what the permission bits mean by reading the man page of <tt>chmod</tt> (this is the command you could use to change those permission bits).
# What are the owner, group, and permissions of <tt>/etc/passwd</tt> and <tt>/etc/shadow</tt>? What are these files used for (and <u>how did you find this out</u>)?
# The <tt>ls</tt> command can be used to get a listing of the files in a directory. What options are passed to <tt>ls</tt> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <tt>bash</tt></u>? List at least 3 differences.

COMP3000 Operating Systems W25: Tutorial 2

2025-01-08T23:23:49Z

Lianyingzhao: Created page with "In this tutorial, you will revisit the lifecyle of a program discussed in class, from source code, to an executable (binary image), and further to loading it into the address space. Then from a different angle, you can see when in execution, how the program (now a process) makes different types of calls to function, and how its memory is laid out. Meanwhile, we can also see how a programmer can make use of existing code (libraries). ==General Instructions (same for all..."

In this tutorial, you will revisit the lifecyle of a program discussed in class, from source code, to an executable (binary image), and further to loading it into the address space. Then from a different angle, you can see when in execution, how the program (now a process) makes different types of calls to function, and how its memory is laid out. Meanwhile, we can also see how a programmer can make use of existing code (libraries).

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Building Your Program==
Assuming you use a compiled programming language like C, you will involve the following steps (implicitly) to build your program:
* Compile source (C) code into assembly code (.s files).
*:You can see it using <tt>gcc -S -O2 [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c hello.c]</tt>.
* Assemble assembly code into machine code placed in object code files (.o files).
*:To avoid the .o file being deleted automatically, you can do the compilation alone without linking by: <tt>gcc -c -O2 hello.c</tt>.
* Link object code files together to create a runnable binary image (ELF files, no extension name).
*:This is the way you did it in Tutorial 1.

==Talking to the OS Kernel via System Calls==
A process on its own has limited access to the system. It cannot directly access any external devices or data sources (e.g., files, keyboard, the screen, networks) on its own. To access these external resources, to allocate memory, or otherwise change its runtime environment, it must make '''system calls'''. So we can say system calls are one way for the contained process to request OS services. Note that system calls run code outside of a process and thus cannot be called like regular function calls. The standard C library provides function wrappers for most commonly-used system calls so they can be accessed like regular C functions. Under the hood, however, these functions make use of special compiler directives in order to generate the machine code necessary to invoke system calls. We will take a closer look later this term.

You can see the system calls produced by a process using the <tt>strace</tt> command.

Keep in mind that this can be a good example to show the orthogonal relationship between root/non-root users and kernel/user modes. System calls are requesting the kernel to do something in the kernel mode on behalf of the calling process. Therefore, any user (even non-root) process making a system call will at certain point run in the kernel mode.

==Static & Dynamic Libraries==
Because of abstraction (e.g., for portability or ease of development), most applications are not self-contained. For instance, you call just a simple <tt>printf()</tt> to output a message instead of drawing pixels on a display using a lot of low-level code. Hence, your program relies on lots of external code.
In compiled languages such as C, external code can be brought into the process through linking. There are two basic types of linking, static and dynamic linking:

* With static linking, code is brought in at compile time (specifically, in the link stage above) and added to the executable. The code is now the same as other application code. (Static libraries are more <tt>.o</tt> files like <tt>hello.o</tt>; in practice, you may see more often <tt>.a</tt> files, which are "archives" of multiple <tt>.o</tt> files.)
* With dynamic linking, a reference to the library code is added to the binary. The actual library code has to later be loaded when the program is executed. This loading will happen before <tt>main()</tt> is called (you will notice a lot of noisy system calls at the beginning to achieve this). Then, at runtime, calling functions in such dynamic libraries will take the form of <b>library calls</b> (which only happens within this process in user space, as opposed to system calls in kernel space).

The dynamic libraries associated with a program binary can be found using the <tt>ldd</tt> command. You can use <tt>ltrace</tt> to see calls to functions that are dynamically linked.

==Tasks/Questions==
===Part A===
Here you will see assembly code, but for now, you do not need to read and understand each line (and it will not be the focus of this course). What you will learn is the ability to extract useful information by skimming the assembly code.
In general, such assembly code is divided into three colums: starting at position/character 1 will be labels/symbols (which you can refer to), followed by mnemonics (instruction opcodes) and last corresponding arguments (operands).
If you find it necessary, you can turn to related documentation/manuals like: [https://en.wikibooks.org/wiki/X86_Assembly/GNU_assembly_syntax x86 Assembly/GNU assembly syntax]
# Looking at the <tt>.s</tt> file produced from <tt>gcc -S -O2 hello.c</tt>, do you see anything familiar and discussed in the lectures of our first week?
# Where (e.g., at which line) is it supposed to call that <tt>printf()</tt>? If it is not doing that, <tt>why?</tt> (you can just mention what you think/believe)
# Generate the .s file for [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut2/syscall-hello.c syscall-hello.c] as well. Observing the two .s files, what do you think is the key difference?
# You can play around by trying to see: how a function in C is reflected in assembly; how to return from a function; how variables/literals are represented; what are '''symbols''' (refer to commands like <tt>nm</tt>); etc.

===Part B===
# By default, gcc-produced binaries are dynamically linked (so at runtime they will require dynamic libraries to be present on the system). To compile a binary that is statically linked (so it has no external runtime library dependencies), instead do this:
#: <tt>gcc -O2 -static hello.c -o hello.static</tt>
#: Then compile a dynamically linked version:
#: <tt>gcc -O2 -z lazy hello.c -o hello.dynamic</tt>
#:How does the size of hello.dynamic compare with that of hello.static? <u>Why?</u>
# Compile the other flavor of hello.c: [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut2/syscall-hello.c syscall-hello.c]. Build one static version and one dynamic version as well.
#: Now considering hello.dynamic, hello.static, syscall-hello.dynamic and syscall-hello.static, see what system calls each program produces by running <tt>strace -o sys-somename.log ./somename.static</tt> (or <tt>./somename.dynamic</tt>). Which version generates more system calls? <u>Why?</u>
#: Note: system calls are saved in the log file <tt>sys-somename.log</tt>. Feel free to save them in a different file for each version.
# See what library calls each program produces by running <tt>ltrace -o lib-somename.log ./somename.dynamic</tt> (or <tt>./somename.static</tt>). Which version generates more library calls? <u>Why?</u>
# Remember when building hello in Tutorial 1 you did not use <tt>-z lazy</tt>. Comparing <tt>hello</tt> with <tt>hello.dynamic</tt>, any difference? <u>Why this difference?</u>
# Use <tt>ldd</tt> to find out what dynamic library dependencies dynamic versions have. What about static versions?
# You did not need to specify the dynamic library dependencies listed by <tt>ldd</tt> but the system figured them out. But this is not always the case. Add the following lines of code to hello.c. Now, if you compile it with <tt>gcc -z lazy hello.c -o hello</tt> (do not use any optimization like <tt>-O2</tt>), what will happen? How can you tell gcc to link with the right library? (Hint: use the option "<tt>-l</tt>".) What is the corresponding library's file name?
#include <math.h>
double val = 9;
printf("The square root of 9 is %f\n", sqrt(val));

===Part C===
Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut2/3000memview.c 3000memview.c], then consider the following questions (if they turn out to be difficult, document your exploration and get some thinking).

# Why are the addresses inconsistent between runs?
# Roughly where does the stack seem to be? The heap? Code? Global variables? (Hints: recall the memory image layout of a process discussed in the lecture; local variables go to the stack; initialized '''data''' and global variables go to the data segment, data allocated at runtime go to the heap.)

COMP3000 Operating Systems W25: Tutorial 1

2025-01-07T16:46:55Z

Lianyingzhao:

This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Getting Started==
For the tutorials, you need to get access to a Linux machine. We strongly suggest you use an SCS Openstack instance (see the instructions below). Alternatively, if you have resource available and prefer to work offline with your desktop/laptop, you can also download <b>this VirtualBox image</b> <small><small>(Edit: the new image will be ready in a few days; before that you can use the [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2023F-2024W/COMP3000-W24.ova W24 image], almost the same except that security-wise no updates were performed)</small></small> (which is the same with the Openstack image, and follow [https://carleton.ca/scs/2019/creating-a-new-host-only-adapter-in-virtualbox/ the instructions] to set up the host-only network adapter, after updating to <b>the latest version of VirtualBox</b>). Note that as our VM is headless (i.e., no GUI desktop), you are supposed to only either use its local command line or connect to it via ssh. This is intended since you are learning to use an OS for which command-line access is the most native and close to its implementation for a better understanding, before you enjoy GUI in your future work. You will need access to a system for the entire semester, ideally the same one.

The concepts covered below are mostly part of standard UNIX/Linux tutorials. Feel free to consult one or more of them. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!

Feel free to discuss this tutorial on Teams.

===Openstack===
'''For emphasis: do not take snapshots!''' (see below)

You can create a VM on the SCS openstack by following [[COMP3000 Operating Systems W25: Openstack Instructions | '''these steps''']]. If you encouter any issue and would like more reading, SCS already has [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/ lots of documentation on openstack], including a [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/openstack-step-by-step-guide/ step-by-step guide].

Make sure of the key steps: ①Carleton VPN, ②refresh your SCS account if needed, ③the COMP3000-W25 project, ④the COMP3000-W25 '''snapshot''' image, ⑤adding the ping-ssh-egress security group, and ⑥associating a floating IP address.

The 192.168.X.X IP addresses are private (and cannot be accessed outside of '''the Openstack cluster'''), the 134.117.X.X floating IP addresses can be accessed from '''the Carleton network''' and will allow you to access the wider Internet.

You need to ssh to your VM instance, to do the tutorial work. Windows 10/11, Ubuntu and MacOS all have SSH clients available from their command lines. Just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. Other tools supporting SSH (e.g., PuTTY) also work.
Once you are prompted to log in, '''the default user is student, default password is student'''. You'll have to change your password once you are first logged in. (If you want to change your password later, use the <tt>passwd</tt> command.)



The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account’s home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed, and you'll make work for the tech staff who have to cancel what you did).'''

===Making use of Man pages===
The <tt>man</tt> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages. For example, if you look at files in an installed software package (e.g., <tt>dpkg -L ftp</tt> to list files in the ftp package) you can see there are a few files such as <tt>/usr/share/man/man1, 5, …</tt> They are the man page entries (sections).

The topics go beyond just software/command manuals, but also include conventions and abstract concepts (e.g., <tt>man syscalls</tt> and <tt>man man-pages</tt>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<tt>man 1 tee</tt>) and a system call (<tt>man 2 tee</tt>). A default will be used if the section is not specified.

For any commands mentioned in the tutorials, you can use <tt>man</tt> to find the usage. The advantage is the exact match with your current environment, compared to searching on the Internet.

Compared to the summary and brief information provided by the <tt>man</tt> command, you can also use the <tt>info</tt> command for a more detailed explanation for certain topics (just give it a try).

==Background==

===The Shell===
The shell or command line provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the operating system.

It works as an interpreter of the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the current default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of such commands (programmed in a scripting language). You will see shell scripts as we move on with this course.

===Permissions===
Your permission to access a file in Linux/Unix is determined by who you are logged in as. All files on the file system (including directories and other special files) have three different sets of permissions. The first set of permissions denotes the allowed file operations for '''the owner of the file''' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for '''a group of users''' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for '''everyone else''' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group. The set of permissions is threefold: read, write, and execute (rwx).

===Environment & Shell Variables===
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.

In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could write <tt>echo $HELLO</tt>).

If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:

X="Important Data"

just defines X as a shell variable. However, if you then type:

export X

X will be turned into an environment variable (no longer private), and so every subsequent program run by the shell will also get X. You can combine both in one line:

export X="Important Data"

This is the idiom for setting environment variables normally. You will be able to better understand the differnces between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).

One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:

export X = "Important Data"

as you now are giving export three arguments, not one.

To delete an environment variable, you can <tt>unset X</tt>.

===Downloading files using commands===
When working with the command line and you need to download a small number of files from a website (i.e., using the Web/HTTP protocol), you can choose one of the two below (use <tt>man</tt> to find out more if interested):

curl -o hello.c [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]
wget [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]

In later tutorials/assignments, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page or a PDF.

===Learning to work with a text console without GUI===
You can take advantage of the graphical user interface (GUI) as much as possible, except when learning to use an OS. The command line interface is closer to the internal implementation of the OS, e.g., the commands being programs you execute, the arguments being what the OS understands, as well as saving multiple layers of indirection such as the window and the desktop subsystems, and graphics rendering.

[https://github.com/tmux/tmux/wiki '''tmux'''] is a terminal multiplexer already installed in our VM (feel free to use your own alternatives). It avoids the need for opening multiple console windows or SSH sessions, which defeats the purpose of not using GUI. More importantly and usefully, tmux can help maintain your active sessions even if you accidentally get disconnected.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

To list and attach to an existing session: <tt>tmux ls</tt> and <tt>tmux attach</tt> (or <tt>tmux attach -t</tt> session-id). Use <tt>man tmux</tt> for more details.

===Compiling & Running Programs===
To compile a program, use <tt>gcc</tt>:

gcc -O2 hello.c -o hello

This compiles it with level 2 optimization and without debugging symbols, and the output program is hello.

To run, you have to specify where it is (the single dot <tt>.</tt> represents your current directory):

./hello

Remember you can always change directories using the <tt>cd</tt> command.

==Tasks/Questions==
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out mutiple ways for each (if you can).
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* Hardware information, e.g., RAM (remaining/total), CPU (model and number of cores), storage (what disks are connected and their capacity).
#* The name (binary path) of the current shell.
# Using the <tt>man</tt> command, find out what the following commands do: <tt>which</tt>, <tt>pwd</tt>, <tt>who</tt>, <tt>whoami</tt>, <tt>whereis</tt>, and <tt>whatis</tt>.
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <tt>cd</tt>) is internal or external?
#* For the program you just compiled above, what happens if you use <tt>hello</tt> to run it instead of <tt>./hello</tt> (and why does that happen)?
#* Figure out where at least three external commands reside on the system.
# Making your own commands: the <tt>PATH</tt> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Now, spend a few minutes to do a quick comparison: what are environment variables like in MS Windows? What are the two types? (hint: just search for "env" by clicking on the magnifier icon next to the Start menu of your Windows; you can also check its command line, if interested.)
# Going back to your Linux, look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# For those same program binaries, figure out what the permission bits mean by reading the man page of <tt>chmod</tt> (this is the command you could use to change those permission bits).
# What are the owner, group, and permissions of <tt>/etc/passwd</tt> and <tt>/etc/shadow</tt>? What are these files used for (and <u>how did you find this out</u>)?
# The <tt>ls</tt> command can be used to get a listing of the files in a directory. What options are passed to <tt>ls</tt> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <tt>bash</tt></u>? List at least 3 differences.

COMP3000 Operating Systems W25: Tutorial 1

2025-01-07T15:57:53Z

Lianyingzhao:

This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Getting Started==
For the tutorials, you need to get access to a Linux machine. We strongly suggest you use an SCS Openstack instance (see the instructions below). Alternatively, if you have resource available and prefer to work offline with your desktop/laptop, you can also download <b>this VirtualBox image</b> <small>Edit: the new image will be ready in a few days; before that you can use the [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2023F-2024W/COMP3000-W24.ova W24 image], almost the same except that security-wise no updates were performed)</small> (which is the same with the Openstack image, and follow [https://carleton.ca/scs/2019/creating-a-new-host-only-adapter-in-virtualbox/ the instructions] to set up the host-only network adapter, after updating to <b>the latest version of VirtualBox</b>). Note that as our VM is headless (i.e., no GUI desktop), you are supposed to only either use its local command line or connect to it via ssh. This is intended since you are learning to use an OS for which command-line access is the most native and close to its implementation for a better understanding, before you enjoy GUI in your future work. You will need access to a system for the entire semester, ideally the same one.

The concepts covered below are mostly part of standard UNIX/Linux tutorials. Feel free to consult one or more of them. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!

Feel free to discuss this tutorial on Teams.

===Openstack===
'''For emphasis: do not take snapshots!''' (see below)

You can create a VM on the SCS openstack by following [[COMP3000 Operating Systems W25: Openstack Instructions | '''these steps''']]. If you encouter any issue and would like more reading, SCS already has [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/ lots of documentation on openstack], including a [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/openstack-step-by-step-guide/ step-by-step guide].

Make sure of the key steps: ①Carleton VPN, ②refresh your SCS account if needed, ③the COMP3000-W25 project, ④the COMP3000-W25 '''snapshot''' image, ⑤adding the ping-ssh-egress security group, and ⑥associating a floating IP address.

The 192.168.X.X IP addresses are private (and cannot be accessed outside of '''the Openstack cluster'''), the 134.117.X.X floating IP addresses can be accessed from '''the Carleton network''' and will allow you to access the wider Internet.

You need to ssh to your VM instance, to do the tutorial work. Windows 10/11, Ubuntu and MacOS all have SSH clients available from their command lines. Just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. Other tools supporting SSH (e.g., PuTTY) also work.
Once you are prompted to log in, '''the default user is student, default password is student'''. You'll have to change your password once you are first logged in. (If you want to change your password later, use the <tt>passwd</tt> command.)



The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account’s home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed, and you'll make work for the tech staff who have to cancel what you did).'''

===Making use of Man pages===
The <tt>man</tt> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages. For example, if you look at files in an installed software package (e.g., <tt>dpkg -L ftp</tt> to list files in the ftp package) you can see there are a few files such as <tt>/usr/share/man/man1, 5, …</tt> They are the man page entries (sections).

The topics go beyond just software/command manuals, but also include conventions and abstract concepts (e.g., <tt>man syscalls</tt> and <tt>man man-pages</tt>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<tt>man 1 tee</tt>) and a system call (<tt>man 2 tee</tt>). A default will be used if the section is not specified.

For any commands mentioned in the tutorials, you can use <tt>man</tt> to find the usage. The advantage is the exact match with your current environment, compared to searching on the Internet.

Compared to the summary and brief information provided by the <tt>man</tt> command, you can also use the <tt>info</tt> command for a more detailed explanation for certain topics (just give it a try).

==Background==

===The Shell===
The shell or command line provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the operating system.

It works as an interpreter of the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the current default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of such commands (programmed in a scripting language). You will see shell scripts as we move on with this course.

===Permissions===
Your permission to access a file in Linux/Unix is determined by who you are logged in as. All files on the file system (including directories and other special files) have three different sets of permissions. The first set of permissions denotes the allowed file operations for '''the owner of the file''' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for '''a group of users''' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for '''everyone else''' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group. The set of permissions is threefold: read, write, and execute (rwx).

===Environment & Shell Variables===
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.

In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could write <tt>echo $HELLO</tt>).

If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:

X="Important Data"

just defines X as a shell variable. However, if you then type:

export X

X will be turned into an environment variable (no longer private), and so every subsequent program run by the shell will also get X. You can combine both in one line:

export X="Important Data"

This is the idiom for setting environment variables normally. You will be able to better understand the differnces between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).

One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:

export X = "Important Data"

as you now are giving export three arguments, not one.

To delete an environment variable, you can <tt>unset X</tt>.

===Downloading files using commands===
When working with the command line and you need to download a small number of files from a website (i.e., using the Web/HTTP protocol), you can choose one of the two below (use <tt>man</tt> to find out more if interested):

curl -o hello.c [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]
wget [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]

In later tutorials/assignments, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page or a PDF.

===Learning to work with a text console without GUI===
You can take advantage of the graphical user interface (GUI) as much as possible, except when learning to use an OS. The command line interface is closer to the internal implementation of the OS, e.g., the commands being programs you execute, the arguments being what the OS understands, as well as saving multiple layers of indirection such as the window and the desktop subsystems, and graphics rendering.

[https://github.com/tmux/tmux/wiki '''tmux'''] is a terminal multiplexer already installed in our VM (feel free to use your own alternatives). It avoids the need for opening multiple console windows or SSH sessions, which defeats the purpose of not using GUI. More importantly and usefully, tmux can help maintain your active sessions even if you accidentally get disconnected.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

To list and attach to an existing session: <tt>tmux ls</tt> and <tt>tmux attach</tt> (or <tt>tmux attach -t</tt> session-id). Use <tt>man tmux</tt> for more details.

===Compiling & Running Programs===
To compile a program, use <tt>gcc</tt>:

gcc -O2 hello.c -o hello

This compiles it with level 2 optimization and without debugging symbols, and the output program is hello.

To run, you have to specify where it is (the single dot <tt>.</tt> represents your current directory):

./hello

Remember you can always change directories using the <tt>cd</tt> command.

==Tasks/Questions==
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out mutiple ways for each (if you can).
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* Hardware information, e.g., RAM (remaining/total), CPU (model and number of cores), storage (what disks are connected and their capacity).
#* The name (binary path) of the current shell.
# Using the <tt>man</tt> command, find out what the following commands do: <tt>which</tt>, <tt>pwd</tt>, <tt>who</tt>, <tt>whoami</tt>, <tt>whereis</tt>, and <tt>whatis</tt>.
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <tt>cd</tt>) is internal or external?
#* For the program you just compiled above, what happens if you use <tt>hello</tt> to run it instead of <tt>./hello</tt> (and why does that happen)?
#* Figure out where at least three external commands reside on the system.
# Making your own commands: the <tt>PATH</tt> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Now, spend a few minutes to do a quick comparison: what are environment variables like in MS Windows? What are the two types? (hint: just search for "env" by clicking on the magnifier icon next to the Start menu of your Windows; you can also check its command line, if interested.)
# Going back to your Linux, look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# For those same program binaries, figure out what the permission bits mean by reading the man page of <tt>chmod</tt> (this is the command you could use to change those permission bits).
# What are the owner, group, and permissions of <tt>/etc/passwd</tt> and <tt>/etc/shadow</tt>? What are these files used for (and <u>how did you find this out</u>)?
# The <tt>ls</tt> command can be used to get a listing of the files in a directory. What options are passed to <tt>ls</tt> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <tt>bash</tt></u>? List at least 3 differences.

COMP3000 Operating Systems W25: Tutorial 1

2025-01-07T15:54:55Z

Lianyingzhao:

This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Getting Started==
For the tutorials, you need to get access to a Linux machine. We strongly suggest you use an SCS Openstack instance (see the instructions below). Alternatively, if you have resource available and prefer to work offline with your desktop/laptop, you can also download <b>this VirtualBox image</b> {{Edit: the new image will be ready in a few days; before that you can use the [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2023F-2024W/COMP3000-W24.ova W24 image], almost the same except that security-wise no updates were performed)}} (which is the same with the Openstack image, and follow [https://carleton.ca/scs/2019/creating-a-new-host-only-adapter-in-virtualbox/ the instructions] to set up the host-only network adapter, after updating to <b>the latest version of VirtualBox</b>). Note that as our VM is headless (i.e., no GUI desktop), you are supposed to only either use its local command line or connect to it via ssh. This is intended since you are learning to use an OS for which command-line access is the most native and close to its implementation for a better understanding, before you enjoy GUI in your future work. You will need access to a system for the entire semester, ideally the same one.

The concepts covered below are mostly part of standard UNIX/Linux tutorials. Feel free to consult one or more of them. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!

Feel free to discuss this tutorial on Teams.

===Openstack===
'''For emphasis: do not take snapshots!''' (see below)

You can create a VM on the SCS openstack by following [[COMP3000 Operating Systems W25: Openstack Instructions | '''these steps''']]. If you encouter any issue and would like more reading, SCS already has [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/ lots of documentation on openstack], including a [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/openstack-step-by-step-guide/ step-by-step guide].

Make sure of the key steps: ①Carleton VPN, ②refresh your SCS account if needed, ③the COMP3000-W25 project, ④the COMP3000-W25 '''snapshot''' image, ⑤adding the ping-ssh-egress security group, and ⑥associating a floating IP address.

The 192.168.X.X IP addresses are private (and cannot be accessed outside of '''the Openstack cluster'''), the 134.117.X.X floating IP addresses can be accessed from '''the Carleton network''' and will allow you to access the wider Internet.

You need to ssh to your VM instance, to do the tutorial work. Windows 10/11, Ubuntu and MacOS all have SSH clients available from their command lines. Just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. Other tools supporting SSH (e.g., PuTTY) also work.
Once you are prompted to log in, '''the default user is student, default password is student'''. You'll have to change your password once you are first logged in. (If you want to change your password later, use the <tt>passwd</tt> command.)



The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account’s home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed, and you'll make work for the tech staff who have to cancel what you did).'''

===Making use of Man pages===
The <tt>man</tt> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages. For example, if you look at files in an installed software package (e.g., <tt>dpkg -L ftp</tt> to list files in the ftp package) you can see there are a few files such as <tt>/usr/share/man/man1, 5, …</tt> They are the man page entries (sections).

The topics go beyond just software/command manuals, but also include conventions and abstract concepts (e.g., <tt>man syscalls</tt> and <tt>man man-pages</tt>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<tt>man 1 tee</tt>) and a system call (<tt>man 2 tee</tt>). A default will be used if the section is not specified.

For any commands mentioned in the tutorials, you can use <tt>man</tt> to find the usage. The advantage is the exact match with your current environment, compared to searching on the Internet.

Compared to the summary and brief information provided by the <tt>man</tt> command, you can also use the <tt>info</tt> command for a more detailed explanation for certain topics (just give it a try).

==Background==

===The Shell===
The shell or command line provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the operating system.

It works as an interpreter of the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the current default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of such commands (programmed in a scripting language). You will see shell scripts as we move on with this course.

===Permissions===
Your permission to access a file in Linux/Unix is determined by who you are logged in as. All files on the file system (including directories and other special files) have three different sets of permissions. The first set of permissions denotes the allowed file operations for '''the owner of the file''' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for '''a group of users''' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for '''everyone else''' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group. The set of permissions is threefold: read, write, and execute (rwx).

===Environment & Shell Variables===
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.

In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could write <tt>echo $HELLO</tt>).

If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:

X="Important Data"

just defines X as a shell variable. However, if you then type:

export X

X will be turned into an environment variable (no longer private), and so every subsequent program run by the shell will also get X. You can combine both in one line:

export X="Important Data"

This is the idiom for setting environment variables normally. You will be able to better understand the differnces between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).

One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:

export X = "Important Data"

as you now are giving export three arguments, not one.

To delete an environment variable, you can <tt>unset X</tt>.

===Downloading files using commands===
When working with the command line and you need to download a small number of files from a website (i.e., using the Web/HTTP protocol), you can choose one of the two below (use <tt>man</tt> to find out more if interested):

curl -o hello.c [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]
wget [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]

In later tutorials/assignments, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page or a PDF.

===Learning to work with a text console without GUI===
You can take advantage of the graphical user interface (GUI) as much as possible, except when learning to use an OS. The command line interface is closer to the internal implementation of the OS, e.g., the commands being programs you execute, the arguments being what the OS understands, as well as saving multiple layers of indirection such as the window and the desktop subsystems, and graphics rendering.

[https://github.com/tmux/tmux/wiki '''tmux'''] is a terminal multiplexer already installed in our VM (feel free to use your own alternatives). It avoids the need for opening multiple console windows or SSH sessions, which defeats the purpose of not using GUI. More importantly and usefully, tmux can help maintain your active sessions even if you accidentally get disconnected.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

To list and attach to an existing session: <tt>tmux ls</tt> and <tt>tmux attach</tt> (or <tt>tmux attach -t</tt> session-id). Use <tt>man tmux</tt> for more details.

===Compiling & Running Programs===
To compile a program, use <tt>gcc</tt>:

gcc -O2 hello.c -o hello

This compiles it with level 2 optimization and without debugging symbols, and the output program is hello.

To run, you have to specify where it is (the single dot <tt>.</tt> represents your current directory):

./hello

Remember you can always change directories using the <tt>cd</tt> command.

==Tasks/Questions==
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out mutiple ways for each (if you can).
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* Hardware information, e.g., RAM (remaining/total), CPU (model and number of cores), storage (what disks are connected and their capacity).
#* The name (binary path) of the current shell.
# Using the <tt>man</tt> command, find out what the following commands do: <tt>which</tt>, <tt>pwd</tt>, <tt>who</tt>, <tt>whoami</tt>, <tt>whereis</tt>, and <tt>whatis</tt>.
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <tt>cd</tt>) is internal or external?
#* For the program you just compiled above, what happens if you use <tt>hello</tt> to run it instead of <tt>./hello</tt> (and why does that happen)?
#* Figure out where at least three external commands reside on the system.
# Making your own commands: the <tt>PATH</tt> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Now, spend a few minutes to do a quick comparison: what are environment variables like in MS Windows? What are the two types? (hint: just search for "env" by clicking on the magnifier icon next to the Start menu of your Windows; you can also check its command line, if interested.)
# Going back to your Linux, look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# For those same program binaries, figure out what the permission bits mean by reading the man page of <tt>chmod</tt> (this is the command you could use to change those permission bits).
# What are the owner, group, and permissions of <tt>/etc/passwd</tt> and <tt>/etc/shadow</tt>? What are these files used for (and <u>how did you find this out</u>)?
# The <tt>ls</tt> command can be used to get a listing of the files in a directory. What options are passed to <tt>ls</tt> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <tt>bash</tt></u>? List at least 3 differences.

COMP3000 Operating Systems W25: Tutorial 1

2025-01-07T15:54:27Z

Lianyingzhao:

This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Getting Started==
For the tutorials, you need to get access to a Linux machine. We strongly suggest you use an SCS Openstack instance (see the instructions below). Alternatively, if you have resource available and prefer to work offline with your desktop/laptop, you can also download <b>this VirtualBox image</b> {{smaller|Edit: the new image will be ready in a few days; before that you can use the [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2023F-2024W/COMP3000-W24.ova W24 image], almost the same except that security-wise no updates were performed)}} (which is the same with the Openstack image, and follow [https://carleton.ca/scs/2019/creating-a-new-host-only-adapter-in-virtualbox/ the instructions] to set up the host-only network adapter, after updating to <b>the latest version of VirtualBox</b>). Note that as our VM is headless (i.e., no GUI desktop), you are supposed to only either use its local command line or connect to it via ssh. This is intended since you are learning to use an OS for which command-line access is the most native and close to its implementation for a better understanding, before you enjoy GUI in your future work. You will need access to a system for the entire semester, ideally the same one.

The concepts covered below are mostly part of standard UNIX/Linux tutorials. Feel free to consult one or more of them. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!

Feel free to discuss this tutorial on Teams.

===Openstack===
'''For emphasis: do not take snapshots!''' (see below)

You can create a VM on the SCS openstack by following [[COMP3000 Operating Systems W25: Openstack Instructions | '''these steps''']]. If you encouter any issue and would like more reading, SCS already has [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/ lots of documentation on openstack], including a [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/openstack-step-by-step-guide/ step-by-step guide].

Make sure of the key steps: ①Carleton VPN, ②refresh your SCS account if needed, ③the COMP3000-W25 project, ④the COMP3000-W25 '''snapshot''' image, ⑤adding the ping-ssh-egress security group, and ⑥associating a floating IP address.

The 192.168.X.X IP addresses are private (and cannot be accessed outside of '''the Openstack cluster'''), the 134.117.X.X floating IP addresses can be accessed from '''the Carleton network''' and will allow you to access the wider Internet.

You need to ssh to your VM instance, to do the tutorial work. Windows 10/11, Ubuntu and MacOS all have SSH clients available from their command lines. Just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. Other tools supporting SSH (e.g., PuTTY) also work.
Once you are prompted to log in, '''the default user is student, default password is student'''. You'll have to change your password once you are first logged in. (If you want to change your password later, use the <tt>passwd</tt> command.)



The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account’s home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed, and you'll make work for the tech staff who have to cancel what you did).'''

===Making use of Man pages===
The <tt>man</tt> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages. For example, if you look at files in an installed software package (e.g., <tt>dpkg -L ftp</tt> to list files in the ftp package) you can see there are a few files such as <tt>/usr/share/man/man1, 5, …</tt> They are the man page entries (sections).

The topics go beyond just software/command manuals, but also include conventions and abstract concepts (e.g., <tt>man syscalls</tt> and <tt>man man-pages</tt>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<tt>man 1 tee</tt>) and a system call (<tt>man 2 tee</tt>). A default will be used if the section is not specified.

For any commands mentioned in the tutorials, you can use <tt>man</tt> to find the usage. The advantage is the exact match with your current environment, compared to searching on the Internet.

Compared to the summary and brief information provided by the <tt>man</tt> command, you can also use the <tt>info</tt> command for a more detailed explanation for certain topics (just give it a try).

==Background==

===The Shell===
The shell or command line provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the operating system.

It works as an interpreter of the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the current default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of such commands (programmed in a scripting language). You will see shell scripts as we move on with this course.

===Permissions===
Your permission to access a file in Linux/Unix is determined by who you are logged in as. All files on the file system (including directories and other special files) have three different sets of permissions. The first set of permissions denotes the allowed file operations for '''the owner of the file''' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for '''a group of users''' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for '''everyone else''' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group. The set of permissions is threefold: read, write, and execute (rwx).

===Environment & Shell Variables===
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.

In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could write <tt>echo $HELLO</tt>).

If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:

X="Important Data"

just defines X as a shell variable. However, if you then type:

export X

X will be turned into an environment variable (no longer private), and so every subsequent program run by the shell will also get X. You can combine both in one line:

export X="Important Data"

This is the idiom for setting environment variables normally. You will be able to better understand the differnces between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).

One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:

export X = "Important Data"

as you now are giving export three arguments, not one.

To delete an environment variable, you can <tt>unset X</tt>.

===Downloading files using commands===
When working with the command line and you need to download a small number of files from a website (i.e., using the Web/HTTP protocol), you can choose one of the two below (use <tt>man</tt> to find out more if interested):

curl -o hello.c [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]
wget [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]

In later tutorials/assignments, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page or a PDF.

===Learning to work with a text console without GUI===
You can take advantage of the graphical user interface (GUI) as much as possible, except when learning to use an OS. The command line interface is closer to the internal implementation of the OS, e.g., the commands being programs you execute, the arguments being what the OS understands, as well as saving multiple layers of indirection such as the window and the desktop subsystems, and graphics rendering.

[https://github.com/tmux/tmux/wiki '''tmux'''] is a terminal multiplexer already installed in our VM (feel free to use your own alternatives). It avoids the need for opening multiple console windows or SSH sessions, which defeats the purpose of not using GUI. More importantly and usefully, tmux can help maintain your active sessions even if you accidentally get disconnected.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

To list and attach to an existing session: <tt>tmux ls</tt> and <tt>tmux attach</tt> (or <tt>tmux attach -t</tt> session-id). Use <tt>man tmux</tt> for more details.

===Compiling & Running Programs===
To compile a program, use <tt>gcc</tt>:

gcc -O2 hello.c -o hello

This compiles it with level 2 optimization and without debugging symbols, and the output program is hello.

To run, you have to specify where it is (the single dot <tt>.</tt> represents your current directory):

./hello

Remember you can always change directories using the <tt>cd</tt> command.

==Tasks/Questions==
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out mutiple ways for each (if you can).
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* Hardware information, e.g., RAM (remaining/total), CPU (model and number of cores), storage (what disks are connected and their capacity).
#* The name (binary path) of the current shell.
# Using the <tt>man</tt> command, find out what the following commands do: <tt>which</tt>, <tt>pwd</tt>, <tt>who</tt>, <tt>whoami</tt>, <tt>whereis</tt>, and <tt>whatis</tt>.
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <tt>cd</tt>) is internal or external?
#* For the program you just compiled above, what happens if you use <tt>hello</tt> to run it instead of <tt>./hello</tt> (and why does that happen)?
#* Figure out where at least three external commands reside on the system.
# Making your own commands: the <tt>PATH</tt> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Now, spend a few minutes to do a quick comparison: what are environment variables like in MS Windows? What are the two types? (hint: just search for "env" by clicking on the magnifier icon next to the Start menu of your Windows; you can also check its command line, if interested.)
# Going back to your Linux, look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# For those same program binaries, figure out what the permission bits mean by reading the man page of <tt>chmod</tt> (this is the command you could use to change those permission bits).
# What are the owner, group, and permissions of <tt>/etc/passwd</tt> and <tt>/etc/shadow</tt>? What are these files used for (and <u>how did you find this out</u>)?
# The <tt>ls</tt> command can be used to get a listing of the files in a directory. What options are passed to <tt>ls</tt> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <tt>bash</tt></u>? List at least 3 differences.

COMP3000 Operating Systems W25: Openstack Instructions

2025-01-07T15:09:18Z

Lianyingzhao:

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

You can create a VM instance on the [https://carleton.ca/scs/tech-support/scs-open-stack/ SCS openstack] cluster for the tutorials, by doing the following:

(obvious steps are skipped)

# [https://carleton.ca/its/help-centre/remote-access/ Connect to Carleton's VPN].
# Log in to [https://openstack-stein.scs.carleton.ca/ https://openstack-stein.scs.carleton.ca/]
# Most of you would need to refresh your SCS account to have the right entitlements for our course (or you may not be able to log in or not listed in the COMP3000-W25 project): https://newacct.scs.carleton.ca/scs_authentication/newacct-policy-form.php
# Switch to the COMP3000-W25 project (top-left corner), and launch an instance (button on the right).
#;[[File:select-project-w25-400px.jpg]]
# Make a choice for necessary items according to the screenshots attached.
#;[[File:boot-source.jpg|300px]] [[File:select-image-w25-580px.jpg]]
#; Flavor
#;[[File:select-flavor-w25-580px.jpg]]
# You must add "ping-ssh-egress" to the security groups, or you'll not be able to ping/ssh to your instance.
#; Security groups:
#;[[File:security-group.jpg|580px]]
# Assign a floating IP.
#;[[File:floating-ip.jpg|170px]]
#; NEVER attempt to create a snapshot
# Then the new IP address (134.117.XXX.XXX) to the right of 192.168.XXX.XXX will be the one (floating IP) you will use to connect to your instance.
# <u>Note down</u> your floating IP and your instance name for subsequent uses for the rest of the term.

File:Select-flavor-w25-580px.jpg

2025-01-07T15:09:09Z

Lianyingzhao:

COMP3000 Operating Systems W25: Openstack Instructions

2025-01-07T15:05:10Z

Lianyingzhao:

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

You can create a VM instance on the [https://carleton.ca/scs/tech-support/scs-open-stack/ SCS openstack] cluster for the tutorials, by doing the following:

(obvious steps are skipped)

# [https://carleton.ca/its/help-centre/remote-access/ Connect to Carleton's VPN].
# Log in to [https://openstack-stein.scs.carleton.ca/ https://openstack-stein.scs.carleton.ca/]
# Most of you would need to refresh your SCS account to have the right entitlements for our course (or you may not be able to log in or not listed in the COMP3000-W25 project): https://newacct.scs.carleton.ca/scs_authentication/newacct-policy-form.php
# Switch to the COMP3000-W25 project (top-left corner), and launch an instance (button on the right).
#;[[File:select-project-w25-400px.jpg]]
# Make a choice for necessary items according to the screenshots attached.
#;[[File:boot-source.jpg|300px]] [[File:select-image-w25-580px.jpg]]
#; Flavor

# You must add "ping-ssh-egress" to the security groups, or you'll not be able to ping/ssh to your instance.
#; Security groups:
#;[[File:security-group.jpg|580px]]
# Assign a floating IP.
#;[[File:floating-ip.jpg|170px]]
#; NEVER attempt to create a snapshot
# Then the new IP address (134.117.XXX.XXX) to the right of 192.168.XXX.XXX will be the one (floating IP) you will use to connect to your instance.
# <u>Note down</u> your floating IP and your instance name for subsequent uses for the rest of the term.

File:Select-image-w25-580px.jpg

2025-01-07T15:05:03Z

Lianyingzhao:

COMP3000 Operating Systems W25: Openstack Instructions

2025-01-07T14:55:58Z

Lianyingzhao:

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

You can create a VM instance on the [https://carleton.ca/scs/tech-support/scs-open-stack/ SCS openstack] cluster for the tutorials, by doing the following:

(obvious steps are skipped)

# [https://carleton.ca/its/help-centre/remote-access/ Connect to Carleton's VPN].
# Log in to [https://openstack-stein.scs.carleton.ca/ https://openstack-stein.scs.carleton.ca/]
# Most of you would need to refresh your SCS account to have the right entitlements for our course (or you may not be able to log in or not listed in the COMP3000-W25 project): https://newacct.scs.carleton.ca/scs_authentication/newacct-policy-form.php
# Switch to the COMP3000-W25 project (top-left corner), and launch an instance (button on the right).
#;[[File:select-project-w25-400px.jpg]]
# Make a choice for necessary items according to the screenshots attached.
#;[[File:boot-source.jpg|300px]] 
#; Flavor

# You must add "ping-ssh-egress" to the security groups, or you'll not be able to ping/ssh to your instance.
#; Security groups:
#;[[File:security-group.jpg|580px]]
# Assign a floating IP.
#;[[File:floating-ip.jpg|170px]]
#; NEVER attempt to create a snapshot
# Then the new IP address (134.117.XXX.XXX) to the right of 192.168.XXX.XXX will be the one (floating IP) you will use to connect to your instance.
# <u>Note down</u> your floating IP and your instance name for subsequent uses for the rest of the term.

COMP3000 Operating Systems W25: Openstack Instructions

2025-01-07T14:55:37Z

Lianyingzhao:

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

You can create a VM instance on the [https://carleton.ca/scs/tech-support/scs-open-stack/ SCS openstack] cluster for the tutorials, by doing the following:

(obvious steps are skipped)

# [https://carleton.ca/its/help-centre/remote-access/ Connect to Carleton's VPN].
# Log in to [https://openstack-stein.scs.carleton.ca/ https://openstack-stein.scs.carleton.ca/]
# Most of you would need to refresh your SCS account to have the right entitlements for our course (or you may not be able to log in or not listed in the COMP3000-W25 project): https://newacct.scs.carleton.ca/scs_authentication/newacct-policy-form.php
# Switch to the COMP3000-W25 project (top-left corner), and launch an instance (button on the right).
#;[[File:select-project-w24-400px.jpg]]
# Make a choice for necessary items according to the screenshots attached.
#;[[File:boot-source.jpg|300px]] 
#; Flavor

# You must add "ping-ssh-egress" to the security groups, or you'll not be able to ping/ssh to your instance.
#; Security groups:
#;[[File:security-group.jpg|580px]]
# Assign a floating IP.
#;[[File:floating-ip.jpg|170px]]
#; NEVER attempt to create a snapshot
# Then the new IP address (134.117.XXX.XXX) to the right of 192.168.XXX.XXX will be the one (floating IP) you will use to connect to your instance.
# <u>Note down</u> your floating IP and your instance name for subsequent uses for the rest of the term.

File:Select-project-w25-400px.jpg

2025-01-07T14:55:19Z

Lianyingzhao:

COMP3000 Operating Systems W25: Tutorial 1

2025-01-04T19:25:50Z

Lianyingzhao: /* Learning to work with a text console without GUI */

This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Getting Started==
For the tutorials, you need to get access to a Linux machine. We strongly suggest you use an SCS Openstack instance (see the instructions below). Alternatively, if you have resource available and prefer to work offline with your desktop/laptop, you can also download <b>this VirtualBox image</b> ('''to be updated''') (which is the same with the Openstack image, and follow [https://carleton.ca/scs/2019/creating-a-new-host-only-adapter-in-virtualbox/ the instructions] to set up the host-only network adapter, after updating to <b>the latest version of VirtualBox</b>). Note that as our VM is headless (i.e., no GUI desktop), you are supposed to only either use its local command line or connect to it via ssh. This is intended since you are learning to use an OS for which command-line access is the most native and close to its implementation for a better understanding, before you enjoy GUI in your future work. You will need access to a system for the entire semester, ideally the same one.

The concepts covered below are mostly part of standard UNIX/Linux tutorials. Feel free to consult one or more of them. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!

Feel free to discuss this tutorial on Teams.

===Openstack===
'''For emphasis: do not take snapshots!''' (see below)

You can create a VM on the SCS openstack by following [[COMP3000 Operating Systems W25: Openstack Instructions | '''these steps''']]. If you encouter any issue and would like more reading, SCS already has [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/ lots of documentation on openstack], including a [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/openstack-step-by-step-guide/ step-by-step guide].

Make sure of the key steps: ①Carleton VPN, ②refresh your SCS account if needed, ③the COMP3000-W25 project, ④the COMP3000-W25 '''snapshot''' image, ⑤adding the ping-ssh-egress security group, and ⑥associating a floating IP address.

The 192.168.X.X IP addresses are private (and cannot be accessed outside of '''the Openstack cluster'''), the 134.117.X.X floating IP addresses can be accessed from '''the Carleton network''' and will allow you to access the wider Internet.

You need to ssh to your VM instance, to do the tutorial work. Windows 10/11, Ubuntu and MacOS all have SSH clients available from their command lines. Just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. Other tools supporting SSH (e.g., PuTTY) also work.
Once you are prompted to log in, '''the default user is student, default password is student'''. You'll have to change your password once you are first logged in. (If you want to change your password later, use the <tt>passwd</tt> command.)



The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account’s home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed, and you'll make work for the tech staff who have to cancel what you did).'''

===Making use of Man pages===
The <tt>man</tt> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages. For example, if you look at files in an installed software package (e.g., <tt>dpkg -L ftp</tt> to list files in the ftp package) you can see there are a few files such as <tt>/usr/share/man/man1, 5, …</tt> They are the man page entries (sections).

The topics go beyond just software/command manuals, but also include conventions and abstract concepts (e.g., <tt>man syscalls</tt> and <tt>man man-pages</tt>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<tt>man 1 tee</tt>) and a system call (<tt>man 2 tee</tt>). A default will be used if the section is not specified.

For any commands mentioned in the tutorials, you can use <tt>man</tt> to find the usage. The advantage is the exact match with your current environment, compared to searching on the Internet.

Compared to the summary and brief information provided by the <tt>man</tt> command, you can also use the <tt>info</tt> command for a more detailed explanation for certain topics (just give it a try).

==Background==

===The Shell===
The shell or command line provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the operating system.

It works as an interpreter of the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the current default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of such commands (programmed in a scripting language). You will see shell scripts as we move on with this course.

===Permissions===
Your permission to access a file in Linux/Unix is determined by who you are logged in as. All files on the file system (including directories and other special files) have three different sets of permissions. The first set of permissions denotes the allowed file operations for '''the owner of the file''' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for '''a group of users''' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for '''everyone else''' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group. The set of permissions is threefold: read, write, and execute (rwx).

===Environment & Shell Variables===
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.

In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could write <tt>echo $HELLO</tt>).

If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:

X="Important Data"

just defines X as a shell variable. However, if you then type:

export X

X will be turned into an environment variable (no longer private), and so every subsequent program run by the shell will also get X. You can combine both in one line:

export X="Important Data"

This is the idiom for setting environment variables normally. You will be able to better understand the differnces between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).

One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:

export X = "Important Data"

as you now are giving export three arguments, not one.

To delete an environment variable, you can <tt>unset X</tt>.

===Downloading files using commands===
When working with the command line and you need to download a small number of files from a website (i.e., using the Web/HTTP protocol), you can choose one of the two below (use <tt>man</tt> to find out more if interested):

curl -o hello.c [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]
wget [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]

In later tutorials/assignments, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page or a PDF.

===Learning to work with a text console without GUI===
You can take advantage of the graphical user interface (GUI) as much as possible, except when learning to use an OS. The command line interface is closer to the internal implementation of the OS, e.g., the commands being programs you execute, the arguments being what the OS understands, as well as saving multiple layers of indirection such as the window and the desktop subsystems, and graphics rendering.

[https://github.com/tmux/tmux/wiki '''tmux'''] is a terminal multiplexer already installed in our VM (feel free to use your own alternatives). It avoids the need for opening multiple console windows or SSH sessions, which defeats the purpose of not using GUI. More importantly and usefully, tmux can help maintain your active sessions even if you accidentally get disconnected.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

To list and attach to an existing session: <tt>tmux ls</tt> and <tt>tmux attach</tt> (or <tt>tmux attach -t</tt> session-id). Use <tt>man tmux</tt> for more details.

===Compiling & Running Programs===
To compile a program, use <tt>gcc</tt>:

gcc -O2 hello.c -o hello

This compiles it with level 2 optimization and without debugging symbols, and the output program is hello.

To run, you have to specify where it is (the single dot <tt>.</tt> represents your current directory):

./hello

Remember you can always change directories using the <tt>cd</tt> command.

==Tasks/Questions==
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out mutiple ways for each (if you can).
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* Hardware information, e.g., RAM (remaining/total), CPU (model and number of cores), storage (what disks are connected and their capacity).
#* The name (binary path) of the current shell.
# Using the <tt>man</tt> command, find out what the following commands do: <tt>which</tt>, <tt>pwd</tt>, <tt>who</tt>, <tt>whoami</tt>, <tt>whereis</tt>, and <tt>whatis</tt>.
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <tt>cd</tt>) is internal or external?
#* For the program you just compiled above, what happens if you use <tt>hello</tt> to run it instead of <tt>./hello</tt> (and why does that happen)?
#* Figure out where at least three external commands reside on the system.
# Making your own commands: the <tt>PATH</tt> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Now, spend a few minutes to do a quick comparison: what are environment variables like in MS Windows? What are the two types? (hint: just search for "env" by clicking on the magnifier icon next to the Start menu of your Windows; you can also check its command line, if interested.)
# Going back to your Linux, look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# For those same program binaries, figure out what the permission bits mean by reading the man page of <tt>chmod</tt> (this is the command you could use to change those permission bits).
# What are the owner, group, and permissions of <tt>/etc/passwd</tt> and <tt>/etc/shadow</tt>? What are these files used for (and <u>how did you find this out</u>)?
# The <tt>ls</tt> command can be used to get a listing of the files in a directory. What options are passed to <tt>ls</tt> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <tt>bash</tt></u>? List at least 3 differences.

COMP3000 Operating Systems W25: Tutorial 1

2025-01-04T19:23:56Z

Lianyingzhao:

This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Getting Started==
For the tutorials, you need to get access to a Linux machine. We strongly suggest you use an SCS Openstack instance (see the instructions below). Alternatively, if you have resource available and prefer to work offline with your desktop/laptop, you can also download <b>this VirtualBox image</b> ('''to be updated''') (which is the same with the Openstack image, and follow [https://carleton.ca/scs/2019/creating-a-new-host-only-adapter-in-virtualbox/ the instructions] to set up the host-only network adapter, after updating to <b>the latest version of VirtualBox</b>). Note that as our VM is headless (i.e., no GUI desktop), you are supposed to only either use its local command line or connect to it via ssh. This is intended since you are learning to use an OS for which command-line access is the most native and close to its implementation for a better understanding, before you enjoy GUI in your future work. You will need access to a system for the entire semester, ideally the same one.

The concepts covered below are mostly part of standard UNIX/Linux tutorials. Feel free to consult one or more of them. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!

Feel free to discuss this tutorial on Teams.

===Openstack===
'''For emphasis: do not take snapshots!''' (see below)

You can create a VM on the SCS openstack by following [[COMP3000 Operating Systems W25: Openstack Instructions | '''these steps''']]. If you encouter any issue and would like more reading, SCS already has [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/ lots of documentation on openstack], including a [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/openstack-step-by-step-guide/ step-by-step guide].

Make sure of the key steps: ①Carleton VPN, ②refresh your SCS account if needed, ③the COMP3000-W25 project, ④the COMP3000-W25 '''snapshot''' image, ⑤adding the ping-ssh-egress security group, and ⑥associating a floating IP address.

The 192.168.X.X IP addresses are private (and cannot be accessed outside of '''the Openstack cluster'''), the 134.117.X.X floating IP addresses can be accessed from '''the Carleton network''' and will allow you to access the wider Internet.

You need to ssh to your VM instance, to do the tutorial work. Windows 10/11, Ubuntu and MacOS all have SSH clients available from their command lines. Just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. Other tools supporting SSH (e.g., PuTTY) also work.
Once you are prompted to log in, '''the default user is student, default password is student'''. You'll have to change your password once you are first logged in. (If you want to change your password later, use the <tt>passwd</tt> command.)



The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account’s home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed, and you'll make work for the tech staff who have to cancel what you did).'''

===Making use of Man pages===
The <tt>man</tt> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages. For example, if you look at files in an installed software package (e.g., <tt>dpkg -L ftp</tt> to list files in the ftp package) you can see there are a few files such as <tt>/usr/share/man/man1, 5, …</tt> They are the man page entries (sections).

The topics go beyond just software/command manuals, but also include conventions and abstract concepts (e.g., <tt>man syscalls</tt> and <tt>man man-pages</tt>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<tt>man 1 tee</tt>) and a system call (<tt>man 2 tee</tt>). A default will be used if the section is not specified.

For any commands mentioned in the tutorials, you can use <tt>man</tt> to find the usage. The advantage is the exact match with your current environment, compared to searching on the Internet.

Compared to the summary and brief information provided by the <tt>man</tt> command, you can also use the <tt>info</tt> command for a more detailed explanation for certain topics (just give it a try).

==Background==

===The Shell===
The shell or command line provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the operating system.

It works as an interpreter of the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the current default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of such commands (programmed in a scripting language). You will see shell scripts as we move on with this course.

===Permissions===
Your permission to access a file in Linux/Unix is determined by who you are logged in as. All files on the file system (including directories and other special files) have three different sets of permissions. The first set of permissions denotes the allowed file operations for '''the owner of the file''' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for '''a group of users''' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for '''everyone else''' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group. The set of permissions is threefold: read, write, and execute (rwx).

===Environment & Shell Variables===
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.

In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could write <tt>echo $HELLO</tt>).

If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:

X="Important Data"

just defines X as a shell variable. However, if you then type:

export X

X will be turned into an environment variable (no longer private), and so every subsequent program run by the shell will also get X. You can combine both in one line:

export X="Important Data"

This is the idiom for setting environment variables normally. You will be able to better understand the differnces between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).

One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:

export X = "Important Data"

as you now are giving export three arguments, not one.

To delete an environment variable, you can <tt>unset X</tt>.

===Downloading files using commands===
When working with the command line and you need to download a small number of files from a website (i.e., using the Web/HTTP protocol), you can choose one of the two below (use <tt>man</tt> to find out more if interested):

curl -o hello.c [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]
wget [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]

In later tutorials/assignments, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page or a PDF.

===Learning to work with a text console without GUI===
You can take advantage of the graphical user interface (GUI) as much as possible, except when learning to use an OS. The command line interface is closer to the internal implementation of the OS, e.g., the commands being programs you execute, the arguments being what the OS understands, as well as saving multiple layers of indirection

[https://github.com/tmux/tmux/wiki '''tmux'''] is a terminal multiplexer already installed in our VM (feel free to use your own alternatives). It avoids the need for opening multiple console windows or SSH sessions, which defeats the purpose of not using GUI. More importantly and usefully, tmux can help maintain your active sessions even if you accidentally get disconnected.

To create a new session: press Ctrl-B and C

To move between sessions: press Ctrl-B and N (or P)

To list and attach to an existing session: <tt>tmux ls</tt> and <tt>tmux attach</tt> (or <tt>tmux attach -t</tt> session-id). Use <tt>man tmux</tt> for more details.

===Compiling & Running Programs===
To compile a program, use <tt>gcc</tt>:

gcc -O2 hello.c -o hello

This compiles it with level 2 optimization and without debugging symbols, and the output program is hello.

To run, you have to specify where it is (the single dot <tt>.</tt> represents your current directory):

./hello

Remember you can always change directories using the <tt>cd</tt> command.

==Tasks/Questions==
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out mutiple ways for each (if you can).
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* Hardware information, e.g., RAM (remaining/total), CPU (model and number of cores), storage (what disks are connected and their capacity).
#* The name (binary path) of the current shell.
# Using the <tt>man</tt> command, find out what the following commands do: <tt>which</tt>, <tt>pwd</tt>, <tt>who</tt>, <tt>whoami</tt>, <tt>whereis</tt>, and <tt>whatis</tt>.
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <tt>cd</tt>) is internal or external?
#* For the program you just compiled above, what happens if you use <tt>hello</tt> to run it instead of <tt>./hello</tt> (and why does that happen)?
#* Figure out where at least three external commands reside on the system.
# Making your own commands: the <tt>PATH</tt> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Now, spend a few minutes to do a quick comparison: what are environment variables like in MS Windows? What are the two types? (hint: just search for "env" by clicking on the magnifier icon next to the Start menu of your Windows; you can also check its command line, if interested.)
# Going back to your Linux, look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# For those same program binaries, figure out what the permission bits mean by reading the man page of <tt>chmod</tt> (this is the command you could use to change those permission bits).
# What are the owner, group, and permissions of <tt>/etc/passwd</tt> and <tt>/etc/shadow</tt>? What are these files used for (and <u>how did you find this out</u>)?
# The <tt>ls</tt> command can be used to get a listing of the files in a directory. What options are passed to <tt>ls</tt> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <tt>bash</tt></u>? List at least 3 differences.

COMP3000 Operating Systems W25: Tutorial 1

2025-01-04T07:11:20Z

Lianyingzhao:

This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Getting Started==
For the tutorials, you need to get access to a Linux machine. We strongly suggest you use an SCS Openstack instance (see the instructions below). Alternatively, if you have resource available and prefer to work offline with your desktop/laptop, you can also download <b>this VirtualBox image</b> ('''to be updated''') (which is the same with the Openstack image, and follow [https://carleton.ca/scs/2019/creating-a-new-host-only-adapter-in-virtualbox/ the instructions] to set up the host-only network adapter, after updating to <b>the latest version of VirtualBox</b>). Note that as our VM is headless (i.e., no GUI desktop), you are supposed to only either use its local command line or connect to it via ssh. This is intended since you are learning to use an OS for which command-line access is the most native and close to its implementation for a better understanding, before you enjoy GUI in your future work. You will need access to a system for the entire semester, ideally the same one.

The concepts covered below are mostly part of standard UNIX/Linux tutorials. Feel free to consult one or more of them. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!

Feel free to discuss this tutorial on Teams.

===Openstack===
'''For emphasis: do not take snapshots!''' (see below)

You can create a VM on the SCS openstack by following [[COMP3000 Operating Systems W25: Openstack Instructions | '''these steps''']]. If you encouter any issue and would like more reading, SCS already has [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/ lots of documentation on openstack], including a [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/openstack-step-by-step-guide/ step-by-step guide].

Make sure of the key steps: ①Carleton VPN, ②refresh your SCS account if needed, ③the COMP3000-W25 project, ④the COMP3000-W25 '''snapshot''' image, ⑤adding the ping-ssh-egress security group, and ⑥associating a floating IP address.

The 192.168.X.X IP addresses are private (and cannot be accessed outside of '''the Openstack cluster'''), the 134.117.X.X floating IP addresses can be accessed from '''the Carleton network''' and will allow you to access the wider Internet.

You need to ssh to your VM instance, to do the tutorial work. Windows 10/11, Ubuntu and MacOS all have SSH clients available from their command lines. Just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. Other tools supporting SSH (e.g., PuTTY) also work.
Once you are prompted to log in, '''the default user is student, default password is student'''. You'll have to change your password once you are first logged in. (If you want to change your password later, use the <tt>passwd</tt> command.)



The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account’s home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed, and you'll make work for the tech staff who have to cancel what you did).'''

===Making use of Man pages===
The <tt>man</tt> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages. For example, if you look at files in an installed software package (e.g., <tt>dpkg -L ftp</tt> to list files in the ftp package) you can see there are a few files such as <tt>/usr/share/man/man1, 5, …</tt> They are the man page entries (sections).

The topics go beyond just software/command manuals, but also include conventions and abstract concepts (e.g., <tt>man syscalls</tt> and <tt>man man-pages</tt>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<tt>man 1 tee</tt>) and a system call (<tt>man 2 tee</tt>). A default will be used if the section is not specified.

For any commands mentioned in the tutorials, you can use <tt>man</tt> to find the usage. The advantage is the exact match with your current environment, compared to searching on the Internet.

Compared to the summary and brief information provided by the <tt>man</tt> command, you can also use the <tt>info</tt> command for a more detailed explanation for certain topics (just give it a try).

==Background==

===The Shell===
The shell or command line provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the operating system.

It works as an interpreter of the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the current default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of such commands (programmed in a scripting language). You will see shell scripts as we move on with this course.

===Permissions===
Your permission to access a file in Linux/Unix is determined by who you are logged in as. All files on the file system (including directories and other special files) have three different sets of permissions. The first set of permissions denotes the allowed file operations for '''the owner of the file''' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for '''a group of users''' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for '''everyone else''' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group. The set of permissions is threefold: read, write, and execute (rwx).

===Environment & Shell Variables===
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.

In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could write <tt>echo $HELLO</tt>).

If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:

X="Important Data"

just defines X as a shell variable. However, if you then type:

export X

X will be turned into an environment variable (no longer private), and so every subsequent program run by the shell will also get X. You can combine both in one line:

export X="Important Data"

This is the idiom for setting environment variables normally. You will be able to better understand the differnces between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).

One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:

export X = "Important Data"

as you now are giving export three arguments, not one.

To delete an environment variable, you can <tt>unset X</tt>.

===Downloading files using commands===
When working with the command line and you need to download a small number of files from a website (i.e., using the Web/HTTP protocol), you can choose one of the two below (use <tt>man</tt> to find out more if interested):

curl -o hello.c [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]
wget [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]

In later tutorials/assignments, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page or a PDF.

===Compiling & Running Programs===
To compile a program, use <tt>gcc</tt>:

gcc -O2 hello.c -o hello

This compiles it with level 2 optimization and without debugging symbols, and the output program is hello.

To run, you have to specify where it is (the single dot <tt>.</tt> represents your current directory):

./hello

Remember you can always change directories using the <tt>cd</tt> command.

==Tasks/Questions==
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out mutiple ways for each (if you can).
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* Hardware information, e.g., RAM (remaining/total), CPU (model and number of cores), storage (what disks are connected and their capacity).
#* The name (binary path) of the current shell.
# Using the <tt>man</tt> command, find out what the following commands do: <tt>which</tt>, <tt>pwd</tt>, <tt>who</tt>, <tt>whoami</tt>, <tt>whereis</tt>, and <tt>whatis</tt>.
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <tt>cd</tt>) is internal or external?
#* For the program you just compiled above, what happens if you use <tt>hello</tt> to run it instead of <tt>./hello</tt> (and why does that happen)?
#* Figure out where at least three external commands reside on the system.
# Making your own commands: the <tt>PATH</tt> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Now, spend a few minutes to do a quick comparison: what are environment variables like in MS Windows? What are the two types? (hint: just search for "env" by clicking on the magnifier icon next to the Start menu of your Windows; you can also check its command line, if interested.)
# Going back to your Linux, look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# For those same program binaries, figure out what the permission bits mean by reading the man page of <tt>chmod</tt> (this is the command you could use to change those permission bits).
# What are the owner, group, and permissions of <tt>/etc/passwd</tt> and <tt>/etc/shadow</tt>? What are these files used for (and <u>how did you find this out</u>)?
# The <tt>ls</tt> command can be used to get a listing of the files in a directory. What options are passed to <tt>ls</tt> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <tt>bash</tt></u>? List at least 3 differences.

COMP3000 Operating Systems W25: Tutorial 1

2025-01-04T07:10:59Z

Lianyingzhao:

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Getting Started==
For the tutorials, you need to get access to a Linux machine. We strongly suggest you use an SCS Openstack instance (see the instructions below). Alternatively, if you have resource available and prefer to work offline with your desktop/laptop, you can also download <b>this VirtualBox image</b> ('''to be updated''') (which is the same with the Openstack image, and follow [https://carleton.ca/scs/2019/creating-a-new-host-only-adapter-in-virtualbox/ the instructions] to set up the host-only network adapter, after updating to <b>the latest version of VirtualBox</b>). Note that as our VM is headless (i.e., no GUI desktop), you are supposed to only either use its local command line or connect to it via ssh. This is intended since you are learning to use an OS for which command-line access is the most native and close to its implementation for a better understanding, before you enjoy GUI in your future work. You will need access to a system for the entire semester, ideally the same one.

The concepts covered below are mostly part of standard UNIX/Linux tutorials. Feel free to consult one or more of them. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!

Feel free to discuss this tutorial on Teams.

===Openstack===
'''For emphasis: do not take snapshots!''' (see below)

You can create a VM on the SCS openstack by following [[COMP3000 Operating Systems W25: Openstack Instructions | '''these steps''']]. If you encouter any issue and would like more reading, SCS already has [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/ lots of documentation on openstack], including a [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/openstack-step-by-step-guide/ step-by-step guide].

Make sure of the key steps: ①Carleton VPN, ②refresh your SCS account if needed, ③the COMP3000-W25 project, ④the COMP3000-W25 '''snapshot''' image, ⑤adding the ping-ssh-egress security group, and ⑥associating a floating IP address.

The 192.168.X.X IP addresses are private (and cannot be accessed outside of '''the Openstack cluster'''), the 134.117.X.X floating IP addresses can be accessed from '''the Carleton network''' and will allow you to access the wider Internet.

You need to ssh to your VM instance, to do the tutorial work. Windows 10/11, Ubuntu and MacOS all have SSH clients available from their command lines. Just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. Other tools supporting SSH (e.g., PuTTY) also work.
Once you are prompted to log in, '''the default user is student, default password is student'''. You'll have to change your password once you are first logged in. (If you want to change your password later, use the <tt>passwd</tt> command.)



The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account’s home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed, and you'll make work for the tech staff who have to cancel what you did).'''

===Making use of Man pages===
The <tt>man</tt> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages. For example, if you look at files in an installed software package (e.g., <tt>dpkg -L ftp</tt> to list files in the ftp package) you can see there are a few files such as <tt>/usr/share/man/man1, 5, …</tt> They are the man page entries (sections).

The topics go beyond just software/command manuals, but also include conventions and abstract concepts (e.g., <tt>man syscalls</tt> and <tt>man man-pages</tt>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<tt>man 1 tee</tt>) and a system call (<tt>man 2 tee</tt>). A default will be used if the section is not specified.

For any commands mentioned in the tutorials, you can use <tt>man</tt> to find the usage. The advantage is the exact match with your current environment, compared to searching on the Internet.

Compared to the summary and brief information provided by the <tt>man</tt> command, you can also use the <tt>info</tt> command for a more detailed explanation for certain topics (just give it a try).

==Background==

===The Shell===
The shell or command line provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the operating system.

It works as an interpreter of the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the current default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of such commands (programmed in a scripting language). You will see shell scripts as we move on with this course.

===Permissions===
Your permission to access a file in Linux/Unix is determined by who you are logged in as. All files on the file system (including directories and other special files) have three different sets of permissions. The first set of permissions denotes the allowed file operations for '''the owner of the file''' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for '''a group of users''' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for '''everyone else''' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group. The set of permissions is threefold: read, write, and execute (rwx).

===Environment & Shell Variables===
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.

In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could write <tt>echo $HELLO</tt>).

If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:

X="Important Data"

just defines X as a shell variable. However, if you then type:

export X

X will be turned into an environment variable (no longer private), and so every subsequent program run by the shell will also get X. You can combine both in one line:

export X="Important Data"

This is the idiom for setting environment variables normally. You will be able to better understand the differnces between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).

One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:

export X = "Important Data"

as you now are giving export three arguments, not one.

To delete an environment variable, you can <tt>unset X</tt>.

===Downloading files using commands===
When working with the command line and you need to download a small number of files from a website (i.e., using the Web/HTTP protocol), you can choose one of the two below (use <tt>man</tt> to find out more if interested):

curl -o hello.c [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]
wget [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]

In later tutorials/assignments, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page or a PDF.

===Compiling & Running Programs===
To compile a program, use <tt>gcc</tt>:

gcc -O2 hello.c -o hello

This compiles it with level 2 optimization and without debugging symbols, and the output program is hello.

To run, you have to specify where it is (the single dot <tt>.</tt> represents your current directory):

./hello

Remember you can always change directories using the <tt>cd</tt> command.

==Tasks/Questions==
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out mutiple ways for each (if you can).
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* Hardware information, e.g., RAM (remaining/total), CPU (model and number of cores), storage (what disks are connected and their capacity).
#* The name (binary path) of the current shell.
# Using the <tt>man</tt> command, find out what the following commands do: <tt>which</tt>, <tt>pwd</tt>, <tt>who</tt>, <tt>whoami</tt>, <tt>whereis</tt>, and <tt>whatis</tt>.
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <tt>cd</tt>) is internal or external?
#* For the program you just compiled above, what happens if you use <tt>hello</tt> to run it instead of <tt>./hello</tt> (and why does that happen)?
#* Figure out where at least three external commands reside on the system.
# Making your own commands: the <tt>PATH</tt> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Now, spend a few minutes to do a quick comparison: what are environment variables like in MS Windows? What are the two types? (hint: just search for "env" by clicking on the magnifier icon next to the Start menu of your Windows; you can also check its command line, if interested.)
# Going back to your Linux, look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# For those same program binaries, figure out what the permission bits mean by reading the man page of <tt>chmod</tt> (this is the command you could use to change those permission bits).
# What are the owner, group, and permissions of <tt>/etc/passwd</tt> and <tt>/etc/shadow</tt>? What are these files used for (and <u>how did you find this out</u>)?
# The <tt>ls</tt> command can be used to get a listing of the files in a directory. What options are passed to <tt>ls</tt> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <tt>bash</tt></u>? List at least 3 differences.

COMP3000 Operating Systems W25: Tutorial 1

2025-01-04T07:09:48Z

Lianyingzhao: Created page with "'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.''' This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu). ==General Instructions (same for all tutorials)== <div class="mw-collapsible mw-collapsed"> Click..."

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).

==General Instructions (same for all tutorials)==

<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''). Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section and include your thinking and exploration (or even confusion) for each question.

The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry under Tools -> Assignments). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.

You should also check in with your assigned TA. Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.

You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.

Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>

==Getting Started==
For the tutorials, you need to get access to a Linux machine. We strongly suggest you use an SCS Openstack instance (see the instructions below). Alternatively, if you have resource available and prefer to work offline with your desktop/laptop, you can also download <b>this VirtualBox image</b> ('''to be updated''') (which is the same with the Openstack image, and follow [https://carleton.ca/scs/2019/creating-a-new-host-only-adapter-in-virtualbox/ the instructions] to set up the host-only network adapter, after updating to <b>the latest version of VirtualBox</b>). Note that as our VM is headless (i.e., no GUI desktop), you are supposed to only either use its local command line or connect to it via ssh. This is intended since you are learning to use an OS for which command-line access is the most native and close to its implementation for a better understanding, before you enjoy GUI in your future work. You will need access to a system for the entire semester, ideally the same one.

The concepts covered below are mostly part of standard UNIX/Linux tutorials. Feel free to consult one or more of them. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!

Feel free to discuss this tutorial on Teams.

===Openstack===
'''For emphasis: do not take snapshots!''' (see below)

You can create a VM on the SCS openstack by following [[COMP3000 Operating Systems W25: Openstack Instructions | '''these steps''']]. If you encouter any issue and would like more reading, SCS already has [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/ lots of documentation on openstack], including a [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/openstack-step-by-step-guide/ step-by-step guide].

Make sure of the key steps: ①Carleton VPN, ②refresh your SCS account if needed, ③the COMP3000-W25 project, ④the COMP3000-W25 '''snapshot''' image, ⑤adding the ping-ssh-egress security group, and ⑥associating a floating IP address.

The 192.168.X.X IP addresses are private (and cannot be accessed outside of '''the Openstack cluster'''), the 134.117.X.X floating IP addresses can be accessed from '''the Carleton network''' and will allow you to access the wider Internet.

You need to ssh to your VM instance, to do the tutorial work. Windows 10/11, Ubuntu and MacOS all have SSH clients available from their command lines. Just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. Other tools supporting SSH (e.g., PuTTY) also work.
Once you are prompted to log in, '''the default user is student, default password is student'''. You'll have to change your password once you are first logged in. (If you want to change your password later, use the <tt>passwd</tt> command.)



The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:

scs-backup janedoe

and it will create a copy of everything in the student account’s home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed, and you'll make work for the tech staff who have to cancel what you did).'''

===Making use of Man pages===
The <tt>man</tt> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages. For example, if you look at files in an installed software package (e.g., <tt>dpkg -L ftp</tt> to list files in the ftp package) you can see there are a few files such as <tt>/usr/share/man/man1, 5, …</tt> They are the man page entries (sections).

The topics go beyond just software/command manuals, but also include conventions and abstract concepts (e.g., <tt>man syscalls</tt> and <tt>man man-pages</tt>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<tt>man 1 tee</tt>) and a system call (<tt>man 2 tee</tt>). A default will be used if the section is not specified.

For any commands mentioned in the tutorials, you can use <tt>man</tt> to find the usage. The advantage is the exact match with your current environment, compared to searching on the Internet.

Compared to the summary and brief information provided by the <tt>man</tt> command, you can also use the <tt>info</tt> command for a more detailed explanation for certain topics (just give it a try).

==Background==

===The Shell===
The shell or command line provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the operating system.

It works as an interpreter of the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the current default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of such commands (programmed in a scripting language). You will see shell scripts as we move on with this course.

===Permissions===
Your permission to access a file in Linux/Unix is determined by who you are logged in as. All files on the file system (including directories and other special files) have three different sets of permissions. The first set of permissions denotes the allowed file operations for '''the owner of the file''' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for '''a group of users''' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for '''everyone else''' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group. The set of permissions is threefold: read, write, and execute (rwx).

===Environment & Shell Variables===
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.

In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could write <tt>echo $HELLO</tt>).

If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:

X="Important Data"

just defines X as a shell variable. However, if you then type:

export X

X will be turned into an environment variable (no longer private), and so every subsequent program run by the shell will also get X. You can combine both in one line:

export X="Important Data"

This is the idiom for setting environment variables normally. You will be able to better understand the differnces between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).

One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:

export X = "Important Data"

as you now are giving export three arguments, not one.

To delete an environment variable, you can <tt>unset X</tt>.

===Downloading files using commands===
When working with the command line and you need to download a small number of files from a website (i.e., using the Web/HTTP protocol), you can choose one of the two below (use <tt>man</tt> to find out more if interested):

curl -o hello.c [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]
wget [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w25/tut1/hello.c]

In later tutorials/assignments, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page or a PDF.

===Compiling & Running Programs===
To compile a program, use <tt>gcc</tt>:

gcc -O2 hello.c -o hello

This compiles it with level 2 optimization and without debugging symbols, and the output program is hello.

To run, you have to specify where it is (the single dot <tt>.</tt> represents your current directory):

./hello

Remember you can always change directories using the <tt>cd</tt> command.

==Tasks/Questions==
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out mutiple ways for each (if you can).
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* Hardware information, e.g., RAM (remaining/total), CPU (model and number of cores), storage (what disks are connected and their capacity).
#* The name (binary path) of the current shell.
# Using the <tt>man</tt> command, find out what the following commands do: <tt>which</tt>, <tt>pwd</tt>, <tt>who</tt>, <tt>whoami</tt>, <tt>whereis</tt>, and <tt>whatis</tt>.
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <tt>cd</tt>) is internal or external?
#* For the program you just compiled above, what happens if you use <tt>hello</tt> to run it instead of <tt>./hello</tt> (and why does that happen)?
#* Figure out where at least three external commands reside on the system.
# Making your own commands: the <tt>PATH</tt> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Now, spend a few minutes to do a quick comparison: what are environment variables like in MS Windows? What are the two types? (hint: just search for "env" by clicking on the magnifier icon next to the Start menu of your Windows; you can also check its command line, if interested.)
# Going back to your Linux, look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# For those same program binaries, figure out what the permission bits mean by reading the man page of <tt>chmod</tt> (this is the command you could use to change those permission bits).
# What are the owner, group, and permissions of <tt>/etc/passwd</tt> and <tt>/etc/shadow</tt>? What are these files used for (and <u>how did you find this out</u>)?
# The <tt>ls</tt> command can be used to get a listing of the files in a directory. What options are passed to <tt>ls</tt> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w24/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <tt>bash</tt></u>? List at least 3 differences.

COMP3000 Operating Systems W25: Downloading files from your Openstack VM

2025-01-04T03:45:25Z

Lianyingzhao: /* Using scp */

COMP3000 Operating Systems W25: Downloading files from your Openstack VM

2025-01-04T03:17:59Z

Lianyingzhao: Created page with "'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.''' You will need to transfer files between the Openstack VM and your own PC from time to time, for example, when submitting your tutorial/assignment work. As your laptop/desktop does not have a fixed IP the VM can connect to, it is always better to initiate the connection <b>from your PC to the VM</b>,..."

COMP3000 Operating Systems 2025W

2025-01-03T07:44:45Z

Lianyingzhao: /* Various ways of connecting to SCS Openstack */

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

==Course Outline and Student Hours==
Aside from the updated version in Brightspace, you can also find it [https://outline.scs.carleton.ca/media/2025/W/COMP3000AW2025/COMP-3000-A-W-2025.pdf here]. Note that only the Brightspace version will contain contact information of the TAs and student hours.

Follow the schedule to meet a TA in the corresponding room. Alternatively, you may also contact the instructor or TAs by email, on MS Teams or booking an appointment to meet online over Zoom.

'''Extended student hours.''' Every one or two weeks, a Q&A session will be held online
with the instructor. This is similar to regular student hours except that multiple students will
be admitted into the session (like a class), and the focus is on lecture topics, e.g., you can ask
the instructor to re-explain a concept. For ad hoc technical problems, e.g., how to set up the
environment for a tutorial, please attend a TA’s student hours. The meeting time varies as intended
and will be announced below and on Brightspace so that if it does not work for a student, it likely will the next time. Priority will be given to students in Section B (if there are too many participants).

==Tutorial Instructions==
Attending tutorials is required (there is a 0.5 participation mark each) and you need to submit the work by the deadline indicated on Brightspace (for the other 1.5 marks). What is submitted is not graded for correctness but effort showing your attempts and thinking. <u>To receive full marks, make sure to include in your submission how you arrived at the solution or other thoughts (if you figured it out) or any confusion/attempts or even disagreements (if you failed to)</u>. After the submission, you should continue to study it if there is still anything you do not understand well, for the assignments and exams.

===[[COMP3000 Operating Systems W25: Openstack Instructions | Instructions for setting up an Openstack instance]]===



===[[COMP3000 Operating Systems W25: Downloading files from your Openstack VM | Downloading files from your Openstack VM]]===

===Tutorial 1===
Jan 9 and Jan 10, 2025: [[COMP3000 Operating Systems W25: Tutorial 1 | Instructions]]

===Tutorial 2===
Jan 16 and Jan 17, 2025: [[COMP3000 Operating Systems W25: Tutorial 2 | Instructions]]

===Tutorial 3===
Jan 23 and Jan 24, 2025: [[COMP3000 Operating Systems W25: Tutorial 3 | Instructions]]

===Tutorial 4===
Jan 30 and Jan 31, 2025: [[COMP3000 Operating Systems W25: Tutorial 4 | Instructions]]

===Tutorial 5===
Feb 6 and Feb 7, 2025: [[COMP3000 Operating Systems W25: Tutorial 5 | Instructions]]

===Tutorial 6===
Feb 27 and Feb 28, 2025: [[COMP3000 Operating Systems W25: Tutorial 6 | Instructions]]

===Tutorial 7===
Mar 6 and Mar 7, 2025: [[COMP3000 Operating Systems W25: Tutorial 7 | Instructions]]

===Tutorial 8===
Mar 13 and Mar 14, 2025: [[COMP3000 Operating Systems W25: Tutorial 8 | Instructions]]

===Tutorial 9===
Mar 20 and Mar 21, 2025: [[COMP3000 Operating Systems W25: Tutorial 9 | Instructions]]

==How to Do Well in This Course==
The goal here is to establish a <u>conceptual model</u> to understand how operating systems work to serve applications. It is not a course like math or data structures. '''To succeed, you need to be more hands-on and focus on figuring out how to find the answers (like learning to fish) instead of the answers alone (like being given a fish)'''. You can NOT expect to do well only by reading course material. You must work with a real OS, with the course material as guidelines. As you progress, you should be gradually able to imagine how various pieces of the OS fit together to function, which will be very useful in your everyday work/life with computers.

If you are asking a lot of "why"s, you are on the right track. Use all available resources (including the course material) as references, like a dictionary, to help answer your "why"s. For example, when you are asked about some information, based on your knowledge, think: 1) which part of the OS should have this information? 2) is it supposed to be available at the point in question? 3) then how do you retrieve that information? e.g., any tool/command/function call?
For sure, this would not be possible if you are just getting started, so a bit of confusion at the beginning is normal. As you move on, the conceptual model will become clearer as the basis for your thinking.

Mandatory textbook reading will be posted on Brightspace, and thus not all of the textbook will be covered. The exams and assignments are based on lectures, tutorials and designated textbook chapters.

The mini-quizzes are designed to ensure that you are following along by attending classes or listening to recorded lectures every week. So, they are straightforward, directly from class discussions (unlike the midterm and final exams).

COMP3000 Operating Systems W25: Connecting to SCS Openstack

2025-01-03T04:22:05Z

Lianyingzhao: /* If you are tired of typing passwords */

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

If you want to access the openstack environment during a video call (or when watching a movie online), please do not leave the Carleton VPN connected. You may also want to avoid your personal online activities going through the VPN or just simply need more convenience. Here are some alternative options. The instructions below assume you have created your VM instance with the VPN. Remember to replace <***> with your actual values.
==ProxyJump==

<span style="color:#0000ff;font-weight:bold">ssh -J <CarletonUserName>@access.scs.carleton.ca <VMUserName>@<VMIP></span>

This above is the easy and recommended way.
If your ssh version is a bit old, you may need to add a key exchange algorithm with:

ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 -J <CarletonUserName>@access.scs.carleton.ca <VMUserName>@<VMIP>

==ProxyJump - persistent==

In your ssh config file (e.g., <tt>~/.ssh/config</tt> for Linux and <tt>C:\Users\<UserName>\.ssh\config</tt> for Windows 10/11):

Host access
HostName access.scs.carleton.ca
KexAlgorithms +diffie-hellman-group1-sha1
User <CarletonUserName>
Host 134.117.*
ProxyJump access
User <VMUserName>

Then you can just <tt>ssh <VMIP></tt>.

==Windows (older versions)==

ssh -o ProxyCommand="C:\Windows\System32\OpenSSH\ssh.exe -W %h:%p <CarletonUserName>@access.scs.carleton.ca" <VMUserName>@<VMIP>

==Legacy (two terminal windows needed, not recommended)==

ssh -L 1200:<VMIP>:22 <CarletonUserName>@access.scs.carleton.ca

[stay logged in, and in another window do the following]

ssh -p 1200 <VMUserName>@localhost

You can choose a port other than 1200.

==If you are tired of typing passwords==
Follow [https://carleton.ca/scs/2019/passwordless-ssh-with-key-pairs/ these instructions] to set up your key pair to log in without having to remember a password. The instructions above for connecting to your instance are still needed, except that instead of typing a password every time you will be automatically logged in with the pre-configured key pair.
You will have a better understanding of this process when we discuss the security topics later in the term.

COMP3000 Operating Systems W25: Connecting to SCS Openstack

2025-01-03T04:17:53Z

Lianyingzhao: Created page with "'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.''' If you want to access the openstack environment during a video call (or when watching a movie online), please do not leave the Carleton VPN connected. You may also want to avoid your personal online activities going through the VPN or just simply need more convenience. Here are some alternative optio..."

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

If you want to access the openstack environment during a video call (or when watching a movie online), please do not leave the Carleton VPN connected. You may also want to avoid your personal online activities going through the VPN or just simply need more convenience. Here are some alternative options. The instructions below assume you have created your VM instance with the VPN. Remember to replace <***> with your actual values.
==ProxyJump==

<span style="color:#0000ff;font-weight:bold">ssh -J <CarletonUserName>@access.scs.carleton.ca <VMUserName>@<VMIP></span>

This above is the easy and recommended way.
If your ssh version is a bit old, you may need to add a key exchange algorithm with:

ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 -J <CarletonUserName>@access.scs.carleton.ca <VMUserName>@<VMIP>

==ProxyJump - persistent==

In your ssh config file (e.g., <tt>~/.ssh/config</tt> for Linux and <tt>C:\Users\<UserName>\.ssh\config</tt> for Windows 10/11):

Host access
HostName access.scs.carleton.ca
KexAlgorithms +diffie-hellman-group1-sha1
User <CarletonUserName>
Host 134.117.*
ProxyJump access
User <VMUserName>

Then you can just <tt>ssh <VMIP></tt>.

==Windows (older versions)==

ssh -o ProxyCommand="C:\Windows\System32\OpenSSH\ssh.exe -W %h:%p <CarletonUserName>@access.scs.carleton.ca" <VMUserName>@<VMIP>

==Legacy (two terminal windows needed, not recommended)==

ssh -L 1200:<VMIP>:22 <CarletonUserName>@access.scs.carleton.ca

[stay logged in, and in another window do the following]

ssh -p 1200 <VMUserName>@localhost

You can choose a port other than 1200.

==If you are tired of typing passwords==
Follow [https://carleton.ca/scs/2019/passwordless-ssh-with-key-pairs/ these instructions] to set up your key pair to log in without having to remember a password. You will have a better understanding of this process when we discuss the security topics later in the term.

COMP3000 Operating Systems W25: Openstack Instructions

2024-12-31T00:19:18Z

Lianyingzhao:

'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.'''

You can create a VM instance on the [https://carleton.ca/scs/tech-support/scs-open-stack/ SCS openstack] cluster for the tutorials, by doing the following:

(obvious steps are skipped)

# [https://carleton.ca/its/help-centre/remote-access/ Connect to Carleton's VPN].
# Log in to [https://openstack-stein.scs.carleton.ca/ https://openstack-stein.scs.carleton.ca/]
# Most of you would need to refresh your SCS account to have the right entitlements for our course (or you may not be able to log in or not listed in the COMP3000-W25 project): https://newacct.scs.carleton.ca/scs_authentication/newacct-policy-form.php
# Switch to the COMP3000-W25 project (top-left corner), and launch an instance (button on the right).

# Make a choice for necessary items according to the screenshots attached.
#;[[File:boot-source.jpg|300px]] 
#; Flavor

# You must add "ping-ssh-egress" to the security groups, or you'll not be able to ping/ssh to your instance.
#; Security groups:
#;[[File:security-group.jpg|580px]]
# Assign a floating IP.
#;[[File:floating-ip.jpg|170px]]
#; NEVER attempt to create a snapshot
# Then the new IP address (134.117.XXX.XXX) to the right of 192.168.XXX.XXX will be the one (floating IP) you will use to connect to your instance.
# <u>Note down</u> your floating IP and your instance name for subsequent uses for the rest of the term.

COMP3000 Operating Systems W25: Openstack Instructions

2024-12-31T00:09:52Z

Lianyingzhao: Created page with "You can create a VM instance on the [https://carleton.ca/scs/tech-support/scs-open-stack/ SCS openstack] cluster for the tutorials, by doing the following: (obvious steps are skipped) # [https://carleton.ca/its/help-centre/remote-access/ Connect to Carleton's VPN]. # Log in to [https://openstack-stein.scs.carleton.ca/ https://openstack-stein.scs.carleton.ca/] # Most of you would need to refresh your SCS account to have the right entitlements for our course (or you may..."

You can create a VM instance on the [https://carleton.ca/scs/tech-support/scs-open-stack/ SCS openstack] cluster for the tutorials, by doing the following:

(obvious steps are skipped)

# [https://carleton.ca/its/help-centre/remote-access/ Connect to Carleton's VPN].
# Log in to [https://openstack-stein.scs.carleton.ca/ https://openstack-stein.scs.carleton.ca/]
# Most of you would need to refresh your SCS account to have the right entitlements for our course (or you may not be able to log in or not listed in the COMP3000-W25 project): https://newacct.scs.carleton.ca/scs_authentication/newacct-policy-form.php
# Switch to the COMP3000-W25 project (top-left corner), and launch an instance (button on the right).

# Make a choice for necessary items according to the screenshots attached.
#;[[File:boot-source.jpg|300px]] 
#; Flavor

# You must add "ping-ssh-egress" to the security groups, or you'll not be able to ping/ssh to your instance.
#; Security groups:
#;[[File:security-group.jpg|580px]]
# Assign a floating IP.
#;[[File:floating-ip.jpg|170px]]
#; NEVER attempt to create a snapshot
# Then the new IP address (134.117.XXX.XXX) to the right of 192.168.XXX.XXX will be the one (floating IP) you will use to connect to your instance.
# <u>Note down</u> your floating IP and your instance name for subsequent uses for the rest of the term.

COMP3000 Operating Systems 2025W

2024-12-27T04:01:29Z

Lianyingzhao: Created page with "'''Note: this page's purpose is to host publicly available material linked from Brightspace. Please use Brightspace as the primary source of information to stay updated.''' ==Course Outline and Student Hours== Aside from the updated version in Brightspace, you can also find it [https://outline.scs.carleton.ca/media/2025/W/COMP3000AW2025/COMP-3000-A-W-2025.pdf here]. Note that only the Brightspace version will contain contact information of the TAs and student hours. Fo..."

COMP5900 COMP4900 2024F: Reading list

2024-10-08T00:40:13Z

Lianyingzhao:

You can only sign up for papers with an asterisk (*) at the end of their title.

If you are looking for more papers (from which you consider choosing to present):

[https://github.com/vschiavoni/sgx-papers Papers about/using Intel SGX]

You can also check out these [https://github.com/Maxul/Awesome-SGX-Open-Source SGX Open Source Projects] (with or without papers).

* Introduction to the course and trusted computing
** [https://www.cs.ox.ac.uk/files/1873/RR-08-11.PDF The ten-page introduction to Trusted Computing]
** [https://ieeexplore.ieee.org/document/7807249 Hardware-Based Trusted Computing Architectures for Isolation and Attestation]
** [https://ieeexplore.ieee.org/document/7345265 Trusted Execution Environment: What It is, and What It is Not]
** [https://dl.acm.org/doi/abs/10.1145/3672392 A Survey of Hardware Improvements to Secure Program Execution]
* Trust
** [https://dl.acm.org/citation.cfm?id=1849988 Bootstrapping Trust in Commodity Computers]
** [https://dl.acm.org/citation.cfm?id=358210 Reflections on Trusting Trust] (Turing Award lecture, 1984)
** [https://dl.acm.org/citation.cfm?id=3186101 SafeKeeper: Protecting Web Passwords using Trusted Execution Environments]
* Application of TC technologies
** [https://www.usenix.org/system/files/conference/osdi16/osdi16-arnautov.pdf SCONE: Secure Linux Containers with Intel SGX] (*) <span style="color:#0000ff;font-weight:bold"> Recommended </span>
** [https://ieeexplore.ieee.org/document/8464097 SGX-Tor: A Secure and Practical Tor Anonymity Network with SGX Enclaves] (*)
** [https://ieeexplore.ieee.org/document/8418608 EnclaveDB: A Secure Database using SGX] (*)
** [https://dl.acm.org/doi/10.1145/3052973.3053034 SGX-Log: Securing System Logs with SGX] (*)
** OS integrity: [https://link.springer.com/chapter/10.1007/978-3-030-29962-0_11 Nighthawk: Transparent System Introspection from Ring -3] (*)
** Hypervisor integrity: [https://ieeexplore.ieee.org/abstract/document/6682894 HyperCheck: A Hardware-Assisted Integrity Monitor] (*)
** Data protection: [https://dl.acm.org/doi/abs/10.1145/3190508.3190518 Pesos: Policy Enhanced Secure Object Store] (*)
* Making TC technologies more adoptable/usable
** [https://dl.acm.org/citation.cfm?id=1352625 Flicker: An Execution Infrastructure for TCB Minimization] (*)
** [https://www.usenix.org/system/files/conference/atc17/atc17-lind.pdf Glamdring: Automatic Application Partitioning for Intel SGX] (*)
** [https://www.usenix.org/conference/atc17/technical-sessions/presentation/tsai Graphene-SGX: A Practical Library OS for Unmodified Applications on SGX] (*) <span style="color:#0000ff;font-weight:bold"> Recommended </span>
** [https://dl.acm.org/doi/pdf/10.1145/3373376.3378469 Occlum: Secure and Efficient Multitasking Inside a Single Enclave of Intel SGX] (*)
** [https://www.usenix.org/system/files/sec20spring_tsai_prepub.pdf Civet: An Efficient Java Partitioning Framework for Hardware Enclaves] (*)
** [https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-hua.pdf vTZ: Virtualizing ARM TrustZone] (*)
** [https://dl.acm.org/doi/abs/10.1145/3319535.3354241 Towards Memory Safe Enclave Programming with Rust-SGX] [Rust + SGX] (*)
** [https://dl.acm.org/doi/abs/10.1145/3427228.3427262 RusTEE: Developing Memory-Safe ARM TrustZone Applications] [Rust + ARM] (*)
** [https://ieeexplore.ieee.org/abstract/document/8945648 SGXPy: Protecting integrity of Python applications with Intel SGX] [Python + SGX] (*)
** [https://dl.acm.org/doi/abs/10.1145/2541940.2541949 Using ARM TrustZone to Build a Trusted Language Runtime for Mobile Applications] [.NET + SGX] (*)
* Side-channel attacks
** [https://ieeexplore.ieee.org/document/8141882 Systematic Classification of Side-Channel Attacks: A Case Study for Mobile Devices]
** [https://www.usenix.org/conference/usenixsecurity18/presentation/bulck Foreshadow: Extracting the Keys to the Intel SGX Kingdom with Transient Out-of-Order Execution] (*)
** [https://www.usenix.org/conference/usenixsecurity18/presentation/lipp Meltdown: Reading Kernel Memory from User Space] (*)
** [https://ieeexplore.ieee.org/abstract/document/8835233 Spectre Attacks: Exploiting Speculative Execution] (*)
** Latest: [https://www.usenix.org/system/files/usenixsecurity23-moghimi.pdf Downfall: Exploiting Speculative Data Gathering] (*)
* Internal misbehavior: memory attacks
** [https://ieeexplore.ieee.org/document/6547101 SoK: Eternal War in Memory] (*)
** [https://link.springer.com/chapter/10.1007/978-3-642-33338-5_5 Memory Errors: The Past, the Present, and the Future]
** Defense: [https://dl.acm.org/doi/abs/10.1145/2976749.2978358 C-FLAT: Control-Flow Attestation for Embedded Systems Software] (*)
** Defense: [https://www.usenix.org/system/files/sec21summer_mirzazade.pdf PTAuth: Temporal Memory Safety via Robust Points-to Authentication] (*)
** Attacking the defense: [https://dl.acm.org/doi/abs/10.1145/3470496.3527429 PACMAN: Attacking ARM Pointer Authentication with Speculative Execution] (*)
** Attacking the defense: [https://arxiv.org/pdf/2406.08719 TIKTAG: Breaking ARM’s Memory Tagging Extension with Speculative Execution] (*)
* Human authenticating machine
** [https://www.usenix.org/legacy/event/hotsec07/tech/full_papers/mccune/mccune.pdf Turtles All The Way Down: Research Challenges in User-Based Attestation]
** [https://fc13.ifca.ai/proc/9-1.pdf Stark: Tamperproof Authentication to Resist Keylogging] (*)
** [https://dl.acm.org/citation.cfm?id=1854103 Evil maid goes after PGP whole disk encryption]
** [https://netsec.ethz.ch/publications/papers/PRISM-2007.pdf PRISM/ Human-Verifiable Code Execution] (*)
* State continuity
** [https://www.microsoft.com/en-us/research/publication/memoir-practical-state-continuity-for-protected-modules/ Memoir: Practical state continuity for protected modules] (*)
** [https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-matetic.pdf ROTE: Rollback Protection for Trusted Execution] (*)
** [https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_strackx.pdf Ariadne: A Minimal Approach to State Continuity] (*)
* Secure input/output
** [https://dl.acm.org/doi/10.1145/3210240.3210334 SeCloak: ARM TrustZone-based Mobile Peripheral Control]
** [https://dl.acm.org/citation.cfm?id=2637225 Building trusted path on untrusted device drivers for mobile devices] (*)
** [https://dl.acm.org/citation.cfm?id=3210338 TruZ-Droid: Integrating TrustZone with Mobile Operating System] (*)
** [https://ieeexplore.ieee.org/document/8859293 Establishing Trusted I/O Paths for SGX Client Systems with Aurora] (*)
** [https://dl.acm.org/doi/10.1145/3210240.3210330 VButton: Practical Attestation of User-driven Operations in Mobile Apps] (*)
** [https://www.ndss-symposium.org/ndss-paper/protection-root-of-trust-for-io-in-compromised-platforms/ ProtectIOn: Root-of-Trust for IO in Compromised Platforms] (*)
** [https://ieeexplore-ieee-org.ezproxy.u-pec.fr/iel7/8826229/8835208/08835331.pdf Fidelius: Protecting User Secrets from Compromised Browsers] (*)
* Proposed hardware improvements
** Fine-grained isolation: [https://www.usenix.org/system/files/conference/usenixsecurity18/sec18-frassetto.pdf IMIX: In-Process Memory Isolation EXtension] (*)
** Memory safety: [https://ieeexplore.ieee.org/abstract/document/7167258 HAFIX: Hardware-Assisted Flow Integrity Extension] (*)
** Integrity monitoring: [https://dl.acm.org/doi/10.1145/3134600.3134622 Co-processor-based Behavior Monitoring: Application to the Detection of Attacks Against the System Management Mode] (*)
** Integrity monitoring: [https://dl.acm.org/doi/abs/10.1145/2508148.2485956 CPU Transparent Protection of OS Kernel and Hypervisor Integrity with Programmable DRAM] (*)
* Proposals based on existing (non-security) hardware support
** [https://dl.acm.org/citation.cfm?id=2660316&dl=ACM&coll=DL PixelVault: Using GPUs for Securing Cryptographic Operations] (*)
** [https://www.usenix.org/system/files/osdi18-volos.pdf Graviton: Trusted Execution Environments on GPUs] (*)
** [https://dl.acm.org/doi/10.1145/3093337.3037716 GRIFFIN: Guarding Control Flows Using Intel Processor Trace] (*)
** [https://www.ndss-symposium.org/wp-content/uploads/2017/09/ndss2017_07-2_Shih_paper.pdf T-SGX: Eradicating Controlled-Channel Attacks Against Enclave Programs] (*)

COMP5900 COMP4900 2024F: Openstack Instructions

2024-09-18T17:18:06Z

Lianyingzhao:

You can create a VM instance on the [https://carleton.ca/scs/tech-support/scs-open-stack/ SCS openstack] cluster for the assignments, by doing the following:

Obvious steps are skipped. You can always reach out to the TA or the instructor for any technical questions.

# [https://carleton.ca/its/help-centre/remote-access/ Connect to Carleton's VPN] (if not on campus).
# Log in to [https://openstack-stein.scs.carleton.ca/ https://openstack-stein.scs.carleton.ca/]
# Most of you would need to refresh your SCS account to have the right entitlements for our course (or you may not be able to log in or not listed in the COMP4900K-F24 project): https://newacct.scs.carleton.ca/scs_authentication/newacct-policy-form.php
# Switch to the COMP4900K-F24 project (top-left corner), and launch an instance (button on the right).
#;[[File:select-project-f24.jpg|300px]]
# Make a choice for necessary items according to the screenshots attached.
#;[[File:boot-source.jpg|300px]] [[File:select-image-f24.jpg|580px]]
#; Flavor
#;[[File:select-flavor-f24.jpg|580px]]
# You must add "ping-ssh-egress" to the security groups, or you'll not be able to ping/ssh to your instance.
#; Security groups:
#;[[File:security-group.jpg|580px]]
# Assign a floating IP.
#;[[File:floating-ip.jpg|170px]]
#; NEVER attempt to create a snapshot
# Then the new IP address (134.117.XXX.XXX) to the right of 192.168.XXX.XXX will be the one (floating IP) you will use to connect to your instance.
# <u>Note down</u> your floating IP and your instance name for subsequent uses for the rest of the term.
# Log in to your instance with username "student" and password "student" (you will be prompted to change your password right away).

COMP5900 COMP4900 2024F: Reading list

2024-09-18T00:56:22Z

Lianyingzhao:

You can only sign up for papers with an asterisk (*) at the end of their title.

If you are looking for more papers (from which you consider choosing to present):

[https://github.com/vschiavoni/sgx-papers Papers about/using Intel SGX]

You can also check out these [https://github.com/Maxul/Awesome-SGX-Open-Source SGX Open Source Projects] (with or without papers).

* Introduction to the course and trusted computing
** [https://www.cs.ox.ac.uk/files/1873/RR-08-11.PDF The ten-page introduction to Trusted Computing]
** [https://ieeexplore.ieee.org/document/7807249 Hardware-Based Trusted Computing Architectures for Isolation and Attestation]
** [https://ieeexplore.ieee.org/document/7345265 Trusted Execution Environment: What It is, and What It is Not]
** [https://dl.acm.org/doi/abs/10.1145/3672392 A Survey of Hardware Improvements to Secure Program Execution]
* Trust
** [https://dl.acm.org/citation.cfm?id=1849988 Bootstrapping Trust in Commodity Computers]
** [https://dl.acm.org/citation.cfm?id=358210 Reflections on Trusting Trust] (Turing Award lecture, 1984)
** [https://dl.acm.org/citation.cfm?id=3186101 SafeKeeper: Protecting Web Passwords using Trusted Execution Environments]
* Application of TC technologies
** [https://www.usenix.org/system/files/conference/osdi16/osdi16-arnautov.pdf SCONE: Secure Linux Containers with Intel SGX] (*) <span style="color:#0000ff;font-weight:bold"> Recommended </span>
** [https://ieeexplore.ieee.org/document/8464097 SGX-Tor: A Secure and Practical Tor Anonymity Network with SGX Enclaves] (*)
** [https://ieeexplore.ieee.org/document/8418608 EnclaveDB: A Secure Database using SGX] (*)
** [https://dl.acm.org/doi/10.1145/3052973.3053034 SGX-Log: Securing System Logs with SGX] (*)
** OS integrity: [https://link.springer.com/chapter/10.1007/978-3-030-29962-0_11 Nighthawk: Transparent System Introspection from Ring -3] (*)
** Hypervisor integrity: [https://ieeexplore.ieee.org/abstract/document/6682894 HyperCheck: A Hardware-Assisted Integrity Monitor] (*)
** Data protection: [https://dl.acm.org/doi/abs/10.1145/3190508.3190518 Pesos: Policy Enhanced Secure Object Store] (*)
* Making TC technologies more adoptable/usable
** [https://dl.acm.org/citation.cfm?id=1352625 Flicker: An Execution Infrastructure for TCB Minimization] (*)
** [https://www.usenix.org/system/files/conference/atc17/atc17-lind.pdf Glamdring: Automatic Application Partitioning for Intel SGX] (*)
** [https://www.usenix.org/conference/atc17/technical-sessions/presentation/tsai Graphene-SGX: A Practical Library OS for Unmodified Applications on SGX] (*) <span style="color:#0000ff;font-weight:bold"> Recommended </span>
** [https://dl.acm.org/doi/pdf/10.1145/3373376.3378469 Occlum: Secure and Efficient Multitasking Inside a Single Enclave of Intel SGX] (*)
** [https://www.usenix.org/system/files/sec20spring_tsai_prepub.pdf Civet: An Efficient Java Partitioning Framework for Hardware Enclaves] (*)
** [https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-hua.pdf vTZ: Virtualizing ARM TrustZone] (*)
** [https://dl.acm.org/doi/abs/10.1145/3319535.3354241 Towards Memory Safe Enclave Programming with Rust-SGX] [Rust + SGX] (*)
** [https://dl.acm.org/doi/abs/10.1145/3427228.3427262 RusTEE: Developing Memory-Safe ARM TrustZone Applications] [Rust + ARM] (*)
** [https://ieeexplore.ieee.org/abstract/document/8945648 SGXPy: Protecting integrity of Python applications with Intel SGX] [Python + SGX] (*)
** [https://dl.acm.org/doi/abs/10.1145/2541940.2541949 Using ARM TrustZone to Build a Trusted Language Runtime for Mobile Applications] [.NET + SGX] (*)
* Side-channel attacks
** [https://ieeexplore.ieee.org/document/8141882 Systematic Classification of Side-Channel Attacks: A Case Study for Mobile Devices]
** [https://www.usenix.org/conference/usenixsecurity18/presentation/bulck Foreshadow: Extracting the Keys to the Intel SGX Kingdom with Transient Out-of-Order Execution] (*)
** [https://www.usenix.org/conference/usenixsecurity18/presentation/lipp Meltdown: Reading Kernel Memory from User Space] (*)
** [https://ieeexplore.ieee.org/abstract/document/8835233 Spectre Attacks: Exploiting Speculative Execution] (*)
** Latest: [https://www.usenix.org/system/files/usenixsecurity23-moghimi.pdf Downfall: Exploiting Speculative Data Gathering] (*)
* Internal misbehavior: memory attacks
** [https://ieeexplore.ieee.org/document/6547101 SoK: Eternal War in Memory] (*)
** [https://link.springer.com/chapter/10.1007/978-3-642-33338-5_5 Memory Errors: The Past, the Present, and the Future]
** Defense: [https://dl.acm.org/doi/abs/10.1145/2976749.2978358 C-FLAT: Control-Flow Attestation for Embedded Systems Software] (*)
** Defense: [https://www.usenix.org/system/files/sec21summer_mirzazade.pdf PTAuth: Temporal Memory Safety via Robust Points-to Authentication] (*)
** Attacking the defense: [https://dl.acm.org/doi/abs/10.1145/3470496.3527429 PACMAN: Attacking ARM Pointer Authentication with Speculative Execution] (*)
* Human authenticating machine
** [https://www.usenix.org/legacy/event/hotsec07/tech/full_papers/mccune/mccune.pdf Turtles All The Way Down: Research Challenges in User-Based Attestation]
** [https://fc13.ifca.ai/proc/9-1.pdf Stark: Tamperproof Authentication to Resist Keylogging] (*)
** [https://dl.acm.org/citation.cfm?id=1854103 Evil maid goes after PGP whole disk encryption]
** [https://netsec.ethz.ch/publications/papers/PRISM-2007.pdf PRISM/ Human-Verifiable Code Execution] (*)
* State continuity
** [https://www.microsoft.com/en-us/research/publication/memoir-practical-state-continuity-for-protected-modules/ Memoir: Practical state continuity for protected modules] (*)
** [https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-matetic.pdf ROTE: Rollback Protection for Trusted Execution] (*)
** [https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_strackx.pdf Ariadne: A Minimal Approach to State Continuity] (*)
* Secure input/output
** [https://dl.acm.org/doi/10.1145/3210240.3210334 SeCloak: ARM TrustZone-based Mobile Peripheral Control]
** [https://dl.acm.org/citation.cfm?id=2637225 Building trusted path on untrusted device drivers for mobile devices] (*)
** [https://dl.acm.org/citation.cfm?id=3210338 TruZ-Droid: Integrating TrustZone with Mobile Operating System] (*)
** [https://ieeexplore.ieee.org/document/8859293 Establishing Trusted I/O Paths for SGX Client Systems with Aurora] (*)
** [https://dl.acm.org/doi/10.1145/3210240.3210330 VButton: Practical Attestation of User-driven Operations in Mobile Apps] (*)
** [https://www.ndss-symposium.org/ndss-paper/protection-root-of-trust-for-io-in-compromised-platforms/ ProtectIOn: Root-of-Trust for IO in Compromised Platforms] (*)
** [https://ieeexplore-ieee-org.ezproxy.u-pec.fr/iel7/8826229/8835208/08835331.pdf Fidelius: Protecting User Secrets from Compromised Browsers] (*)
* Proposed hardware improvements
** Fine-grained isolation: [https://www.usenix.org/system/files/conference/usenixsecurity18/sec18-frassetto.pdf IMIX: In-Process Memory Isolation EXtension] (*)
** Memory safety: [https://ieeexplore.ieee.org/abstract/document/7167258 HAFIX: Hardware-Assisted Flow Integrity Extension] (*)
** Integrity monitoring: [https://dl.acm.org/doi/10.1145/3134600.3134622 Co-processor-based Behavior Monitoring: Application to the Detection of Attacks Against the System Management Mode] (*)
** Integrity monitoring: [https://dl.acm.org/doi/abs/10.1145/2508148.2485956 CPU Transparent Protection of OS Kernel and Hypervisor Integrity with Programmable DRAM] (*)
* Proposals based on existing (non-security) hardware support
** [https://dl.acm.org/citation.cfm?id=2660316&dl=ACM&coll=DL PixelVault: Using GPUs for Securing Cryptographic Operations] (*)
** [https://www.usenix.org/system/files/osdi18-volos.pdf Graviton: Trusted Execution Environments on GPUs] (*)
** [https://dl.acm.org/doi/10.1145/3093337.3037716 GRIFFIN: Guarding Control Flows Using Intel Processor Trace] (*)
** [https://www.ndss-symposium.org/wp-content/uploads/2017/09/ndss2017_07-2_Shih_paper.pdf T-SGX: Eradicating Controlled-Channel Attacks Against Enclave Programs] (*)

COMP5900 COMP4900 2024F: Reading list

2024-09-18T00:45:11Z

Lianyingzhao:

You can only sign up for papers with an asterisk (*) at the end of their title.

If you are looking for more papers (from which you consider choosing to present):

[https://github.com/vschiavoni/sgx-papers Papers about/using Intel SGX]

You can also check out these [https://github.com/Maxul/Awesome-SGX-Open-Source SGX Open Source Projects] (with or without papers).

* Introduction to the course and trusted computing
** [https://www.cs.ox.ac.uk/files/1873/RR-08-11.PDF The ten-page introduction to Trusted Computing]
** [https://ieeexplore.ieee.org/document/7807249 Hardware-Based Trusted Computing Architectures for Isolation and Attestation]
** [https://ieeexplore.ieee.org/document/7345265 Trusted Execution Environment: What It is, and What It is Not]
** [https://dl.acm.org/doi/abs/10.1145/3672392 A Survey of Hardware Improvements to Secure Program Execution]
* Trust
** [https://dl.acm.org/citation.cfm?id=1849988 Bootstrapping Trust in Commodity Computers]
** [https://dl.acm.org/citation.cfm?id=358210 Reflections on Trusting Trust] (Turing Award lecture, 1984)
** [https://dl.acm.org/citation.cfm?id=3186101 SafeKeeper: Protecting Web Passwords using Trusted Execution Environments]
* Application of TC technologies
** [https://www.usenix.org/system/files/conference/osdi16/osdi16-arnautov.pdf SCONE: Secure Linux Containers with Intel SGX] (*) <span style="color:#0000ff;font-weight:bold"> Recommended </span>
** [https://ieeexplore.ieee.org/document/8464097 SGX-Tor: A Secure and Practical Tor Anonymity Network with SGX Enclaves] (*)
** [https://ieeexplore.ieee.org/document/8418608 EnclaveDB: A Secure Database using SGX] (*)
** [https://dl.acm.org/doi/10.1145/3052973.3053034 SGX-Log: Securing System Logs with SGX] (*)
** OS integrity: [https://link.springer.com/chapter/10.1007/978-3-030-29962-0_11 Nighthawk: Transparent System Introspection from Ring -3] (*)
** Hypervisor integrity: [https://ieeexplore.ieee.org/abstract/document/6682894 HyperCheck: A Hardware-Assisted Integrity Monitor] (*)
** Data protection: [https://dl.acm.org/doi/abs/10.1145/3190508.3190518 Pesos: Policy Enhanced Secure Object Store] (*)
* Making TC technologies more adoptable/usable
** [https://dl.acm.org/citation.cfm?id=1352625 Flicker: An Execution Infrastructure for TCB Minimization] (*)
** [https://www.usenix.org/system/files/conference/atc17/atc17-lind.pdf Glamdring: Automatic Application Partitioning for Intel SGX] (*)
** [https://dl.acm.org/doi/pdf/10.1145/3373376.3378469 Occlum: Secure and Efficient Multitasking Inside a Single Enclave of Intel SGX] (*)
** [https://www.usenix.org/system/files/sec20spring_tsai_prepub.pdf Civet: An Efficient Java Partitioning Framework for Hardware Enclaves] (*)
** [https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-hua.pdf vTZ: Virtualizing ARM TrustZone] (*)
** [https://dl.acm.org/doi/abs/10.1145/3319535.3354241 Towards Memory Safe Enclave Programming with Rust-SGX] [Rust + SGX] (*)
** [https://dl.acm.org/doi/abs/10.1145/3427228.3427262 RusTEE: Developing Memory-Safe ARM TrustZone Applications] [Rust + ARM] (*)
** [https://ieeexplore.ieee.org/abstract/document/8945648 SGXPy: Protecting integrity of Python applications with Intel SGX] [Python + SGX] (*)
** [https://dl.acm.org/doi/abs/10.1145/2541940.2541949 Using ARM TrustZone to Build a Trusted Language Runtime for Mobile Applications] [.NET + SGX] (*)
* Side-channel attacks
** [https://ieeexplore.ieee.org/document/8141882 Systematic Classification of Side-Channel Attacks: A Case Study for Mobile Devices]
** [https://www.usenix.org/conference/usenixsecurity18/presentation/bulck Foreshadow: Extracting the Keys to the Intel SGX Kingdom with Transient Out-of-Order Execution] (*)
** [https://www.usenix.org/conference/usenixsecurity18/presentation/lipp Meltdown: Reading Kernel Memory from User Space] (*)
** [https://ieeexplore.ieee.org/abstract/document/8835233 Spectre Attacks: Exploiting Speculative Execution] (*)
** Latest: [https://www.usenix.org/system/files/usenixsecurity23-moghimi.pdf Downfall: Exploiting Speculative Data Gathering] (*)
* Internal misbehavior: memory attacks
** [https://ieeexplore.ieee.org/document/6547101 SoK: Eternal War in Memory] (*)
** [https://link.springer.com/chapter/10.1007/978-3-642-33338-5_5 Memory Errors: The Past, the Present, and the Future]
** Defense: [https://dl.acm.org/doi/abs/10.1145/2976749.2978358 C-FLAT: Control-Flow Attestation for Embedded Systems Software] (*)
** Defense: [https://www.usenix.org/system/files/sec21summer_mirzazade.pdf PTAuth: Temporal Memory Safety via Robust Points-to Authentication] (*)
** Attacking the defense: [https://dl.acm.org/doi/abs/10.1145/3470496.3527429 PACMAN: Attacking ARM Pointer Authentication with Speculative Execution] (*)
* Human authenticating machine
** [https://www.usenix.org/legacy/event/hotsec07/tech/full_papers/mccune/mccune.pdf Turtles All The Way Down: Research Challenges in User-Based Attestation]
** [https://fc13.ifca.ai/proc/9-1.pdf Stark: Tamperproof Authentication to Resist Keylogging] (*)
** [https://dl.acm.org/citation.cfm?id=1854103 Evil maid goes after PGP whole disk encryption]
** [https://netsec.ethz.ch/publications/papers/PRISM-2007.pdf PRISM/ Human-Verifiable Code Execution] (*)
* State continuity
** [https://www.microsoft.com/en-us/research/publication/memoir-practical-state-continuity-for-protected-modules/ Memoir: Practical state continuity for protected modules] (*)
** [https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-matetic.pdf ROTE: Rollback Protection for Trusted Execution] (*)
** [https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_strackx.pdf Ariadne: A Minimal Approach to State Continuity] (*)
* Secure input/output
** [https://dl.acm.org/doi/10.1145/3210240.3210334 SeCloak: ARM TrustZone-based Mobile Peripheral Control]
** [https://dl.acm.org/citation.cfm?id=2637225 Building trusted path on untrusted device drivers for mobile devices] (*)
** [https://dl.acm.org/citation.cfm?id=3210338 TruZ-Droid: Integrating TrustZone with Mobile Operating System] (*)
** [https://ieeexplore.ieee.org/document/8859293 Establishing Trusted I/O Paths for SGX Client Systems with Aurora] (*)
** [https://dl.acm.org/doi/10.1145/3210240.3210330 VButton: Practical Attestation of User-driven Operations in Mobile Apps] (*)
** [https://www.ndss-symposium.org/ndss-paper/protection-root-of-trust-for-io-in-compromised-platforms/ ProtectIOn: Root-of-Trust for IO in Compromised Platforms] (*)
** [https://ieeexplore-ieee-org.ezproxy.u-pec.fr/iel7/8826229/8835208/08835331.pdf Fidelius: Protecting User Secrets from Compromised Browsers] (*)
* Proposed hardware improvements
** Fine-grained isolation: [https://www.usenix.org/system/files/conference/usenixsecurity18/sec18-frassetto.pdf IMIX: In-Process Memory Isolation EXtension] (*)
** Memory safety: [https://ieeexplore.ieee.org/abstract/document/7167258 HAFIX: Hardware-Assisted Flow Integrity Extension] (*)
** Integrity monitoring: [https://dl.acm.org/doi/10.1145/3134600.3134622 Co-processor-based Behavior Monitoring: Application to the Detection of Attacks Against the System Management Mode] (*)
** Integrity monitoring: [https://dl.acm.org/doi/abs/10.1145/2508148.2485956 CPU Transparent Protection of OS Kernel and Hypervisor Integrity with Programmable DRAM] (*)
* Proposals based on existing (non-security) hardware support
** [https://dl.acm.org/citation.cfm?id=2660316&dl=ACM&coll=DL PixelVault: Using GPUs for Securing Cryptographic Operations] (*)
** [https://www.usenix.org/system/files/osdi18-volos.pdf Graviton: Trusted Execution Environments on GPUs] (*)
** [https://dl.acm.org/doi/10.1145/3093337.3037716 GRIFFIN: Guarding Control Flows Using Intel Processor Trace] (*)
** [https://www.ndss-symposium.org/wp-content/uploads/2017/09/ndss2017_07-2_Shih_paper.pdf T-SGX: Eradicating Controlled-Channel Attacks Against Enclave Programs] (*)

File:Select-project-f24.jpg

2024-09-10T22:49:44Z

Lianyingzhao: Lianyingzhao uploaded a new version of File:Select-project-f24.jpg

File:Select-image-f24.jpg

2024-09-10T22:44:34Z

Lianyingzhao: Lianyingzhao uploaded a new version of File:Select-image-f24.jpg

File:Select-flavor-f24.jpg

2024-09-10T22:41:20Z

Lianyingzhao: Lianyingzhao reverted File:Select-flavor-f24.jpg to an old version