COMP3000 Operating Systems 2023F Tutorials: Tutorial 1: Difference between revisions

From Soma-notes
Abdou (talk | contribs)
Created page with "This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu). ==General Instructions (same for all tutorials)== <div class="mw-collapsible mw-collapsed"> Click on Expand to view. <div class="mw-collapsible-content"> Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers..."
 
Abdou (talk | contribs)
 
(26 intermediate revisions by the same user not shown)
Line 1: Line 1:
This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).  
This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).  
==General Instructions (same for all tutorials)==
<div class="mw-collapsible mw-collapsed">
Click on Expand to view.
<div class="mw-collapsible-content">
Tutorials are graded based on participation and effort ('''so no need to struggle to have the “correct” answers — what matters is the attempts and sufficient thinking reflected in your answers'''), but you should still turn in your work. Submit your answers on Brightspace as a single text file named "<username>-comp3000-t<n>.txt" (where username is your MyCarletonOne username and n is the tutorial number). The first four lines of this file should be "COMP 3000 Tutorial <n>", your name, student number, and the date of submission. Your submitted answers must at least respond to all the items in the '''Tasks/Questions''' section.
The deadline is usually four days after the tutorial date (see the actual deadline on the submission entry). Note that the submission entry is enforced by the system, so you may fail to get the effort marks even if it is one minute past the deadline.
You should also check in with your assigned TA (in person or by responding to a poll if online). Your TA will be your first point of contact when you have questions or encounter any issues during the tutorial session.
You get 1.5 marks for submitting answers that shows your effort and 0.5 for checking in, making this tutorial worth 2 points total.
Read through the instructions before starting your work to get an overall picture. When source files are needed, you can download them by clicking on the hyperlink.
</div>
</div>


==Getting Started==
==Getting Started==
For the tutorials, you need to get access to a Linux machine. We strongly suggest you use an SCS Openstack instance (see the instructions below). Alternatively, if you have resource available and prefer to work offline with your desktop/laptop, you can also download [https://git.scs.carleton.ca/downloads/CourseVirtualMachines/2022F-2023W/COMP3000-W23.ova <b>this VirtualBox image</b>] (which is the same with the Openstack image). You will need access to a system for the entire semester, ideally the same one.
The concepts covered below are mostly part of a standard tutorial on UNIX/Linux. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!
 
The concepts covered below are mostly part of standard UNIX/Linux tutorials. Feel free to consult one or more of them. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!
 
Feel free to discuss this tutorial on Teams.
 
===Openstack===
'''For emphasis: do not take snapshots!''' (see below)
 
You can create a VM on the SCS openstack by following [[COMP3000 Operating Systems W23: Openstack Instructions | '''these steps''']]. If you encouter any issue and would like more reading, SCS already has [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/ lots of documentation on openstack], including a [https://carleton.ca/scs/tech-support/scs-open-stack/openstack-technical-support/openstack-step-by-step-guide/ step-by-step guide].
 
Make sure of the key steps: ①Carleton VPN, ②refresh your SCS account if needed, ③the COMP3000-W23 project, ④the COMP3000-W23 '''snapshot''' image, ⑤adding the ping-ssh-egress security group, and ⑥associating a floating IP address.
 
The 192.168.X.X IP addresses are private (and cannot be accessed outside of '''the Openstack cluster'''), the 134.117.X.X floating IP addresses can be accessed from '''the Carleton network''' and will allow you to access the wider Internet.
 
You need to ssh to your VM instance, to do the tutorial work. Windows 10/11, Ubuntu and MacOS all have SSH clients available from their command lines. Just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. Other tools supporting SSH (e.g., PuTTY) also work.
Once you are prompted to log in, the default user is student, default password is student. You'll have to change your password once you are first logged in. (If you want to change your password later, use the <tt>passwd</tt> command.)
 
For various alternative ways to acess your instance, refer to here (using the <tt>-J</tt> option is recommended; take a look).
 
The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type:
 
scs-backup janedoe
 
and it will create a copy of everything in the student account’s home directory (note: you can customize it) in a directory called "COMP3000VM-backup" in your SCS home directory. You can ssh/sftp to <tt>access.scs.carleton.ca</tt> in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.
 
'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed, and you'll make work for the tech staff who have to cancel what you did).'''


===Making use of Man pages===
===Making use of Man pages===
The <tt>man</tt> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages.  For example, if you look at files in an installed software package (e.g., <tt>dpkg -L ftp</tt> to list files in the ftp package) you can see there are a few files such as <tt>/usr/share/man/man1, 5, …</tt> They are the man page entries (sections).
The <code>man</code> (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages.  For example, if you look at files in an installed software package (e.g., <code>dpkg -L ftp</code> to list files in the ftp package) you can see there are a few files such as <code>/usr/share/man/man1, 5, …</code> They are the man page entries (sections).


The topics go beyond just software/command manuals, but also include conventions and abstract concepts (e.g., <tt>man syscalls</tt> and <tt>man man-pages</tt>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<tt>man 1 tee</tt>) and a system call (<tt>man 2 tee</tt>). A default will be used if the section is not specified.
The topics go beyond just software/command manuals. They also include conventions and abstract concepts (e.g., <code>man syscalls</code> and <code>man man-pages</code>) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (<code>man 1 tee</code>) and a system call (<code>man 2 tee</code>). A default will be used if the section is not specified.


For any commands mentioned in the tutorials, you can use <tt>man</tt> to find the usage. The advantage is the exact match with your current environment, compared to searching on the Internet.
For any commands mentioned in the tutorials, you can use <code>man</code> to find the usage. The advantage is that the man page documentation provides an exact match to your current environment, compared to searching the Internet.


Compared to the summary and brief information provided by the <tt>man</tt> command, you can also use the <tt>info</tt> command for a more detailed explanation for certain topics (just give it a try).
Compared to the summary and brief information provided by the <code>man</code> command, you can also use the <code>info</code> command for a more detailed explanation for certain topics (just give it a try).


==Background==
==Background==


===The Shell===
===The Shell===
The shell or command line provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the operating system.
The shell, also called "command line" or more commonly "terminal", provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the OS.


It works as an interpreter of the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the current default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of such commands (programmed in a scripting language). You will see shell scripts as we move on with this course.
It works as an interpreter to the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include <code>bash</code> (the default we are using in Ubuntu), <code>ksh</code>, <code>zsh</code>, <code>csh</code>, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a [https://en.wikipedia.org/wiki/Shell_script shell script], which is a sequence of commands (programmed in a scripting language). You will see shell scripts as we navigate this course.


===Permissions===
===File Permissions===
Your permission to access a file in Linux/Unix is determined by who you are logged in as. All files on the file system (including directories and other special files) have three different sets of permissions. The first set of permissions denotes the allowed file operations for '''the owner of the file''' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for '''a group of users''' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for '''everyone else''' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group. The set of permissions is threefold: read, write, and execute (rwx).
Each file on the system (including directories and other special files) has an owner (user), and belongs to a group.
 
File permissions in Linux are: read, write, and execute, denoted <code>rwx</code>. Each file has three sets of <code>rwx</code> permissions. The first set denotes the type of access allowed to ''the owner of the file'' ('''rwx'''rwxrwx). The second set of permissions denotes the allowed file operations for ''a group of users'' (rwx'''rwx'''rwx). The third set of permissions denotes the allowed file operations for ''everyone else'' (rwxrwx'''rwx'''). A file is always owned by someone and is always associated with a group.  
 
Whether or not you will be allowed to ''access'' (read from, write to, or execute) a file in Linux/Unix is determined by who (i.e., which user) you are logged in as, and the "rwxrwxrwx" permissions on that file.


===Environment & Shell Variables===
===Environment & Shell Variables===
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.
Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.


In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could write <tt>echo $HELLO</tt>).
In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a <code>$</code> sign (e.g., to output the value in the HELLO environment variable, one could type the command <code>echo $HELLO</code>).


If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:
If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:
Line 83: Line 44:
  export X="Important Data"
  export X="Important Data"


This is the idiom for setting environment variables normally. You will be able to better understand the differnces between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).  
This is the idiom for setting environment variables normally. You will be able to better understand the differences between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the <code>$</code> sign).  


One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:  
One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:  
Line 91: Line 52:
as you now are giving export three arguments, not one.
as you now are giving export three arguments, not one.


To delete an environment variable, you can <tt>unset X</tt>.
To delete an environment variable, you can use the command <code>unset</code>. For example:
 
unset X


===Downloading files using commands===
===Downloading files using commands===
When working with the command line and you need to download a small number of files from a website (i.e., using the Web/HTTP protocol), you can choose one of the two below (use <tt>man</tt> to find out more if interested):
When working with the command line and you need to download a small number of files from a website, you can choose one of the two below:
 
curl -o hello.c [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut1/hello.c https://people.scs.carleton.ca/~abdou/comp3000/f23/tut1/hello.c]
wget [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut1/hello.c https://people.scs.carleton.ca/~abdou/comp3000/f23/tut1/hello.c]


curl -o hello.c [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut1/hello.c]
Use <code>man curl</code> and <code>man wget</code> to find out more about these commands, if you are interested.
wget [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut1/hello.c https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut1/hello.c]


In later tutorials, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page or a PDF.
In later tutorials, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page.


===Compiling & Running Programs===
===Compiling & Running Programs===
Line 112: Line 77:
  ./hello
  ./hello


Remember you can always change directories using the <tt>cd</tt> command.
Remember you can always change directories using the <code>cd</code> command.


==Tasks/Questions==
==Tasks/Questions==
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out mutiple ways for each (if you can).
# When you have first logged in to a shell, <u>how</u> (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out multiple ways for each (if you can).
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
#* RAM (remaining/total), disk space (remaining/total), and CPU.
#* RAM (remaining/total), disk space (remaining/total), and CPU.
#* The name (binary path) of the current shell.
#* The name (binary path) of the current shell.
# Using the <tt>man</tt> command, find out what the following commands do: <tt>which</tt>, <tt>pwd</tt>, <tt>who</tt>, <tt>whoami</tt>, <tt>whereis</tt>, and <tt>whatis</tt>.  
# Using the <code>man</code> command, find out what the following commands do: <code>which</code>, <code>pwd</code>, <code>who</code>, <code>whoami</code>, <code>whereis</code>, and <code>whatis</code>.  
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <tt>cd</tt>) is internal or external?  
# Linux commands can be classified as internal (built into the shell) and external (separate program binaries). <u>How can you tell</u> if a specific command (e.g., <code>cd</code>) is internal or external?  
#* For the program you just compiled above, what if you used <tt>hello</tt> to run it instead of <tt>./hello</tt> (and why)?
#* For the program you just compiled above, what if you used <code>hello</code> to run it instead of <code>./hello</code> (and why)?
#* Figure out where at least three external commands reside on the system.  
#* Figure out where at least three external commands reside on the system.  
# Making your own commands: the <tt>PATH</tt> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Making your own commands: the <code>PATH</code> environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
# Look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# Look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
# For those same program binaries, figure out what the permission bits mean by reading the man page of <tt>chmod</tt> (this is the command you could use to change those permission bits).  
# For those same program binaries, figure out what the permission bits mean by reading the man page of <code>chmod</code> (this is the command you could use to change those permission bits).  
# What are the owner, group, and permissions of <tt>/etc/passwd</tt> and <tt>/etc/shadow</tt>? What are these files used for (and <u>how did you find this out</u>)?
# What are the owner, group, and permissions of <code>/etc/passwd</code> and <code>/etc/shadow</code>? What are these files used for (and <u>how did you find this out</u>)?
# What does it mean to have execute permission on a directory?
# What does it mean to have execute permission on a directory?
# The <tt>ls</tt> command can be used to get a listing of the files in a directory. What options are passed to <tt>ls</tt> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# The <code>ls</code> command can be used to get a listing of the files in a directory. What options are passed to <code>ls</code> to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
# Compile and run [https://people.scs.carleton.ca/~lianyingzhao/comp3000/w23/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <tt>bash</tt></u>? List at least 3 differences.
# Compile and run [https://people.scs.carleton.ca/~abdou/comp3000/f23/tut1/csimpleshell.c <tt>csimpleshell.c</tt>]. <u>How does its functionality compare to that of <code>bash</code></u>? List at least 3 differences.

Latest revision as of 03:55, 22 September 2023

This first tutorial will prepare you for subsequent tutorials. You will familiarize yourself with the OS environment and, in particular, learn the basics of command-line interaction in Linux (Ubuntu).

Getting Started

The concepts covered below are mostly part of a standard tutorial on UNIX/Linux. However, remember that you are trying to build a conceptual model of how things work. Thus, don't memorize commands; instead, try to understand how things fit together, learn how to find out the answer with that model, and ask questions when things don't work as expected!

Making use of Man pages

The man (short for manual) command is a way to access the built-in software documentation system, whose entries come with the system and software packages. For example, if you look at files in an installed software package (e.g., dpkg -L ftp to list files in the ftp package) you can see there are a few files such as /usr/share/man/man1, 5, … They are the man page entries (sections).

The topics go beyond just software/command manuals. They also include conventions and abstract concepts (e.g., man syscalls and man man-pages) that can serve as an info center. There are multiple sections for different purposes, which can be specified as the first argument, e.g., 1 for general commands, 2 for system calls, 3 for library functions, etc. For instance, tee is both a command (man 1 tee) and a system call (man 2 tee). A default will be used if the section is not specified.

For any commands mentioned in the tutorials, you can use man to find the usage. The advantage is that the man page documentation provides an exact match to your current environment, compared to searching the Internet.

Compared to the summary and brief information provided by the man command, you can also use the info command for a more detailed explanation for certain topics (just give it a try).

Background

The Shell

The shell, also called "command line" or more commonly "terminal", provides a text interface for running programs. While not as visually pleasing as a graphical interface, the shell provides a clearer representation of the functionality provided by the OS.

It works as an interpreter to the commands the user enters, i.e., taking commands and sending them to the OS for execution. Typical shells include bash (the default we are using in Ubuntu), ksh, zsh, csh, etc. You will learn how to change the current shell in Tutorial 4. Most modern shells also allow for the creation of a shell script, which is a sequence of commands (programmed in a scripting language). You will see shell scripts as we navigate this course.

File Permissions

Each file on the system (including directories and other special files) has an owner (user), and belongs to a group.

File permissions in Linux are: read, write, and execute, denoted rwx. Each file has three sets of rwx permissions. The first set denotes the type of access allowed to the owner of the file (rwxrwxrwx). The second set of permissions denotes the allowed file operations for a group of users (rwxrwxrwx). The third set of permissions denotes the allowed file operations for everyone else (rwxrwxrwx). A file is always owned by someone and is always associated with a group.

Whether or not you will be allowed to access (read from, write to, or execute) a file in Linux/Unix is determined by who (i.e., which user) you are logged in as, and the "rwxrwxrwx" permissions on that file.

Environment & Shell Variables

Environment variables on both Linux and Windows are variable-value pairs that define important context-related information (such as the name of the current user, the current language, the timezone) for applications. The key advantage of environment variables is that they are available right when a program starts - they are given to it by the operating system.

In Linux, these environment variables can be printed on the command line in most shells by referring to the variable name prefixed with a $ sign (e.g., to output the value in the HELLO environment variable, one could type the command echo $HELLO).

If a variable is private to the shell, it is called a shell variable. In bash, by default all variables are first shell variables. A shell variable becomes an environment variable, if you "export" it. For example:

X="Important Data"

just defines X as a shell variable. However, if you then type:

export X

X will be turned into an environment variable (no longer private), and so every subsequent program run by the shell will also get X. You can combine both in one line:

export X="Important Data"

This is the idiom for setting environment variables normally. You will be able to better understand the differences between shell variables and environment variables in later tutorials and assignments. Typically, you can access shell and environment variables using the same mechanism (the $ sign).

One thing to remember with the above is that spaces are used to separate arguments in bash and most other UNIX shells. Thus, it is an error to type:

export X = "Important Data"

as you now are giving export three arguments, not one.

To delete an environment variable, you can use the command unset. For example:

unset X

Downloading files using commands

When working with the command line and you need to download a small number of files from a website, you can choose one of the two below:

curl -o hello.c https://people.scs.carleton.ca/~abdou/comp3000/f23/tut1/hello.c
wget https://people.scs.carleton.ca/~abdou/comp3000/f23/tut1/hello.c

Use man curl and man wget to find out more about these commands, if you are interested.

In later tutorials, this will be useful as you need source files to do your work, for which the URL will be just a hyperlink on this wiki page.

Compiling & Running Programs

To compile a program, use gcc:

gcc -O2 hello.c -o hello

This compiles it with level 2 optimization and without debugging symbols, and the output program is hello.

To run, you have to specify where it is (the single dot . represents your current directory):

./hello

Remember you can always change directories using the cd command.

Tasks/Questions

  1. When you have first logged in to a shell, how (i.e., using what commands?) can you find out information about the environment? Consider the following as examples. Try to find out multiple ways for each (if you can).
    • The version of your Linux distribution (in our case, Ubuntu) and the version of your Linux kernel.
    • RAM (remaining/total), disk space (remaining/total), and CPU.
    • The name (binary path) of the current shell.
  2. Using the man command, find out what the following commands do: which, pwd, who, whoami, whereis, and whatis.
  3. Linux commands can be classified as internal (built into the shell) and external (separate program binaries). How can you tell if a specific command (e.g., cd) is internal or external?
    • For the program you just compiled above, what if you used hello to run it instead of ./hello (and why)?
    • Figure out where at least three external commands reside on the system.
  4. Making your own commands: the PATH environment variable lists the directories the shell uses to search for external commands. Where can you find documentation on it? How can you add the current directory (whichever directory you are currently in) to PATH? Then, how to make that change permanent? Try to identify multiple ways. You can now revisit the question about "hello" above.
  5. Look at the permissions of the program binaries of the external commands you have just found above. Who owns them? What group are they in?
  6. For those same program binaries, figure out what the permission bits mean by reading the man page of chmod (this is the command you could use to change those permission bits).
  7. What are the owner, group, and permissions of /etc/passwd and /etc/shadow? What are these files used for (and how did you find this out)?
  8. What does it mean to have execute permission on a directory?
  9. The ls command can be used to get a listing of the files in a directory. What options are passed to ls to see: the permission bits above; all the files within a directory (including hidden files)? How to make a file hidden?
  10. Compile and run csimpleshell.c. How does its functionality compare to that of bash? List at least 3 differences.