Operating Systems (Fall 2019): Difference between revisions

From Soma-notes
No edit summary
 
(14 intermediate revisions by 2 users not shown)
Line 104: Line 104:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2019F Lecture 9|Lecture 9]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/file-devices.pdf I/O Devices], [http://pages.cs.wisc.edu/~remzi/OSTEP/file-disks.pdf Hard disk drives], [http://pages.cs.wisc.edu/~remzi/OSTEP/file-implementation.pdf Filesystem implementation], [http://pages.cs.wisc.edu/~remzi/OSTEP/file-journaling.pdf FSCK and Journaling]
       <p>[[Operating Systems 2019F Lecture 9|Lecture 9]]
       </p>
       </p>
       </td>
       </td>
Line 144: Line 144:
       </td>
       </td>
       <td>
       <td>
       <p>Midterm Exam
       <p>Midterm Exam [https://homeostasis.scs.carleton.ca/~soma/os-2019f/solutions/comp3000-midterm-2019f-sol.pdf (solutions)]
       </p>
       </p>
       </td>
       </td>
Line 180: Line 180:
     <tr>
     <tr>
       <td>
       <td>
       <p>Nov. 6
       <p><del>Nov. 6</del> Nov. 4
       </p>
       </p>
       </td>
       </td>
Line 270: Line 270:
     <tr>
     <tr>
       <td>
       <td>
       <p>TBD
       <p>Dec. 13, 1 PM
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>Final Exam
       <p>[[Operating Systems 2019F Final Exam Review|Final Exam Review in MC 5050]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Dec. 17, 7 PM
      </p>
      </td>
      <td>
      <p>Final Exam in AH, rows 12-23 [https://homeostasis.scs.carleton.ca/~soma/os-2019f/solutions/comp3000-final-2019f-sol.pdf (solutions)]
       </p>
       </p>
       </td>
       </td>
Line 282: Line 292:
==Tutorials==
==Tutorials==


You may complete tutorials either by
'''Note this has changed for Tutorials 3 and 4.'''
* attending your assigned tutorial and working there, or
 
* doing the work on your own and demonstrating to your assigned TA that you've completed the work
You may complete tutorials in lab or on your own.  Submit your answers on culearn, they are due before the next week but are accepted with no grade penalty up until the related assignment is due. (For example, Tutorials 1 and 2 must be completed before Assignment 1 is submitted.)
You must finish the tutorials before the assignment based on them is due. (For example, Tutorials 1 and 2 must be completed before Assignment 1 is submitted.)  Exceptions will be made only if there are extenuating circumstances.
 
Your answers will be graded based on participation, not understanding.  Basically, if you make a reasonable attempt at the whole tutorial you'll get 4/4.  Partial attempts will potentially be given partial credit; however, you may improve your tutorial mark as long as it is still being accepted. Exceptions will be made only if there are extenuating circumstances.
 
While you may collaborate with others and use outside resources, you should indicate the names of who you worked with and cite the sources you used.


You can find your TA by looking at the feedback portion of the "TA" grade item in cuLearn.
The words you submit should be your own.  Please don't copy-paste answers from others, because if you do, you are subject to being reported to the Dean for plagiarism.
 
Note that your assigned TA will be marking your work.  You can find your TA by looking at the feedback portion of the "TA" grade item in cuLearn.


<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
Line 324: Line 339:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2019F: Tutorial 3|3000test]]
       <p>[[Operating Systems 2019F: Tutorial 3|Memory & System Calls]]
       </p>
       </p>
       </td>
       </td>
Line 334: Line 349:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2019F: Tutorial 4|Filesystems]]
       <p>[[Operating Systems 2019F: Tutorial 4|3000test]]
       </p>
       </p>
       </td>
       </td>
Line 344: Line 359:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2019F: Tutorial 5|Kernel modules]]
       <p>[[Operating Systems 2019F: Tutorial 5|Filesystems]]
       </p>
       </p>
       </td>
       </td>
Line 354: Line 369:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2019F: Tutorial 6|Kernel memory management]]
       <p>[[Operating Systems 2019F: Tutorial 6|eBPF]]
       </p>
       </p>
       </td>
       </td>
Line 364: Line 379:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2019F: Tutorial 7|EBPF: bcc tools]]
       <p>[[Operating Systems 2019F: Tutorial 7|Linux kernel modules]]
       </p>
       </p>
       </td>
       </td>
Line 375: Line 390:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2019F: Tutorial 8|3000pc]]
       <p>[[Operating Systems 2019F: Tutorial 8|Kernel memory management]]
       </p>
       </p>
       </td>
       </td>
Line 385: Line 400:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2019F: Tutorial 9|TOCTTOU]]
       <p>[[Operating Systems 2019F: Tutorial 9|3000rootkit]]
       </p>
       </p>
       </td>
       </td>
Line 426: Line 441:
     <tr>
     <tr>
       <td>
       <td>
       <p>Nov. 15
       <p>Nov. 24
       </p>
       </p>
       </td>
       </td>
Line 436: Line 451:
     <tr>
     <tr>
       <td>
       <td>
       <p>Dec. 6
       <p>Dec. 4
       </p>
       </p>
       </td>
       </td>
Line 451: Line 466:


You may use other Linux distributions to complete the assigned work; there will be differences, however, in some aspects (such as installing software), particularly if you use a distribution not based on Ubuntu or Debian.
You may use other Linux distributions to complete the assigned work; there will be differences, however, in some aspects (such as installing software), particularly if you use a distribution not based on Ubuntu or Debian.
===Openstack===
You should create a VM on the new SCS openstack cluster at [https://openstack-stein.scs.carleton.ca openstack-stein.scs.carleton.ca] and do your work there.  Documentation on the cluster is [https://carleton.ca/scs/tech-support/scs-open-stack/ here].  While you don't need a persistent VM for this lab, it will be important for future tuturials - and it is nice to have your work stick around when you leave the lab.
Make sure you have added the ssh-ping security group to your network interface and that you have associated a floating IP address with your instance.  The 192.168.X.X IP addresses are private (and cannot be accessed outside of the openstack cluster), the 134.117.X.X floating IP addresses can be accessed from the Carleton network and will allow you to access the wider Internet.
Note that you must be on the Carleton network to use openstack.  When you are off campus, connect using the  [https://carleton.ca/its/help-centre/remote-access/ Carleton VPN].
Create a VM using the latest COMP 3000 snapshot image.  Please create a machine with two VCPUs.  The user is student, default password is student.  Please change your password after you first connect to your machine (using the passwd command).
'''Make sure you connect via ssh.'''  DO NOT use the openstack web console, it is glitchy!  Windows 10 and MacOS have ssh clients available from their command lines, just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM.  PuTTY also works, and you can use x2go.
The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines.  So if your SCS username is janedoe, you can type
  scs-backup janedoe
and it will create a copy of everything in the student account in a directory called "COMP3000VM-backup" in your home directory.  You can ssh/sftp to access.scs.carleton.ca in order to access this copy of your VM's files.  You should do backups at the end of every session and before you do anything dangerous.
The scs-backup bash function is listed below.  Feel free to adapt to your own needs; however, realize that rsync is a very powerful command that can delete arbitrary files at the specified destination (and in fact that is what the listed command does to the backup directory).  If you are changing any arguments, be sure to test with the -n option so you can see what will happen!
'''Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed).'''


==Online Resources==
==Online Resources==
Line 460: Line 497:
==Tutorial Grading==
==Tutorial Grading==


Each week you will get a progress grade from 0-4, given to you by a TA.  If you are being diligent, you should be able to get 4's every week.  The easiest way to get your grade is to come to tutorial and meet with your TA; alternately, you can meet a TA in their office hours or, at their discretion, discuss things with them online.  Tutorials from before the Midterm must be completed by the Midterm, and Tutorials after the Midterm must be completed by the last day of classes (if you wish to receive credit).
Each week you will get a progress grade from 0-4, given to you by a TA.  If you are being diligent, you should be able to get 4's every week.  The easiest way to get your grade is to come to tutorial and meet with your TA; alternately, you can meet a TA in their office hours or, at their discretion, discuss things with them online.  Tutorials must be completed before the assignment on which they are based is due (if you wish to receive credit).


Tutorial grades will be given out as follows:
Tutorial grades will be given out as follows:
Line 469: Line 506:
It should be easy to get a 3 if you show up and attempt the material.  To get a 4, however, you need to be demonstrating that you are conducting experiments and actively engaging with the material in a way that is likely to lead to success.
It should be easy to get a 3 if you show up and attempt the material.  To get a 4, however, you need to be demonstrating that you are conducting experiments and actively engaging with the material in a way that is likely to lead to success.


It is possible to upgrade your grade on a tutorial at any time before the midterm (for the first four tutorials) or before the last day of classes (for the last five tutorials).
It is possible to upgrade your grade on a tutorial at any time before the assignment on which they are based is due.


==How to Succeed and How to Fail in COMP 3000==
==How to Succeed and How to Fail in COMP 3000==

Latest revision as of 23:18, 19 March 2020

Course Outline

Here is the course outline for COMP 3000: Operating Systems.

Lectures & Tests

Note that the topics below are primarily chapters from the class textbook, Operating Systems: Three Easy Pieces. Note that while introductory and summary dialogues are not linked below, they are worth reading for an informal take on the material.

Assigned readings are subject to change, please check this page each week.

Date

Topic

Sept. 4

Lecture 1: Introduction

Sept. 6

Lecture 2: Process API

Sept. 11

Lecture 3: Processes

Sept. 13

Lecture 4: Files

Sept. 18

Lecture 5: Address Spaces (Dialogue)

Sept. 20

Lecture 6

Sept. 25

Lecture 7

Sept. 27

Lecture 8

Oct. 2

Lecture 9

Oct. 4

Lecture 10

Oct. 9

Lecture 11

Oct. 11

Lecture 12: Midterm Review

Oct. 16

Midterm Exam (solutions)

Oct. 18

Lecture 13

Oct. 30

Lecture 14

Nov. 1

Lecture 15

Nov. 6 Nov. 4

Lecture 16

Nov. 8

Lecture 17

Nov. 13

Lecture 18

Nov. 15

Lecture 19

Nov. 20

Lecture 20

Nov. 22

Lecture 21

Nov. 27

Lecture 22

Nov. 29

Lecture 23

Dec. 4

Lecture 24

Dec. 13, 1 PM

Final Exam Review in MC 5050

Dec. 17, 7 PM

Final Exam in AH, rows 12-23 (solutions)

Tutorials

Note this has changed for Tutorials 3 and 4.

You may complete tutorials in lab or on your own. Submit your answers on culearn, they are due before the next week but are accepted with no grade penalty up until the related assignment is due. (For example, Tutorials 1 and 2 must be completed before Assignment 1 is submitted.)

Your answers will be graded based on participation, not understanding. Basically, if you make a reasonable attempt at the whole tutorial you'll get 4/4. Partial attempts will potentially be given partial credit; however, you may improve your tutorial mark as long as it is still being accepted. Exceptions will be made only if there are extenuating circumstances.

While you may collaborate with others and use outside resources, you should indicate the names of who you worked with and cite the sources you used.

The words you submit should be your own. Please don't copy-paste answers from others, because if you do, you are subject to being reported to the Dean for plagiarism.

Note that your assigned TA will be marking your work. You can find your TA by looking at the feedback portion of the "TA" grade item in cuLearn.

Date

Tutorials

Sept. 9-13

Processes

Sept. 16-20

3000shell

Sept. 23-27

Memory & System Calls

Sept. 30-Oct. 4

3000test

Oct. 28-Nov. 1

Filesystems

Nov. 4-8

eBPF

Nov. 11-15

Linux kernel modules

Nov. 18-22

Kernel memory management

Nov. 25-29

3000rootkit

Assignments

Assignments are due before class on the due dates (2:30 PM).

Due Date

Assignments

Sept. 27

Assignment 1

Oct. 11

Assignment 2

Nov. 24

Assignment 3

Dec. 4

Assignment 4

Course Software

In this course we will primarily working with Ubuntu, a widely-used family of Linux distributions. We will be using Ubuntu Server on the SCS's Openstack installation (accessible only from the Carleton network).

You may use other Linux distributions to complete the assigned work; there will be differences, however, in some aspects (such as installing software), particularly if you use a distribution not based on Ubuntu or Debian.

Openstack

You should create a VM on the new SCS openstack cluster at openstack-stein.scs.carleton.ca and do your work there. Documentation on the cluster is here. While you don't need a persistent VM for this lab, it will be important for future tuturials - and it is nice to have your work stick around when you leave the lab.

Make sure you have added the ssh-ping security group to your network interface and that you have associated a floating IP address with your instance. The 192.168.X.X IP addresses are private (and cannot be accessed outside of the openstack cluster), the 134.117.X.X floating IP addresses can be accessed from the Carleton network and will allow you to access the wider Internet.

Note that you must be on the Carleton network to use openstack. When you are off campus, connect using the Carleton VPN.

Create a VM using the latest COMP 3000 snapshot image. Please create a machine with two VCPUs. The user is student, default password is student. Please change your password after you first connect to your machine (using the passwd command).

Make sure you connect via ssh. DO NOT use the openstack web console, it is glitchy! Windows 10 and MacOS have ssh clients available from their command lines, just type "ssh student@<IP address>" where the IP address is the floating IP address you assigned to your VM. PuTTY also works, and you can use x2go.

The image provides an "scs-backup" command that will backup the student user's directory to the SCS linux machines. So if your SCS username is janedoe, you can type

 scs-backup janedoe

and it will create a copy of everything in the student account in a directory called "COMP3000VM-backup" in your home directory. You can ssh/sftp to access.scs.carleton.ca in order to access this copy of your VM's files. You should do backups at the end of every session and before you do anything dangerous.

The scs-backup bash function is listed below. Feel free to adapt to your own needs; however, realize that rsync is a very powerful command that can delete arbitrary files at the specified destination (and in fact that is what the listed command does to the backup directory). If you are changing any arguments, be sure to test with the -n option so you can see what will happen!

Note that you cannot take snapshots of your VM, so please don't try (it will keep trying and never succeed).

Online Resources

Other than the textbook, there are many, many other good online sources of information about operating systems. Here are a few that may be of interest:

Tutorial Grading

Each week you will get a progress grade from 0-4, given to you by a TA. If you are being diligent, you should be able to get 4's every week. The easiest way to get your grade is to come to tutorial and meet with your TA; alternately, you can meet a TA in their office hours or, at their discretion, discuss things with them online. Tutorials must be completed before the assignment on which they are based is due (if you wish to receive credit).

Tutorial grades will be given out as follows:

  • 2: student attended the tutorial (50%)
  • 3: student worked on most of the tutorial material (75%)
  • 4: student is learning the material in a way that should lead to success on assignment and the exams (100%)

It should be easy to get a 3 if you show up and attempt the material. To get a 4, however, you need to be demonstrating that you are conducting experiments and actively engaging with the material in a way that is likely to lead to success.

It is possible to upgrade your grade on a tutorial at any time before the assignment on which they are based is due.

How to Succeed and How to Fail in COMP 3000

While the use of outside resources is acceptable, even encouraged, use them to achieve understanding rather than answers.

It is very possible to get full tutorial marks by doing little more than showing up, and assignments can be quickly finished (and you can even get most of the marks) by working with partners, looking at old assignments, and doing lots of online searches. If this is all you do, however, you will fail the tests and you won't get a very good grade in the course.

Successful students use the tutorials and assignments as learning opportunities. They look at the code and ask "what does that do?" - not just in a general sense, but by looking at every line and asking why it is there. They modify the code in order to better understand it, and when they are confused they ask questions until they get the answers they need. This sort of learning takes time, generally much more than the time allocated for lectures and tutorials. A tutorial is only "finished" when you understand essentially all the material in it - not when 80 minutes are up.

If you find you are confused about the concepts, make sure to read and review the textbook and the Linux man pages. They provide much deeper answers than most online resources. The tutorials are designed to help you ask the right questions - the textbook and manual pages (and the lectures) help provide the answers.

Take the tutorials seriously. Understand the answers for all the questions on the assignments. Use the textbook and manual pages. Ask questions.

Good luck!