Operating Systems (Fall 2015): Difference between revisions

From Soma-notes
 
(45 intermediate revisions by the same user not shown)
Line 2: Line 2:


[[Operating Systems (Fall 2015) Course Outline|Here]] is the course outline for COMP 3000: Operating Systems.
[[Operating Systems (Fall 2015) Course Outline|Here]] is the course outline for COMP 3000: Operating Systems.
==Course Project==
===What's in your final report?===
If you managed to implement something substantial, your report should be organized as follows:
* Title, Authors, Date, and "Class project for COMP 3000: Operating Systems" (or something like that).
* The idea (intro)
* Design/high level implementation
* implementation
** What problems did you encounter?
** How did you overcome them?
* Evaluation
** Does it work?
** How do you know it works?  Explain how you tested your work.
** In particular, make sure you test your code with some significant load (e.g., building a kernel) so that you have some evidence that your modifications didn't make the kernel unstable.
* Conclusion
** contributions
** limitations
** future work
* Statement of Joint Work
** If this was a group project, please explain here what role each co-author had in generating the ideas for the project, doing the actual work, and writing the report.
* References
** Be sure to cite any resources (including web pages) that significantly helped you in your work.  Please do not use footnotes for citations; they should all be endnotes.
* Appendix: code (generally in patch form)
** Please also include code inline if it makes sense as part of the implementation and evaluation sections.
If, in the end, you more learned about how various parts of the kernel worked but you didn't managed to accomplish most of your goals, you should instead organize your report around what you learned about the kernel and how you learned it.  Thus, the "design" and "implementation" portions are more about the small experiments you designed and implemented to test how the kernel works, with the evaluation part saying what results you got and what that taught you.  In this case you'll in effect have multiple design, implementation, and evaluation sections, so instead organize your paper around each experiment, i.e., make a section for each experiment which describes what you did and what you learned.
It will be much easier to get a good grade with the first strategy (implementing and documenting a single project) rather than the second.  However, if through your experiments you have learned a substantial amount about the kernel, writing up what you have learned and how you learned it can potentially result in a high grade.
===Grading Criteria===
Marks for projects will be given in the following categories:
* Technical accomplishment (40%)
* Creativity (20%)
* Writing quality (40%)
** Grammar, spelling
** Organization
** Clarity/Understandability
A technically competent, well-written report will receive approximately a B grade.  Exceptional projects will receive A's.  Lower graded reports will have some clear faults in terms of writing, technical accomplishment, or creativity.
Note that there is no fixed length requirement.  However, projects that are less that 5 pages are unlikely to get a high grade as that means that less than a page was spent on each section, on average.  The implementation and evaluation sections should be the longest ones in your report.  An overly short evaluation section may lead to suspicion that you code does not work.
'''In order to get a grade for the final project, you need to pass the oral exam.'''  You need to pass the oral exam because otherwise we cannot be sure that you have done the work of the project.  If you manage to complete a project successfully, either alone or in a group, the oral exam is very easy.
If you want to schedule an oral exam and you cannot find a slot, please email Prof. Somayaji and let him know your availability.
===Generating Patches===
Given the size of the Linux kernel, even at the level of individual files, you do not want to include most of that code in your report.  Instead, in general you should generate a patch using the diff command:
  diff -u <old file/directory> <new file/directory>
diff can compare files or entire directories.  Feel free to edit the output of diff in order to make your code changes easy to understand.  Alternately, you can just include code snippets by hand and mark your changes in a way that should be clear to any reader.  Whatever you do, it should be crystal clear the difference between '''code that you wrote, code that you found on the Internet/by your peers, and code that was already in the Linux kernel'''.
(We call the output of the diff command patch files because they can be used to [https://en.wikipedia.org/wiki/Patch_%28Unix%29 automatically make changes to source code].)
===Formatting your report===
You are welcome to use Microsoft Word or Libreoffice to write your report.  You may find it easier to generate an aesthetically pleasing report, however, if you use LaTeX or a front end for LaTeX such as [http://pandoc.org/ pandoc].  While I will not be grading based on appearance, nicely formatted documents are always appreciated.
In general I prefer larger text, e.g., 12 point text with one inch margins; however, if your report looks "good" I won't care.  If you use LaTeX, though, please don't use the default Computer Modern fonts.
Please submit a PDF document or a very nicely formatted plain text file.  Please, please do not submit a paper in MS Word, LibreOffice, or other format.


==Lectures and Exams==
==Lectures and Exams==


'''This is currently all wrong!'''
Note that the topics below are primarily chapters from the class textbook, [http://pages.cs.wisc.edu/~remzi/OSTEP/ Operating Systems: Three Easy Pieces].  Note that while introductory and summary dialogues are not linked below, they are worth reading for an informal take on the material.


Note that the topics below are primarily chapters from the class textbook, [http://pages.cs.wisc.edu/~remzi/OSTEP/ Operating Systems: Three Easy Pieces].  Note that while introductory and summary dialogues are not linked below, they are worth reading for an informal take on the material.
Assigned readings are subject to change, please check this page each week.


<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
Line 35: Line 102:
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 2|Lecture 2]]: (Video only) [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-intro.pdf Processes], [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-api.pdf Process API]
       <p>[[Operating Systems 2015F Lecture 2|Lecture 2]]: (Video only) [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-intro.pdf Processes], [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-api.pdf Process API]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Sept. 11
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F Lecture 3|Lecture 3]]: (Video only) [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-mechanisms.pdf Limited Direct Execution]
       </p>
       </p>
       </td>
       </td>
Line 54: Line 111:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 4|Lecture 4]]:
       <p>[[Operating Systems 2015F Lecture 3|Lecture 3]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-mechanisms.pdf Limited Direct Execution], [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched.pdf CPU Scheduling], [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-mlfq.pdf MLFQ]  </p>
[http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched.pdf CPU Scheduling],
[http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-mlfq.pdf MLFQ]  </p>
       </td>
       </td>
     </tr>
     </tr>
Line 65: Line 120:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 5|Lecture 5]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-lottery.pdf Lottery Scheduling], [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-multi.pdf Multi-CPU scheduling]
       <p>[[Operating Systems 2015F Lecture 4|Lecture 4]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-intro.pdf Address Spaces], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-api.pdf Memory API]
       </p>
       </p>
       </td>
       </td>
Line 75: Line 130:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 6|Lecture 6]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-intro.pdf Address Spaces], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-api.pdf Memory API]
       <p>[[Operating Systems 2015F Lecture 5|Lecture 5]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/file-disks.pdf Hard Disk Drives], [http://pages.cs.wisc.edu/~remzi/OSTEP/file-intro.pdf File and Directories]
       </p>
       </p>
       </td>
       </td>
Line 85: Line 140:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 7|Lecture 7]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-mechanism.pdf Address Translation], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-segmentation.pdf Segmentation]
       <p>[[Operating Systems 2015F Lecture 6|Lecture 6]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/file-devices.pdf I/O Devices], [http://pages.cs.wisc.edu/~remzi/OSTEP/file-journaling.pdf FSCK and Journaling]
       </p>
       </p>
       </td>
       </td>
Line 95: Line 150:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 8|Lecture 8]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-freespace.pdf Free space management], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-paging.pdf Paging]
       <p>[[Operating Systems 2015F Lecture 7|Lecture 7]]</p>
      </p>
       </td>
       </td>
     </tr>
     </tr>
Line 105: Line 159:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 9|Lecture 9]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-intro.pdf Concurrency and Threads], [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-locks.pdf Locks]
       <p>[[Operating Systems 2015F Lecture 8|Lecture 8]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/file-implementation.pdf File System Implementation]
       </p>
       </p>
       </td>
       </td>
Line 115: Line 169:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 10|Lecture 10]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-locks-usage.pdf Concurrent Data Structures], [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-api.pdf Thread API]
       <p>[[Operating Systems 2015F Lecture 9|Lecture 9]] (first half), Test 1 (in class)</p>
      </p>
       </td>
       </td>
     </tr>
     </tr>
Line 125: Line 178:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 11|Lecture 11]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf Condition Variables], [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-sema.pdf Semaphores], [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-bugs.pdf Concurrency Problems]
       <p>[[Operating Systems 2015F Lecture 10|Lecture 10]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-intro.pdf Concurrency and Threads], [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-locks.pdf Locks],
       </p>
       </p>
       </td>
       </td>
Line 135: Line 188:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 12|Lecture 12]]: Test 1 Review
       <p>[[Operating Systems 2015F Lecture 11|Lecture 11]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-mechanism.pdf Address Translation], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-tlbs.pdf TLBs]
       </p>
       </p>
       </td>
       </td>
Line 145: Line 198:
       </td>
       </td>
       <td>
       <td>
       <p>Test 1 (in class)</p>
       <p>[[Operating Systems 2015F Lecture 12|Lecture 12]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-segmentation.pdf Segmentation], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-freespace.pdf Free space management]
      </p>
       </td>
       </td>
     </tr>
     </tr>
Line 154: Line 208:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 13|Lecture 13]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-tlbs.pdf TLBs], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-smalltables.pdf Smaller Tables]</p>
       <p>[[Operating Systems 2015F Lecture 13|Lecture 13]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-paging.pdf Paging]</p>
       </td>
       </td>
     </tr>
     </tr>
Line 163: Line 217:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 14|Lecture 14]]</p>
       <p>[[Operating Systems 2015F Lecture 14|Lecture 14]]: pH</p>
       </td>
       </td>
     </tr>
     </tr>
Line 172: Line 226:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 15|Lecture 15]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/file-disks.pdf Hard Disk Drives],
       <p>[[Operating Systems 2015F Lecture 15|Lecture 15]]
[http://pages.cs.wisc.edu/~remzi/OSTEP/file-intro.pdf File and Directories]
       </p>
       </p>
       </td>
       </td>
Line 183: Line 236:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 16|Lecture 16]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/file-journaling.pdf FSCK and Journaling]
       <p>[[Operating Systems 2015F Lecture 16|Lecture 16]]: Kernel modification walkthrough
       </p>
       </p>
       </td>
       </td>
Line 193: Line 246:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 17|Lecture 17]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/file-devices.pdf I/O Devices]
       <p>[[Operating Systems 2015F Lecture 17|Lecture 17]]: Graphics
       </p>
       </p>
       </td>
       </td>
Line 203: Line 256:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 18|Lecture 18]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/file-implementation.pdf File System Implementation], [http://pages.cs.wisc.edu/~remzi/OSTEP/file-ffs.pdf FFS]
       <p>[[Operating Systems 2015F Lecture 18|Lecture 18]]: Device Drivers
       </p>
       </p>
       </td>
       </td>
Line 213: Line 266:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 19|Lecture 19]]: Test 2 Review
       <p>[[Operating Systems 2015F Lecture 19|Lecture 19]]: Cloud Computing
       </p>
       </p>
       </td>
       </td>
Line 223: Line 276:
       </td>
       </td>
       <td>
       <td>
       <p>Test 2 (in class)
       <p>[[Operating Systems 2015F Lecture 20|Lecture 20]] (first half), Test 2 (second half)
       </p>
       </p>
       </td>
       </td>
Line 233: Line 286:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 20|Lecture 20]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/dist-intro.pdf Distributed Systems], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys.pdf Swapping: Mechanisms], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys-policy.pdf Swapping: Policies]
       <p>[[Operating Systems 2015F Lecture 21|Lecture 21]]: Research
       </p>
       </p>
       </td>
       </td>
Line 243: Line 296:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 21|Lecture 21]]: [http://pages.cs.wisc.edu/~remzi/OSTEP/dist-nfs.pdf NFS], [http://pages.cs.wisc.edu/~remzi/OSTEP/dist-afs.pdf AFS] (optional readings)
       <p>[[Operating Systems 2015F Lecture 22|Lecture 22]]: Security and malware
       </p>
       </p>
       </td>
       </td>
Line 253: Line 306:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 22|Lecture 22]]: TBA
       <p>[[Operating Systems 2015F Lecture 23|Lecture 23]]: Other Operating Systems
       </p>
       </p>
       </td>
       </td>
Line 263: Line 316:
       </td>
       </td>
       <td>
       <td>
       <p>[[Operating Systems 2015F Lecture 23|Lecture 23]]: TBA
       <p>[[Operating Systems 2015F Lecture 24|Lecture 24]]: The Future
       </p>
       </p>
       </td>
       </td>
     </tr>
     </tr>
</table>
</table>
Other chapters: [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-lottery.pdf Lottery Scheduling], [http://pages.cs.wisc.edu/~remzi/OSTEP/cpu-sched-multi.pdf Multi-CPU scheduling], [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-locks-usage.pdf Concurrent Data Structures], [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-api.pdf Thread API], [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-cv.pdf Condition Variables], [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-sema.pdf Semaphores], [http://pages.cs.wisc.edu/~remzi/OSTEP/threads-bugs.pdf Concurrency Problems], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-smalltables.pdf Smaller Tables] , [http://pages.cs.wisc.edu/~remzi/OSTEP/file-ffs.pdf FFS], [http://pages.cs.wisc.edu/~remzi/OSTEP/dist-intro.pdf Distributed Systems], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys.pdf Swapping: Mechanisms], [http://pages.cs.wisc.edu/~remzi/OSTEP/vm-beyondphys-policy.pdf Swapping: Policies], [http://pages.cs.wisc.edu/~remzi/OSTEP/dist-nfs.pdf NFS], [http://pages.cs.wisc.edu/~remzi/OSTEP/dist-afs.pdf AFS]
==Tutorials==
Each week you will get a progress grade from 0-4, given to you by a TA.  If you are being diligent, you should be able to get 4's every week.  The easiest way to get your grade is to come to tutorial and meet with your TA; alternately, you can meet a TA in their office hours or, at their discretion, discuss things with them online.
<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
  <tr valign="top">
    <th>
    <p align="left">Date</p>
    </th>
    <th>
    <p align="left">Tutorials</p>
    </th>
  </tr>
    <tr>
      <td>
      <p>Sept. 18,22
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Tutorial 1|UNIX Introduction]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Sept. 25, 29
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Tutorial 2|Device files, filesystems, and kernel modules]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Oct. 2, 6
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Tutorial 3|kernel modules and using the source]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Oct. 9, 13
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Tutorial 4|Filesystem concurrency]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Oct. 16, 20
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Tutorial 5|Openstack]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Oct. 23, Nov. 3
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Tutorial 6|Catch up/Project help]]
      </p>
      </td>
    </tr>
        <tr>
      <td>
      <p>Nov. 6, 10
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Tutorial 7|Modifying system calls]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Nov. 13, 17
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Tutorial 8|kprobes, calling userspace]]
      </p>
      </td>
    </tr>
    <tr>
    <tr>
      <td>
      <p>Nov. 20, 24
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Tutorial 9|Kernel map, projects]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Nov. 27, Dec. 1
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2014F: Tutorial 10|Projects]]
      </p>
      </td>
    </tr>
</table>
==Assignments==
<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
  <tr valign="top">
    <th>
    <p align="left">Due Date</p>
    </th>
    <th>
    <p align="left">Assignments</p>
    </th>
  </tr>
    <tr>
      <td>
      <p>Sept. 23
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Assignment 1|Assignment 1]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Sept. 30
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Assignment 2|Assignment 2]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Oct. 6
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Assignment 3|Assignment 3]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Nov. 5
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Assignment 4|Assignment 4]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Nov. 18
      </p>
      </td>
      <td>
      <p>[[Operating Systems 2015F: Assignment 5|Assignment 5]]
      </p>
      </td>
    </tr>
</table>
==Course Software==
In this course we will primarily working with [http://www.ubuntu.com/ Ubuntu], a widely-used family of Linux distributions.  We will be using its [http://www.lubuntu.net/ Lubuntu] variant in Virtualbox earlier in the term; later on we will transition to using Ubuntu Server on the SCS's [http://openstack.scs.carleton.ca/ Openstack installation] (accessible only from the Carleton network).
You may use other Linux distributions to complete the assigned work and do your project; there will be differences, however, in some aspects (such as installing software), particularly if you use a distribution not based on Ubuntu or Debian.
===Using Virtualbox in the labs===
In the SCS labs you should be able to run the course VM by starting Virtualbox (listed in the Applications menu) and selecting the COMP 2406/3000 virtual machine image.  After the VM has fully booted you will be automatically logged into the student account; this account has admin privileges and its password is "tneduts!".
We highly recommend running your VM in full-screen mode (select from the menu, not by maximizing the window).  Do all of your work inside of the VM; it should be fast enough and you won't have any issues with sharing files or with firewalls/network connectivity.
You can save the work you do from the course VM (in the student account) to your SCS account and restore it to any other copy of the class VM (on your machines or in the labs) by running using the following commands:
  save3000 <SCS username>
  restore3000 <SCS username>
  compare3000 <SCS username>
If you use these commands, '''use them consistently'''.  That means run <tt>restore3000</tt> when you first log in, and run <tt>save3000</tt> just before logging out.  If you don't do this, you will '''erase''' the work that you had done previously when you save.
If you forgot to restore and you want to save, try running this:
  rsync -a -v --progress ~/ <SCS username>@access.scs.carleton.ca:COMP3000/
This is the same as the <tt>save3000</tt> command minus the options (--delete and --force) that deletes files in the destination that don't exist in the source.  As a check, you may want to add the <tt>-n</tt> option to do a dry run.
===Running the Virtualbox VM on your own machines===
If you want to run the VM appliance on your own system (running essentially any desktop operating system you want), just download the [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406%20&%203000,%20Fall%202015.ova virtual appliance file] and import.  The SHA1 hash of this file is:
    a8a70ec2e1b49699f4de29c872ecec7cee21888f [http://homeostasis.scs.carleton.ca/~soma/VMs/COMP%202406%20&%203000,%20Fall%202015.ova COMP 2406 & 3000, Fall 2015.ova]
On Windows you can compute this hash for your downloaded file using the command [http://support.microsoft.com/kb/889768 <tt>FCIV -sha1 COMP 2406-3000 Fall 2014.ova</tt>].  If the hash is different from above, your download has been corrupted.
If the application is not VirtualBox, you'll need to:
* Have the VM platform ignore any errors in the structure of the appliance found during the import process;
* Uninstall the VirtualBox guest additions by typing starting a terminal application and running
  sudo apt-get purge virtualbox-guest-x11 virtualbox-guest-utils
* Install your platform's own Linux guest additions, if available.

Latest revision as of 20:38, 2 December 2015

Course Outline

Here is the course outline for COMP 3000: Operating Systems.

Course Project

What's in your final report?

If you managed to implement something substantial, your report should be organized as follows:

  • Title, Authors, Date, and "Class project for COMP 3000: Operating Systems" (or something like that).
  • The idea (intro)
  • Design/high level implementation
  • implementation
    • What problems did you encounter?
    • How did you overcome them?
  • Evaluation
    • Does it work?
    • How do you know it works? Explain how you tested your work.
    • In particular, make sure you test your code with some significant load (e.g., building a kernel) so that you have some evidence that your modifications didn't make the kernel unstable.
  • Conclusion
    • contributions
    • limitations
    • future work
  • Statement of Joint Work
    • If this was a group project, please explain here what role each co-author had in generating the ideas for the project, doing the actual work, and writing the report.
  • References
    • Be sure to cite any resources (including web pages) that significantly helped you in your work. Please do not use footnotes for citations; they should all be endnotes.
  • Appendix: code (generally in patch form)
    • Please also include code inline if it makes sense as part of the implementation and evaluation sections.

If, in the end, you more learned about how various parts of the kernel worked but you didn't managed to accomplish most of your goals, you should instead organize your report around what you learned about the kernel and how you learned it. Thus, the "design" and "implementation" portions are more about the small experiments you designed and implemented to test how the kernel works, with the evaluation part saying what results you got and what that taught you. In this case you'll in effect have multiple design, implementation, and evaluation sections, so instead organize your paper around each experiment, i.e., make a section for each experiment which describes what you did and what you learned.

It will be much easier to get a good grade with the first strategy (implementing and documenting a single project) rather than the second. However, if through your experiments you have learned a substantial amount about the kernel, writing up what you have learned and how you learned it can potentially result in a high grade.

Grading Criteria

Marks for projects will be given in the following categories:

  • Technical accomplishment (40%)
  • Creativity (20%)
  • Writing quality (40%)
    • Grammar, spelling
    • Organization
    • Clarity/Understandability

A technically competent, well-written report will receive approximately a B grade. Exceptional projects will receive A's. Lower graded reports will have some clear faults in terms of writing, technical accomplishment, or creativity.

Note that there is no fixed length requirement. However, projects that are less that 5 pages are unlikely to get a high grade as that means that less than a page was spent on each section, on average. The implementation and evaluation sections should be the longest ones in your report. An overly short evaluation section may lead to suspicion that you code does not work.

In order to get a grade for the final project, you need to pass the oral exam. You need to pass the oral exam because otherwise we cannot be sure that you have done the work of the project. If you manage to complete a project successfully, either alone or in a group, the oral exam is very easy.

If you want to schedule an oral exam and you cannot find a slot, please email Prof. Somayaji and let him know your availability.

Generating Patches

Given the size of the Linux kernel, even at the level of individual files, you do not want to include most of that code in your report. Instead, in general you should generate a patch using the diff command:

 diff -u <old file/directory> <new file/directory>

diff can compare files or entire directories. Feel free to edit the output of diff in order to make your code changes easy to understand. Alternately, you can just include code snippets by hand and mark your changes in a way that should be clear to any reader. Whatever you do, it should be crystal clear the difference between code that you wrote, code that you found on the Internet/by your peers, and code that was already in the Linux kernel.

(We call the output of the diff command patch files because they can be used to automatically make changes to source code.)

Formatting your report

You are welcome to use Microsoft Word or Libreoffice to write your report. You may find it easier to generate an aesthetically pleasing report, however, if you use LaTeX or a front end for LaTeX such as pandoc. While I will not be grading based on appearance, nicely formatted documents are always appreciated.

In general I prefer larger text, e.g., 12 point text with one inch margins; however, if your report looks "good" I won't care. If you use LaTeX, though, please don't use the default Computer Modern fonts.

Please submit a PDF document or a very nicely formatted plain text file. Please, please do not submit a paper in MS Word, LibreOffice, or other format.

Lectures and Exams

Note that the topics below are primarily chapters from the class textbook, Operating Systems: Three Easy Pieces. Note that while introductory and summary dialogues are not linked below, they are worth reading for an informal take on the material.

Assigned readings are subject to change, please check this page each week.

Date

Topic

Sept. 2

Lecture 1: Introduction

Sept. 9

Lecture 2: (Video only) Processes, Process API

Sept. 16

Lecture 3: Limited Direct Execution, CPU Scheduling, MLFQ

Sept. 18

Lecture 4: Address Spaces, Memory API

Sept. 23

Lecture 5: Hard Disk Drives, File and Directories

Sept. 25

Lecture 6: I/O Devices, FSCK and Journaling

Sept. 30

Lecture 7

Oct. 2

Lecture 8: File System Implementation

Oct. 7

Lecture 9 (first half), Test 1 (in class)

Oct. 9

Lecture 10: Concurrency and Threads, Locks,

Oct. 14

Lecture 11: Address Translation, TLBs

Oct. 16

Lecture 12: Segmentation, Free space management

Oct. 21

Lecture 13: Paging

Oct. 23

Lecture 14: pH

Nov. 4

Lecture 15

Nov. 6

Lecture 16: Kernel modification walkthrough

Nov. 11

Lecture 17: Graphics

Nov. 13

Lecture 18: Device Drivers

Nov. 18

Lecture 19: Cloud Computing

Nov. 20

Lecture 20 (first half), Test 2 (second half)

Nov. 25

Lecture 21: Research

Nov. 27

Lecture 22: Security and malware

Dec. 2

Lecture 23: Other Operating Systems

Dec. 4

Lecture 24: The Future

Other chapters: Lottery Scheduling, Multi-CPU scheduling, Concurrent Data Structures, Thread API, Condition Variables, Semaphores, Concurrency Problems, Smaller Tables , FFS, Distributed Systems, Swapping: Mechanisms, Swapping: Policies, NFS, AFS

Tutorials

Each week you will get a progress grade from 0-4, given to you by a TA. If you are being diligent, you should be able to get 4's every week. The easiest way to get your grade is to come to tutorial and meet with your TA; alternately, you can meet a TA in their office hours or, at their discretion, discuss things with them online.

Date

Tutorials

Sept. 18,22

UNIX Introduction

Sept. 25, 29

Device files, filesystems, and kernel modules

Oct. 2, 6

kernel modules and using the source

Oct. 9, 13

Filesystem concurrency

Oct. 16, 20

Openstack

Oct. 23, Nov. 3

Catch up/Project help

Nov. 6, 10

Modifying system calls

Nov. 13, 17

kprobes, calling userspace

Nov. 20, 24

Kernel map, projects

Nov. 27, Dec. 1

Projects

Assignments

Due Date

Assignments

Sept. 23

Assignment 1

Sept. 30

Assignment 2

Oct. 6

Assignment 3

Nov. 5

Assignment 4

Nov. 18

Assignment 5

Course Software

In this course we will primarily working with Ubuntu, a widely-used family of Linux distributions. We will be using its Lubuntu variant in Virtualbox earlier in the term; later on we will transition to using Ubuntu Server on the SCS's Openstack installation (accessible only from the Carleton network).

You may use other Linux distributions to complete the assigned work and do your project; there will be differences, however, in some aspects (such as installing software), particularly if you use a distribution not based on Ubuntu or Debian.

Using Virtualbox in the labs

In the SCS labs you should be able to run the course VM by starting Virtualbox (listed in the Applications menu) and selecting the COMP 2406/3000 virtual machine image. After the VM has fully booted you will be automatically logged into the student account; this account has admin privileges and its password is "tneduts!".

We highly recommend running your VM in full-screen mode (select from the menu, not by maximizing the window). Do all of your work inside of the VM; it should be fast enough and you won't have any issues with sharing files or with firewalls/network connectivity.

You can save the work you do from the course VM (in the student account) to your SCS account and restore it to any other copy of the class VM (on your machines or in the labs) by running using the following commands:

 save3000 <SCS username>
 restore3000 <SCS username>
 compare3000 <SCS username>

If you use these commands, use them consistently. That means run restore3000 when you first log in, and run save3000 just before logging out. If you don't do this, you will erase the work that you had done previously when you save.

If you forgot to restore and you want to save, try running this:

  rsync -a -v --progress ~/ <SCS username>@access.scs.carleton.ca:COMP3000/

This is the same as the save3000 command minus the options (--delete and --force) that deletes files in the destination that don't exist in the source. As a check, you may want to add the -n option to do a dry run.

Running the Virtualbox VM on your own machines

If you want to run the VM appliance on your own system (running essentially any desktop operating system you want), just download the virtual appliance file and import. The SHA1 hash of this file is:

   a8a70ec2e1b49699f4de29c872ecec7cee21888f COMP 2406 & 3000, Fall 2015.ova

On Windows you can compute this hash for your downloaded file using the command FCIV -sha1 COMP 2406-3000 Fall 2014.ova. If the hash is different from above, your download has been corrupted.

If the application is not VirtualBox, you'll need to:

  • Have the VM platform ignore any errors in the structure of the appliance found during the import process;
  • Uninstall the VirtualBox guest additions by typing starting a terminal application and running
  sudo apt-get purge virtualbox-guest-x11 virtualbox-guest-utils
  • Install your platform's own Linux guest additions, if available.