Difference between revisions of "Distributed OS: Winter 2015"

From Soma-notes
Jump to navigation Jump to search
 
(40 intermediate revisions by 2 users not shown)
Line 3: Line 3:
[[Distributed OS: Winter 2015 Course Outline|Here]] is the course outline.  It should see only minor modifications during the semester.
[[Distributed OS: Winter 2015 Course Outline|Here]] is the course outline.  It should see only minor modifications during the semester.


==Topics==
==Assigned Readings==


===Early Systems (Jan. 12)===
===January 12, 2015===


The Early Internet:
The Early Internet:
Line 18: Line 18:
* [http://en.wikipedia.org/wiki/The_Mother_of_All_Demos Wikipedia's page on "The Mother of all Demos"]
* [http://en.wikipedia.org/wiki/The_Mother_of_All_Demos Wikipedia's page on "The Mother of all Demos"]


===The Early Web===
===January 19, 2015===
 
* [https://archive.org/details/02Kahle000673 Berners-Lee et al., "World-Wide Web: The Information Universe" (1992)], pp. 52-58
* [http://www.youtube.com/watch?v=72nfrhXroo8 Alex Wright, "The Web That Wasn't" (2007)], Google Tech Talk
 
===UNIX and Plan 9===


* [http://en.wikipedia.org/wiki/Multics Wikipedia article on Multics]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/unix.pdf Dennis M. Ritchie and Ken Thompson, "The UNIX Time-Sharing System" (1974)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/unix.pdf Dennis M. Ritchie and Ken Thompson, "The UNIX Time-Sharing System" (1974)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2014w/presotto-plan9.pdf Presotto et. al, Plan 9, A Distributed System (1991)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2014w/pike-plan9.pdf Pike et al., Plan 9 from Bell Labs (1995)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/sandberg-nfs.pdf Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/ousterhout-sprite.pdf John Ousterhout et al., "The Sprite Network Operating System" (1987)]


===NFS and AFS===
===January 26, 2015===


* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/sandberg-nfs.pdf Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/cheriton-v.pdf David R. Cheriton, "The V Distributed System." (1988)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/tanenbaum-amoeba.pdf Andrew Tannenbaum et al., "The Amoeba System" (1990)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/clouds-dasgupta.pdf Partha Dasgupta et al., "The Clouds Distributed Operating System" (1991)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/howard-afs.pdf John H. Howard et al., "Scale and Performance in a Distributed File System" (1988)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/howard-afs.pdf John H. Howard et al., "Scale and Performance in a Distributed File System" (1988)]


===GFS and Ceph===
===February 2, 2015===
 
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/clouds-dasgupta.pdf Partha Dasgupta et al., "The Clouds Distributed Operating System" (1991)] (continued from last class)
* [http://homeostasis.scs.carleton.ca/~soma/distos/2014w/presotto-plan9.pdf Presotto et. al, Plan 9, A Distributed System (1991)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2014w/pike-plan9.pdf Pike et al., Plan 9 from Bell Labs (1995)]
* [http://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]
* [http://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]
* [http://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].


===Chubby===
Read but no need for a response:
* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]
* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]


* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]
===February 9, 2015===


===Oceanstore===
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (2000)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (2000)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (2003)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (2003)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/adya-farsite-intro.pdf Atul Adya et al.,"FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment" (2002)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/bolosky-farsite-retro.pdf William J. Bolosky et al., "The Farsite Project: A Retrospective" (2007)]
'''ALSO''' Present paper & topic to class for final project!
===February 23, 2015===
* [http://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].
* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]


===Farsite===
===March 2, 2015===


* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/adya-farsite-intro.pdf Atul Adya et al.,"FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment" (2002)]
* [https://archive.org/details/02Kahle000673 Berners-Lee et al., "World-Wide Web: The Information Universe" (1992)], pp. 52-58
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/bolosky-farsite-retro.pdf William J. Bolosky et al., "The Farsite Project: A Retrospective" (2007)]
* [http://www.youtube.com/watch?v=72nfrhXroo8 Alex Wright, "The Web That Wasn't" (2007)], Google Tech Talk


===Public Resource Computing===
===March 9, 2015===


* Anderson et al., "SETI@home: An Experiment in Public-Resource Computing" (CACM 2002) [http://dx.doi.org/10.1145/581571.581573 (DOI)] [http://dl.acm.org.proxy.library.carleton.ca/citation.cfm?id=581573 (Proxy)]
* Anderson et al., "SETI@home: An Experiment in Public-Resource Computing" (CACM 2002) [http://dx.doi.org/10.1145/581571.581573 (DOI)] [http://dl.acm.org.proxy.library.carleton.ca/citation.cfm?id=581573 (Proxy)]
* Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004) [http://dx.doi.org/10.1109/GRID.2004.14 (DOI)] [http://ieeexplore.ieee.org.proxy.library.carleton.ca/stamp/stamp.jsp?tp=&arnumber=1382809 (Proxy)]
* Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004) [http://dx.doi.org/10.1109/GRID.2004.14 (DOI)] [http://ieeexplore.ieee.org.proxy.library.carleton.ca/stamp/stamp.jsp?tp=&arnumber=1382809 (Proxy)]
* [http://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)]
* [http://dl.acm.org/citation.cfm?doid=2517349.2522738 Murray et al., "Naiad: a timely dataflow system" (SOSP 2013)]


===Distributed Hash Tables===
===March 16, 2015===


* [http://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]
* [http://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]
* [http://en.wikipedia.org/wiki/Kademlia Wikipedia's article on Kademlia]
* [http://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf Maymounkov and Mazieres, "Kademlia: A Peer-to-peer information system based on the XOR Metric" (2002)]
* [http://en.wikipedia.org/wiki/Tapestry_%28DHT%29 Wikipedia's article on Tapestry]
* [http://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]
* [http://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]
* [https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Geambasu.pdf Geambasu et al., "Comet: An active distributed key-value store" (OSDI 2010)]


===Structured Data===
===March 23, 2015===


* [http://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]
* [http://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]
* [https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Geambasu.pdf Geambasu et al., "Comet: An active distributed key-value store" (OSDI 2010)]
* [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)]
 
===March 30, 2015===
 
* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)]
* [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/gordon Gordon et al., "COMET: Code Offload by Migrating Execution Transparently" (OSDI 2012)]
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)]
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/zhang Zhang et al., "Customizable and Extensible Deployment for Mobile/Cloud Applications" (OSDI 2014)]
 
==Project Selection Help==


===Computational Models===
To develop your literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.


* [http://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)]
To begin selecting a paper, I suggest that you:
* [http://dl.acm.org/citation.cfm?doid=2517349.2522738 Murray et al., "Naiad: a timely dataflow system" (SOSP 2013)]
* search on Google Scholar using keywords relating to your interests, and/or
* browse the proceedings of major conferences that publish work related to distributed operating systems.


===Literature Review Help===
The main operating system conferences are [https://www.usenix.org/conferences/byname/179 OSDI] and ACM SOSP ([http://sosp.org/ sosp.org],[http://dl.acm.org/event.cfm?id=RE208&CFID=475138068&CFTOKEN=43996267 ACM DL]).  Note that not all the work here is on distributed operating systems!  Also, many other conferences publish some work related to distributed operating systems, e.g. [https://www.usenix.org/conferences/byname/178 NSDI].


* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]
* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]


==Notes==
==Notes==


<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0">
<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0" class="wikitable">
   <tr valign="top">
   <tr valign="top">
     <th>
     <th>
Line 106: Line 131:
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Session 2|Session 2]]
       <p>[[DistOS 2015W Session 2|Session 2]]
       </p>
       </p>
       </td>
       </td>
Line 112: Line 137:
     <tr>
     <tr>
       <td>
       <td>
       <p>Jan. 14
       <p>Jan. 19
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Lecture 3|Lecture 3]]
       <p>[[DistOS 2015W Session 3|Session 3]]
       </p>
       </p>
       </td>
       </td>
Line 122: Line 147:
     <tr>
     <tr>
       <td>
       <td>
       <p>Jan. 16
       <p>Jan. 26
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Lecture 4|Lecture 4]]
       <p>[[DistOS 2015W Session 4|Session 4]]
       </p>
       </p>
       </td>
       </td>
Line 132: Line 157:
     <tr>
     <tr>
       <td>
       <td>
       <p>Jan. 21
       <p>Feb. 2
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Lecture 5|Lecture 5]]
       <p>[[DistOS 2015W Session 5|Session 5]]
       </p>
       </p>
       </td>
       </td>
Line 142: Line 167:
     <tr>
     <tr>
       <td>
       <td>
       <p>Jan. 23
       <p>Feb. 9
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Lecture 6|Lecture 6]]
       <p>[[DistOS 2015W Session 6|Session 6]]
       </p>
       </p>
       </td>
       </td>
Line 152: Line 177:
     <tr>
     <tr>
       <td>
       <td>
       <p>Jan. 28
       <p>Feb. 23
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Lecture 7|Lecture 7]]
       <p>[[DistOS 2015W Session 7|Session 7]]
       </p>
       </p>
       </td>
       </td>
Line 162: Line 187:
     <tr>
     <tr>
       <td>
       <td>
       <p>Jan. 30
       <p>Mar. 2
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Lecture 8|Lecture 8]]
       <p>[[DistOS 2015W Session 8|Session 8]], <b>[http://homeostasis.scs.carleton.ca/~soma/distos/2015w/comp4000-2015w-midterm.pdf Midterm]</b> (4 PM) and Proposal due
       </p>
       </p>
       </td>
       </td>
Line 172: Line 197:
     <tr>
     <tr>
       <td>
       <td>
       <p>Feb. 4
       <p>Mar. 9
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Lecture 9|Lecture 9]]
       <p>[[DistOS 2015W Session 9|Session 9]]
       </p>
       </p>
       </td>
       </td>
Line 182: Line 207:
     <tr>
     <tr>
       <td>
       <td>
       <p>Feb. 6
       <p>Mar. 16
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Lecture 10|Lecture 10]]
       <p>[[DistOS 2015W Session 10|Session 10]]
       </p>
       </p>
       </td>
       </td>
Line 192: Line 217:
     <tr>
     <tr>
       <td>
       <td>
       <p>Feb. 11
       <p>Mar. 23
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Lecture 11|Lecture 11]]
       <p>[[DistOS 2015W Session 11|Session 11]]
       </p>
       </p>
       </td>
       </td>
Line 202: Line 227:
     <tr>
     <tr>
       <td>
       <td>
       <p>Feb. 13
       <p>Mar. 30
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Lecture 12|Lecture 12]]<br>'''Project Proposals Due'''
       <p>[[DistOS 2015W Session 12|Session 12]]
       </p>
       </p>
       </td>
       </td>
Line 212: Line 237:
     <tr>
     <tr>
       <td>
       <td>
       <p>Feb. 25
       <p>Apr. 6
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>[[DistOS 2014W Midterm Review|Midterm Review]]
       <p>Final Project Presentations
       </p>
       </p>
       </td>
       </td>
Line 222: Line 247:
     <tr>
     <tr>
       <td>
       <td>
       <p>Feb. 27
       <p>April 13, 2 PM
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
       <p>Midterm Exam
       <p>'''Final Exam'''
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Mar. 4
      </p>
      </td>
      <td>
      <p>[[DistOS 2014W Lecture 14|Lecture 14]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Mar. 6
      </p>
      </td>
      <td>
      <p>[[DistOS 2014W Lecture 15|Lecture 15]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Mar. 11
      </p>
      </td>
      <td>
      <p>[[DistOS 2014W Lecture 16|Lecture 16]]
       </p>
       </p>
       </td>
       </td>
Line 262: Line 257:
     <tr>
     <tr>
       <td>
       <td>
       <p>Mar. 13
       <p>April 20, Noon
       </p>
       </p>
       </td>
       </td>
       <td>
       <td>
      <p>[[DistOS 2014W Lecture 17|Lecture 17]]
       <p>'''Final Project Due'''
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Mar. 18
      </p>
      </td>
      <td>
      <p>[[DistOS 2014W Lecture 18|Lecture 18]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Mar. 20
      </p>
      </td>
      <td>
      <p>[[DistOS 2014W Lecture 19|Lecture 19]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Mar. 25
      </p>
      </td>
      <td>
      <p>[[DistOS 2014W Lecture 20|Lecture 20]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Mar. 27
      </p>
      </td>
      <td>
      <p>[[DistOS 2014W Lecture 21|Lecture 21]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Apr. 1
      </p>
      </td>
      <td>
      <p>[[DistOS 2014W Lecture 22|Lecture 22]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Apr. 3
      </p>
      </td>
      <td>
      <p>[[DistOS 2014W Lecture 23|Lecture 23]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>Apr. 8
      </p>
      </td>
      <td>
      <p>[[DistOS 2014W Lecture 24|Lecture 24]]
      </p>
      </td>
    </tr>
    <tr>
      <td>
      <p>April 24, 2 PM
      </p>
      </td>
      <td>
       <p>'''Final Exam'''
       </p>
       </p>
       </td>
       </td>
     </tr>
     </tr>
</table>
</table>

Latest revision as of 14:26, 11 April 2015

Course Outline

Here is the course outline. It should see only minor modifications during the semester.

Assigned Readings

January 12, 2015

The Early Internet:

The Alto:

The Mother of All Demos:

January 19, 2015

January 26, 2015

February 2, 2015

Read but no need for a response:

February 9, 2015

ALSO Present paper & topic to class for final project!

February 23, 2015

March 2, 2015

March 9, 2015

March 16, 2015

March 23, 2015

March 30, 2015

Project Selection Help

To develop your literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.

To begin selecting a paper, I suggest that you:

  • search on Google Scholar using keywords relating to your interests, and/or
  • browse the proceedings of major conferences that publish work related to distributed operating systems.

The main operating system conferences are OSDI and ACM SOSP (sosp.org,ACM DL). Note that not all the work here is on distributed operating systems! Also, many other conferences publish some work related to distributed operating systems, e.g. NSDI.


Notes

Date

Topic

Jan. 5

Session 1

Jan. 12

Session 2

Jan. 19

Session 3

Jan. 26

Session 4

Feb. 2

Session 5

Feb. 9

Session 6

Feb. 23

Session 7

Mar. 2

Session 8, Midterm (4 PM) and Proposal due

Mar. 9

Session 9

Mar. 16

Session 10

Mar. 23

Session 11

Mar. 30

Session 12

Apr. 6

Final Project Presentations

April 13, 2 PM

Final Exam

April 20, Noon

Final Project Due