Distributed OS: Fall 2019: Difference between revisions

From Soma-notes
Line 98: Line 98:
===[[DistOS 2019F 2019-10-28|October 28, 2019]] (online)===
===[[DistOS 2019F 2019-10-28|October 28, 2019]] (online)===


BOINC & Tapestry
* Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004) [http://dx.doi.org/10.1109/GRID.2004.14 (DOI)] [http://ieeexplore.ieee.org.proxy.library.carleton.ca/stamp/stamp.jsp?tp=&arnumber=1382809 (Proxy)]
* [http://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]
Background (optional but helpful):
* [http://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]
* [http://en.wikipedia.org/wiki/Kademlia Wikipedia's article on Kademlia]
* [http://en.wikipedia.org/wiki/Tapestry_%28DHT%29 Wikipedia's article on Tapestry]


===[[DistOS 2019F 2019-10-30|October 30, 2019]] (online)===
===[[DistOS 2019F 2019-10-30|October 30, 2019]] (online)===
Farsite & Oceanstore
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/adya-farsite-intro.pdf Atul Adya et al.,"FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment" (2002)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/bolosky-farsite-retro.pdf William J. Bolosky et al., "The Farsite Project: A Retrospective" (2007)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (2000)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (2003)]


Project Proposal due.
Project Proposal due.


===[[DistOS 2019F 2019-11-04|November 4, 2019]] (online)===
===[[DistOS 2019F 2019-11-04|November 4, 2019]] (online)===
GFS & Chubby
* [http://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]
* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]


===[[DistOS 2019F 2019-11-06|November 6, 2019]] (online)===
===[[DistOS 2019F 2019-11-06|November 6, 2019]] (online)===
MapReduce & BigTable
* [http://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)]
* [http://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]


===[[DistOS 2019F 2019-11-11|November 11, 2019]] (online)===
===[[DistOS 2019F 2019-11-11|November 11, 2019]] (online)===
Line 127: Line 150:
===Other readings===
===Other readings===


Farsite & Oceanstore
Ceph
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/adya-farsite-intro.pdf Atul Adya et al.,"FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment" (2002)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/bolosky-farsite-retro.pdf William J. Bolosky et al., "The Farsite Project: A Retrospective" (2007)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (2000)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (2003)]
 
* Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004) [http://dx.doi.org/10.1109/GRID.2004.14 (DOI)] [http://ieeexplore.ieee.org.proxy.library.carleton.ca/stamp/stamp.jsp?tp=&arnumber=1382809 (Proxy)]
 
 
* [http://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]
* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]
 
 
* [http://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].
* [http://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].
* [http://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]
Background (optional but helpful):
* [http://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]
* [http://en.wikipedia.org/wiki/Kademlia Wikipedia's article on Kademlia]
* [http://en.wikipedia.org/wiki/Tapestry_%28DHT%29 Wikipedia's article on Tapestry]


 
Spanner & TensorFlow
* [http://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]
* [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)]
* [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)]
* [http://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)]
* [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)]
* [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)]


Dynamo & Cassandra
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]


Haystack & f4
* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)]
* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)]
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)]
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)]

Revision as of 14:57, 23 October 2019

Course Outline

Here is the course outline.

Project Help

To develop your literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.

To begin selecting a paper, I suggest that you:

  • search on Google Scholar using keywords relating to your interests, and/or
  • browse the proceedings of major conferences that publish work related to distributed operating systems.

The main operating system conferences are OSDI and ACM SOSP (sosp.org,ACM DL). Note that not all the work here is on distributed operating systems! Also, many other conferences publish some work related to distributed operating systems, e.g. NSDI.

To help you write a literature review or the background of a research paper, read the following:

Class Schedule & Readings

September 4, 2019 (in person)

None

September 9, 2019 (online)

The Early Internet:

September 11, 2019 (online)

The Mother of All Demos:

September 16, 2019 (online)

The Alto:

September 18, 2019 (online)

Multics & UNIX:

Optional: Browse around the Multicians website.

September 23, 2019 (online)

LOCUS & NFS

September 25, 2019 (online)

Remote Procedure Calls

September 30, 2019 (online)

Distributed Shared Memory

October 2, 2019 (online)

Sprite, AFS, & Literature reviews

October 7, 2019 (online)

Amoeba & Clouds

October 9, 2019 (online)

Plan 9 & Inferno (no group report)

Second half of class will be a review for the midterm.

October 16, 2019 (in person)

Mid-term exam (in class)

October 28, 2019 (online)

BOINC & Tapestry

Background (optional but helpful):

October 30, 2019 (online)

Farsite & Oceanstore

Project Proposal due.

November 4, 2019 (online)

GFS & Chubby

November 6, 2019 (online)

MapReduce & BigTable


November 11, 2019 (online)

November 13, 2019 (online)

November 18, 2019 (online)

November 20, 2019 (online)

November 25, 2019 (online)

November 27, 2019 (online)

December 2, 2019 (online)

December 4, 2019 (online)

December 6, 2019 (online)

Other readings

Ceph

Spanner & TensorFlow

Dynamo & Cassandra

Haystack & f4

Containers & Orchestration

"Serverless Computing"