Difference between revisions of "Distributed OS: Fall 2017"
Jump to navigation
Jump to search
| (13 intermediate revisions by the same user not shown) | |||
| Line 72: | Line 72: | ||
* [http://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)] | * [http://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)] | ||
* Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004) [http://dx.doi.org/10.1109/GRID.2004.14 (DOI)] [http://ieeexplore.ieee.org.proxy.library.carleton.ca/stamp/stamp.jsp?tp=&arnumber=1382809 (Proxy)] | * Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004) [http://dx.doi.org/10.1109/GRID.2004.14 (DOI)] [http://ieeexplore.ieee.org.proxy.library.carleton.ca/stamp/stamp.jsp?tp=&arnumber=1382809 (Proxy)] | ||
===October 27, 2017=== | |||
[[DistOS Midterm Review Fall 2017|Midterm review (optional)]] | |||
===October 31, 2017=== | ===October 31, 2017=== | ||
Midterm exam (COMP 4000 students only) | [http://homeostasis.scs.carleton.ca/~soma/distos/2017f/comp4000-2017f-midterm.pdf Midterm exam] (COMP 4000 students only) | ||
Project outline due | Project outline due | ||
===November 2, 2017=== | ===November 2, 2017=== | ||
Botnets and Distributed OS (Discussion) | |||
===November 7, 2017=== | |||
* [http://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)] | * [http://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)] | ||
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)] | * [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)] | ||
===November | ===November 9, 2017=== | ||
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)] | * [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)] | ||
* [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)] | * [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)] | ||
===November | ===November 14, 2017=== | ||
* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)] | * [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)] | ||
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)] | * [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)] | ||
===November | ===November 16, 2017=== | ||
* Qi Huang et al., "SVE: Distributed Video Processing at Facebook Scale" (SOSP 2017) | * [https://dl.acm.org/citation.cfm?id=3132775 Qi Huang et al., "SVE: Distributed Video Processing at Facebook Scale" (SOSP 2017)] | ||
* [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)] | * [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)] | ||
===November | ===November 21, 2017=== | ||
* [http://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)]. | * [http://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)]. | ||
===November 23, 2017=== | ===November 23, 2017=== | ||
(US Thanksgiving) | No class (US Thanksgiving) | ||
===November 28, 2017=== | ===November 28, 2017=== | ||
* [http://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)] | |||
Background (optional but helpful): | |||
* [http://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables] | |||
* [http://en.wikipedia.org/wiki/Kademlia Wikipedia's article on Kademlia] | |||
* [http://en.wikipedia.org/wiki/Tapestry_%28DHT%29 Wikipedia's article on Tapestry] | |||
===November 30, 2017=== | ===November 30, 2017=== | ||
Class wrap-up discussion/Final Exam Review | |||
===December 5, 2017=== | ===December 5, 2017=== | ||
Project presentations | Project presentations: Vidhi, Khaja, Mrinalini, Yu, Weipeng | ||
===December 7, 2017=== | ===December 7, 2017=== | ||
Project presentations | Project presentations: Vanja, Gurvir, Reza, Gangesh, Amardev | ||
===December 12 & 15, 2017=== | |||
Final Exam (COMP 4000), Dec. 12, 2 PM in TB 236 | |||
Final Projects due on Dec. 15th (COMP 5102) | |||
==Project Help== | ==Project Help== | ||
Latest revision as of 20:50, 11 December 2017
Course Outline
Here is the course outline. It should see only minor modifications during the semester.
Assigned Readings
September 12, 2017
The Early Internet:
- Robert E. Kahn, "Resource-Sharing Computer Communications Networks" (1972) (DOI)
- Computer Networks: The Heralds of Resource Sharing (1972) - video
The Mother of All Demos:
- Doug Engelbart Institute, "Doug's 1968 Demo". You may want to focus on the highlights or the annotated clips.
- Wikipedia's page on "The Mother of all Demos"
September 14, 2017
The Alto:
September 19, 2017
- Wikipedia article on Multics
- Dennis M. Ritchie and Ken Thompson, "The UNIX Time-Sharing System" (1974)
Optional: Browse around the Multicians website.
September 21, 2017
- Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)
- John Ousterhout et al., "The Sprite Network Operating System" (1987)
September 26, 2017
- David R. Cheriton, "The V Distributed System." (1988)
- Andrew Tannenbaum et al., "The Amoeba System" (1990)
- Partha Dasgupta et al., "The Clouds Distributed Operating System" (1991)
September 28, 2017
- Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)
- John H. Howard et al., "Scale and Performance in a Distributed File System" (1988)
October 3, 2017
- John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (2000)
- Sean Rhea et al., "Pond: the OceanStore Prototype" (2003)
October 5, 2017
- Atul Adya et al.,"FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment" (2002)
- William J. Bolosky et al., "The Farsite Project: A Retrospective" (2007)
October 10, 2017
- Presotto et. al, Plan 9, A Distributed System (1991)
- Pike et al., Plan 9 from Bell Labs (1995)
- Harvey, "What Is a Literature Review?" (DOC) (PPT)
- Taylor, "The Literature Review: A Few Tips On Conducting It"
October 12, 2017
October 17, 2017
October 19, 2017
- Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)
- Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004) (DOI) (Proxy)
October 27, 2017
October 31, 2017
Midterm exam (COMP 4000 students only)
Project outline due
November 2, 2017
Botnets and Distributed OS (Discussion)
November 7, 2017
- Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)
- DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)
November 9, 2017
- Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)
- Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)
November 14, 2017
- Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)
- Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)
November 16, 2017
- Qi Huang et al., "SVE: Distributed Video Processing at Facebook Scale" (SOSP 2017)
- Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)
November 21, 2017
November 23, 2017
No class (US Thanksgiving)
November 28, 2017
Background (optional but helpful):
- Wikipedia's article on Distributed Hash Tables
- Wikipedia's article on Kademlia
- Wikipedia's article on Tapestry
November 30, 2017
Class wrap-up discussion/Final Exam Review
December 5, 2017
Project presentations: Vidhi, Khaja, Mrinalini, Yu, Weipeng
December 7, 2017
Project presentations: Vanja, Gurvir, Reza, Gangesh, Amardev
December 12 & 15, 2017
Final Exam (COMP 4000), Dec. 12, 2 PM in TB 236
Final Projects due on Dec. 15th (COMP 5102)
Project Help
To develop your literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.
To begin selecting a paper, I suggest that you:
- search on Google Scholar using keywords relating to your interests, and/or
- browse the proceedings of major conferences that publish work related to distributed operating systems.
The main operating system conferences are OSDI and ACM SOSP (sosp.org,ACM DL). Note that not all the work here is on distributed operating systems! Also, many other conferences publish some work related to distributed operating systems, e.g. NSDI.
To help you write a literature review or the background of a research paper, read the following:
- Harvey, "What Is a Literature Review?" (DOC) (PPT)
- Taylor, "The Literature Review: A Few Tips On Conducting It"