View source for Distributed OS: Fall 2021

==Course Outline==

[[Distributed OS: Fall 2021 Course Outline|Here]] is the course outline.

==Experiences==

* [[DistOS 2021F Experience 1|Experience 1]], due November 2, 2021
* [[DistOS 2021F Experience 2|Experience 2]], due November 30, 2021

===Using Openstack===

The experiences (and potentially your project) will involve using the SCS Openstack cluster:
* [[DistOS 2021F: Using Openstack|How to use Openstack (for COMP 4000/5102)]]
* [https://openstack.scs.carleton.ca SCS Openstack web console] (only available from the Carleton network)

==Classes==

More information will be posted here by the first class.

'''Note:''' If you don't have access to the course information on brightspace, please email Prof. Somayaji, you may need to be manually added.  (If you're at UofO or you recently added COMP 5102, you probably need to be added manually.)  On Brightspace the course should show up as COMP 4000 no matter whether you enrolled in it or COMP 5102.

==Project Help==

To develop a literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.

To begin selecting a paper, I suggest that you:
* search on Google Scholar using keywords relating to your interests, and/or
* browse the proceedings of major conferences that publish work related to distributed operating systems.

The main operating system conferences are [https://www.usenix.org/conferences/byname/179 OSDI] and ACM SOSP ([http://sosp.org/ sosp.org],[http://dl.acm.org/event.cfm?id=RE208&CFID=475138068&CFTOKEN=43996267 ACM DL]).  Note that not all the work here is on distributed operating systems!  Also, many other conferences publish some work related to distributed operating systems, e.g. [https://www.usenix.org/conferences/byname/178 NSDI].

To help you write a literature review or the background of a research paper, read the following:
* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]
* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]

Implementation experiences will depend upon your technical background and personal interests.  If you have an idea, please discuss with Anil or William as soon as you can.

==Readings & Classes==

===[[DistOS 2021F 2021-09-09|September 9, 2021]]===

Introduction

===[[DistOS 2021F 2021-09-14|September 14, 2021]]===
UNIX
* [https://www.youtube.com/watch?v=tc4ROCJYbm0 The UNIX System: Making Computers More Productive (1982)], video
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/unix.pdf Dennis M. Ritchie and Ken Thompson, "The UNIX Time-Sharing System" (1974)]
Note that the video covers the main points of the paper.

===[[DistOS 2021F 2021-09-16|September 16, 2021]]===
LOCUS
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)]

===[[DistOS 2021F 2021-09-21|September 21, 2021]]===
NFS
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/sandberg-nfs.pdf Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)]

===[[DistOS 2021F 2021-09-23|September 23, 2021]]===
Distributed Shared Memory
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/protic-overview.pdf Jelica Protic et al., "Distributed Shared Memory: Concepts and Systems" (1996)]

===[[DistOS 2021F 2021-09-28|September 28, 2021]]===
Plan 9
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/presotto-plan9.pdf Presotto et. al, Plan 9, A Distributed System (1991)]
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/pike-plan9.pdf Pike et al., Plan 9 from Bell Labs (1995)]

===[[DistOS 2021F 2021-09-30|September 30, 2021]]===
Solaris Zones
* [https://www.usenix.org/legacy/publications/library/proceedings/lisa04/tech/full_papers/price/price.pdf Price & Tucker, "Solaris Zones: Operating System Support for Consolidating Commercial Workloads" (LISA 2004)]
* [https://www.usenix.org/system/files/login/articles/963-galvin.pdf Galvin, "Solaris 10 Containers" (;LOGIN 2005)]

===[[DistOS 2021F 2021-10-05|October 5, 2021]]===

NASD & GFS
* [https://homeostasis.scs.carleton.ca/~soma/distos/2008-03-10/gibson-nasd.pdf Garth A. Gibson et al., "A Cost-Effective, High-Bandwidth Storage Architecture" (1998)]
* [https://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]

===[[DistOS 2021F 2021-10-07|October 7, 2021]]===

Chubby & ZooKeeper
* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]
* [http://static.usenix.org/event/atc10/tech/full_papers/Hunt.pdf Hunt et al., "ZooKeeper: Wait-free coordination for Internet-scale systems" (USENIX ATC 2010)] [https://www.usenix.org/legacy/multimedia/atc10hunt (video)]

===[[DistOS 2021F 2021-10-12|October 12, 2021]]===

BigTable & MapReduce
* [https://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]
* [https://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)] (be sure to read paper)

===[[DistOS 2021F 2021-10-14|October 14, 2021]]===

Omega & Borg
* [https://research.google/pubs/pub41684.pdf Schwarzkopf et al., "Omega: flexible, scalable schedulers for large compute clusters" (EuroSys 2013)]
* [https://research.google/pubs/pub43438.pdf Verma et al., "Large-scale cluster management at Google with Borg" (EuroSys 2015)]

===[[DistOS 2021F 2021-10-19|October 19, 2021]]===

Midterm Review

===October 21, 2021===

'''Midterm'''

===[[DistOS 2021F 2021-11-02|November 2, 2021]]===

Cassandra & Dynamo
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]

===[[DistOS 2021F 2021-11-04|November 04, 2021]]===

Haystack & F4
* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)]
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)]

===[[DistOS 2021F 2021-11-09|November 09, 2021]]===

'''Proposal due by 11:59 PM'''

Spanner & Tensorflow
* [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)]
* [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)]

===[[DistOS 2021F 2021-11-11|November 11, 2021]]===

Ceph
* [https://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].
* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/weil2006-crush.pdf Weil et al., CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data (SC 2006)]

===[[DistOS 2021F 2021-11-16|November 16, 2021]]===

OceanStore & BOINC
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (SIGPLAN 2000)]
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (FAST 2003)]
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/anderson-boinc.pdf Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004)]

===[[DistOS 2021F 2021-11-18|November 18, 2021]]===

Tapestry & Delos
* [https://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]
* [https://www.usenix.org/system/files/osdi20-balakrishnan.pdf Balakrishnan et al., "Virtual Consensus in Delos" (OSDI 2020)]

Background (optional but helpful):
* [https://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]

===December 19, 2021===

'''Final Exam''' (Online, 2-5 PM)