Distributed OS: Fall 2021: Difference between revisions

← Older edit

Latest revision as of 00:15, 10 December 2021

Course Outline

Here is the course outline.

Experiences

Experience 1, due November 2, 2021
Experience 2, due December 10, 2021

Using Openstack

The experiences (and potentially your project) will involve using the SCS Openstack cluster:

How to use Openstack (for COMP 4000/5102)
SCS Openstack web console (only available from the Carleton network)

Classes

More information will be posted here by the first class.

Note: If you don't have access to the course information on brightspace, please email Prof. Somayaji, you may need to be manually added. (If you're at UofO or you recently added COMP 5102, you probably need to be added manually.) On Brightspace the course should show up as COMP 4000 no matter whether you enrolled in it or COMP 5102.

Project Help

To develop a literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.

To begin selecting a paper, I suggest that you:

search on Google Scholar using keywords relating to your interests, and/or
browse the proceedings of major conferences that publish work related to distributed operating systems.

The main operating system conferences are OSDI and ACM SOSP (sosp.org,ACM DL). Note that not all the work here is on distributed operating systems! Also, many other conferences publish some work related to distributed operating systems, e.g. NSDI.

To help you write a literature review or the background of a research paper, read the following:

Harvey, "What Is a Literature Review?" (DOC) (PPT)
Taylor, "The Literature Review: A Few Tips On Conducting It"

Implementation experiences will depend upon your technical background and personal interests. If you have an idea, please discuss with Anil or William as soon as you can.

Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)

September 21, 2021

NFS

Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)

September 23, 2021

Distributed Shared Memory

Jelica Protic et al., "Distributed Shared Memory: Concepts and Systems" (1996)

October 21, 2021

Midterm

Wikipedia's article on Distributed Hash Tables

November 23, 2021

Cryptocurrency origins

November 25, 2021

No class, US Thanksgiving

November 30, 2021

Ethereum & NFTs

December 2, 2021

Ethereum Security & The Future

December 7, 2021

Project Presentations (Informal)

December 9, 2021

Class wrap up & Exam review

December 19, 2021

Final Exam (Online, 2-5 PM)

@@ Line 3: / Line 3: @@
 [[Distributed OS: Fall 2021 Course Outline|Here]] is the course outline.
-==Using Openstack==
+==Experiences==
+* [[DistOS 2021F Experience 1|Experience 1]], due November 2, 2021
+* [[DistOS 2021F Experience 2|Experience 2]], due December 10, 2021
+===Using Openstack===
 The experiences (and potentially your project) will involve using the SCS Openstack cluster:
@@ Line 15: / Line 20: @@
 '''Note:''' If you don't have access to the course information on brightspace, please email Prof. Somayaji, you may need to be manually added.  (If you're at UofO or you recently added COMP 5102, you probably need to be added manually.)  On Brightspace the course should show up as COMP 4000 no matter whether you enrolled in it or COMP 5102.
-==Readings==
+==Project Help==
+To develop a literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.
-===September 14, 2021===
+To begin selecting a paper, I suggest that you:
+* search on Google Scholar using keywords relating to your interests, and/or
+* browse the proceedings of major conferences that publish work related to distributed operating systems.
+The main operating system conferences are [https://www.usenix.org/conferences/byname/179 OSDI] and ACM SOSP ([http://sosp.org/ sosp.org],[http://dl.acm.org/event.cfm?id=RE208&CFID=475138068&CFTOKEN=43996267 ACM DL]).  Note that not all the work here is on distributed operating systems!  Also, many other conferences publish some work related to distributed operating systems, e.g. [https://www.usenix.org/conferences/byname/178 NSDI].
+To help you write a literature review or the background of a research paper, read the following:
+* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]
+* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]
+Implementation experiences will depend upon your technical background and personal interests.  If you have an idea, please discuss with Anil or William as soon as you can.
+==Readings & Classes==
+===[[DistOS 2021F 2021-09-09|September 9, 2021]]===
+Introduction
+===[[DistOS 2021F 2021-09-14|September 14, 2021]]===
 UNIX
 * [https://www.youtube.com/watch?v=tc4ROCJYbm0 The UNIX System: Making Computers More Productive (1982)], video
@@ Line 23: / Line 48: @@
 Note that the video covers the main points of the paper.
-===September 16, 2021===
+===[[DistOS 2021F 2021-09-16|September 16, 2021]]===
 LOCUS
 * [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)]
-===September 21, 2021===
+===[[DistOS 2021F 2021-09-21|September 21, 2021]]===
 NFS
 * [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/sandberg-nfs.pdf Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)]
-===September 23, 2021===
+===[[DistOS 2021F 2021-09-23|September 23, 2021]]===
 Distributed Shared Memory
 * [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/protic-overview.pdf Jelica Protic et al., "Distributed Shared Memory: Concepts and Systems" (1996)]
-===September 28, 2021===
+===[[DistOS 2021F 2021-09-28|September 28, 2021]]===
 Plan 9
 * [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/presotto-plan9.pdf Presotto et. al, Plan 9, A Distributed System (1991)]
 * [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/pike-plan9.pdf Pike et al., Plan 9 from Bell Labs (1995)]
-===September 30, 2021===
+===[[DistOS 2021F 2021-09-30|September 30, 2021]]===
 Solaris Zones
 * [https://www.usenix.org/legacy/publications/library/proceedings/lisa04/tech/full_papers/price/price.pdf Price & Tucker, "Solaris Zones: Operating System Support for Consolidating Commercial Workloads" (LISA 2004)]
 * [https://www.usenix.org/system/files/login/articles/963-galvin.pdf Galvin, "Solaris 10 Containers" (;LOGIN 2005)]
+===[[DistOS 2021F 2021-10-05|October 5, 2021]]===
+NASD & GFS
+* [https://homeostasis.scs.carleton.ca/~soma/distos/2008-03-10/gibson-nasd.pdf Garth A. Gibson et al., "A Cost-Effective, High-Bandwidth Storage Architecture" (1998)]
+* [https://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]
+===[[DistOS 2021F 2021-10-07|October 7, 2021]]===
+Chubby & ZooKeeper
+* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]
+* [http://static.usenix.org/event/atc10/tech/full_papers/Hunt.pdf Hunt et al., "ZooKeeper: Wait-free coordination for Internet-scale systems" (USENIX ATC 2010)] [https://www.usenix.org/legacy/multimedia/atc10hunt (video)]
+===[[DistOS 2021F 2021-10-12|October 12, 2021]]===
+BigTable & MapReduce
+* [https://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]
+* [https://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)] (be sure to read paper)
+===[[DistOS 2021F 2021-10-14|October 14, 2021]]===
+Omega & Borg
+* [https://research.google/pubs/pub41684.pdf Schwarzkopf et al., "Omega: flexible, scalable schedulers for large compute clusters" (EuroSys 2013)]
+* [https://research.google/pubs/pub43438.pdf Verma et al., "Large-scale cluster management at Google with Borg" (EuroSys 2015)]
+===[[DistOS 2021F 2021-10-19|October 19, 2021]]===
+Midterm Review
+===October 21, 2021===
+'''Midterm'''
+===[[DistOS 2021F 2021-11-02|November 2, 2021]]===
+Cassandra & Dynamo
+* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]
+* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]
+===[[DistOS 2021F 2021-11-04|November 04, 2021]]===
+Haystack & F4
+* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)]
+* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)]
+===[[DistOS 2021F 2021-11-09|November 09, 2021]]===
+'''Proposal due by 11:59 PM'''
+Spanner & Tensorflow
+* [https://www.usenix.org/conference/osdi12/technical-sessions/presentation/corbett Corbett et al., "Spanner: Google’s Globally-Distributed Database" (OSDI 2012)]
+* [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)]
+===[[DistOS 2021F 2021-11-11|November 11, 2021]]===
+Ceph
+* [https://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].
+* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/weil2006-crush.pdf Weil et al., CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data (SC 2006)]
+===[[DistOS 2021F 2021-11-16|November 16, 2021]]===
+OceanStore & BOINC
+* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (SIGPLAN 2000)]
+* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (FAST 2003)]
+* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/anderson-boinc.pdf Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004)]
+===[[DistOS 2021F 2021-11-18|November 18, 2021]]===
+Tapestry & Delos
+* [https://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]
+* [https://www.usenix.org/system/files/osdi20-balakrishnan.pdf Balakrishnan et al., "Virtual Consensus in Delos" (OSDI 2020)]
+Background (optional but helpful):
+* [https://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]
+===[[DistOS 2021F 2021-11-23|November 23, 2021]]===
+Cryptocurrency origins
+* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/lamport1982-generals.pdf Lamport et al., "The Byzantine Generals Problem" (TOPLAS 1982)]
+* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/nakamoto2008-bitcoin.pdf Nakamoto, "Bitcoin: A Peer-to-Peer Electronic Cash System" (2008)]
+* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/nakamoto2008-email.pdf Nakamoto, "Re: Bitcoin P2P e-cash paper" (2008)]
+* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/lemieux2013-satoshi.pdf Lemieux, "Who is Satoshi Nakamoto?" (Regulation, Fall 2013)]
+===November 25, 2021===
+No class, US Thanksgiving
+===[[DistOS 2021F 2021-11-30|November 30, 2021]]===
+Ethereum & NFTs
+* [https://files.gitter.im/ethereum/yellowpaper/VIyt/Paper.pdf Wood, Ethereum: A secure decentralised generalised transaction ledger (2014)]
+* [https://arxiv.org/pdf/2105.07447 Wang et al., "Non-Fungible Token (NFT): Overview, Evaluation, Opportunities and Challenges" (2021)]
+===[[DistOS 2021F 2021-12-02|December 2, 2021]]===
+Ethereum Security & The Future
+* [https://dl.acm.org/doi/pdf/10.1145/3391195 Chen et al, "A survey on ethereum systems security: Vulnerabilities, attacks, and defenses" (CSUR 2020)]
+* [https://homeostasis.scs.carleton.ca/~soma/pubs/burgess-nspw2018.pdf Burgess & Somayaji, "After the BlockCloud Apocalypse" (NSPW 2018)]
+===December 7, 2021===
+Project Presentations (Informal)
+===[[DistOS 2021F 2021-12-09|December 9, 2021]]===
+Class wrap up & Exam review
+===December 19, 2021===
+'''Final Exam''' (Online, 2-5 PM)