Distributed OS: Fall 2021: Difference between revisions

From Soma-notes
 
(16 intermediate revisions by 2 users not shown)
Line 6: Line 6:


* [[DistOS 2021F Experience 1|Experience 1]], due November 2, 2021
* [[DistOS 2021F Experience 1|Experience 1]], due November 2, 2021
* [[DistOS 2021F Experience 2|Experience 2]], due November 30, 2021
* [[DistOS 2021F Experience 2|Experience 2]], due December 10, 2021


===Using Openstack===
===Using Openstack===
Line 19: Line 19:


'''Note:''' If you don't have access to the course information on brightspace, please email Prof. Somayaji, you may need to be manually added.  (If you're at UofO or you recently added COMP 5102, you probably need to be added manually.)  On Brightspace the course should show up as COMP 4000 no matter whether you enrolled in it or COMP 5102.
'''Note:''' If you don't have access to the course information on brightspace, please email Prof. Somayaji, you may need to be manually added.  (If you're at UofO or you recently added COMP 5102, you probably need to be added manually.)  On Brightspace the course should show up as COMP 4000 no matter whether you enrolled in it or COMP 5102.
==Project Help==
To develop a literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.
To begin selecting a paper, I suggest that you:
* search on Google Scholar using keywords relating to your interests, and/or
* browse the proceedings of major conferences that publish work related to distributed operating systems.
The main operating system conferences are [https://www.usenix.org/conferences/byname/179 OSDI] and ACM SOSP ([http://sosp.org/ sosp.org],[http://dl.acm.org/event.cfm?id=RE208&CFID=475138068&CFTOKEN=43996267 ACM DL]).  Note that not all the work here is on distributed operating systems!  Also, many other conferences publish some work related to distributed operating systems, e.g. [https://www.usenix.org/conferences/byname/178 NSDI].
To help you write a literature review or the background of a research paper, read the following:
* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]
* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]
Implementation experiences will depend upon your technical background and personal interests.  If you have an idea, please discuss with Anil or William as soon as you can.


==Readings & Classes==
==Readings & Classes==
Line 86: Line 102:
'''Midterm'''
'''Midterm'''


===[[DistOS 2021F 2021-11-02|November 2, 2021]] (online)===
===[[DistOS 2021F 2021-11-02|November 2, 2021]]===


Cassandra & Dynamo
Cassandra & Dynamo
Line 92: Line 108:
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]


===[[DistOS 2021F 2021-11-04|November 04, 2021]] (online)===
===[[DistOS 2021F 2021-11-04|November 04, 2021]]===


Haystack & F4
Haystack & F4
Line 98: Line 114:
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)]
* [https://www.usenix.org/conference/osdi14/technical-sessions/presentation/muralidhar Muralidhar et al., "f4: Facebook's Warm BLOB Storage System" (OSDI 2014)]


===[[DistOS 2021F 2021-11-09|November 09, 2021]] (online)===
===[[DistOS 2021F 2021-11-09|November 09, 2021]]===
 
'''Proposal due by 11:59 PM'''


Spanner & Tensorflow
Spanner & Tensorflow
Line 104: Line 122:
* [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)]
* [https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi Martin Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning" (OSDI 2016)]


===[[DistOS 2021F 2021-11-11|November 11, 2021]] (online)===
===[[DistOS 2021F 2021-11-11|November 11, 2021]]===


Ceph
Ceph
* [http://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].
* [https://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].
...
* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/weil2006-crush.pdf Weil et al., CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data (SC 2006)]
 
===[[DistOS 2021F 2021-11-16|November 16, 2021]]===
 
OceanStore & BOINC
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (SIGPLAN 2000)]
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (FAST 2003)]
* [https://homeostasis.scs.carleton.ca/~soma/distos/fall2008/anderson-boinc.pdf Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004)]
 
===[[DistOS 2021F 2021-11-18|November 18, 2021]]===
 
Tapestry & Delos
* [https://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]
* [https://www.usenix.org/system/files/osdi20-balakrishnan.pdf Balakrishnan et al., "Virtual Consensus in Delos" (OSDI 2020)]
 
Background (optional but helpful):
* [https://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]
 
===[[DistOS 2021F 2021-11-23|November 23, 2021]]===
 
Cryptocurrency origins
* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/lamport1982-generals.pdf Lamport et al., "The Byzantine Generals Problem" (TOPLAS 1982)]
* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/nakamoto2008-bitcoin.pdf Nakamoto, "Bitcoin: A Peer-to-Peer Electronic Cash System" (2008)]
* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/nakamoto2008-email.pdf Nakamoto, "Re: Bitcoin P2P e-cash paper" (2008)]
* [https://homeostasis.scs.carleton.ca/~soma/distos/2021f/papers/lemieux2013-satoshi.pdf Lemieux, "Who is Satoshi Nakamoto?" (Regulation, Fall 2013)]
 
===November 25, 2021===
 
No class, US Thanksgiving
 
===[[DistOS 2021F 2021-11-30|November 30, 2021]]===
 
Ethereum & NFTs
* [https://files.gitter.im/ethereum/yellowpaper/VIyt/Paper.pdf Wood, Ethereum: A secure decentralised generalised transaction ledger (2014)]
* [https://arxiv.org/pdf/2105.07447 Wang et al., "Non-Fungible Token (NFT): Overview, Evaluation, Opportunities and Challenges" (2021)]
 
===[[DistOS 2021F 2021-12-02|December 2, 2021]]===
 
Ethereum Security & The Future
* [https://dl.acm.org/doi/pdf/10.1145/3391195 Chen et al, "A survey on ethereum systems security: Vulnerabilities, attacks, and defenses" (CSUR 2020)]
* [https://homeostasis.scs.carleton.ca/~soma/pubs/burgess-nspw2018.pdf Burgess & Somayaji, "After the BlockCloud Apocalypse" (NSPW 2018)]
 
===December 7, 2021===
 
Project Presentations (Informal)
 
===[[DistOS 2021F 2021-12-09|December 9, 2021]]===
 
Class wrap up & Exam review
 
===December 19, 2021===
 
'''Final Exam''' (Online, 2-5 PM)

Latest revision as of 00:15, 10 December 2021

Course Outline

Here is the course outline.

Experiences

Using Openstack

The experiences (and potentially your project) will involve using the SCS Openstack cluster:

Classes

More information will be posted here by the first class.

Note: If you don't have access to the course information on brightspace, please email Prof. Somayaji, you may need to be manually added. (If you're at UofO or you recently added COMP 5102, you probably need to be added manually.) On Brightspace the course should show up as COMP 4000 no matter whether you enrolled in it or COMP 5102.

Project Help

To develop a literature review or research proposal, start with a single research paper that you find interesting and that is related to distributed operating systems in some way.

To begin selecting a paper, I suggest that you:

  • search on Google Scholar using keywords relating to your interests, and/or
  • browse the proceedings of major conferences that publish work related to distributed operating systems.

The main operating system conferences are OSDI and ACM SOSP (sosp.org,ACM DL). Note that not all the work here is on distributed operating systems! Also, many other conferences publish some work related to distributed operating systems, e.g. NSDI.

To help you write a literature review or the background of a research paper, read the following:

Implementation experiences will depend upon your technical background and personal interests. If you have an idea, please discuss with Anil or William as soon as you can.

Readings & Classes

September 9, 2021

Introduction

September 14, 2021

UNIX

Note that the video covers the main points of the paper.

September 16, 2021

LOCUS

September 21, 2021

NFS

September 23, 2021

Distributed Shared Memory

September 28, 2021

Plan 9

September 30, 2021

Solaris Zones

October 5, 2021

NASD & GFS

October 7, 2021

Chubby & ZooKeeper

October 12, 2021

BigTable & MapReduce

October 14, 2021

Omega & Borg

October 19, 2021

Midterm Review

October 21, 2021

Midterm

November 2, 2021

Cassandra & Dynamo

November 04, 2021

Haystack & F4

November 09, 2021

Proposal due by 11:59 PM

Spanner & Tensorflow

November 11, 2021

Ceph

November 16, 2021

OceanStore & BOINC

November 18, 2021

Tapestry & Delos

Background (optional but helpful):

November 23, 2021

Cryptocurrency origins

November 25, 2021

No class, US Thanksgiving

November 30, 2021

Ethereum & NFTs

December 2, 2021

Ethereum Security & The Future

December 7, 2021

Project Presentations (Informal)

December 9, 2021

Class wrap up & Exam review

December 19, 2021

Final Exam (Online, 2-5 PM)