Soma: Created page with "==Readings== SCOPE & Yarn * [https://homeostasis.scs.carleton.ca/~soma/distos/2019f/chaiken2008-scope.pdf Chaiken et al., "SCOPE: Easy and Efficient Parallel Processing of Ma..."

2020-03-20T02:16:04Z

Created page with "==Readings== SCOPE & Yarn * [https://homeostasis.scs.carleton.ca/~soma/distos/2019f/chaiken2008-scope.pdf Chaiken et al., "SCOPE: Easy and Efficient Parallel Processing of Ma..."

New page

==Readings==

SCOPE & Yarn
* [https://homeostasis.scs.carleton.ca/~soma/distos/2019f/chaiken2008-scope.pdf Chaiken et al., "SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets" (PVLDB 2008)]
* [https://homeostasis.scs.carleton.ca/~soma/distos/2019f/vavilapalli2013-yarn.pdf Vavilapalli et al., "Apache Hadoop YARN: Yet Another Resource Negotiator" (SoCC 2013)]
* Optional: [https://homeostasis.scs.carleton.ca/~soma/distos/2019f/isard2007-dryad.pdf Isard et al., "Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks" (Eurosys 2007)]

==Discussion Questions==

* Both papers start off with a criticism of MapReduce but then go off in very different directions. What are those directions?
* How close is SCOPE to SQL? Why didn't they just use SQL?
* What is Cosmos? How is it related to SCOPE? Dryad?
* How similar is Cosmos to other systems we have discussed?
* What were the requirements for YARN, and how were they achieved?
* Why do you think Microsoft didn't commercialize SCOPE or Cosmos? See [https://social.msdn.microsoft.com/Forums/en-US/5aba7d2f-2e82-4b10-a2f9-e5fea2647076/difference-between-cosmos-and-scope-vs-data-lake-store-and-usql this question] comparing Cosmos/SCOPE to Data Lake Store and U-SQL, which apparently are built on YARN and WebHDFS.

DistOS 2019F 2019-11-27 - Revision history

Soma: Created page with "==Readings== SCOPE & Yarn * [https://homeostasis.scs.carleton.ca/~soma/distos/2019f/chaiken2008-scope.pdf Chaiken et al., "SCOPE: Easy and Efficient Parallel Processing of Ma..."