DistOS 2014W Lecture 21

Presentation

Marking

marked mostly on presentation, not content
basically we want to communicate the basic structure of the paper, and do so in a way that isn't boring

Test

Content

concrete, not "head in the clouds"
present the area
compare and contrast the papers
10 minutes talk, 5 minutes feedback
basic argument
basic references

Form

show the work we've done on paper
try to get feedback
think of it as a rough draft
try to get people to read the paper
enthusiasm
powerpoints are easier
don't read slides
no whole sentences on slides
look at talks by Mark Shuttleworth

MapReduce

A clever observation that a simple solution could solve most distributed problems. It's all about programming to an abstraction that is efficiently parallelizable. Note that it's not actually a simple solution, because it sits atop a mountain of code. It requires something like BigTables which requires something like GFS, which requires something like Chubby.

Simplification
Interestingly large scale problems can be implemented with this
Easy to program, powerful for certain classes, it scales like no ones business.
Kind of empoweraged model

Programming to an abstraction that is efficiently parllel. We have learnt all about infrastructure until now. Classic OS abstractions were about files.

Naiad

Where MapReduce was suited for a specific family of solutions, Naiad tries to generalize the solution to apply parallelization to a much wider family. Naiad supports MapReduce style solutions, but also many other solutions. However, the tradeoff was simplicity. It's like we took MapReduce and took away its low barrier to entry. The idea is to create a constrained graph that can easily be parallelized.

Complicated than Map Reduce
Talks about Timely dataflow graphs
Its all about Graph algorithms - Graph abstraction
Restrictions on graphs so that they can be mapped to parllel computation
How to fit anything to this model is a big question.
More general than map reduce
Not very useful.