DistOS 2014W Lecture 24

The Landscape of Parallel Computing Research: A View from Berkeley

What sort of applications can you expect to run on distributed OS/parallize?
How do you scale up
We can't rely on processor improvements to provide speed-ups
The proposed computational models that need more processor power don't really apply to regular
Users would see the advances with games primarily
More reliance in cloud computing in recent years
In general you can model modern programs (Java, C, etc.) as finite state machines which are not parallizable
Today we deal with processor limitations by using "experts" to build the system which results in a very specialized solution usually in the cloud
Authors have found the problem but not really the process

7 Dwarfs

Dense Linear Algebra (hard to parallelize)
Sparse Linear Algebra
Spectral Methods
N-Body Methods
Structured Grids
Unstructured Grids
Monte Carlo (parallelizable)

Extended Dwarfs

Combinational Logic
Graph Traversal
Dynamic Programming
Backtrack/Branch + Bound
Construct Graphical Models
Finite State Machines (hardest to parallelize)

Features

Pretty impressive on getting everyone to sign off on the report
Connection to MapReduce
Programs that run on distributed operating systems - applications that can be expected to be massively parallel - what sort of computational model is needed - Abstractions needed on top of the stack.
Predictions about the processing power
GPU's do have 1000 or more cores
Desktop cores have not gotten that fast over the past years. They just don't run fast enough.
Games are the only things that can't be run over the time on single thread
Low power
Being able to run a smart phone with 100's of transistors - stalled with the sequential processing
Why do we need the additional processing power for ? - Games - Games - Games
Doomsday of the IT industry
Massive change in mobile and cloud over the past five years
Linux a very general operating system when it started at first - hard coded to 8086 processor specialized to run on the box. Now it runs everywhere. It has all the abstractions dealing with various aspects of hardware, architecture. Multiple layers abstraction because it was useful.

Summary

Standard programming languages can be modeled as finite state machines (their execution models are finite state machines). It is advisable to concede defeat in the area of parallelizing finite state machines and focus on parallelizing other areas. We need to find multiple system architectures that can deal with each of these problems. How do we find abstractions to deal with these things?

Generalizations are rarely generalized in the right way. Generalization has to be made at some level, but at the same time, you do not want to make generalizations without a clear justification. Otherwise, you’re likely to generalize it wrong.

Class Feedback for the Course

Challenging questions given in advance to lead and direct the group discussions
Having required questions on the Wiki
Share and highlight any good reading responses with the rest of the class
Having essay assignments replace essay format of midterm
Incorporate more information about the history of computer science