Soma-notes - User contributions [en]

DistOS 2015W Session 10

2015-03-24T17:30:06Z

Dwils098:

Feel free to tweak the questions!

==Kademlia==
Members: Kirill, Deep, Jason, Hassan

* '''Why are DHTs relevant to distributed OSs?'''
** Using many system, having repetition
** DHT to distributed content over multiple nodes
** Decentralized therefore peer-to-peer

* '''How is content divided?'''
** File hashes
** Node ID to locate the value
** 160 bit key space, binary tree for partition and searching down the tree

* '''How is the network traversed?'''
** Highest prefix and increase the number of digit it used to get close to the target nodes

* '''What trust assumptions does the system make?'''
** DHT by itself is insecure
** The academic and practitioner communities have realized that all current DHT designs suffer from a security weakness, known as the Sybil attack
** K-buckets
*** Binary tree with each node having k-buckets as leaf
*** Any given k-node is very unlikely to fail within an hour of each other
*** New nodes are only inserted when there is room in the bucket or the oldest node doesn't respond
** Uses UDP therefore packets are often lost

* '''Performance constraints?'''
** Binary tree traversing, therefore traversing is maximum O(log n)

* '''What non-DHT internet infrastructure would you replace with a DHT? How suitable is Kademlia for this purpose?'''
** DNS
** Any kind of meta-data service

==Comet==
Members: Mohamed Ahmed, Apoorv Sangal, Ambalica Sharma

* Why are DHTs relevant to distributed OSs?
DHT is an infrastructure than enables many clients to share information, and scale to handle node arrival, departure and failure. DHT's serve many of the design goals of disbtributed operating systems. The paper states that "DHTs are increasingly used to support a variety of distributed applications, such as file-sharing, distributed resource tracking, end-system multicast, publish-subscribe systems, distributed search engines"

* How is content divided?
One of the three main components of the comet system is a routing substrate which
implements the value/node mapping. This allows a client to find the node htat stores
a specific data item. Since Comet uses a DHT implementation, routing occurs by applying
a hash function to the key to compute node ID's that store the associated value.

* How is the network traversed?

* What trust assumptions does the system make?
Assumes clients re untrusted autonomous nodes.

A client node running Comet should be protected from the execution of handlers
e.g. an executing handler cannot corrupt the node or use unlimited resources.
Handlers should not be able to mount messaging attacks on other nodes.

Users downloading Comet must trust it and have guarantees about its behavior. For this reason, Comet enforces four important restrictions:
1. Limited knowledge: an ASO is not aware of other objects
or resources stored on the same node and has no
direct way to learn about them.
2. Limited access: an object handler can manipulate only its own value and cannot modify the values of other objects on its storage node.
3. Limited communication: an active storage object cannot
send arbitrary messages over the network.
4. Limited resource consumption: an ASO’s resource usage is strictly bounded, e.g., the system limits the amount of computation and memory it can consume.

* Performance constraints?

* What non-DHT internet infrastructure would you replace with a DHT? How suitable is Comet for this purpose?

==Tapestry==
Members: Ashley, Dany, Alexis, Khaled

* Why are DHTs relevant to distributed OSs?

Because they provide a way to distribute information over large networks (distributed key/value store).

* How is content divided?

Uses consistent hashing (SHA-1), upon node creation (join) creates optimal routing table.

* How is the network traversed?

You look at your neighbours, you see which neighbour is closest to your destination, and recurse.

* What trust assumptions does the system make?

It assumes the system is entirely trustworthy from adversaries. While network failures may happen and nodes may go down, no node will explicitly try to mess with the network.

* Performance constraints?

O(log n) access times to any given node. Best effort publishing/unpublishing via decentralized object location routing.

* What non-DHT internet infrastructure would you replace with a DHT? How suitable is Tapestry for this purpose?

== Other Resources ==
*[http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1610546&tag=1 A survey and comparison of peer-to-peer overlay network schemes]
*[http://link.springer.com/article/10.1007/s12083-012-0157-3 Collaborative Applications over Peer-to-Peer Systems – Challenges and Solutions]

DistOS 2015W Session 10

2015-03-16T19:16:12Z

Dwils098: /* Tapestry */

Feel free to tweak the questions!

==Kademlia==
Members: Kirill, Deep, Jason, Hassan

* '''Why are DHTs relevant to distributed OSs?'''
** Using many system, having repetition
** DHT to distributed content over multiple nodes
** Decentralized therefore peer-to-peer

* '''How is content divided?'''
** File hashes
** Node ID to locate the value
** 160 bit key space, binary tree for partition and searching down the tree

* '''How is the network traversed?'''
** Highest prefix and increase the number of digit it used to get close to the target nodes

* '''What trust assumptions does the system make?'''
** DHT by itself is insecure
** The academic and practitioner communities have realized that all current DHT designs suffer from a security weakness, known as the Sybil attack
** K-buckets
*** Binary tree with each node having k-buckets as leaf
*** Any given k-node is very unlikely to fail within an hour of each other
*** New nodes are only inserted when there is room in the bucket or the oldest node doesn't respond
** Uses UDP therefore packets are often lost

* '''Performance constraints?'''
** Binary tree traversing, therefore traversing is maximum O(log n)

* '''What non-DHT internet infrastructure would you replace with a DHT? How suitable is Kademlia for this purpose?'''
** DNS
** Any kind of meta-data service

==Comet==
Members: Mohamed Ahmed, Apoorv Sangal, Ambalica Sharma

* Why are DHTs relevant to distributed OSs?
DHT is an infrastructure than enables many clients to share information, and scale to handle node arrival, departure and failure. DHT's serve many of the design goals of disbtributed operating systems. The paper states that "DHTs are increasingly used to support a variety of distributed applications, such as file-sharing, distributed resource tracking, end-system multicast, publish-subscribe systems, distributed search engines"

* How is content divided?
One of the three main components of the comet system is a routing substrate which
implements the value/node mapping. This allows a client to find the node htat stores
a specific data item. Since Comet uses a DHT implementation, routing occurs by applying
a hash function to the key to compute node ID's that store the associated value.

* How is the network traversed?

* What trust assumptions does the system make?
A client node running Comet should be protected from the execution of handlers
e.g. an executing handler cannot corrupt the node or use unlimited resources.
Handlers should not be able to mount messaging attacks on other nodes.

Users downloading Comet must trust it and have guarantees about its behavior. For this reason, Comet enforces four important restrictions:
1. Limited knowledge: an ASO is not aware of other objects
or resources stored on the same node and has no
direct way to learn about them.
2. Limited access: an object handler can manipulate only its own value and cannot modify the values of other objects on its storage node.
3. Limited communication: an active storage object cannot
send arbitrary messages over the network.
4. Limited resource consumption: an ASO’s resource usage is strictly bounded, e.g., the system limits the amount of computation and memory it can consume.

* Performance constraints?

* What non-DHT internet infrastructure would you replace with a DHT? How suitable is Kademlia for this purpose?

==Tapestry==
Members: Ashley, Dany, Alexis

* Why are DHTs relevant to distributed OSs?

Because they provide a way to distribute information over large networks (distributed key/value store).

* How is content divided?

Uses consistent hashing (SHA-1), upon node creation (join) creates optimal routing table.

* How is the network traversed?

You look at your neighbours, you see which neighbour is closest to your destination, and recurse.

* What trust assumptions does the system make?

It assumes the system is entirely trustworthy from adversaries. While network failures may happen and nodes may go down, no node will explicitly try to mess with the network.

* Performance constraints?

O(log n) access times to any given node. Best effort publishing/unpublishing via decentralized object location routing.

* What non-DHT internet infrastructure would you replace with a DHT? How suitable is Tapestry for this purpose?

DistOS 2015W Session 10

2015-03-16T19:11:09Z

Dwils098: /* Tapestry */

Feel free to tweak the questions!

==Kademlia==
Members: Kirill, Deep, Jason, Hassan

* Why are DHTs relevant to distributed OSs?

* How is content divided?

* How is the network traversed?

* What trust assumptions does the system make?

* Performance constraints?

* What non-DHT internet infrastructure would you replace with a DHT? How suitable is Kademlia for this purpose?

==Comet==
Members: Mohamed Ahmed, Apoorv Sangal, Ambalica Sharma

* Why are DHTs relevant to distributed OSs?
DHT is an infrastructure than enables many clients to share information, and scale to handle node arrival, departure and failure. DHT's serve many of the design goals of disbtributed operating systems. The paper states that "DHTs are increasingly used to support a variety of distributed applications, such as file-sharing, distributed resource tracking, end-system multicast, publish-subscribe systems, distributed search engines"

* How is content divided?

One of the three main components of the comet system is a routing substrate which
implements the value/node mapping. This allows a client to find the node htat stores
a specific data item. Since Comet uses a DHT implementation, routing occurs by applying
a hash function to the key to compute node ID's that store the associated value.

* How is the network traversed?

* What trust assumptions does the system make?

A client node running COmet should be protected from the execution of handlers
e.g. an executing handler cannot corrupt the node or use unlimited resources.
Handlers should not be able to mount messaging attacks on other nodes

* Performance constraints?

* What non-DHT internet infrastructure would you replace with a DHT? How suitable is Kademlia for this purpose?

==Tapestry==
Members:

* Why are DHTs relevant to distributed OSs?

Because they provide a way to distribute information over large networks (distributed key/value store).

* How is content divided?

Uses consistent hashing (SHA-1), upon node creation (join) creates optimal routing table.

* How is the network traversed?

You look at your neighbours, you see which neighbour is closest to your destination, and recurse.

* What trust assumptions does the system make?

It assumes the system is entirely trustworthy from adversaries. While network failures may happen and nodes may go down, no node will explicitly try to mess with the network.

* Performance constraints?

O(log n) access times to any given node. Best effort publishing/unpublishing via decentralized object location routing.

* What non-DHT internet infrastructure would you replace with a DHT? How suitable is Tapestry for this purpose?

DistOS 2015W Session 10

2015-03-16T18:59:50Z

Dwils098: /* Tapestry */

Feel free to tweak the questions!

==Kademlia==
Members:

* Why are DHTs relevant to distributed OSs?

* How is content divided?

* How is the network traversed?

* What trust assumptions does the system make?

* Performance constraints?

* What non-DHT internet infrastructure would you replace with a DHT? How suitable is Kademlia for this purpose?

==Comet==
Members: Mohamed Ahmed, Apoorv Sangal, Ambalica Sharma

* Why are DHTs relevant to distributed OSs?

* How is content divided?

* How is the network traversed?

* What trust assumptions does the system make?

* Performance constraints?

* What non-DHT internet infrastructure would you replace with a DHT? How suitable is Kademlia for this purpose?

==Tapestry==
Members:

* Why are DHTs relevant to distributed OSs?

Because they provide a way to distribute information over large networks (distributed key/value store).

* How is content divided?

* How is the network traversed?

* What trust assumptions does the system make?

* Performance constraints?

* What non-DHT internet infrastructure would you replace with a DHT? How suitable is Tapestry for this purpose?

DistOS 2015W Session 4

2015-01-29T16:25:16Z

Dwils098:

= Amebo Operating System =

=== Capablities: ===
* Pointer to the object
* Capability assigning right to perform to some operation to the object ticket
* Communicate wide area network
* a kind of ticket or key that allows the holder of the capa- bility to perform some (not neces- sarily all)
* Each user process owns some collection of capabilities, which together define the set of objects it may access and the types of operations that my ne performed on each
* After the server has performed the operation, it sends back a reply message that unblocks the client
* Sending messages, blocking and accepting forms the remote procedure call that can be encapsulate using to make entire remote operation look like local procedure
* Second field: used by the sever to identify which of its objects is being addressed server port and object number identify object which operation to performed
* Generates 48-bit random number
* The third field is the right field which contains a bit map telling which operation the holder of the capability may performed

=== Thread Management: ===
* Same process have multiple thread and each process has its own registered counter and stack
* Behave like process
* It can synchronized using mutex semaphore
* File: Multiple thread,
* Blocked when there's multiple threads
* Buttlet thread the mutex
* The careful reader may have noticed that user process can pull 813kbytes/sec

= The V Distributed System =

* First tent in V design: High Performance communication is the most critical facility for distributed systems.
* Second; The protocols, not the software, define the system.
* Third; a relatively small operating system kernel can implement the basic protocols and services providing a simple network-transparent process, address space & communication model.

=== Ideas that significantly affected the design ===
* Shared Memory.
* Dealing with group of entities same as they deal w/individual entities.
* Efficient file caching mechanism using the virtual memory caching mechanism.

=== Design Decisions ===
* Designed for a cluster of workstations with high speed network access ( only really supports LAN ).
* Abstract the physical architecture of the participating workstations, by defining common protocols providing well-defined interfaces.

DistOS 2015W Session 4

2015-01-29T16:22:31Z

Dwils098: /* The V Distributed System */

'''Amebo Operating System: Capablities'''
* Pointer to the object
* Capability assigning right to perform to some operation to the object ticket
* Communicate wide area network
* a kind of ticket or key that allows the holder of the capa- bility to perform some (not neces- sarily all)
* Each user process owns some collection of capabilities, which together define the set of objects it may access and the types of operations that my ne performed on each
* After the server has performed the operation, it sends back a reply message that unblocks the client
* Sending messages, blocking and accepting forms the remote procedure call that can be encapsulate using to make entire remote operation look like local procedure
* Second field: used by the sever to identify which of its objects is being addressed server port and object number identify object which operation to performed
* Generates 48-bit random number
* The third field is the right field which contains a bit map telling which operation the holder of the capability may performed

'''Thread Management:'''
• Same process have multiple thread and each process has its own registered counter and stack
• Behave like process
• It can synchronized using mutex semaphore
• File: Multiple thread,
• Blocked when there's multiple threads
• Buttlet thread the mutex
• The careful reader may have noticed that user process can pull 813kbytes/sec

= The V Distributed System =

* First tent in V design: High Performance communication is the most critical facility for distributed systems.
* Second; The protocols, not the software, define the system.
* Third; a relatively small operating system kernel can implement the basic protocols and services providing a simple network-transparent process, address space & communication model.

=== Ideas that significantly affected the design ===
* Shared Memory.
* Dealing with group of entities same as they deal w/individual entities.
* Efficient file caching mechanism using the virtual memory caching mechanism.

=== Design Decisions ===
* Designed for a cluster of workstations with high speed network access ( only really supports LAN ).
* Abstract the physical architecture of the participating workstations, by defining common protocols providing well-defined interfaces.

DistOS 2015W Session 1

2015-01-27T16:40:16Z

Dwils098: /* Operating System Examples */

Notes for the first session that happened on Jan. 5th, 2015.

= Course Outline =

Undergrad Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% lecture Notes/Wiki Contributions
* 25% Midterm
* 35% Final Exam

Grads Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% Lectures Notes/Wiki contributions
* 10% Project Proposal
* 15% Project Presentation
* 35% Final Project

=== Project ===
* A literature review of distributing operations
* Research proposal on a problem related to distributed systems

= Discussion =

== Q: What do you think of when you hear 'Distributed System'? ==

=== Group 1 ===
* Sharing Resources
* Spreading Work Loads
* Scheduling
* Process migration
* Different nodes have different purpose
* Google
* Parallel running processes
* Nodes
* Resource Allocation across multiple nodes
* Scheduling multiple nodes
* Resources availability among nodes
* Problem: across multiple machine

=== Group 2 ===
* Request comes from (usually) one computer and processing is usually handled by more than one computer
* Tasks are divided into small parts which can be processed individually before coming back together
* Usually deals with large scale data sets
* Globalization
* Fault tolerance
* Usually emphasize distributed agents to not work on certain schedule
* Examples: Map Reduce, cloud, cloud software (Google Drive)

=== Group 3 ===
* Separated, networked machines
* Coordinated by similar or identical software
* Error recovery/redundancy
* No controlled storage
* Coordinated communication facilitating operation on coordinated task
* Leader/hierarchy for task delegation

=== Group 4 ===
* Distributed, multiple systems
* OS: The root level system that operates a computer system
* Dist. OS more complexity

=== Prof Discussion Notes ===
Key words: network, parallel, fault tolerant, redundancy, complexity

== Distributed OS is kind of OS ==

'''But what is an OS in the first place?'''
* Connection software and hardware
* Resource allocation
* Abstraction layers
* Makes it easy to run higher level programs on different computers
* Sharing: resources are split up and each process is isolated, this improves security and programming (no need to worry about sharing)
* Virtual memory, process scheduling

'''What is a distributed system?'''
* Focused on "network transparency", in order to provide to the user the sense that he/she interacts with a single machine as if all the resources were local.
* Do the things users take for granted on regular computers
* Does not scale relatively to the number of resources in the system (i.e.: 100 computers, does not equal a distributed system that is 100 times faster than a single computer)
* In a way, opposite of an operating system: instead of splitting up a single computer into many, it takes many and tries to merge into one
* Resources can be divided as you want
* Centralized control solves some issues, but has issues of its own
* Since it cannot truly act as a single computer, we fake it as much as possible - and it works in some scenarios
* The more specialized the task, the better it will scale (i.e.: grid computing with very specific computations that are heavily parallel in nature)
* Interact it, process update, callback,
* Gmail so responsible - it perfected , downloads in your browser
* Cache - they predict what the processor going to do

[[File:Dos.png|frame|center|Distributed Operating System represented as a Venn diagram. OS denotes an operating system, and DS denotes a distributed system. The black center represents a distributed operating system; a system with properties of an operating system and a distributed system.]]

== Operating System Examples ==
'''Mobile devices - Phones'''
* iOS
* Android

'''Embedded OS'''
* Linux
* QNX
* xBSD firewalls

'''Desktop'''
* Windows
* OSX
* Chrome OS

'''Server'''
* Windows
* Linux
* BSD

'''Main Frames'''
* OS/400

Is the cloud an OS? Important question.

'''Cloud'''
* MPI
* AWS
* Google App Engine

No, they are at best proto-OSs because the abstractions they provide are very leaky. They only provide limited APIs.

[[File:C_ontology.png|thumb|center|400px|This showcases the possible services that clouds can provide, to an extent we can consider the whole as being an "OS".]]

--> I do not agree, it really depends on how expansive is your definition of cloud... (i.e.: IaaS provides the ability to the user to spin a full fledged OS on a VM (for which you have no explicit idea of the physical architecture, it could be part of a 96 SBC server-rack connected to dedicated HDD/SDD server-rack, etc.) ) Again this is semantics, but I feel we need to clarify the terminology.

== Pick a system and show how it is and is not an OS ==

=== Group 1 ===
{| class="wikitable" width="100%"
|+ '''BOINC'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Very parallel
| Can handle only very specific (trivially parallelizable) types of problems
|-
| Availability
| Abstraction layer is poor
|-
| Scheduling
|-
| Networked
|-
| Nodes may come and leave as they like (it is active when the computer is idle)
|-
| Same problem at the same time
|-
| Redundant/Fault tolerant
|-
| Allocation can be handle by system
|-
| Large problem managed by one system
|-
| Anyone can submit projects
|}

=== Group 2 ===
{| class="wikitable" width="100%"
|+ '''Facebook'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| OK abstraction (for human communication/control)
| Doesn't really take a single resources and split it up into smaller ones
|-
| The programming API is stable
| The human API (user interface) is not stable
|-
| Able to separate resources such as wall posts, photos, etc
| Limited control
|}

=== Group 3 ===
{| class="wikitable" width="100%"
|+ '''Google Docs (Drive)'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Has a file system
| Somehow still not a true OS
|-
| Provides hardware abstraction (users don't care how the requests are carried out)
|-
| Has an API
|}

=== Group 4 ===
{| class="wikitable" width="100%"
|+ '''LinkedIn'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Supports multiple location/users
| Specific functionality (not abstract)
|-
| Multiple servers around the world
|-
| Resource/Security management
|-
| Networked
|-
| Cross-platform (Desktop/Mobile)
|}

DistOS 2015W Session 1

2015-01-27T16:38:18Z

Dwils098: /* Operating System Examples */

Notes for the first session that happened on Jan. 5th, 2015.

= Course Outline =

Undergrad Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% lecture Notes/Wiki Contributions
* 25% Midterm
* 35% Final Exam

Grads Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% Lectures Notes/Wiki contributions
* 10% Project Proposal
* 15% Project Presentation
* 35% Final Project

=== Project ===
* A literature review of distributing operations
* Research proposal on a problem related to distributed systems

= Discussion =

== Q: What do you think of when you hear 'Distributed System'? ==

=== Group 1 ===
* Sharing Resources
* Spreading Work Loads
* Scheduling
* Process migration
* Different nodes have different purpose
* Google
* Parallel running processes
* Nodes
* Resource Allocation across multiple nodes
* Scheduling multiple nodes
* Resources availability among nodes
* Problem: across multiple machine

=== Group 2 ===
* Request comes from (usually) one computer and processing is usually handled by more than one computer
* Tasks are divided into small parts which can be processed individually before coming back together
* Usually deals with large scale data sets
* Globalization
* Fault tolerance
* Usually emphasize distributed agents to not work on certain schedule
* Examples: Map Reduce, cloud, cloud software (Google Drive)

=== Group 3 ===
* Separated, networked machines
* Coordinated by similar or identical software
* Error recovery/redundancy
* No controlled storage
* Coordinated communication facilitating operation on coordinated task
* Leader/hierarchy for task delegation

=== Group 4 ===
* Distributed, multiple systems
* OS: The root level system that operates a computer system
* Dist. OS more complexity

=== Prof Discussion Notes ===
Key words: network, parallel, fault tolerant, redundancy, complexity

== Distributed OS is kind of OS ==

'''But what is an OS in the first place?'''
* Connection software and hardware
* Resource allocation
* Abstraction layers
* Makes it easy to run higher level programs on different computers
* Sharing: resources are split up and each process is isolated, this improves security and programming (no need to worry about sharing)
* Virtual memory, process scheduling

'''What is a distributed system?'''
* Focused on "network transparency", in order to provide to the user the sense that he/she interacts with a single machine as if all the resources were local.
* Do the things users take for granted on regular computers
* Does not scale relatively to the number of resources in the system (i.e.: 100 computers, does not equal a distributed system that is 100 times faster than a single computer)
* In a way, opposite of an operating system: instead of splitting up a single computer into many, it takes many and tries to merge into one
* Resources can be divided as you want
* Centralized control solves some issues, but has issues of its own
* Since it cannot truly act as a single computer, we fake it as much as possible - and it works in some scenarios
* The more specialized the task, the better it will scale (i.e.: grid computing with very specific computations that are heavily parallel in nature)
* Interact it, process update, callback,
* Gmail so responsible - it perfected , downloads in your browser
* Cache - they predict what the processor going to do

[[File:Dos.png|frame|center|Distributed Operating System represented as a Venn diagram. OS denotes an operating system, and DS denotes a distributed system. The black center represents a distributed operating system; a system with properties of an operating system and a distributed system.]]

== Operating System Examples ==
'''Mobile devices - Phones'''
* iOS
* Android

'''Embedded OS'''
* Linux
* QNX
* xBSD firewalls

'''Desktop'''
* Windows
* OSX
* Chrome OS

'''Server'''
* Windows
* Linux
* BSD

'''Main Frames'''
* OS/400

Is the cloud an OS? Important question.

'''Cloud'''
* MPI
* AWS
* Google App Engine

No, they are at best proto-OSs because the abstractions they provide are very leaky. They only provide limited APIs.

[[File:C_ontology.png|frame|center|This showcases the possible services that clouds can provide, to an extent we can consider the whole as being an "OS".]]

--> I do not agree, it really depends on how expansive is your definition of cloud... (i.e.: IaaS provides the ability to the user to spin a full fledged OS on a VM (for which you have no explicit idea of the physical architecture, it could be part of a 96 SBC server-rack connected to dedicated HDD/SDD server-rack, etc.) ) Again this is semantics, but I feel we need to clarify the terminology.

== Pick a system and show how it is and is not an OS ==

=== Group 1 ===
{| class="wikitable" width="100%"
|+ '''BOINC'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Very parallel
| Can handle only very specific (trivially parallelizable) types of problems
|-
| Availability
| Abstraction layer is poor
|-
| Scheduling
|-
| Networked
|-
| Nodes may come and leave as they like (it is active when the computer is idle)
|-
| Same problem at the same time
|-
| Redundant/Fault tolerant
|-
| Allocation can be handle by system
|-
| Large problem managed by one system
|-
| Anyone can submit projects
|}

=== Group 2 ===
{| class="wikitable" width="100%"
|+ '''Facebook'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| OK abstraction (for human communication/control)
| Doesn't really take a single resources and split it up into smaller ones
|-
| The programming API is stable
| The human API (user interface) is not stable
|-
| Able to separate resources such as wall posts, photos, etc
| Limited control
|}

=== Group 3 ===
{| class="wikitable" width="100%"
|+ '''Google Docs (Drive)'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Has a file system
| Somehow still not a true OS
|-
| Provides hardware abstraction (users don't care how the requests are carried out)
|-
| Has an API
|}

=== Group 4 ===
{| class="wikitable" width="100%"
|+ '''LinkedIn'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Supports multiple location/users
| Specific functionality (not abstract)
|-
| Multiple servers around the world
|-
| Resource/Security management
|-
| Networked
|-
| Cross-platform (Desktop/Mobile)
|}

DistOS 2015W Session 1

2015-01-27T16:36:31Z

Dwils098: /* Operating System Examples */

Notes for the first session that happened on Jan. 5th, 2015.

= Course Outline =

Undergrad Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% lecture Notes/Wiki Contributions
* 25% Midterm
* 35% Final Exam

Grads Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% Lectures Notes/Wiki contributions
* 10% Project Proposal
* 15% Project Presentation
* 35% Final Project

=== Project ===
* A literature review of distributing operations
* Research proposal on a problem related to distributed systems

= Discussion =

== Q: What do you think of when you hear 'Distributed System'? ==

=== Group 1 ===
* Sharing Resources
* Spreading Work Loads
* Scheduling
* Process migration
* Different nodes have different purpose
* Google
* Parallel running processes
* Nodes
* Resource Allocation across multiple nodes
* Scheduling multiple nodes
* Resources availability among nodes
* Problem: across multiple machine

=== Group 2 ===
* Request comes from (usually) one computer and processing is usually handled by more than one computer
* Tasks are divided into small parts which can be processed individually before coming back together
* Usually deals with large scale data sets
* Globalization
* Fault tolerance
* Usually emphasize distributed agents to not work on certain schedule
* Examples: Map Reduce, cloud, cloud software (Google Drive)

=== Group 3 ===
* Separated, networked machines
* Coordinated by similar or identical software
* Error recovery/redundancy
* No controlled storage
* Coordinated communication facilitating operation on coordinated task
* Leader/hierarchy for task delegation

=== Group 4 ===
* Distributed, multiple systems
* OS: The root level system that operates a computer system
* Dist. OS more complexity

=== Prof Discussion Notes ===
Key words: network, parallel, fault tolerant, redundancy, complexity

== Distributed OS is kind of OS ==

'''But what is an OS in the first place?'''
* Connection software and hardware
* Resource allocation
* Abstraction layers
* Makes it easy to run higher level programs on different computers
* Sharing: resources are split up and each process is isolated, this improves security and programming (no need to worry about sharing)
* Virtual memory, process scheduling

'''What is a distributed system?'''
* Focused on "network transparency", in order to provide to the user the sense that he/she interacts with a single machine as if all the resources were local.
* Do the things users take for granted on regular computers
* Does not scale relatively to the number of resources in the system (i.e.: 100 computers, does not equal a distributed system that is 100 times faster than a single computer)
* In a way, opposite of an operating system: instead of splitting up a single computer into many, it takes many and tries to merge into one
* Resources can be divided as you want
* Centralized control solves some issues, but has issues of its own
* Since it cannot truly act as a single computer, we fake it as much as possible - and it works in some scenarios
* The more specialized the task, the better it will scale (i.e.: grid computing with very specific computations that are heavily parallel in nature)
* Interact it, process update, callback,
* Gmail so responsible - it perfected , downloads in your browser
* Cache - they predict what the processor going to do

[[File:Dos.png|frame|center|Distributed Operating System represented as a Venn diagram. OS denotes an operating system, and DS denotes a distributed system. The black center represents a distributed operating system; a system with properties of an operating system and a distributed system.]]

== Operating System Examples ==
'''Mobile devices - Phones'''
* iOS
* Android

'''Embedded OS'''
* Linux
* QNX
* xBSD firewalls

'''Desktop'''
* Windows
* OSX
* Chrome OS

'''Server'''
* Windows
* Linux
* BSD

'''Main Frames'''
* OS/400

Is the cloud an OS? Important question.

'''Cloud'''
* MPI
* AWS
* Google App Engine

No, they are at best proto-OSs because the abstractions they provide are very leaky. They only provide limited APIs.

[[File:C_ontology.png|center]]

--> I do not agree, it really depends on how expansive is your definition of cloud... (i.e.: IaaS provides the ability to the user to spin a full fledged OS on a VM (for which you have no explicit idea of the physical architecture, it could be part of a 96 SBC server-rack connected to dedicated HDD/SDD server-rack, etc.) ) Again this is semantics, but I feel we need to clarify the terminology.

== Pick a system and show how it is and is not an OS ==

=== Group 1 ===
{| class="wikitable" width="100%"
|+ '''BOINC'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Very parallel
| Can handle only very specific (trivially parallelizable) types of problems
|-
| Availability
| Abstraction layer is poor
|-
| Scheduling
|-
| Networked
|-
| Nodes may come and leave as they like (it is active when the computer is idle)
|-
| Same problem at the same time
|-
| Redundant/Fault tolerant
|-
| Allocation can be handle by system
|-
| Large problem managed by one system
|-
| Anyone can submit projects
|}

=== Group 2 ===
{| class="wikitable" width="100%"
|+ '''Facebook'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| OK abstraction (for human communication/control)
| Doesn't really take a single resources and split it up into smaller ones
|-
| The programming API is stable
| The human API (user interface) is not stable
|-
| Able to separate resources such as wall posts, photos, etc
| Limited control
|}

=== Group 3 ===
{| class="wikitable" width="100%"
|+ '''Google Docs (Drive)'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Has a file system
| Somehow still not a true OS
|-
| Provides hardware abstraction (users don't care how the requests are carried out)
|-
| Has an API
|}

=== Group 4 ===
{| class="wikitable" width="100%"
|+ '''LinkedIn'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Supports multiple location/users
| Specific functionality (not abstract)
|-
| Multiple servers around the world
|-
| Resource/Security management
|-
| Networked
|-
| Cross-platform (Desktop/Mobile)
|}

File:C ontology.png

2015-01-27T16:35:29Z

Dwils098:

DistOS 2015W Session 1

2015-01-27T16:30:56Z

Dwils098: /* Operating System Examples */

Notes for the first session that happened on Jan. 5th, 2015.

= Course Outline =

Undergrad Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% lecture Notes/Wiki Contributions
* 25% Midterm
* 35% Final Exam

Grads Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% Lectures Notes/Wiki contributions
* 10% Project Proposal
* 15% Project Presentation
* 35% Final Project

=== Project ===
* A literature review of distributing operations
* Research proposal on a problem related to distributed systems

= Discussion =

== Q: What do you think of when you hear 'Distributed System'? ==

=== Group 1 ===
* Sharing Resources
* Spreading Work Loads
* Scheduling
* Process migration
* Different nodes have different purpose
* Google
* Parallel running processes
* Nodes
* Resource Allocation across multiple nodes
* Scheduling multiple nodes
* Resources availability among nodes
* Problem: across multiple machine

=== Group 2 ===
* Request comes from (usually) one computer and processing is usually handled by more than one computer
* Tasks are divided into small parts which can be processed individually before coming back together
* Usually deals with large scale data sets
* Globalization
* Fault tolerance
* Usually emphasize distributed agents to not work on certain schedule
* Examples: Map Reduce, cloud, cloud software (Google Drive)

=== Group 3 ===
* Separated, networked machines
* Coordinated by similar or identical software
* Error recovery/redundancy
* No controlled storage
* Coordinated communication facilitating operation on coordinated task
* Leader/hierarchy for task delegation

=== Group 4 ===
* Distributed, multiple systems
* OS: The root level system that operates a computer system
* Dist. OS more complexity

=== Prof Discussion Notes ===
Key words: network, parallel, fault tolerant, redundancy, complexity

== Distributed OS is kind of OS ==

'''But what is an OS in the first place?'''
* Connection software and hardware
* Resource allocation
* Abstraction layers
* Makes it easy to run higher level programs on different computers
* Sharing: resources are split up and each process is isolated, this improves security and programming (no need to worry about sharing)
* Virtual memory, process scheduling

'''What is a distributed system?'''
* Focused on "network transparency", in order to provide to the user the sense that he/she interacts with a single machine as if all the resources were local.
* Do the things users take for granted on regular computers
* Does not scale relatively to the number of resources in the system (i.e.: 100 computers, does not equal a distributed system that is 100 times faster than a single computer)
* In a way, opposite of an operating system: instead of splitting up a single computer into many, it takes many and tries to merge into one
* Resources can be divided as you want
* Centralized control solves some issues, but has issues of its own
* Since it cannot truly act as a single computer, we fake it as much as possible - and it works in some scenarios
* The more specialized the task, the better it will scale (i.e.: grid computing with very specific computations that are heavily parallel in nature)
* Interact it, process update, callback,
* Gmail so responsible - it perfected , downloads in your browser
* Cache - they predict what the processor going to do

[[File:Dos.png|frame|center|Distributed Operating System represented as a Venn diagram. OS denotes an operating system, and DS denotes a distributed system. The black center represents a distributed operating system; a system with properties of an operating system and a distributed system.]]

== Operating System Examples ==
'''Mobile devices - Phones'''
* iOS
* Android

'''Embedded OS'''
* Linux
* QNX
* xBSD firewalls

'''Desktop'''
* Windows
* OSX
* Chrome OS

'''Server'''
* Windows
* Linux
* BSD

'''Main Frames'''
* OS/400

Is the cloud an OS? Important question.

'''Cloud'''
* MPI
* AWS
* Google App Engine

No, they are at best proto-OSs because the abstractions they provide are very leaky. They only provide limited APIs.

--> I do not agree, it really depends on how expansive is your definition of cloud... (i.e.: IaaS provides the ability to the user to spin a full fledged OS on a VM (for which you have no explicit idea of the physical architecture, it could be part of a 96 SBC server-rack connected to dedicated HDD/SDD server-rack, etc.) ) Again this is semantics, but I feel we need to clarify the terminology.

== Pick a system and show how it is and is not an OS ==

=== Group 1 ===
{| class="wikitable" width="100%"
|+ '''BOINC'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Very parallel
| Can handle only very specific (trivially parallelizable) types of problems
|-
| Availability
| Abstraction layer is poor
|-
| Scheduling
|-
| Networked
|-
| Nodes may come and leave as they like (it is active when the computer is idle)
|-
| Same problem at the same time
|-
| Redundant/Fault tolerant
|-
| Allocation can be handle by system
|-
| Large problem managed by one system
|-
| Anyone can submit projects
|}

=== Group 2 ===
{| class="wikitable" width="100%"
|+ '''Facebook'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| OK abstraction (for human communication/control)
| Doesn't really take a single resources and split it up into smaller ones
|-
| The programming API is stable
| The human API (user interface) is not stable
|-
| Able to separate resources such as wall posts, photos, etc
| Limited control
|}

=== Group 3 ===
{| class="wikitable" width="100%"
|+ '''Google Docs (Drive)'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Has a file system
| Somehow still not a true OS
|-
| Provides hardware abstraction (users don't care how the requests are carried out)
|-
| Has an API
|}

=== Group 4 ===
{| class="wikitable" width="100%"
|+ '''LinkedIn'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Supports multiple location/users
| Specific functionality (not abstract)
|-
| Multiple servers around the world
|-
| Resource/Security management
|-
| Networked
|-
| Cross-platform (Desktop/Mobile)
|}

DistOS 2015W Session 1

2015-01-27T16:24:21Z

Dwils098: /* Distributed OS is kind of OS */

Notes for the first session that happened on Jan. 5th, 2015.

= Course Outline =

Undergrad Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% lecture Notes/Wiki Contributions
* 25% Midterm
* 35% Final Exam

Grads Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% Lectures Notes/Wiki contributions
* 10% Project Proposal
* 15% Project Presentation
* 35% Final Project

=== Project ===
* A literature review of distributing operations
* Research proposal on a problem related to distributed systems

= Discussion =

== Q: What do you think of when you hear 'Distributed System'? ==

=== Group 1 ===
* Sharing Resources
* Spreading Work Loads
* Scheduling
* Process migration
* Different nodes have different purpose
* Google
* Parallel running processes
* Nodes
* Resource Allocation across multiple nodes
* Scheduling multiple nodes
* Resources availability among nodes
* Problem: across multiple machine

=== Group 2 ===
* Request comes from (usually) one computer and processing is usually handled by more than one computer
* Tasks are divided into small parts which can be processed individually before coming back together
* Usually deals with large scale data sets
* Globalization
* Fault tolerance
* Usually emphasize distributed agents to not work on certain schedule
* Examples: Map Reduce, cloud, cloud software (Google Drive)

=== Group 3 ===
* Separated, networked machines
* Coordinated by similar or identical software
* Error recovery/redundancy
* No controlled storage
* Coordinated communication facilitating operation on coordinated task
* Leader/hierarchy for task delegation

=== Group 4 ===
* Distributed, multiple systems
* OS: The root level system that operates a computer system
* Dist. OS more complexity

=== Prof Discussion Notes ===
Key words: network, parallel, fault tolerant, redundancy, complexity

== Distributed OS is kind of OS ==

'''But what is an OS in the first place?'''
* Connection software and hardware
* Resource allocation
* Abstraction layers
* Makes it easy to run higher level programs on different computers
* Sharing: resources are split up and each process is isolated, this improves security and programming (no need to worry about sharing)
* Virtual memory, process scheduling

'''What is a distributed system?'''
* Focused on "network transparency", in order to provide to the user the sense that he/she interacts with a single machine as if all the resources were local.
* Do the things users take for granted on regular computers
* Does not scale relatively to the number of resources in the system (i.e.: 100 computers, does not equal a distributed system that is 100 times faster than a single computer)
* In a way, opposite of an operating system: instead of splitting up a single computer into many, it takes many and tries to merge into one
* Resources can be divided as you want
* Centralized control solves some issues, but has issues of its own
* Since it cannot truly act as a single computer, we fake it as much as possible - and it works in some scenarios
* The more specialized the task, the better it will scale (i.e.: grid computing with very specific computations that are heavily parallel in nature)
* Interact it, process update, callback,
* Gmail so responsible - it perfected , downloads in your browser
* Cache - they predict what the processor going to do

[[File:Dos.png|frame|center|Distributed Operating System represented as a Venn diagram. OS denotes an operating system, and DS denotes a distributed system. The black center represents a distributed operating system; a system with properties of an operating system and a distributed system.]]

== Operating System Examples ==
'''Mobile devices - Phones'''
* iOS
* Android

'''Embedded OS'''
* Linux
* QNX
* xBSD firewalls

'''Desktop'''
* Windows
* OSX
* Chrome OS

'''Server'''
* Windows
* Linux
* BSD

'''Main Frames'''
* OS/400

Is the cloud an OS? Important question.

'''Cloud'''
* MPI
* AWS
* Google App Engine

No, they are at best proto-OSs because the abstractions they provide are very leaky. They only provide limited APIs.

== Pick a system and show how it is and is not an OS ==

=== Group 1 ===
{| class="wikitable" width="100%"
|+ '''BOINC'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Very parallel
| Can handle only very specific (trivially parallelizable) types of problems
|-
| Availability
| Abstraction layer is poor
|-
| Scheduling
|-
| Networked
|-
| Nodes may come and leave as they like (it is active when the computer is idle)
|-
| Same problem at the same time
|-
| Redundant/Fault tolerant
|-
| Allocation can be handle by system
|-
| Large problem managed by one system
|-
| Anyone can submit projects
|}

=== Group 2 ===
{| class="wikitable" width="100%"
|+ '''Facebook'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| OK abstraction (for human communication/control)
| Doesn't really take a single resources and split it up into smaller ones
|-
| The programming API is stable
| The human API (user interface) is not stable
|-
| Able to separate resources such as wall posts, photos, etc
| Limited control
|}

=== Group 3 ===
{| class="wikitable" width="100%"
|+ '''Google Docs (Drive)'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Has a file system
| Somehow still not a true OS
|-
| Provides hardware abstraction (users don't care how the requests are carried out)
|-
| Has an API
|}

=== Group 4 ===
{| class="wikitable" width="100%"
|+ '''LinkedIn'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Supports multiple location/users
| Specific functionality (not abstract)
|-
| Multiple servers around the world
|-
| Resource/Security management
|-
| Networked
|-
| Cross-platform (Desktop/Mobile)
|}

DistOS 2015W Session 1

2015-01-27T16:23:49Z

Dwils098: /* Distributed OS is kind of OS */

Notes for the first session that happened on Jan. 5th, 2015.

= Course Outline =

Undergrad Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% lecture Notes/Wiki Contributions
* 25% Midterm
* 35% Final Exam

Grads Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% Lectures Notes/Wiki contributions
* 10% Project Proposal
* 15% Project Presentation
* 35% Final Project

=== Project ===
* A literature review of distributing operations
* Research proposal on a problem related to distributed systems

= Discussion =

== Q: What do you think of when you hear 'Distributed System'? ==

=== Group 1 ===
* Sharing Resources
* Spreading Work Loads
* Scheduling
* Process migration
* Different nodes have different purpose
* Google
* Parallel running processes
* Nodes
* Resource Allocation across multiple nodes
* Scheduling multiple nodes
* Resources availability among nodes
* Problem: across multiple machine

=== Group 2 ===
* Request comes from (usually) one computer and processing is usually handled by more than one computer
* Tasks are divided into small parts which can be processed individually before coming back together
* Usually deals with large scale data sets
* Globalization
* Fault tolerance
* Usually emphasize distributed agents to not work on certain schedule
* Examples: Map Reduce, cloud, cloud software (Google Drive)

=== Group 3 ===
* Separated, networked machines
* Coordinated by similar or identical software
* Error recovery/redundancy
* No controlled storage
* Coordinated communication facilitating operation on coordinated task
* Leader/hierarchy for task delegation

=== Group 4 ===
* Distributed, multiple systems
* OS: The root level system that operates a computer system
* Dist. OS more complexity

=== Prof Discussion Notes ===
Key words: network, parallel, fault tolerant, redundancy, complexity

== Distributed OS is kind of OS ==

'''But what is an OS in the first place?'''
* Connection software and hardware
* Resource allocation
* Abstraction layers
* Makes it easy to run higher level programs on different computers
* Sharing: resources are split up and each process is isolated, this improves security and programming (no need to worry about sharing)
* Virtual memory, process scheduling

'''What is a distributed system?'''
* Focused on "network transparency", in order to provide to the user the sense that he/she interacts with a single machine as if all the resources were local.
* Do the things users take for granted on regular computers
* Does not scale relatively to the number of resources in the system (i.e.: 100 computers, does not equal a distributed system that is 100 times faster than a single computer)
* In a way, opposite of an operating system: instead of splitting up a single computer into many, it takes many and tries to merge into one
* Resources can be divided as you want
* Centralized control solves some issues, but has issues of its own
* Since it cannot truly act as a single computer, we fake it as much as possible - and it works in some scenarios
* The more specialized the task, the better it will scale (i.e.: grid computing with very specific computation that are heavily parallel in nature)
* Interact it, process update, callback,
* Gmail so responsible - it perfected , downloads in your browser
* Cache - they predict what the processor going to do

[[File:Dos.png|frame|center|Distributed Operating System represented as a Venn diagram. OS denotes an operating system, and DS denotes a distributed system. The black center represents a distributed operating system; a system with properties of an operating system and a distributed system.]]

== Operating System Examples ==
'''Mobile devices - Phones'''
* iOS
* Android

'''Embedded OS'''
* Linux
* QNX
* xBSD firewalls

'''Desktop'''
* Windows
* OSX
* Chrome OS

'''Server'''
* Windows
* Linux
* BSD

'''Main Frames'''
* OS/400

Is the cloud an OS? Important question.

'''Cloud'''
* MPI
* AWS
* Google App Engine

No, they are at best proto-OSs because the abstractions they provide are very leaky. They only provide limited APIs.

== Pick a system and show how it is and is not an OS ==

=== Group 1 ===
{| class="wikitable" width="100%"
|+ '''BOINC'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Very parallel
| Can handle only very specific (trivially parallelizable) types of problems
|-
| Availability
| Abstraction layer is poor
|-
| Scheduling
|-
| Networked
|-
| Nodes may come and leave as they like (it is active when the computer is idle)
|-
| Same problem at the same time
|-
| Redundant/Fault tolerant
|-
| Allocation can be handle by system
|-
| Large problem managed by one system
|-
| Anyone can submit projects
|}

=== Group 2 ===
{| class="wikitable" width="100%"
|+ '''Facebook'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| OK abstraction (for human communication/control)
| Doesn't really take a single resources and split it up into smaller ones
|-
| The programming API is stable
| The human API (user interface) is not stable
|-
| Able to separate resources such as wall posts, photos, etc
| Limited control
|}

=== Group 3 ===
{| class="wikitable" width="100%"
|+ '''Google Docs (Drive)'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Has a file system
| Somehow still not a true OS
|-
| Provides hardware abstraction (users don't care how the requests are carried out)
|-
| Has an API
|}

=== Group 4 ===
{| class="wikitable" width="100%"
|+ '''LinkedIn'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Supports multiple location/users
| Specific functionality (not abstract)
|-
| Multiple servers around the world
|-
| Resource/Security management
|-
| Networked
|-
| Cross-platform (Desktop/Mobile)
|}