DistOS 2015W Session 1: Difference between revisions

From Soma-notes
Created page with "1st Lecture - January 5th 2015 COURSE OUTLINE Undergrad Grading Scheme: 15% Class Participation 15% Reading Responses 10% lecture Notes/Wiki Contributions 25% Midterm ..."
 
Dwils098 (talk | contribs)
13 edits
 
(9 intermediate revisions by 3 users not shown)
Line 1: Line 1:
1st Lecture - January 5th 2015  
Notes for the first session that happened on Jan. 5th, 2015.




COURSE OUTLINE
= Course Outline =


Undergrad Grading Scheme:
Undergrad Grading Scheme
15% Class Participation  
* 15% Class Participation  
15% Reading Responses
* 15% Reading Responses
10% lecture Notes/Wiki Contributions  
* 10% lecture Notes/Wiki Contributions  
25% Midterm  
* 25% Midterm  
35% Final Exam  
* 35% Final Exam  


Grads Grading Scheme:
Grads Grading Scheme
15% Class Participation  
* 15% Class Participation  
15% Reading Responses
* 15% Reading Responses
10% Lectures Notes/Wiki contributions
* 10% Lectures Notes/Wiki contributions
10% Project Proposal  
* 10% Project Proposal  
15% Project Presentation  
* 15% Project Presentation  
35% Final Project  
* 35% Final Project  


Project:
=== Project ===
A literature review of Distributing operating
* A literature review of distributing operations
Research proposal on a problem related to DS
* Research proposal on a problem related to distributed systems




A large software system
= Discussion =


== Q: What do you think of when you hear 'Distributed System'? ==


Response: How you feel, What you Learned


=== Group 1 ===
* Sharing Resources
* Spreading Work Loads
* Scheduling
* Process migration
* Different nodes have different purpose
* Google
* Parallel running processes
* Nodes
* Resource Allocation across multiple nodes
* Scheduling multiple nodes
* Resources availability among nodes
* Problem: across multiple machine


LECTURES 1 : Discussion (What is Distributed System/OS)
=== Group 2 ===
Distributing System
* Request comes from (usually) one computer and processing is usually handled by more than one computer
- Sharing Resources
* Tasks are divided into small parts which can be processed individually before coming back together
- Spreading Work Loads
* Usually deals with large scale data sets
- Scheduling
* Globalization
- Process migration
* Fault tolerance
- Different nodes have different purpose
* Usually emphasize distributed agents to not work on certain schedule
- Google
* Examples: Map Reduce, cloud, cloud software (Google Drive)
- Parallel running processes
- Nodes 
- Resource Allocation across multiple nodes
- Scheduling multiple nodes
- Resources availability among nodes
- Problem: across multiple machine,
- Questions:


'''Group2:'''
=== Group 3 ===
- Request come from one or more computer and handles by many other computers
* Separated, networked machines
- Tasks are divided into small, which are dealt with then re-integrate
* Coordinated by similar or identical software
- Usually deal with large scale datasets
* Error recovery/redundancy
- Globalization
* No controlled storage
- Fault tolerant
* Coordinated communication facilitating operation on coordinated task
- Usually emphasize distributed agents to not work on certain schedule
* Leader/hierarchy for task delegation


'''Group3''':
=== Group 4 ===
- Separated networked machine
* Distributed, multiple systems
- Coordinated with similar with similar identical software
* OS: The root level system that operates a computer system  
○ Error recovery, redundancy
* Dist. OS more complexity  
- No control storage
- Coordinated communication facilitating operation on coordinated task
- Leader/hierarchy for task delegation
○ Example: Map Reduce , cloud, cloud software(Google Drive)
'''Group4''':
- Distributed, multiple systems
- OS: The root level system that operates a computer system  
- Dist. OS more complexity  


'''Prof Discussion Notes:'''
=== Prof Discussion Notes ===
'''Key words:'''
Key words: network, parallel, fault tolerant, redundancy, complexity
Network
Parallel
Full tolerant  
Redundancy
Complexity
== Distributed OS is kind of OS ==
Distributing OS, is kind of OS
'''But what is an OS in the first place?'''
''' What is an OS?'''
* Connection software and hardware
○ Connection software and hardware
* Resource allocation
○ Recourses allocation  
* Abstraction layers
○ Abstraction  
* Makes it easy to run higher level programs on different computers
○ Transfer what you have and what you want to program on , Want to make it many computer you want to program on  
* Sharing: resources are split up and each process is isolated, this improves security and programming (no need to worry about sharing)
○ Sharing: Recourse spilt up , OS turn the computer you want to program , some other computer may want other part of resources
* Virtual memory, process scheduling
○ Virtual memory, Process scheduling  


      '''What is Distributed System?'''
'''What is a distributed system?'''
○ Run of one node but really on many nodes
* Focused on "network transparency", in order to provide to the user the sense that he/she interacts with a single machine as if all the resources were local.
○ Do the thing we take granted with regular computer  
* Do the things users take for granted on regular computers
○ You want to program on computer - Cannot do many computer - lets you treat n computer as if its one numbers, expect you have 1000x resources
* Does not scale relatively to the number of resources in the system (i.e.: 100 computers, does not equal a distributed system that is 100 times faster than a single computer)
○ Opposite of OS 
* In a way, opposite of an operating system: instead of splitting up a single computer into many, it takes many and tries to merge into one
○ Resources can be divided as you want  
* Resources can be divided as you want  
○ Implement centralized control - problem solution
* Centralized control solves some issues, but has issues of its own
○ You cannot take many and make it one -> We fake it , in certain context, in certain environment
* Since it cannot truly act as a single computer, we fake it as much as possible - and it works in some scenarios
○ More specific task you try to do , the more it scale  
* The more specialized the task, the better it will scale (i.e.: grid computing with very specific computations that are heavily parallel in nature)
○ Interact it, process update, callback,  
* Interact it, process update, callback,  
○ Gmail so responsible  - it perfected , downloads in your browser  
* Gmail so responsible  - it perfected , downloads in your browser  
○ Cache  - they predict what the processor going to do  
* Cache  - they predict what the processor going to do


[[File:Dos.png|frame|center|Distributed Operating System represented as a Venn diagram. OS denotes an operating system, and DS denotes a distributed system. The black center represents a distributed operating system; a system with properties of an operating system and a distributed system.]]


'''Systems''':
== Operating System Examples ==
Mobile devices - Phones  
'''Mobile devices - Phones'''
IOS
* iOS
Android
* Android
''' Embedded OS:'''  
'''Embedded OS'''  
Linux, QNX, xBSD firewalls  
* Linux
* QNX
* xBSD firewalls  
'''Desktop''':
'''Desktop'''
Windows  
* Windows  
MacOS
* OSX
Chrome OS
* Chrome OS
'''Server''':
'''Server'''
Windows, Server
* Windows
Linux
* Linux
BSD
* BSD
''' Main Frames:'''
'''Main Frames'''
OS/400
* OS/400
Cloud is n server? IMP Question
Is the cloud an OS? Important question.
'''Cloud''':
'''Cloud'''
MPI
* MPI
AWS
* AWS
Google App Engine  
* Google App Engine  
Is it OS?
No, they are at best proto-OSs because the abstractions they provide are very leaky. They only provide limited APIs.
No, it is a best porto OS , because of abstraction, you don’t know what in the background
 
[[File:C_ontology.png|thumb|center|400px|This showcases the possible services that clouds can provide, to an extent we can consider the whole as being an "OS".]]
'''Cloud''': quite OS, u do stuff in API
 
--> I do not agree, it really depends on how expansive is your definition of cloud... (i.e.:  IaaS provides the ability to the user to spin a full fledged OS on a VM (for which you have no explicit idea of the physical architecture, it could be part of a 96 SBC server-rack connected to dedicated HDD/SDD server-rack, etc.) ) Again this is semantics, but I feel we need to clarify the terminology.
 
== Pick a system and show how it is and is not an OS ==
''' Group Discussion(2): BOINC - Pick a system  == Similar to OS or function like OS'''
 
 
Similarity:
=== Group 1 ===
○ Lot of parallelization
{| class="wikitable" width="100%"
○ Availability  
|+ '''BOINC'''
○ Scheduling  
|-
○  Network
! Similarities to a traditional OS
○ Connection leave and join whenever you want at idle time
! Differences from a traditional OS
○ Same problem at the same time  
|-
○ Redundancy
| Very parallel
○  Allocation can be handle by system  
| Can handle only very specific (trivially parallelizable) types of problems
○ Large problem managed by one system  
|-
○ Submit project 
| Availability
○ Fault tolerance
| Abstraction layer is poor
Different OS:
|-
○ VERY VERY specific
| Scheduling
○ Studying at home
|-
| Networked
Similarity BOINC:
|-
○ Its networked
| Nodes may come and leave as they like (it is active when the computer is idle)
○ Takes arbitrary, parallelized problems and scale then
|-
○ Allocates problem re
| Same problem at the same time
Differences:
|-
○ Bad abstraction  
| Redundant/Fault tolerant
○ Cant run centralized program
|-
| Allocation can be handle by system
|-
| Large problem managed by one system
Group 2:
|-
FaceBook
| Anyone can submit projects
Pro:
|}
○ Abstracts over human connection (makes it easier )
 
○ Has a programming API/platform
=== Group 2 ===
○ ? Able to separate resources such as wall posts, photos, etc
{| class="wikitable" width="100%"
Cons:
|+ '''Facebook'''
○ Doesn't really take a single resources and spilt it up into smaller ones
|-
○ APIs arents very stable
! Similarities to a traditional OS
○ Not a lot of control
! Differences from a traditional OS
|-
| OK abstraction (for human communication/control)
| Doesn't really take a single resources and split it up into smaller ones
|-
| The programming API is stable
| The human API (user interface) is not stable
|-
| Able to separate resources such as wall posts, photos, etc
| Limited control
|}
 
Group3:
=== Group 3 ===
Google Docs
{| class="wikitable" width="100%"
Pro:
|+ '''Google Docs (Drive)'''
○ File System(resources management )
|-
○ Hardware abstraction
! Similarities to a traditional OS
○ API google script
! Differences from a traditional OS
Cons:
|-
○ Is it really?
| Has a file system
| Somehow still not a true OS
|-
| Provides hardware abstraction (users don't care how the requests are carried out)
|-
| Has an API
|}
Group 4:
=== Group 4 ===
LinkedIn  
{| class="wikitable" width="100%"
Pro:
|+ '''LinkedIn'''
○ Supports multiple location/Users(Recourses)
|-
○ Multiple servers around the worls
! Similarities to a traditional OS
○ Recourses mgmt
! Differences from a traditional OS
○ Security Mgmt
|-
○ Networked System
| Supports multiple location/users
○ Consistency
| Specific functionality (not abstract)
○ Platform Free(Desktop/Mobile)
|-
○ Specific Functionality(Not abstract)
| Multiple servers around the world
|-
| Resource/Security management
|-
| Networked
|-
| Cross-platform (Desktop/Mobile)
|}

Latest revision as of 16:40, 27 January 2015

Notes for the first session that happened on Jan. 5th, 2015.


Course Outline

Undergrad Grading Scheme

  • 15% Class Participation
  • 15% Reading Responses
  • 10% lecture Notes/Wiki Contributions
  • 25% Midterm
  • 35% Final Exam

Grads Grading Scheme

  • 15% Class Participation
  • 15% Reading Responses
  • 10% Lectures Notes/Wiki contributions
  • 10% Project Proposal
  • 15% Project Presentation
  • 35% Final Project

Project

  • A literature review of distributing operations
  • Research proposal on a problem related to distributed systems


Discussion

Q: What do you think of when you hear 'Distributed System'?

Group 1

  • Sharing Resources
  • Spreading Work Loads
  • Scheduling
  • Process migration
  • Different nodes have different purpose
  • Google
  • Parallel running processes
  • Nodes
  • Resource Allocation across multiple nodes
  • Scheduling multiple nodes
  • Resources availability among nodes
  • Problem: across multiple machine

Group 2

  • Request comes from (usually) one computer and processing is usually handled by more than one computer
  • Tasks are divided into small parts which can be processed individually before coming back together
  • Usually deals with large scale data sets
  • Globalization
  • Fault tolerance
  • Usually emphasize distributed agents to not work on certain schedule
  • Examples: Map Reduce, cloud, cloud software (Google Drive)

Group 3

  • Separated, networked machines
  • Coordinated by similar or identical software
  • Error recovery/redundancy
  • No controlled storage
  • Coordinated communication facilitating operation on coordinated task
  • Leader/hierarchy for task delegation

Group 4

  • Distributed, multiple systems
  • OS: The root level system that operates a computer system
  • Dist. OS more complexity

Prof Discussion Notes

Key words: network, parallel, fault tolerant, redundancy, complexity


Distributed OS is kind of OS

But what is an OS in the first place?

  • Connection software and hardware
  • Resource allocation
  • Abstraction layers
  • Makes it easy to run higher level programs on different computers
  • Sharing: resources are split up and each process is isolated, this improves security and programming (no need to worry about sharing)
  • Virtual memory, process scheduling


What is a distributed system?

  • Focused on "network transparency", in order to provide to the user the sense that he/she interacts with a single machine as if all the resources were local.
  • Do the things users take for granted on regular computers
  • Does not scale relatively to the number of resources in the system (i.e.: 100 computers, does not equal a distributed system that is 100 times faster than a single computer)
  • In a way, opposite of an operating system: instead of splitting up a single computer into many, it takes many and tries to merge into one
  • Resources can be divided as you want
  • Centralized control solves some issues, but has issues of its own
  • Since it cannot truly act as a single computer, we fake it as much as possible - and it works in some scenarios
  • The more specialized the task, the better it will scale (i.e.: grid computing with very specific computations that are heavily parallel in nature)
  • Interact it, process update, callback,
  • Gmail so responsible - it perfected , downloads in your browser
  • Cache - they predict what the processor going to do
Distributed Operating System represented as a Venn diagram. OS denotes an operating system, and DS denotes a distributed system. The black center represents a distributed operating system; a system with properties of an operating system and a distributed system.

Operating System Examples

Mobile devices - Phones

  • iOS
  • Android

Embedded OS

  • Linux
  • QNX
  • xBSD firewalls

Desktop

  • Windows
  • OSX
  • Chrome OS

Server

  • Windows
  • Linux
  • BSD

Main Frames

  • OS/400

Is the cloud an OS? Important question.

Cloud

  • MPI
  • AWS
  • Google App Engine

No, they are at best proto-OSs because the abstractions they provide are very leaky. They only provide limited APIs.

This showcases the possible services that clouds can provide, to an extent we can consider the whole as being an "OS".

--> I do not agree, it really depends on how expansive is your definition of cloud... (i.e.: IaaS provides the ability to the user to spin a full fledged OS on a VM (for which you have no explicit idea of the physical architecture, it could be part of a 96 SBC server-rack connected to dedicated HDD/SDD server-rack, etc.) ) Again this is semantics, but I feel we need to clarify the terminology.

Pick a system and show how it is and is not an OS

Group 1

BOINC
Similarities to a traditional OS Differences from a traditional OS
Very parallel Can handle only very specific (trivially parallelizable) types of problems
Availability Abstraction layer is poor
Scheduling
Networked
Nodes may come and leave as they like (it is active when the computer is idle)
Same problem at the same time
Redundant/Fault tolerant
Allocation can be handle by system
Large problem managed by one system
Anyone can submit projects

Group 2

Facebook
Similarities to a traditional OS Differences from a traditional OS
OK abstraction (for human communication/control) Doesn't really take a single resources and split it up into smaller ones
The programming API is stable The human API (user interface) is not stable
Able to separate resources such as wall posts, photos, etc Limited control


Group 3

Google Docs (Drive)
Similarities to a traditional OS Differences from a traditional OS
Has a file system Somehow still not a true OS
Provides hardware abstraction (users don't care how the requests are carried out)
Has an API

Group 4

LinkedIn
Similarities to a traditional OS Differences from a traditional OS
Supports multiple location/users Specific functionality (not abstract)
Multiple servers around the world
Resource/Security management
Networked
Cross-platform (Desktop/Mobile)
Retrieved from "https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2015W_Session_1&oldid=19728"