Difference between revisions of "DistOS 2015W Session 1"

From Soma-notes
Jump to navigation Jump to search
(Created page with "1st Lecture - January 5th 2015 COURSE OUTLINE Undergrad Grading Scheme: 15% Class Participation 15% Reading Responses 10% lecture Notes/Wiki Contributions 25% Midterm ...")
 
(General clean-up (grammar, spelling, wiki-markup, etc))
Line 1: Line 1:
1st Lecture - January 5th 2015  
Notes for the first session that happened on Jan. 5th, 2015.




COURSE OUTLINE
= Course Outline =


Undergrad Grading Scheme:
Undergrad Grading Scheme
15% Class Participation  
* 15% Class Participation  
15% Reading Responses
* 15% Reading Responses
10% lecture Notes/Wiki Contributions  
* 10% lecture Notes/Wiki Contributions  
25% Midterm  
* 25% Midterm  
35% Final Exam  
* 35% Final Exam  


Grads Grading Scheme:
Grads Grading Scheme
15% Class Participation  
* 15% Class Participation  
15% Reading Responses
* 15% Reading Responses
10% Lectures Notes/Wiki contributions
* 10% Lectures Notes/Wiki contributions
10% Project Proposal  
* 10% Project Proposal  
15% Project Presentation  
* 15% Project Presentation  
35% Final Project  
* 35% Final Project  


Project:
=== Project ===
A literature review of Distributing operating
* A literature review of distributing operations
Research proposal on a problem related to DS
* Research proposal on a problem related to distributed systems




A large software system
= Discussion =


== Q: What do you think of when you hear 'Distributed System'? ==


Response: How you feel, What you Learned


=== Group 1 ===
* Sharing Resources
* Spreading Work Loads
* Scheduling
* Process migration
* Different nodes have different purpose
* Google
* Parallel running processes
* Nodes
* Resource Allocation across multiple nodes
* Scheduling multiple nodes
* Resources availability among nodes
* Problem: across multiple machine


LECTURES 1 : Discussion (What is Distributed System/OS)
=== Group 2 ===
Distributing System
* Request comes from (usually) one computer and processing is usually handled by more than one computer
- Sharing Resources
* Tasks are divided into small parts which can be processed individually before coming back together
- Spreading Work Loads
* Usually deals with large scale data sets
- Scheduling
* Globalization
- Process migration
* Fault tolerance
- Different nodes have different purpose
* Usually emphasize distributed agents to not work on certain schedule
- Google
- Parallel running processes
- Nodes 
- Resource Allocation across multiple nodes
- Scheduling multiple nodes
- Resources availability among nodes
- Problem: across multiple machine, 
- Questions:


'''Group2:'''
=== Group 3 ===
- Request come from one or more computer and handles by many other computers
* Separated, networked machines
- Tasks are divided into small, which are dealt with then re-integrate
* Coordinated by similar or identical software
- Usually deal with large scale datasets
* Error recovery/redundancy
- Globalization
* No controlled storage
- Fault tolerant
* Coordinated communication facilitating operation on coordinated task
- Usually emphasize distributed agents to not work on certain schedule
* Leader/hierarchy for task delegation
* Examples: Map Reduce, cloud, cloud software (Google Drive)


'''Group3''':
=== Group 4 ===
- Separated networked machine
* Distributed, multiple systems
- Coordinated with similar with similar identical software
* OS: The root level system that operates a computer system  
○ Error recovery, redundancy
* Dist. OS more complexity  
- No control storage
- Coordinated communication facilitating operation on coordinated task
- Leader/hierarchy for task delegation
○ Example: Map Reduce , cloud, cloud software(Google Drive)
'''Group4''':
- Distributed, multiple systems
- OS: The root level system that operates a computer system  
- Dist. OS more complexity  


'''Prof Discussion Notes:'''
=== Prof Discussion Notes ===
'''Key words:'''
Key words: network, parallel, fault tolerant, redundancy, complexity
Network
Parallel
Full tolerant  
Redundancy
Complexity
== Distributed OS is kind of OS ==
Distributing OS, is kind of OS
'''But what is an OS in the first place?'''
''' What is an OS?'''
* Connection software and hardware
○ Connection software and hardware
* Resource allocation
○ Recourses allocation  
* Abstraction layers
○ Abstraction  
* Makes it easy to run higher level programs on different computers
○ Transfer what you have and what you want to program on , Want to make it many computer you want to program on  
* Sharing: resources are split up and each process is isolated, this improves security and programming (no need to worry about sharing)
○ Sharing: Recourse spilt up , OS turn the computer you want to program , some other computer may want other part of resources
* Virtual memory, process scheduling
○ Virtual memory, Process scheduling  


      '''What is Distributed System?'''
'''What is a distributed system?'''
○ Run of one node but really on many nodes
* Feels like it is only a single node, but runs on many
○ Do the thing we take granted with regular computer
* Do the things users take for granted on regular computers
○ You want to program on computer - Cannot do many computer - lets you treat n computer as if its one numbers, expect you have 1000x resources
* Doesn't actually work as a single (general use) computer - with 1000 computers one would expect it to be like a single computer with 1000 times the resources, but it's not
○ Opposite of OS 
* In a way, opposite of an operating system: instead of splitting up a single computer into many, it takes many and tries to merge into one
○ Resources can be divided as you want  
* Resources can be divided as you want  
○ Implement centralized control - problem solution
* Centralized control solves some issues, but has issues of its own
○ You cannot take many and make it one -> We fake it , in certain context, in certain environment
* Since it cannot truly act as a single computer, we fake it as much as possible - and it works in some scenarios
○ More specific task you try to do , the more it scale  
* The more specialized the task, the better it will scale
○ Interact it, process update, callback,  
* Interact it, process update, callback,  
○ Gmail so responsible  - it perfected , downloads in your browser  
* Gmail so responsible  - it perfected , downloads in your browser  
○ Cache  - they predict what the processor going to do  
* Cache  - they predict what the processor going to do  


 
== Operating System Examples ==
'''Systems''':
'''Mobile devices - Phones'''
Mobile devices - Phones  
* iOS
IOS
* Android
Android
''' Embedded OS:'''  
'''Embedded OS'''  
Linux, QNX, xBSD firewalls  
* Linux
* QNX
* xBSD firewalls  
'''Desktop''':
'''Desktop'''
Windows  
* Windows  
MacOS
* OSX
Chrome OS
* Chrome OS
'''Server''':
'''Server'''
Windows, Server
* Windows
Linux
* Linux
BSD
* BSD
''' Main Frames:'''
'''Main Frames'''
OS/400
* OS/400
Cloud is n server? IMP Question
Is the cloud an OS? Important question.
'''Cloud''':
'''Cloud'''
MPI
* MPI
AWS
* AWS
Google App Engine  
* Google App Engine  
Is it OS?
No, they are at best proto-OSs because the abstractions they provide are very leaky. They only provide limited APIs.
No, it is a best porto OS , because of abstraction, you don’t know what in the background
 
 
'''Cloud''': quite OS, u do stuff in API
== Pick a system and show how it is and is not an OS ==
 
 
=== Group 1 ===
''' Group Discussion(2): BOINC - Pick a system  == Similar to OS or function like OS'''
{| class="wikitable" width="100%"
|+ '''BOINC'''
Similarity:
|-
○ Lot of parallelization
! Similarities to a traditional OS
○ Availability  
! Differences from a traditional OS
○ Scheduling  
|-
○  Network
| Very parallel
○ Connection leave and join whenever you want at idle time
| Can handle only very specific (trivially parallelizable) types of problems
○ Same problem at the same time  
|-
○ Redundancy
| Availability
○  Allocation can be handle by system  
| Abstraction layer is poor
○ Large problem managed by one system  
|-
○ Submit project 
| Scheduling
○ Fault tolerance
|-
Different OS:
| Networked
○ VERY VERY specific
|-
○ Studying at home
| Nodes may come and leave as they like (it is active when the computer is idle)
|-
Similarity BOINC:
| Same problem at the same time
○ Its networked
|-
○ Takes arbitrary, parallelized problems and scale then
| Redundant/Fault tolerant
○ Allocates problem re
|-
Differences:
| Allocation can be handle by system
○ Bad abstraction  
|-
○ Cant run centralized program
| Large problem managed by one system
|-
| Anyone can submit projects
|}
Group 2:
 
FaceBook
=== Group 2 ===
Pro:
{| class="wikitable" width="100%"
○ Abstracts over human connection (makes it easier )
|+ '''Facebook'''
○ Has a programming API/platform
|-
○ ? Able to separate resources such as wall posts, photos, etc
! Similarities to a traditional OS
Cons:
! Differences from a traditional OS
○ Doesn't really take a single resources and spilt it up into smaller ones
|-
○ APIs arents very stable
| OK abstraction (for human communication/control)
○ Not a lot of control
| Doesn't really take a single resources and split it up into smaller ones
|-
| The programming API is stable
| The human API (user interface) is not stable
|-
| Able to separate resources such as wall posts, photos, etc
| Limited control
|}
 
Group3:
=== Group 3 ===
Google Docs
{| class="wikitable" width="100%"
Pro:
|+ '''Google Docs (Drive)'''
○ File System(resources management )
|-
○ Hardware abstraction
! Similarities to a traditional OS
○ API google script
! Differences from a traditional OS
Cons:
|-
○ Is it really?
| Has a file system
| Somehow still not a true OS
|-
| Provides hardware abstraction (users don't care how the requests are carried out)
|-
| Has an API
|}
Group 4:
=== Group 4 ===
LinkedIn  
{| class="wikitable" width="100%"
Pro:
|+ '''LinkedIn'''
○ Supports multiple location/Users(Recourses)
|-
○ Multiple servers around the worls
! Similarities to a traditional OS
○ Recourses mgmt
! Differences from a traditional OS
○ Security Mgmt
|-
○ Networked System
| Supports multiple location/users
○ Consistency
| Specific functionality (not abstract)
○ Platform Free(Desktop/Mobile)
|-
○ Specific Functionality(Not abstract)
| Multiple servers around the world
|-
| Resource/Security management
|-
| Networked
|-
| Cross-platform (Desktop/Mobile)
|}

Revision as of 16:05, 11 January 2015

Notes for the first session that happened on Jan. 5th, 2015.


Course Outline

Undergrad Grading Scheme

  • 15% Class Participation
  • 15% Reading Responses
  • 10% lecture Notes/Wiki Contributions
  • 25% Midterm
  • 35% Final Exam

Grads Grading Scheme

  • 15% Class Participation
  • 15% Reading Responses
  • 10% Lectures Notes/Wiki contributions
  • 10% Project Proposal
  • 15% Project Presentation
  • 35% Final Project

Project

  • A literature review of distributing operations
  • Research proposal on a problem related to distributed systems


Discussion

Q: What do you think of when you hear 'Distributed System'?

Group 1

  • Sharing Resources
  • Spreading Work Loads
  • Scheduling
  • Process migration
  • Different nodes have different purpose
  • Google
  • Parallel running processes
  • Nodes
  • Resource Allocation across multiple nodes
  • Scheduling multiple nodes
  • Resources availability among nodes
  • Problem: across multiple machine

Group 2

  • Request comes from (usually) one computer and processing is usually handled by more than one computer
  • Tasks are divided into small parts which can be processed individually before coming back together
  • Usually deals with large scale data sets
  • Globalization
  • Fault tolerance
  • Usually emphasize distributed agents to not work on certain schedule

Group 3

  • Separated, networked machines
  • Coordinated by similar or identical software
  • Error recovery/redundancy
  • No controlled storage
  • Coordinated communication facilitating operation on coordinated task
  • Leader/hierarchy for task delegation
  • Examples: Map Reduce, cloud, cloud software (Google Drive)

Group 4

  • Distributed, multiple systems
  • OS: The root level system that operates a computer system
  • Dist. OS more complexity

Prof Discussion Notes

Key words: network, parallel, fault tolerant, redundancy, complexity


Distributed OS is kind of OS

But what is an OS in the first place?

  • Connection software and hardware
  • Resource allocation
  • Abstraction layers
  • Makes it easy to run higher level programs on different computers
  • Sharing: resources are split up and each process is isolated, this improves security and programming (no need to worry about sharing)
  • Virtual memory, process scheduling


What is a distributed system?

  • Feels like it is only a single node, but runs on many
  • Do the things users take for granted on regular computers
  • Doesn't actually work as a single (general use) computer - with 1000 computers one would expect it to be like a single computer with 1000 times the resources, but it's not
  • In a way, opposite of an operating system: instead of splitting up a single computer into many, it takes many and tries to merge into one
  • Resources can be divided as you want
  • Centralized control solves some issues, but has issues of its own
  • Since it cannot truly act as a single computer, we fake it as much as possible - and it works in some scenarios
  • The more specialized the task, the better it will scale
  • Interact it, process update, callback,
  • Gmail so responsible - it perfected , downloads in your browser
  • Cache - they predict what the processor going to do


Operating System Examples

Mobile devices - Phones

  • iOS
  • Android

Embedded OS

  • Linux
  • QNX
  • xBSD firewalls

Desktop

  • Windows
  • OSX
  • Chrome OS

Server

  • Windows
  • Linux
  • BSD

Main Frames

  • OS/400

Is the cloud an OS? Important question.

Cloud

  • MPI
  • AWS
  • Google App Engine

No, they are at best proto-OSs because the abstractions they provide are very leaky. They only provide limited APIs.


Pick a system and show how it is and is not an OS

Group 1

BOINC
Similarities to a traditional OS Differences from a traditional OS
Very parallel Can handle only very specific (trivially parallelizable) types of problems
Availability Abstraction layer is poor
Scheduling
Networked
Nodes may come and leave as they like (it is active when the computer is idle)
Same problem at the same time
Redundant/Fault tolerant
Allocation can be handle by system
Large problem managed by one system
Anyone can submit projects

Group 2

Facebook
Similarities to a traditional OS Differences from a traditional OS
OK abstraction (for human communication/control) Doesn't really take a single resources and split it up into smaller ones
The programming API is stable The human API (user interface) is not stable
Able to separate resources such as wall posts, photos, etc Limited control


Group 3

Google Docs (Drive)
Similarities to a traditional OS Differences from a traditional OS
Has a file system Somehow still not a true OS
Provides hardware abstraction (users don't care how the requests are carried out)
Has an API

Group 4

LinkedIn
Similarities to a traditional OS Differences from a traditional OS
Supports multiple location/users Specific functionality (not abstract)
Multiple servers around the world
Resource/Security management
Networked
Cross-platform (Desktop/Mobile)
Retrieved from "https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2015W_Session_1&oldid=19625"