Soma-notes - User contributions [en]

DistOS 2015W Session 3

2015-03-15T17:53:49Z

Ericadamski: /* Locus */

= Multics =

'''Team:''' Sameer, Shivjot, Ambalica, Veena

It came into being in the 1960s and it completely vanished in 2000s. It was started by Bell, General Electric and MIT but Bell backed out of the project in 1969.
Multics is a time sharing OS which provides multitasking and multiprogramming. It is not a distributed OS but it a centralized system which was written in the assembly language.

It provides following features:
# Utility Computing
# Access Control Lists
# Single level storage
# Dynamic linking
#* Sharded libraries or files can be loaded and linked to Random Access Memory at run time
# Hot swapping
# Multiprocessing System
# Ring oriented Security
#* It provides number of levels of authorization within the computer system
#* Still present in some form today, inside both processors (like x86) and operating systems

= Unix =

Unix in its original conception is a small, minimal API system designed by two guys from Bell Labs. It was essentially a OS that would be easy to grasp for an programmer but not much beyond that. The UNIX OS ran on one computer, and terminals ran from that one computer. Thus it is not a distributed operating system as it is centralized and implements time sharing. In fact, it didn't even have support for networking in the first version.

The C language was created specifically for Unix, as the creators wanted to create a machine-agnostic language for the operating system.

Most features from Unix are still available in present day Unix-based systems. For example, the shell, with its piping capabilities, is still used today in its original form.

= Sun NFS =

The Sun NFS OS implemented networking using RPC connections. These connections are not secure. Sun wanted to encrypt these RPC connections but encryption would result in government regulations that Sun wanted to avoid in order to sell NFS over seas.

<blockquote>
Sun wanted to secure their NFS with encryption, but at the time encryption was regulated like munitions in the United States. Exporting any product that had encryption was impossible, but Sun needed those sales abroad. To avoid these regulations, Sun decided to sell the insecure NFS version of the system.
</blockquote>

'''Team''': Mert

= Locus =

# Not scalable
#* The synchronization algorithms were so slow, they only managed to run it on five computers
#* Every computer stores a copy of every file
#* Also used CAS to manage files
# Not efficient with abstractions
#* Trying to distribute files and processes
#Allowed for process migration
#Transparency
#* It provided network transparency to “disguise” its distributed context.
#Dynamic reconfiguration. (It adapts topology changes.)

Locus has lots of similarities with the today's systems. It uses replication and partitioning which are employed in cloud and distributed systems.

'''Team''': Eric

= Sprite =

''' Team''': Jamie, Hassan, Khaled

Sprite had the following Design Features:
# Network Transparency
# Process Migration, file transfer between computers
#* User could initiate a process migration to an idle machine, and if the machine was no longer idle; due to it being used by another user, the system would take care of the process migration to another machine
# Handling Cache Consistency
#* Sequential file sharing ==> By using a version number for each file
#* Concurrent write sharing ==> Disable cache to clients, enable write-blocking and other methods
# Implemented a caching system that sped up performance
# Implemented a log structured file system
#* They realized that with increasing amounts of RAM in computers which can be used for caching, writes to the disk were the main bottleneck, not reads.
#* Log structured file-systems are optimized for writes, as changes to previous data are appended at the current position.
#* This allows for very fast, sequential writes.
#* Example: SSD (Solid-state disks)

The main features to take away from the Sprite system is that it implemented a log structured file system, and implemented caching to increase performance.

DistOS 2015W Session 9

2015-03-10T00:45:55Z

Ericadamski: /* Naiad */

== BONIC ==

*Public Resource Computing Platform
*Gives scientists the ability to use large amounts of computation resources.
*The clients do not connect directly with each other but instead they talk to a central server located at Berkley
*The goals of Bonic are:
:*1) reduce the barriers of entry
:*2) Share resources among autonomous projects
:*3) Support diverse applications
:*4) Reward participants.
A BONIC application can be identified by a single master URL, which serves as the homepage as well as the directory of the servers.

== SETI@Home ==

*Uses public resource computing to analyze radio signals to find extraterrestrial intelligence
*Need good quality telescope to search for radio signals, and lots of computational power, which was unavailable locally
*It has not yet found extraterrestrial intelligence, but its has established credibility of public resource computing projects which are given by the public
*Uses BONIC as a backbone for the project
*Uses relational database to store information on a large scale, further it uses a multi-threaded server to distribute work to clients

== MapReduce ==

*A programming model presented by Google to do large scale parallel computations
*Uses the <code>Map()</code> and <code>Reduce()</code> functions from functional style programming languages
:*Map (Filtering)
::*Takes a function and applies it to all elements of the given data set
:*Reduce (Summary)
::*Accumulates results from the data set using a given function

== Naiad ==

*A programming model similar to <code>MapReduce</code> but with streaming capabilities so that data results are almost instantaneous
*A distributed system for executing data parallel cyclic dataflow programs offering high throughput and low latency
*Aims to provide a general purpose system which will fulfill the requirements and the will also support wide variety of high level programming models.
*Real Time Applications:
:*Batch iterative Machine Learning:
VW, an open source distributed machine learning performs iteration in 3 phases: each process updates local state; processes independently training on local data; and process jointly performed global average which is All Reduce.
:*Streaming Acyclic Computation
When compared to a system called [http://research.microsoft.com/apps/pubs/default.aspx?id=163832 Kineograph] ( also done by Microsoft ), which processes twitter handles and provides counts of the occurrence of hashtags as well as links between popular tags, was written using Naiad in 26 lines of code and ran close to 2X faster.

DistOS 2015W Session 9

2015-03-10T00:36:59Z

Ericadamski: /* Naiad */

DistOS 2015W Session 9

2015-03-10T00:34:55Z

Ericadamski: /* MapReduce */

DistOS 2015W Session 9

2015-03-10T00:32:57Z

Ericadamski: /* MapReduce */

DistOS 2015W Session 9

2015-03-10T00:31:11Z

Ericadamski:

DistOS 2015W Session 9

2015-03-10T00:28:23Z

Ericadamski:

DistOS 2015W Session 9

2015-03-10T00:25:26Z

Ericadamski:

'''BONIC'''
*Public Resource Computing Platform
*Gives scientists the ability to use large amounts of computation resources.
*The clients do not connect directly with each other but instead they talk to a central server located at Berkley
*The goals of Bonic are:
:*1) reduce the barriers of entry
:*2) Share resources among autonomous projects
:*3) Support diverse applications
:*4) Reward participants.
A BONIC application can be identified by a single master URL, which serves as the homepage as well as the directory of the servers.

'''SETI@Home'''
*Uses public resource computing to analyze radio signals to find extraterrestrial intelligence
*Need good quality telescope to search for radio signals, and lots of computational power, which was unavailable locally
*It has not yet found extraterrestrial intelligence, but its has established credibility of public resource computing projects which are given by the public
*Uses BONIC as a backbone for the project
*Uses relational database to store information on a large scale, further it uses a multi-threaded server to distribute work to clients

DistOS 2015W Session 6

2015-02-09T21:54:54Z

Ericadamski: /* Group 2 */

==Midterm==

The midterm from last year [http://homeostasis.scs.carleton.ca/~soma/distos/2015w/comp4000-2014w-midterm.pdf is now available].

==Group 1==

'''Team:''' Kirill, Jamie, Alexis, Veena, Khaled, Hassan

{| class="wikitable"
|-
!
! FARSITE
! OceanStore
|-
! Fault Tolerance
| Used Byzantine Fault Tolerance Algorithm - Did not manage well
| Used Byzantine Fault Tolerance Algorithm - Did not manage well
|-
! Cryptography
| Trusted Certificates
| A strong cryptographic algorithm on read-only operations
|-
! Implementation
| Did not mention what programming they used, but it was based on Windows. They did not implement the file system
| Implemented in JAVA
|-
! Scalability
| Scalable to a University or large corporations, maximum 105
| Worldwide scalability, maximum 1010
|-
! File Usage
| Was designed for general purpose files
| Was designed for small file sizes
|-
! Scope
| All clients sharing the available resources
| Transient centralized service
|-
! Object Model
| Didn't use the object model
| Used the object model
|}

==Group 2==
Team Members: Apoorv, Ambalica, Ashley, Eric, Mert, Shivjot

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0" class="wikitable">
<tr>
<td>Farsite</td>
<td>OceanStore</td>
</tr>
<tr>
<td>Implemented Content Leases</td>
<td>Update Model handled data consistency, no Leases</td>
</tr>
<tr>
<td>Single tier, p2p model</td>
<td>Two tier, server client model</td>
</tr>
<tr>
<td>Scope of ten to the five</td>
<td>Global Scope (ten to the ten)</td>
</tr>
<tr>
<td>Cryptographic public, private key security</td>
<td>Read and write privileges</td>
</tr>
<tr>
<td>Randomized data replication</td>
<td>Nomadic Data concept</td>
</tr>
</table>

==Group 3==
DANY, MOE, DEEP, SAMEER, TROY

'''FARSITE'''

'''Security'''
• Cascading certificates system through directory hierarchy
• Keys
• Three types of certificates
• CFS required to authorized certificate
• Because directory groups only modify their shared state via a Byzantine-fault-tolerant protocol, we trust the group not to make
an incorrect update to directory metadata. This metadata includes an access control list (ACL) of public keys of all users
who are authorized writers to that directory and to files in it
• Both file content and user-sensitive metadata (meaning file and directory names) are encrypted for privacy.

'''System Architecture'''
• Client Monitor, directory group, file host
• When space runs out in directory group, delegate’s ownership to sub tree to other delegate group.

'''OCEANSTORE'''

'''Security'''
• GUID and ACLs used for write, encryption used for reads.
• To prevent unauthorized reads, it encrypts
all data in the system that is not completely public and distributes the encryption key to those users with read permission