Soma-notes - User contributions [en]

DistOS 2015W Session 4

2015-02-28T17:01:23Z

Aimbot: /* Supported architectures */ Changed HTML to wiki markup

= Andrew File System =
AFS (Andrew File System) was set up as a direct response to NFS. Essentially universities found issues when they tried to scale NFS in a way that would allow them to share files amongst their staff effectively. AFS was more scalable than NFS because read-write operations happened locally before they were committed to the server (data store).

Since AFS copied files locally when they were opened and only sent the data back when they were closed, all operations during that time are very fast and do need the network. NFS works with files remotely, so there is no data to transfer when opening/closing the file, making those operations instant.

There are several problems with this design, however.
* The local system must have enough space to temporarily store the file.
* Opening and closing the files requires a lot of bandwidth for large files. To read even a single byte, the entire file must be retrieved (later versions remedied this).
* If the close operation fails, the system will not have the updated version of the file. Many programs don't even check the return value of the close operation, giving users the false impression that everything went well.

Given all this, AFS was suitable for working with small files, not large ones, limiting its usefulness. It is also notoriously annoying to set up as it is geared towards university-sized networks, further limiting its success.

= Amoeba Operating System =

=== Capablities: ===
* Pointer to the object
* Capability assigning right to perform to some operation to the object ticket
* Communicate wide area network
* a kind of ticket or key that allows the holder of the capa- bility to perform some (not neces- sarily all)
* Each user process owns some collection of capabilities, which together define the set of objects it may access and the types of operations that my ne performed on each
* After the server has performed the operation, it sends back a reply message that unblocks the client
* Sending messages, blocking and accepting forms the remote procedure call that can be encapsulate using to make entire remote operation look like local procedure
* Second field: used by the sever to identify which of its objects is being addressed server port and object number identify object which operation to performed
* Generates 48-bit random number
* The third field is the right field which contains a bit map telling which operation the holder of the capability may performed
* X11 Window management

=== Thread Management: ===
* Same process have multiple thread and each process has its own registered counter and stack
* Behave like process
* It can synchronized using mutex semaphore
* File: Multiple thread,
* Blocked when there's multiple threads
* Buttlet thread the mutex
* The careful reader may have noticed that user process can pull 813kbytes/sec

= Unique features =

== Pool processors ==
Pool processors are group of CPUs that are dynamically allocated as user needs. When a program is executed, any of the available processor run.

== Supported architectures ==
Many different processor architectures are supported including:
* i80386 (Pentium)
* 68K
* SPARC

= The V Distributed System =

* First tent in V design: High Performance communication is the most critical facility for distributed systems.
* Second; The protocols, not the software, define the system.
* Third; a relatively small operating system kernel can implement the basic protocols and services providing a simple network-transparent process, address space & communication model.

=== Ideas that significantly affected the design ===
* Shared Memory.
* Dealing with group of entities same as they deal w/individual entities.
* Efficient file caching mechanism using the virtual memory caching mechanism.

=== Design Decisions ===
* Designed for a cluster of workstations with high speed network access ( only really supports LAN ).
* Abstract the physical architecture of the participating workstations, by defining common protocols providing well-defined interfaces.

V was run on LAN, and its developers developed a really fast IPC protocol which allowed for it to be a fasted distributed operating system in a small geographic area. Aside from the IPC protocols, V also implemented RPC calls in the background.

V uses the strong consistency model. This model can cause issues with concurrency because in V files are a memory space. Thus two different users accessing the same file and in fact accessing the same memory location. This could result in issues unless there is an effective implementation to deal with multiple versions, etc.

VMTP protocol was used for communication. It supports request-respond behavior. Besides, it provides transparency, group communication facility and flow control. It is pretty much like TCP.

DistOS 2015W Session 7

2015-02-28T16:59:14Z

Aimbot: Formatting, updates and additions to both sections.

= Ceph =
Unlike GFS that was talked about previously, this is a general purpose distributed file system. It follows the same general model of distribution as GFS and Amoeba.

== Main Components ==
* Client

* Cluster of Object Storage Devices (OSD)
** It basically stores data and metadata and clients communicate directly with it to perform IO operations
** Data is stored in objects (variable size chunks)

* Meta-data Server (MDS)
** It is used to manage the file and directories. Clients interact with it to perform metadata operations like open, rename. It manages the capabilities of a client.
** Clients '''do not''' need to access MDSs to find where data is stored, improving scalability (more on that below)

== Key Features ==
* Decoupled data and metadata

* Dynamic Distributed Metadata Management

** It distributes the metadata among multiple metadata servers using dynamic sub-tree partitioning, meaning folders that get used more often get their meta-data replicated to more servers, spreading the load. This happens completely automatically

* Object based storage
** Uses cluster of OSDs to form a Reliable Autonomic Distributed Object-Store (RADOS) for Ceph failure detection and recovery

* CRUSH (Controlled, Replicated, Under Scalable, Hashing)
** The hashing algorithm used to calculate the location of object instead of looking for them
** This significantly reduces the load on the MDSs
** Responsible for automatically moving data when ODSs are added or removed (can be simplified as ''location = CRUSH(filename) % num_servers'')
** The CRUSH paper on Ceph’s website can be [http://ceph.com/papers/weil-crush-sc06.pdf viewed here]

* RADOS (Reliable Autonomic Distributed Object-Store) is the object store for Ceph

= Chubby =
It is a coarse grained lock service, made and used internally by Google, that serves multiple clients with small number of servers (chubby cell).

== System Components ==
* Chubby Cell
** Handles the actual locks
** Mainly consists of 5 servers known as replicas
** Consensus protocol ([https://en.wikipedia.org/wiki/Paxos_(computer_science) Paxos]) is used to elect the master from replicas

* Client
** Used by programs to request and use locks

== Main Features ==
* Implemented as a semi POSIX-compliant file-system with a 256KB file limit
** Permissions are for files only, not folders, thus breaking some compatibility
** Trivial to use by programs; just use the standard ''fopen()'' family of calls

* Uses consensus algorithm among a set of servers to agree on who is the master that is in charge of the metadata

* Meant for locks that last hours or days, not seconds (thus, "coarse grained")

* A master server can handle tens of thousands of simultaneous connections
** This can be further improved by using caching servers as most of the traffic are keep-alive messages

* Used by GFS for electing master server

* Also used by Google as a nameserver

== Issues ==
* Due to the use of Paxos (the only algorithm for this problem), the chubby cell is limited to only 5 servers. While this limits fault-tolerance, in practice this is more than enough.
* Since it has a file-system client interface, many programmers tend to abuse the system and need education (even inside Google)

DistOS 2015W Session 3

2015-02-07T19:42:21Z

Aimbot: Cleaned up, fixed typos, expanded some information.

= Multics =

'''Team:''' Sameer, Shivjot, Ambalica, Veena

It came into being in the 1960s and it completely vanished in 2000s. It was started by Bell, General Electric and MIT but Bell backed out of the project in 1969.
Multics is a time sharing OS which provides multitasking and multiprogramming. It is not a distributed OS but it a centralized system which was written in the assembly language.

It provides following features:
# Utility Computing
# Access Control Lists
# Single level storage
# Dynamic linking
#* Sharded libraries or files can be loaded and linked to Random Access Memory at run time
# Hot swapping
# Multiprocessing System
# Ring oriented Security
#* It provides number of levels of authorization within the computer system
#* Still present in some form today, inside both processors (like x86) and operating systems

= Unix =

Unix in its original conception is a small, minimal API system designed by two guys from Bell Labs. It was essentially a OS that would be easy to grasp for an programmer but not much beyond that. The UNIX OS ran on one computer, and terminals ran from that one computer. Thus it is not a distributed operating system as it is centralized and implements time sharing. In fact, it didn't even have support for networking in the first version.

The C language was created specifically for Unix, as the creators wanted to create a machine-agnostic language for the operating system.

Most features from Unix are still available in present day Unix-based systems. For example, the shell, with its piping capabilities, is still used today in its original form.

= Sun NFS =

The Sun NFS OS implemented networking using RPC connections. These connections are not secure. Sun wanted to encrypt these RPC connections but encryption would result in government regulations that Sun wanted to avoid in order to sell NFS over seas.

<blockquote>
Sun wanted to secure their NFS with encryption, but at the time encryption was regulated like munitions in the United States. Exporting any product that had encryption was impossible, but Sun needed those sales abroad. To avoid these regulations, Sun decided to sell the insecure NFS version of the system.
</blockquote>

'''Team''': Mert

= Locus =

# Not scalable
#* The synchronization algorithms were so slow, they only managed to run it on five computers
#* Every computer stores a copy of every file
#* Also used CAS to manage files
# Not efficient with abstractions
#* Trying to distribute files and processes
#Allowed for process migration
#Transparency
#* It provided network transparency to “disguise” its distributed context.
#Dynamic reconfiguration. (It adapts topology changes.)

Locus has lots of similarities with the today's systems. It uses replication and partitioning which are employed in cloud and distributed systems.

= Sprite =

''' Team''': Jamie, Hassan, Khaled

Sprite had the following Design Features:
# Network Transparency
# Process Migration, file transfer between computers
#* User could initiate a process migration to an idle machine, and if the machine was no longer idle; due to it being used by another user, the system would take care of the process migration to another machine
# Handling Cache Consistency
#* Sequential file sharing ==> By using a version number for each file
#* Concurrent write sharing ==> Disable cache to clients, enable write-blocking and other methods
# Implemented a caching system that sped up performance
# Implemented a log structured file system
#* They realized that with increasing amounts of RAM in computers which can be used for caching, writes to the disk were the main bottleneck, not reads.
#* Log structured file-systems are optimized for writes, as changes to previous data are appended at the current position.
#* This allows for very fast, sequential writes.
#* Example: SSD (Solid-state disks)

The main features to take away from the Sprite system is that it implemented a log structured file system, and implemented caching to increase performance.

DistOS 2015W Session 4

2015-02-07T17:34:43Z

Aimbot: Updated the AFS section.

= Andrew File System =
AFS (Andrew File System) was set up as a direct response to NFS. Essentially universities found issues when they tried to scale NFS in a way that would allow them to share files amongst their staff effectively. AFS was more scalable than NFS because read-write operations happened locally before they were committed to the server (data store).

Since AFS copied files locally when they were opened and only sent the data back when they were closed, all operations during that time are very fast and do need the network. NFS works with files remotely, so there is no data to transfer when opening/closing the file, making those operations instant.

There are several problems with this design, however.
* The local system must have enough space to temporarily store the file.
* Opening and closing the files requires a lot of bandwidth for large files. To read even a single byte, the entire file must be retrieved (later versions remedied this).
* If the close operation fails, the system will not have the updated version of the file. Many programs don't even check the return value of the close operation, giving users the false impression that everything went well.

Given all this, AFS was suitable for working with small files, not large ones, limiting its usefulness. It is also notoriously annoying to set up as it is geared towards university-sized networks, further limiting its success.

= Amoeba Operating System =

=== Capablities: ===
* Pointer to the object
* Capability assigning right to perform to some operation to the object ticket
* Communicate wide area network
* a kind of ticket or key that allows the holder of the capa- bility to perform some (not neces- sarily all)
* Each user process owns some collection of capabilities, which together define the set of objects it may access and the types of operations that my ne performed on each
* After the server has performed the operation, it sends back a reply message that unblocks the client
* Sending messages, blocking and accepting forms the remote procedure call that can be encapsulate using to make entire remote operation look like local procedure
* Second field: used by the sever to identify which of its objects is being addressed server port and object number identify object which operation to performed
* Generates 48-bit random number
* The third field is the right field which contains a bit map telling which operation the holder of the capability may performed
* X11 Window management

=== Thread Management: ===
* Same process have multiple thread and each process has its own registered counter and stack
* Behave like process
* It can synchronized using mutex semaphore
* File: Multiple thread,
* Blocked when there's multiple threads
* Buttlet thread the mutex
* The careful reader may have noticed that user process can pull 813kbytes/sec

= The V Distributed System =

* First tent in V design: High Performance communication is the most critical facility for distributed systems.
* Second; The protocols, not the software, define the system.
* Third; a relatively small operating system kernel can implement the basic protocols and services providing a simple network-transparent process, address space & communication model.

=== Ideas that significantly affected the design ===
* Shared Memory.
* Dealing with group of entities same as they deal w/individual entities.
* Efficient file caching mechanism using the virtual memory caching mechanism.

=== Design Decisions ===
* Designed for a cluster of workstations with high speed network access ( only really supports LAN ).
* Abstract the physical architecture of the participating workstations, by defining common protocols providing well-defined interfaces.

V was run on LAN, and its developers developed a really fast IPC protocol which allowed for it to be a fasted distributed operating system in a small geographic area. Aside from the IPC protocols, V also implemented RPC calls in the background.

V uses the strong consistency model. This model can cause issues with concurrency because in V files are a memory space. Thus two different users accessing the same file and in fact accessing the same memory location. This could result in issues unless there is an effective implementation to deal with multiple versions, etc.

VMTP protocol was used for communication. It supports request-respond behavior. Besides, it provides transparency, group communication facility and flow control. It is pretty much like TCP.

DistOS 2015W Session 5

2015-02-07T15:56:24Z

Aimbot: Formatted and added to the GFS section.

= The Clouds Distributed Operating System =
It is a distributed OS running on a set of computers that are interconnected by a group of network. It basically unifies different computers into a single component.

The OS is based on 2 patterns:
* Message Based OS
* Object Based OS

== Object Thread Model ==

The structure of this is based on object thread model. It has set of objects which are defined by the class. Objects respond to messages. Sending message to object causes object to execute the method and then reply back.

The system has '''active objects''' and '''passive objects'''.

# Active objects are the objects which have one or more processes associated with them and further they can communicate with the external environment.
# Passive objects are those that currently do not have an active thread executing in them.

The content of the Clouds data is long lived. Since the memory is implemented as a single-level store, the data exists forever and can survive system crashes and shut downs.

== Threads ==

The threads are the logical path of execution that traverse objects and executes code in them. The Clouds thread is not bound to a single address space. Several threads can enter an object simultaneously and execute concurrently. The nature of the Clouds object prohibits a thread from accessing any data outside the current address space in which it is executing.

== Interaction Between Objects and Threads ==

# Inter object interfaces are procedural
# Invocations work across machine boundaries
# Objects in clouds unify concept of persistent storage and memory to create address space, thus making the programming simpler.
# Control flow achieved by threads invoking objects.

== Clouds Environment ==

# Integrates set of homogeneous machines into one seamless environment
# There are three logical categories of machines- Compute Server, User Workstation and Data server.

= Plan 9 =

Plan 9 is a general purpose, multi-user and mobile computing environment physically distributed across machines. Development of the system began in the late 1980s. The system was built at Bell Labs - the birth place of Unix. The original Unix OS had no support for networking, and there were many attempts over the years by others to create distributed systems with Unix compatibility. Plan 9, however, is distributed systems done following the original Unix philosophy.

The goals of this system were:
# To built a distributed system that can be centrally administered.
# Be cost effective using cheap, modern microcomputers.

The distribution itself is transparent to most programs. This property is made possible by 2 properties:
# A per process group namespace.
# Uniform access to most resources by representing them as a file.

== Unix Compatibility ==

The commands, libraries and system calls are similar to that of Unix and therefore a casual user cannot distinguish between these two. The problems in UNIX were too deep to fix but still the various ideas were brought along. The problems addressed badly by UNIX were improved. Old tools were dropped and others were polished and reused.

== Unique Features ==

What actually distinguishes Plan 9 is its '''organization'''. Plan 9 is divided along the lines of service function.
* CPU services and terminals use same kernel.
* Users may choose to run programs locally or remotely on CPU servers.
* It lets the user choose whether they want a distributed or centralized system.

The design of Plan 9 is based on 3 principles:
# Resources are named and accessed like files in hierarchical file system.
# Standard protocol 9P.
# Disjoint hierarchical provided by different services are joined together into single private hierarchical file name space.

=== Virtual Namespaces ===

In a virtual namespace, a user boots a terminal or connects to a CPU server and then a new process group is created. Processes in group can either add to or rearrange their name space using two system calls - mount and bind.

* '''Mount''' is used to attach new file system to a point in name space.
* '''Bind''' is used to attach a kernel resident (existing, mounted) file system to name space and also arrange pieces of name space.
* There is also '''unbind''' which undoes the effects of the other two calls.

Namespaces in Plan 9 are on a per-process basis. While everything had a way reference resources with a unique name, using mount and bind, every process could also build their namespace as they saw fit.

Since most resources are in the form of files (and folders), the term ''namespace'' really only refers to the filesystem layout.

=== Parallel Programming ===
The parallel programming has two aspects:
* Kernel provides simple process model and carefully designed system calls for synchronization.
* Programming language supports concurrent programming.

== Legacy ==

Even though Plan 9 is no longer developed, the good ideas from the system still exist today. For example, the ''/proc'' virtual filesystem which displays current process information in the form of files exists in moden Linux kernels.

= Google File System =

It is scalable, distributed file system for large, data intensive applications. It is crafted to Google's unique needs as a search engine company.

Unlike most filesystems, GFS must be implemented by individual applications and is not part of the kernel. While this introduces some technical overhead, it gives the system more freedom to implement or not implement certain non-standard features.

== Architecture ==

The architecture of the Google file system consists of a single master, multiple chunk-servers and multiple clients. Chunk servers are used to store the data in uniformly sized chunks. Each chunk is identified by globally unique 64 bit handle assigned by master at the time of creation. The chunks are split into 64KB blocks, each with its own hashsum for data integrity checks. The chunks are replicated between servers, 3 by default. The master maintains all the file system metadata which includes the namespace and chunk location.

Each of the chunks is 64 MB large (contrast this to the typical filesystem sectors of 512 or 4096 bytes) as the system is meant to hold enormous amount of data - namely the internet. The large chunk size is also important for the scalability of the system - the larger the chunk size, the less metadata the master server has to store for any given amount of data. With the current size, the master server is able to store the entirety of the metadata in memory, increasing performance by a significant margin.

== Operation ==

Master and Chunk server communication consists of
# checking whether there is any chunk-server is down.
# checking if any file is corrupted.
# deleting stale chunks

When a client want to do some operations on the chunks
# it first asks the master server for the list of servers that store the parts of a file it wants to access
# it receives a list of chunk servers, with multiple servers for each chunk
# it finally communicates with the the chunk servers to perform the operation

The system is geared towards appends and sequential reads. This is why the master server responds with multiple server addresses for each chunk - the client can then request a small piece from each server, increasing the data throughput linearly with the number of servers. Writes, in general, are in the form of a special ''append'' system call. When appending, there is no chance that two clients will want to write to the same location at the same time. This helps avoid any potential synchronization issues. If there are multiple appends to the same file at the same time, the chunk servers are free to order them as they wish (chunks on each server are not guaranteed to be byte-for-byte identical). While a problem in the general sense, this is good enough for Google's needs.

== Redundancy ==

GFS is built with failure in mind. The system expects that at any time, there is some server or disk that is malfunctioning. The system deals with the failures as follows.

=== Chunk Servers ===

By default, chunks are replicated to three servers. This exact number depends on the application in question doing the write. When a chunk server finds that some of its data is corrupt, it grabs the data from other servers to repair itself{{Citation needed}}.

=== Master Server ===

For efficiency, there is only a single live master server at a time. While not making the system completely distributed, it avoid many synchronization problems and suits Google's needs. At any point in time, there are multiple read-only master servers that copy metadata from the currently live master. Should it go down, they will serve any read operations from the clients, until one of the hot spares is promoted to being the new live master server.

DistOS 2015W Session 5

2015-02-07T15:23:36Z

Aimbot: Formatted and edited Plan 9 section

= The Clouds Distributed Operating System =
It is a distributed OS running on a set of computers that are interconnected by a group of network. It basically unifies different computers into a single component.

The OS is based on 2 patterns:
* Message Based OS
* Object Based OS

== Object Thread Model ==

The structure of this is based on object thread model. It has set of objects which are defined by the class. Objects respond to messages. Sending message to object causes object to execute the method and then reply back.

The system has '''active objects''' and '''passive objects'''.

# Active objects are the objects which have one or more processes associated with them and further they can communicate with the external environment.
# Passive objects are those that currently do not have an active thread executing in them.

The content of the Clouds data is long lived. Since the memory is implemented as a single-level store, the data exists forever and can survive system crashes and shut downs.

== Threads ==

The threads are the logical path of execution that traverse objects and executes code in them. The Clouds thread is not bound to a single address space. Several threads can enter an object simultaneously and execute concurrently. The nature of the Clouds object prohibits a thread from accessing any data outside the current address space in which it is executing.

== Interaction Between Objects and Threads ==

# Inter object interfaces are procedural
# Invocations work across machine boundaries
# Objects in clouds unify concept of persistent storage and memory to create address space, thus making the programming simpler.
# Control flow achieved by threads invoking objects.

== Clouds Environment ==

# Integrates set of homogeneous machines into one seamless environment
# There are three logical categories of machines- Compute Server, User Workstation and Data server.

= Plan 9 =

Plan 9 is a general purpose, multi-user and mobile computing environment physically distributed across machines. Development of the system began in the late 1980s. The system was built at Bell Labs - the birth place of Unix. The original Unix OS had no support for networking, and there were many attempts over the years by others to create distributed systems with Unix compatibility. Plan 9, however, is distributed systems done following the original Unix philosophy.

The goals of this system were:
# To built a distributed system that can be centrally administered.
# Be cost effective using cheap, modern microcomputers.

The distribution itself is transparent to most programs. This property is made possible by 2 properties:
# A per process group namespace.
# Uniform access to most resources by representing them as a file.

== Unix Compatibility ==

The commands, libraries and system calls are similar to that of Unix and therefore a casual user cannot distinguish between these two. The problems in UNIX were too deep to fix but still the various ideas were brought along. The problems addressed badly by UNIX were improved. Old tools were dropped and others were polished and reused.

== Unique Features ==

What actually distinguishes Plan 9 is its '''organization'''. Plan 9 is divided along the lines of service function.
* CPU services and terminals use same kernel.
* Users may choose to run programs locally or remotely on CPU servers.
* It lets the user choose whether they want a distributed or centralized system.

The design of Plan 9 is based on 3 principles:
# Resources are named and accessed like files in hierarchical file system.
# Standard protocol 9P.
# Disjoint hierarchical provided by different services are joined together into single private hierarchical file name space.

=== Virtual Namespaces ===

In a virtual namespace, a user boots a terminal or connects to a CPU server and then a new process group is created. Processes in group can either add to or rearrange their name space using two system calls - mount and bind.

* '''Mount''' is used to attach new file system to a point in name space.
* '''Bind''' is used to attach a kernel resident (existing, mounted) file system to name space and also arrange pieces of name space.
* There is also '''unbind''' which undoes the effects of the other two calls.

Namespaces in Plan 9 are on a per-process basis. While everything had a way reference resources with a unique name, using mount and bind, every process could also build their namespace as they saw fit.

Since most resources are in the form of files (and folders), the term ''namespace'' really only refers to the filesystem layout.

=== Parallel Programming ===
The parallel programming has two aspects:
* Kernel provides simple process model and carefully designed system calls for synchronization.
* Programming language supports concurrent programming.

== Legacy ==

Even though Plan 9 is no longer developed, the good ideas from the system still exist today. For example, the ''/proc'' virtual filesystem which displays current process information in the form of files exists in moden Linux kernels.

= Google File System =

It is scalable file system for large distributed data intensive applications. The design is driven by providing previous applications workloads and technical environments, both current and anticipated.

The architecture of the Google file system consists of a single master, multiple chunk-servers and multiple clients. These chunk-servers store the data or file in unit of named chunks. Each chunk is identified by globally unique 64 bit chunk handle assigned by master at the end of the time of chunk creation. For more reliability and availability chunks are replicated on more chunk servers. The master maintains all the file system meta data which include the name space, chunk location and also the access control information.

Master and Chunk-Server Communication:
a) To check whether there is any chunk-server is down
b) To check if any file is corrupted.
c) Whether to create or delete any chunk.

Operation of GFS:
a) Client communicate with master to get the matadata.
b) client get chunk location from matadata.
c) Communicate with the one of that chunk-server to retrieve the data to perform operations on it.

DistOS 2015W Session 5

2015-02-07T14:57:00Z

Aimbot: Formatted the clouds section

Distributed OS: Winter 2015

2015-01-14T21:56:27Z

Aimbot: /* Notes */ Updated link to second set of notes.

==Course Outline==

[[Distributed OS: Winter 2015 Course Outline|Here]] is the course outline. It should see only minor modifications during the semester.

==Assigned Readings==

===January 12, 2015===

The Early Internet:
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/kahn1972-resource.pdf Robert E. Kahn, "Resource-Sharing Computer Communications Networks" (1972)] [http://dx.doi.org/10.1109/PROC.1972.8911 (DOI)]
* [https://archive.org/details/ComputerNetworks_TheHeraldsOfResourceSharing Computer Networks: The Heralds of Resource Sharing (1972)] - video

The Alto:
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/alto.pdf Thacker et al., "Alto: A Personal computer" (1979)] ([https://archive.org/details/bitsavers_xeroxparcttoAPersonalComputer_6560658 archive.org])

The Mother of All Demos:
* [http://www.dougengelbart.org/firsts/dougs-1968-demo.html Doug Engelbart Institute, "Doug's 1968 Demo"]. You may want to focus on the [http://dougengelbart.org/events/1968-demo-highlights.html highlights] or the [http://sloan.stanford.edu/MouseSite/1968Demo.html annotated clips].
* [http://en.wikipedia.org/wiki/The_Mother_of_All_Demos Wikipedia's page on "The Mother of all Demos"]

===January 19, 2015===

* [http://en.wikipedia.org/wiki/Multics Wikipedia article on Multics]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/unix.pdf Dennis M. Ritchie and Ken Thompson, "The UNIX Time-Sharing System" (1974)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/sandberg-nfs.pdf Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/ousterhout-sprite.pdf John Ousterhout et al., "The Sprite Network Operating System" (1987)]

===January 26, 2015===

* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/cheriton-v.pdf David R. Cheriton, "The V Distributed System." (1988)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/tanenbaum-amoeba.pdf Andrew Tannenbaum et al., "The Amoeba System" (1990)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/clouds-dasgupta.pdf Partha Dasgupta et al., "The Clouds Distributed Operating System" (1991)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/howard-afs.pdf John H. Howard et al., "Scale and Performance in a Distributed File System" (1988)]

==Other Readings==

===The Early Web===

* [https://archive.org/details/02Kahle000673 Berners-Lee et al., "World-Wide Web: The Information Universe" (1992)], pp. 52-58
* [http://www.youtube.com/watch?v=72nfrhXroo8 Alex Wright, "The Web That Wasn't" (2007)], Google Tech Talk

===Plan 9===

* [http://homeostasis.scs.carleton.ca/~soma/distos/2014w/presotto-plan9.pdf Presotto et. al, Plan 9, A Distributed System (1991)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2014w/pike-plan9.pdf Pike et al., Plan 9 from Bell Labs (1995)]

===GFS and Ceph===
* [http://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]
* [http://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].

===Chubby===

* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]

===Oceanstore===

* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (2000)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (2003)]

===Farsite===

* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/adya-farsite-intro.pdf Atul Adya et al.,"FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment" (2002)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/bolosky-farsite-retro.pdf William J. Bolosky et al., "The Farsite Project: A Retrospective" (2007)]

===Public Resource Computing===

* Anderson et al., "SETI@home: An Experiment in Public-Resource Computing" (CACM 2002) [http://dx.doi.org/10.1145/581571.581573 (DOI)] [http://dl.acm.org.proxy.library.carleton.ca/citation.cfm?id=581573 (Proxy)]
* Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004) [http://dx.doi.org/10.1109/GRID.2004.14 (DOI)] [http://ieeexplore.ieee.org.proxy.library.carleton.ca/stamp/stamp.jsp?tp=&arnumber=1382809 (Proxy)]

===Distributed Hash Tables===

* [http://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]
* [http://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]

===Structured Data===

* [http://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]
* [https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Geambasu.pdf Geambasu et al., "Comet: An active distributed key-value store" (OSDI 2010)]

===Specialized Storage===

* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)]

===Computational Models===

* [http://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)]
* [http://dl.acm.org/citation.cfm?doid=2517349.2522738 Murray et al., "Naiad: a timely dataflow system" (SOSP 2013)]

===Literature Review Help===

* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]
* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]

==Notes==

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0" class="wikitable">
<tr valign="top">
<th>
Date
</th>
<th>
Topic
</th>
</tr>
<tr>
<td>
Jan. 5

</td>
<td>
[[DistOS 2015W Session 1|Session 1]]

</td>
</tr>
<tr>
<td>
Jan. 12

</td>
<td>
[[DistOS 2015W Session 2|Session 2]]

</td>
</tr>
<tr>
<td>
Jan. 19

</td>
<td>
[[DistOS 2015W Session 3|Session 3]]

</td>
</tr>
<tr>
<td>
Jan. 26

</td>
<td>
[[DistOS 2015W Session 4|Session 4]]

</td>
</tr>
<tr>
<td>
Feb. 2

</td>
<td>
[[DistOS 2015W Session 5|Session 5]]

</td>
</tr>
<tr>
<td>
Feb. 9

</td>
<td>
[[DistOS 2015W Session 6|Session 6]]

</td>
</tr>
<tr>
<td>
Feb. 23

</td>
<td>
[[DistOS 2015W Session 7|Session 7]]

</td>
</tr>
<tr>
<td>
Mar. 2

</td>
<td>
[[DistOS 2015W Session 8|Session 8]]

</td>
</tr>
<tr>
<td>
Mar. 9

</td>
<td>
[[DistOS 2015W Session 9|Session 9]]

</td>
</tr>
<tr>
<td>
Mar. 16

</td>
<td>
[[DistOS 2015W Session 10|Session 10]]

</td>
</tr>
<tr>
<td>
Mar. 23

</td>
<td>
[[DistOS 2015W Session 11|Session 11]]

</td>
</tr>
<tr>
<td>
Mar. 30

</td>
<td>
[[DistOS 2015W Session 12|Session 12]]

</td>
</tr>
<tr>
<td>
Apr. 6

</td>
<td>
[[DistOS 2015W Project Presentations|Final Project Presentations]]

</td>
</tr>
<tr>
<td>
TBA

</td>
<td>
'''Final Exam'''

</td>
</tr>
</table>

DistOS 2014W Session 2

2015-01-14T21:55:32Z

Aimbot: Aimbot moved page DistOS 2014W Session 2 to DistOS 2015W Session 2: Wrong year

#REDIRECT [[DistOS 2015W Session 2]]

DistOS 2015W Session 2

2015-01-14T21:55:32Z

Aimbot: Aimbot moved page DistOS 2014W Session 2 to DistOS 2015W Session 2: Wrong year

= Reading Response Discussion =

== Mother of All Demos: ==

'''Team:''' Jason, Kirill, Sravan, Agustin, Hassan Ambalica, Apoorv, Khaled

* 1968, lead by Doug Inglebard
* Initial public display of many modern technologies
* One computer with multiple remote terminals
* Video conferencing
* Computer mouse (and coined the term)
* Word processing, rudimentary copy and paste
* Dynamic file linking (hypertext)
* Revision control/version control/source control
* Collaborative real-time editor
** User privilege control in that user can provide read-only access, read-write privilege to file
* Corded keyboard
** Marco keyboard that allows messages quickly, while using mouse at same time
* Not really a distributed operating system, but great start because multiple users at different terminals could share same resources

== Early Internet ==

== Alto ==

'''Team:''' Tory, Veena, Sameer, Mert, Deep, Nameet, Moe

* Initially developed around 1973 by Xerox, upgraded over next few years
* High speed network connectivity (3 Mbps @ 1km distance)
* Connected up to 256 machines
* Protocol similar to cross between UDP/TCP, before TCP invented
* Allowed sharing of printers
* Also allowed distribution of files across computers (redundancy/reliability)
* Sort of early cloud
* Allowed for remote debugging, store error logs
* Allowed machines to use processing power of others
* Much of the time it would be idling, which is amazing at a time when computers cost a fortune

= Professor lead discussion 1 =

A true distributed operating system does not actually exist, its more of a dream.

You'll find in the history of trying to achieve this dream of a distributed operating system there has always been a road block as a result of a technical issue. People would come up with a solution to this technical issue but then they would find that some other technical issue would crop up. For example 'The Mother of All Demos' tried to build a distributed operating system but they had no networking. They had to point a television camera at the computer monitor to be able to demo their concept. The early internet came along and dealt with the issue of no networking. However the early internet itself had other technical issues.

A common buzz word during the early days of development was '''time sharing''', which is an old term for multiple processes running simultaneously. However, at the time it referred to multiple users sharing the CPU cycles on a single computer. Today, a single user's many processes using a single CPU is much more common.

= Discussion: Easy on one computer, hard on multiple computers =

{| class="wikitable" width="100%"
|+ '''Instant Messaging'''
|-
! Why is it easy?
! Why is it hard?
|-
| Can't be out sync
| Synchronization issues
* Server-client modal
** If two or more servers they get requests same time, they have to figure out how to synchronize the data
* Peer-to-Peer
** Each peer has to figure out how to sync each message
|-
| Only one data store/source
|-
|}

{| class="wikitable" width="100%"
|+ '''Mortal Kombat/Twitch Gaming'''
|-
! Why is it easy?
! Why is it hard?
|-
|}

{| class="wikitable" width="100%"
|+ '''Photo Album'''
|-
! Why is it easy?
! Why is it hard?
|-
|}

=== Commonalities ===
* Synchronization
* Bandwidth
* Reliability/Fault Tolerance
* Interoperability
* Discovery
* Routing
** In modern system this issue has been mostly abstracted away
** Classic example would be Wireless Ad-hoc networking

The above list is not hard on a single system because all the cores have equal access to all the resources. Also as a result of excellent engineering modern single systems can very effective deal with any errors that may result.

User:Aimbot

2015-01-14T03:21:31Z

Aimbot: Created page with "https://i.imgur.com/PqtnWTP.jpg"

https://i.imgur.com/PqtnWTP.jpg

Distributed OS: Winter 2015

2015-01-14T03:18:12Z

Aimbot: Added standard wikipedia table CSS class to the table of notes.

==Course Outline==

[[Distributed OS: Winter 2015 Course Outline|Here]] is the course outline. It should see only minor modifications during the semester.

==Assigned Readings==

===January 12, 2015===

The Early Internet:
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/kahn1972-resource.pdf Robert E. Kahn, "Resource-Sharing Computer Communications Networks" (1972)] [http://dx.doi.org/10.1109/PROC.1972.8911 (DOI)]
* [https://archive.org/details/ComputerNetworks_TheHeraldsOfResourceSharing Computer Networks: The Heralds of Resource Sharing (1972)] - video

The Alto:
* [https://homeostasis.scs.carleton.ca/~soma/distos/2014w/alto.pdf Thacker et al., "Alto: A Personal computer" (1979)] ([https://archive.org/details/bitsavers_xeroxparcttoAPersonalComputer_6560658 archive.org])

The Mother of All Demos:
* [http://www.dougengelbart.org/firsts/dougs-1968-demo.html Doug Engelbart Institute, "Doug's 1968 Demo"]. You may want to focus on the [http://dougengelbart.org/events/1968-demo-highlights.html highlights] or the [http://sloan.stanford.edu/MouseSite/1968Demo.html annotated clips].
* [http://en.wikipedia.org/wiki/The_Mother_of_All_Demos Wikipedia's page on "The Mother of all Demos"]

===January 19, 2015===

* [http://en.wikipedia.org/wiki/Multics Wikipedia article on Multics]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/unix.pdf Dennis M. Ritchie and Ken Thompson, "The UNIX Time-Sharing System" (1974)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/walker-locus.pdf Bruce Walker et al., "The LOCUS Distributed Operating System." (1983)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/sandberg-nfs.pdf Russel Sandberg et al., "Design and Implementation of the Sun Network Filesystem" (1985)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/ousterhout-sprite.pdf John Ousterhout et al., "The Sprite Network Operating System" (1987)]

===January 26, 2015===

* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-21/cheriton-v.pdf David R. Cheriton, "The V Distributed System." (1988)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/tanenbaum-amoeba.pdf Andrew Tannenbaum et al., "The Amoeba System" (1990)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-01-28/clouds-dasgupta.pdf Partha Dasgupta et al., "The Clouds Distributed Operating System" (1991)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2008-02-11/howard-afs.pdf John H. Howard et al., "Scale and Performance in a Distributed File System" (1988)]

==Other Readings==

===The Early Web===

* [https://archive.org/details/02Kahle000673 Berners-Lee et al., "World-Wide Web: The Information Universe" (1992)], pp. 52-58
* [http://www.youtube.com/watch?v=72nfrhXroo8 Alex Wright, "The Web That Wasn't" (2007)], Google Tech Talk

===Plan 9===

* [http://homeostasis.scs.carleton.ca/~soma/distos/2014w/presotto-plan9.pdf Presotto et. al, Plan 9, A Distributed System (1991)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/2014w/pike-plan9.pdf Pike et al., Plan 9 from Bell Labs (1995)]

===GFS and Ceph===
* [http://research.google.com/archive/gfs-sosp2003.pdf Sanjay Ghemawat et al., "The Google File System" (SOSP 2003)]
* [http://www.usenix.org/events/osdi06/tech/weil.html Weil et al., Ceph: A Scalable, High-Performance Distributed File System (OSDI 2006)].

===Chubby===

* [https://www.usenix.org/legacy/events/osdi06/tech/burrows.html Burrows, The Chubby Lock Service for Loosely-Coupled Distributed Systems (OSDI 2006)]

===Oceanstore===

* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/oceanstore-sigplan.pdf John Kubiatowicz et al., "OceanStore: An Architecture for Global-Scale Persistent Storage" (2000)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/fast2003-pond.pdf Sean Rhea et al., "Pond: the OceanStore Prototype" (2003)]

===Farsite===

* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/adya-farsite-intro.pdf Atul Adya et al.,"FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment" (2002)]
* [http://homeostasis.scs.carleton.ca/~soma/distos/fall2008/bolosky-farsite-retro.pdf William J. Bolosky et al., "The Farsite Project: A Retrospective" (2007)]

===Public Resource Computing===

* Anderson et al., "SETI@home: An Experiment in Public-Resource Computing" (CACM 2002) [http://dx.doi.org/10.1145/581571.581573 (DOI)] [http://dl.acm.org.proxy.library.carleton.ca/citation.cfm?id=581573 (Proxy)]
* Anderson, "BOINC: A System for Public-Resource Computing and Storage" (Grid Computing 2004) [http://dx.doi.org/10.1109/GRID.2004.14 (DOI)] [http://ieeexplore.ieee.org.proxy.library.carleton.ca/stamp/stamp.jsp?tp=&arnumber=1382809 (Proxy)]

===Distributed Hash Tables===

* [http://en.wikipedia.org/wiki/Distributed_hash_table Wikipedia's article on Distributed Hash Tables]
* [http://pdos.csail.mit.edu/~strib/docs/tapestry/tapestry_jsac03.pdf Zhao et al, "Tapestry: A Resilient Global-Scale Overlay for Service Deployment" (JSAC 2003)]

===Structured Data===

* [http://research.google.com/archive/bigtable-osdi06.pdf Chang et al., "BigTable: A Distributed Storage System for Structured Data" (OSDI 2006)]
* [http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf DeCandia et al., "Dynamo: Amazon’s Highly Available Key-value Store" (SOSP 2007)]
* [http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf Lakshman & Malik, "Cassandra - A Decentralized Structured Storage System" (LADIS 2009)]
* [https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Geambasu.pdf Geambasu et al., "Comet: An active distributed key-value store" (OSDI 2010)]

===Specialized Storage===

* [http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf Beaver et al., "Finding a needle in Haystack: Facebook’s photo storage" (OSDI 2010)]

===Computational Models===

* [http://research.google.com/archive/mapreduce.html Dean & Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters" (OSDI 2004)]
* [http://dl.acm.org/citation.cfm?doid=2517349.2522738 Murray et al., "Naiad: a timely dataflow system" (SOSP 2013)]

===Literature Review Help===

* Harvey, "What Is a Literature Review?" [http://www.cs.cmu.edu/~missy/WritingaLiteratureReview.doc (DOC)] [http://www.cs.cmu.edu/~missy/Writing_a_Literature_Review.ppt (PPT)]
* [http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review Taylor, "The Literature Review: A Few Tips On Conducting It"]

==Notes==

<table style="width: 100%;" border="1" cellpadding="4" cellspacing="0" class="wikitable">
<tr valign="top">
<th>
Date
</th>
<th>
Topic
</th>
</tr>
<tr>
<td>
Jan. 5

</td>
<td>
[[DistOS 2015W Session 1|Session 1]]

</td>
</tr>
<tr>
<td>
Jan. 12

</td>
<td>
[[DistOS 2014W Session 2|Session 2]]

</td>
</tr>
<tr>
<td>
Jan. 19

</td>
<td>
[[DistOS 2015W Session 3|Session 3]]

</td>
</tr>
<tr>
<td>
Jan. 26

</td>
<td>
[[DistOS 2015W Session 4|Session 4]]

</td>
</tr>
<tr>
<td>
Feb. 2

</td>
<td>
[[DistOS 2015W Session 5|Session 5]]

</td>
</tr>
<tr>
<td>
Feb. 9

</td>
<td>
[[DistOS 2015W Session 6|Session 6]]

</td>
</tr>
<tr>
<td>
Feb. 23

</td>
<td>
[[DistOS 2015W Session 7|Session 7]]

</td>
</tr>
<tr>
<td>
Mar. 2

</td>
<td>
[[DistOS 2015W Session 8|Session 8]]

</td>
</tr>
<tr>
<td>
Mar. 9

</td>
<td>
[[DistOS 2015W Session 9|Session 9]]

</td>
</tr>
<tr>
<td>
Mar. 16

</td>
<td>
[[DistOS 2015W Session 10|Session 10]]

</td>
</tr>
<tr>
<td>
Mar. 23

</td>
<td>
[[DistOS 2015W Session 11|Session 11]]

</td>
</tr>
<tr>
<td>
Mar. 30

</td>
<td>
[[DistOS 2015W Session 12|Session 12]]

</td>
</tr>
<tr>
<td>
Apr. 6

</td>
<td>
[[DistOS 2015W Project Presentations|Final Project Presentations]]

</td>
</tr>
<tr>
<td>
TBA

</td>
<td>
'''Final Exam'''

</td>
</tr>
</table>

DistOS 2015W Session 2

2015-01-14T03:15:49Z

Aimbot: /* Professor lead discussion 1 */ Updated time sharing definition.

DistOS 2015W Session 2

2015-01-14T03:09:38Z

Aimbot: /* Mother of All Demos: */ Minor updates, clarifications

= Reading Response Discussion =

== Mother of All Demos: ==

'''Team:''' Jason, Kirill, Sravan, Agustin, Hassan Ambalica, Apoorv, Khaled

* 1968, lead by Doug Inglebard
* Initial public display of many modern technologies
* One computer with multiple remote terminals
* Video conferencing
* Computer mouse (and coined the term)
* Word processing, rudimentary copy and paste
* Dynamic file linking (hypertext)
* Revision control/version control/source control
* Collaborative real-time editor
** User privilege control in that user can provide read-only access, read-write privilege to file
* Corded keyboard
** Marco keyboard that allows messages quickly, while using mouse at same time
* Not really a distributed operating system, but great start because multiple users at different terminals could share same resources

== Early Internet ==

== Alto ==

'''Team:''' Tory, Veena, Sameer, Mert, Deep, Nameet, Moe

* Initially developed around 1973 by Xerox, upgraded over next few years
* High speed network connectivity (3 Mbps @ 1km distance)
* Connected up to 256 machines
* Protocol similar to cross between UDP/TCP, before TCP invented
* Allowed sharing of printers
* Also allowed distribution of files across computers (redundancy/reliability)
* Sort of early cloud
* Allowed for remote debugging, store error logs
* Allowed machines to use processing power of others
* Much of the time it would be idling, which is amazing at a time when computers cost a fortune

= Professor lead discussion 1 =

A true distributed operating system does not actually exist, its more of a dream.

You'll find in the history of trying to achieve this dream of a distributed operating system there has always been a road block as a result of a technical issue. People would come up with a solution to this technical issue but then they would find that some other technical issue would crop up. For example 'The Mother of All Demos' tried to build a distributed operating system but they had no networking. They had to point a television camera at the computer monitor to be able to demo their concept. The early internet came along and dealt with the issue of no networking. However the early internet itself had other technical issues.

A common buzz word during the early days of development was '''Time Sharing'''. Time sharing is about the sharing of resources between users. Today the concept of time sharing is similar to sharing a processor between multiple processes.

= Discussion: Easy on one computer, hard on multiple computers =

{| class="wikitable" width="100%"
|+ '''Instant Messaging'''
|-
! Why is it easy?
! Why is it hard?
|-
| Can't be out sync
| Synchronization issues
* Server-client modal
** If two or more servers they get requests same time, they have to figure out how to synchronize the data
* Peer-to-Peer
** Each peer has to figure out how to sync each message
|-
| Only one data store/source
|-
|}

{| class="wikitable" width="100%"
|+ '''Mortal Kombat/Twitch Gaming'''
|-
! Why is it easy?
! Why is it hard?
|-
|}

{| class="wikitable" width="100%"
|+ '''Photo Album'''
|-
! Why is it easy?
! Why is it hard?
|-
|}

=== Commonalities ===
* Synchronization
* Bandwidth
* Reliability/Fault Tolerance
* Interoperability
* Discovery
* Routing
** In modern system this issue has been mostly abstracted away
** Classic example would be Wireless Ad-hoc networking

The above list is not hard on a single system because all the cores have equal access to all the resources. Also as a result of excellent engineering modern single systems can very effective deal with any errors that may result.

DistOS 2015W Session 2

2015-01-14T03:03:52Z

Aimbot: /* Alto */ Clarifications, minor updates

= Reading Response Discussion =

== Mother of All Demos: ==

Team: Jason, Kirill, Sravan, Agustin, Hassan Ambalica, Apoorv, Khaled

* 1968, Doug Inglebard
* One computer with multiple terminals controlling
* Video conferencing
* Computer mouse
* Word processing, rudimentary copy and paste
* Dynamic file linking
* Revision control/version control/source control
* Collaborative real-time editor
** User privilege control in that user can provide read-only access, read-write privilege to file
* Cord keyboard
** Marco keyboard that allows messages quickly, while using mouse at same time
* Not really a distributed operating system, but great start because multiple users at different terminals could share same resources

== Early Internet ==

== Alto ==

'''Team:''' Tory, Veena, Sameer, Mert, Deep, Nameet, Moe

* Initially developed around 1973 by Xerox, upgraded over next few years
* High speed network connectivity (3 Mbps @ 1km distance)
* Connected up to 256 machines
* Protocol similar to cross between UDP/TCP, before TCP invented
* Allowed sharing of printers
* Also allowed distribution of files across computers (redundancy/reliability)
* Sort of early cloud
* Allowed for remote debugging, store error logs
* Allowed machines to use processing power of others
* Much of the time it would be idling, which is amazing at a time when computers cost a fortune

= Professor lead discussion 1 =

A true distributed operating system does not actually exist, its more of a dream.

You'll find in the history of trying to achieve this dream of a distributed operating system there has always been a road block as a result of a technical issue. People would come up with a solution to this technical issue but then they would find that some other technical issue would crop up. For example 'The Mother of All Demos' tried to build a distributed operating system but they had no networking. They had to point a television camera at the computer monitor to be able to demo their concept. The early internet came along and dealt with the issue of no networking. However the early internet itself had other technical issues.

A common buzz word during the early days of development was '''Time Sharing'''. Time sharing is about the sharing of resources between users. Today the concept of time sharing is similar to sharing a processor between multiple processes.

= Discussion: Easy on one computer, hard on multiple computers =

{| class="wikitable" width="100%"
|+ '''Instant Messaging'''
|-
! Why is it easy?
! Why is it hard?
|-
| Can't be out sync
| Synchronization issues
* Server-client modal
** If two or more servers they get requests same time, they have to figure out how to synchronize the data
* Peer-to-Peer
** Each peer has to figure out how to sync each message
|-
| Only one data store/source
|-
|}

{| class="wikitable" width="100%"
|+ '''Mortal Kombat/Twitch Gaming'''
|-
! Why is it easy?
! Why is it hard?
|-
|}

{| class="wikitable" width="100%"
|+ '''Photo Album'''
|-
! Why is it easy?
! Why is it hard?
|-
|}

=== Commonalities ===
* Synchronization
* Bandwidth
* Reliability/Fault Tolerance
* Interoperability
* Discovery
* Routing
** In modern system this issue has been mostly abstracted away
** Classic example would be Wireless Ad-hoc networking

The above list is not hard on a single system because all the cores have equal access to all the resources. Also as a result of excellent engineering modern single systems can very effective deal with any errors that may result.

DistOS 2015W Session 1

2015-01-11T20:05:17Z

Aimbot: General clean-up (grammar, spelling, wiki-markup, etc)

Notes for the first session that happened on Jan. 5th, 2015.

= Course Outline =

Undergrad Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% lecture Notes/Wiki Contributions
* 25% Midterm
* 35% Final Exam

Grads Grading Scheme
* 15% Class Participation
* 15% Reading Responses
* 10% Lectures Notes/Wiki contributions
* 10% Project Proposal
* 15% Project Presentation
* 35% Final Project

=== Project ===
* A literature review of distributing operations
* Research proposal on a problem related to distributed systems

= Discussion =

== Q: What do you think of when you hear 'Distributed System'? ==

=== Group 1 ===
* Sharing Resources
* Spreading Work Loads
* Scheduling
* Process migration
* Different nodes have different purpose
* Google
* Parallel running processes
* Nodes
* Resource Allocation across multiple nodes
* Scheduling multiple nodes
* Resources availability among nodes
* Problem: across multiple machine

=== Group 2 ===
* Request comes from (usually) one computer and processing is usually handled by more than one computer
* Tasks are divided into small parts which can be processed individually before coming back together
* Usually deals with large scale data sets
* Globalization
* Fault tolerance
* Usually emphasize distributed agents to not work on certain schedule

=== Group 3 ===
* Separated, networked machines
* Coordinated by similar or identical software
* Error recovery/redundancy
* No controlled storage
* Coordinated communication facilitating operation on coordinated task
* Leader/hierarchy for task delegation
* Examples: Map Reduce, cloud, cloud software (Google Drive)

=== Group 4 ===
* Distributed, multiple systems
* OS: The root level system that operates a computer system
* Dist. OS more complexity

=== Prof Discussion Notes ===
Key words: network, parallel, fault tolerant, redundancy, complexity

== Distributed OS is kind of OS ==

'''But what is an OS in the first place?'''
* Connection software and hardware
* Resource allocation
* Abstraction layers
* Makes it easy to run higher level programs on different computers
* Sharing: resources are split up and each process is isolated, this improves security and programming (no need to worry about sharing)
* Virtual memory, process scheduling

'''What is a distributed system?'''
* Feels like it is only a single node, but runs on many
* Do the things users take for granted on regular computers
* Doesn't actually work as a single (general use) computer - with 1000 computers one would expect it to be like a single computer with 1000 times the resources, but it's not
* In a way, opposite of an operating system: instead of splitting up a single computer into many, it takes many and tries to merge into one
* Resources can be divided as you want
* Centralized control solves some issues, but has issues of its own
* Since it cannot truly act as a single computer, we fake it as much as possible - and it works in some scenarios
* The more specialized the task, the better it will scale
* Interact it, process update, callback,
* Gmail so responsible - it perfected , downloads in your browser
* Cache - they predict what the processor going to do

== Operating System Examples ==
'''Mobile devices - Phones'''
* iOS
* Android

'''Embedded OS'''
* Linux
* QNX
* xBSD firewalls

'''Desktop'''
* Windows
* OSX
* Chrome OS

'''Server'''
* Windows
* Linux
* BSD

'''Main Frames'''
* OS/400

Is the cloud an OS? Important question.

'''Cloud'''
* MPI
* AWS
* Google App Engine

No, they are at best proto-OSs because the abstractions they provide are very leaky. They only provide limited APIs.

== Pick a system and show how it is and is not an OS ==

=== Group 1 ===
{| class="wikitable" width="100%"
|+ '''BOINC'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Very parallel
| Can handle only very specific (trivially parallelizable) types of problems
|-
| Availability
| Abstraction layer is poor
|-
| Scheduling
|-
| Networked
|-
| Nodes may come and leave as they like (it is active when the computer is idle)
|-
| Same problem at the same time
|-
| Redundant/Fault tolerant
|-
| Allocation can be handle by system
|-
| Large problem managed by one system
|-
| Anyone can submit projects
|}

=== Group 2 ===
{| class="wikitable" width="100%"
|+ '''Facebook'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| OK abstraction (for human communication/control)
| Doesn't really take a single resources and split it up into smaller ones
|-
| The programming API is stable
| The human API (user interface) is not stable
|-
| Able to separate resources such as wall posts, photos, etc
| Limited control
|}

=== Group 3 ===
{| class="wikitable" width="100%"
|+ '''Google Docs (Drive)'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Has a file system
| Somehow still not a true OS
|-
| Provides hardware abstraction (users don't care how the requests are carried out)
|-
| Has an API
|}

=== Group 4 ===
{| class="wikitable" width="100%"
|+ '''LinkedIn'''
|-
! Similarities to a traditional OS
! Differences from a traditional OS
|-
| Supports multiple location/users
| Specific functionality (not abstract)
|-
| Multiple servers around the world
|-
| Resource/Security management
|-
| Networked
|-
| Cross-platform (Desktop/Mobile)
|}