Soma-notes - User contributions [en]

DistOS 2014W Lecture 19

2014-03-20T15:17:15Z

Hariz:

== Dynamo ==

*Key value-store.
*Build a distributed storage system:
*Scale
*Simple: key-value
*Highly available
*Guarantee Service Level Agreements (SLA).
*high concurrent.
* no dynamic routing.
* 0-hop DHT: means it is doe not have information when deliver packet from node to another , it has direct link to the destination
* Dynamo sacrifices consistency under certain failure scenarios.
* it has partition algorithm.
*Consistent hashing: the output range of a hash function is treated as a fixed circular space or “ring”.
*Key is linear and the nodes is partition.
*”Virtual Nodes”: Each node can be responsible for more than one virtual node.
*Each data item is replicated at N hosts.
*“preference list”: The list of nodes that is responsible for storing a particular key.
* Sacrifice strong consistency for availability
* it work with 100 servers,it is not more big.

== Bigtable ==

* BigTable is a distributed storage system for managing structured data.
* Designed to scale to a very large size
* it stores the column together ,the raw is web pages and the column is the contents.
* Each pages have incoming links
* A BigTable is a sparse, distributed persistent multi-dimensional sorted map.
* it have a many columns and it is look as table.
* Each raw has arbitrary column.
* It is multi daimination map.
* An SSTable provides a persistent,ordered immutable map from keys to values, where both keys and values are arbitrary byte strings.
* Large tables broken into tablets at row boundaries and each raw Tablet holds contiguous range of rows.
* Metadata operations: Create/delete tables, column families, change metadata.

== General talk ==

* read the introdutcion and conclusion for each paper and thanking about cases in the paper more than look to how the author solve the proble.

DistOS 2014W Lecture 19

2014-03-20T15:09:27Z

Hariz:

DistOS 2014W Lecture 19

2014-03-20T15:04:22Z

Hariz:

*
== Dynamo ==

*Key value-store.
*Build a distributed storage system:
*Scale
*Simple: key-value
*Highly available
*Guarantee Service Level Agreements (SLA).
*high concurrent.
* no dynamic routing.
* 0-hop DHT: means it is doe not have information when deliver packet from node to another , it has direct link to the destination
* Dynamo sacrifices consistency under certain failure scenarios.
* it has partition algorithm.
*Consistent hashing: the output range of a hash function is treated as a fixed circular space or “ring”.
*Key is linear and the nodes is partition.
*”Virtual Nodes”: Each node can be responsible for more than one virtual node.
*Each data item is replicated at N hosts.
*“preference list”: The list of nodes that is responsible for storing a particular key.
* Sacrifice strong consistency for availability
* it work with 100 servers,it is not more big.

*
== Bigtable ==

* BigTable is a distributed storage system for managing structured data.
* Designed to scale to a very large size
* it stores the column together ,the raw is web pages and the column is the contents.
* Each pages have incoming links
* A BigTable is a sparse, distributed persistent multi-dimensional sorted map.
* it have a many columns and it is look as table.
* Each raw has arbitrary column.
* It is multi daimination map.
* An SSTable provides a persistent,ordered immutable map from keys to values, where both keys and values are arbitrary byte strings.
* Large tables broken into tablets at row boundaries and each raw Tablet holds contiguous range of rows.
* Metadata operations: Create/delete tables, column families, change metadata
*

DistOS 2014W Lecture 19

2014-03-20T14:59:28Z

Hariz:

DistOS 2014W Lecture 19

2014-03-20T14:54:33Z

Hariz:

DistOS 2014W Lecture 19

2014-03-20T14:49:57Z

Hariz:

DistOS 2014W Lecture 19

2014-03-20T14:49:43Z

Hariz:

*
== Dynamo ==

*Key value-store.
*Build a distributed storage system:
*Scale
*Simple: key-value
*Highly available
*Guarantee Service Level Agreements (SLA).
*high concurrent.
* no dynamic routing.
* 0-hop DHT: means it is doe not have information when deliver packet from node to another , it has direct link to the destination
* Dynamo sacrifices consistency under certain failure scenarios.
* it has partition algorithm.
*Consistent hashing: the output range of a hash function is treated as a fixed circular space or “ring”.
*Key is linear and the nodes is partition.
*”Virtual Nodes”: Each node can be responsible for more than one virtual node.
*Each data item is replicated at N hosts.
*“preference list”: The list of nodes that is responsible for storing a particular key.
* Sacrifice strong consistency for availability
* it work with 100 servers,it is not more big.

*Bigtable

* BigTable is a distributed storage system for managing structured data.
* Designed to scale to a very large size
* it stores the column together ,the raw is web pages and the column is the contents.
* Each pages have incoming links
* A BigTable is a sparse, distributed persistent multi-dimensional sorted map.
* it have a many columns and it is look as table.
* Each raw has arbitrary column.
* It is multi daimination map.

DistOS 2014W Lecture 19

2014-03-20T14:49:09Z

Hariz:

* Dynamo

*Key value-store.
*Build a distributed storage system:
*Scale
*Simple: key-value
*Highly available
*Guarantee Service Level Agreements (SLA).
*high concurrent.
* no dynamic routing.
* 0-hop DHT: means it is doe not have information when deliver packet from node to another , it has direct link to the destination
* Dynamo sacrifices consistency under certain failure scenarios.
* it has partition algorithm.
*Consistent hashing: the output range of a hash function is treated as a fixed circular space or “ring”.
*Key is linear and the nodes is partition.
*”Virtual Nodes”: Each node can be responsible for more than one virtual node.
*Each data item is replicated at N hosts.
*“preference list”: The list of nodes that is responsible for storing a particular key.
* Sacrifice strong consistency for availability
* it work with 100 servers,it is not more big.

*Bigtable

* BigTable is a distributed storage system for managing structured data.
* Designed to scale to a very large size
* it stores the column together ,the raw is web pages and the column is the contents.
* Each pages have incoming links
* A BigTable is a sparse, distributed persistent multi-dimensional sorted map.
* it have a many columns and it is look as table.
* Each raw has arbitrary column.
* It is multi daimination map.

DistOS 2014W Lecture 19

2014-03-20T14:44:58Z

Hariz:

DistOS 2014W Lecture 19

2014-03-20T14:43:20Z

Hariz:

DistOS 2014W Lecture 19

2014-03-20T14:29:30Z

Hariz:

* Dynamo

* Key value-store.
-*Build a distributed storage system:
*Scale
*Simple: key-value
*Highly available
*Guarantee Service Level Agreements (SLA).
*high concurrent.
* no dynamic routing.
* 0-hop DHT: means it is doe not have information when deliver packet from node to another , it has direct link to the destination
* Dynamo sacrifices consistency under certain failure scenarios.
* it has partition algorithm.
*Consistent hashing: the output range of a hash function is treated as a fixed circular space or “ring”.
*Key is linear and the nodes is partition.
*”Virtual Nodes”: Each node can be responsible for more than one virtual node.
*Each data item is replicated at N hosts.
*“preference list”: The list of nodes that is responsible for storing a particular key.

DistOS 2014W Lecture 19

2014-03-20T14:19:47Z

Hariz:

DistOS 2014W Lecture 19

2014-03-20T14:15:55Z

Hariz:

* Dynamo

* Key value-store.
-*Build a distributed storage system:
*Scale
*Simple: key-value
*Highly available
*Guarantee Service Level Agreements (SLA).
*high concurrent.

DistOS 2014W Lecture 19

2014-03-20T14:15:05Z

Hariz: Created page with " * Dynamo - Key value-store. - Build a distributed storage system: -Scale -Simple: key-value -Highly available -Guarantee Service Level Agreements (SLA). - high concurrent."

* Dynamo

- Key value-store.
- Build a distributed storage system:
-Scale
-Simple: key-value
-Highly available
-Guarantee Service Level Agreements (SLA).
- high concurrent.

DistOS 2014W Lecture 18

2014-03-18T15:13:54Z

Hariz:

== Distributed Hash Table Overview ==

A Distributed Hash Table (DHT) is a fast lookup structure of <key,value> pairs,
distributed across many nodes in a network. Keys are hashed to generate the
index at which the value can be found. Depending on the nature of the hash
function, only exact queries may be returned. Each node has a partial view of
the hash table, as opposed to a full replica. This has given rise to a number
of different search techniques:
* A centralized server may maintain a list of all keys and associated nodes
at which the value can be found. This method involves a single point of failure.
* Each node may query all connected nodes. This method has performance and
scalability shortcomings
* The keyspace can be partitioned such that nodes will maintain the values
for keys that hash to similar indices (e.g., within a certain hamming distance).
This method is not suitable for all applications, as it involves
bandwidth-intensive migration of data stored in the DHT.

===Tapestry:===
Tapestry is an overlay network which makes use of a DHT to provide routing for
distributed applications. Similar to IP routing, not all nodes need to be
directly connected to each other: they can query a subset of neighbours for
information about which nodes are responsible for certain parts of the keyspace.
Routing is performed in such a way that nodes are aware of their ''distance''
to the object being queried. Hence objects can be located with low latency
without the need to migrate actual object data between nodes. It would be interesting
to see if Tapestry could be adapted for anonymity similar to ant-colony routing
schemes.

**Tapestry
* Distributed .
* Simple key-value store.
* using DHT ( distributed hash table).
* look up table contains : key and value
* DNS as tree but Tapestry as hercically structure.

**More dtails about Tapestry:
** how the information flow?
* each nod has neighbour table which that contains the node neighbour number.

** Tapestry API:
* have four operations called
* each node has ID and each endpoint has GUID (Globally unique identifier).

**Tapestry look like operating system.
* it has two models,one is built on UDP protocol and the other on TCP protocol.

DistOS 2014W Lecture 18

2014-03-18T15:06:49Z

Hariz:

**Tapestry
* Distributed .
* Simple key-value store.
* using DHT ( distributed hash table).
* look up table contains : key and value
* DNS as tree but Tapestry as hercically structure.

**More dtails about Tapestry:
** how the information flow?
* each nod has neighbour table which that contains the node neighbour number.

** Tapestry API:
* have four operations called
* each node has ID and each endpoint has GUID (Globally unique identifier).

**Tapestry look like operating system.

DistOS 2014W Lecture 18

2014-03-18T15:00:18Z

Hariz:

DistOS 2014W Lecture 18

2014-03-18T14:33:16Z

Hariz:

DistOS 2014W Lecture 18

2014-03-18T14:27:24Z

Hariz: Created page with "**Tapestry * Distributed . * Simple key-value store. * using DHT ( distributed hash table). * look up table contains : key and value * DNS as tree but Tapestry as hercically s..."

**Tapestry
* Distributed .
* Simple key-value store.
* using DHT ( distributed hash table).
* look up table contains : key and value
* DNS as tree but Tapestry as hercically structure,

DistOS 2014W Lecture 14

2014-03-04T16:14:13Z

Hariz: /* Technology */

=OceanStore=

==What is the dream?==
* High availabitility, universally available.
* Utility managed by multiple parties
* Highly redundant, fault tolerant
* Basic assumption was that servers would NOT be trusted.

* Highly persistent
** Effective archival
** Everything saved, nothing deleted. "Commits"

* Service was untrusted
** Held opaque data.
* Would be used for more than files. DB's, etc.

==Why did the dream die?==

=== Technology ===
* The trust model is the most attractive feature which ultimately killed it.
** The untrusted assumption was a huge burden on the system. Forced technical
limitations Made them uncompetitive.
** It is just easier to trust a given system
** Every system is compromisable despite this mistrust
* Pub key system reduces usability
** If you loose your key, you're S.O.L.
*security
**there is no security mechanism in servers side.
**can not now who access the data
*economic side
**The economic model is unconvincing as defined. The authors suggest that a collection of companies will host OceanStore servers, and consumers will buy capacity (not unlike web-hosting of today).

===Use Cases===
* Subset of the features already exist
** Black berry. Google.
** Current services owned by one company, not many providers.
** Can not sell back your services as a user.
*** ex. Can not sell your extra storage back to the utility.

==Pond: What insights?==

* They actually built it.

* Can't assume the use of any infrastructure, so they rebuild everything!
** Built over the internet.
** Tapestry
** GUID for object indentification. Object naming scheme.

==Benchmarks==
* Really good read speed, really bad write speed.

===Storage overhead===
* How much are they increasing the storage needed to implement their storage model.
* Factor of 4.8x the space needed (you'll have 1/5th the storage)
* Expensive, but good value (data is backed up, replicated, etc..)

==Update performance==
* No data is mutated. It is diffed and archived.
* Creating a new version of an object and distributing that object.

==Other stuff==
* Byzantine fault tolerance
** Assuming certain actors are malicious

DistOS 2014W Lecture 14

2014-03-04T16:12:57Z

Hariz: /* Technology */

=OceanStore=

==What is the dream?==
* High availabitility, universally available.
* Utility managed by multiple parties
* Highly redundant, fault tolerant
* Basic assumption was that servers would NOT be trusted.

* Highly persistent
** Effective archival
** Everything saved, nothing deleted. "Commits"

* Service was untrusted
** Held opaque data.
* Would be used for more than files. DB's, etc.

==Why did the dream die?==

=== Technology ===
* The trust model is the most attractive feature which ultimately killed it.
** The untrusted assumption was a huge burden on the system. Forced technical
limitations Made them uncompetitive.
** It is just easier to trust a given system
** Every system is compromisable despite this mistrust
* Pub key system reduces usability
** If you loose your key, you're S.O.L.
*security
**there is no security mechanism in servers side.
**can not now who access the data

===Use Cases===
* Subset of the features already exist
** Black berry. Google.
** Current services owned by one company, not many providers.
** Can not sell back your services as a user.
*** ex. Can not sell your extra storage back to the utility.

==Pond: What insights?==

* They actually built it.

* Can't assume the use of any infrastructure, so they rebuild everything!
** Built over the internet.
** Tapestry
** GUID for object indentification. Object naming scheme.

==Benchmarks==
* Really good read speed, really bad write speed.

===Storage overhead===
* How much are they increasing the storage needed to implement their storage model.
* Factor of 4.8x the space needed (you'll have 1/5th the storage)
* Expensive, but good value (data is backed up, replicated, etc..)

==Update performance==
* No data is mutated. It is diffed and archived.
* Creating a new version of an object and distributing that object.

==Other stuff==
* Byzantine fault tolerance
** Assuming certain actors are malicious

DistOS 2014W Lecture 8

2014-01-30T16:19:00Z

Hariz: /* Group 3 */

==Group 1==
nfs:

1) per operating traffic
2) rpc based
3) unreliable

AFS:

1) design for 5000 clients
2) high integrity.

==Group 2==
NFS:

1) more UNIX like
2) portable
3) use UDP
4) it is not minimize network traffic.
5) used VNODE
6) not have much hardware equipment

AFS:

1) better scalability
2) better security.
3) minimize network traffic.
4) less UNIX like

==Group 3==

NFS:

1) cache assumption invalid.

2) no locking

3) bad security

AFS:

1) cache assumption valid

2) locking

3) good security.

==Group 4==

DistOS 2014W Lecture 8

2014-01-30T16:14:25Z

Hariz: /* Group 2 */

DistOS 2014W Lecture 8

2014-01-30T16:10:13Z

Hariz: /* Group 1 */

==Group 1==
nfs:

1) per operating traffic
2) rpc based
3) unreliable

AFS:

1) design for 5000 clients
2) high integrity.

==Group 2==

==Group 3==

==Group 4==

DistOS 2014W Lecture 5

2014-01-23T17:22:54Z

Hariz: /* Class review */

= Introduction =
; Operating system
: The software that turns the computer you have into the one you want (Anil)

* What sort of computer did we want to have?
* What sort of abstractions did they want to be easy? Hard?
* What could we build with the internet (not just WAN, but also LAN)?
* Most dreams people had of their computers smacked into the wall of reality.

= MOAD review in groups =

* Chorded keyboard unfortunately obscure, partly because the attendees disagreed with the long-term investment of training the user.
* View control → hyperlinking system, but in a lightweight (more like nanoweight) markup language.
* Ad-hoc ticketing system
* Ad-hoc messaging system
** Used on a time-sharing systme with shared storage,
* Primitive revision control system
* Different vocabulary:
** Bug and bug smear (mouse and trail)
** Point rather than click

= Class review =

* Doug died Jul 2 2013
* Doug himself called it an “online system”, rather than offline composition of code using card punchers as was common in the day.
* What became of the tech:
** Chorded keyboards:
*** Exist but obscure
** Pre-ARPANET network:
*** Time-sharing mainframe
*** 13 workstations
*** Telephone and television circuit
** Mouse
*** “I sometimes apologize for calling it a mouse”
** Collaborative document editing integrated with screen sharing
** Videoconferencing
*** Part of the vision, but more for the demo at the time,
** Hyperlinks
*** The web on a mainframe
** Languages
*** Metalanguages
**** “Part and parcel of their entire vision of augmenting human intelligence.”
**** You must teach the computer about the language you are using.
**** They were the use case. It was almost designed more for augmenting programmer intelligence rather than human intelligence.
*** It was normal for the time to build new languages (domain-specific) for new systems. Nowadays, we standardize on one but develop large APIs, at the expense of conciseness. We look for short-term benefits; we minimize programmer effort.
*** Compiler compiler
** Freeze-pane
** Folding—Zoomable UI (ZUI)
*** Lots of systems do it, but not the default
*** Much easier to just present everything.
** Technologies the required further investment got left behind.
* The NLS had little to no security
** There was a minimal notion of a user
** There was a utopian aspect. Meanwhile, the Mac had no utopian aspect. Data exchange was through floppies. Any network was small, local, ad-hoc, and among trusted peers.
** The system wasn't envisioned to scale up to masses of people who didn't trust each other.
** How do you enforce secrecy.
* Part of the reason for lack of adoption of some of the tech was hardware. We can posit that a bigger reason would be infrastructure.
* Differentiate usability of system from usability of vision
** What was missing was the polish, the ‘sexiness’, and the intuitiveness of later systems like the Apple II and the Lisa.
** The usability of the later Alto is still less than commercial systems.
*** The word processor was modal, which is apt to confuse unmotivated and untrained users.
* In the context of the Mother of All Demos, the Alto doesn't seem entirely revolutionary. Xerox PARC raided his team. They almost had a GUI; rather they had what we call today a virtual console, with a few things above.
* What happens with visionaries that present a big vision is that the spectators latch onto specific aspects.
* To be comfortable with not adopting the vision, one must ostracize the visionary. People pay attention to things that fit into their world view.
* Use cases of networking have changed little, though the means did
* Fundamentally a resource-sharing system; everything is shared, unlike later systems where you would need to explicitly do so. Resources shared fundamentally sense to share: documents, printers, etc.
* Resource sharing was never enough. '''Information-sharing''' was the focus.

“Mother of all demos” is nickname for Engelbart who could make the computers help humans become smarter.

*More interesting in this works that:
"His idea included seeing computing devices as a means to communicate and retrieve information, rather than just crunch numbers. This idea is represented in NLS”On-Line system”.

*Some information about NLS system:
1) NLS was a revolutionary computer collaboration system from the 1960s.
2) Designed by Douglas Engelbart and implemented by researchers at the Augmentation Research Center (ARC) at the Stanford Research Institute (SRI).
3) The NLS system was the first to employ the practical use of :
a) hypertext links,
b) the mouse,
c) raster-scan video monitors,
d) information organized by relevance,
e) screen windowing,
f) presentation programs,
g) and other modern computing concepts.

= Alto review =

* Fundamentally a personal computer
* Applications:
** Drawing program with curves and arcs for drawing
** Hardware design tools (mostly logic boards)
** Time server
* Less designed for reading than the NLS. More designed around paper. Xerox had a laser printer, and you would read what you printed. Hypertext was deprioritized, unlike the NLS vision had focused on what could not be expressed on paper.
* Xerox had almost an obsession with making documents print beautifully.

User:Hariz

2014-01-21T21:47:46Z

Hariz: Mother of all demos

“Mother of all demos” is nickname for Engelbart who could make the computers help humans become smarter. His idea included seeing computing devices as a means to communicate and retrieve information, rather than just crunch numbers. This idea is represented in NLS”On-Line system”.
Mother of all
NLS was a revolutionary computer collaboration system from the 1960s. Designed by Douglas Engelbart and implemented by researchers at the Augmentation Research Center (ARC) at the Stanford Research Institute (SRI). The NLS system was the first to employ the practical use of hypertext links, the mouse, raster-scan video monitors, information organized by relevance, screen windowing, presentation programs, and other modern computing concepts.

Talk:DistOS 2014W Lecture 5

2014-01-21T21:46:02Z

Hariz: Mother of all demos

“Mother of all demos” is nickname for Engelbart who could make the computers help humans become smarter. His idea included seeing computing devices as a means to communicate and retrieve information, rather than just crunch numbers. This idea is represented in NLS”On-Line system”.

NLS was a revolutionary computer collaboration system from the 1960s. Designed by Douglas Engelbart and implemented by researchers at the Augmentation Research Center (ARC) at the Stanford Research Institute (SRI). The NLS system was the first to employ the practical use of hypertext links, the mouse, raster-scan video monitors, information organized by relevance, screen windowing, presentation programs, and other modern computing concepts.