https://homeostasis.scs.carleton.ca/wiki/api.php?action=feedcontributions&feedformat=atom&user=Nelaturuk 2026-05-02T02:27:33Z User contributions MediaWiki 1.42.1 https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_24&diff=19004 2014-04-08T14:39:29Z

<p>Nelaturuk: /* Features */</p> <hr /> <div>===The Landscape of Parallel Computing Research: A View from Berkeley===<br /> * What sort of applications can you expect to run on distributed OS/parallize?<br /> * How do you scale up<br /> * We can't rely on processor improvements to provide speed-ups<br /> * The proposed computational models that need more processor power don't really apply to regular<br /> * Users would see the advances with games primarily<br /> * More reliance in cloud computing in recent years<br /> <br /> ==7 Dwarfs==<br /> * Dense Linear Algebra<br /> ** Hard to parallize<br /> * Sparse Linear Algebra<br /> * Spectral Methods<br /> * N-Body Methods<br /> * Structured Grids<br /> * Unstructured Grids<br /> * Monte Carlo<br /> ==Extended Dwarfs==<br /> * Combinational Logic<br /> * Graph Traversal<br /> * Dynamic Programming<br /> * Backtrack/Branch + Bound<br /> * Construct Graphical Models<br /> * Finite State Machines<br /> <br /> ===Features===<br /> * Pretty impressive on getting everyone to sign off on the report<br /> * Connection to MapReduce <br /> * Programs that run on distributed operating systems - applications that can be expected to be massively parallel - what sort of computational model is needed - Abstractions needed on top of the stack. <br /> * Predictions about the processing power<br /> * GPU's do have 1000 or more cores<br /> * Desktop cores have not gotten that fast over the past years. They just don't run fast enough. <br /> * Games are the only things that can't be run over the time on single thread<br /> * Low power <br /> * Being able to run a smart phone with 100's of transistors - stalled with the sequential processing<br /> * Why do we need the additional processing power for ? - Games - Games - Games<br /> * Doomsday of the IT industry <br /> * Massive change in mobile and cloud over the past five years<br /> * Linux a very general operating system when it started at first - hard coded to 8086 processor specialized to run on the box. Now it runs everywhere. It has all the abstractions dealing with various aspects of hardware, architecture. Multiple layers abstraction because it was useful.</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_24&diff=19003 2014-04-08T14:39:10Z

<p>Nelaturuk: /* Features */</p> <hr /> <div>===The Landscape of Parallel Computing Research: A View from Berkeley===<br /> * What sort of applications can you expect to run on distributed OS/parallize?<br /> * How do you scale up<br /> * We can't rely on processor improvements to provide speed-ups<br /> * The proposed computational models that need more processor power don't really apply to regular<br /> * Users would see the advances with games primarily<br /> * More reliance in cloud computing in recent years<br /> <br /> ==7 Dwarfs==<br /> * Dense Linear Algebra<br /> ** Hard to parallize<br /> * Sparse Linear Algebra<br /> * Spectral Methods<br /> * N-Body Methods<br /> * Structured Grids<br /> * Unstructured Grids<br /> * Monte Carlo<br /> ==Extended Dwarfs==<br /> * Combinational Logic<br /> * Graph Traversal<br /> * Dynamic Programming<br /> * Backtrack/Branch + Bound<br /> * Construct Graphical Models<br /> * Finite State Machines<br /> <br /> ===Features===<br /> * Pretty impressive on getting everyone to sign off on the report<br /> * Connection to MapReduce <br /> * Programs that run on distributed operating systems - applications that can be expected to be massively parallel - what sort of computational model is needed - Abstractions needed on top of the stack. <br /> * Predictions about the processing power<br /> * GPU's do have 1000 or more cores<br /> * Desktop cores have not gotten that fast over the past years. They just don't run fast enough. <br /> * Games are the only things that can't be run over the time on single thread<br /> * Low power <br /> * Being able to run a smart phone with 100's of transistors - stalled with the sequential processing<br /> * Why do we need the additional processing power for ? - Games - Games - Games<br /> * Doomsday of the IT industry <br /> * Massive change in mobile and cloud over the past five years<br /> * Linux a very general operating system when it started at first - hard coded to 8086 processor specialized to run on the box. Now it runs everywhere. It has all the abstractions dealing with various aspects of hardware, architecture.</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_24&diff=19002 2014-04-08T14:38:31Z

<p>Nelaturuk: /* Features */</p> <hr /> <div>===The Landscape of Parallel Computing Research: A View from Berkeley===<br /> * What sort of applications can you expect to run on distributed OS/parallize?<br /> * How do you scale up<br /> * We can't rely on processor improvements to provide speed-ups<br /> * The proposed computational models that need more processor power don't really apply to regular<br /> * Users would see the advances with games primarily<br /> * More reliance in cloud computing in recent years<br /> <br /> ==7 Dwarfs==<br /> * Dense Linear Algebra<br /> ** Hard to parallize<br /> * Sparse Linear Algebra<br /> * Spectral Methods<br /> * N-Body Methods<br /> * Structured Grids<br /> * Unstructured Grids<br /> * Monte Carlo<br /> ==Extended Dwarfs==<br /> * Combinational Logic<br /> * Graph Traversal<br /> * Dynamic Programming<br /> * Backtrack/Branch + Bound<br /> * Construct Graphical Models<br /> * Finite State Machines<br /> <br /> ===Features===<br /> * Pretty impressive on getting everyone to sign off on the report<br /> * Connection to MapReduce <br /> * Programs that run on distributed operating systems - applications that can be expected to be massively parallel - what sort of computational model is needed - Abstractions needed on top of the stack. <br /> * Predictions about the processing power<br /> * GPU's do have 1000 or more cores<br /> * Desktop cores have not gotten that fast over the past years. They just don't run fast enough. <br /> * Games are the only things that can't be run over the time on single thread<br /> * Low power <br /> * Being able to run a smart phone with 100's of transistors - stalled with the sequential processing<br /> * Why do we need the additional processing power for ? - Games - Games - Games<br /> * Doomsday of the IT industry <br /> * Massive change in mobile and cloud over the past five years<br /> * Linux a very general operating system when it started at first - hard coded to 8086 processor specialized to run on the box</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_24&diff=19001 2014-04-08T14:31:50Z

<p>Nelaturuk: /* Features */</p> <hr /> <div>===The Landscape of Parallel Computing Research: A View from Berkeley===<br /> * What sort of applications can you expect to run on distributed OS/parallize?<br /> * How do you scale up<br /> * We can't rely on processor improvements to provide speed-ups<br /> * The proposed computational models that need more processor power don't really apply to regular<br /> * Users would see the advances with games primarily<br /> * More reliance in cloud computing in recent years<br /> <br /> ==7 Dwarfs==<br /> * Dense Linear Algebra<br /> ** Hard to parallize<br /> * Sparse Linear Algebra<br /> * Spectral Methods<br /> * N-Body Methods<br /> * Structured Grids<br /> * Unstructured Grids<br /> * Monte Carlo<br /> ==Extended Dwarfs==<br /> * Combinational Logic<br /> * Graph Traversal<br /> * Dynamic Programming<br /> * Backtrack/Branch + Bound<br /> * Construct Graphical Models<br /> * Finite State Machines<br /> <br /> ===Features===<br /> * Pretty impressive on getting everyone to sign off on the report<br /> * Connection to MapReduce <br /> * Programs that run on distributed operating systems - applications that can be expected to be massively parallel - what sort of computational model is needed - Abstractions needed on top of the stack. <br /> * Predictions about the processing power<br /> * GPU's do have 1000 or more cores<br /> * Desktop cores have not gotten that fast over the past years. They just don't run fast enough. <br /> * Games are the only things that can't be run over the time on single thread<br /> * Low power <br /> * Being able to run a smart phone with 100's of transistors - stalled with the sequential processing<br /> * Why do we need the additional processing power for ? - Games - Games - Games<br /> * Doomsday of the IT industry <br /> * Massive change in mobile and cloud over the past five years</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_24&diff=19000 2014-04-08T14:31:36Z

<p>Nelaturuk: </p> <hr /> <div>===The Landscape of Parallel Computing Research: A View from Berkeley===<br /> * What sort of applications can you expect to run on distributed OS/parallize?<br /> * How do you scale up<br /> * We can't rely on processor improvements to provide speed-ups<br /> * The proposed computational models that need more processor power don't really apply to regular<br /> * Users would see the advances with games primarily<br /> * More reliance in cloud computing in recent years<br /> <br /> ==7 Dwarfs==<br /> * Dense Linear Algebra<br /> ** Hard to parallize<br /> * Sparse Linear Algebra<br /> * Spectral Methods<br /> * N-Body Methods<br /> * Structured Grids<br /> * Unstructured Grids<br /> * Monte Carlo<br /> ==Extended Dwarfs==<br /> * Combinational Logic<br /> * Graph Traversal<br /> * Dynamic Programming<br /> * Backtrack/Branch + Bound<br /> * Construct Graphical Models<br /> * Finite State Machines<br /> <br /> ===Features===<br /> * Pretty impressive on getting everyone to sign off on the report<br /> * Connection to MapReduce <br /> * Programs that run on distributed operating systems - applications that can be expected to be massively parallel - what sort of computational model is needed - Abstractions needed on top of the stack. <br /> * Predictions about the processing power<br /> * GPU's do have 1000 or more cores<br /> * Desktop cores have not gotten that fast over the past years. They just don't run fast enough. <br /> * Games are the only things that can't be run over the time on single thread<br /> * Low power <br /> * Being able to run a smart phone with 100's of transistors - stalled with the sequential processing<br /> * Why do we need the additional processing power for ? - Games - Games - Games<br /> * Doomsday of the IT industry <br /> * Massive change in mobile and cloud over the past five years<br /> Dwarfs : <br /> * Dense linear algebra - Sparse linear algebra - Spectral methods - Body methods - n-body methods - structured grids - unstructured grids - Monte carlo - Combinational logic - Graph traversal - Dynamic programming - Backtrack/Branch and bound - Construct graphical models - Finite state machines <br /> * Of these some can be programmed parallel and some are suitable for sequential</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18978 2014-04-03T15:18:36Z

<p>Nelaturuk: /* Distributed Stream Processing - Ronak Chaudhari */</p> <hr /> <div>'''Presentations'''<br /> ===Distributed Shared Memory Systems - Mojgan===<br /> * Introduction to DSM systems<br /> * Advantages and Disadvantages<br /> * Classification of DSM systems<br /> * Design considerations<br /> * Examples of DSM systems<br /> - OpenSSI<br /> - Mermaid<br /> - MOSIX<br /> - DDM<br /> <br /> ===Survey: Fault Tolerance in Distributed File System - Mohammed===<br /> * Abstract<br /> * Introductions<br /> ** About fault tolerance in any distributed system. Comparison between different file systems. <br /> ** Whats more suitable for Mobile based systems. <br /> ** Why satisfaction high for fault tolerance is one of the main issues for DFS's ? <br /> * Replication and fault tolerance<br /> ** What is the Replica and Placement policy? What is the synchronization? What is its benefit? <br /> - Synchronous Method<br /> - Asynchronous Method<br /> - Semi-Asynchronous Method<br /> * Cache consistency and fault tolerance<br /> ** What is the cache? What is its benefit? Cache consistency? <br /> - Write only Read Many (WORM)<br /> - Transactional Locking - Read and write locks<br /> - Leasing<br /> * Example DFS mentioned in the paper<br /> ** Google File Systems<br /> ** HDFS <br /> ** MOOSEFS<br /> ** iRODS<br /> ** GlusterFS<br /> ** Lustre<br /> ** Ceph<br /> ** PARADISE for mobile<br /> * Conclusion<br /> <br /> ===Survey on Control Plane Frameworks for Software Defined Networking - Sijo===<br /> * Introduction<br /> ** Traditional Networks - Control Plane and Forwarding Plane<br /> ** Software Defined Networking<br /> - Proposes decoupling of layers into independent layers<br /> - Network entities or nodes are specialized elements which does the forwarding <br /> - Control applications do not need to worry about installation of the underlying network<br /> * Theme, Argument Outline<br /> - Look at various control frameworks proposed<br /> * Controller Platforms<br /> - Centralized and Distributed approaches<br /> - Identify the need to use in controller platforms<br /> - For centralized it started with NOX - Maestro - Beacon - Floodlight - POX - OpenDayLight<br /> - For Distributed : ONIX - Hyperflow - YANC - ONOS<br /> - Leverage parallel processing capabilities<br /> * In detail about two systems: <br /> ** ONIX<br /> ** ONOS <br /> * References<br /> <br /> ===Metadata management in Distributed File System - Sandarbh===<br /> * What is metadata? <br /> - Define by bare-minimum functions for MDS (Metadata Server)<br /> - Monitor the performance of DFS so that it can be used further<br /> - Structure of metadata in Paper<br /> * Why is Metadata management difficult? <br /> - 50% file operations are metadata operations<br /> - Size of metadata<br /> - Distribute the load evenly across all MDS<br /> - Be able to handle thousands of clients <br /> - Be able to handle file/directory permission change<br /> - Recover data if some MDS goes down<br /> - Be POSIX compliant<br /> - Be able to scale- addition of new MDS shoudn't cause ripples<br /> - Contrasting goals - replication and consistency - Average case improvements vs guaranteed performance for each access<br /> * Static sub-tree partitioning<br /> - Advantage - Clients know which MDS to contact for the file - Prefix caching <br /> - Disadvantage - Directory hot spot formation<br /> * Static hashing based partitioning <br /> - Hash the filename or File identifier and assign to MDS<br /> - Advantage - Distributes load evenly - Gets rid of hotpsot info<br /> - Disadvantage <br /> * Don't ask me where your server is approach<br /> - Ex : Ceph , GlusterFS, OceanStore, Hierarchical Bloom filters, Cassandra<br /> - Responsibilities - Replica mgmt, Consistency, Access control, Recover metadata in case of crash, Talk to each other to handle the load dynamically <br /> * What's not in the slides <br /> - Not focused on replication of metadata<br /> - Semantic based search<br /> * Structure of the survey<br /> - Conventional metadata systems<br /> - No-metadata approach<br /> - Metadata approach of the file systems designed for specific goals 0 GFS, Haystack etcs<br /> - Evolution history<br /> - Comparison with in ctageory<br /> - Cover reliability and consistency part<br /> - Summarize learnings with expected trends<br /> <br /> ===Distributed Stream Processing - Ronak Chaudhari===<br /> * About Stream processing<br /> - Data streams <br /> - DBMS vs Stream processing <br /> * Applications<br /> - Monitoring applications<br /> - Militia applications<br /> - Financial analysis<br /> - Tracking applications<br /> * Aurora <br /> - Process incoming streams<br /> - It has its own query algebra<br /> - System Model - Query Model - Runtime Architecture <br /> - QOS criteria<br /> - SQuAL - Query algebra<br /> - Aurora GUI<br /> - Challenges in distribute operation<br /> * Aurora vs Medusa<br /> * Medusa<br /> - Architecture<br /> - Addition to Aurora - Lookup and Brain<br /> - Failure detection<br /> - Transfer of processing <br /> - System API<br /> - Load management <br /> - High availability <br /> - Benefits<br /> * References</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18977 2014-04-03T15:18:18Z

<p>Nelaturuk: /* Ronak Chaudhari */</p> <hr /> <div>'''Presentations'''<br /> ===Distributed Shared Memory Systems - Mojgan===<br /> * Introduction to DSM systems<br /> * Advantages and Disadvantages<br /> * Classification of DSM systems<br /> * Design considerations<br /> * Examples of DSM systems<br /> - OpenSSI<br /> - Mermaid<br /> - MOSIX<br /> - DDM<br /> <br /> ===Survey: Fault Tolerance in Distributed File System - Mohammed===<br /> * Abstract<br /> * Introductions<br /> ** About fault tolerance in any distributed system. Comparison between different file systems. <br /> ** Whats more suitable for Mobile based systems. <br /> ** Why satisfaction high for fault tolerance is one of the main issues for DFS's ? <br /> * Replication and fault tolerance<br /> ** What is the Replica and Placement policy? What is the synchronization? What is its benefit? <br /> - Synchronous Method<br /> - Asynchronous Method<br /> - Semi-Asynchronous Method<br /> * Cache consistency and fault tolerance<br /> ** What is the cache? What is its benefit? Cache consistency? <br /> - Write only Read Many (WORM)<br /> - Transactional Locking - Read and write locks<br /> - Leasing<br /> * Example DFS mentioned in the paper<br /> ** Google File Systems<br /> ** HDFS <br /> ** MOOSEFS<br /> ** iRODS<br /> ** GlusterFS<br /> ** Lustre<br /> ** Ceph<br /> ** PARADISE for mobile<br /> * Conclusion<br /> <br /> ===Survey on Control Plane Frameworks for Software Defined Networking - Sijo===<br /> * Introduction<br /> ** Traditional Networks - Control Plane and Forwarding Plane<br /> ** Software Defined Networking<br /> - Proposes decoupling of layers into independent layers<br /> - Network entities or nodes are specialized elements which does the forwarding <br /> - Control applications do not need to worry about installation of the underlying network<br /> * Theme, Argument Outline<br /> - Look at various control frameworks proposed<br /> * Controller Platforms<br /> - Centralized and Distributed approaches<br /> - Identify the need to use in controller platforms<br /> - For centralized it started with NOX - Maestro - Beacon - Floodlight - POX - OpenDayLight<br /> - For Distributed : ONIX - Hyperflow - YANC - ONOS<br /> - Leverage parallel processing capabilities<br /> * In detail about two systems: <br /> ** ONIX<br /> ** ONOS <br /> * References<br /> <br /> ===Metadata management in Distributed File System - Sandarbh===<br /> * What is metadata? <br /> - Define by bare-minimum functions for MDS (Metadata Server)<br /> - Monitor the performance of DFS so that it can be used further<br /> - Structure of metadata in Paper<br /> * Why is Metadata management difficult? <br /> - 50% file operations are metadata operations<br /> - Size of metadata<br /> - Distribute the load evenly across all MDS<br /> - Be able to handle thousands of clients <br /> - Be able to handle file/directory permission change<br /> - Recover data if some MDS goes down<br /> - Be POSIX compliant<br /> - Be able to scale- addition of new MDS shoudn't cause ripples<br /> - Contrasting goals - replication and consistency - Average case improvements vs guaranteed performance for each access<br /> * Static sub-tree partitioning<br /> - Advantage - Clients know which MDS to contact for the file - Prefix caching <br /> - Disadvantage - Directory hot spot formation<br /> * Static hashing based partitioning <br /> - Hash the filename or File identifier and assign to MDS<br /> - Advantage - Distributes load evenly - Gets rid of hotpsot info<br /> - Disadvantage <br /> * Don't ask me where your server is approach<br /> - Ex : Ceph , GlusterFS, OceanStore, Hierarchical Bloom filters, Cassandra<br /> - Responsibilities - Replica mgmt, Consistency, Access control, Recover metadata in case of crash, Talk to each other to handle the load dynamically <br /> * What's not in the slides <br /> - Not focused on replication of metadata<br /> - Semantic based search<br /> * Structure of the survey<br /> - Conventional metadata systems<br /> - No-metadata approach<br /> - Metadata approach of the file systems designed for specific goals 0 GFS, Haystack etcs<br /> - Evolution history<br /> - Comparison with in ctageory<br /> - Cover reliability and consistency part<br /> - Summarize learnings with expected trends<br /> <br /> ===Distributed Stream Processing - Ronak Chaudhari===<br /> * About Stream processing<br /> - Data streams <br /> - DBMS vs Stream processing <br /> * Applications<br /> - Monitoring applications<br /> - Militia applications<br /> - Financial analysis<br /> - Tracking applications<br /> * Aurora <br /> - Process incoming streams<br /> - It has its own query algebra<br /> - System Model - Query Model - Runtime Architecture <br /> - QOS criteria<br /> - SQuAL - Query algebra<br /> - Aurora GUI<br /> - Challenges in distribute operation<br /> * Aurora vs Medusa<br /> * Medusa<br /> - Architecture<br /> - Addition to Aurora - Lookup and Brain<br /> - Failure detection<br /> - Transfer of processing <br /> - System API<br /> - Load management <br /> - High availability <br /> - Benefits</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18976 2014-04-03T15:05:08Z

<p>Nelaturuk: /* Sandarbh */</p> <hr /> <div>'''Presentations'''<br /> ===Distributed Shared Memory Systems - Mojgan===<br /> * Introduction to DSM systems<br /> * Advantages and Disadvantages<br /> * Classification of DSM systems<br /> * Design considerations<br /> * Examples of DSM systems<br /> - OpenSSI<br /> - Mermaid<br /> - MOSIX<br /> - DDM<br /> <br /> ===Survey: Fault Tolerance in Distributed File System - Mohammed===<br /> * Abstract<br /> * Introductions<br /> ** About fault tolerance in any distributed system. Comparison between different file systems. <br /> ** Whats more suitable for Mobile based systems. <br /> ** Why satisfaction high for fault tolerance is one of the main issues for DFS's ? <br /> * Replication and fault tolerance<br /> ** What is the Replica and Placement policy? What is the synchronization? What is its benefit? <br /> - Synchronous Method<br /> - Asynchronous Method<br /> - Semi-Asynchronous Method<br /> * Cache consistency and fault tolerance<br /> ** What is the cache? What is its benefit? Cache consistency? <br /> - Write only Read Many (WORM)<br /> - Transactional Locking - Read and write locks<br /> - Leasing<br /> * Example DFS mentioned in the paper<br /> ** Google File Systems<br /> ** HDFS <br /> ** MOOSEFS<br /> ** iRODS<br /> ** GlusterFS<br /> ** Lustre<br /> ** Ceph<br /> ** PARADISE for mobile<br /> * Conclusion<br /> <br /> ===Survey on Control Plane Frameworks for Software Defined Networking - Sijo===<br /> * Introduction<br /> ** Traditional Networks - Control Plane and Forwarding Plane<br /> ** Software Defined Networking<br /> - Proposes decoupling of layers into independent layers<br /> - Network entities or nodes are specialized elements which does the forwarding <br /> - Control applications do not need to worry about installation of the underlying network<br /> * Theme, Argument Outline<br /> - Look at various control frameworks proposed<br /> * Controller Platforms<br /> - Centralized and Distributed approaches<br /> - Identify the need to use in controller platforms<br /> - For centralized it started with NOX - Maestro - Beacon - Floodlight - POX - OpenDayLight<br /> - For Distributed : ONIX - Hyperflow - YANC - ONOS<br /> - Leverage parallel processing capabilities<br /> * In detail about two systems: <br /> ** ONIX<br /> ** ONOS <br /> * References<br /> <br /> ===Metadata management in Distributed File System - Sandarbh===<br /> * What is metadata? <br /> - Define by bare-minimum functions for MDS (Metadata Server)<br /> - Monitor the performance of DFS so that it can be used further<br /> - Structure of metadata in Paper<br /> * Why is Metadata management difficult? <br /> - 50% file operations are metadata operations<br /> - Size of metadata<br /> - Distribute the load evenly across all MDS<br /> - Be able to handle thousands of clients <br /> - Be able to handle file/directory permission change<br /> - Recover data if some MDS goes down<br /> - Be POSIX compliant<br /> - Be able to scale- addition of new MDS shoudn't cause ripples<br /> - Contrasting goals - replication and consistency - Average case improvements vs guaranteed performance for each access<br /> * Static sub-tree partitioning<br /> - Advantage - Clients know which MDS to contact for the file - Prefix caching <br /> - Disadvantage - Directory hot spot formation<br /> * Static hashing based partitioning <br /> - Hash the filename or File identifier and assign to MDS<br /> - Advantage - Distributes load evenly - Gets rid of hotpsot info<br /> - Disadvantage <br /> * Don't ask me where your server is approach<br /> - Ex : Ceph , GlusterFS, OceanStore, Hierarchical Bloom filters, Cassandra<br /> - Responsibilities - Replica mgmt, Consistency, Access control, Recover metadata in case of crash, Talk to each other to handle the load dynamically <br /> * What's not in the slides <br /> - Not focused on replication of metadata<br /> - Semantic based search<br /> * Structure of the survey<br /> - Conventional metadata systems<br /> - No-metadata approach<br /> - Metadata approach of the file systems designed for specific goals 0 GFS, Haystack etcs<br /> - Evolution history<br /> - Comparison with in ctageory<br /> - Cover reliability and consistency part<br /> - Summarize learnings with expected trends<br /> <br /> ===Ronak Chaudhari===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18975 2014-04-03T14:50:12Z

<p>Nelaturuk: /* Survey on Control Plane Frameworks for Software Defined Networking - Sijo */</p> <hr /> <div>'''Presentations'''<br /> ===Distributed Shared Memory Systems - Mojgan===<br /> * Introduction to DSM systems<br /> * Advantages and Disadvantages<br /> * Classification of DSM systems<br /> * Design considerations<br /> * Examples of DSM systems<br /> - OpenSSI<br /> - Mermaid<br /> - MOSIX<br /> - DDM<br /> <br /> ===Survey: Fault Tolerance in Distributed File System - Mohammed===<br /> * Abstract<br /> * Introductions<br /> ** About fault tolerance in any distributed system. Comparison between different file systems. <br /> ** Whats more suitable for Mobile based systems. <br /> ** Why satisfaction high for fault tolerance is one of the main issues for DFS's ? <br /> * Replication and fault tolerance<br /> ** What is the Replica and Placement policy? What is the synchronization? What is its benefit? <br /> - Synchronous Method<br /> - Asynchronous Method<br /> - Semi-Asynchronous Method<br /> * Cache consistency and fault tolerance<br /> ** What is the cache? What is its benefit? Cache consistency? <br /> - Write only Read Many (WORM)<br /> - Transactional Locking - Read and write locks<br /> - Leasing<br /> * Example DFS mentioned in the paper<br /> ** Google File Systems<br /> ** HDFS <br /> ** MOOSEFS<br /> ** iRODS<br /> ** GlusterFS<br /> ** Lustre<br /> ** Ceph<br /> ** PARADISE for mobile<br /> * Conclusion<br /> <br /> ===Survey on Control Plane Frameworks for Software Defined Networking - Sijo===<br /> * Introduction<br /> ** Traditional Networks - Control Plane and Forwarding Plane<br /> ** Software Defined Networking<br /> - Proposes decoupling of layers into independent layers<br /> - Network entities or nodes are specialized elements which does the forwarding <br /> - Control applications do not need to worry about installation of the underlying network<br /> * Theme, Argument Outline<br /> - Look at various control frameworks proposed<br /> * Controller Platforms<br /> - Centralized and Distributed approaches<br /> - Identify the need to use in controller platforms<br /> - For centralized it started with NOX - Maestro - Beacon - Floodlight - POX - OpenDayLight<br /> - For Distributed : ONIX - Hyperflow - YANC - ONOS<br /> - Leverage parallel processing capabilities<br /> * In detail about two systems: <br /> ** ONIX<br /> ** ONOS <br /> * References<br /> <br /> ===Sandarbh===<br /> ===Ronak Chaudhari===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18974 2014-04-03T14:47:34Z

<p>Nelaturuk: /* Sijo */</p> <hr /> <div>'''Presentations'''<br /> ===Distributed Shared Memory Systems - Mojgan===<br /> * Introduction to DSM systems<br /> * Advantages and Disadvantages<br /> * Classification of DSM systems<br /> * Design considerations<br /> * Examples of DSM systems<br /> - OpenSSI<br /> - Mermaid<br /> - MOSIX<br /> - DDM<br /> <br /> ===Survey: Fault Tolerance in Distributed File System - Mohammed===<br /> * Abstract<br /> * Introductions<br /> ** About fault tolerance in any distributed system. Comparison between different file systems. <br /> ** Whats more suitable for Mobile based systems. <br /> ** Why satisfaction high for fault tolerance is one of the main issues for DFS's ? <br /> * Replication and fault tolerance<br /> ** What is the Replica and Placement policy? What is the synchronization? What is its benefit? <br /> - Synchronous Method<br /> - Asynchronous Method<br /> - Semi-Asynchronous Method<br /> * Cache consistency and fault tolerance<br /> ** What is the cache? What is its benefit? Cache consistency? <br /> - Write only Read Many (WORM)<br /> - Transactional Locking - Read and write locks<br /> - Leasing<br /> * Example DFS mentioned in the paper<br /> ** Google File Systems<br /> ** HDFS <br /> ** MOOSEFS<br /> ** iRODS<br /> ** GlusterFS<br /> ** Lustre<br /> ** Ceph<br /> ** PARADISE for mobile<br /> * Conclusion<br /> <br /> ===Survey on Control Plane Frameworks for Software Defined Networking - Sijo===<br /> * Introduction<br /> ** Traditional Networks - Control Plane and Forwarding Plane<br /> ** Software Defined Networking<br /> - Proposes decoupling of layers into independent layers<br /> - Network entities or nodes are specialized elements which does the forwarding <br /> - Control applications do not need to worry about installation of the underlying network<br /> * Theme, Argument Outline<br /> - Look at various control frameworks proposed<br /> * Controller Platforms<br /> - Centralized and Distributed approaches<br /> - Identify the need to use in controller platforms<br /> - For centralized it started with NOX - Maestro - Beacon - Floodlight - POX - OpenDayLight<br /> - For Distributed : ONIX - Hyperflow - YANC - ONOS<br /> - Leverage parallel processing capabilities<br /> * In detail about two systems: <br /> ** ONIX<br /> ** ONOS<br /> * References<br /> <br /> ===Sandarbh===<br /> ===Ronak Chaudhari===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18973 2014-04-03T14:36:03Z

<p>Nelaturuk: /* Survey: Fault Tolerance in Distributed File System - Mohammed */</p> <hr /> <div>'''Presentations'''<br /> ===Distributed Shared Memory Systems - Mojgan===<br /> * Introduction to DSM systems<br /> * Advantages and Disadvantages<br /> * Classification of DSM systems<br /> * Design considerations<br /> * Examples of DSM systems<br /> - OpenSSI<br /> - Mermaid<br /> - MOSIX<br /> - DDM<br /> <br /> ===Survey: Fault Tolerance in Distributed File System - Mohammed===<br /> * Abstract<br /> * Introductions<br /> ** About fault tolerance in any distributed system. Comparison between different file systems. <br /> ** Whats more suitable for Mobile based systems. <br /> ** Why satisfaction high for fault tolerance is one of the main issues for DFS's ? <br /> * Replication and fault tolerance<br /> ** What is the Replica and Placement policy? What is the synchronization? What is its benefit? <br /> - Synchronous Method<br /> - Asynchronous Method<br /> - Semi-Asynchronous Method<br /> * Cache consistency and fault tolerance<br /> ** What is the cache? What is its benefit? Cache consistency? <br /> - Write only Read Many (WORM)<br /> - Transactional Locking - Read and write locks<br /> - Leasing<br /> * Example DFS mentioned in the paper<br /> ** Google File Systems<br /> ** HDFS <br /> ** MOOSEFS<br /> ** iRODS<br /> ** GlusterFS<br /> ** Lustre<br /> ** Ceph<br /> ** PARADISE for mobile<br /> * Conclusion<br /> <br /> ===Sijo===<br /> ===Sandarbh===<br /> ===Ronak Chaudhari===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18972 2014-04-03T14:31:52Z

<p>Nelaturuk: /* Survey: Fault Tolerance in Distributed File System - Mohammed */</p> <hr /> <div>'''Presentations'''<br /> ===Distributed Shared Memory Systems - Mojgan===<br /> * Introduction to DSM systems<br /> * Advantages and Disadvantages<br /> * Classification of DSM systems<br /> * Design considerations<br /> * Examples of DSM systems<br /> - OpenSSI<br /> - Mermaid<br /> - MOSIX<br /> - DDM<br /> <br /> ===Survey: Fault Tolerance in Distributed File System - Mohammed===<br /> * Abstract<br /> * Introductions<br /> ** About fault tolerance in any distributed system. Comparison between different file systems. <br /> ** Whats more suitable for Mobile based systems. <br /> ** Why satisfaction high for fault tolerance is one of the main issues for DFS's ? <br /> * Replication and fault tolerance<br /> ** What is the Replica and Placement policy? What is the synchronization? What is its benefit? <br /> - Synchronous Method<br /> - Asynchronous Method<br /> - Semi-Asynchronous Method<br /> * Cache consistency and fault tolerance<br /> ** What is the cache? What is its benefit? Cache consistency? <br /> - Write only Read Many (WORM)<br /> - Transactional Locking<br /> - Leasing<br /> * Conclusion<br /> <br /> ===Sijo===<br /> ===Sandarbh===<br /> ===Ronak Chaudhari===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18971 2014-04-03T14:27:58Z

<p>Nelaturuk: /* Mohammed */</p> <hr /> <div>'''Presentations'''<br /> ===Distributed Shared Memory Systems - Mojgan===<br /> * Introduction to DSM systems<br /> * Advantages and Disadvantages<br /> * Classification of DSM systems<br /> * Design considerations<br /> * Examples of DSM systems<br /> - OpenSSI<br /> - Mermaid<br /> - MOSIX<br /> - DDM<br /> <br /> ===Survey: Fault Tolerance in Distributed File System - Mohammed===<br /> * Abstract<br /> * Introductions<br /> ** About fault tolerance in any distributed system. Comparison between different file systems. Whats more suitable for Mobile based systems. Why satisfaction high for fault tolerance is one of the main issues for DFS's ? <br /> <br /> * Replication and fault tolerance<br /> * Cache consistency and fault tolerance<br /> * Conclusion<br /> <br /> ===Sijo===<br /> ===Sandarbh===<br /> ===Ronak Chaudhari===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18970 2014-04-03T14:23:50Z

<p>Nelaturuk: /* Mojgan */</p> <hr /> <div>'''Presentations'''<br /> ===Distributed Shared Memory Systems - Mojgan===<br /> * Introduction to DSM systems<br /> * Advantages and Disadvantages<br /> * Classification of DSM systems<br /> * Design considerations<br /> * Examples of DSM systems<br /> - OpenSSI<br /> - Mermaid<br /> - MOSIX<br /> - DDM<br /> <br /> ===Mohammed===<br /> ===Sijo===<br /> ===Sandarbh===<br /> ===Ronak Chaudhari===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18969 2014-04-03T14:01:24Z

<p>Nelaturuk: /* Presentations */</p> <hr /> <div>'''Presentations'''<br /> ===Mojgan===<br /> ===Mohammed===<br /> ===Sijo===<br /> ===Sandarbh===<br /> ===Ronak Chaudhari===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18968 2014-04-03T13:59:55Z

<p>Nelaturuk: /* Mohammed */</p> <hr /> <div>===Presentations===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18967 2014-04-03T13:59:48Z

<p>Nelaturuk: /* Sijo */</p> <hr /> <div>===Presentations===<br /> ===Mohammed===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18966 2014-04-03T13:59:40Z

<p>Nelaturuk: /* Sandarbh */</p> <hr /> <div>===Presentations===<br /> ===Mohammed===<br /> ===Sijo===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18965 2014-04-03T13:59:33Z

<p>Nelaturuk: /* =Ronak Chaudhari */</p> <hr /> <div>===Presentations===<br /> ===Mohammed===<br /> ===Sijo===<br /> ===Sandarbh===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18964 2014-04-03T13:59:25Z

<p>Nelaturuk: /* Mojgan */</p> <hr /> <div>===Presentations===<br /> ===Mohammed===<br /> ===Sijo===<br /> ===Sandarbh===<br /> ===Ronak Chaudhari==</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_23&diff=18963 2014-04-03T13:58:55Z

<p>Nelaturuk: Created page with &quot;===Presentations=== ===Mojgan=== ===Mohammed=== ===Sijo=== ===Sandarbh=== ===Ronak Chaudhari==&quot;</p> <hr /> <div>===Presentations===<br /> ===Mojgan===<br /> ===Mohammed===<br /> ===Sijo===<br /> ===Sandarbh===<br /> ===Ronak Chaudhari==</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_22&diff=18961 2014-04-02T19:09:18Z

<p>Nelaturuk: /* Object Based Storage Systems - Keerthi Nelaturu */</p> <hr /> <div>===Presentations===<br /> ===Gehana===<br /> <br /> ===Object Based Storage Systems - Keerthi Nelaturu===<br /> '''Abstract'''<br /> Object-based storage implementations have been there since 1990’s. Mostly this architecture is used for developing scalable distributed file systems. It is surprising that we Object Oriented concepts can be used even in storage systems. <br /> <br /> '''Project Report Outline'''<br /> This paper starts with a comparison on why we would prefer object-based storage systems to File or Block storage systems. A basic architecture will be presented discussing the common elements like security, load balancing, object attributes management methodology etc. that are part of this storage system. This will include an analysis on some of the other object storage systems like HYDRA, LUSTRE, SOSS, OBFS and Differstore.<br /> Further into the paper, we would concentrate more on how object-based storage is useful for implementing massive image processing systems like Haystack and OBSI.<br /> <br /> ===Simon &amp; Peter===<br /> ===Adam===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_22&diff=18960 2014-04-02T19:08:49Z

<p>Nelaturuk: /* Bo */</p> <hr /> <div>===Presentations===<br /> ===Gehana===<br /> <br /> ===Object Based Storage Systems - Keerthi Nelaturu===<br /> '''Abstract'''<br /> Object-based storage implementations have been there since 1990’s. Mostly this architecture is used for developing scalable distributed file systems. It is surprising that we Object Oriented concepts can be used even in storage systems. <br /> <br /> Project Report Outline: <br /> This paper starts with a comparison on why we would prefer object-based storage systems to File or Block storage systems. A basic architecture will be presented discussing the common elements like security, load balancing, object attributes management methodology etc. that are part of this storage system. This will include an analysis on some of the other object storage systems like HYDRA, LUSTRE, SOSS, OBFS and Differstore.<br /> Further into the paper, we would concentrate more on how object-based storage is useful for implementing massive image processing systems like Haystack and OBSI.<br /> <br /> ===Simon &amp; Peter===<br /> ===Adam===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_22&diff=18959 2014-04-02T19:08:15Z

<p>Nelaturuk: /* Object Based Storage Systems - Keerthi Nelaturu */</p> <hr /> <div>===Presentations===<br /> ===Gehana===<br /> <br /> ===Object Based Storage Systems - Keerthi Nelaturu===<br /> '''Abstract'''<br /> Object-based storage implementations have been there since 1990’s. Mostly this architecture is used for developing scalable distributed file systems. It is surprising that we Object Oriented concepts can be used even in storage systems. <br /> <br /> Project Report Outline: <br /> This paper starts with a comparison on why we would prefer object-based storage systems to File or Block storage systems. A basic architecture will be presented discussing the common elements like security, load balancing, object attributes management methodology etc. that are part of this storage system. This will include an analysis on some of the other object storage systems like HYDRA, LUSTRE, SOSS, OBFS and Differstore.<br /> Further into the paper, we would concentrate more on how object-based storage is useful for implementing massive image processing systems like Haystack and OBSI.<br /> <br /> ===Simon &amp; Peter===<br /> ===Bo===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_22&diff=18958 2014-04-02T19:07:43Z

<p>Nelaturuk: /* Presentations */</p> <hr /> <div>===Presentations===<br /> ===Gehana===<br /> <br /> ===Object Based Storage Systems - Keerthi Nelaturu===<br /> '''Abstract'''<br /> Object-based storage implementations have been there since 1990’s. Mostly this architecture is used for developing scalable distributed file systems. It is surprising that we Object Oriented concepts can be used even in storage systems. <br /> <br /> Project Report Outline: <br /> This paper starts with a comparison on why we would prefer object-based storage systems to File or Block storage systems. A basic architecture will be presented discussing the common elements like security, load balancing, object attributes management methodology etc. that are part of this storage system. This will include an analysis on some of the other object storage systems like HYDRA, LUSTRE, SOSS, OBFS and Differstore.<br /> Further into the paper, we would concentrate more on how object-based storage is useful for implementing massive image processing systems like Haystack and OBSI.</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_22&diff=18957 2014-04-02T19:07:20Z

<p>Nelaturuk: /* Presentations */</p> <hr /> <div>===Presentations===<br /> ===Object Based Storage Systems - Keerthi Nelaturu===<br /> '''Abstract'''<br /> Object-based storage implementations have been there since 1990’s. Mostly this architecture is used for developing scalable distributed file systems. It is surprising that we Object Oriented concepts can be used even in storage systems. <br /> <br /> Project Report Outline: <br /> This paper starts with a comparison on why we would prefer object-based storage systems to File or Block storage systems. A basic architecture will be presented discussing the common elements like security, load balancing, object attributes management methodology etc. that are part of this storage system. This will include an analysis on some of the other object storage systems like HYDRA, LUSTRE, SOSS, OBFS and Differstore.<br /> Further into the paper, we would concentrate more on how object-based storage is useful for implementing massive image processing systems like Haystack and OBSI.</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_22&diff=18956 2014-04-02T19:05:17Z

<p>Nelaturuk: Created page with &quot;===Presentations===&quot;</p> <hr /> <div>===Presentations===</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_21&diff=18945 2014-03-27T21:01:15Z

<p>Nelaturuk: /* Test */</p> <hr /> <div><br /> == Presentation ==<br /> <br /> === Marking ===<br /> <br /> * marked mostly on presentation, not content<br /> * basically we want to communicate the basic structure of the paper, and do so in a way that isn't boring<br /> <br /> === Content ===<br /> <br /> * concrete, not &quot;head in the clouds&quot;<br /> * present the area<br /> * compare and contrast the papers<br /> * 10 minutes talk, 5 minutes feedback<br /> * basic argument<br /> * basic references<br /> <br /> === Form ===<br /> <br /> * show the work we've done on paper<br /> * try to get feedback<br /> * think of it as a rough draft<br /> * try to get people to read the paper<br /> * enthusiasm<br /> * powerpoints are easier<br /> * don't read slides<br /> * no whole sentences on slides<br /> * look at talks by Mark Shuttleworth<br /> <br /> == MapReduce ==<br /> <br /> A clever observation that a simple solution could solve most distributed problems. It's all about programming to an abstraction that is efficiently parallelizable. Note that it's not actually a simple solution, because it sits atop a mountain of code. It requires something like BigTables which requires something like GFS, which requires something like Chubby.<br /> <br /> <br /> * Simplification<br /> * Interestingly large scale problems can be implemented with this<br /> * Easy to program, powerful for certain classes, it scales like no ones business. <br /> * Kind of empoweraged model<br /> Programming to an abstraction that is efficiently parllel. We have learnt all about infrastructure until now. <br /> Classic OS abstractions were about files.<br /> <br /> == Naiad ==<br /> <br /> Where MapReduce was suited for a specific family of solutions, Naiad tries to generalize the solution to apply parallelization to a much wider family. Naiad supports MapReduce style solutions, but also many other solutions. However, the tradeoff was simplicity. It's like we took MapReduce and took away its low barrier to entry. The idea is to create a constrained graph that can easily be parallelized.<br /> * Complicated than Map Reduce<br /> * Talks about Timely dataflow graphs <br /> * Its all about Graph algorithms - Graph abstraction<br /> * Restrictions on graphs so that they can be mapped to parllel computation<br /> * How to fit anything to this model is a big question. <br /> * More general than map reduce <br /> * Not very useful.</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_21&diff=18944 2014-03-27T21:00:58Z

<p>Nelaturuk: /* Marking */</p> <hr /> <div><br /> == Presentation ==<br /> <br /> === Marking ===<br /> <br /> * marked mostly on presentation, not content<br /> * basically we want to communicate the basic structure of the paper, and do so in a way that isn't boring<br /> <br /> === Test ===<br /> <br /> === Content ===<br /> <br /> * concrete, not &quot;head in the clouds&quot;<br /> * present the area<br /> * compare and contrast the papers<br /> * 10 minutes talk, 5 minutes feedback<br /> * basic argument<br /> * basic references<br /> <br /> === Form ===<br /> <br /> * show the work we've done on paper<br /> * try to get feedback<br /> * think of it as a rough draft<br /> * try to get people to read the paper<br /> * enthusiasm<br /> * powerpoints are easier<br /> * don't read slides<br /> * no whole sentences on slides<br /> * look at talks by Mark Shuttleworth<br /> <br /> == MapReduce ==<br /> <br /> A clever observation that a simple solution could solve most distributed problems. It's all about programming to an abstraction that is efficiently parallelizable. Note that it's not actually a simple solution, because it sits atop a mountain of code. It requires something like BigTables which requires something like GFS, which requires something like Chubby.<br /> <br /> <br /> * Simplification<br /> * Interestingly large scale problems can be implemented with this<br /> * Easy to program, powerful for certain classes, it scales like no ones business. <br /> * Kind of empoweraged model<br /> Programming to an abstraction that is efficiently parllel. We have learnt all about infrastructure until now. <br /> Classic OS abstractions were about files.<br /> <br /> == Naiad ==<br /> <br /> Where MapReduce was suited for a specific family of solutions, Naiad tries to generalize the solution to apply parallelization to a much wider family. Naiad supports MapReduce style solutions, but also many other solutions. However, the tradeoff was simplicity. It's like we took MapReduce and took away its low barrier to entry. The idea is to create a constrained graph that can easily be parallelized.<br /> * Complicated than Map Reduce<br /> * Talks about Timely dataflow graphs <br /> * Its all about Graph algorithms - Graph abstraction<br /> * Restrictions on graphs so that they can be mapped to parllel computation<br /> * How to fit anything to this model is a big question. <br /> * More general than map reduce <br /> * Not very useful.</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_21&diff=18936 2014-03-27T15:21:51Z

<p>Nelaturuk: /* Naiad */</p> <hr /> <div><br /> == Presentation ==<br /> <br /> === Marking ===<br /> <br /> * marked mostly on presentation, not content<br /> * basically we want to communicate the basic structure of the paper, and do so in a way that isn't boring<br /> <br /> === Content ===<br /> <br /> * concrete, not &quot;head in the clouds&quot;<br /> * present the area<br /> * compare and contrast the papers<br /> * 10 minutes talk, 5 minutes feedback<br /> * basic argument<br /> * basic references<br /> <br /> === Form ===<br /> <br /> * show the work we've done on paper<br /> * try to get feedback<br /> * think of it as a rough draft<br /> * try to get people to read the paper<br /> * enthusiasm<br /> * powerpoints are easier<br /> * don't read slides<br /> * no whole sentences on slides<br /> * look at talks by Mark Shuttleworth<br /> <br /> == MapReduce ==<br /> <br /> A clever observation that a simple solution could solve most distributed problems. It's all about programming to an abstraction that is efficiently parallelizable. Note that it's not actually a simple solution, because it sits atop a mountain of code. It requires something like BigTables which requires something like GFS, which requires something like Chubby.<br /> <br /> <br /> * Simplification<br /> * Interestingly large scale problems can be implemented with this<br /> * Easy to program, powerful for certain classes, it scales like no ones business. <br /> * Kind of empoweraged model<br /> Programming to an abstraction that is efficiently parllel. We have learnt all about infrastructure until now. <br /> Classic OS abstractions were about files.<br /> <br /> == Naiad ==<br /> <br /> Where MapReduce was suited for a specific family of solutions, Naiad tries to generalize the solution to apply parallelization to a much wider family. Naiad supports MapReduce style solutions, but also many other solutions. However, the tradeoff was simplicity. It's like we took MapReduce and took away its low barrier to entry. The idea is to create a constrained graph that can easily be parallelized.<br /> * Complicated than Map Reduce<br /> * Talks about Timely dataflow graphs <br /> * Its all about Graph algorithms - Graph abstraction<br /> * Restrictions on graphs so that they can be mapped to parllel computation<br /> * How to fit anything to this model is a big question. <br /> * More general than map reduce <br /> * Not very useful.</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_21&diff=18935 2014-03-27T15:21:37Z

<p>Nelaturuk: /* MapReduce */</p> <hr /> <div><br /> == Presentation ==<br /> <br /> === Marking ===<br /> <br /> * marked mostly on presentation, not content<br /> * basically we want to communicate the basic structure of the paper, and do so in a way that isn't boring<br /> <br /> === Content ===<br /> <br /> * concrete, not &quot;head in the clouds&quot;<br /> * present the area<br /> * compare and contrast the papers<br /> * 10 minutes talk, 5 minutes feedback<br /> * basic argument<br /> * basic references<br /> <br /> === Form ===<br /> <br /> * show the work we've done on paper<br /> * try to get feedback<br /> * think of it as a rough draft<br /> * try to get people to read the paper<br /> * enthusiasm<br /> * powerpoints are easier<br /> * don't read slides<br /> * no whole sentences on slides<br /> * look at talks by Mark Shuttleworth<br /> <br /> == MapReduce ==<br /> <br /> A clever observation that a simple solution could solve most distributed problems. It's all about programming to an abstraction that is efficiently parallelizable. Note that it's not actually a simple solution, because it sits atop a mountain of code. It requires something like BigTables which requires something like GFS, which requires something like Chubby.<br /> <br /> <br /> * Simplification<br /> * Interestingly large scale problems can be implemented with this<br /> * Easy to program, powerful for certain classes, it scales like no ones business. <br /> * Kind of empoweraged model<br /> Programming to an abstraction that is efficiently parllel. We have learnt all about infrastructure until now. <br /> Classic OS abstractions were about files.<br /> <br /> == Naiad ==<br /> <br /> Where MapReduce was suited for a specific family of solutions, Naiad tries to generalize the solution to apply parallelization to a much wider family. Naiad supports MapReduce style solutions, but also many other solutions. However, the tradeoff was simplicity. It's like we took MapReduce and took away its low barrier to entry. The idea is to create a constrained graph that can easily be parallelized.</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_20&diff=18916 2014-03-25T15:28:06Z

<p>Nelaturuk: /* Cassandra */</p> <hr /> <div><br /> == Cassandra ==<br /> <br /> Cassandra is essentially running a BigTable interface on top of a Dynamo infrastructure. BigTable uses GFS' built-in replication and Chubby for locking. Cassandra uses gossip algorithms: [http://dl.acm.org/citation.cfm?id=1529983 Scuttlebutt]. Apache Zookeeper is used for distributed configuration.<br /> <br /> Casandra - <br /> GFS type cluster which big table depends on <br /> Lighter weight <br /> All most of the readings are part of Apache<br /> More designed for online updates for interactive lower latency <br /> Once they write to disk they only read back<br /> Scalable multi master database with no single point of failure<br /> Reason for not giving out the complete detail on the table schema<br /> Probably not just inbox search<br /> All data in one row of a table <br /> Its not a key-value store. Big blob of data. <br /> Gossip based protocol - Scufflebutt<br /> Fixed circular ring <br /> Consistency issue not addressed at all. Does writes in an immutable way. Never change them. <br /> <br /> Discussions<br /> <br /> Athero GPL<br /> Cassandra vs Big table<br /> BigTable is not part of Hadoop. <br /> BigTable &quot; Funny thing on top of GFS&quot;<br /> History of HDFS<br /> Hbase<br /> Why two projects with same notion are supported?<br /> Apache as a community. For any tool in CS particularly software tools, its actually important to have more <br /> <br /> than one good implementation. Only time it doesnt happen because of market realities. <br /> Older style network protocol - token rings<br /> What sort of computational systems avoid changing data? <br /> Functional programming languages - <br /> Whats different from classic c and c++. Tries to eliminate side effects. Functional there is no assignment.Data is just binded you associate a name with a value. Garbage collection. No mutation of data. <br /> Systems talking about implementing functional like semantics.<br /> <br /> == Comet ==<br /> <br /> The major idea behind Comet is triggers/callbacks. There is an extensive literature in extensible operating systems, basically adding code to the operating system to better suit my application. &quot;Generally, extensible systems suck.&quot; -[[User:Soma]]<br /> <br /> [https://www.usenix.org/conference/osdi10/comet-active-distributed-key-value-store The presentation video of Comet]<br /> <br /> Cassandra<br /> + <br /> Cassandra is essentially running a BigTable interface on top of a Dynamo infrastructure. BigTable uses GFS' built-in replication and Chubby for locking. Cassandra uses gossip algorithms: [http://dl.acm.org/citation.cfm?id=1529983 Scuttlebutt]. Apache Zookeeper is used for distributed configuration.<br /> <br /> − <br /> Google developed its technology interbally and used for competitive advantage.<br /> + <br /> == Comet ==<br /> − <br /> Facebook developed its technology in open source manner.<br /> + <br /> <br /> − <br /> Gpl 3 you have to provide code with binary<br /> + <br /> [https://www.usenix.org/conference/osdi10/comet-active-distributed-key-value-store The presentation video of Comet]<br /> − <br /> In AGPL addtional service also be given with source code.<br /> + <br /> − <br /> <br /> + <br /> − <br /> Bigtable needs gfs.cassandra depends on server's file system.Anil feels cassndra cluster is easy to setup.<br /> + <br /> − <br /> <br /> + <br /> − <br /> Bigtable is designed for batch updates.cassandra is for hqbdling realtime stuff.<br /> + <br /> − <br /> <br /> + <br /> − <br /> Schema design is explained in inbox example.it does does not give clarity about how table will look like.Anil thinks they store lot data with messages which makes table crappy.<br /> + <br /> − <br /> <br /> + <br /> − <br /> Cassandra is design for high speed access and online operation.<br /> + <br /> − <br /> <br /> + <br /> − <br /> Zookeeper is similar to chhuby<br /> + <br /> − <br /> Zookeeper is for node level information<br /> + <br /> − <br /> Gossip is more about key partitioning<br /> + <br /> − <br /> <br /> + <br /> − <br /> Zookeeper is for configuration of new node.<br /> + <br /> − <br /> <br /> + <br /> − <br /> It writes in immutable way like functional programming.there is no assignment in functional programming.</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_15&diff=18738 2014-03-06T20:57:51Z

<p>Nelaturuk: Designing Exercise</p> <hr /> <div>'''Designing Exercise'''<br /> <br /> Can we do any kind of distributed system without crypto? We can't trust crypto...<br /> <br /> What are the main features we need to consider for such a system ? <br /> *Limited Sharing<br /> *Integrity<br /> *Availability<br /> <br /> Perhaps probabilistically...<br /> <br /> Want to be able to put data in, have it distributed, and be able to get it out on some other machine. This kind of sharing would need identification or authentication process.<br /> <br /> Availability: &quot;distribute the crap out of it&quot;, doesn't need crypto. No corruption of data. <br /> <br /> Integrity: hashing, but we assume hashes can be forged. If we want to know that we got the same file, then simply send each other the file and compare.<br /> <br /> '''Note on Project Proposal''' <br /> * Date has been extended until next week. As Prof said some of the proposals are not completely up to mark.</div>

Nelaturuk https://homeostasis.scs.carleton.ca/wiki/index.php?title=DistOS_2014W_Lecture_6&diff=18478 2014-01-23T18:12:43Z

<p>Nelaturuk: Discussion on &quot;The Early Web&quot;</p> <hr /> <div><br /> <br /> == Group Discussion on &quot;The Early Web ==<br /> <br /> <br /> Questions to discuss:<br /> <br /> : 1. How do you think the web would have been if not like the present way? <br /> : 2. What kind of infrastructure changes would you like to make? <br /> <br /> '''Group 1'''<br /> : Relatively satisfied with the present structure of the web some changes suggested are in the below areas: <br /> * Make use of the greater potential of Protocols <br /> * More communication and interaction capabilities.<br /> * Implementation changes in the present payment method systems. Example usage of &quot;Micro-computation&quot; - a discussion we would get back to in future classes. Also, Cryptographic currencies.<br /> * Augmented reality.<br /> * More towards individual privacy. <br /> <br /> '''Group 2'''<br /> * Information to be classified in detail <br /> ** Organize things on web. Ex: Yahoo indexers<br /> ** Suggestion for the need of Universal Decimal System an idea by Paul Otlet to be considered. <br /> ** In the end it comes to semantic web<br /> * Information redundancy<br /> * Information verification<br /> <br /> '''Group 3'''<br /> * What we want to keep <br /> ** Linking mechanisms<br /> ** Minimum permissions to publish<br /> * What we don't like<br /> ** Relying on one source for document <br /> ** Privacy links for security<br /> * Proposal <br /> ** Peer-peer to distributed mechanisms for documenting<br /> ** Reverse links with caching - distributed cache<br /> ** More availability for user - what happens when system fails? <br /> ** Key management to be considered - Is it good to have centralized or distributed mechanism? <br /> <br /> '''Group 4'''<br /> * An idea of web searching for us <br /> * A suggestion of a different web if it would have been implemented by &quot;AI&quot; people<br /> ** AI programs searching for data - A notion already being implemented by Google slowly.<br /> * Generate report forums<br /> * HTML equivalent is inspired by the AI communication<br /> * Higher semantics apart from just indexing the data<br /> ** Problem : &quot;How to bridge the semantic gap?&quot;<br /> ** Search for more data patterns</div>

Nelaturuk