DistOS 2014W Lecture 23: Difference between revisions
Line 58: | Line 58: | ||
* References | * References | ||
===Sandarbh=== | ===Metadata management in Distributed File System - Sandarbh=== | ||
* What is metadata? | |||
- Define by bare-minimum functions for MDS (Metadata Server) | |||
- Monitor the performance of DFS so that it can be used further | |||
- Structure of metadata in Paper | |||
* Why is Metadata management difficult? | |||
- 50% file operations are metadata operations | |||
- Size of metadata | |||
- Distribute the load evenly across all MDS | |||
- Be able to handle thousands of clients | |||
- Be able to handle file/directory permission change | |||
- Recover data if some MDS goes down | |||
- Be POSIX compliant | |||
- Be able to scale- addition of new MDS shoudn't cause ripples | |||
- Contrasting goals - replication and consistency - Average case improvements vs guaranteed performance for each access | |||
* Static sub-tree partitioning | |||
- Advantage - Clients know which MDS to contact for the file - Prefix caching | |||
- Disadvantage - Directory hot spot formation | |||
* Static hashing based partitioning | |||
- Hash the filename or File identifier and assign to MDS | |||
- Advantage - Distributes load evenly - Gets rid of hotpsot info | |||
- Disadvantage | |||
* Don't ask me where your server is approach | |||
- Ex : Ceph , GlusterFS, OceanStore, Hierarchical Bloom filters, Cassandra | |||
- Responsibilities - Replica mgmt, Consistency, Access control, Recover metadata in case of crash, Talk to each other to handle the load dynamically | |||
* What's not in the slides | |||
- Not focused on replication of metadata | |||
- Semantic based search | |||
* Structure of the survey | |||
- Conventional metadata systems | |||
- No-metadata approach | |||
- Metadata approach of the file systems designed for specific goals 0 GFS, Haystack etcs | |||
- Evolution history | |||
- Comparison with in ctageory | |||
- Cover reliability and consistency part | |||
- Summarize learnings with expected trends | |||
===Ronak Chaudhari=== | ===Ronak Chaudhari=== |
Revision as of 15:05, 3 April 2014
Presentations
- Introduction to DSM systems
- Advantages and Disadvantages
- Classification of DSM systems
- Design considerations
- Examples of DSM systems
- OpenSSI - Mermaid - MOSIX - DDM
Survey: Fault Tolerance in Distributed File System - Mohammed
- Abstract
- Introductions
- About fault tolerance in any distributed system. Comparison between different file systems.
- Whats more suitable for Mobile based systems.
- Why satisfaction high for fault tolerance is one of the main issues for DFS's ?
- Replication and fault tolerance
- What is the Replica and Placement policy? What is the synchronization? What is its benefit?
- Synchronous Method - Asynchronous Method - Semi-Asynchronous Method
- Cache consistency and fault tolerance
- What is the cache? What is its benefit? Cache consistency?
- Write only Read Many (WORM) - Transactional Locking - Read and write locks - Leasing
- Example DFS mentioned in the paper
- Google File Systems
- HDFS
- MOOSEFS
- iRODS
- GlusterFS
- Lustre
- Ceph
- PARADISE for mobile
- Conclusion
Survey on Control Plane Frameworks for Software Defined Networking - Sijo
- Introduction
- Traditional Networks - Control Plane and Forwarding Plane
- Software Defined Networking
- Proposes decoupling of layers into independent layers - Network entities or nodes are specialized elements which does the forwarding - Control applications do not need to worry about installation of the underlying network
- Theme, Argument Outline
- Look at various control frameworks proposed
- Controller Platforms
- Centralized and Distributed approaches - Identify the need to use in controller platforms - For centralized it started with NOX - Maestro - Beacon - Floodlight - POX - OpenDayLight - For Distributed : ONIX - Hyperflow - YANC - ONOS - Leverage parallel processing capabilities
- In detail about two systems:
- ONIX
- ONOS
- References
Metadata management in Distributed File System - Sandarbh
- What is metadata?
- Define by bare-minimum functions for MDS (Metadata Server) - Monitor the performance of DFS so that it can be used further - Structure of metadata in Paper
- Why is Metadata management difficult?
- 50% file operations are metadata operations - Size of metadata - Distribute the load evenly across all MDS - Be able to handle thousands of clients - Be able to handle file/directory permission change - Recover data if some MDS goes down - Be POSIX compliant - Be able to scale- addition of new MDS shoudn't cause ripples - Contrasting goals - replication and consistency - Average case improvements vs guaranteed performance for each access
- Static sub-tree partitioning
- Advantage - Clients know which MDS to contact for the file - Prefix caching - Disadvantage - Directory hot spot formation
- Static hashing based partitioning
- Hash the filename or File identifier and assign to MDS - Advantage - Distributes load evenly - Gets rid of hotpsot info - Disadvantage
- Don't ask me where your server is approach
- Ex : Ceph , GlusterFS, OceanStore, Hierarchical Bloom filters, Cassandra - Responsibilities - Replica mgmt, Consistency, Access control, Recover metadata in case of crash, Talk to each other to handle the load dynamically
- What's not in the slides
- Not focused on replication of metadata - Semantic based search
- Structure of the survey
- Conventional metadata systems - No-metadata approach - Metadata approach of the file systems designed for specific goals 0 GFS, Haystack etcs - Evolution history - Comparison with in ctageory - Cover reliability and consistency part - Summarize learnings with expected trends