COMP 3000 Essay 1 2010 Question 11

Question

Why are object stores an increasingly attractive building block for filesystems (as opposed to block-based stores)? Explain.

Answer

Introduction

Each year we are faced with growing storage needs as the world's information increases exponentially and business' are increasingly choosing to archive and retain all the data they produce. The storage industry has been able to keep up with demand with matching increases in storage capacity. Unfortunately the interfaces between clients and storage devices has remained unchanged since the 1950's. The dominate storage mechanism is still block-based storage technology. This has been sufficient for meeting most needs of modern businesses, but as we enter an age where "store everything, forever" is the common mantra of storage administrators and unstructured data with little meta-data is the norm, we have to look for technology that can provide better scalability, business intelligence, and management while ensuring security and data access speed of traditional storage solutions.

Enter object based storage devices (OSD). Object storage uses objects that consists of data and meta-data that describe the object. They are accessed with defined methods such as read and write and carry a unique ID. They manage all necessary low-level storage, space management, and security functions. This storage technology has the potential to address some of the problems with block-based storage.

With increased scalability, better security through per-object level access and insured integrity of data with unique hash key's for each object along with some benefits in management and business intelligence with rich meta-data, OSD can be seen as a viable alternative to improve the standard architectures of storage area network (SAN) and network-attached storage (NAS).

Overview of Block-Based Storage

Hard disks as a storage medium date back to the 1950's with the introduction of the IBM 350 disk storage unit.^[1] Hard disks store data in blocks, which are a fixed length series' of bytes. Since early devices like the IBM 350, the interface that the operating system uses to communicate with the hard disk has remained mostly the same.^[2] This interface simply allows the operating system to read or write to blocks on the disk. This means that the goal of abstracting stored data into related groups or into human-understandable constructs such as objects or files is left completely in the space of the operating system's filesystem. For example, when the filesystem wants to write data to a file it must translate that into what block on the disk to write to. In this way, the scope of a filesystem extends from high level constructs like files to low level constructs like blocks. This wide scope is necessary because of the simple interface presented to the filesystem that must be abstracted up to the complex expectations of a user.

Multiple standards exist to implement this interface. The small computer system interface (SCSI) standards, which have been around in one form or another since the late 1970s, are popular with industry. Parallel ATA, another standard which was designed in the 1980s, continues today in the form of Serial ATA (SATA). However, even though these standards have been around for a long time, "the logical interface, or the command set, has seen only minor additions"^[3](Bandulet). This means that the functionality that the command set allows has also remained mostly the same, since the functionality must be built on top of these commands.

Comparison of object and block based stores

Scalability

The Association of Storage Networking Professionals estimates that there were over 1 million full-time or part-time storage administrators in 2004. A survey of over one thousand ASNP members indicates that 20% of them manage over 100 terabytes of data. Many Fortune 500 companies are known to be approaching 1 petabyte of data, that is, assuming 100 gigabytes per drive, 10,000 individual drives. ^[4]

Most block based storage systems contain many layers of metadata. There are also various types of virtualized systems that contain metadata to deal with device diversity or remapping of blocks for archiving or duplication. Manageability of metadata becomes more and more complex and has a major impact on scalability. We see this in SAN and NAS systems.

Although SAN filesystems have the benefits of shared access for scalability, coordination of this shared access leads to scalability problems. File systems must coordinate allocation of blocks and for clients to share read-write access, they must coordinate usage of data blocks through metadata.

NAS systems use metadata to map blocks into files and manage file security within a single system. This is done through a single NAS head which usually has thousands of gigabytes of storage behind it. ^[5] Metadata is shared among separate metadata servers remote from the hosts. Space allocation management on different storage system layers and applications that add policy and management metadata individually is spread throughout the system. Metadata becomes very hard to manage.

With an OSD interface, however, metadata is associated and stored directly with each data object and is automatically carried between layers and across devices. Space allocation and management metadata are the responsibility of the storage device. ^[6] This allows metadata layers to be folded, reducing server overhead and processing, and allows for larger clusters of storage compared with traditional block-based interfaces.

Integrity

Block based file systems in archive solutions usually have no built in mechanisms for assuring data integrity. A common best practice is to conduct frequent backups, which adds to the complexity of using file systems for archiving and scalability. The mechanisms for ensuring data integrity in OSDs have mechanisms that operate differently from block store systems.

One of the major problems with storage at the block level is that if there is an error in a block, it is almost impossible to determine what part of the file system is affected. It may be the case that the error in a particular block may not even contain any data. This usually happens during a backup procedure or when a controller is organizing data.

OSDs provide a level of abstraction that hides the fact that a disk device has blocks. It no longer matters to the file system manager what kind of disk drive is being used, it only worries about managing objects. This is done through managing metadata as well as maintaining internal copies of its metadata. Hence, OSDs have knowledge of its object layout even though one or more groups of objects are on different OSDs. In this way OSDs know what kind of space is being used or unused and can scan and correct errors without losing data. In the event of a failure in recovering a file or a number of files, traditional systems may have to do a complete file system restore. However, an OSDs awareness of its object layout enables it to recover data specific to a byte range and thus restore files in an efficient manner.

References

[1] IBM 350 Disk Storage Unit

[2] M. Mesnier, G. R. Ganger, and E. Riedel. Object-Based Storage. IEEE Communications Magazine, 41(8), August 2003.

[3] Object-Based Storage Devices Christian Bandulet, July 2007

[4] Seagate

[5] Foundations of Network Storage

[6] Dell Object Storage Overview