Talk:COMP 3000 Essay 1 2010 Question 11
Quick Overview
So I hope i'm not the only one who was wondering "What are object stores?" when reading the question. I don't think the textbook mentions it but I didn't read through the filesystems chapter very thoroughly. Here's where some quick googling has got me:
Most storage devices divide their storage up into blocks, a fixed length sequence of bytes. The interface that storage devices provide to the rest of the system is pretty simple. It's essentially "Here, you can read to or write to blocks, have fun". This is block-based storage.
Object-based storage is different. The interface it presents to the rest of the system is more sophisticated. Instead of directly accessing blocks on the disk, the system accesses objects. Objects are like a level of abstraction on top of blocks. Objects can be variable sized, read/written to, created, and deleted. The device itself handles mapping these objects to blocks and all the issues that come with that, rather than the OS.
Here's some papers that give an overview of object-based storage:
Object Storage: The Future Building Block for Storage Systems
I think if you just look those up on google scholar you can access the pdf without even being inside carleton's network.
--Mbingham 23:56, 1 October 2010 (UTC)
Some more links
I haven't been reading many academic papers on the subject so those links will be very useful.
If I may add to this. I read articles on object storage here:
and
I can add that metadata is much richer in an object store context. Searching for files and grouping related files together is much easier with the context information that metadata supplies for objects. I'm beginning to read:
--Myagi 10:39, 5 October 2010 (UTC)
I'm going to write a version of my essay out over the long weekend with headings and references and put it up on the wiki. I'd like to know who and how many people are working on this essay but dunno if that's possible. We'll see what we do from there I guess? I was thinking we just homogenize all of the information we write into one unified essay.
--Myagi 10:42, 6 October 2010 (UTC)
- I think there's 6 people in our group, though there might only be 5. I'll be working on this over the long weekend too. I was thinking maybe we should try to get a rough outline up, thursday or friday. Since Prof Somayaji mentioned that this should have the format of an essay, maybe we could start with what our main argument is?
- I was thinking something like objects stores are becoming more attractive because the demands on filesystems has changed, but the interface has not been updated to accomodate these changes. Then we could go into an explanation of block based storage, how it fails to meet the needs placed on modern FSs, then how object stores solves these problems. What do you think?
- --Mbingham 01:55, 7 October 2010 (UTC)
- You don't need to write your own independent essay on the wiki. Let's just add info as it comes along. I'll be completely without internet access this weekend, but I'll try to bring some background reading with me. Expect lots of edits from me starting Monday night/Tuesday morning.
- --Dagar 12:59, 7 October 2010 (UTC)
- Sounds good! I think that's a good idea for a thesis statement and we should have a concrete one by Thurs/Fri. Although I'm not absolutely clear about the interface not being updated? I think the object store SCSI standard is constantly being ratified and now they have an OSD-3 draft. T10 OSD Working Drafts. But then again I'm probably misunderstanding something...
- --Myagi 10:08, 7 October 2010 (UTC)
- I didn't mean that the object interface hadn't been updated, I meant that the block interface hasn't been updated to reflect the changing requirements put on storage. Since the block interface is still largely the same as it was decades ago (read/write to blocks) it is unable to handle the new requirements. Object stores look attractive because they are designed to deal with issues like scalability, integrity, security, etc. Sorry for the confusion, I hope it makes more sense now, haha.
- --Mbingham 15:44, 7 October 2010 (UTC)
- I gotcha, thanks for explaining! I'd say that would be a great thesis statement then: Object stores are becoming more attractive because the demands on filesystems has changed and the block store interface has not been updated to accommodate these changes. We can work from there. I think we can address the inadequacies of block based storage after stating our thesis and then for the body, we point out how object stores deal with issues of scalability, integrity, security as well as flexibility. And then some kind of nice tie up reiterating our thesis.
- --Myagi 12:50, 7 October 2010 (UTC)
I mine as well put my contribution here. I'm willing to move or change it for the sake of organizing this discussion page.
Essay outline
Introduction
- Thesis Statement: Object stores are becoming more attractive because the demands on filesystems has changed and the block store interface has not been updated to accommodate these changes.
- What will be discussed
- Current state of block based storage - Brief overview of object store - Scalability - Integrity - Security
Block based storage
Overview of OSD
Scalability
- Metadata is associated and stored directly with data objects and carried between layers and across devices
- Space allocation delegated to storage device
- Reduced overhead and processing by server allowing larger clusters of storage
Integrity
- OSD's have knowledge of its object layout
- Unlike block stores, OSD's can recover data specific to a byte range
- OSD's know what space is being unused in this way - Can scan and correct errors without losing data
- OSD's maintain internal copies of metadata
- User doesn't have to do a complete file system restore for the sake of one or few unrecoverable files - OSD's can identify the byte range lost and restore the file efficiently
Security
- Suited for network based storage
- Associate security attributes directly with data object
- Security requests handled directly by storage device
- Computer system can access OSD device by providing cryptographically secure credentials(capability) that the OSD device can validate
- This can prevent malicious access from unauthorized requests or accidental access from misconfigured machines
Conclusion
- Reiteration of thesis statement
--Myagi 18:15, 7 October 2010 (UTC)
Other
-instead of storing filesytems in terms of blocks, you store in terms of objects.
-extents, named extents
-objects fancier because they can move around.
-extra level of abstraction and indirection
-files made of objects, objects made of blocks