Revision as of 02:58, 13 October 2010

Question

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

Note to Group

If you have yet to participate in the discussion, please at least leave your name and notify us on what you plan on working on.

Answer

Newton OS

In progress...

BeOS

Working on it...

WinFS

History Pre-WinFS

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove NTFS file system and instead use a relational object-oriented file storage which was based on SQL server 8.0. It was supposed to be implemented in windows 2003 Server. But then, in 2000 Windows announced that Storage + was to be forgotten and Relational File System (RFS) was his successor. RFS was supposed to be included in SQL server 2000, but never made the cut. A factor that we can't forget, in 2000, Oracle announced a new file system which was a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.

We heard in 2002 about a new file system that would be present in Longhorn, renamed Vista. The system would be once again based on its predecessor, in this case RFS, but it would have as requirement to run on top of a NTFS file system. WinFS was included in a few public build for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable later, but was definitely cut for good in 2006 from Vista.

Brief concept of WinFS

With today’s data, we are facing a crisis of finding what we want when we want at a reasonable speed on our own computer. The amount of file extension we can find is astonishing. We can’t name half of them nor know what they do. We store data in many different ways simple file with different extension, in different kinds of database which make finding, relating and acting quite difficult to achieve.

Microsoft had an idea to solve this problem, by using a relational database as file System, where data would be simply data. To understand how WinFS file system works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in normal database, but there are multiple relations between tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to " Enable people to Find, Relate, and Act on their information." As already pointed out, data files are so broad nowadays, with that many file formats using complicated data storage methods our " current file system does not know how to collect and find information within these new types of data." But with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relate to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time find all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies and whatever else we would want find in the same request. We have to search for them one by one. If we treat data as data, once again, we can simple search in the tables of the database the pictures, received emails, videos, documents for the name Bob, and present them to the user. As we can see, data can be related to each other with key word, in our example it was the name Bob, but it can be anything. What if data could do action that follow specific rules? This is exactly what WinFS wanted to implement. " WinFS Rules are a built-in component of the system that allows you to tell the system how to work with, sort, and deliver your data". Also, it could make use of other applications on the system. So if we received a picture from our friend Bob, we could automatically transfer it elsewhere.

The last point that WinFs was aiming for, was to run WinFS on top of NTFS. Basically, WinFS would scan all the data in NTFS file system, and put it into its database. Thus it would work as a file system, but it would be totally dependent of NTFS.

Fate of WinFS

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package but instead deliver it into the next MS SQL server, which was SQL server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release." but instead, they will keep working on it and some "may be used by other Microsoft products going forward." So we can see that it didn't become a file system, but some of the logistic will go into Microsoft’s database software.

Why

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answer that " No. In fact, the Beta was coming together really well. " He then replied that the technology used wasn't easy to build on, so they had to rewrite some parts, but that wouldn't have caused the end of it. Some others also speculate that no serious software used it, nor did it receive the attention needed from the developer to have a good start.

Palm OS

History

Graffiti, a hand-recognition software which was believed to be doomed from the very beginning became Palm OS. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it and so there was no companies that wanted to make the hardware for it. The company Palm then decided to make their own hardware with Graffiti as the OS which is known as Palm OS today.

Database

Palm OS does not use a relational or XML database. It has two types, record and resource database.

Record Database

A record database is a collection of records (relocatable blocks of memory). Each record can only store up to 64KB of memory. The record database has a header which stores information onto the database this information, includes the creator ID, its name, version, etc. Also the database has information that is unique to the record, the location of the record, a unique id, and an attribute which contains delete, dirty, busy, and secret bit [1]. The database is stored in the storage heap (where permanent data resides).

Resource Database

Palm OS applications are stored in the resource database. One of the resources in this database contains code, another resource contains the application’s name, and another the application’s icon, and the rest contains the forms, alerts, menus, strings, and other elements of the application [1]. The resources come directly from the storage heap. The difference between the record database and resource database is how the information is stored and how it is named. Resource database no longer stores unique Ids and attributes instead it stores an ID number and a type (four-character constant). Resources are called by using the ID number and the type.

Problems

Record and resource database are stored in storage/database heap which is stored in the RAM (Random Access Memory). Database heap has a limit of 64KB of memory and a record had to be small enough to fit in there, this made memory hard to manage. A problem arose in versions before OS 3.0, because database heap was divided into smaller parts this allowed for free space in memory but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap.

Ex: A record is 50kb in size and there are 2 database heaps which can only hold 64KB each, a total of 128KB of memory. Lets say these two database heaps are half full therefore there is 64kb of memory is occupied and 64KB of free memory, but the record of size 40kb cannot be stored because a record has to be stored into a database heap and each of these heaps actually have 32kb of free memory.

Reference

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

I don't know how to properly show the references

@@ Line 62: / Line 62: @@
 ===Problems===
 Record and resource database are stored in storage/database heap which is stored in the RAM (Random Access Memory). Database heap has a limit of 64KB of memory and a record had to be small enough to fit in there, this made memory hard to manage. A problem arose in versions before OS 3.0, because database heap was divided into smaller parts this allowed for free space in memory but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap.
+Ex: A record is 50kb in size and there are 2 database heaps which can only hold 64KB each, a total of 128KB of memory. Lets say these two database heaps are half full therefore there is 64kb of memory is occupied and 64KB of free memory, but the record of size 40kb cannot be stored because a record has to be stored into a database heap and each of these heaps actually have 32kb of free memory.
 ===Reference===