Soma-notes - User contributions [en]

COMP 3000 Essay 2 2010 Question 5

2010-12-03T02:28:01Z

Dustinmartin: /* Good */

==Paper==
'''Title:''' [http://www.usenix.org/events/osdi10/tech/full_papers/Wu.pdf Bypassing Races in Live Applications with Execution Filters]

'''Authors:''' Jingyue Wu, Heming Cui, Junfeng Yang

'''Affiliations:''' Computer Science Department, Columbia University

'''Supplementary Information:''' Video available [http://homeostasis.scs.carleton.ca/osdi/video/wu.mp4 here] as well as [http://homeostasis.scs.carleton.ca/osdi/slides/wu.pdf slides]

==Background Concepts==
A race condition is a system flaw that “occurs when two threads access a shared variable at the same time." Race conditions can be very complex, time consuming and expensive to fix. Unfortunately, the most challenging part of race condition is not fixing it, but rather finding it. Race conditions are notorious for being extremely difficult to find, isolate and recreate. To help ease this process, the authors of this paper, Jingyue Wu, Heming Cui, Junfeng Yang, propose the adoption of LOOM.

LOOM is a system which dynamically locates areas which may be susceptible to race condition errors and allows the race condition to be potentially fixed. The power of LOOM rests in its ability to operate on live applications in real time. This is possible thanks to its evacuation algorithm which injects execution filters to fix race conditions at runtime. Execution filters, otherwise known as request filtering, allow you to inspect the request before and after the main logic is executed. By leveraging execution filters as the means for correcting race conditions, LOOM is able to operate with very little performance overhead and is a highly scalable as the number of application threads increases.

The authors tested LOOM on existing real world race conditions found in common applications. The tests found that all tested race conditions were solved, with little performance overhead, in a scalable and easy to implement manner.

This paper consists of multiple terms which must be familiar to the reader in order to assist in reading the Bypassing Races in Live Applications with Execution Filters paper. These terms are listed and explained below:

'''Deadlock:''' Deadlocks usually occur within the context of two threads. One thread tries to lock a variable that the other thread has already locked and vice versa. The result of this is that each thread is waiting for each other's thread to release the variable. Thus a deadlock occurs and nothing can happen.

'''Evacuation''' The process of proactively pausing and changing states of code sections so that those sections can be filtered for proper processing

'''Execution Filters:''' Otherwise known as request filtering. Request filters allow you to inspect the request before and after the main logic is executed. These are mutual exclusion filters in the context of this paper.

'''Function Quiescence''' The process of pausing and altering states, in order to avoid race conditions and overlapping between code segments.

'''Hot Patches:''' "Hot patching provides a mechanism to update system files without rebooting or stopping services and processes."[[#References | [1]]]

'''Hybrid Instrumentation Engine:''' "Instrumentation refers to an ability to monitor or measure the level of a product's performance, to diagnose errors and writing trace information." [[#References | [2]]] Instrument programs can have low runtime overhead, but instrumentation has to be done at compile time. Dynamic instrumentation can update programs at runtime but incur high overhead. A hybrid instrumentation is an implementation of combined static and dynamic instrumentation.

'''Lock:''' A lock is a way of limiting access to a common resource when using multiple threads. Lock and unlock methods are usually called at the beginning and end of a target method, respectively. It simply denies access when it is locked, and allows access when not locked. "Mutual exclusion locks (mutexes) are a common method of serializing thread execution. Mutual exclusion locks synchronize threads, usually by ensuring that only one thread at a time executes a critical section of code. Mutex locks can also preserve single-threaded code." [[#References | [3]]]

'''Mutex:''' Mutex (Mutual Exclusion) is an algorithm which prevents race conditions on a resource. Essentially, it forces any threads that are trying to access the resource to wait until the current thread accessing the resource has completed using it.

'''Race Condition:''' "A race condition occurs when two threads access a shared variable at the same time." [[#References | [4]]]

'''Semaphore:''' Semaphores are basically a special type of flag and generalize a down and up state (sleep or wakeup). The down operation checks to see if the value is greater than 0 and if so, decrements the value and uses up one stored wakeup. If the value is 0, the process is put to sleep. These steps are all done in a single indivisible atomic action. It is guaranteed that once a semaphore operation has started, no other process can access the semaphore until the operation has been completed or blocked. Semaphores are an essential part of solving synchronization problems. [[#References | [5]]]

==Research problem==
===Problem being addressed===

With the rise of multiple core systems, multi-threaded programs are often prone to race conditions. Races are hard to detect, test and debug. Many systems are designed to detect, reproduce and diagnose race conditions, but these do not directly address the actual race. This is normally dealt with via a software update which would often require a software restart and potentially introduces new bugs.[[#References | [7]]] Patches require you to be aware of the cause of the problem, and take time to produce, test, and install, leaving a potential user of the software waiting, potentially for months. Due to the immaturity of current race detectors, this paper explains a new approach to race detection and workarounds through the use of LOOM. The goal being to quickly address the actual symptom, the race condition, in real time on live systems. This is in opposition to the conventional approach which focuses on the cause of the race condition, an unknown bug in the software. By targeting the race condition itself, the system is able to keep running without a software update or even a reset.

===Related work===
Two common solutions to fixing deployed races are software updates and hot patches. Software updates require restarts whereas hot patches applies patches to live systems. However, relying on conventional patches can lead to new errors and could be unsafe, due to a multithreaded applications complexity. Releasing a reliable patch takes time, but developers often resort to more efficient fixes rather than placing proper locks in the application due to performance or work pressure.

Using a QUIESCE function to "temporarily suspend...incoming messages on an IUCV path"[[#References | [6]]], these paths can later be reactivated and run as normal. This is not efficient for fixing a race condition because it only delays the problem in an attempt to avoid conflict. Although this does allow for a certain extent of safety it does not come near the reliability and flexibility of LOOM. Speed, reliability, flexibility and ease of use are all areas in which LOOM is demonstrated as being better than a QUIESCE function.

Another similar system to LOOM is STUMP[[#References | [8]]]. STUMP is a system for releasing live updates for multi-threaded or single-threaded programs written in C. It has the ability to provide arbitrary patches to source code in running systems without requiring a reset. These patches require considerable annotation and preparation as source code modifications are considered to be unsafe. Unlike STUMP, LOOM does not operate on the source code and is considered more safe because of this.

The most recent system for live updates to the kernels of Operating Systems is called Ksplice [[#References | [9]]]. It allows users to update the Linux kernel without resetting. This can be done either completely automatically if the code does not chance any data structures, or with on average 17 lines of code for an update that would otherwise require a reset. [[#References | [9]]] It does this by operating on the object layer.

==Contribution==
===Current solution expressed===
Compared to traditional solutions, LOOM differs in its approach to race fixes. It is designed to quickly develop safe, optimized, temporary workarounds while a concrete solution is developed. LOOM is also very easy to use. LOOM is compiled with a developer's application as a plugin and kept separate from the source code. The plugin will inject the LOOM Update Engine into the application binary.

Mutual exclusion filters are written by the developer and synced with the source code to filter out any racy threads. The filter's code declaration is easy to understand and can be inserted in a code region that needs to be mutually exclusive. The developer does not need to deal with low level operations such as lock, unlock and semaphore operations. Users can then download the filter and apply it to the application while it is still live.

LOOM is flexible in that developers can make trade-offs between performance and reliability in their application. These can include making two code regions mutually exclusive when accessing different objects; or, with a significant decrease in performance, making them run in single threaded mode.

An evacuation algorithm is used for safety as to not introduce new errors. A critical region is marked using static analysis. All threads in the critical region are then evacuated. After the evacuation is executed, the execution filter is installed and then the threads are resumed after a live update pause is done at a safe location.

LOOM's hybrid instrumentation engine is used to reduce its overhead. The engine statically changes an application's binary to anticipate dynamic updates.

The evaluation of LOOM was based on overhead, scalability, reliability, availability and timeliness. These were demonstrated using Apache and MySQL in conjunction with the multithreaded ApacheBench and SysBench, respectively.

Through multiple tests the advantages of LOOM were proven. Overhead was tested in a comparison of LOOM during normal runtime. The effects of LOOM on Apache and MySQL were minimal, (~1.83% and ~4% respectively) causing it to be a viable as a runtime fix for race errors. To test scalability the team discovered that on 32 server threads, the overhead was still low: under 3% and 12% respectively. Reliability is one of the strongest facets of the LOOM system as it fixed all of the race conditions studied. To demonstrate LOOM's reliability, it was compared against a conventional restart-based software update. In this test the software update was clearly slower, requiring time to reset itself, where LOOM running a live update had almost no effect on the throughput. Lastly the timeliness of the installation of LOOM's fixes was demonstrated in a simple example. The LOOM based fix completed in 368ms whereas the function quiescence fix took the max test time (1 hour) and was not finished.

====Why is it any better than what came before?====
Previously, the two standard ways of fixing deployed race conditions were system updates and hot patches. LOOM is a superior choice to both these options for a number of reasons.

Unlike LOOM, the system update approach requires that the system be rebooted before the fix can be implemented. With desktop applications, rebooting a system is acceptable. However, servers often cannot reboot because requests are coming from external sources and are expected to be processed.

While hot patches do not require a system reboot, they do have their own specific vulnerabilities. Namely, it is very difficult to apply a patch that corrects the error, or errors, but leaves the rest of the system unaffected. Often when correcting a race condition via a hot patch, others can appear. The main concern with hot patches however, is that their development is a time consuming process. A process which until developed and deployed, leaves the race condition vulnerable and exposed. The paper chronicles a real world Mozilla race condition whose hot patch took nearly 8 years of development to correct. All the while the vulnerability was exposed to Mozilla users.

Flaws common to both system updates and hot patches are that they are very difficult to properly develop, slow to implement, and result in potentially unsafe ad hoc solutions that are not scalable.

Conversely, LOOM is easy to use, fast to implement, highly flexible, scalable, and safe to use

==Critique==
===Good===
The authors of this essay effectively convey their findings by staying focused on the thesis as well as supporting their topics with relevant examples and data. Examples throughout the paper, particularly the MySQL example, ensure that the use of execution filters is clear to the reader. All of the examples are well documented and some (ex. Figure 2) are simplified to avoid confusing the reader with too much unnecessary information. The references throughout backup the reliability of the paper and let the reader to verify information from the sources.

The essay flows well and the information is delivered in a well put together order, allowing the reader to learn enough about LOOM (or any of the sub-topics involved in the explanation) before being informed about the next relative subject. The conclusion summarizes the paper in a clear and concise manner.

In terms of the technology that this paper serves to introduce, the offering is very strong. The authors make aggressive assertions stating the strengths of LOOM and back these assertions with unequivocal results through extensive testing. Specifically, the assertions made are that LOOM can dynamically locate and correct areas which may be susceptible to race conditions on live systems in real time, in a scalable, flexible, easy to use, safe to use, and fast to implement manor. The testing presented to support these assertions were nine real world race conditions of which each was successfully located and corrected, with little overhead.

===Not-So-Good===
One of the problems with this paper is that some of the examples are oversimplified. For example, Figure 9 attempts to represent the evacuation process. Unfortunately, this causes the problem to seem trivial.

The writers are also biased towards LOOM. Although they do admit the limitations of LOOM, they do not elaborate any further. They promote LOOM without discussing possible problems with it, such as the clients running LOOM may decide not to fix the race conditions and let the program continue to run with LOOM as a permanent fix. This may cause further errors in the long term life of the program.

==References==
[1] "Introduction to Hotpatching".'' Microsoft TechNet ''. Microsoft Corporation, 2010. Web. Accessed: Dec 1st 2010. [http://technet.microsoft.com/en-us/library/cc781109%28WS.10%29.aspx <http://technet.microsoft.com/en-us/library/cc781109(WS.10).aspx>].

[2] "Introduction to Instrumentation and Tracing". '' MSDN ''. Microsoft Corporation, 2010. Web. Accessed: Dec 2nd 2010. [http://msdn.microsoft.com/en-us/library/aa983649%28VS.71%29.aspx <http://msdn.microsoft.com/en-us/library/aa983649(VS.71).aspx>]

[3] Marshall, A. D. "Further Threads Programming:Synchronization.". '' Cardiff School of Comp. Sci. and Info. ''. Cardiff University, 1999. Web. Accessed: Dec 2nd 2010. [http://www.cs.cf.ac.uk/Dave/C/node31.html#SECTION003110000000000000000 <http://www.cs.cf.ac.uk/Dave/C/node31.html#SECTION003110000000000000000>]

[4] "Description of race conditions and deadlocks". '' Microsoft Support ''. Microsoft Corporation, December 6, 2006. Revision: 2.3. Web. Accessed: Dec 2nd 2010. [http://support.microsoft.com/kb/317723 <http://support.microsoft.com/kb/317723>]

[5] Tanenbaum, A. S. Modern Operating Systems (3rd Edition), page 128, 2008. Print.

[6] "QUIESCE Function." 'IBM' IBM Corporation, 2008. Web. Accessed: Dec 2nd 2010. [http://publib.boulder.ibm.com/infocenter/zvm/v5r3/index.jsp?topic=/com.ibm.zvm.v53.hcpb4/hcse5b21270.htm <http://publib.boulder.ibm.com/infocenter/zvm/v5r3/index.jsp?topic=/com.ibm.zvm.v53.hcpb4/hcse5b21270.htm>]

[7] Lu, Shan; Park, Soyeon; Seo, Eunsoo; Zhou, Yuanyuan. "Learning from Mistakes — A Comprehensive Study on Real World Concurrency Bug Characteristics". '' CiteSeerX ''. Dept. of Comp. Sci. at Univ. of Illinois, 2008. Web. Accessed Dec 2nd 2010. [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.121.1203 <http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.121.1203>]

[8] Neamtiu, Iulian; Hicks, Michael. "Safe and Timely Dynamic Updates for Multi-threaded Programs". ''ACM Digital Library''. Association for Computing Machinery, 2009. Web. Accessed: Dec 2nd 2010. [http://portal.acm.org/citation.cfm?id=1542479 <http://portal.acm.org/citation.cfm?id=1542479>]

[9] "Ksplice: Automatic Rebootless Kernel Updates". ''Ksplice''. Massachusetts Institute of Technology, April 2009. Web. Accessed Dec 2nd 2010. [http://www.ksplice.com/doc/ksplice.pdf <http://www.ksplice.com/doc/ksplice.pdf>]

COMP 3000 Essay 2 2010 Question 5

2010-12-02T21:20:08Z

Dustinmartin: /* Problem being addressed */

==Paper==
'''Title:''' [http://www.usenix.org/events/osdi10/tech/full_papers/Wu.pdf Bypassing Races in Live Applications with Execution Filters]

'''Authors:''' Jingyue Wu, Heming Cui, Junfeng Yang

'''Affiliations:''' Computer Science Department, Columbia University

'''Supplementary Information:''' Video available [http://homeostasis.scs.carleton.ca/osdi/video/wu.mp4 here] as well as [http://homeostasis.scs.carleton.ca/osdi/slides/wu.pdf slides]

==Background Concepts==
A race condition is a system flaw that “occurs when two threads access a shared variable at the same time." Race conditions can be very complex, time consuming and expensive to fix. Unfortunately, the most challenging part of race condition is not fixing it, but rather finding it. Race conditions are notorious for being extremely difficult to find, isolate and recreate. To help ease this process, the authors of this paper, Jingyue Wu, Heming Cui, Junfeng Yang, propose the adoption of LOOM.

LOOM is a system which dynamically locates and corrects areas which may be susceptible to race condition errors. The power of LOOM rests in its ability to operate on live applications in real time. This is possible thanks to its evacuation algorithm which injects execution filters to fix race conditions at runtime. Execution filters, otherwise known as request filtering, allow you to inspect the request before and after the main logic is executed. By leveraging execution filters as the means for correcting race conditions, LOOM is able to operate with very little performance overhead and is a highly scalable as the number of application threads increases.

The authors tested LOOM on existing real world race conditions found in common applications. The tests found that all tested race conditions were solved, with little performance overhead, in a scalable and easy to implement manner.

This paper consists of multiple terms which must be familiar to the reader in order to assist in reading the Bypassing Races in Live Applications with Execution Filters paper. These terms are listed and explained below:

'''Deadlock:''' Deadlocks usually occur within the context of two threads. One thread tries to lock a variable that the other thread has already locked and vice versa. The result of this is that each thread is waiting for each other's thread to release the variable. Thus a deadlock occurs and nothing can happen.

'''Evacuation''' The process of proactively pausing and changing states of code sections so that those sections can be filtered for proper processing

'''Execution Filters:''' Otherwise known as request filtering. Request filters allow you to inspect the request before and after the main logic is executed. These are mutual exclusion filters in the context of this paper.

'''Function Quiescence''' The process of pausing and altering states, in order to avoid race conditions and overlapping between code segments.

'''Hot Patches:''' "Hot patching provides a mechanism to update system files without rebooting or stopping services and processes."[[#References | [1]]]

'''Hybrid Instrumentation Engine:''' "Instrumentation refers to an ability to monitor or measure the level of a product's performance, to diagnose errors and writing trace information." [[#References | [2]]] Instrument programs can have low runtime overhead, but instrumentation has to be done at compile time. Dynamic instrumentation can update programs at runtime but incur high overhead. A hybrid instrumentation is an implementation of combined static and dynamic instrumentation.

'''Lock:''' A lock is a way of limiting access to a common resource when using multiple threads. Lock and unlock methods are usually called at the beginning and end of a target method, respectively. "Mutual exclusion locks (mutexes) are a common method of serializing thread execution. Mutual exclusion locks synchronize threads, usually by ensuring that only one thread at a time executes a critical section of code. Mutex locks can also preserve single-threaded code." [[#References | [3]]]

'''Mutex:''' Unable to be both true at the same time.

'''Race Condition:''' "A race condition occurs when two threads access a shared variable at the same time." [[#References | [4]]]

'''Semaphore:''' Semaphores are basically a special type of flag and generalize a down and up state (sleep or wakeup). The down operation checks to see if the value is greater than 0 and if so, decrements the value and uses up one stored wakeup. If the value is 0, the process is put to sleep. These steps are all done in a single indivisible atomic action. It is guaranteed that once a semaphore operation has started, no other process can access the semaphore until the operation has been completed or blocked. Semaphores are an essential part of solving synchronization problems. [[#References | [5]]]

==Research problem==
===Problem being addressed===

With the rise of multiple core systems, multi-threaded programs are often prone to race conditions. Races are hard to detect, test and debug. Many systems are designed to detect, reproduce and diagnose race conditions, but these do not directly address the actual race. This is normally dealt with via a software update which would often require a software restart and potentially introduces new bugs.[[#References | [7]]] Patches require you to be aware of the cause of the problem, and take time to produce, test, and install, leaving a potential user of the software waiting, potentially for months. Due to the immaturity of current race detectors, this paper explains a new approach to race detection and workarounds through the use of LOOM. The goal being to quickly address the actual symptom, the race condition, in real time on live systems. This is in opposition to the conventional approach which focuses on the cause of the race condition, an unknown bug in the software. By targeting the race condition itself, the system is able to keep running without a software update or even a reset.

===Related work===
Two common solutions to fixing deployed races are software updates and hot patches. Software updates require restarts whereas hot patches applies patches to live systems. However, relying on conventional patches can lead to new errors and could be unsafe, due to a multithreaded applications complexity. Releasing a reliable patch takes time, but developers often resort to more efficient fixes rather than placing proper locks in the application due to performance or work pressure.

Using a QUIESCE function to "temporarily suspend...incoming messages on an IUCV path"[[#References | [6]]], these paths can later be reactivated and run as normal. This is not efficient for fixing a race condition because it only delays the problem in an attempt to avoid conflict. Although this does allow for a certain extent of safety it does not come near the reliability and flexibility of LOOM. Speed, reliability, flexibility and ease of use are all areas in which LOOM is demonstrated as being better than a QUIESCE function.

Another similar system to LOOM is STUMP[[#References | [8]]]. STUMP is a system for releasing live updates for multi-threaded or single-threaded programs written in C. It has the ability to provide arbitrary patches to source code in running systems without requiring a reset. These patches require considerable annotation and preparation as source code modifications are considered to be unsafe. Unlike STUMP, LOOM does not operate on the source code and is considered more safe because of this.

The most recent system for live updates to the kernels of Operating Systems is called Ksplice [[#References | [9]]]. It allows users to update the Linux kernel without resetting. This can be done either completely automatically if the code does not chance any data structures, or with on average 17 lines of code for an update that would otherwise require a reset. [[#References | [9]]] It does this by operating on the object layer.

==Contribution==
===Current solution expressed===
Compared to traditional solutions, LOOM differs in its approach to race fixes. It is designed to quickly develop safe, optimized, temporary workarounds while a concrete solution is developed. LOOM is also very easy to use. LOOM is compiled with a developer's application as a plugin and kept separate from the source code. The plugin will inject the LOOM Update Engine into the application binary.

Mutual exclusion filters are written by the developer and synced with the source code to filter out any racy threads. The filter's code declaration is easy to understand and can be inserted in a code region that needs to be mutually exclusive. The developer does not need to deal with low level operations such as lock, unlock and semaphore operations. Users can then download the filter and apply it to the application while it is still live.

LOOM is flexible in that developers can make trade-offs between performance and reliability in their application. These can include making two code regions mutually exclusive when accessing different objects; or, with a significant decrease in performance, making them run in single threaded mode.

An evacuation algorithm is used for safety as to not introduce new errors. A critical region is marked using static analysis. All threads in the critical region are then evacuated. After the evacuation is executed, the execution filter is installed and then the threads are resumed after a live update pause is done at a safe location.

LOOM's hybrid instrumentation engine is used to reduce its overhead. The engine statically changes an application's binary to anticipate dynamic updates.

The evaluation of LOOM was based on overhead, scalability, reliability, availability and timeliness. These were demonstrated using Apache and MySQL in conjunction with the multithreaded ApacheBench and SysBench, respectively.

Through multiple tests the advantages of LOOM were proven. Overhead was tested in a comparison of LOOM during normal runtime. The effects of LOOM on Apache and MySQL were minimal, (~1.83% and ~4% respectively) causing it to be a viable as a runtime fix for race errors. To test scalability the team discovered that on 32 server threads, the overhead was still low: under 3% and 12% respectively. Reliability is one of the strongest facets of the LOOM system as it fixed all of the race conditions studied. To demonstrate LOOM's reliability, it was compared against a conventional restart-based software update. In this test the software update was clearly slower, requiring time to reset itself, where LOOM running a live update had almost no effect on the throughput. Lastly the timeliness of the installation of LOOM's fixes was demonstrated in a simple example. The LOOM based fix completed in 368ms whereas the function quiescence fix took the max test time (1 hour) and was not finished.

====Why is it any better than what came before?====
Previously, the two standard ways of fixing deployed race conditions were system updates and hot patches. LOOM is a superior choice to both these options for a number of reasons.

Unlike LOOM, the system update approach requires that the system be rebooted before the fix can be implemented. With desktop applications, rebooting a system is acceptable. However, servers often cannot reboot because requests are coming from external sources and are expected to be processed.

While hot patches do not require a system reboot, they do have their own specific vulnerabilities. Namely, it is very difficult to apply a patch that corrects the error, or errors, but leaves the rest of the system unaffected. Often when correcting a race condition via a hot patch, others can appear. The main concern with hot patches however, is that their development is a time consuming process. A process which until developed and deployed, leaves the race condition vulnerable and exposed. The paper chronicles a real world Mozilla race condition whose hot patch took nearly 8 years of development to correct. All the while the vulnerability was exposed to Mozilla users.

Flaws common to both system updates and hot patches are that they are very difficult to properly develop, slow to implement, and result in potentially unsafe ad hoc solutions that are not scalable.

Conversely, LOOM is easy to use, fast to implement, highly flexible, scalable, and safe to use

==Critique==
===Good===
The authors of this essay effectively convey their findings by staying focused on the thesis as well as supporting their topics with relevant examples and data. Examples throughout the paper, particularly the MySQL example, ensure that the use of execution filters is clear to the reader. All of the examples are well documented and some (ex. Figure 2) are simplified to avoid confusing the reader with too much unnecessary information. The references throughout backup the reliability of the paper and let the reader to verify information from the sources.

The essay flows well and the information is delivered in a well put together order, allowing the reader to learn enough about LOOM (or any of the sub-topics involved in the explanation) before being informed about the next relative subject. The conclusion summarizes the paper in a clear and concise manner.

===Not-So-Good===
One of the problems with this paper is that some of the examples are oversimplified. For example, Figure 9 attempts to represent the evacuation process. Unfortunately, this causes the problem to seem trivial.

The writers are also biased towards LOOM. Although they do admit the limitations of LOOM, they do not elaborate any further. They promote LOOM without discussing possible problems with it, such as the clients running LOOM may decide not to fix the race conditions and let the program continue to run with LOOM as a permanent fix. This may cause further errors in the long term life of the program.

==References==
[1] "Introduction to Hotpatching".'' Microsoft TechNet ''. Microsoft Corporation, 2010. Web. Accessed: Dec 1st 2010. [http://technet.microsoft.com/en-us/library/cc781109%28WS.10%29.aspx <http://technet.microsoft.com/en-us/library/cc781109(WS.10).aspx>].

[2] "Introduction to Instrumentation and Tracing". '' MSDN ''. Microsoft Corporation, 2010. Web. Accessed: Dec 2nd 2010. [http://msdn.microsoft.com/en-us/library/aa983649%28VS.71%29.aspx <http://msdn.microsoft.com/en-us/library/aa983649(VS.71).aspx>]

[3] Marshall, A. D. "Further Threads Programming:Synchronization.". '' Cardiff School of Comp. Sci. and Info. ''. Cardiff University, 1999. Web. Accessed: Dec 2nd 2010. [http://www.cs.cf.ac.uk/Dave/C/node31.html#SECTION003110000000000000000 <http://www.cs.cf.ac.uk/Dave/C/node31.html#SECTION003110000000000000000>]

[4] "Description of race conditions and deadlocks". '' Microsoft Support ''. Microsoft Corporation, December 6, 2006. Revision: 2.3. Web. Accessed: Dec 2nd 2010. [http://support.microsoft.com/kb/317723 <http://support.microsoft.com/kb/317723>]

[5] Tanenbaum, A. S. Modern Operating Systems (3rd Edition), page 128, 2008. Print.

[6] "QUIESCE Function." 'IBM' IBM Corporation, 2008. Web. Accessed: Dec 2nd 2010. [http://publib.boulder.ibm.com/infocenter/zvm/v5r3/index.jsp?topic=/com.ibm.zvm.v53.hcpb4/hcse5b21270.htm <http://publib.boulder.ibm.com/infocenter/zvm/v5r3/index.jsp?topic=/com.ibm.zvm.v53.hcpb4/hcse5b21270.htm>]

[7] Lu, Shan; Park, Soyeon; Seo, Eunsoo; Zhou, Yuanyuan. "Learning from Mistakes — A Comprehensive Study on Real World Concurrency Bug Characteristics". '' CiteSeerX ''. Dept. of Comp. Sci. at Univ. of Illinois, 2008. Web. Accessed Dec 2nd 2010. [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.121.1203 <http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.121.1203>]

[8] Neamtiu, Iulian; Hicks, Michael. "Safe and Timely Dynamic Updates for Multi-threaded Programs". ''ACM Digital Library''. Association for Computing Machinery, 2009. Web. Accessed: Dec 2nd 2010. [http://portal.acm.org/citation.cfm?id=1542479 <http://portal.acm.org/citation.cfm?id=1542479>]

[9] "Ksplice: Automatic Rebootless Kernel Updates". ''Ksplice''. Massachusetts Institute of Technology, April 2009. Web. Accessed Dec 2nd 2010. [http://www.ksplice.com/doc/ksplice.pdf <http://www.ksplice.com/doc/ksplice.pdf>]

COMP 3000 Essay 2 2010 Question 5

2010-12-02T14:58:50Z

Dustinmartin: /* Why is it any better than what came before? */

==Paper==
'''Title:''' [http://www.usenix.org/events/osdi10/tech/full_papers/Wu.pdf Bypassing Races in Live Applications with Execution Filters]

'''Authors:''' Jingyue Wu, Heming Cui, Junfeng Yang

'''Affiliations:''' Computer Science Department, Columbia University

'''Supplementary Information:''' Video available [http://homeostasis.scs.carleton.ca/osdi/video/wu.mp4 here] as well as [http://homeostasis.scs.carleton.ca/osdi/slides/wu.pdf slides]

==Background Concepts==
A race condition is a system flaw that “occurs when two threads access a shared variable at the same time." Race conditions can be very complex, time consuming and expensive to fix. Unfortunately, the most challenging part of race condition is not fixing it, but rather finding it. Race conditions are notorious for being extremely difficult to find, isolate and recreate. To help ease this process, the authors of this paper, Jingyue Wu, Heming Cui, Junfeng Yang, propose the adoption of LOOM.

LOOM is a system which dynamically locates and corrects areas which may be susceptible to race condition errors. The power of LOOM rests in its ability to operate on live applications in real time. This is possible thanks to its evacuation algorithm which injects execution filters to fix race conditions at runtime. Execution filters, otherwise known as request filtering, allow you to inspect the request before and after the main logic is executed. By leveraging execution filters as the means for correcting race conditions, LOOM is able to operate with very little performance overhead and is a highly scalable as the number of application threads increases.

The authors tested LOOM on existing real world race conditions found in common applications. The tests found that all tested race conditions were solved, with little performance overhead, in a scalable and easy to implement manner.

This paper consists of multiple terms which must be familiar to the reader in order to assist in reading the Bypassing Races in Live Applications with Execution Filters paper. These terms are listed and explained below:

'''Deadlock:''' Deadlocks usually occur within the context of two threads. One thread tries to lock a variable that the other thread has already locked and vice versa. The result of this is that each thread is waiting for each other's thread to release the variable. Thus a deadlock occurs and nothing can happen.

'''Evacuation''' The process of proactively pausing and changing states of code sections so that those sections can be filtered for proper processing

'''Execution Filters:''' Otherwise known as request filtering. Request filters allow you to inspect the request before and after the main logic is executed. These are mutual exclusion filters in the context of this paper.

'''Function Quiescence''' The process of pausing and altering states, in order to avoid race conditions and overlapping between code segments.

'''Hot Patches:''' "Hot patching provides a mechanism to update system files without rebooting or stopping services and processes."[[#References | [1]]]

'''Hybrid Instrumentation Engine:''' "Instrumentation refers to an ability to monitor or measure the level of a product's performance, to diagnose errors and writing trace information." [[#References | [2]]] Instrument programs can have low runtime overhead, but instrumentation has to be done at compile time. Dynamic instrumentation can update programs at runtime but incur high overhead. A hybrid instrumentation is an implementation of combined static and dynamic instrumentation.

'''Lock:''' A lock is a way of limiting access to a common resource when using multiple threads. Lock and unlock methods are usually called at the beginning and end of a target method, respectively. "Mutual exclusion locks (mutexes) are a common method of serializing thread execution. Mutual exclusion locks synchronize threads, usually by ensuring that only one thread at a time executes a critical section of code. Mutex locks can also preserve single-threaded code." [[#References | [3]]]

'''Mutex:''' Unable to be both true at the same time.

'''Race Condition:''' "A race condition occurs when two threads access a shared variable at the same time." [[#References | [4]]]

'''Semaphore:''' Semaphores are basically a special type of flag and generalize a down and up state (sleep or wakeup). The down operation checks to see if the value is greater than 0 and if so, decrements the value and uses up one stored wakeup. If the value is 0, the process is put to sleep. These steps are all done in a single indivisible atomic action. It is guaranteed that once a semaphore operation has started, no other process can access the semaphore until the operation has been completed or blocked. Semaphores are an essential part of solving synchronization problems. [[#References | [5]]]

==Research problem==
===Problem being addressed===
With the rise of multiple core systems, multithreaded programs are often prone to race conditions. Races are hard to detect, test and debug. Due to the immaturity of current race detectors, this paper explains a new approach to race detection and workarounds through the use of LOOM.
===Related work===
Two common solutions to fixing deployed races are software updates and hot patches. Software updates require restarts whereas hot patches applies patches to live systems. However, relying on conventional patches can lead to new errors and could be unsafe, due to a multithreaded applications complexity. Releasing a reliable patch takes time, but developers often resort to more efficient fixes rather than placing proper locks in the application due to performance or work pressure.

Using a QUIESCE function to "temporarily suspend...incoming messages on an IUCV path"[[#References | [6]]], these paths can later be reactivated and run as normal. This is not efficient for fixing a race condition because it only delays the problem in an attempt to avoid conflict. Although this does allow for a certain extent of safety it does not come near the reliability and flexibility of LOOM. Speed, reliability, flexibility and ease of use are all areas in which LOOM is demonstrated as being better than a QUIESCE function.

==Contribution==
===Current solution expressed===
Compared to traditional solutions, LOOM differs in its approach to race fixes. It is designed to quickly develop safe, optimized, temporary workarounds while a concrete solution is developed. LOOM is also very easy to use. LOOM is compiled with a developer's application as a plugin and kept separate from the source code. The plugin will inject the LOOM Update Engine into the application binary.

Mutual exclusion filters are written by the developer and synced with the source code to filter out any racy threads. The filter's code declaration is easy to understand and can be inserted in a code region that needs to be mutually exclusive. The developer does not need to deal with low level operations such as lock, unlock and semaphore operations. Users can then download the filter and apply it to the application while it is still live.

LOOM is flexible in that developers can make trade-offs between performance and reliability in their application. These can include making two code regions mutually exclusive when accessing different objects; or, with a significant decrease in performance, making them run in single threaded mode.

An evacuation algorithm is used for safety as to not introduce new errors. A critical region is marked using static analysis. All threads in the critical region are then evacuated. After the evacuation is executed, the execution filter is installed and then the threads are resumed after a live update pause is done at a safe location.

LOOM's hybrid instrumentation engine is used to reduce its overhead. The engine statically changes an application's binary to anticipate dynamic updates.

The evaluation of LOOM was based on overhead, scalability, reliability, availability and timeliness. These were demonstrated using Apache and MySQL in conjunction with the multithreaded ApacheBench and SysBench, respectively.

Through multiple tests the advantages of LOOM were proven. Overhead was tested in a comparison of LOOM during normal runtime. The effects of LOOM on Apache and MySQL were minimal, (~1.83% and ~4% respectively) causing it to be a viable as a runtime fix for race errors. To test scalability the team discovered that on 32 server threads, the overhead was still low: under 3% and 12% respectively. Reliability is one of the strongest facets of the LOOM system as it fixed all of the race conditions studied. To demonstrate LOOM's reliability, it was compared against a conventional restart-based software update. In this test the software update was clearly slower, requiring time to reset itself, where LOOM running a live update had almost no effect on the throughput. Lastly the timeliness of the installation of LOOM's fixes was demonstrated in a simple example. The LOOM based fix completed in 368ms whereas the function quiescence fix took the max test time (1 hour) and was not finished.

====Why is it any better than what came before?====
Previously, the two standard ways of fixing deployed race conditions were system updates and hot patches. LOOM is a superior choice to both these options for a number of reasons.

Unlike LOOM, the system update approach requires that the system be rebooted before the fix can be implemented. With desktop applications, rebooting a system is acceptable. However, servers often cannot reboot because requests are coming from external sources and are expected to be processed.

While hot patches do not require a system reboot, they do have their own specific vulnerabilities. Namely, it is very difficult to apply a patch that corrects the error, or errors, but leaves the rest of the system unaffected. Often when correcting a race condition via a hot patch, others can appear. The main concern with hot patches however, is that their development is a time consuming process. A process which until developed and deployed, leaves the race condition vulnerable and exposed. The paper chronicles a real world Mozilla race condition whose hot patch took nearly 8 years of development to correct. All the while the vulnerability was exposed to Mozilla users.

Flaws common to both system updates and hot patches are that they are very difficult to properly develop, slow to implement, and result in potentially unsafe ad hoc solutions that are not scalable.

Conversely, LOOM is easy to use, fast to implement, highly flexible, scalable, and safe to use.

==Critique==
===Good===
The authors of this essay effectively convey their findings by staying focused on the thesis as well as supporting their topics with relevant examples and data. Examples throughout the paper, particularly the MySQL example, ensure that the use of execution filters is clear to the reader. All of the examples are well documented and some (ex. Figure 2) are simplified to avoid confusing the reader with too much unnecessary information. The references throughout backup the reliability of the paper and let the reader to verify information from the sources.

The essay flows well and the information is delivered in a well put together order, allowing the reader to learn enough about LOOM (or any of the sub-topics involved in the explanation) before being informed about the next relative subject. The conclusion summarizes the paper in a clear and concise manner.

===Not-So-Good===
One of the problems with this paper is that some of the examples are oversimplified. For example, Figure 9 attempts to represent the evacuation process. Unfortunately, this causes the problem to seem trivial.

The writers are also biased towards LOOM. Although they do admit the limitations of LOOM, they do not elaborate any further. They promote LOOM without discussing possible problems with it, such as the clients running LOOM may decide not to fix the race conditions and let the program continue to run with LOOM as a permanent fix. This may cause further errors in the long term life of the program.

==References==
[1] Introduction to Hotpatching. [http://technet.microsoft.com/en-us/library/cc781109%28WS.10%29.aspx http://technet.microsoft.com/en-us/library/cc781109(WS.10).aspx].

[2] Introduction to Instrumentation and Tracing. [http://msdn.microsoft.com/en-us/library/aa983649%28VS.71%29.aspx http://msdn.microsoft.com/en-us/library/aa983649(VS.71).aspx]

[3] A. D. Marshall. Further Threads Programming:Synchronization. Cardiff University, 1999 [http://www.cs.cf.ac.uk/Dave/C/node31.html#SECTION003110000000000000000 HTML]

[4] Description of race conditions and deadlocks. [http://support.microsoft.com/kb/317723 http://support.microsoft.com/kb/317723]

[5] A. S. Tanenbaum. Modern Operating Systems (3rd Edition), page 128, 2008

[6] QUIESCE Function. IBM [http://publib.boulder.ibm.com/infocenter/zvm/v5r3/index.jsp?topic=/com.ibm.zvm.v53.hcpb4/hcse5b21270.htm]

[7] Lu, Shan; Park, Soyeon; Seo, Eunsoo; Zhou, Yuanyuan. "Learning from Mistakes — A Comprehensive Study on Real World Concurrency Bug Characteristics". '"CiteSeerX"'. Dept. of Comp. Sci. at Univ. of Illinois, 2008. Accessed Dec 1st 2010. [http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.121.1203 <http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.121.1203>]

COMP 3000 Essay 2 2010 Question 5

2010-11-30T17:33:36Z

Dustinmartin: /* Current solution expressed */

==Paper==
'''Title:''' [http://www.usenix.org/events/osdi10/tech/full_papers/Wu.pdf Bypassing Races in Live Applications with Execution Filters]

'''Authors:''' Jingyue Wu, Heming Cui, Junfeng Yang

'''Affiliations:''' Computer Science Department, Columbia University

'''Supplementary Information:''' Video available [http://homeostasis.scs.carleton.ca/osdi/video/wu.mp4 here] as well as [http://homeostasis.scs.carleton.ca/osdi/slides/wu.pdf slides]

==Background Concepts==
A race condition is a system flaw that “occurs when two threads access a shared variable at the same time." Race conditions can be very complex, time consuming and expensive to fix. Unfortunately, the most challenging part of race condition is not fixing it, but rather find it. Race conditions are notorious for being extremely difficult to find, isolate and recreate. To help ease this process, the authors of this paper, Jingyue Wu, Heming Cui, Junfeng Yang, propose the adoption of LOOM.

LOOM is a system which dynamically locates and corrects areas which may be susceptible to race condition errors. The power of LOOM rests in its ability to operate on live applications in real time. This is possible thanks to its evacuation algorithm which injects execution filters to fix race conditions at runtime. Execution filters, otherwise known as request filtering, allow you to inspect the request before and after the main logic is executed. By leveraging execution filters as the means for correcting race conditions, LOOM is able to operate with very little performance overhead and is a highly scalable as the number of application threads increases.

The authors tested LOOM on existing real world race conditions found in common applications. The tests found that all tested race conditions were solved, with little performance overhead, in a scalable and easy to implement manor.

This paper consists of multiple terms which must be familiar to the reader in order to assist in reading the Bypassing Races in Live Applications with Execution Filters paper. These terms are listed and explained below:

'''Deadlock:''' Deadlocks usually occur within the context of two threads. One thread tries to lock a variable that the other thread has already locked and vice versa. The result of this is that each thread is waiting for each others thread to release the variable. Thus a deadlock occurs and nothing can happen.

'''Evacuation''' The process of proactively pausing and changing states of code sections so that those sections can be filtered for proper processing

'''Execution Filters:''' Otherwise known as request filtering. Request filters allow you to inspect the request before and after the main logic is executed. These are mutual exclusion filters in the context of this paper.

'''Function Quiescence''' The process of pausing and altering states, in order to avoid race conditions and overlapping between code segments.

'''Hot Patches:''' "Hot patching provides a mechanism to update system files without rebooting or stopping services and processes."[[#References | [1]]]

'''Hybrid Instrumentation Engine:''' "Instrumentation refers to an ability to monitor or measure the level of a product's performance, to diagnose errors and writing trace information." [[#References | [2]]] Instrument programs can have low runtime overhead, but instrumentation has to be done at compile time. Dynamic instrumentation can update programs at runtime but incur high overhead. A hybrid instrumentation is an implementation of combined static and dynamic instrumentation.

'''Lock:''' A lock is a way of limiting access to a common resource when using multiple threads. Lock and unlock methods are usually called at the beginning and end of a target method, respectively. "Mutual exclusion locks (mutexes) are a common method of serializing thread execution. Mutual exclusion locks synchronize threads, usually by ensuring that only one thread at a time executes a critical section of code. Mutex locks can also preserve single-threaded code." [[#References | [3]]]

'''Mutex:''' Unable to be both true at the same time.

'''Race Condition:''' "A race condition occurs when two threads access a shared variable at the same time." [[#References | [4]]]

'''Semaphore:''' Semaphores are basically a special type of flag and generalize a down and up state(sleep or wakeup). The down operation checks to see if the value is greater than 0 and if so, decrements the value and uses up one stored wakeup. If the value is 0, the process is put to sleep. These steps are all done in a single indivisible atomic action. It is guaranteed that once a semaphore operation has started, no other process can access the semaphore until the operation has been completed or blocked. Semaphores are an essential part of solving synchronization problems. [[#References | [5]]]

==Research problem==
===Problem being addressed===
With the rise of multiple core systems, multithreaded programs are often prone to race conditions. Races are hard to detect, test and debug. Due to the immaturity of current race detectors, this paper explains a new approach to race detection and work arounds through the use of LOOM.
===Related work===
Two common solutions to fixing deployed races are software updates and hot patches. Software updates require restarts whereas hot patches applies patches to live systems. However, relying on conventional patches can lead to new errors and could be unsafe, due to a multithreaded applications complexity. Releasing a reliable patch takes time, but developers often resort to more efficient fixes rather than placing proper locks in the application due to performance or work pressure.

Using a QUIESCE function to "temporarily suspend...incoming messages on an IUCV path"[[#References | [6]]] These paths can later be reactivated and run as normal. This is not an efficient for of fixing a race condition because it only delays the problem in an attempt to avoid conflict. Although this does allow for a certain extent of safety it does not come near the reliability and flexibility of LOOM. Speed, reliability, flexibility and ease of use are all areas in which LOOM is demonstrated as being better than a QUIESCE function.

==Contribution==
===Current solution expressed===
Compared to traditional solutions, LOOM differs in its approach to race fixes. It is designed to quickly develop safe, optimized, temporary workarounds while a concrete solution is developed. LOOM is also very easy to use. LOOM is compiled with a developers application as a plugin and kept separate from the source code. The plugin will inject the LOOM update into the application binary.

Mutual exclusion filters are written by the developer and synced with the source code to filter out any racy threads. The code declaration used is easy to understand and can be inserted in a code region that need to be mutually exclusive. The developer does not need to deal with low level operations such as lock, unlock and semaphore operations. Users can then download the filter and apply it to the application while it is still live.

LOOM is flexible in that developers can make trade-offs in performance and reliability in their application in conjunction with LOOM. These can include making two code regions mutually exclusive even when accessing different objects or with extreme measures, making them run in single threaded mode.

An evacuation algorithm is used for safety as to not introduce new errors. A critical region is marked using static analysis. All threads in the critical region are then evacuated. After the evacuation is executed, the execution filter is installed and then the threads are resumed after a live update pause is done at a safe location.

LOOM's hybrid instrumentation engine is used to reduce its overhead. The engine statically changes an applications binary to anticipate dynamic updates.

Evaluation of LOOM was based on overhead, scalability, reliability, availability and timeliness. These were demonstrated using Apache and MySQL in conjunction with the multithreaded ApacheBench and SysBench, respectively.

Through multiple tests LOOM proves its worth. Overhead is tested in a comparison of LOOM during normal runtime. The effects of LOOM on Apache and MySQL are minimal, (~1.83% and ~4% respectively) making it an obvious choice as a runtime fix for race errors. To test scalability the team discovered that on 32 server threads, the over head was still under 3% and 12% respectively. Reliability is one of the strongest facets of the LOOM system. It fixed all of the race condition errors that it was tested against, proving that it has immense power as a reliable form of fix. To assure reliability LOOM was paired against a conventional restart-based software update. In this test the software update was clearly slower, requiring time to reset itself, where LOOM running a live update had almost no effect on the throughput. Lastly the timeliness of LOOM was demonstrated using a simple example, showing a LOOM based fix in 368ms whereas the function quiescence fix took the max test time (1 hour) and still did not finish.

====Why is it any better than what came before?====
Previously, the two standard ways of fixing deployed race conditions were system updates and hot patches. LOOM is a superior choice to both these options for a number of reasons.

Unlike LOOM, the system update approach requires that the system be rebooted before the fix can be implemented. With desktop applications, this can sometimes be considered acceptable. However, server applications often do not have the luxury of being able to reboot because requests are coming from external sources and are expected to be processed.

While hot patches do not require a system reboot, they do have their own specific vulnerabilities. Namely, it is very difficult to apply a patch that corrects the error, or errors, but leaves the rest of the system unaffected. Often when correcting a race condition via a hot patch, others can appear. The main concern with hot patches however, is that their development is a time consuming process. A process which until developed and deployed, leaves the race condition vulnerable and exposed. The paper chronicles a real world Mozilla race condition whose hot patch took nearly 8 years of development to correct, all the while the vulnerability was exposed to all Mozilla users.

Flaws common to both the system update and hot patch approach are they very difficult to properly development, slow to implement, and result in potential unsafe ad hoc solutions that are not scalable. Conversely, LOOM is easy to use, fast to implement, highly flexible, scalable, and safe to use.

==Critique==
===Good===
The authors of this essay are efficient at delivering the information surrounding their thesis both in staying focused on the main thesis as well as backing up their topics with relevant examples and data. This helps to keep the thesis paramount throughout the paper. Examples throughout the paper, particularly the MySQL example ensure that the use of execution filters is clear to the reader. All of the examples are well documented and some (ex. Figure 2) are simplified as to not confuse the reader with too much unnecessary information. References throughout the writing backup the reliability of the paper and let the user keep track of the sources to properly check information and sources.

The whole essay flows well and the information is delivered in a well put together order, allowing the reader to learn enough about LOOM (or any of the sub-topics involved in the explanation) before being informed about the next relative subject. The paper ends with a conclusion that does a good job of wrapping up the whole paper in a clear and concise manner.

===Not-So-Good===
One of the problems with this paper is that although many of the examples are simplified in order to expedite the understanding of the user, some are a little oversimplified. For example, Figure 9 is a graphic that attempts to represent the evacuation process in a visual manner. Unfortunately, this ends up making the problem seem almost trivial and does little more than water down the information.

The writers are also a little bit one sided (with understandable reason) on the topic. Although they do admit the limitations of LOOM, they do not spend much time discussing any problems later. There is a large amount of play-up for LOOM without much discussion of the possible problems with it, such as the clients running LOOM may decide not to fix the race conditions and rather just let the program continue to run with LOOM as a permanent fix. This may cause further errors in the long term life of the program.

==References==
[1] Introduction to Hotpatching. [http://technet.microsoft.com/en-us/library/cc781109%28WS.10%29.aspx http://technet.microsoft.com/en-us/library/cc781109(WS.10).aspx].

[2] Introduction to Instrumentation and Tracing. [http://msdn.microsoft.com/en-us/library/aa983649%28VS.71%29.aspx http://msdn.microsoft.com/en-us/library/aa983649(VS.71).aspx]

[3] A. D. Marshall. Further Threads Programming:Synchronization. Cardiff University, 1999 [http://www.cs.cf.ac.uk/Dave/C/node31.html#SECTION003110000000000000000 HTML]

[4] Description of race conditions and deadlocks. [http://support.microsoft.com/kb/317723 http://support.microsoft.com/kb/317723]

[5] A. S. Tanenbaum. Modern Operating Systems (3rd Edition), page 128, 2008

[6] QUIESCE Function. IBM [http://publib.boulder.ibm.com/infocenter/zvm/v5r3/index.jsp?topic=/com.ibm.zvm.v53.hcpb4/hcse5b21270.htm]

Talk:COMP 3000 Essay 2 2010 Question 5

2010-11-24T22:20:27Z

Dustinmartin: /* Why is it any better than what came before? */

Maybe we can all add our names below so we know who's still in this course? --[[User:Myagi|Myagi]] 12:38, 14 November 2010 (UTC)

Group members:

* Michael Yagi
* Nicolas Lessard
* Julie Powers
* Derek Langlois
* Dustin Martin

Jeffrey Francom contacted me earlier so I know he is also still in the course. <strike>Now we are only waiting on Dustin Martin.</strike> Everyone has been accounted for. [[User:J powers|J powers]] 18:07, 15 November 2010 (UTC)

Just kicking things off. Feel free to make suggestions or change anything. --[[User:Myagi|Myagi]] 11:36, 17 November 2010 (UTC)

Edited and filled out the critique section. Edited a little bit here and there. --[[User:Afranco2|Afranco2]] 17:41, 22 November 2010 (UTC)

Moved stuff to the front page and cleaned up references. Still waiting for people to expand if possible. Also, spellcheck ;) --[[User:Myagi|Myagi]] 10:37, 24 November 2010 (UTC)

==Essay==
===Paper===
<blockquote>The paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.</blockquote>
* Title: [http://www.usenix.org/events/osdi10/tech/full_papers/Wu.pdf Bypassing Races in Live Applications with Execution Filters]
* Authors: Jingyue Wu, Heming Cui, Junfeng Yang
* Affiliations: Computer Science Department, Columbia University
* Supplementary Information: [http://homeostasis.scs.carleton.ca/osdi/video/wu.mp4 Video], [http://homeostasis.scs.carleton.ca/osdi/slides/wu.pdf Slides]

===Background Concepts===
<blockquote>Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.</blockquote>

-------------
A race condition is a system flaw that “occurs when two threads access a shared variable at the same time." Race conditions can be very complex, time consuming and expensive to fix. Unfortunately, the most challenging part of race condition is not fixing it, but rather find it. Race conditions are notorious for being extremely difficult to find, isolate and recreate. To help ease this process, the authors of this paper, Jingyue Wu, Heming Cui, Junfeng Yang, propose the adoption of LOOM.

LOOM is a system which dynamically locates and corrects areas which may be susceptible to race condition errors. The power of LOOM rests in its ability to operate on live applications in real time. This is possible thanks to its evacuation algorithm which injects execution filters to fix race conditions at runtime. Execution filters, otherwise known as request filtering, allow you to inspect the request before and after the main logic is executed. By leveraging execution filters as the means for correcting race conditions, LOOM is able to operate with very little performance overhead and is a highly scalable as the number of application threads increases.

The authors tested LOOM on existing real world race conditions found in common applications. The tests found that all tested race conditions were solved, with little performance overhead, in a scalable and easy to implement manor.
-------------

This paper consists of multiple terms which must be familiar to the reader in order to assist in reading the Bypassing Races in Live Applications with Execution Filters paper. These terms are listed and explained below:

* Race Condition: "A race condition occurs when two threads access a shared variable at the same time." [http://support.microsoft.com/kb/317723 Race Condition]
* Execution Filters: Otherwise known as request filtering. Request filters allow you to inspect the request before and after the main logic is executed. These are mutual exclusion filters in the context of this paper.
* Hot Patches: "Hot patching provides a mechanism to update system files without rebooting or stopping services and processes."[http://technet.microsoft.com/en-us/library/cc781109%28WS.10%29.aspx Hot Patching]
* Hybrid Instrumentation Engine: "Instrumentation refers to an ability to monitor or measure the level of a product's performance, to diagnose errors and writing trace information." [http://msdn.microsoft.com/en-us/library/aa983649%28VS.71%29.aspx Instrumentation] Instrument programs can have low runtime overhead, but instrumentation has to be done at compile time. Dynamic instrumentation can update programs at runtime but incur high overhead. A hybrid instrumentation is an implementation of combined static and dynamic instrumentation.
* Lock: A lock is a way of limiting access to a common resource when using multiple threads. Lock and unlock methods are usually called at the beginning and end of a target method, respectively. "Mutual exclusion locks (mutexes) are a common method of serializing thread execution. Mutual exclusion locks synchronize threads, usually by ensuring that only one thread at a time executes a critical section of code. Mutex locks can also preserve single-threaded code." [http://www.cs.cf.ac.uk/Dave/C/node31.html#SECTION003110000000000000000 Mutex Locks]
* Mutex: Unable to be both true at the same time.
* Semaphore: "A semaphore is a protected variable or abstract data type that provides a simple but useful abstraction for controlling access by multiple processes to a common resource in a parallel programming environment." [http://en.wikipedia.org/wiki/Semaphore_%28programming%29 Semaphore]

===Research problem===
<blockquote>What is the research problem being addressed by the paper? How does this problem relate to past related work?</blockquote>
====Problem being addressed====
With the rise of multiple core systems, multithreaded programs are often prone to race conditions. Races are hard to detect, test and debug. Due to the immaturity of current race detectors, this paper explains a new approach to race detection and work arounds through the use of LOOM.
====Past related work====
Two common solutions to fixing deployed races are software updates and hot patches. Software updates require restarts whereas hot patches applies patches to live systems. However, relying on conventional patches can lead to new errors and could be unsafe, due to a multithreaded applications complexity. Releasing a reliable patch takes time, but developers often resort to more efficient fixes rather than placing proper locks in the application due to performance or work pressure.
===Contribution===
<blockquote>What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)</blockquote>
====Current solution expressed====
Compared to traditional solutions, LOOM differs in its approach to race fixes. It is designed to quickly develop safe, optimized, temporary workarounds while a concrete solution is developed. LOOM is also very easy to use. LOOM is compiled with a developers application as a plugin and kept separate from the source code. The plugin will inject the LOOM update into the application binary.

Mutual exclusion filters are written by the developer and synced with the source code to filter out any racy threads. The code declaration used is easy to understand and can be inserted in a code region that need to be mutually exclusive. The developer does not need to deal with low level operations such as lock, unlock and semaphore operations. Users can then download the filter and apply it to the application while it is still live.

LOOM is flexible in that developers can make trade-offs in performance and reliability in their application in conjunction with LOOM. These can include making two code regions mutually exclusive even when accessing different objects or with extreme measures, making them run in single threaded mode.

An evacuation algorithm is used for safety as to not introduce new errors. A critical region is marked using static analysis. All threads in the critical region are then evacuated. After the evacuation is executed, the execution filter is installed and then the threads are resumed after a live update pause is done at a safe location.

LOOM's hybrid instrumentation engine is used to reduce its overhead. The engine statically changes an applications binary to anticipate dynamic updates.

Evaluation of LOOM was based on overhead, scalability, reliability, availability and timeliness. These were demonstrated using Apache and MySQL in conjunction with the multithreaded ApacheBench and SysBench, respectively.

-------------
====Why is it any better than what came before?====
Previously, the two standard ways of fixing deployed race conditions were system updates and hot patches. LOOM is a superior choice to both these options for a number of reasons.

Unlike LOOM, the system update approach requires that the system be rebooted before the fix can be implemented. With desktop applications, this can sometimes be considered acceptable. However, server applications often do not have the luxury of being able to reboot because requests are coming from external sources and are expected to be processed.

While hot patches do not require a system reboot, they do have their own specific vulnerabilities. Namely, it is very difficult to apply a patch that corrects the error, or errors, but leaves the rest of the system unaffected. Often when correcting a race condition via a hot patch, others can appear. The main concern with hot patches however, is that their development is a time consuming process. A process which until developed and deployed, leaves the race condition vulnerable and exposed. The paper chronicles a real world Mozilla race condition whose hot patch took nearly 8 years of development to correct, all the while the vulnerability was exposed to all Mozilla users.

Flaws common to both the system update and hot patch approach are they very difficult to properly development, slow to implement, and result in potential unsafe ad hoc solutions that are not scalable. Conversely, LOOM is easy to use, fast to implement, highly flexible, scalable, and safe to use.
-------------

===Critique===
<blockquote>What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.</blockquote>
====Good====
The authors of this essay are efficient at delivering the information surrounding their thesis both in staying focused on the main thesis as well as backing up thier topics with relevant examples and data. This helps to keep the thesis paramount throughout the paper. Examples throughout the paper, particularly the MySQL example ensure that the use of execution filters is clear to the reader. All of the examples are well documented and some (ex. Figure 2) are simplified as to not confuse the reader with too much unnessicary information. References throughout the writing backup the reliability of the paper and let the user keep track of the sources to properly check information and sources.

The whole essay flows well and the information is delievered in a well put together order, allowing the reader to learn enough about LOOM (or any of the sub-topics involved in the explination) before being informed about the next relative subject. The paper ends with a conclusion that does a good job of wrapping up the whole paper in a clear and concise manner.

====Not-So-Good====
One of the problems with this paper is that although many of the examples are simplified in order to expediate the understanding of the user, some are a little oversimplified. For example, Figure 9 is a graphic that attempts to represent the evacuation process in a visual manner. Unfortunatly, this ends up making the problem seem almost trivial and does little more than water down the information.

The writers are also a little bit one sided (with understandable reason) on the topic. Although they do admit the limitations of LOOM, they do not spend much time discussing any problems later. There is a large amount of play-up for LOOM without much discussion of the possible problems with it, such as the clients running LOOM may decide not to fix the race conditions and rather just let the program continue to run with LOOM as a permanent fix. This may cause further errors in the long term life of the program.

===References===
<blockquote>You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.</blockquote>

Talk:COMP 3000 Essay 2 2010 Question 5

2010-11-24T22:08:37Z

Dustinmartin: /* Contribution */

Maybe we can all add our names below so we know who's still in this course? --[[User:Myagi|Myagi]] 12:38, 14 November 2010 (UTC)

Group members:

* Michael Yagi
* Nicolas Lessard
* Julie Powers
* Derek Langlois
* Dustin Martin

Jeffrey Francom contacted me earlier so I know he is also still in the course. <strike>Now we are only waiting on Dustin Martin.</strike> Everyone has been accounted for. [[User:J powers|J powers]] 18:07, 15 November 2010 (UTC)

Just kicking things off. Feel free to make suggestions or change anything. --[[User:Myagi|Myagi]] 11:36, 17 November 2010 (UTC)

Edited and filled out the critique section. Edited a little bit here and there. --[[User:Afranco2|Afranco2]] 17:41, 22 November 2010 (UTC)

Moved stuff to the front page and cleaned up references. Still waiting for people to expand if possible. Also, spellcheck ;) --[[User:Myagi|Myagi]] 10:37, 24 November 2010 (UTC)

==Essay==
===Paper===
<blockquote>The paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.</blockquote>
* Title: [http://www.usenix.org/events/osdi10/tech/full_papers/Wu.pdf Bypassing Races in Live Applications with Execution Filters]
* Authors: Jingyue Wu, Heming Cui, Junfeng Yang
* Affiliations: Computer Science Department, Columbia University
* Supplementary Information: [http://homeostasis.scs.carleton.ca/osdi/video/wu.mp4 Video], [http://homeostasis.scs.carleton.ca/osdi/slides/wu.pdf Slides]

===Background Concepts===
<blockquote>Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.</blockquote>

-------------
A race condition is a system flaw that “occurs when two threads access a shared variable at the same time." Race conditions can be very complex, time consuming and expensive to fix. Unfortunately, the most challenging part of race condition is not fixing it, but rather find it. Race conditions are notorious for being extremely difficult to find, isolate and recreate. To help ease this process, the authors of this paper, Jingyue Wu, Heming Cui, Junfeng Yang, propose the adoption of LOOM.

LOOM is a system which dynamically locates and corrects areas which may be susceptible to race condition errors. The power of LOOM rests in its ability to operate on live applications in real time. This is possible thanks to its evacuation algorithm which injects execution filters to fix race conditions at runtime. Execution filters, otherwise known as request filtering, allow you to inspect the request before and after the main logic is executed. By leveraging execution filters as the means for correcting race conditions, LOOM is able to operate with very little performance overhead and is a highly scalable as the number of application threads increases.

The authors tested LOOM on existing real world race conditions found in common applications. The tests found that all tested race conditions were solved, with little performance overhead, in a scalable and easy to implement manor.
-------------

This paper consists of multiple terms which must be familiar to the reader in order to assist in reading the Bypassing Races in Live Applications with Execution Filters paper. These terms are listed and explained below:

* Race Condition: "A race condition occurs when two threads access a shared variable at the same time." [http://support.microsoft.com/kb/317723 Race Condition]
* Execution Filters: Otherwise known as request filtering. Request filters allow you to inspect the request before and after the main logic is executed. These are mutual exclusion filters in the context of this paper.
* Hot Patches: "Hot patching provides a mechanism to update system files without rebooting or stopping services and processes."[http://technet.microsoft.com/en-us/library/cc781109%28WS.10%29.aspx Hot Patching]
* Hybrid Instrumentation Engine: "Instrumentation refers to an ability to monitor or measure the level of a product's performance, to diagnose errors and writing trace information." [http://msdn.microsoft.com/en-us/library/aa983649%28VS.71%29.aspx Instrumentation] Instrument programs can have low runtime overhead, but instrumentation has to be done at compile time. Dynamic instrumentation can update programs at runtime but incur high overhead. A hybrid instrumentation is an implementation of combined static and dynamic instrumentation.
* Lock: A lock is a way of limiting access to a common resource when using multiple threads. Lock and unlock methods are usually called at the beginning and end of a target method, respectively. "Mutual exclusion locks (mutexes) are a common method of serializing thread execution. Mutual exclusion locks synchronize threads, usually by ensuring that only one thread at a time executes a critical section of code. Mutex locks can also preserve single-threaded code." [http://www.cs.cf.ac.uk/Dave/C/node31.html#SECTION003110000000000000000 Mutex Locks]
* Mutex: Unable to be both true at the same time.
* Semaphore: "A semaphore is a protected variable or abstract data type that provides a simple but useful abstraction for controlling access by multiple processes to a common resource in a parallel programming environment." [http://en.wikipedia.org/wiki/Semaphore_%28programming%29 Semaphore]

===Research problem===
<blockquote>What is the research problem being addressed by the paper? How does this problem relate to past related work?</blockquote>
====Problem being addressed====
With the rise of multiple core systems, multithreaded programs are often prone to race conditions. Races are hard to detect, test and debug. Due to the immaturity of current race detectors, this paper explains a new approach to race detection and work arounds through the use of LOOM.
====Past related work====
Two common solutions to fixing deployed races are software updates and hot patches. Software updates require restarts whereas hot patches applies patches to live systems. However, relying on conventional patches can lead to new errors and could be unsafe, due to a multithreaded applications complexity. Releasing a reliable patch takes time, but developers often resort to more efficient fixes rather than placing proper locks in the application due to performance or work pressure.
===Contribution===
<blockquote>What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)</blockquote>
====Current solution expressed====
Compared to traditional solutions, LOOM differs in its approach to race fixes. It is designed to quickly develop safe, optimized, temporary workarounds while a concrete solution is developed. LOOM is also very easy to use. LOOM is compiled with a developers application as a plugin and kept separate from the source code. The plugin will inject the LOOM update into the application binary.

Mutual exclusion filters are written by the developer and synced with the source code to filter out any racy threads. The code declaration used is easy to understand and can be inserted in a code region that need to be mutually exclusive. The developer does not need to deal with low level operations such as lock, unlock and semaphore operations. Users can then download the filter and apply it to the application while it is still live.

LOOM is flexible in that developers can make trade-offs in performance and reliability in their application in conjunction with LOOM. These can include making two code regions mutually exclusive even when accessing different objects or with extreme measures, making them run in single threaded mode.

An evacuation algorithm is used for safety as to not introduce new errors. A critical region is marked using static analysis. All threads in the critical region are then evacuated. After the evacuation is executed, the execution filter is installed and then the threads are resumed after a live update pause is done at a safe location.

LOOM's hybrid instrumentation engine is used to reduce its overhead. The engine statically changes an applications binary to anticipate dynamic updates.

Evaluation of LOOM was based on overhead, scalability, reliability, availability and timeliness. These were demonstrated using Apache and MySQL in conjunction with the multithreaded ApacheBench and SysBench, respectively.

-------------
====Why is it any better than what came before?====
Previously, the two standard ways of fixing deployed race conditions were system updates and hot patches. LOOM is a superior choice to both these options for a number of reasons.

First, the main criticism of the system update approach is that this requires that the system be rebooted. With desktop applications, this can sometimes be considered acceptable. However, server applications often do not have the luxury of being able to reboot because requests are coming from external sources and are expected to be processed.

Second, while hot patches do not require a system reboot, they do have vulnerabilities. Namely, it is very difficult to apply a patch that corrects the error, or errors, but that leaves the rest of the system unaffected. Often when correcting one error, others can appear. The main concern with hot patches however, is that their development is a time consuming process. A process which until developed and deployed, leaves the race condition vulnerable and exposed. The paper chronicles a real world Mozilla race condition whose hot patch took nearly 8 years of development to correct, all the while the vulnerability was exposed to all Mozilla users.
-------------

===Critique===
<blockquote>What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.</blockquote>
====Good====
The authors of this essay are efficient at delivering the information surrounding their thesis both in staying focused on the main thesis as well as backing up thier topics with relevant examples and data. This helps to keep the thesis paramount throughout the paper. Examples throughout the paper, particularly the MySQL example ensure that the use of execution filters is clear to the reader. All of the examples are well documented and some (ex. Figure 2) are simplified as to not confuse the reader with too much unnessicary information. References throughout the writing backup the reliability of the paper and let the user keep track of the sources to properly check information and sources.

The whole essay flows well and the information is delievered in a well put together order, allowing the reader to learn enough about LOOM (or any of the sub-topics involved in the explination) before being informed about the next relative subject. The paper ends with a conclusion that does a good job of wrapping up the whole paper in a clear and concise manner.

====Not-So-Good====
One of the problems with this paper is that although many of the examples are simplified in order to expediate the understanding of the user, some are a little oversimplified. For example, Figure 9 is a graphic that attempts to represent the evacuation process in a visual manner. Unfortunatly, this ends up making the problem seem almost trivial and does little more than water down the information.

The writers are also a little bit one sided (with understandable reason) on the topic. Although they do admit the limitations of LOOM, they do not spend much time discussing any problems later. There is a large amount of play-up for LOOM without much discussion of the possible problems with it, such as the clients running LOOM may decide not to fix the race conditions and rather just let the program continue to run with LOOM as a permanent fix. This may cause further errors in the long term life of the program.

===References===
<blockquote>You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.</blockquote>

Talk:COMP 3000 Essay 2 2010 Question 5

2010-11-24T15:28:03Z

Dustinmartin: /* Background Concepts */

Maybe we can all add our names below so we know who's still in this course? --[[User:Myagi|Myagi]] 12:38, 14 November 2010 (UTC)

Group members:

* Michael Yagi
* Nicolas Lessard
* Julie Powers
* Derek Langlois
* Dustin Martin

Jeffrey Francom contacted me earlier so I know he is also still in the course. <strike>Now we are only waiting on Dustin Martin.</strike> Everyone has been accounted for. [[User:J powers|J powers]] 18:07, 15 November 2010 (UTC)

Just kicking things off. Feel free to make suggestions or change anything. --[[User:Myagi|Myagi]] 11:36, 17 November 2010 (UTC)

Edited and filled out the critique section. Edited a little bit here and there. --[[User:Afranco2|Afranco2]] 17:41, 22 November 2010 (UTC)

==Essay==
===Paper===
<blockquote>The paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.</blockquote>
* Title: [http://www.usenix.org/events/osdi10/tech/full_papers/Wu.pdf Bypassing Races in Live Applications with Execution Filters]
* Authors: Jingyue Wu, Heming Cui, Junfeng Yang
* Affiliations: Computer Science Department, Columbia University
* Supplementary Information: [http://homeostasis.scs.carleton.ca/osdi/video/wu.mp4 Video], [http://homeostasis.scs.carleton.ca/osdi/slides/wu.pdf Slides]

===Background Concepts===
<blockquote>Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.</blockquote>

-------------
A race condition is a system flaw that “occurs when two threads access a shared variable at the same time." Race conditions can be very complex, time consuming and expensive to fix. Unfortunately, the most challenging part of race condition is not fixing it, but rather find it. Race conditions are notorious for being extremely difficult to find, isolate and recreate. To help ease this process, the authors of this paper, Jingyue Wu, Heming Cui, Junfeng Yang, propose the adoption of LOOM.

LOOM is a system which dynamically locates and corrects areas which may be susceptible to race condition errors. The power of LOOM rests in its ability to operate on live applications in real time. This is possible thanks to its evacuation algorithm which injects execution filters to fix race conditions at runtime. Execution filters, otherwise known as request filtering, allow you to inspect the request before and after the main logic is executed. By leveraging execution filters as the means for correcting race conditions, LOOM is able to operate with very little performance overhead and is a highly scalable as the number of application threads increases.

The authors tested LOOM on existing real world race conditions found in common applications. The tests found that all tested race conditions were solved, with little performance overhead, in a scalable and easy to implement manor.
-------------

This paper consists of multiple terms which must be familiar to the reader in order to assist in reading the Bypassing Races in Live Applications with Execution Filters paper. These terms are listed and explained below:

* Race Condition: "A race condition occurs when two threads access a shared variable at the same time." [http://support.microsoft.com/kb/317723 Race Condition]
* Execution Filters: Otherwise known as request filtering. Request filters allow you to inspect the request before and after the main logic is executed. These are mutual exclusion filters in the context of this paper.
* Hot Patches: "Hot patching provides a mechanism to update system files without rebooting or stopping services and processes."[http://technet.microsoft.com/en-us/library/cc781109%28WS.10%29.aspx Hot Patching]
* Hybrid Instrumentation Engine: "Instrumentation refers to an ability to monitor or measure the level of a product's performance, to diagnose errors and writing trace information." [http://msdn.microsoft.com/en-us/library/aa983649%28VS.71%29.aspx Instrumentation] Instrument programs can have low runtime overhead, but instrumentation has to be done at compile time. Dynamic instrumentation can update programs at runtime but incur high overhead. A hybrid instrumentation is an implementation of combined static and dynamic instrumentation.
* Lock: A lock is a way of limiting access to a common resource when using multiple threads. Lock and unlock methods are usually called at the beginning and end of a target method, respectively. "Mutual exclusion locks (mutexes) are a common method of serializing thread execution. Mutual exclusion locks synchronize threads, usually by ensuring that only one thread at a time executes a critical section of code. Mutex locks can also preserve single-threaded code." [http://www.cs.cf.ac.uk/Dave/C/node31.html#SECTION003110000000000000000 Mutex Locks]
* Mutex: Unable to be both true at the same time.
* Semaphore: "A semaphore is a protected variable or abstract data type that provides a simple but useful abstraction for controlling access by multiple processes to a common resource in a parallel programming environment." [http://en.wikipedia.org/wiki/Semaphore_%28programming%29 Semaphore]

===Research problem===
<blockquote>What is the research problem being addressed by the paper? How does this problem relate to past related work?</blockquote>
====Problem being addressed====
With the rise of multiple core systems, multithreaded programs are often prone to race conditions. Races are hard to detect, test and debug. Due to the immaturity of current race detectors, this paper explains a new approach to race detection and work arounds through the use of LOOM.
====Past related work====
Two common solutions to fixing deployed races are software updates and hot patches. Software updates require restarts whereas hot patches applies patches to live systems. However, relying on conventional patches can lead to new errors and could be unsafe, due to a multithreaded applications complexity. Releasing a reliable patch takes time, but developers often resort to more efficient fixes rather than placing proper locks in the application due to performance or work pressure.
===Contribution===
<blockquote>What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)</blockquote>
====Current solution expressed====
Compared to traditional solutions, LOOM differs in its approach to race fixes. It is designed to quickly develop safe, optimized, temporary workarounds while a concrete solution is developed. LOOM is also very easy to use. LOOM is compiled with a developers application as a plugin and kept separate from the source code. The plugin will inject the LOOM update into the application binary.

Mutual exclusion filters are written by the developer and synced with the source code to filter out any racy threads. The code declaration used is easy to understand and can be inserted in a code region that need to be mutually exclusive. The developer does not need to deal with low level operations such as lock, unlock and semaphore operations. Users can then download the filter and apply it to the application while it is still live.

LOOM is flexible in that developers can make trade-offs in performance and reliability in their application in conjunction with LOOM. These can include making two code regions mutually exclusive even when accessing different objects or with extreme measures, making them run in single threaded mode.

An evacuation algorithm is used for safety as to not introduce new errors. A critical region is marked using static analysis. All threads in the critical region are then evacuated. After the evacuation is executed, the execution filter is installed and then the threads are resumed after a live update pause is done at a safe location.

LOOM's hybrid instrumentation engine is used to reduce its overhead. The engine statically changes an applications binary to anticipate dynamic updates.

Evaluation of LOOM was based on overhead, scalability, reliability, availability and timeliness. These were demonstrated using Apache and MySQL in conjunction with the multithreaded ApacheBench and SysBench, respectively.

===Critique===
<blockquote>What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.</blockquote>
====Good====
The authors of this essay are efficient at delivering the information surrounding their thesis both in staying focused on the main thesis as well as backing up thier topics with relevant examples and data. This helps to keep the thesis paramount throughout the paper. Examples throughout the paper, particularly the MySQL example ensure that the use of execution filters is clear to the reader. All of the examples are well documented and some (ex. Figure 2) are simplified as to not confuse the reader with too much unnessicary information. References throughout the writing backup the reliability of the paper and let the user keep track of the sources to properly check information and sources.

The whole essay flows well and the information is delievered in a well put together order, allowing the reader to learn enough about LOOM (or any of the sub-topics involved in the explination) before being informed about the next relative subject. The paper ends with a conclusion that does a good job of wrapping up the whole paper in a clear and concise manner.

====Not-So-Good====
One of the problems with this paper is that although many of the examples are simplified in order to expediate the understanding of the user, some are a little oversimplified. For example, Figure 9 is a graphic that attempts to represent the evacuation process in a visual manner. Unfortunatly, this ends up making the problem seem almost trivial and does little more than water down the information.

The writers are also a little bit one sided (with understandable reason) on the topic. Although they do admit the limitations of LOOM, they do not spend much time discussing any problems later. There is a large amount of play-up for LOOM without much discussion of the possible problems with it, such as the clients running LOOM may decide not to fix the race conditions and rather just let the program continue to run with LOOM as a permanent fix. This may cause further errors in the long term life of the program.

===References===
<blockquote>You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.</blockquote>

Talk:COMP 3000 Essay 2 2010 Question 5

2010-11-24T15:22:48Z

Dustinmartin: /* Background Concepts */

Maybe we can all add our names below so we know who's still in this course? --[[User:Myagi|Myagi]] 12:38, 14 November 2010 (UTC)

Group members:

* Michael Yagi
* Nicolas Lessard
* Julie Powers
* Derek Langlois
* Dustin Martin

Jeffrey Francom contacted me earlier so I know he is also still in the course. <strike>Now we are only waiting on Dustin Martin.</strike> Everyone has been accounted for. [[User:J powers|J powers]] 18:07, 15 November 2010 (UTC)

Just kicking things off. Feel free to make suggestions or change anything. --[[User:Myagi|Myagi]] 11:36, 17 November 2010 (UTC)

Edited and filled out the critique section. Edited a little bit here and there. --[[User:Afranco2|Afranco2]] 17:41, 22 November 2010 (UTC)

==Essay==
===Paper===
<blockquote>The paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.</blockquote>
* Title: [http://www.usenix.org/events/osdi10/tech/full_papers/Wu.pdf Bypassing Races in Live Applications with Execution Filters]
* Authors: Jingyue Wu, Heming Cui, Junfeng Yang
* Affiliations: Computer Science Department, Columbia University
* Supplementary Information: [http://homeostasis.scs.carleton.ca/osdi/video/wu.mp4 Video], [http://homeostasis.scs.carleton.ca/osdi/slides/wu.pdf Slides]

===Background Concepts===
<blockquote>Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.</blockquote>

-------------
A race condition is a system flaw that “occurs when two threads access a shared variable at the same time." Race conditions can be very complex, time consuming and expensive to fix. Unfortunately, the most challenging part of race condition is not fixing it, but rather find it. Race conditions are notorious for being extremely difficult to find, isolate and recreate. To help ease this process, the authors of this paper, Jingyue Wu, Heming Cui, Junfeng Yang, propose the adoption of LOOM.

LOOM is a system which dynamically locates and corrects areas which may be susceptible to race condition errors. The power of LOOM rests in its ability to operate on live applications in real time. To accomplish this, LOOM injects execution filters to filter out the potential race conditions. Execution filiters, otherwise known as request filtering, allow you to inspect the request before and after the main logic is executed. By leveraging execution filters as the means for correcting race conditions, LOOM is able to operate with very little performance overhead and is a highly scalable as the number of application threads increases.
-------------

This paper consists of multiple terms which must be familiar to the reader in order to assist in reading the Bypassing Races in Live Applications with Execution Filters paper. These terms are listed and explained below:

* Race Condition: "A race condition occurs when two threads access a shared variable at the same time." [http://support.microsoft.com/kb/317723 Race Condition]
* Execution Filters: Otherwise known as request filtering. Request filters allow you to inspect the request before and after the main logic is executed. These are mutual exclusion filters in the context of this paper.
* Hot Patches: "Hot patching provides a mechanism to update system files without rebooting or stopping services and processes."[http://technet.microsoft.com/en-us/library/cc781109%28WS.10%29.aspx Hot Patching]
* Hybrid Instrumentation Engine: "Instrumentation refers to an ability to monitor or measure the level of a product's performance, to diagnose errors and writing trace information." [http://msdn.microsoft.com/en-us/library/aa983649%28VS.71%29.aspx Instrumentation] Instrument programs can have low runtime overhead, but instrumentation has to be done at compile time. Dynamic instrumentation can update programs at runtime but incur high overhead. A hybrid instrumentation is an implementation of combined static and dynamic instrumentation.
* Lock: A lock is a way of limiting access to a common resource when using multiple threads. Lock and unlock methods are usually called at the beginning and end of a target method, respectively. "Mutual exclusion locks (mutexes) are a common method of serializing thread execution. Mutual exclusion locks synchronize threads, usually by ensuring that only one thread at a time executes a critical section of code. Mutex locks can also preserve single-threaded code." [http://www.cs.cf.ac.uk/Dave/C/node31.html#SECTION003110000000000000000 Mutex Locks]
* Mutex: Unable to be both true at the same time.
* Semaphore: "A semaphore is a protected variable or abstract data type that provides a simple but useful abstraction for controlling access by multiple processes to a common resource in a parallel programming environment." [http://en.wikipedia.org/wiki/Semaphore_%28programming%29 Semaphore]

===Research problem===
<blockquote>What is the research problem being addressed by the paper? How does this problem relate to past related work?</blockquote>
====Problem being addressed====
With the rise of multiple core systems, multithreaded programs are often prone to race conditions. Races are hard to detect, test and debug. Due to the immaturity of current race detectors, this paper explains a new approach to race detection and work arounds through the use of LOOM.
====Past related work====
Two common solutions to fixing deployed races are software updates and hot patches. Software updates require restarts whereas hot patches applies patches to live systems. However, relying on conventional patches can lead to new errors and could be unsafe, due to a multithreaded applications complexity. Releasing a reliable patch takes time, but developers often resort to more efficient fixes rather than placing proper locks in the application due to performance or work pressure.
===Contribution===
<blockquote>What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)</blockquote>
====Current solution expressed====
Compared to traditional solutions, LOOM differs in its approach to race fixes. It is designed to quickly develop safe, optimized, temporary workarounds while a concrete solution is developed. LOOM is also very easy to use. LOOM is compiled with a developers application as a plugin and kept separate from the source code. The plugin will inject the LOOM update into the application binary.

Mutual exclusion filters are written by the developer and synced with the source code to filter out any racy threads. The code declaration used is easy to understand and can be inserted in a code region that need to be mutually exclusive. The developer does not need to deal with low level operations such as lock, unlock and semaphore operations. Users can then download the filter and apply it to the application while it is still live.

LOOM is flexible in that developers can make trade-offs in performance and reliability in their application in conjunction with LOOM. These can include making two code regions mutually exclusive even when accessing different objects or with extreme measures, making them run in single threaded mode.

An evacuation algorithm is used for safety as to not introduce new errors. A critical region is marked using static analysis. All threads in the critical region are then evacuated. After the evacuation is executed, the execution filter is installed and then the threads are resumed after a live update pause is done at a safe location.

LOOM's hybrid instrumentation engine is used to reduce its overhead. The engine statically changes an applications binary to anticipate dynamic updates.

Evaluation of LOOM was based on overhead, scalability, reliability, availability and timeliness. These were demonstrated using Apache and MySQL in conjunction with the multithreaded ApacheBench and SysBench, respectively.

===Critique===
<blockquote>What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.</blockquote>
====Good====
The authors of this essay are efficient at delivering the information surrounding their thesis both in staying focused on the main thesis as well as backing up thier topics with relevant examples and data. This helps to keep the thesis paramount throughout the paper. Examples throughout the paper, particularly the MySQL example ensure that the use of execution filters is clear to the reader. All of the examples are well documented and some (ex. Figure 2) are simplified as to not confuse the reader with too much unnessicary information. References throughout the writing backup the reliability of the paper and let the user keep track of the sources to properly check information and sources.

The whole essay flows well and the information is delievered in a well put together order, allowing the reader to learn enough about LOOM (or any of the sub-topics involved in the explination) before being informed about the next relative subject. The paper ends with a conclusion that does a good job of wrapping up the whole paper in a clear and concise manner.

====Not-So-Good====
One of the problems with this paper is that although many of the examples are simplified in order to expediate the understanding of the user, some are a little oversimplified. For example, Figure 9 is a graphic that attempts to represent the evacuation process in a visual manner. Unfortunatly, this ends up making the problem seem almost trivial and does little more than water down the information.

The writers are also a little bit one sided (with understandable reason) on the topic. Although they do admit the limitations of LOOM, they do not spend much time discussing any problems later. There is a large amount of play-up for LOOM without much discussion of the possible problems with it, such as the clients running LOOM may decide not to fix the race conditions and rather just let the program continue to run with LOOM as a permanent fix. This may cause further errors in the long term life of the program.

===References===
<blockquote>You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.</blockquote>

Talk:COMP 3000 Essay 2 2010 Question 5

2010-11-22T20:56:37Z

Dustinmartin:

Maybe we can all add our names below so we know who's still in this course? --[[User:Myagi|Myagi]] 12:38, 14 November 2010 (UTC)

Group members:

* Michael Yagi
* Nicolas Lessard
* Julie Powers
* Derek Langlois
* Dustin Martin

Jeffrey Francom contacted me earlier so I know he is also still in the course. <strike>Now we are only waiting on Dustin Martin.</strike> Everyone has been accounted for. [[User:J powers|J powers]] 18:07, 15 November 2010 (UTC)

Just kicking things off. Feel free to make suggestions or change anything. --[[User:Myagi|Myagi]] 11:36, 17 November 2010 (UTC)

Edited and filled out the critique section. Edited a little bit here and there. --[[User:Afranco2|Afranco2]] 17:41, 22 November 2010 (UTC)

==Essay==
===Paper===
<blockquote>The paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.</blockquote>
* Title: [http://www.usenix.org/events/osdi10/tech/full_papers/Wu.pdf Bypassing Races in Live Applications with Execution Filters]
* Authors: Jingyue Wu, Heming Cui, Junfeng Yang
* Affiliations: Computer Science Department, Columbia University
* Supplementary Information: [http://homeostasis.scs.carleton.ca/osdi/video/wu.mp4 Video], [http://homeostasis.scs.carleton.ca/osdi/slides/wu.pdf Slides]

===Background Concepts===
<blockquote>Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.</blockquote>

This paper consists of multiple terms which must be familiar to the reader in order to assist in reading the Bypassing Races in Live Applications with Execution Filters paper. These terms are listed and explained below:

* Execution Filters: Otherwise known as request filtering. Request filters allow you to inspect the request before and after the main logic is executed. These are mutual exclusion filters in the context of this paper.
* Hot Patches: "Hot patching is the application of patches without shutting down and restarting the system or the program concerned. This addresses problems related to unavailability of service provided by the system or the program. A patch that can be applied in this way is called a hot patch. [http://en.wikipedia.org/wiki/Patch_%28computing%29#Hot_patching Hot Patching]
* Hybrid Instrumentation Engine: Instrumentation refers to an ability to monitor or measure the level of a product's performance, to diagnose errors and writing trace information. [http://en.wikipedia.org/wiki/Instrumentation_%28computer_programming%29 Instrumentation] Instrument programs can have low runtime overhead, but instrumentation has to be done at compile time. Dynamic instrumentation can update programs at runtime but incur high overhead. A hybrid instrumentation is an implementation of combined static and dynamic instrumentation.
* Lock: "In computer science, a lock is a synchronization mechanism for enforcing limits on access to a resource in an environment where there are many threads of execution. Locks are one way of enforcing concurrency control policies." [http://en.wikipedia.org/wiki/Lock_%28computer_science%29 Lock]
* Mutex: Mutually Exclusive; Unable to be both true at the same time.

===Research problem===
<blockquote>What is the research problem being addressed by the paper? How does this problem relate to past related work?</blockquote>
====Problem being addressed====
With the rise of multiple core systems, multithreaded programs are often prone to race conditions. Races are hard to detect, test and debug. Due to the immaturity of current race detectors, this paper explains a new approach to race detection and work arounds through the use of LOOM.
====Past related work====
Two common solutions to fixing deployed races are software updates and hot patches. Software updates require restarts whereas hot patches applies patches to live systems. However, relying on conventional patches can lead to new errors and could be unsafe, due to a multithreaded applications complexity. Releasing a reliable patch takes time, but developers often resort to more efficient fixes rather than placing proper locks in the application due to performance or work pressure.
===Contribution===
<blockquote>What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)</blockquote>
====Current solution expressed====
Compared to traditional solutions, LOOM differs in its approach to race fixes. It is designed to quickly develop safe, optimized, temporary workarounds while a concrete solution is developed. LOOM is also very easy to use. LOOM is compiled with a developers application as a plugin and kept separate from the source code. The plugin will inject the LOOM update into the application binary.

Mutual exclusion filters are written by the developer and synced with the source code to filter out any racy threads. The code declaration used is easy to understand and can be inserted in a code region that need to be mutually exclusive. The developer does not need to deal with low level operations such as lock, unlock and semaphore operations. Users can then download the filter and apply it to the application while it is still live.

LOOM is flexible in that developers can make trade-offs in performance and reliability in their application in conjunction with LOOM. These can include making two code regions mutually exclusive even when accessing different objects or with extreme measures, making them run in single threaded mode.

An evacuation algorithm is used for safety as to not introduce new errors. A critical region is marked using static analysis. All threads in the critical region are then evacuated. After the evacuation is executed, the execution filter is installed and then the threads are resumed after a live update pause is done at a safe location.

LOOM's hybrid instrumentation engine is used to reduce its overhead. The engine statically changes an applications binary to anticipate dynamic updates.

Evaluation of LOOM was based on overhead, scalability, reliability, availability and timeliness. These were demonstrated using Apache and MySQL in conjunction with the multithreaded ApacheBench and SysBench, respectively.

===Critique===
<blockquote>What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.</blockquote>
====Good====
The authors of this essay are efficient at delivering the information surrounding their thesis both in staying focused on the main thesis as well as backing up thier topics with relevant examples and data. This helps to keep the thesis paramount throughout the paper. Examples throughout the paper, particularly the MySQL example ensure that the use of execution filters is clear to the reader. All of the examples are well documented and some (ex. Figure 2) are simplified as to not confuse the reader with too much unnessicary information. References throughout the writing backup the reliability of the paper and let the user keep track of the sources to properly check information and sources.

The whole essay flows well and the information is delievered in a well put together order, allowing the reader to learn enough about LOOM (or any of the sub-topics involved in the explination) before being informed about the next relative subject. The paper ends with a conclusion that does a good job of wrapping up the whole paper in a clear and concise manner.

====Not-So-Good====
One of the problems with this paper is that although many of the examples are simplified in order to expediate the understanding of the user, some are a little oversimplified. For example, Figure 9 is a graphic that attempts to represent the evacuation process in a visual manner. Unfortunatly, this ends up making the problem seem almost trivial and does little more than water down the information.

The writers are also a little bit one sided (with understandable reason) on the topic. Although they do admit the limitations of LOOM, they do not spend much time discussing any problems later. There is a large amount of play-up for LOOM without much discussion of the possible problems with it, such as the clients running LOOM may decide not to fix the race conditions and rather just let the program continue to run with LOOM as a permanent fix. This may cause further errors in the long term life of the program.

===References===
<blockquote>You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.</blockquote>

COMP 3000 Essay 1 2010 Question 12

2010-10-15T11:19:33Z

Dustinmartin: /* Why */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful.

We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system.

Unlike the Palm OS, WinFS was not as fortunate and leads a different path, we will examine their fate in the next section.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp [5]]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs [6]]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting[8]]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx [9]], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx[9]] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now [10]] This shows that the team might have had some management problem, which evidently led to the termination of the project.

We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton [11]]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html [12]]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html [13]].

Newton OS uses a database-like store for data. The object storage system within Newton OS has frames, slots, soups, and union soups to handle the data for the OS. The frame is essentially the same as a record database, as that used in Palm OS. “Frames are flexible and can represent a wide variety of structures.”[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]] A slot is like a database field. The slot is used to hold single pieces of data and these slots are contained in a frame. Related frames are stored in soups, Newton's equivalent of a database, that are automatically indexed and can be queried by the index directly, or searched through by keywords. A union soup is a collection of soups of the same name[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html [12]].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys [15]]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [16]]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html [17]][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/ [18]], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/ [19]]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html [20]], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html [21]].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod [22]]. Pixo was subsequently acquired by Apple after the shipping of the first iPods.

We will now explore BeOs in the next section.

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS [23]]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 [25]] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 [26]] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will explore ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html [27]]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4 [28]]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html [29]]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832 [30]]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php [31]]

Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf [32]]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf [33]], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html [34]]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html [35]]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf [36]]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/ [37]].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Gloor, G. PalmOS from the Ground Up - http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2] McKeehan, J., & Rhodes, N. (2002). Palm OS programming: the Developer’s guide. Sebastopol, California, USA: O'Reilly and Associates, Inc.

[3] René Guadalupe Cruz Flores, Juan Ivan Nieto Hipólito, Gabriel López Morteo, and Elitania Jiménez García. - Mobile Operating Systems: a Comparative Analysis in Personal Devices. http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4] Company Website for Garnet OS - http://www.access-company.com/products/platforms/garnet/index.html

[5] Blog of established tech reporter, Paul Thurrott - http://www.winsupersite.com/showcase/winfs_preview.asp

[6] Presentation on WinFS by Ajilion consulting - http://www.slideshare.net/Severus/winfs

[7] MSDN Introduction to WinFS - http://msdn.microsoft.com/en-US/library/aa480687.aspx

[8] Video of the WinFS team discussing WinFS - http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting

[9] Update on WinFS from MSDN - http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx

[10] Interview with WinFS project lead on MSDN - http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now

[11] Getting Started Newton video (that came with the product) - http://www.youtube.com/watch?v=64QuJdJmCbA&feature=player_embedded#!

[12] History of Newton by Pen Computing Magazine - http://www.pencomputing.com/frames/newton_obituary.html

[13] CNET news announcing end of Newton - http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html

[14] Bey C., et al. (1996). Newton Programmer’s Guide for Newton 2.0. Cupertino, California, USA: Apple Computer, Inc. - http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php

[15] http://www.youtube.com/watch?v=xc3JzS0K3ys

[16] Analysis by an Apple-centric magazine - http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html

[17] Opinion piece via CNET - http://news.cnet.com/8301-13512_3-9754359-23.html

[18] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/

[19] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/

[20] CNET reporting on uncertain future of Newton - http://news.cnet.com/2100-1001-206664.html

[21] Apple announcing the discontinuation of Newton - http://www.apple.com/ca/press/1998/02/NewtonDisco.html

[22] San Fransisco Chronicle discussing Pixio - http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod

[23] http://en.wikipedia.org/wiki/BeOS

[24] Giampaolo, D. Practical File System Design with the Be File System. Be Inc. - http://www.letterp.com/~dbg/practical-file-system-design.pdf

[25] History of file systems on ARS Technica - http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

[26] http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

[27] Archive of the Namesys product page for Reiser4 - http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html

[28] http://en.wikipedia.org/wiki/Reiser4

[29] CNET discussing end of Namesys - http://news.cnet.com/8301-13580_3-9851703-39.html

[30] Reiser, H. Kernal Korner: Trees in the Reiser file system. Namesys. - http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832

[31] Analysis of the structure of ReiserFS, by an employee of Namesys - http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php

[32] Bar, M. (2001). ReiserFS for Linux. In Linux File Systems. McGraw-Hill Companies. - http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

[33] Sun file system benchmarks - http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf

[34] Commentry on ReiserFS by Wietse Venema, PhD - http://archives.neohapsis.com/archives/postfix/2001-05/1749.html

[35] Discussion between kernel developers on the University of Indiana Mailing List - http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html

[36] Arpaci-Dusseau, A., Arpaci-Dusseau, R, Prabhakaran, V. Analysis and Evolution of Journaling File Systems. University of Wisconsin, Madison. - http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

[37] Discussion of dropping ReiserFS, by the kernel team lead for SUSE, Jeff Mahoney - http://lwn.net/Articles/202780/

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on Palm OS programming

*https://docs.google.com/document/d/1MgtOOuggnKvutmqpL2rR8ZD4vuKs9wYObX0ifGujUaA/edit?authkey=COnbkrIB# Group discussion 10/13/2010

COMP 3000 Essay 1 2010 Question 12

2010-10-15T11:18:47Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful.

We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system.

Unlike the Palm OS, WinFS was not as fortunate and leads a different path, we will examine their fate in the next section.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp [5]]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs [6]]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting[8]]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx [9]], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx[9]] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now [10]] This shows that the team might have had some management problem, which evidently led to the termination of the project.

We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton [11]]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html [12]]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html [13]].

Newton OS uses a database-like store for data. The object storage system within Newton OS has frames, slots, soups, and union soups to handle the data for the OS. The frame is essentially the same as a record database, as that used in Palm OS. “Frames are flexible and can represent a wide variety of structures.”[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]] A slot is like a database field. The slot is used to hold single pieces of data and these slots are contained in a frame. Related frames are stored in soups, Newton's equivalent of a database, that are automatically indexed and can be queried by the index directly, or searched through by keywords. A union soup is a collection of soups of the same name[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html [12]].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys [15]]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [16]]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html [17]][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/ [18]], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/ [19]]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html [20]], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html [21]].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod [22]]. Pixo was subsequently acquired by Apple after the shipping of the first iPods.

And now we will explore BeOs.

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS [23]]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 [25]] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 [26]] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will explore ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html [27]]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4 [28]]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html [29]]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832 [30]]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php [31]]

Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf [32]]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf [33]], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html [34]]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html [35]]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf [36]]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/ [37]].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Gloor, G. PalmOS from the Ground Up - http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2] McKeehan, J., & Rhodes, N. (2002). Palm OS programming: the Developer’s guide. Sebastopol, California, USA: O'Reilly and Associates, Inc.

[3] René Guadalupe Cruz Flores, Juan Ivan Nieto Hipólito, Gabriel López Morteo, and Elitania Jiménez García. - Mobile Operating Systems: a Comparative Analysis in Personal Devices. http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4] Company Website for Garnet OS - http://www.access-company.com/products/platforms/garnet/index.html

[5] Blog of established tech reporter, Paul Thurrott - http://www.winsupersite.com/showcase/winfs_preview.asp

[6] Presentation on WinFS by Ajilion consulting - http://www.slideshare.net/Severus/winfs

[7] MSDN Introduction to WinFS - http://msdn.microsoft.com/en-US/library/aa480687.aspx

[8] Video of the WinFS team discussing WinFS - http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting

[9] Update on WinFS from MSDN - http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx

[10] Interview with WinFS project lead on MSDN - http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now

[11] Getting Started Newton video (that came with the product) - http://www.youtube.com/watch?v=64QuJdJmCbA&feature=player_embedded#!

[12] History of Newton by Pen Computing Magazine - http://www.pencomputing.com/frames/newton_obituary.html

[13] CNET news announcing end of Newton - http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html

[14] Bey C., et al. (1996). Newton Programmer’s Guide for Newton 2.0. Cupertino, California, USA: Apple Computer, Inc. - http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php

[15] http://www.youtube.com/watch?v=xc3JzS0K3ys

[16] Analysis by an Apple-centric magazine - http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html

[17] Opinion piece via CNET - http://news.cnet.com/8301-13512_3-9754359-23.html

[18] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/

[19] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/

[20] CNET reporting on uncertain future of Newton - http://news.cnet.com/2100-1001-206664.html

[21] Apple announcing the discontinuation of Newton - http://www.apple.com/ca/press/1998/02/NewtonDisco.html

[22] San Fransisco Chronicle discussing Pixio - http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod

[23] http://en.wikipedia.org/wiki/BeOS

[24] Giampaolo, D. Practical File System Design with the Be File System. Be Inc. - http://www.letterp.com/~dbg/practical-file-system-design.pdf

[25] History of file systems on ARS Technica - http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

[26] http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

[27] Archive of the Namesys product page for Reiser4 - http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html

[28] http://en.wikipedia.org/wiki/Reiser4

[29] CNET discussing end of Namesys - http://news.cnet.com/8301-13580_3-9851703-39.html

[30] Reiser, H. Kernal Korner: Trees in the Reiser file system. Namesys. - http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832

[31] Analysis of the structure of ReiserFS, by an employee of Namesys - http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php

[32] Bar, M. (2001). ReiserFS for Linux. In Linux File Systems. McGraw-Hill Companies. - http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

[33] Sun file system benchmarks - http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf

[34] Commentry on ReiserFS by Wietse Venema, PhD - http://archives.neohapsis.com/archives/postfix/2001-05/1749.html

[35] Discussion between kernel developers on the University of Indiana Mailing List - http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html

[36] Arpaci-Dusseau, A., Arpaci-Dusseau, R, Prabhakaran, V. Analysis and Evolution of Journaling File Systems. University of Wisconsin, Madison. - http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

[37] Discussion of dropping ReiserFS, by the kernel team lead for SUSE, Jeff Mahoney - http://lwn.net/Articles/202780/

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on Palm OS programming

*https://docs.google.com/document/d/1MgtOOuggnKvutmqpL2rR8ZD4vuKs9wYObX0ifGujUaA/edit?authkey=COnbkrIB# Group discussion 10/13/2010

COMP 3000 Essay 1 2010 Question 12

2010-10-15T11:17:45Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful.

We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system.

Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp [5]]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs [6]]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting[8]]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx [9]], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx[9]] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now [10]] This shows that the team might have had some management problem, which evidently led to the termination of the project.

We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton [11]]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html [12]]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html [13]].

Newton OS uses a database-like store for data. The object storage system within Newton OS has frames, slots, soups, and union soups to handle the data for the OS. The frame is essentially the same as a record database, as that used in Palm OS. “Frames are flexible and can represent a wide variety of structures.”[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]] A slot is like a database field. The slot is used to hold single pieces of data and these slots are contained in a frame. Related frames are stored in soups, Newton's equivalent of a database, that are automatically indexed and can be queried by the index directly, or searched through by keywords. A union soup is a collection of soups of the same name[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html [12]].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys [15]]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [16]]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html [17]][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/ [18]], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/ [19]]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html [20]], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html [21]].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod [22]]. Pixo was subsequently acquired by Apple after the shipping of the first iPods.

And now we will explore BeOs.

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS [23]]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 [25]] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 [26]] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will explore ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html [27]]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4 [28]]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html [29]]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832 [30]]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php [31]]

Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf [32]]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf [33]], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html [34]]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html [35]]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf [36]]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/ [37]].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Gloor, G. PalmOS from the Ground Up - http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2] McKeehan, J., & Rhodes, N. (2002). Palm OS programming: the Developer’s guide. Sebastopol, California, USA: O'Reilly and Associates, Inc.

[3] René Guadalupe Cruz Flores, Juan Ivan Nieto Hipólito, Gabriel López Morteo, and Elitania Jiménez García. - Mobile Operating Systems: a Comparative Analysis in Personal Devices. http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4] Company Website for Garnet OS - http://www.access-company.com/products/platforms/garnet/index.html

[5] Blog of established tech reporter, Paul Thurrott - http://www.winsupersite.com/showcase/winfs_preview.asp

[6] Presentation on WinFS by Ajilion consulting - http://www.slideshare.net/Severus/winfs

[7] MSDN Introduction to WinFS - http://msdn.microsoft.com/en-US/library/aa480687.aspx

[8] Video of the WinFS team discussing WinFS - http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting

[9] Update on WinFS from MSDN - http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx

[10] Interview with WinFS project lead on MSDN - http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now

[11] Getting Started Newton video (that came with the product) - http://www.youtube.com/watch?v=64QuJdJmCbA&feature=player_embedded#!

[12] History of Newton by Pen Computing Magazine - http://www.pencomputing.com/frames/newton_obituary.html

[13] CNET news announcing end of Newton - http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html

[14] Bey C., et al. (1996). Newton Programmer’s Guide for Newton 2.0. Cupertino, California, USA: Apple Computer, Inc. - http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php

[15] http://www.youtube.com/watch?v=xc3JzS0K3ys

[16] Analysis by an Apple-centric magazine - http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html

[17] Opinion piece via CNET - http://news.cnet.com/8301-13512_3-9754359-23.html

[18] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/

[19] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/

[20] CNET reporting on uncertain future of Newton - http://news.cnet.com/2100-1001-206664.html

[21] Apple announcing the discontinuation of Newton - http://www.apple.com/ca/press/1998/02/NewtonDisco.html

[22] San Fransisco Chronicle discussing Pixio - http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod

[23] http://en.wikipedia.org/wiki/BeOS

[24] Giampaolo, D. Practical File System Design with the Be File System. Be Inc. - http://www.letterp.com/~dbg/practical-file-system-design.pdf

[25] History of file systems on ARS Technica - http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

[26] http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

[27] Archive of the Namesys product page for Reiser4 - http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html

[28] http://en.wikipedia.org/wiki/Reiser4

[29] CNET discussing end of Namesys - http://news.cnet.com/8301-13580_3-9851703-39.html

[30] Reiser, H. Kernal Korner: Trees in the Reiser file system. Namesys. - http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832

[31] Analysis of the structure of ReiserFS, by an employee of Namesys - http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php

[32] Bar, M. (2001). ReiserFS for Linux. In Linux File Systems. McGraw-Hill Companies. - http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

[33] Sun file system benchmarks - http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf

[34] Commentry on ReiserFS by Wietse Venema, PhD - http://archives.neohapsis.com/archives/postfix/2001-05/1749.html

[35] Discussion between kernel developers on the University of Indiana Mailing List - http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html

[36] Arpaci-Dusseau, A., Arpaci-Dusseau, R, Prabhakaran, V. Analysis and Evolution of Journaling File Systems. University of Wisconsin, Madison. - http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

[37] Discussion of dropping ReiserFS, by the kernel team lead for SUSE, Jeff Mahoney - http://lwn.net/Articles/202780/

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on Palm OS programming

*https://docs.google.com/document/d/1MgtOOuggnKvutmqpL2rR8ZD4vuKs9wYObX0ifGujUaA/edit?authkey=COnbkrIB# Group discussion 10/13/2010

COMP 3000 Essay 1 2010 Question 12

2010-10-15T11:17:27Z

Dustinmartin: /* Why */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful.

We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp [5]]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs [6]]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting[8]]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx [9]], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx[9]] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now [10]] This shows that the team might have had some management problem, which evidently led to the termination of the project.

We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton [11]]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html [12]]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html [13]].

Newton OS uses a database-like store for data. The object storage system within Newton OS has frames, slots, soups, and union soups to handle the data for the OS. The frame is essentially the same as a record database, as that used in Palm OS. “Frames are flexible and can represent a wide variety of structures.”[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]] A slot is like a database field. The slot is used to hold single pieces of data and these slots are contained in a frame. Related frames are stored in soups, Newton's equivalent of a database, that are automatically indexed and can be queried by the index directly, or searched through by keywords. A union soup is a collection of soups of the same name[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html [12]].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys [15]]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [16]]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html [17]][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/ [18]], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/ [19]]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html [20]], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html [21]].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod [22]]. Pixo was subsequently acquired by Apple after the shipping of the first iPods.

And now we will explore BeOs.

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS [23]]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 [25]] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 [26]] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will explore ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html [27]]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4 [28]]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html [29]]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832 [30]]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php [31]]

Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf [32]]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf [33]], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html [34]]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html [35]]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf [36]]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/ [37]].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Gloor, G. PalmOS from the Ground Up - http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2] McKeehan, J., & Rhodes, N. (2002). Palm OS programming: the Developer’s guide. Sebastopol, California, USA: O'Reilly and Associates, Inc.

[3] René Guadalupe Cruz Flores, Juan Ivan Nieto Hipólito, Gabriel López Morteo, and Elitania Jiménez García. - Mobile Operating Systems: a Comparative Analysis in Personal Devices. http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4] Company Website for Garnet OS - http://www.access-company.com/products/platforms/garnet/index.html

[5] Blog of established tech reporter, Paul Thurrott - http://www.winsupersite.com/showcase/winfs_preview.asp

[6] Presentation on WinFS by Ajilion consulting - http://www.slideshare.net/Severus/winfs

[7] MSDN Introduction to WinFS - http://msdn.microsoft.com/en-US/library/aa480687.aspx

[8] Video of the WinFS team discussing WinFS - http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting

[9] Update on WinFS from MSDN - http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx

[10] Interview with WinFS project lead on MSDN - http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now

[11] Getting Started Newton video (that came with the product) - http://www.youtube.com/watch?v=64QuJdJmCbA&feature=player_embedded#!

[12] History of Newton by Pen Computing Magazine - http://www.pencomputing.com/frames/newton_obituary.html

[13] CNET news announcing end of Newton - http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html

[14] Bey C., et al. (1996). Newton Programmer’s Guide for Newton 2.0. Cupertino, California, USA: Apple Computer, Inc. - http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php

[15] http://www.youtube.com/watch?v=xc3JzS0K3ys

[16] Analysis by an Apple-centric magazine - http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html

[17] Opinion piece via CNET - http://news.cnet.com/8301-13512_3-9754359-23.html

[18] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/

[19] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/

[20] CNET reporting on uncertain future of Newton - http://news.cnet.com/2100-1001-206664.html

[21] Apple announcing the discontinuation of Newton - http://www.apple.com/ca/press/1998/02/NewtonDisco.html

[22] San Fransisco Chronicle discussing Pixio - http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod

[23] http://en.wikipedia.org/wiki/BeOS

[24] Giampaolo, D. Practical File System Design with the Be File System. Be Inc. - http://www.letterp.com/~dbg/practical-file-system-design.pdf

[25] History of file systems on ARS Technica - http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

[26] http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

[27] Archive of the Namesys product page for Reiser4 - http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html

[28] http://en.wikipedia.org/wiki/Reiser4

[29] CNET discussing end of Namesys - http://news.cnet.com/8301-13580_3-9851703-39.html

[30] Reiser, H. Kernal Korner: Trees in the Reiser file system. Namesys. - http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832

[31] Analysis of the structure of ReiserFS, by an employee of Namesys - http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php

[32] Bar, M. (2001). ReiserFS for Linux. In Linux File Systems. McGraw-Hill Companies. - http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

[33] Sun file system benchmarks - http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf

[34] Commentry on ReiserFS by Wietse Venema, PhD - http://archives.neohapsis.com/archives/postfix/2001-05/1749.html

[35] Discussion between kernel developers on the University of Indiana Mailing List - http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html

[36] Arpaci-Dusseau, A., Arpaci-Dusseau, R, Prabhakaran, V. Analysis and Evolution of Journaling File Systems. University of Wisconsin, Madison. - http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

[37] Discussion of dropping ReiserFS, by the kernel team lead for SUSE, Jeff Mahoney - http://lwn.net/Articles/202780/

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on Palm OS programming

*https://docs.google.com/document/d/1MgtOOuggnKvutmqpL2rR8ZD4vuKs9wYObX0ifGujUaA/edit?authkey=COnbkrIB# Group discussion 10/13/2010

COMP 3000 Essay 1 2010 Question 12

2010-10-15T11:17:07Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful.

We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp [5]]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs [6]]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting[8]]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx [9]], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx[9]] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now [10]] This shows that the team might have had some management problem, which evidently led to the termination of the project.

We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton [11]]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html [12]]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html [13]].

Newton OS uses a database-like store for data. The object storage system within Newton OS has frames, slots, soups, and union soups to handle the data for the OS. The frame is essentially the same as a record database, as that used in Palm OS. “Frames are flexible and can represent a wide variety of structures.”[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]] A slot is like a database field. The slot is used to hold single pieces of data and these slots are contained in a frame. Related frames are stored in soups, Newton's equivalent of a database, that are automatically indexed and can be queried by the index directly, or searched through by keywords. A union soup is a collection of soups of the same name[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html [12]].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys [15]]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [16]]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html [17]][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/ [18]], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/ [19]]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html [20]], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html [21]].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod [22]]. Pixo was subsequently acquired by Apple after the shipping of the first iPods.

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS [23]]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 [25]] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 [26]] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will explore ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html [27]]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4 [28]]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html [29]]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832 [30]]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php [31]]

Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf [32]]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf [33]], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html [34]]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html [35]]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf [36]]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/ [37]].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Gloor, G. PalmOS from the Ground Up - http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2] McKeehan, J., & Rhodes, N. (2002). Palm OS programming: the Developer’s guide. Sebastopol, California, USA: O'Reilly and Associates, Inc.

[3] René Guadalupe Cruz Flores, Juan Ivan Nieto Hipólito, Gabriel López Morteo, and Elitania Jiménez García. - Mobile Operating Systems: a Comparative Analysis in Personal Devices. http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4] Company Website for Garnet OS - http://www.access-company.com/products/platforms/garnet/index.html

[5] Blog of established tech reporter, Paul Thurrott - http://www.winsupersite.com/showcase/winfs_preview.asp

[6] Presentation on WinFS by Ajilion consulting - http://www.slideshare.net/Severus/winfs

[7] MSDN Introduction to WinFS - http://msdn.microsoft.com/en-US/library/aa480687.aspx

[8] Video of the WinFS team discussing WinFS - http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting

[9] Update on WinFS from MSDN - http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx

[10] Interview with WinFS project lead on MSDN - http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now

[11] Getting Started Newton video (that came with the product) - http://www.youtube.com/watch?v=64QuJdJmCbA&feature=player_embedded#!

[12] History of Newton by Pen Computing Magazine - http://www.pencomputing.com/frames/newton_obituary.html

[13] CNET news announcing end of Newton - http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html

[14] Bey C., et al. (1996). Newton Programmer’s Guide for Newton 2.0. Cupertino, California, USA: Apple Computer, Inc. - http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php

[15] http://www.youtube.com/watch?v=xc3JzS0K3ys

[16] Analysis by an Apple-centric magazine - http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html

[17] Opinion piece via CNET - http://news.cnet.com/8301-13512_3-9754359-23.html

[18] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/

[19] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/

[20] CNET reporting on uncertain future of Newton - http://news.cnet.com/2100-1001-206664.html

[21] Apple announcing the discontinuation of Newton - http://www.apple.com/ca/press/1998/02/NewtonDisco.html

[22] San Fransisco Chronicle discussing Pixio - http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod

[23] http://en.wikipedia.org/wiki/BeOS

[24] Giampaolo, D. Practical File System Design with the Be File System. Be Inc. - http://www.letterp.com/~dbg/practical-file-system-design.pdf

[25] History of file systems on ARS Technica - http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

[26] http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

[27] Archive of the Namesys product page for Reiser4 - http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html

[28] http://en.wikipedia.org/wiki/Reiser4

[29] CNET discussing end of Namesys - http://news.cnet.com/8301-13580_3-9851703-39.html

[30] Reiser, H. Kernal Korner: Trees in the Reiser file system. Namesys. - http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832

[31] Analysis of the structure of ReiserFS, by an employee of Namesys - http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php

[32] Bar, M. (2001). ReiserFS for Linux. In Linux File Systems. McGraw-Hill Companies. - http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

[33] Sun file system benchmarks - http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf

[34] Commentry on ReiserFS by Wietse Venema, PhD - http://archives.neohapsis.com/archives/postfix/2001-05/1749.html

[35] Discussion between kernel developers on the University of Indiana Mailing List - http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html

[36] Arpaci-Dusseau, A., Arpaci-Dusseau, R, Prabhakaran, V. Analysis and Evolution of Journaling File Systems. University of Wisconsin, Madison. - http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

[37] Discussion of dropping ReiserFS, by the kernel team lead for SUSE, Jeff Mahoney - http://lwn.net/Articles/202780/

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on Palm OS programming

*https://docs.google.com/document/d/1MgtOOuggnKvutmqpL2rR8ZD4vuKs9wYObX0ifGujUaA/edit?authkey=COnbkrIB# Group discussion 10/13/2010

COMP 3000 Essay 1 2010 Question 12

2010-10-15T11:16:43Z

Dustinmartin: /* Why */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful.

We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

And now we will explore WinFS.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp [5]]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs [6]]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting[8]]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx [9]], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx[9]] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now [10]] This shows that the team might have had some management problem, which evidently led to the termination of the project.

We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton [11]]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html [12]]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html [13]].

Newton OS uses a database-like store for data. The object storage system within Newton OS has frames, slots, soups, and union soups to handle the data for the OS. The frame is essentially the same as a record database, as that used in Palm OS. “Frames are flexible and can represent a wide variety of structures.”[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]] A slot is like a database field. The slot is used to hold single pieces of data and these slots are contained in a frame. Related frames are stored in soups, Newton's equivalent of a database, that are automatically indexed and can be queried by the index directly, or searched through by keywords. A union soup is a collection of soups of the same name[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html [12]].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys [15]]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [16]]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html [17]][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/ [18]], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/ [19]]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html [20]], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html [21]].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod [22]]. Pixo was subsequently acquired by Apple after the shipping of the first iPods.

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS [23]]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 [25]] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 [26]] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will explore ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html [27]]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4 [28]]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html [29]]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832 [30]]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php [31]]

Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf [32]]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf [33]], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html [34]]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html [35]]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf [36]]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/ [37]].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Gloor, G. PalmOS from the Ground Up - http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2] McKeehan, J., & Rhodes, N. (2002). Palm OS programming: the Developer’s guide. Sebastopol, California, USA: O'Reilly and Associates, Inc.

[3] René Guadalupe Cruz Flores, Juan Ivan Nieto Hipólito, Gabriel López Morteo, and Elitania Jiménez García. - Mobile Operating Systems: a Comparative Analysis in Personal Devices. http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4] Company Website for Garnet OS - http://www.access-company.com/products/platforms/garnet/index.html

[5] Blog of established tech reporter, Paul Thurrott - http://www.winsupersite.com/showcase/winfs_preview.asp

[6] Presentation on WinFS by Ajilion consulting - http://www.slideshare.net/Severus/winfs

[7] MSDN Introduction to WinFS - http://msdn.microsoft.com/en-US/library/aa480687.aspx

[8] Video of the WinFS team discussing WinFS - http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting

[9] Update on WinFS from MSDN - http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx

[10] Interview with WinFS project lead on MSDN - http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now

[11] Getting Started Newton video (that came with the product) - http://www.youtube.com/watch?v=64QuJdJmCbA&feature=player_embedded#!

[12] History of Newton by Pen Computing Magazine - http://www.pencomputing.com/frames/newton_obituary.html

[13] CNET news announcing end of Newton - http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html

[14] Bey C., et al. (1996). Newton Programmer’s Guide for Newton 2.0. Cupertino, California, USA: Apple Computer, Inc. - http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php

[15] http://www.youtube.com/watch?v=xc3JzS0K3ys

[16] Analysis by an Apple-centric magazine - http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html

[17] Opinion piece via CNET - http://news.cnet.com/8301-13512_3-9754359-23.html

[18] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/

[19] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/

[20] CNET reporting on uncertain future of Newton - http://news.cnet.com/2100-1001-206664.html

[21] Apple announcing the discontinuation of Newton - http://www.apple.com/ca/press/1998/02/NewtonDisco.html

[22] San Fransisco Chronicle discussing Pixio - http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod

[23] http://en.wikipedia.org/wiki/BeOS

[24] Giampaolo, D. Practical File System Design with the Be File System. Be Inc. - http://www.letterp.com/~dbg/practical-file-system-design.pdf

[25] History of file systems on ARS Technica - http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

[26] http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

[27] Archive of the Namesys product page for Reiser4 - http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html

[28] http://en.wikipedia.org/wiki/Reiser4

[29] CNET discussing end of Namesys - http://news.cnet.com/8301-13580_3-9851703-39.html

[30] Reiser, H. Kernal Korner: Trees in the Reiser file system. Namesys. - http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832

[31] Analysis of the structure of ReiserFS, by an employee of Namesys - http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php

[32] Bar, M. (2001). ReiserFS for Linux. In Linux File Systems. McGraw-Hill Companies. - http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

[33] Sun file system benchmarks - http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf

[34] Commentry on ReiserFS by Wietse Venema, PhD - http://archives.neohapsis.com/archives/postfix/2001-05/1749.html

[35] Discussion between kernel developers on the University of Indiana Mailing List - http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html

[36] Arpaci-Dusseau, A., Arpaci-Dusseau, R, Prabhakaran, V. Analysis and Evolution of Journaling File Systems. University of Wisconsin, Madison. - http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

[37] Discussion of dropping ReiserFS, by the kernel team lead for SUSE, Jeff Mahoney - http://lwn.net/Articles/202780/

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on Palm OS programming

*https://docs.google.com/document/d/1MgtOOuggnKvutmqpL2rR8ZD4vuKs9wYObX0ifGujUaA/edit?authkey=COnbkrIB# Group discussion 10/13/2010

COMP 3000 Essay 1 2010 Question 12

2010-10-15T11:16:14Z

Dustinmartin: /* Answer */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful.

We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

And now we will explore WinFS.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp [5]]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs [6]]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting[8]]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx [9]], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx[9]] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now [10]] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton [11]]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html [12]]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html [13]].

Newton OS uses a database-like store for data. The object storage system within Newton OS has frames, slots, soups, and union soups to handle the data for the OS. The frame is essentially the same as a record database, as that used in Palm OS. “Frames are flexible and can represent a wide variety of structures.”[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]] A slot is like a database field. The slot is used to hold single pieces of data and these slots are contained in a frame. Related frames are stored in soups, Newton's equivalent of a database, that are automatically indexed and can be queried by the index directly, or searched through by keywords. A union soup is a collection of soups of the same name[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html [12]].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys [15]]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [16]]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html [17]][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/ [18]], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/ [19]]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html [20]], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html [21]].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod [22]]. Pixo was subsequently acquired by Apple after the shipping of the first iPods.

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS [23]]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 [25]] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 [26]] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will explore ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html [27]]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4 [28]]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html [29]]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832 [30]]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php [31]]

Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf [32]]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf [33]], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html [34]]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html [35]]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf [36]]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/ [37]].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Gloor, G. PalmOS from the Ground Up - http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2] McKeehan, J., & Rhodes, N. (2002). Palm OS programming: the Developer’s guide. Sebastopol, California, USA: O'Reilly and Associates, Inc.

[3] René Guadalupe Cruz Flores, Juan Ivan Nieto Hipólito, Gabriel López Morteo, and Elitania Jiménez García. - Mobile Operating Systems: a Comparative Analysis in Personal Devices. http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4] Company Website for Garnet OS - http://www.access-company.com/products/platforms/garnet/index.html

[5] Blog of established tech reporter, Paul Thurrott - http://www.winsupersite.com/showcase/winfs_preview.asp

[6] Presentation on WinFS by Ajilion consulting - http://www.slideshare.net/Severus/winfs

[7] MSDN Introduction to WinFS - http://msdn.microsoft.com/en-US/library/aa480687.aspx

[8] Video of the WinFS team discussing WinFS - http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting

[9] Update on WinFS from MSDN - http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx

[10] Interview with WinFS project lead on MSDN - http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now

[11] Getting Started Newton video (that came with the product) - http://www.youtube.com/watch?v=64QuJdJmCbA&feature=player_embedded#!

[12] History of Newton by Pen Computing Magazine - http://www.pencomputing.com/frames/newton_obituary.html

[13] CNET news announcing end of Newton - http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html

[14] Bey C., et al. (1996). Newton Programmer’s Guide for Newton 2.0. Cupertino, California, USA: Apple Computer, Inc. - http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php

[15] http://www.youtube.com/watch?v=xc3JzS0K3ys

[16] Analysis by an Apple-centric magazine - http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html

[17] Opinion piece via CNET - http://news.cnet.com/8301-13512_3-9754359-23.html

[18] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/

[19] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/

[20] CNET reporting on uncertain future of Newton - http://news.cnet.com/2100-1001-206664.html

[21] Apple announcing the discontinuation of Newton - http://www.apple.com/ca/press/1998/02/NewtonDisco.html

[22] San Fransisco Chronicle discussing Pixio - http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod

[23] http://en.wikipedia.org/wiki/BeOS

[24] Giampaolo, D. Practical File System Design with the Be File System. Be Inc. - http://www.letterp.com/~dbg/practical-file-system-design.pdf

[25] History of file systems on ARS Technica - http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

[26] http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

[27] Archive of the Namesys product page for Reiser4 - http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html

[28] http://en.wikipedia.org/wiki/Reiser4

[29] CNET discussing end of Namesys - http://news.cnet.com/8301-13580_3-9851703-39.html

[30] Reiser, H. Kernal Korner: Trees in the Reiser file system. Namesys. - http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832

[31] Analysis of the structure of ReiserFS, by an employee of Namesys - http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php

[32] Bar, M. (2001). ReiserFS for Linux. In Linux File Systems. McGraw-Hill Companies. - http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

[33] Sun file system benchmarks - http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf

[34] Commentry on ReiserFS by Wietse Venema, PhD - http://archives.neohapsis.com/archives/postfix/2001-05/1749.html

[35] Discussion between kernel developers on the University of Indiana Mailing List - http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html

[36] Arpaci-Dusseau, A., Arpaci-Dusseau, R, Prabhakaran, V. Analysis and Evolution of Journaling File Systems. University of Wisconsin, Madison. - http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

[37] Discussion of dropping ReiserFS, by the kernel team lead for SUSE, Jeff Mahoney - http://lwn.net/Articles/202780/

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on Palm OS programming

*https://docs.google.com/document/d/1MgtOOuggnKvutmqpL2rR8ZD4vuKs9wYObX0ifGujUaA/edit?authkey=COnbkrIB# Group discussion 10/13/2010

COMP 3000 Essay 1 2010 Question 12

2010-10-15T11:15:45Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

And now we will explore WinFS.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp [5]]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs [6]]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting[8]]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx [9]], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx[9]] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now [10]] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton [11]]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html [12]]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html [13]].

Newton OS uses a database-like store for data. The object storage system within Newton OS has frames, slots, soups, and union soups to handle the data for the OS. The frame is essentially the same as a record database, as that used in Palm OS. “Frames are flexible and can represent a wide variety of structures.”[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]] A slot is like a database field. The slot is used to hold single pieces of data and these slots are contained in a frame. Related frames are stored in soups, Newton's equivalent of a database, that are automatically indexed and can be queried by the index directly, or searched through by keywords. A union soup is a collection of soups of the same name[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html [12]].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys [15]]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [16]]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html [17]][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/ [18]], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/ [19]]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html [20]], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html [21]].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod [22]]. Pixo was subsequently acquired by Apple after the shipping of the first iPods.

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS [23]]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 [25]] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 [26]] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will explore ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html [27]]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4 [28]]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html [29]]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832 [30]]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php [31]]

Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf [32]]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf [33]], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html [34]]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html [35]]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf [36]]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/ [37]].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Gloor, G. PalmOS from the Ground Up - http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2] McKeehan, J., & Rhodes, N. (2002). Palm OS programming: the Developer’s guide. Sebastopol, California, USA: O'Reilly and Associates, Inc.

[3] René Guadalupe Cruz Flores, Juan Ivan Nieto Hipólito, Gabriel López Morteo, and Elitania Jiménez García. - Mobile Operating Systems: a Comparative Analysis in Personal Devices. http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4] Company Website for Garnet OS - http://www.access-company.com/products/platforms/garnet/index.html

[5] Blog of established tech reporter, Paul Thurrott - http://www.winsupersite.com/showcase/winfs_preview.asp

[6] Presentation on WinFS by Ajilion consulting - http://www.slideshare.net/Severus/winfs

[7] MSDN Introduction to WinFS - http://msdn.microsoft.com/en-US/library/aa480687.aspx

[8] Video of the WinFS team discussing WinFS - http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting

[9] Update on WinFS from MSDN - http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx

[10] Interview with WinFS project lead on MSDN - http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now

[11] Getting Started Newton video (that came with the product) - http://www.youtube.com/watch?v=64QuJdJmCbA&feature=player_embedded#!

[12] History of Newton by Pen Computing Magazine - http://www.pencomputing.com/frames/newton_obituary.html

[13] CNET news announcing end of Newton - http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html

[14] Bey C., et al. (1996). Newton Programmer’s Guide for Newton 2.0. Cupertino, California, USA: Apple Computer, Inc. - http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php

[15] http://www.youtube.com/watch?v=xc3JzS0K3ys

[16] Analysis by an Apple-centric magazine - http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html

[17] Opinion piece via CNET - http://news.cnet.com/8301-13512_3-9754359-23.html

[18] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/

[19] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/

[20] CNET reporting on uncertain future of Newton - http://news.cnet.com/2100-1001-206664.html

[21] Apple announcing the discontinuation of Newton - http://www.apple.com/ca/press/1998/02/NewtonDisco.html

[22] San Fransisco Chronicle discussing Pixio - http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod

[23] http://en.wikipedia.org/wiki/BeOS

[24] Giampaolo, D. Practical File System Design with the Be File System. Be Inc. - http://www.letterp.com/~dbg/practical-file-system-design.pdf

[25] History of file systems on ARS Technica - http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

[26] http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

[27] Archive of the Namesys product page for Reiser4 - http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html

[28] http://en.wikipedia.org/wiki/Reiser4

[29] CNET discussing end of Namesys - http://news.cnet.com/8301-13580_3-9851703-39.html

[30] Reiser, H. Kernal Korner: Trees in the Reiser file system. Namesys. - http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832

[31] Analysis of the structure of ReiserFS, by an employee of Namesys - http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php

[32] Bar, M. (2001). ReiserFS for Linux. In Linux File Systems. McGraw-Hill Companies. - http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

[33] Sun file system benchmarks - http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf

[34] Commentry on ReiserFS by Wietse Venema, PhD - http://archives.neohapsis.com/archives/postfix/2001-05/1749.html

[35] Discussion between kernel developers on the University of Indiana Mailing List - http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html

[36] Arpaci-Dusseau, A., Arpaci-Dusseau, R, Prabhakaran, V. Analysis and Evolution of Journaling File Systems. University of Wisconsin, Madison. - http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

[37] Discussion of dropping ReiserFS, by the kernel team lead for SUSE, Jeff Mahoney - http://lwn.net/Articles/202780/

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on Palm OS programming

*https://docs.google.com/document/d/1MgtOOuggnKvutmqpL2rR8ZD4vuKs9wYObX0ifGujUaA/edit?authkey=COnbkrIB# Group discussion 10/13/2010

COMP 3000 Essay 1 2010 Question 12

2010-10-15T11:15:18Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp [5]]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs [6]]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting[8]]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx [9]], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx[9]] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now [10]] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton [11]]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html [12]]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html [13]].

Newton OS uses a database-like store for data. The object storage system within Newton OS has frames, slots, soups, and union soups to handle the data for the OS. The frame is essentially the same as a record database, as that used in Palm OS. “Frames are flexible and can represent a wide variety of structures.”[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]] A slot is like a database field. The slot is used to hold single pieces of data and these slots are contained in a frame. Related frames are stored in soups, Newton's equivalent of a database, that are automatically indexed and can be queried by the index directly, or searched through by keywords. A union soup is a collection of soups of the same name[http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php [14]].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html [12]].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys [15]]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [16]]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html [17]][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html [18]]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/ [18]], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/ [19]]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html [20]], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html [21]].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod [22]]. Pixo was subsequently acquired by Apple after the shipping of the first iPods.

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS [23]]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 [25]] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 [26]] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf [24]] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will explore ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html [27]]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4 [28]]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html [29]]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832 [30]]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php [31]]

Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf [32]]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf [33]], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html [34]]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html [35]]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf [36]]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/ [37]].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Gloor, G. PalmOS from the Ground Up - http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2] McKeehan, J., & Rhodes, N. (2002). Palm OS programming: the Developer’s guide. Sebastopol, California, USA: O'Reilly and Associates, Inc.

[3] René Guadalupe Cruz Flores, Juan Ivan Nieto Hipólito, Gabriel López Morteo, and Elitania Jiménez García. - Mobile Operating Systems: a Comparative Analysis in Personal Devices. http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4] Company Website for Garnet OS - http://www.access-company.com/products/platforms/garnet/index.html

[5] Blog of established tech reporter, Paul Thurrott - http://www.winsupersite.com/showcase/winfs_preview.asp

[6] Presentation on WinFS by Ajilion consulting - http://www.slideshare.net/Severus/winfs

[7] MSDN Introduction to WinFS - http://msdn.microsoft.com/en-US/library/aa480687.aspx

[8] Video of the WinFS team discussing WinFS - http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting

[9] Update on WinFS from MSDN - http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx

[10] Interview with WinFS project lead on MSDN - http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now

[11] Getting Started Newton video (that came with the product) - http://www.youtube.com/watch?v=64QuJdJmCbA&feature=player_embedded#!

[12] History of Newton by Pen Computing Magazine - http://www.pencomputing.com/frames/newton_obituary.html

[13] CNET news announcing end of Newton - http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html

[14] Bey C., et al. (1996). Newton Programmer’s Guide for Newton 2.0. Cupertino, California, USA: Apple Computer, Inc. - http://newtonpda.com/docs/NewtonProgrammerGuide20/53.php

[15] http://www.youtube.com/watch?v=xc3JzS0K3ys

[16] Analysis by an Apple-centric magazine - http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html

[17] Opinion piece via CNET - http://news.cnet.com/8301-13512_3-9754359-23.html

[18] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/

[19] BNET reporting on Apple merger - http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/

[20] CNET reporting on uncertain future of Newton - http://news.cnet.com/2100-1001-206664.html

[21] Apple announcing the discontinuation of Newton - http://www.apple.com/ca/press/1998/02/NewtonDisco.html

[22] San Fransisco Chronicle discussing Pixio - http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod

[23] http://en.wikipedia.org/wiki/BeOS

[24] Giampaolo, D. Practical File System Design with the Be File System. Be Inc. - http://www.letterp.com/~dbg/practical-file-system-design.pdf

[25] History of file systems on ARS Technica - http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

[26] http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

[27] Archive of the Namesys product page for Reiser4 - http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html

[28] http://en.wikipedia.org/wiki/Reiser4

[29] CNET discussing end of Namesys - http://news.cnet.com/8301-13580_3-9851703-39.html

[30] Reiser, H. Kernal Korner: Trees in the Reiser file system. Namesys. - http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832

[31] Analysis of the structure of ReiserFS, by an employee of Namesys - http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php

[32] Bar, M. (2001). ReiserFS for Linux. In Linux File Systems. McGraw-Hill Companies. - http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

[33] Sun file system benchmarks - http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf

[34] Commentry on ReiserFS by Wietse Venema, PhD - http://archives.neohapsis.com/archives/postfix/2001-05/1749.html

[35] Discussion between kernel developers on the University of Indiana Mailing List - http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html

[36] Arpaci-Dusseau, A., Arpaci-Dusseau, R, Prabhakaran, V. Analysis and Evolution of Journaling File Systems. University of Wisconsin, Madison. - http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

[37] Discussion of dropping ReiserFS, by the kernel team lead for SUSE, Jeff Mahoney - http://lwn.net/Articles/202780/

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on Palm OS programming

*https://docs.google.com/document/d/1MgtOOuggnKvutmqpL2rR8ZD4vuKs9wYObX0ifGujUaA/edit?authkey=COnbkrIB# Group discussion 10/13/2010

COMP 3000 Essay 1 2010 Question 12

2010-10-14T22:31:21Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning,[url=http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T20:24:53Z

Dustinmartin: /* Conclusion */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning,<ref>[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]</ref>. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these database-like systems that were examined, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{reflist}}
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T20:24:41Z

Dustinmartin: /* Answer */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning,<ref>[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]</ref>. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

-------

Looking at these database-like systems that were examined, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{reflist}}
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T20:21:17Z

Dustinmartin: /* Conclusion */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning,<ref>[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]</ref>. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

-------

Looking at these database-like systems that were examined, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{reflist}}
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T20:06:49Z

Dustinmartin: /* Conclusion */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

Looking at these database-like systems that were examined, it becomes apparent that despite their potential, each did not live up to its expectations.

y useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{reflist}}
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T20:00:55Z

Dustinmartin: /* Answer */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

What about something like this ?

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{reflist}}
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T19:43:59Z

Dustinmartin: /* File System */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

What about something like this ?

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system that use a database as file system , all chosen for certain features that made sense in that application. We’ll visit five different operating systems that used a database as file system. Both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking. Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{reflist}}
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T19:38:34Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

What about something like this ?

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system that use a database as file system , all chosen for certain features that made sense in that application. We’ll visit five different operating systems that used a database as file system. Both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking. Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{reflist}}
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T19:12:19Z

Dustinmartin: /* Operating System */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

What about something like this ?

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system that use a database as file system , all chosen for certain features that made sense in that application. We’ll visit five different operating systems that used a database as file system. Both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking. Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===
Ultimately, BeOS was acquired by Palm

BeOs

ktop users and a competitor to Mac OS and Microsoft Windows. However, it was ultimately unable to achieve a significant market share and proved commercially unviable for Be Inc. The company was acquired by Palm Inc. and today BeOS is mainly used and developed by a small population of enthusiasts.
The open-source OS Haiku is designed to start up where BeOS left off. Alpha 2 of Haiku was released in May 2010.[1]

Microsoft execlusive lisence with hardware manufacture made harder to enter the market

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{reflist}}
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:56:42Z

Dustinmartin: /* Answer */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

What about something like this ?

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system that use a database as file system , all chosen for certain features that made sense in that application. We’ll visit five different operating systems that used a database as file system. Both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===
Ultimately, BeOS was acquired by Palm

BeOs

ktop users and a competitor to Mac OS and Microsoft Windows. However, it was ultimately unable to achieve a significant market share and proved commercially unviable for Be Inc. The company was acquired by Palm Inc. and today BeOS is mainly used and developed by a small population of enthusiasts.
The open-source OS Haiku is designed to start up where BeOS left off. Alpha 2 of Haiku was released in May 2010.[1]

Microsoft execlusive lisence with hardware manufacture made harder to enter the market

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:50:12Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

What about something like this ?

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system that use a database as file system , all chosen for certain features that made sense in that application. We’ll visit 5 differents operating systems that used a database as file system. First palmOS used database mainly for efficiently. WinFS was a file system for the operating system Windows that used database for efficiently too. NewtonOS was produced to be used on a desktop machine, but then became one of the best operating system for pda with a very good security. BeOS was a new operating system and decided to use a database for file system because of efficiently and for compability reason. ReiserFS was more of an experience that turned into a new file system using database. We will first examine the trials of PalmOS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===
Ultimately, BeOS was acquired by Palm

BeOs

ktop users and a competitor to Mac OS and Microsoft Windows. However, it was ultimately unable to achieve a significant market share and proved commercially unviable for Be Inc. The company was acquired by Palm Inc. and today BeOS is mainly used and developed by a small population of enthusiasts.
The open-source OS Haiku is designed to start up where BeOS left off. Alpha 2 of Haiku was released in May 2010.[1]

Microsoft execlusive lisence with hardware manufacture made harder to enter the market

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:44:12Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===
The fate of BeOS

Microsoft execlusive lisence with hardware manufacture made harder to enter the market

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Talk:COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:43:19Z

Dustinmartin: /* Note to Group */

http://portal.acm.org.proxy.library.carleton.ca/citation.cfm?id=364716.364719&coll=Portal&dl=GUIDE&CFID=107313098&CFTOKEN=63043022

http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php It looks to be pretty accurate

http://www.ntfs.com/winfs_basics.htm

http://blogs.msdn.com/b/winfs/ The team blog

http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now <-- If you are bored and want to learn about WinFs, 54 min video.

http://www.splorp.com/newton/faq/newton-faq-nos.html#IIID3c an overview of the Newton OS filesystem to help you out --Rannath

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 BeOS --Rannath

I found some other stuff for Palm OS data base, but didn't find any good yet.

All the above links give good explanation of the subject. I will try to update later.

-Jean-Benoit

----

I found a little bit more in depth about NewtonOS:
http://lowendmac.com/orchard/06/john-sculley-newton-origin.html

Also, another name for Palm OS is Garnet OS. I haven't had time yet to look too much into this, but the wiki has many links to references and to the main site:
http://en.wikipedia.org/wiki/Palm_OS

Edit:
Since there are 5 database systems listed, and 5 people in the group, we can each pick one to work on. If anyone needs help with the one they pick, holla! Also, if you find another database system you want to work on, just add it to the list and put your name beside it.

WinFS - Jean-Benoit

ReiserFS - Sarah L

Newton OS - Daniel

PalmOS/Garnet OS - Tuan Pham

BeOS - Xi Chen

Other (list)...

-Daniel

----
Just to check how everybody are going? There is still two people who haven't picked an FileSystem, is there a problem or you simply didn't write your name next to it?

Also, for others i found an other database System made by Oracle. It is called : Internet File System. http://download.oracle.com/docs/html/A90434_01/toc.htm Here a good link to get started.

I'll probably posted in the page what I have written so far later today.

Also, to keep consistence, I don't know how you are writing yours, but mine will look something like that :
Short history of Winfs, predecessor , etc. (by short i mean roughly 200-300 words)
Short explanation on how WinFs works.
It's fate and Why

If you have any other idea, please share it.

-Jean-Benoit

----
Hey I am curious to how we are going to put all this together, are we each writing an essay for our chosen OS and then edit all the essays into one big one?

-Tuan Pham

----

I was thinking that we could simply have an introduction to express the general advantages and disadvantages of Database as filesystem, then create
sections for our different OS. Then at the end a conclusion.

But for that to work, we need most of our essay to be done in the next 2 days or so. Tuesday during the day we would need all the essay to be done so we can edit each other and write introduction + conclusion.

What do you think ? Any idea?

Edit : I just posted what I have so far. If you have any idea to add anything , just let me know. Also, English isn't my first language, so feel free to fix any of my errors.

I'll add the reference tomorow

-Jean-Benoit

----
Yeah i think we should do that but how about the two section that does not have the other group member's names beside?

- Tuan Pham

----

I will have to post my part of the project sometime Wednesday (most likely before 2pm). I apologize if this causes some problems for the group, but I have an assignment for another class that needs to be done by Wednesday, and I have little time Tuesday to work on this (work and classes). Here is a little summary of Newton OS to get an idea. I will make sure that by Wednesday afternoon I will have at least a near-finished copy of my work (some input may be required :p ).

Newton OS was used on PDAs and doesn’t have a file system, but instead uses a shallow database system. The OS considers each inserted card, as well as its internal storage, as separate stores. Each store contains either read/write databases or read-only objects. Newton eventually was cancelled and became an Apple subsidiary company, then shortly thereafter was reabsorbed into Apple. Developers from the Newton project helped with the iPod OS and the project may have had a hand in the iPhone’s development.

Edit: I left a notice to the group on the front page. Hopefully the other two members will see the notice. And about the two other sections: If we have at least 3 sections, we should be good, but hopefully we hear word from the other two members...

- Daniel

----
I just posted what i have so far. I am having difficulties finding the fate and why so if you guys know anything let me know. Also let me know if you have any suggestions for my part.

- Tuan Pham
----
Sorry, I couldn't post this weekend. I'll gladly work on MurderFS/ReiserFS, and I can definitely finish before Wednesday afternoon (2ish).

I really like how the WinFS sectioned is structured (although maybe condense the fate/why sections for readability?), it seems easy to follow. I agree we should all use the same sections (with additional subsections if need be). It'll make it look a lot cleaner/better put together.

- Sarah

Sorry, guys
Seems like I am the last member of the group, I am now doing works on the BeOS, and some stuff will be posted tonight or tomorrow morning.
Again, sorry for showing up late.

Xi Chen

Hi guys, Luc (lab TA ¬.¬) here, I'm seeing some good collaboration here, but I'm concerned about how coherent the end product will be. I think you guys need to spend some time working out the central thesis and how the various components will advance that point before everybody goes off to work on their little piece.

--[[User:3maisons|3maisons]] 20:14, 12 October 2010 (UTC)

----
I agree. I believe the prof said today that he wants a thesis and we use the examples (WinFS, ReiserFS, Newton OS, PalmOS/Garnet OS, and BeOS) to prove our argument. So we should discuss a thesis and how to format the essay

- Tuan Pham

That sounds good, and you guys thinking about meeting up sometime in lab?
or we can chat through gTalk or the wiki.

ps: my gmail acct is xintai1985@gmail.com
--Xi Chen

---

It could be a good idea to meet up to put everything together.
Or atleast use gTalk/msn/ whatever else people would like to use.
As we only have 2 days remaining, what about tonight ?

And as Tuan said, we would need a thesis subject and then prove why it fails most of the time.
I'm down to work on it whenever you guys are ready (or so obviously). Just send me an email to : jbout13@hotmail.com

-Jean-Benoit

----

I will be available until 2pm today (Wednesday) and then from 7pm onwards. I think we should discuss the thesis and such tonight (if possible), then finish the essay up after class tomorrow morning. My email is demongyro@gmail.com.

-Daniel

----

Let's try to be all around tonight for 7:30 ish ? so we can get thesis and start writing introduction + conclusion and to put everything together.

We either can go to Carleton, or msn / Gmail. (or any other option I forgot) I'm down for both of these.

xintai1985@gmail.com Xi Chen 613-799-5893(in case)

jbout13@hotmail.com Jean-Benoit

demongyro@gmail.com Daniel - on gmail

tuan-pham@hotmail.com Tuan Pham - lets do it over msn/gtalk

Edit : Daniel and I are on Gmail, gtalk.
Add him or me to join the conversation.

Jean-Benoit

Guys add my gmail, t.m.pham3@gmail.com - Tuan Pham

----
http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php
http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf
http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

so i don't lose these

Reiser4, Part II: Designing Trees that Cache Well
Performance Evaluation of Linux File Systems for Data Warehousing Workloads

sarah
----
moving this here because i'm a little ocd...

=Note to Group=

=Contact Details=
Dustin Martin: 613-325-9427 dustin.adam.martin@gmail.com

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:40:55Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===
The fate of BeOS

Microsoft execlusive lisence with hardware manufacture made harder to enter the market

Schmalensee didn't mention that Be had trou- 34
ble giving away the BeOS operating system.
Gassée approached a number of PC manufacturers
to see if they would include BeOS
on their machines and give users the chance
to switch between two operating systems.
Gassée found, to no one's real surprise,
that Microsoft's contracts with manufacturers
made it difficult, if not practically impossible,
to get BeOS in customers' hands. Microsoft
controlled much of what the user got to see
and insisted on almost total control over the
viewer's experience. Schmalensee didn't
mention these details in his testimony. BeOS

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:40:23Z

Dustinmartin: /* File System */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===

Microsoft execlusive lisence with hardware manufacture made harder to enter the market

Schmalensee didn't mention that Be had trou- 34
ble giving away the BeOS operating system.
Gassée approached a number of PC manufacturers
to see if they would include BeOS
on their machines and give users the chance
to switch between two operating systems.
Gassée found, to no one's real surprise,
that Microsoft's contracts with manufacturers
made it difficult, if not practically impossible,
to get BeOS in customers' hands. Microsoft
controlled much of what the user got to see
and insisted on almost total control over the
viewer's experience. Schmalensee didn't
mention these details in his testimony. BeOS

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:38:16Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the journaling file system implemented by BFS, stores all data in a journaled database, but allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===

Microsoft execlusive lisence with hardware manufacture made harder to enter the market

Schmalensee didn't mention that Be had trou- 34
ble giving away the BeOS operating system.
Gassée approached a number of PC manufacturers
to see if they would include BeOS
on their machines and give users the chance
to switch between two operating systems.
Gassée found, to no one's real surprise,
that Microsoft's contracts with manufacturers
made it difficult, if not practically impossible,
to get BeOS in customers' hands. Microsoft
controlled much of what the user got to see
and insisted on almost total control over the
viewer's experience. Schmalensee didn't
mention these details in his testimony. BeOS

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:35:12Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the journaling file system implemented by BFS, stores all data in a journaled database, but allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===

Microsoft execlusive lisence with hardware manufacture made harder to enter the market

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:28:57Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database stores?**

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the journaling file system implemented by BFS, stores all data in a journaled database, but allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===

Schmalensee didn't mention that Be had trou- 34
ble giving away the BeOS operating system.
Gassée approached a number of PC manufacturers
to see if they would include BeOS
on their machines and give users the chance
to switch between two operating systems.
Gassée found, to no one's real surprise,
that Microsoft's contracts with manufacturers
made it difficult, if not practically impossible,
to get BeOS in customers' hands. Microsoft
controlled much of what the user got to see
and insisted on almost total control over the
viewer's experience. Schmalensee didn't
mention these details in his testimony. BeOS

Microsoft execlusive lisence with hardware manufacture made harder to enter the market

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:26:55Z

Dustinmartin: /* BeOS */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database stores?**

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the journaling file system implemented by BFS, stores all data in a journaled database, but allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:25:17Z

Dustinmartin: /* File System */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database stores?**

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the journaling file system implemented by BFS, stores all data in a journaled database, but allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:06:49Z

Dustinmartin: /* File System */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the journaling file system implemented by BFS, stores all data in a journaled database, but allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any posible data format of their choosing.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T17:27:42Z

Dustinmartin: /* History */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so made WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

**Obscure 1st sentence: data would be simply data? Elaborate.**
Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be simply data. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

**Reference for quotation needed**
As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, our "...current file system does not know how to collect and find information within these new types of data", but with data treated as data inside a database, we could find what we are looking for quite easily.

**"In our...to Bob." Needs to be easier to read/understand. Also, treat data as data? Elaborate. Also, any reference?**
Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time find all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.

What if data could do actions that follow a specific set of rules? This is exactly what WinFS wanted to implement. "WinFS Rules are a built-in component of the system that allows you to tell the system how to work with, sort, and deliver your data." Also, it could make use of other applications on the system. So if we received a picture from our friend Bob, we could automatically transfer it elsewhere. [http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developer to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensure that it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and is the default file system on several Linux distributions, most notably SUSE for a short while. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 stopped being commercially supported [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback for any file system.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only recieving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great improvement on the traditional file system. However, for the average desktop, object stores preform somewhat less efficiently than the traditional model, due to scalability issues. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice. It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage, so there is no guarantee that object stores will become a widely used technology even if these problems are worked out.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T17:27:33Z

Dustinmartin: /* Database Structure */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so made WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

==Palm OS==

===History===
**edited**

PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

**Obscure 1st sentence: data would be simply data? Elaborate.**
Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be simply data. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

**Reference for quotation needed**
As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, our "...current file system does not know how to collect and find information within these new types of data", but with data treated as data inside a database, we could find what we are looking for quite easily.

**"In our...to Bob." Needs to be easier to read/understand. Also, treat data as data? Elaborate. Also, any reference?**
Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time find all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.

What if data could do actions that follow a specific set of rules? This is exactly what WinFS wanted to implement. "WinFS Rules are a built-in component of the system that allows you to tell the system how to work with, sort, and deliver your data." Also, it could make use of other applications on the system. So if we received a picture from our friend Bob, we could automatically transfer it elsewhere. [http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developer to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensure that it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and is the default file system on several Linux distributions, most notably SUSE for a short while. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 stopped being commercially supported [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback for any file system.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only recieving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great improvement on the traditional file system. However, for the average desktop, object stores preform somewhat less efficiently than the traditional model, due to scalability issues. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice. It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage, so there is no guarantee that object stores will become a widely used technology even if these problems are worked out.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T17:27:21Z

Dustinmartin: /* Problem */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so made WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

==Palm OS==

===History===
**edited**

PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
**edited**

Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

**Obscure 1st sentence: data would be simply data? Elaborate.**
Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be simply data. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

**Reference for quotation needed**
As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, our "...current file system does not know how to collect and find information within these new types of data", but with data treated as data inside a database, we could find what we are looking for quite easily.

**"In our...to Bob." Needs to be easier to read/understand. Also, treat data as data? Elaborate. Also, any reference?**
Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time find all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.

What if data could do actions that follow a specific set of rules? This is exactly what WinFS wanted to implement. "WinFS Rules are a built-in component of the system that allows you to tell the system how to work with, sort, and deliver your data." Also, it could make use of other applications on the system. So if we received a picture from our friend Bob, we could automatically transfer it elsewhere. [http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developer to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensure that it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and is the default file system on several Linux distributions, most notably SUSE for a short while. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 stopped being commercially supported [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback for any file system.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only recieving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great improvement on the traditional file system. However, for the average desktop, object stores preform somewhat less efficiently than the traditional model, due to scalability issues. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice. It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage, so there is no guarantee that object stores will become a widely used technology even if these problems are worked out.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T17:26:52Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so made WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

==Palm OS==

===History===
**edited**

PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
**edited**

Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
**edited**

Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

**Obscure 1st sentence: data would be simply data? Elaborate.**
Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be simply data. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

**Reference for quotation needed**
As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, our "...current file system does not know how to collect and find information within these new types of data", but with data treated as data inside a database, we could find what we are looking for quite easily.

**"In our...to Bob." Needs to be easier to read/understand. Also, treat data as data? Elaborate. Also, any reference?**
Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time find all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.

What if data could do actions that follow a specific set of rules? This is exactly what WinFS wanted to implement. "WinFS Rules are a built-in component of the system that allows you to tell the system how to work with, sort, and deliver your data." Also, it could make use of other applications on the system. So if we received a picture from our friend Bob, we could automatically transfer it elsewhere. [http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developer to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensure that it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and is the default file system on several Linux distributions, most notably SUSE for a short while. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 stopped being commercially supported [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback for any file system.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only recieving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great improvement on the traditional file system. However, for the average desktop, object stores preform somewhat less efficiently than the traditional model, due to scalability issues. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice. It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage, so there is no guarantee that object stores will become a widely used technology even if these problems are worked out.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T17:26:15Z

Dustinmartin: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so made WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

==Palm OS==

===History===
**edited**

PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
**edited**

Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
**edited**

Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In Palm OS 3.0, their solution to the problem was to create a single large storage heap instead of multiple smaller heaps.

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

**Obscure 1st sentence: data would be simply data? Elaborate.**
Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be simply data. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

**Reference for quotation needed**
As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, our "...current file system does not know how to collect and find information within these new types of data", but with data treated as data inside a database, we could find what we are looking for quite easily.

**"In our...to Bob." Needs to be easier to read/understand. Also, treat data as data? Elaborate. Also, any reference?**
Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time find all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.

What if data could do actions that follow a specific set of rules? This is exactly what WinFS wanted to implement. "WinFS Rules are a built-in component of the system that allows you to tell the system how to work with, sort, and deliver your data." Also, it could make use of other applications on the system. So if we received a picture from our friend Bob, we could automatically transfer it elsewhere. [http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developer to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensure that it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and is the default file system on several Linux distributions, most notably SUSE for a short while. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 stopped being commercially supported [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback for any file system.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only recieving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great improvement on the traditional file system. However, for the average desktop, object stores preform somewhat less efficiently than the traditional model, due to scalability issues. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice. It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage, so there is no guarantee that object stores will become a widely used technology even if these problems are worked out.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T14:52:53Z

Dustinmartin: /* Brief Concept */

COMP 3000 Essay 1 2010 Question 12

2010-10-14T14:42:33Z

Dustinmartin: /* Brief Concept */

COMP 3000 Essay 1 2010 Question 12

2010-10-14T14:41:10Z

Dustinmartin: /* Brief Concept */

COMP 3000 Essay 1 2010 Question 12

2010-10-13T17:01:41Z

Dustinmartin: /* Note to Group */

=Question=

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

=Note to Group=

If you have yet to participate in the discussion, please at least leave your name and notify us on what you plan on working on.

17:01, 13 October 2010 (UTC)17:01, 13 October 2010 (UTC)17:01, 13 October 2010 (UTC)17:01, 13 October 2010 (UTC)17:01, 13 October 2010 (UTC)17:01, 13 October 2010 (UTC)
Hi Team, sorry for being so late getting going on this project. Like I said in class on tuesday, my midterm today has been taking up most of my focus. I can cover ReiserFS and am definitely able to help with the introduction, conclusion, formatting, that kind of thing.

Also, in class there was mention of a meeting tomorrow. Is that right after class?

Thanks,
Dustin.
17:01, 13 October 2010 (UTC)17:01, 13 October 2010 (UTC)17:01, 13 October 2010 (UTC)17:01, 13 October 2010 (UTC)17:01, 13 October 2010 (UTC)17:01, 13 October 2010 (UTC)

=Answer=

==Newton OS==

In progress...

==BeOS==

Working on it...

==WinFS==

===History Pre-WinFS===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove NTFS file system and instead use a relational object-oriented file storage which was based on SQL server 8.0. It was supposed to be implemented in windows 2003 Server. But then, in 2000 Windows announced that Storage + was to be forgotten and Relational File System (RFS) was his successor. RFS was supposed to be included in SQL server 2000, but never made the cut. A factor that we can't forget, in 2000, Oracle announced a new file system which was a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.

We heard in 2002 about a new file system that would be present in Longhorn, renamed Vista. The system would be once again based on its predecessor, in this case RFS, but it would have as requirement to run on top of a NTFS file system. WinFS was included in a few public build for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable later, but was definitely cut for good in 2006 from Vista.

===Brief concept of WinFS ===

With today’s data, we are facing a crisis of finding what we want when we want at a reasonable speed on our own computer. The amount of file extension we can find is astonishing. We can’t name half of them nor know what they do. We store data in many different ways simple file with different extension, in different kinds of database which make finding, relating and acting quite difficult to achieve. [2]

Microsoft had an idea to solve this problem, by using a relational database as file System, where data would be simply data. To understand how WinFS file system works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in normal database, but there are multiple relations between tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to " Enable people to Find, Relate, and Act on their information." [3]
As already pointed out, data files are so broad nowadays, with that many file formats using complicated data storage methods our " current file system does not know how to collect and find information within these new types of data." But with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relate to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time find all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies and whatever else we would want find in the same request. We have to search for them one by one. If we treat data as data, once again, we can simple search in the tables of the database the pictures, received emails, videos, documents for the name Bob, and present them to the user. As we can see, data can be related to each other with key word, in our example it was the name Bob, but it can be anything.
What if data could do action that follow specific rules? This is exactly what WinFS wanted to implement. " WinFS Rules are a built-in component of the system that allows you to tell the system how to work with, sort, and deliver your data". Also, it could make use of other applications on the system. So if we received a picture from our friend Bob, we could automatically transfer it elsewhere.

The last point that WinFs was aiming for, was to run WinFS on top of NTFS. Basically, WinFS would scan all the data in NTFS file system, and put it into its database. Thus it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package but instead deliver it into the next MS SQL server, which was SQL server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release."[4] but instead, they will keep working on it and some "may be used by other Microsoft products going forward."[4] So we can see that it didn't become a file system, but some of the logistic will go into Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answer that " No. In fact, the Beta was coming together really well. "[4] He then replied that the technology used wasn't easy to build on, so they had to rewrite some parts, but that wouldn't have caused the end of it. Some others also speculate that no serious software used it, nor did it receive the attention needed from the developer to have a good start. In an interview with Channel 9, Quentin Clark said "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[5] This show that the team might had some management problem, which lead to the termination of the project.

==Palm OS==

===History===
Graffiti, a hand-recognition software which was believed to be doomed from the very beginning became Palm OS. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it and so there was no companies that wanted to make the hardware for it. The company Palm then decided to make their own hardware with Graffiti as the OS which is known as Palm OS today.

===Database===
Palm OS does not use a relational or XML database. It has two types, record and resource database.

====Record Database====
A record database is a collection of records (blocks of memory). Each record can only store up to 64KB of memory. The record database has a header which stores information onto the database this information, includes the creator ID, its name, version, etc. Also the database has information that is unique to the record, the location of the record, a unique id, and an attribute which contains delete, dirty, busy, and secret bit [1]. The database is stored in the storage heap (where permanent data resides).

====Resource Database====
Palm OS applications are stored in the resource database. One of the resources in this database contains code, another resource contains the application’s name, and another the application’s icon, and the rest contains the forms, alerts, menus, strings, and other elements of the application [1]. The resources come directly from the storage heap. The difference between the record database and resource database is how the information is stored and how it is named. Resource database no longer stores unique Ids and attributes instead it stores an ID number and a type (four-character constant). Resources are called by using the ID number and the type.

===Problems===
Record and resource database are stored in storage/database heap which is stored in the RAM (Random Access Memory). Database heap has a limit of 64KB of memory and a record had to be small enough to fit in there, this made memory hard to manage. A problem arose in versions before OS 3.0, because database heap was divided into smaller parts this allowed for free space in memory but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap.

Ex: A record is 50kb in size and there are 3 database heaps which can only hold 64KB each, a total of 192KB of memory. Let's say these three database heaps are half full therefore there is 96KB of free memory, but the record of size 50kb cannot be stored because not a single heap has enough memory to store the record.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

* I don't know how to properly show the references

[2]http://www.slideshare.net/Severus/winfs : Presentation made by John Keefauver,Manager .net Business Developement Ajilon Consulting

[3]http://msdn.microsoft.com/en-US/library/aa480687.aspx : Blog of Thomas Rizzo , Director in the Microsoft SQL Server group. 2004

[4]http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx Team blog of WinFS,Quentin Clark WinFS Product management team 2005

[5]http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now Interview with Quentin Clark WinFS Product management team 2005