Soma-notes - User contributions [en]

Talk:COMP 3000 Essay 2 2010 Question 10

2010-12-02T04:14:27Z

Tpham3: /* Notes to Group */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

=Notes to Group=

'''We might as well work directly on the main page'''

I think I've moved all the existing text to the main page, as things were being edited in 2 places. Hope that's okay. --[[User:Dagar|Dagar]] 23:01, 30 November 2010 (UTC)

that's fine! hey guys are we good with the information that we gathered? i think we should find external links on mClock if possible. --[[User:npatel1|npatel1]]

I can't any external links on mClock--[[User:tpham|tpham]]

hey, guys! that's what we I've gotten so far on the main page; please check/edit/reference accordingly to essay requirement! (how come two of our other group members did not contribute towards the essay?) --[[User:npatel1|npatel1]]

Hey, so I think tag assignment, tag adjustment, and request scheduler might be something important but i don't quite understand it if someone can take a look at it and see what they can come up with that would be helpful its on page 5 --[[User:tpham|tpham]]

=Group Members=
Please leave your name and email address if you are in the group

*[[User:Dagar|Daniel Agar]] - dagar@scs.carleton.ca
*[[User:xchen6|Xi Chen]] - xintai1985@gmail.com
*[[User:npatel1|Niravkumar Patel]] - npatel1@connect.carleton.ca
*[[User:tpham3|Tuan Pham]] - tpham3@connect.carleton.ca
*[[User:aellebla|Aaron Leblanc]] - aellebla@connect.carleton.ca
*[[User:naseido|Nisrin Abou-Seido]] - naseido@connect.carleton.ca

=Layout=
==Paper==

'''Authors''':

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

'''Link to the paper''':

[http://www.usenix.org/events/osdi10/tech/full_papers/Gulati.pdf mClock: Handling Throughput Variability for Hypervisor IO Scheduling]

==Background Concepts==

[old]

Hypervisors are responsible for multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upper bound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upper bound limits. This is interesting because virtualization has been very successful; people are comfortable with putting multiple VM on one HOST without worrying about the performance of each VM on another. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically. mClock is a better alternate because it supports all controls in a single algorithm, handles variable and unknown capacity, and fast to compute. This is interesting because there is a limit control on VM allocation, it does not weaken as each VM gets added on, and mClock reservations are met.

more about mclock here

mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines.

: Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper. Heres some notes i took from the paper. I did not want to put it on the main page because it is directly copied from the paper:

The intuitive idea behind the mClock algorithm is to logically interleave a constraint-based scheduler and a weight-based scheduler in a fine-grained manner. The constraint-based scheduler ensures that VMs receive at least their minimum reserved service and no more than the upper limit in a time interval, while the weight-based scheduler allocates the remaining throughput to achieve proportional sharing.

mClock uses two main ideas: multiple real-time clocks and dynamic clock selection.

Each VM IO request is assigned three tags, one for each clock: a reservation tag R, a limit tag L, and a proportional share tag P for weight-based allocation. Different clocks are used to keep track of each of the three controls, and tags based on one of the clocks are dynamically chosen to do the constraint-based or weight-based scheduling.

==Research problem==
: What is the research problem being addressed by the paper? How does this problem relate to past related work?

[old]

We use today, a very primitive kind of IO resource allocation in modern hypervisors. Currently an algorithm called PARDA (Proportional Allocation of Resources in Distributed storage Access) 1 is used to allocate IO resources to each VM running on a particular storage device. Unfortunately, the IO resource allocation algorithm of the hosts use a fair-scheduler called SFQ (Start-time Fair Queuing) 2. What this means is that PARDA allocates IO resources to VMs proportional to the number of IO shares on the host, but each host uses a fair scheduler which divides the IO shares amongst the VMs equally. This leads to the problem that whenever another VM is added or another background application is run on one of the VMs, all the other VMs suffer a huge performance lose. In the case of adding another VM, there is a 40% performance drop. This is completely unacceptable when applications have minimum performance requirements to run effectively. An application with minimum resource requirements can be running fine on any given VM, but as soon as the load on the shared storage device increases, the application would run poorly, or could potentially crash.

Related work
- http://www.fortuitous.com/docs/whitepapers/Linux2.6-Perf-Intro.pdf
-Linux schedulers (CFQ, SFQ)

==Contribution==
: What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)

[old]

This paper addresses the current limitations of IO resource allocation for hypervisors. The paper has proposed a new and more efficient algorithm to allocate IO resources. Older methods were limited solely by providing proportional shares. mClock incorporates proportional shares, as well as a minimum reservation of IO resources, and a maximum reservation.

Older methods of IO resource allocation had a terrible performance lose. Whenever the load on the shared storage device was increased, or when another VM was added, the performance of all hosts would drop considerably. Older methods provided unreliable IO management of hypervisors

mClock was able to present VMs with a guaranteed minimum reservation of IO resources. This means that application performance will never drop below a certain point. This provides much better application stability on each of the VMs, and better overall performance.

"dmClock (used for cluster-based storage systems) runs a modified version of mClock at each server. There is only one modification to the algorithm to account for the distributed model in the Tag-Assignment component." - from the paper

The mClock algorithm uses a tag-based scheduler with some modifications; like the tag-based schedulers all requests are assigned tags and scheduled in order of their tag values, the modifications includes the ability to use “multiple tags based on three controls and dynamically decide which tag to use for scheduling, while still synchronizing idle clients”

*'''Storage IO allocation is hard'''

*'''mClockcontributions'''
•Supports reservation, limit and shares in one place
•Handles variable IO performance seen by hypervisor
•Can be used for other resources such as CPU, memory & Network IO allocation as well

*'''Future work'''
•Better estimation of reservation capacity in terms of IOPS
•Add priority control along with RLS
•Mechanisms to set R, L,S and other controls to meet application-level goals

==Critique==
: What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

[old]

The article introduces the mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. This algorithm, mClock, is able to meet those controls in varying capacity. The good thing about this is that the algorithm proves efficient in clustered architectures. Moreover, it provides greater isolation between VMs.

In this paper there were many terms that were used but never explained, such as orders (used in the graphs), LUN, PARDA, etc. Also, I did not like the way the calculations were written in sentences, "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1". To me this was very messy and made me skip through the calculations part of the sentence.<math></math>

: Tuan,i think the term PARDA was explained in the article. It stands for Proportional Allocation of Resources in Distributed storage Access. It was basically a priority queue for the storage devices and VMs.--[[User:Aellebla|Aaron Leblanc]] 22:47, 30 November 2010 (UTC)

: I see, I'll have to read it over again

COMP 3000 Essay 2 2010 Question 10

2010-12-02T04:05:29Z

Tpham3: /* Contribution */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

==Paper==

[http://www.usenix.org/events/osdi10/tech/full_papers/Gulati.pdf mClock: Handling Throughput Variability for Hypervisor IO Scheduling]

'''Authors''':

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==

: [Added and made changes to Background Concepts! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: VM usage is increasing significantly on a daily basis – whether by large gaming firms or university students. SFQ(D) performed fairly for low-intensity workloads. However, as the workload with VMs multiplied, the constant need for faster performance and efficiency rose. Hypervisors required a resource-allocation algorithm in order to meet the need for high performance VMs running concurrently; mClock was the answer Gulati, Varman, and Merchant proposed, to aid hypervisors.

: Hypervisors are responsible for multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upper bound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upper bound limits. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically.

: mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Evidently, it is the better alternate to SQF because it supports all controls in a single algorithm, handles variable and unknown capacity, and is fast to compute. The algorithm does not weaken the performance level as each VM gets added on, and mClock reservations are met. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines.

==Research problem==
: [Added and made changes to Research Problem! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: Today, we use a very primitive kind of IO resource allocation in modern hypervisors. Currently an algorithm called PARDA (Proportional Allocation of Resources in Distributed storage Access) [[#Foot1|1]] is being used to allocate IO resources to each VM running on a particular storage device. Unfortunately, the IO resource allocation algorithm of the hosts use a fair-scheduler called SFQ (Start-time Fair Queuing) [[#Foot2|2]]. What this means is that PARDA allocates IO resources to VMs proportional to the number of IO shares on the host, but each host uses a fair scheduler which divides the IO shares amongst the VMs equally. This leads to the main problem; whenever another VM is added or another background application is run on one of the VMs, all other VMs suffer a huge performance loss; a 40% performance drop. This is completely unacceptable when applications have minimum performance requirements to run effectively. An application with minimum resource requirements can be running fine on any given VM, however as soon as the stress load on the shared storage device increases, the application might fail to run smoothly, or worse, crash. To resolve this issue of resource allocation and performance, mClock is introduced and tested against SFQ.

==Contribution==

: [Added and made changes to Contribution! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: This paper addresses the current limitations of IO resource allocation for hypervisors. It has proposed a new and more efficient algorithm to allocate IO resources. Older methods were limited solely by providing proportional shares, such as SFQ. mClock incorporates proportional shares, as well as a minimum reservation of IO resources, and a maximum reservation.

: Older methods of IO resource allocation had a terrible performance loss. Whenever the load on the shared storage device was increased, or when another VM was added, the performance of all hosts would drop significantly. Also, these older methods provided unreliable IO management of hypervisors. Conversely, mClock was able to present VMs with a guaranteed minimum reservation of IO resources. This means that application performance will never drop below a certain point. This provides much better application stability on each of the VMs, and better overall performance and efficiency level, compared to older methods; for instance, SFQ.

: The mClock algorithm uses a tag-based scheduler with some modifications; like the tag-based schedulers all IO requests are assigned tags and scheduled in order of their tag values, the modifications includes the ability to use “multiple tags based on three controls and dynamically decide which tag to use for scheduling, while still synchronizing idle clients”. [[#Foot2|2]]

: mClock also uses both constraint-based and weight-based schedulers. Constraint-based scheduler makes sure that “VMs receive their minimum reserved service and no more than their upper limit in a time interval. Weight-based scheduler allocates the remaining throughput to achieve proportional sharing”. [[#Foot2|2]]

: Another contribution was the introduction of Distributed mClock or dmClock which basically runs an altered version of mClock at each server. dmClock is mainly used for cluster-based storage system which are rising as centralized disk arrays, and better than the alternates in terms of cost. The reservation in this modified algorithm gives higher preference to non-idle VMs to attain high performance. dmClock proved to be effective with a simple, modified mClock algorithm which does not require complex synchronizations between servers.

==Critique==

: [Added and made changes to Critique! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: The article introduces the mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. This algorithm, mClock, is able to meet those controls in varying capacity. The good thing about this is that the algorithm proves to be efficient in clustered architectures due to better resource allocation while providing greater isolation between VMs. mClock allows users to be comfortable when working with multiple VMs on one HOST without the constant worry of performance levels, with each VM add-on.

: The paper proposes a better, and effective alternative to SFQ and other older methods; the mClock algorithm which efficiently handles multiple VMs in a throughput environment (LUN, PARDA).
: One aspect of the writing style , "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1" [[#Foot2|2]]. The style of displaying these calculations depicts a messy, unorganized styled.<math></math>

==References==

: 1 A. Gulati, I. Ahmad, and C. Waldspurger. PARDA: Proportional
Allocation of Resources in Distributed Storage
Access. In (FAST ’09) Proceedings of the Seventh Usenix
Conference on File and Storage Technologies, Feb 2009.

: 2 W. Jin, J. S. Chase, and J. Kaur. Interposed proportional
sharing for a storage service utility. In ACM SIGMET-
RICS, 2004. [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.72.7012&rep=rep1&type=pdf Interposed proportional sharing for a storage service utility]

COMP 3000 Essay 2 2010 Question 10

2010-12-02T04:05:08Z

Tpham3: /* Contribution */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

==Paper==

[http://www.usenix.org/events/osdi10/tech/full_papers/Gulati.pdf mClock: Handling Throughput Variability for Hypervisor IO Scheduling]

'''Authors''':

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==

: [Added and made changes to Background Concepts! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: VM usage is increasing significantly on a daily basis – whether by large gaming firms or university students. SFQ(D) performed fairly for low-intensity workloads. However, as the workload with VMs multiplied, the constant need for faster performance and efficiency rose. Hypervisors required a resource-allocation algorithm in order to meet the need for high performance VMs running concurrently; mClock was the answer Gulati, Varman, and Merchant proposed, to aid hypervisors.

: Hypervisors are responsible for multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upper bound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upper bound limits. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically.

: mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Evidently, it is the better alternate to SQF because it supports all controls in a single algorithm, handles variable and unknown capacity, and is fast to compute. The algorithm does not weaken the performance level as each VM gets added on, and mClock reservations are met. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines.

==Research problem==
: [Added and made changes to Research Problem! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: Today, we use a very primitive kind of IO resource allocation in modern hypervisors. Currently an algorithm called PARDA (Proportional Allocation of Resources in Distributed storage Access) [[#Foot1|1]] is being used to allocate IO resources to each VM running on a particular storage device. Unfortunately, the IO resource allocation algorithm of the hosts use a fair-scheduler called SFQ (Start-time Fair Queuing) [[#Foot2|2]]. What this means is that PARDA allocates IO resources to VMs proportional to the number of IO shares on the host, but each host uses a fair scheduler which divides the IO shares amongst the VMs equally. This leads to the main problem; whenever another VM is added or another background application is run on one of the VMs, all other VMs suffer a huge performance loss; a 40% performance drop. This is completely unacceptable when applications have minimum performance requirements to run effectively. An application with minimum resource requirements can be running fine on any given VM, however as soon as the stress load on the shared storage device increases, the application might fail to run smoothly, or worse, crash. To resolve this issue of resource allocation and performance, mClock is introduced and tested against SFQ.

==Contribution==

: [Added and made changes to Contribution! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: This paper addresses the current limitations of IO resource allocation for hypervisors. It has proposed a new and more efficient algorithm to allocate IO resources. Older methods were limited solely by providing proportional shares, such as SFQ. mClock incorporates proportional shares, as well as a minimum reservation of IO resources, and a maximum reservation.

: Older methods of IO resource allocation had a terrible performance loss. Whenever the load on the shared storage device was increased, or when another VM was added, the performance of all hosts would drop significantly. Also, these older methods provided unreliable IO management of hypervisors. Conversely, mClock was able to present VMs with a guaranteed minimum reservation of IO resources. This means that application performance will never drop below a certain point. This provides much better application stability on each of the VMs, and better overall performance and efficiency level, compared to older methods; for instance, SFQ.

: The mClock algorithm uses a tag-based scheduler with some modifications; like the tag-based schedulers all IO requests are assigned tags and scheduled in order of their tag values, the modifications includes the ability to use “multiple tags based on three controls and dynamically decide which tag to use for scheduling, while still synchronizing idle clients”. [[#Foot2|2]]

: mClock also uses both constraint-based and weight-based schedulers. Constraint-based scheduler makes sure that “VMs receive their minimum reserved service and no more than their upper limit in a time interval. Weight-based scheduler allocates the remaining throughput to achieve proportional sharing”. [[#Foot2|2]]</sup

: Another contribution was the introduction of Distributed mClock or dmClock which basically runs an altered version of mClock at each server. dmClock is mainly used for cluster-based storage system which are rising as centralized disk arrays, and better than the alternates in terms of cost. The reservation in this modified algorithm gives higher preference to non-idle VMs to attain high performance. dmClock proved to be effective with a simple, modified mClock algorithm which does not require complex synchronizations between servers.

==Critique==

: [Added and made changes to Critique! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: The article introduces the mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. This algorithm, mClock, is able to meet those controls in varying capacity. The good thing about this is that the algorithm proves to be efficient in clustered architectures due to better resource allocation while providing greater isolation between VMs. mClock allows users to be comfortable when working with multiple VMs on one HOST without the constant worry of performance levels, with each VM add-on.

: The paper proposes a better, and effective alternative to SFQ and other older methods; the mClock algorithm which efficiently handles multiple VMs in a throughput environment (LUN, PARDA).
: One aspect of the writing style , "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1" [[#Foot2|2]]. The style of displaying these calculations depicts a messy, unorganized styled.<math></math>

==References==

: 1 A. Gulati, I. Ahmad, and C. Waldspurger. PARDA: Proportional
Allocation of Resources in Distributed Storage
Access. In (FAST ’09) Proceedings of the Seventh Usenix
Conference on File and Storage Technologies, Feb 2009.

: 2 W. Jin, J. S. Chase, and J. Kaur. Interposed proportional
sharing for a storage service utility. In ACM SIGMET-
RICS, 2004. [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.72.7012&rep=rep1&type=pdf Interposed proportional sharing for a storage service utility]

COMP 3000 Essay 2 2010 Question 10

2010-12-02T04:04:50Z

Tpham3: /* Contribution */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

==Paper==

[http://www.usenix.org/events/osdi10/tech/full_papers/Gulati.pdf mClock: Handling Throughput Variability for Hypervisor IO Scheduling]

'''Authors''':

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==

: [Added and made changes to Background Concepts! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: VM usage is increasing significantly on a daily basis – whether by large gaming firms or university students. SFQ(D) performed fairly for low-intensity workloads. However, as the workload with VMs multiplied, the constant need for faster performance and efficiency rose. Hypervisors required a resource-allocation algorithm in order to meet the need for high performance VMs running concurrently; mClock was the answer Gulati, Varman, and Merchant proposed, to aid hypervisors.

: Hypervisors are responsible for multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upper bound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upper bound limits. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically.

: mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Evidently, it is the better alternate to SQF because it supports all controls in a single algorithm, handles variable and unknown capacity, and is fast to compute. The algorithm does not weaken the performance level as each VM gets added on, and mClock reservations are met. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines.

==Research problem==
: [Added and made changes to Research Problem! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: Today, we use a very primitive kind of IO resource allocation in modern hypervisors. Currently an algorithm called PARDA (Proportional Allocation of Resources in Distributed storage Access) [[#Foot1|1]] is being used to allocate IO resources to each VM running on a particular storage device. Unfortunately, the IO resource allocation algorithm of the hosts use a fair-scheduler called SFQ (Start-time Fair Queuing) [[#Foot2|2]]. What this means is that PARDA allocates IO resources to VMs proportional to the number of IO shares on the host, but each host uses a fair scheduler which divides the IO shares amongst the VMs equally. This leads to the main problem; whenever another VM is added or another background application is run on one of the VMs, all other VMs suffer a huge performance loss; a 40% performance drop. This is completely unacceptable when applications have minimum performance requirements to run effectively. An application with minimum resource requirements can be running fine on any given VM, however as soon as the stress load on the shared storage device increases, the application might fail to run smoothly, or worse, crash. To resolve this issue of resource allocation and performance, mClock is introduced and tested against SFQ.

==Contribution==

: [Added and made changes to Contribution! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: This paper addresses the current limitations of IO resource allocation for hypervisors. It has proposed a new and more efficient algorithm to allocate IO resources. Older methods were limited solely by providing proportional shares, such as SFQ. mClock incorporates proportional shares, as well as a minimum reservation of IO resources, and a maximum reservation.

: Older methods of IO resource allocation had a terrible performance loss. Whenever the load on the shared storage device was increased, or when another VM was added, the performance of all hosts would drop significantly. Also, these older methods provided unreliable IO management of hypervisors. Conversely, mClock was able to present VMs with a guaranteed minimum reservation of IO resources. This means that application performance will never drop below a certain point. This provides much better application stability on each of the VMs, and better overall performance and efficiency level, compared to older methods; for instance, SFQ.

: The mClock algorithm uses a tag-based scheduler with some modifications; like the tag-based schedulers all IO requests are assigned tags and scheduled in order of their tag values, the modifications includes the ability to use “multiple tags based on three controls and dynamically decide which tag to use for scheduling, while still synchronizing idle clients”. [[#Foot2|2]]</sup

: mClock also uses both constraint-based and weight-based schedulers. Constraint-based scheduler makes sure that “VMs receive their minimum reserved service and no more than their upper limit in a time interval. Weight-based scheduler allocates the remaining throughput to achieve proportional sharing”. [[#Foot2|2]]</sup

: Another contribution was the introduction of Distributed mClock or dmClock which basically runs an altered version of mClock at each server. dmClock is mainly used for cluster-based storage system which are rising as centralized disk arrays, and better than the alternates in terms of cost. The reservation in this modified algorithm gives higher preference to non-idle VMs to attain high performance. dmClock proved to be effective with a simple, modified mClock algorithm which does not require complex synchronizations between servers.

==Critique==

: [Added and made changes to Critique! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: The article introduces the mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. This algorithm, mClock, is able to meet those controls in varying capacity. The good thing about this is that the algorithm proves to be efficient in clustered architectures due to better resource allocation while providing greater isolation between VMs. mClock allows users to be comfortable when working with multiple VMs on one HOST without the constant worry of performance levels, with each VM add-on.

: The paper proposes a better, and effective alternative to SFQ and other older methods; the mClock algorithm which efficiently handles multiple VMs in a throughput environment (LUN, PARDA).
: One aspect of the writing style , "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1" [[#Foot2|2]]. The style of displaying these calculations depicts a messy, unorganized styled.<math></math>

==References==

: 1 A. Gulati, I. Ahmad, and C. Waldspurger. PARDA: Proportional
Allocation of Resources in Distributed Storage
Access. In (FAST ’09) Proceedings of the Seventh Usenix
Conference on File and Storage Technologies, Feb 2009.

: 2 W. Jin, J. S. Chase, and J. Kaur. Interposed proportional
sharing for a storage service utility. In ACM SIGMET-
RICS, 2004. [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.72.7012&rep=rep1&type=pdf Interposed proportional sharing for a storage service utility]

COMP 3000 Essay 2 2010 Question 10

2010-12-02T03:42:02Z

Tpham3: /* Contribution */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

==Paper==

[http://www.usenix.org/events/osdi10/tech/full_papers/Gulati.pdf mClock: Handling Throughput Variability for Hypervisor IO Scheduling]

'''Authors''':

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==

: [Added and made changed to Background Concepts! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: VM usage is increasing significantly on a daily basis – whether by large gaming firms or university students. SFQ(D) performed fairly for low-intensity workloads. However, as the workload with VMs multiplied, the constant need for faster performance and efficiency rose. Hypervisors required a resource-allocation algorithm in order to meet the need for high performance VMs running concurrently; mClock was the answer Gulati, Varman, and Merchant proposed, to aid hypervisors.

: Hypervisors are responsible for multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upper bound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upper bound limits. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically.

: mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Evidently, it is the better alternate to SQF because it supports all controls in a single algorithm, handles variable and unknown capacity, and is fast to compute. The algorithm does not weaken the performance level as each VM gets added on, and mClock reservations are met. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines.

==Research problem==
: [Added and made changed to Research Problem! please edit if you guys find mistake(s)]--[[User:npatel1|npatel1]]

: Today, we use a very primitive kind of IO resource allocation in modern hypervisors. Currently an algorithm called PARDA (Proportional Allocation of Resources in Distributed storage Access) [[#Foot1|1]] is being used to allocate IO resources to each VM running on a particular storage device. Unfortunately, the IO resource allocation algorithm of the hosts use a fair-scheduler called SFQ (Start-time Fair Queuing) [[#Foot2|2]]. What this means is that PARDA allocates IO resources to VMs proportional to the number of IO shares on the host, but each host uses a fair scheduler which divides the IO shares amongst the VMs equally. This leads to the main problem; whenever another VM is added or another background application is run on one of the VMs, all other VMs suffer a huge performance loss; a 40% performance drop. This is completely unacceptable when applications have minimum performance requirements to run effectively. An application with minimum resource requirements can be running fine on any given VM, however as soon as the stress load on the shared storage device increases, the application might fail to run smoothly, or worse, crash. To resolve this issue of resource allocation and performance, mClock is introduced and tested against SFQ.

==Contribution==
This paper addresses the current limitations of IO resource allocation for hypervisors. The paper has proposed a new and more efficient algorithm to allocate IO resources. Older methods were limited solely by providing proportional shares. mClock incorporates proportional shares, as well as a minimum reservation of IO resources, and a maximum reservation.

Older methods of IO resource allocation had a terrible performance lose. Whenever the load on the shared storage device was increased, or when another VM was added, the performance of all hosts would drop considerably. Older methods provided unreliable IO management of hypervisors

mClock was able to present VMs with a guaranteed minimum reservation of IO resources. This means that application performance will never drop below a certain point. This provides much better application stability on each of the VMs, and better overall performance.

"dmClock (used for cluster-based storage systems) runs a modified version of mClock at each server. There is only one modification to the algorithm to account for the distributed model in the Tag-Assignment component." - from the paper

The mClock algorithm uses a tag-based scheduler with some modifications; like the tag-based schedulers all requests are assigned tags and scheduled in order of their tag values, the modifications includes the ability to use “multiple tags based on three controls and dynamically decide which tag to use for scheduling, while still synchronizing idle clients”

==Critique==

: [Added and made changed to Critique! please edit if you guys find mistake(s)]

: The article introduces the mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. This algorithm, mClock, is able to meet those controls in varying capacity. The good thing about this is that the algorithm proves to be efficient in clustered architectures due to better resource allocation while providing greater isolation between VMs. mClock allows users to be comfortable when working with multiple VMs on one HOST without the constant worry of performance levels, with each VM add-on.

: The paper proposes a better, and effective alternative to SFQ and other older methods; the mClock algorithm which efficiently handles multiple VMs in a throughput environment (LUN, PARDA).
: One aspect of the writing style , "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1". The style of displaying these calculations depicts a messy, unorganized styled.<math></math>

==References==

1 A. Gulati, I. Ahmad, and C. Waldspurger. PARDA: Proportional
Allocation of Resources in Distributed Storage
Access. In (FAST ’09) Proceedings of the Seventh Usenix
Conference on File and Storage Technologies, Feb 2009.

2 W. Jin, J. S. Chase, and J. Kaur. Interposed proportional
sharing for a storage service utility. In ACM SIGMET-
RICS, 2004. [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.72.7012&rep=rep1&type=pdf Interposed proportional sharing for a storage service utility]

COMP 3000 Essay 2 2010 Question 10

2010-12-02T02:40:45Z

Tpham3: /* Research problem */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

==Paper==

[http://www.usenix.org/events/osdi10/tech/full_papers/Gulati.pdf mClock: Handling Throughput Variability for Hypervisor IO Scheduling]

'''Authors''':

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==

Hypervisors are responsible for multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are: reservation, where the minimum bounds are set, the limit where the maximum upper bound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upper bound limits. This is interesting because virtualization has been very successful; people are comfortable with putting multiple VM on one HOST without worrying about the performance of each VM on another. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically. mClock is a better alternate because it supports all controls in a single algorithm, handles variable and unknown capacity, and fast to compute. This is interesting because there is a limit control on VM allocation, it does not weaken as each VM gets added on, and mClock reservations are met.

: more about mclock here

mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines.

==Research problem==
Today, we use a very primitive kind of IO resource allocation in modern hypervisors. Currently an algorithm called PARDA (Proportional Allocation of Resources in Distributed storage Access) [[#Foot1|1]] is used to allocate IO resources to each VM running on a particular storage device. Unfortunately, the IO resource allocation algorithm of the hosts use a fair-scheduler called SFQ (Start-time Fair Queuing) [[#Foot2|2]]. What this means is that PARDA allocates IO resources to VMs proportional to the number of IO shares on the host, but each host uses a fair scheduler which divides the IO shares amongst the VMs equally. This leads to the problem that whenever another VM is added or another background application is run on one of the VMs, all the other VMs suffer a huge performance lose. In the case of adding another VM, there is a 40% performance drop. This is completely unacceptable when applications have minimum performance requirements to run effectively. An application with minimum resource requirements can be running fine on any given VM, but as soon as the load on the shared storage device increases, the application would run poorly, or could potentially crash.

==Contribution==
This paper addresses the current limitations of IO resource allocation for hypervisors. The paper has proposed a new and more efficient algorithm to allocate IO resources. Older methods were limited solely by providing proportional shares. mClock incorporates proportional shares, as well as a minimum reservation of IO resources, and a maximum reservation.

Older methods of IO resource allocation had a terrible performance lose. Whenever the load on the shared storage device was increased, or when another VM was added, the performance of all hosts would drop considerably. Older methods provided unreliable IO management of hypervisors

mClock was able to present VMs with a guaranteed minimum reservation of IO resources. This means that application performance will never drop below a certain point. This provides much better application stability on each of the VMs, and better overall performance.

"dmClock (used for cluster-based storage systems) runs a modified version of mClock at each server. There is only one modification to the algorithm to account for the distributed model in the Tag-Assignment component." - from the paper

==Critique==
The article introduces the mClock algorithm which handles multiple VMs in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. This algorithm, mClock, is able to meet those controls in varying capacity.

The good thing about this is that the algorithm proves efficient in clustered architectures. Moreover, it provides greater isolation between VMs.

In this paper there were many terms that were used but never explained, such as orders (used in the graphs), LUN, PARDA, etc.
One aspect of the writing style , "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1". To me this was very messy and made me skip through the calculations part of the sentence.<math></math>

==References==

1 A. Gulati, I. Ahmad, and C. Waldspurger. PARDA: Proportional
Allocation of Resources in Distributed Storage
Access. In (FAST ’09) Proceedings of the Seventh Usenix
Conference on File and Storage Technologies, Feb 2009.

2 W. Jin, J. S. Chase, and J. Kaur. Interposed proportional
sharing for a storage service utility. In ACM SIGMET-
RICS, 2004. [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.72.7012&rep=rep1&type=pdf Interposed proportional sharing for a storage service utility]

Talk:COMP 3000 Essay 2 2010 Question 10

2010-12-01T18:49:45Z

Tpham3: /* Background Concepts */

Talk:COMP 3000 Essay 2 2010 Question 10

2010-12-01T18:20:52Z

Tpham3: /* Research problem */

Talk:COMP 3000 Essay 2 2010 Question 10

2010-12-01T18:20:28Z

Tpham3: /* Research problem */

Talk:COMP 3000 Essay 2 2010 Question 10

2010-12-01T18:10:12Z

Tpham3: /* Critique */

Talk:COMP 3000 Essay 2 2010 Question 10

2010-12-01T18:09:03Z

Tpham3: /* Notes to Group */

Talk:COMP 3000 Essay 2 2010 Question 10

2010-11-27T03:10:46Z

Tpham3: /* Contribution */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

=Notes to Group=

=Group Members=
Please leave your name and email address if you are in the group

*[[User:Dagar|Daniel Agar]] - dagar@scs.carleton.ca
*[[User:xchen6|Xi Chen]] - xintai1985@gmail.com
*[[User:npatel1|Niravkumar Patel]] - npatel1@connect.carleton.ca
*[[User:tpham3|Tuan Pham]] - tpham3@connect.carleton.ca
*[[User:aellebla|Aaron Leblanc]] - aellebla@connect.carleton.ca
*[[User:naseido|Nisrin Abou-Seido]] - naseido@connect.carleton.ca

=Layout=
==Paper==
: the paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==
: Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

: Hypervisors are responsible were multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upperbound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upperbound limits. This is interesting because virtualization has been very successful; people are comfortable with putting multiple VM on one HOST without worrying about the performance of each VM on another. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically. mClock is a better alternate because it supports all controls in a single algorithm, handles variable and unknown capacity, and fast to compute. This is interesting because there is a limit control on VM allocation, it does not weaken as each VM gets added on, and mClock reservations are met. [[User:npatel1|Niravkumar Patel]]

: Here abit more on mClock!

: mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines. [[User:npatel1|Niravkumar Patel]]

==Research problem==
: What is the research problem being addressed by the paper? How does this problem relate to past related work?
==Contribution==
: What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)

: - "dmClock (used for cluster-based storage systems) runs a modified version of mClock at each server. There is only one modification to the algorithm to account for the distributed model in the Tag-Assignment component." - from the paper [[User:tpham3|Tuan Pham]]

==Critique==
: What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

: The article introduces a mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. mClock is able to meet those controls in varying capacity. the good thing about this is that the algorithms proves efficient in clustered architectures. Moreover, mClock provides greater isolation between VMs. [[User:npatel1|Niravkumar Patel]]

: In this paper there were many terms that were used but never explained, such as orders (used in the graphs), LUN, PARDA, etc. Also i did not like the way the calcualtions were written in sentences, "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1". To me this was very messy and made me skip through the calculations part of the sentence. [[User:tpham3|Tuan Pham]]

==References==
: You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

Talk:COMP 3000 Essay 2 2010 Question 10

2010-11-27T03:10:22Z

Tpham3: /* Contribution */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

=Notes to Group=

=Group Members=
Please leave your name and email address if you are in the group

*[[User:Dagar|Daniel Agar]] - dagar@scs.carleton.ca
*[[User:xchen6|Xi Chen]] - xintai1985@gmail.com
*[[User:npatel1|Niravkumar Patel]] - npatel1@connect.carleton.ca
*[[User:tpham3|Tuan Pham]] - tpham3@connect.carleton.ca
*[[User:aellebla|Aaron Leblanc]] - aellebla@connect.carleton.ca
*[[User:naseido|Nisrin Abou-Seido]] - naseido@connect.carleton.ca

=Layout=
==Paper==
: the paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==
: Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

: Hypervisors are responsible were multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upperbound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upperbound limits. This is interesting because virtualization has been very successful; people are comfortable with putting multiple VM on one HOST without worrying about the performance of each VM on another. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically. mClock is a better alternate because it supports all controls in a single algorithm, handles variable and unknown capacity, and fast to compute. This is interesting because there is a limit control on VM allocation, it does not weaken as each VM gets added on, and mClock reservations are met. [[User:npatel1|Niravkumar Patel]]

: Here abit more on mClock!

: mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines. [[User:npatel1|Niravkumar Patel]]

==Research problem==
: What is the research problem being addressed by the paper? How does this problem relate to past related work?
==Contribution==
: What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)

: - "dmClock (used for cluster-based storage systems) runs a modified version of mClock at each server. There is only one modification to the algorithm to account for the distributed model in the Tag-Assignment component." - from the paper

==Critique==
: What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

: The article introduces a mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. mClock is able to meet those controls in varying capacity. the good thing about this is that the algorithms proves efficient in clustered architectures. Moreover, mClock provides greater isolation between VMs. [[User:npatel1|Niravkumar Patel]]

: In this paper there were many terms that were used but never explained, such as orders (used in the graphs), LUN, PARDA, etc. Also i did not like the way the calcualtions were written in sentences, "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1". To me this was very messy and made me skip through the calculations part of the sentence. [[User:tpham3|Tuan Pham]]

==References==
: You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

Talk:COMP 3000 Essay 2 2010 Question 10

2010-11-27T03:08:46Z

Tpham3: /* Contribution */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

=Notes to Group=

=Group Members=
Please leave your name and email address if you are in the group

*[[User:Dagar|Daniel Agar]] - dagar@scs.carleton.ca
*[[User:xchen6|Xi Chen]] - xintai1985@gmail.com
*[[User:npatel1|Niravkumar Patel]] - npatel1@connect.carleton.ca
*[[User:tpham3|Tuan Pham]] - tpham3@connect.carleton.ca
*[[User:aellebla|Aaron Leblanc]] - aellebla@connect.carleton.ca
*[[User:naseido|Nisrin Abou-Seido]] - naseido@connect.carleton.ca

=Layout=
==Paper==
: the paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==
: Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

: Hypervisors are responsible were multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upperbound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upperbound limits. This is interesting because virtualization has been very successful; people are comfortable with putting multiple VM on one HOST without worrying about the performance of each VM on another. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically. mClock is a better alternate because it supports all controls in a single algorithm, handles variable and unknown capacity, and fast to compute. This is interesting because there is a limit control on VM allocation, it does not weaken as each VM gets added on, and mClock reservations are met. [[User:npatel1|Niravkumar Patel]]

: Here abit more on mClock!

: mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines. [[User:npatel1|Niravkumar Patel]]

==Research problem==
: What is the research problem being addressed by the paper? How does this problem relate to past related work?
==Contribution==
: What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)

: - "dmClock runs a modified version of mClock at each server. There is only one modification to the algorithm to account for the distributed model in the Tag-Assignment component." - from the paper

==Critique==
: What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

: The article introduces a mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. mClock is able to meet those controls in varying capacity. the good thing about this is that the algorithms proves efficient in clustered architectures. Moreover, mClock provides greater isolation between VMs. [[User:npatel1|Niravkumar Patel]]

: In this paper there were many terms that were used but never explained, such as orders (used in the graphs), LUN, PARDA, etc. Also i did not like the way the calcualtions were written in sentences, "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1". To me this was very messy and made me skip through the calculations part of the sentence. [[User:tpham3|Tuan Pham]]

==References==
: You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

Talk:COMP 3000 Essay 2 2010 Question 10

2010-11-27T03:08:12Z

Tpham3: /* Contribution */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

=Notes to Group=

=Group Members=
Please leave your name and email address if you are in the group

*[[User:Dagar|Daniel Agar]] - dagar@scs.carleton.ca
*[[User:xchen6|Xi Chen]] - xintai1985@gmail.com
*[[User:npatel1|Niravkumar Patel]] - npatel1@connect.carleton.ca
*[[User:tpham3|Tuan Pham]] - tpham3@connect.carleton.ca
*[[User:aellebla|Aaron Leblanc]] - aellebla@connect.carleton.ca
*[[User:naseido|Nisrin Abou-Seido]] - naseido@connect.carleton.ca

=Layout=
==Paper==
: the paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==
: Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

: Hypervisors are responsible were multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upperbound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upperbound limits. This is interesting because virtualization has been very successful; people are comfortable with putting multiple VM on one HOST without worrying about the performance of each VM on another. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically. mClock is a better alternate because it supports all controls in a single algorithm, handles variable and unknown capacity, and fast to compute. This is interesting because there is a limit control on VM allocation, it does not weaken as each VM gets added on, and mClock reservations are met. [[User:npatel1|Niravkumar Patel]]

: Here abit more on mClock!

: mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines. [[User:npatel1|Niravkumar Patel]]

==Research problem==
: What is the research problem being addressed by the paper? How does this problem relate to past related work?
==Contribution==
: What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)

- dmClock runs a modified version of mClock at each server. There is only one modification to the algorithm to account for the distributed model in the Tag-Assignment component.

==Critique==
: What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

: The article introduces a mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. mClock is able to meet those controls in varying capacity. the good thing about this is that the algorithms proves efficient in clustered architectures. Moreover, mClock provides greater isolation between VMs. [[User:npatel1|Niravkumar Patel]]

: In this paper there were many terms that were used but never explained, such as orders (used in the graphs), LUN, PARDA, etc. Also i did not like the way the calcualtions were written in sentences, "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1". To me this was very messy and made me skip through the calculations part of the sentence. [[User:tpham3|Tuan Pham]]

==References==
: You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

Talk:COMP 3000 Essay 2 2010 Question 10

2010-11-27T03:06:16Z

Tpham3: /* Critique */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

=Notes to Group=

=Group Members=
Please leave your name and email address if you are in the group

*[[User:Dagar|Daniel Agar]] - dagar@scs.carleton.ca
*[[User:xchen6|Xi Chen]] - xintai1985@gmail.com
*[[User:npatel1|Niravkumar Patel]] - npatel1@connect.carleton.ca
*[[User:tpham3|Tuan Pham]] - tpham3@connect.carleton.ca
*[[User:aellebla|Aaron Leblanc]] - aellebla@connect.carleton.ca
*[[User:naseido|Nisrin Abou-Seido]] - naseido@connect.carleton.ca

=Layout=
==Paper==
: the paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==
: Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

: Hypervisors are responsible were multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upperbound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upperbound limits. This is interesting because virtualization has been very successful; people are comfortable with putting multiple VM on one HOST without worrying about the performance of each VM on another. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically. mClock is a better alternate because it supports all controls in a single algorithm, handles variable and unknown capacity, and fast to compute. This is interesting because there is a limit control on VM allocation, it does not weaken as each VM gets added on, and mClock reservations are met. [[User:npatel1|Niravkumar Patel]]

: Here abit more on mClock!

: mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines. [[User:npatel1|Niravkumar Patel]]

==Research problem==
: What is the research problem being addressed by the paper? How does this problem relate to past related work?
==Contribution==
: What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)
==Critique==
: What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

: The article introduces a mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. mClock is able to meet those controls in varying capacity. the good thing about this is that the algorithms proves efficient in clustered architectures. Moreover, mClock provides greater isolation between VMs. [[User:npatel1|Niravkumar Patel]]

: In this paper there were many terms that were used but never explained, such as orders (used in the graphs), LUN, PARDA, etc. Also i did not like the way the calcualtions were written in sentences, "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1". To me this was very messy and made me skip through the calculations part of the sentence. [[User:tpham3|Tuan Pham]]

==References==
: You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

Talk:COMP 3000 Essay 2 2010 Question 10

2010-11-27T03:05:32Z

Tpham3: /* Critique */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

=Notes to Group=

=Group Members=
Please leave your name and email address if you are in the group

*[[User:Dagar|Daniel Agar]] - dagar@scs.carleton.ca
*[[User:xchen6|Xi Chen]] - xintai1985@gmail.com
*[[User:npatel1|Niravkumar Patel]] - npatel1@connect.carleton.ca
*[[User:tpham3|Tuan Pham]] - tpham3@connect.carleton.ca
*[[User:aellebla|Aaron Leblanc]] - aellebla@connect.carleton.ca
*[[User:naseido|Nisrin Abou-Seido]] - naseido@connect.carleton.ca

=Layout=
==Paper==
: the paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==
: Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

: Hypervisors are responsible were multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upperbound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upperbound limits. This is interesting because virtualization has been very successful; people are comfortable with putting multiple VM on one HOST without worrying about the performance of each VM on another. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically. mClock is a better alternate because it supports all controls in a single algorithm, handles variable and unknown capacity, and fast to compute. This is interesting because there is a limit control on VM allocation, it does not weaken as each VM gets added on, and mClock reservations are met. [[User:npatel1|Niravkumar Patel]]

: Here abit more on mClock!

: mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines. [[User:npatel1|Niravkumar Patel]]

==Research problem==
: What is the research problem being addressed by the paper? How does this problem relate to past related work?
==Contribution==
: What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)
==Critique==
: What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

: The article introduces a mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. mClock is able to meet those controls in varying capacity. the good thing about this is that the algorithms proves efficient in clustered architectures. Moreover, mClock provides greater isolation between VMs. [[User:npatel1|Niravkumar Patel]]

: In this paper there were many terms that were used but never explained, such as orders (used in the graphs), LUN, PARDA, etc. Also i did not like the way the calcualtions were written in sentences, "For a small reference IO size of 8KB and using typical values for mechanical delay Tm = 5ms and peak transfer rate, Bpeak = 60 MB/s, the numerator = Lat1*(1 + 8/300) ≈ Lat1". To me this was very messy and made me skip through the calculations part of the sentence.

==References==
: You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

Talk:COMP 3000 Essay 2 2010 Question 10

2010-11-25T16:25:40Z

Tpham3: /* Group Members */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

=Notes to Group=

=Group Members=
Please leave your name and email address if you are in the group

*[[User:Dagar|Daniel Agar]] - dagar@scs.carleton.ca
*[[User:xchen6|Xi Chen]] - xintai1985@gmail.com
*[[User:npatel1|Niravkumar Patel]] - npatel1@connect.carleton.ca
*[[User:tpham3|Tuan Pham]] - tpham3@connect.carleton.ca
*[[User:aellebla|Aaron Leblanc]] - aellebla@connect.carleton.ca
*[[User:naseido|Nisrin Abou-Seido]] - naseido@connect.carleton.ca

=Layout=
==Paper==
: the paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.

: Ajay Gulati VMware Inc. Palo Alto, CA, 94304 agulati@vmware.com

: Arif Merchant HP Labs Palo Alto, CA 94304 arif.merchant@acm.org

: Peter J. Varman Rice University Houston, TX, 77005 pjv@rice.edu

==Background Concepts==
: Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.

: Hypervisors are responsible were multiplexing hardware resources between virtual machines while providing isolation to an extent, using resource management. The three controls used are reservation where the minimum bounds are set, the limit where the maximum upperbound on the allocation is set, and shares which proportionally allocate the resources according to the certain weight each VM has, and also depending on the reservation and upperbound limits. This is interesting because virtualization has been very successful; people are comfortable with putting multiple VM on one HOST without worrying about the performance of each VM on another. However the contention for I/O resources can suddenly lower a VM’s allocation; the available throughput can change with time, and adjustments to allocations must be made dynamically. mClock is a better alternate because it supports all controls in a single algorithm, handles variable and unknown capacity, and fast to compute. This is interesting because there is a limit control on VM allocation, it does not weaken as each VM gets added on, and mClock reservations are met. [[User:npatel1|Niravkumar Patel]]

: Here abit more on mClock!

: mClock is a resource-allocation algorithm that helps hypervisors manage I/O requests from multiple virtual machines simultaneously. Essentially, mClock dynamically adjusts the proportions of resources each VM receives based on how active each VM currently is. While mClock constantly changes the physical resource allocation to each VM, it lets each VM hold onto the illusion that it has full control of all system resources. As a result, performance can be increased for VMs that need it, without letting the others know that “their” resources are being distributed to other machines. [[User:npatel1|Niravkumar Patel]]

==Research problem==
: What is the research problem being addressed by the paper? How does this problem relate to past related work?
==Contribution==
: What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)
==Critique==
: What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.

: The article introduces a mClock algorithm which handles multiple VM in a variable throughput environment. The Quality of Service (QoS) requirements for a VM are expressed as a minimum reservation, a maximum limit, and a proportional share. mClock is able to meet those controls in varying capacity. the good thing about this is that the algorithms proves efficient in clustered architectures. Moreover, mClock provides greater isolation between VMs. [[User:npatel1|Niravkumar Patel]]

==References==
: You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

Talk:COMP 3000 Essay 2 2010 Question 10

2010-11-15T16:12:26Z

Tpham3: /* Group Members */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

=Group Members=
Please leave your name and email address if you are in the group

*[[User:Dagar|Daniel Agar]] - dagar@scs.carleton.ca
*[[User:xchen6|Xi Chen]] - xintai1985@gmail.com
*[[User:npatel1|Kumar Patel]] - npatel1@scs.carleton.ca
*[[User:tpham3|Tuan Pham]] - tpham3@scs.carleton.ca

===Waiting for response===
*Abou-Seido Nisrin naseido
*Leblanc Aaron aellebla

=Layout=
==Paper==
: the paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.
==Background Concepts==
: Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.
==Research problem==
: What is the research problem being addressed by the paper? How does this problem relate to past related work?
==Contribution==
: What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)
==Critique==
: What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
==References==
: You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

Talk:COMP 3000 Essay 2 2010 Question 10

2010-11-15T16:12:17Z

Tpham3: /* Group Members */

'''mClock: Handling Throughput Variability for Hypervisor IO Scheduling'''

=Group Members=
Please leave your name and email address if you are in the group

*[[User:Dagar|Daniel Agar]] - dagar@scs.carleton.ca
*[[User:xchen6|Xi Chen]] - xintai1985@gmail.com
*[[User:npatel1|Kumar Patel]] - npatel1@scs.carleton.ca
*[[User:tpham3|Tuan Pham]] - tpham3@scs.carleton.ca

===Waiting for response===
*Abou-Seido Nisrin naseido
*Leblanc Aaron aellebla
*Pham Tuan tpham3

=Layout=
==Paper==
: the paper's title, authors, and their affiliations. Include a link to the paper and any particularly helpful supplementary information.
==Background Concepts==
: Explain briefly the background concepts and ideas that your fellow classmates will need to know first in order to understand your assigned paper.
==Research problem==
: What is the research problem being addressed by the paper? How does this problem relate to past related work?
==Contribution==
: What are the research contribution(s) of this work? Specifically, what are the key research results, and what do they mean? (What was implemented? Why is it any better than what came before?)
==Critique==
: What is good and not-so-good about this paper? You may discuss both the style and content; be sure to ground your discussion with specific references. Simple assertions that something is good or bad is not enough - you must explain why.
==References==
: You will almost certainly have to refer to other resources; please cite these resources in the style of citation of the papers assigned (inlined numbered references). Place your bibliographic entries in this section.

Talk:COMP 3000 Essay 1 2010 Question 12

2010-10-15T20:42:17Z

Tpham3: /* Contact Details */

http://portal.acm.org.proxy.library.carleton.ca/citation.cfm?id=364716.364719&coll=Portal&dl=GUIDE&CFID=107313098&CFTOKEN=63043022

http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php It looks to be pretty accurate

http://www.ntfs.com/winfs_basics.htm

http://blogs.msdn.com/b/winfs/ The team blog

http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now <-- If you are bored and want to learn about WinFs, 54 min video.

http://www.splorp.com/newton/faq/newton-faq-nos.html#IIID3c an overview of the Newton OS filesystem to help you out --Rannath

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 BeOS --Rannath

I found some other stuff for Palm OS data base, but didn't find any good yet.

All the above links give good explanation of the subject. I will try to update later.

-Jean-Benoit

----

I found a little bit more in depth about NewtonOS:
http://lowendmac.com/orchard/06/john-sculley-newton-origin.html

Also, another name for Palm OS is Garnet OS. I haven't had time yet to look too much into this, but the wiki has many links to references and to the main site:
http://en.wikipedia.org/wiki/Palm_OS

Edit:
Since there are 5 database systems listed, and 5 people in the group, we can each pick one to work on. If anyone needs help with the one they pick, holla! Also, if you find another database system you want to work on, just add it to the list and put your name beside it.

WinFS - Jean-Benoit

ReiserFS - Sarah L

Newton OS - Daniel

PalmOS/Garnet OS - Tuan Pham

BeOS - Xi Chen

Other (list)...

-Daniel

----
Just to check how everybody are going? There is still two people who haven't picked an FileSystem, is there a problem or you simply didn't write your name next to it?

Also, for others i found an other database System made by Oracle. It is called : Internet File System. http://download.oracle.com/docs/html/A90434_01/toc.htm Here a good link to get started.

I'll probably posted in the page what I have written so far later today.

Also, to keep consistence, I don't know how you are writing yours, but mine will look something like that :
Short history of Winfs, predecessor , etc. (by short i mean roughly 200-300 words)
Short explanation on how WinFs works.
It's fate and Why

If you have any other idea, please share it.

-Jean-Benoit

----
Hey I am curious to how we are going to put all this together, are we each writing an essay for our chosen OS and then edit all the essays into one big one?

-Tuan Pham

----

I was thinking that we could simply have an introduction to express the general advantages and disadvantages of Database as filesystem, then create
sections for our different OS. Then at the end a conclusion.

But for that to work, we need most of our essay to be done in the next 2 days or so. Tuesday during the day we would need all the essay to be done so we can edit each other and write introduction + conclusion.

What do you think ? Any idea?

Edit : I just posted what I have so far. If you have any idea to add anything , just let me know. Also, English isn't my first language, so feel free to fix any of my errors.

I'll add the reference tomorow

-Jean-Benoit

----
Yeah i think we should do that but how about the two section that does not have the other group member's names beside?

- Tuan Pham

----

I will have to post my part of the project sometime Wednesday (most likely before 2pm). I apologize if this causes some problems for the group, but I have an assignment for another class that needs to be done by Wednesday, and I have little time Tuesday to work on this (work and classes). Here is a little summary of Newton OS to get an idea. I will make sure that by Wednesday afternoon I will have at least a near-finished copy of my work (some input may be required :p ).

Newton OS was used on PDAs and doesn’t have a file system, but instead uses a shallow database system. The OS considers each inserted card, as well as its internal storage, as separate stores. Each store contains either read/write databases or read-only objects. Newton eventually was cancelled and became an Apple subsidiary company, then shortly thereafter was reabsorbed into Apple. Developers from the Newton project helped with the iPod OS and the project may have had a hand in the iPhone’s development.

Edit: I left a notice to the group on the front page. Hopefully the other two members will see the notice. And about the two other sections: If we have at least 3 sections, we should be good, but hopefully we hear word from the other two members...

- Daniel

----
I just posted what i have so far. I am having difficulties finding the fate and why so if you guys know anything let me know. Also let me know if you have any suggestions for my part.

- Tuan Pham
----
Sorry, I couldn't post this weekend. I'll gladly work on MurderFS/ReiserFS, and I can definitely finish before Wednesday afternoon (2ish).

I really like how the WinFS sectioned is structured (although maybe condense the fate/why sections for readability?), it seems easy to follow. I agree we should all use the same sections (with additional subsections if need be). It'll make it look a lot cleaner/better put together.

- Sarah

Sorry, guys
Seems like I am the last member of the group, I am now doing works on the BeOS, and some stuff will be posted tonight or tomorrow morning.
Again, sorry for showing up late.

Xi Chen

Hi guys, Luc (lab TA ¬.¬) here, I'm seeing some good collaboration here, but I'm concerned about how coherent the end product will be. I think you guys need to spend some time working out the central thesis and how the various components will advance that point before everybody goes off to work on their little piece.

--[[User:3maisons|3maisons]] 20:14, 12 October 2010 (UTC)

----
I agree. I believe the prof said today that he wants a thesis and we use the examples (WinFS, ReiserFS, Newton OS, PalmOS/Garnet OS, and BeOS) to prove our argument. So we should discuss a thesis and how to format the essay

- Tuan Pham

That sounds good, and you guys thinking about meeting up sometime in lab?
or we can chat through gTalk or the wiki.

ps: my gmail acct is xintai1985@gmail.com
--Xi Chen

---

It could be a good idea to meet up to put everything together.
Or atleast use gTalk/msn/ whatever else people would like to use.
As we only have 2 days remaining, what about tonight ?

And as Tuan said, we would need a thesis subject and then prove why it fails most of the time.
I'm down to work on it whenever you guys are ready (or so obviously). Just send me an email to : jbout13@hotmail.com

-Jean-Benoit

----

I will be available until 2pm today (Wednesday) and then from 7pm onwards. I think we should discuss the thesis and such tonight (if possible), then finish the essay up after class tomorrow morning. My email is demongyro@gmail.com.

-Daniel

----

Let's try to be all around tonight for 7:30 ish ? so we can get thesis and start writing introduction + conclusion and to put everything together.

We either can go to Carleton, or msn / Gmail. (or any other option I forgot) I'm down for both of these.

xintai1985@gmail.com Xi Chen 613-799-5893(in case)

jbout13@hotmail.com Jean-Benoit

demongyro@gmail.com Daniel - on gmail

tuan-pham@hotmail.com Tuan Pham - lets do it over msn/gtalk

Edit : Daniel and I are on Gmail, gtalk.
Add him or me to join the conversation.

Jean-Benoit

Guys add my gmail, t.m.pham3@gmail.com - Tuan Pham

----
http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php
http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf
http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

so i don't lose these

Reiser4, Part II: Designing Trees that Cache Well
Performance Evaluation of Linux File Systems for Data Warehousing Workloads

sarah
----
moving this here because i'm a little ocd...

=Note to Group=

=Contact Details=
Dustin Martin: 613-325-9427 dustin.adam.martin@gmail.com

http://en.wikipedia.org/wiki/Wikipedia:Citing_Wikipedia

COMP 3000 Essay 1 2010 Question 12

2010-10-15T00:02:27Z

Tpham3: /* Other Links */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp [5]]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs [6]]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx [7]], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting[8]]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx [9]], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx[10]] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx [11]] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now [12]] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

[3]http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4]http://www.access-company.com/products/platforms/garnet/index.html

[5]http://www.winsupersite.com/showcase/winfs_preview.asp

[6]http://www.slideshare.net/Severus/winfs

[7]http://msdn.microsoft.com/en-US/library/aa480687.aspx

[8]http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting

[9]http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx

[10]http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx

[11]http://msdn.microsoft.com/en-US/library/aa480687.aspx

[12]http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on Palm OS programming

*http://www.letterp.com/~dbg/practical-file-system-design.pdf, Practical File System Design with the Be File System - Dominic Giampaolo

*http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 - A Chinese article talks about Be,Inc. BeOS and their fate

*http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 - From BFS to ZFS: past, present, and future of file systems

*http://en.wikipedia.org/wiki/BeOS - BeOS--Wikipedia

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:57:30Z

Tpham3: /* Reference */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

[3]http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4]http://www.access-company.com/products/platforms/garnet/index.html

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on how to Palm Os programming

*http://www.letterp.com/~dbg/practical-file-system-design.pdf, Practical File System Design with the Be File System - Dominic Giampaolo

*http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 - A Chinese article talks about Be,Inc. BeOS and their fate

*http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 - From BFS to ZFS: past, present, and future of file systems

*http://en.wikipedia.org/wiki/BeOS - BeOS--Wikipedia

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:56:38Z

Tpham3: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [3]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

[3]http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4]http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[5]http://www.access-company.com/products/platforms/garnet/index.html

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on how to Palm Os programming

*http://www.letterp.com/~dbg/practical-file-system-design.pdf, Practical File System Design with the Be File System - Dominic Giampaolo

*http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 - A Chinese article talks about Be,Inc. BeOS and their fate

*http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 - From BFS to ZFS: past, present, and future of file systems

*http://en.wikipedia.org/wiki/BeOS - BeOS--Wikipedia

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:55:58Z

Tpham3: /* Reference */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [4]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

[3]http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on how to Palm Os programming

*http://www.letterp.com/~dbg/practical-file-system-design.pdf, Practical File System Design with the Be File System - Dominic Giampaolo

*http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 - A Chinese article talks about Be,Inc. BeOS and their fate

*http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 - From BFS to ZFS: past, present, and future of file systems

*http://en.wikipedia.org/wiki/BeOS - BeOS--Wikipedia

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:55:42Z

Tpham3: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [4]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [4]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on how to Palm Os programming

*http://www.letterp.com/~dbg/practical-file-system-design.pdf, Practical File System Design with the Be File System - Dominic Giampaolo

*http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 - A Chinese article talks about Be,Inc. BeOS and their fate

*http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 - From BFS to ZFS: past, present, and future of file systems

*http://en.wikipedia.org/wiki/BeOS - BeOS--Wikipedia

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:54:58Z

Tpham3: /* Other Links */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [4]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [5]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html - Information on Palm OS

*http://mobile.eric-poncet.com/palm/tutorial/db.html - Tutorial on how to Palm Os programming

*http://www.letterp.com/~dbg/practical-file-system-design.pdf, Practical File System Design with the Be File System - Dominic Giampaolo

*http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 - A Chinese article talks about Be,Inc. BeOS and their fate

*http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 - From BFS to ZFS: past, present, and future of file systems

*http://en.wikipedia.org/wiki/BeOS - BeOS--Wikipedia

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:54:00Z

Tpham3: /* Other Links */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [4]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [5]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html

*http://mobile.eric-poncet.com/palm/tutorial/db.html

*http://www.letterp.com/~dbg/practical-file-system-design.pdf, Practical File System Design with the Be File System - Dominic Giampaolo

*http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235 - A Chinese article talks about Be,Inc. BeOS and their fate

*http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 - From BFS to ZFS: past, present, and future of file systems

*http://en.wikipedia.org/wiki/BeOS - BeOS--Wikipedia

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:52:39Z

Tpham3: /* Reference */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [4]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [5]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

==Other Links==

*http://membres.multimania.fr/microfirst/palm/pdb.html

*http://mobile.eric-poncet.com/palm/tutorial/db.html

*Practical File System Design with the Be File System, Dominic Giampaolo

*http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

*http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

*http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

*http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:51:47Z

Tpham3: /* Reference */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [4]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [5]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:51:23Z

Tpham3: /* Problem */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record[2]. This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [4]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [5]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:51:03Z

Tpham3: /* Palm OS */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [2]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [2]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [2] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf [4]] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html [5]] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:49:43Z

Tpham3: /* Reference */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [BOOK REF]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [BOOK REF]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:49:08Z

Tpham3: /* History */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [BOOK REF]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [BOOK REF]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:48:44Z

Tpham3: /* History */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf [1]]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [BOOK REF]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [BOOK REF]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:47:53Z

Tpham3: /* Database Structure */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]". Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications. Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [BOOK REF]. The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [BOOK REF]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:46:56Z

Tpham3: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]". Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS, WinFS was not as fortunate and leads a different path.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:45:26Z

Tpham3: /* Fate */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]". Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system. Unlike the Palm OS

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:43:54Z

Tpham3: /* History */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]". Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:42:40Z

Tpham3: /* History */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf Source 1]". Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:40:30Z

Tpham3: /* Reference */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning,<ref>"Author, Publisher, Date of Publication, [url=http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf Source 1]", Today</ref>. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

Practical File System Design with the Be File System, Dominic Giampaolo

http://www.letterp.com/~dbg/practical-file-system-design.pdf

A Chinese article talks about Be,Inc. BeOS and their fate

http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235

From BFS to ZFS: past, present, and future of file systems

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7

BeOS--Wikipedia

http://en.wikipedia.org/wiki/BeOS

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:37:43Z

Tpham3: /* History */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning,<ref>"Author, Publisher, Date of Publication, [url=http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf Source 1]", Today</ref>. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{Reflist}}

COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:34:51Z

Tpham3: /* Reference */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system using a database driven file system, each developed to satisfy its individual needs. We will visit five different operating systems that used a database as file system. First, both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. Lastly, ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.
The success of each of these file systems is intrinsically tied to that of its host operating system, and vice versa. One cannot succeed without the other. With that in mind, by closely examining each file system and operating system combination individually, it is clear that none have been successful. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning,[url=http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project. We will now talk about NewtonOS.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.[http://en.wikipedia.org/wiki/BeOS]

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking.[http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7] Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

And now we will exam ReiserFS.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these examples of database-like systems, it becomes apparent that despite their potential, each did not live up to its expectations. Since the success of a file system is tied to that of the operating system, a harmonious relationship must exist between the two. In each case defined above, Palm OS, WinFS, Newton OS, BeOS and ReiserFS, that relationship did not manifest into success. There has not yet been developed a highly flexible design that would be useful on the average system. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch from a tried and true system to a newer and possibly unstable technology. In the case of database-like file systems, the few implementations that are available to the public are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that database-like stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{Reflist}}

Talk:COMP 3000 Essay 1 2010 Question 12

2010-10-14T23:03:32Z

Tpham3: /* Contact Details */

http://portal.acm.org.proxy.library.carleton.ca/citation.cfm?id=364716.364719&coll=Portal&dl=GUIDE&CFID=107313098&CFTOKEN=63043022

http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf

http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php It looks to be pretty accurate

http://www.ntfs.com/winfs_basics.htm

http://blogs.msdn.com/b/winfs/ The team blog

http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now <-- If you are bored and want to learn about WinFs, 54 min video.

http://www.splorp.com/newton/faq/newton-faq-nos.html#IIID3c an overview of the Newton OS filesystem to help you out --Rannath

http://arstechnica.com/hardware/news/2008/03/past-present-future-file-systems.ars/7 BeOS --Rannath

I found some other stuff for Palm OS data base, but didn't find any good yet.

All the above links give good explanation of the subject. I will try to update later.

-Jean-Benoit

----

I found a little bit more in depth about NewtonOS:
http://lowendmac.com/orchard/06/john-sculley-newton-origin.html

Also, another name for Palm OS is Garnet OS. I haven't had time yet to look too much into this, but the wiki has many links to references and to the main site:
http://en.wikipedia.org/wiki/Palm_OS

Edit:
Since there are 5 database systems listed, and 5 people in the group, we can each pick one to work on. If anyone needs help with the one they pick, holla! Also, if you find another database system you want to work on, just add it to the list and put your name beside it.

WinFS - Jean-Benoit

ReiserFS - Sarah L

Newton OS - Daniel

PalmOS/Garnet OS - Tuan Pham

BeOS - Xi Chen

Other (list)...

-Daniel

----
Just to check how everybody are going? There is still two people who haven't picked an FileSystem, is there a problem or you simply didn't write your name next to it?

Also, for others i found an other database System made by Oracle. It is called : Internet File System. http://download.oracle.com/docs/html/A90434_01/toc.htm Here a good link to get started.

I'll probably posted in the page what I have written so far later today.

Also, to keep consistence, I don't know how you are writing yours, but mine will look something like that :
Short history of Winfs, predecessor , etc. (by short i mean roughly 200-300 words)
Short explanation on how WinFs works.
It's fate and Why

If you have any other idea, please share it.

-Jean-Benoit

----
Hey I am curious to how we are going to put all this together, are we each writing an essay for our chosen OS and then edit all the essays into one big one?

-Tuan Pham

----

I was thinking that we could simply have an introduction to express the general advantages and disadvantages of Database as filesystem, then create
sections for our different OS. Then at the end a conclusion.

But for that to work, we need most of our essay to be done in the next 2 days or so. Tuesday during the day we would need all the essay to be done so we can edit each other and write introduction + conclusion.

What do you think ? Any idea?

Edit : I just posted what I have so far. If you have any idea to add anything , just let me know. Also, English isn't my first language, so feel free to fix any of my errors.

I'll add the reference tomorow

-Jean-Benoit

----
Yeah i think we should do that but how about the two section that does not have the other group member's names beside?

- Tuan Pham

----

I will have to post my part of the project sometime Wednesday (most likely before 2pm). I apologize if this causes some problems for the group, but I have an assignment for another class that needs to be done by Wednesday, and I have little time Tuesday to work on this (work and classes). Here is a little summary of Newton OS to get an idea. I will make sure that by Wednesday afternoon I will have at least a near-finished copy of my work (some input may be required :p ).

Newton OS was used on PDAs and doesn’t have a file system, but instead uses a shallow database system. The OS considers each inserted card, as well as its internal storage, as separate stores. Each store contains either read/write databases or read-only objects. Newton eventually was cancelled and became an Apple subsidiary company, then shortly thereafter was reabsorbed into Apple. Developers from the Newton project helped with the iPod OS and the project may have had a hand in the iPhone’s development.

Edit: I left a notice to the group on the front page. Hopefully the other two members will see the notice. And about the two other sections: If we have at least 3 sections, we should be good, but hopefully we hear word from the other two members...

- Daniel

----
I just posted what i have so far. I am having difficulties finding the fate and why so if you guys know anything let me know. Also let me know if you have any suggestions for my part.

- Tuan Pham
----
Sorry, I couldn't post this weekend. I'll gladly work on MurderFS/ReiserFS, and I can definitely finish before Wednesday afternoon (2ish).

I really like how the WinFS sectioned is structured (although maybe condense the fate/why sections for readability?), it seems easy to follow. I agree we should all use the same sections (with additional subsections if need be). It'll make it look a lot cleaner/better put together.

- Sarah

Sorry, guys
Seems like I am the last member of the group, I am now doing works on the BeOS, and some stuff will be posted tonight or tomorrow morning.
Again, sorry for showing up late.

Xi Chen

Hi guys, Luc (lab TA ¬.¬) here, I'm seeing some good collaboration here, but I'm concerned about how coherent the end product will be. I think you guys need to spend some time working out the central thesis and how the various components will advance that point before everybody goes off to work on their little piece.

--[[User:3maisons|3maisons]] 20:14, 12 October 2010 (UTC)

----
I agree. I believe the prof said today that he wants a thesis and we use the examples (WinFS, ReiserFS, Newton OS, PalmOS/Garnet OS, and BeOS) to prove our argument. So we should discuss a thesis and how to format the essay

- Tuan Pham

That sounds good, and you guys thinking about meeting up sometime in lab?
or we can chat through gTalk or the wiki.

ps: my gmail acct is xintai1985@gmail.com
--Xi Chen

---

It could be a good idea to meet up to put everything together.
Or atleast use gTalk/msn/ whatever else people would like to use.
As we only have 2 days remaining, what about tonight ?

And as Tuan said, we would need a thesis subject and then prove why it fails most of the time.
I'm down to work on it whenever you guys are ready (or so obviously). Just send me an email to : jbout13@hotmail.com

-Jean-Benoit

----

I will be available until 2pm today (Wednesday) and then from 7pm onwards. I think we should discuss the thesis and such tonight (if possible), then finish the essay up after class tomorrow morning. My email is demongyro@gmail.com.

-Daniel

----

Let's try to be all around tonight for 7:30 ish ? so we can get thesis and start writing introduction + conclusion and to put everything together.

We either can go to Carleton, or msn / Gmail. (or any other option I forgot) I'm down for both of these.

xintai1985@gmail.com Xi Chen 613-799-5893(in case)

jbout13@hotmail.com Jean-Benoit

demongyro@gmail.com Daniel - on gmail

tuan-pham@hotmail.com Tuan Pham - lets do it over msn/gtalk

Edit : Daniel and I are on Gmail, gtalk.
Add him or me to join the conversation.

Jean-Benoit

Guys add my gmail, t.m.pham3@gmail.com - Tuan Pham

----
http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php
http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf
http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf

so i don't lose these

Reiser4, Part II: Designing Trees that Cache Well
Performance Evaluation of Linux File Systems for Data Warehousing Workloads

sarah
----
moving this here because i'm a little ocd...

=Note to Group=

=Contact Details=
Dustin Martin: 613-325-9427 dustin.adam.martin@gmail.com

794-five zero five six Jean-Benoit

899-8826 Tuan Pham

http://en.wikipedia.org/wiki/Wikipedia:Citing_Wikipedia

COMP 3000 Essay 1 2010 Question 12

2010-10-14T19:49:48Z

Tpham3: /* Database Structure */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

What about something like this ?

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system that use a database as file system , all chosen for certain features that made sense in that application. We’ll visit five different operating systems that used a database as file system. Both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking. Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{reflist}}
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T19:47:50Z

Tpham3: /* Database Structure */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

What about something like this ?

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.
Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, web servers and more. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, and file systems that function as databases.

There have been various implementations of operating system that use a database as file system , all chosen for certain features that made sense in that application. We’ll visit five different operating systems that used a database as file system. Both Palm OS and WinFS implemented a database file system for their operating systems for improve efficiency. NewtonOS was produced to be used on a desktop machine, but evolved into one of the best and most secure PDA operating systems. BeOS was a new operating system and decided to use a database driven file system to leverage both improved efficiency and compability. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux. We will first examine the trials of Palm OS.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
"Palm OS does not use a relational or XML database"[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===
BeOS is an operating system that was first developed from scratch in 1991 to run a proprietary hardware. The goal for BeOs was to compete with both Apple and Microsoft, by creating a new platform. Since both Apple and Microsoft were putting considerable effort into covering legacy versions of their systems, BeOs hoped to gain ground through efficiency.[http://www.letterp.com/~dbg/practical-file-system-design.pdf]

BeOS was designed to be a better use on modern hardware. Compared to Windows and Mac OS, it greatly improved on digital media performance by using modern hardware facilities like symmetric multiprocessing, pervasive multithreading and preemptive multitasking. Compared with other modern operating system, pervasive multithreading is allowing the system to better utilize system resources. It also has a simple and clear GUI and powerful command-line interface through BASH shell. However, since the system is not UNIX based, it is also compatible with POSIX. Furthermore, the API of BeOS is developed under C++ which facilitate application development.

But since BeOS is a fully multithreaded system, programs which are written for the system need to be really well programmed in order to stop deadlock or errors from doing multiple simultaneous threads.[http://translate.google.com/translate?hl=en&sl=zh-CN&tl=en&u=http://www.techcn.com.cn/index.php%3Fdoc-view-135996%235] Also not a lot of hardware is supported by BeBox, the machine designed for BeOS. And maintaining the system is another problem due to database like file system.

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the BFS journaling file system was implemented by BFS to be a high performance file system that stores all data in a database, but also allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. This is because the system reliability is placed in the hands of the programmer.

===Fate===
In the mid-1990's BeOs was pursued by Apple to serve as their new operating system. BeOs demanded a purchase price that Apple was not willing to pay and instead Apple decided to acquire NeXT and implemented NeXTSTEP OS. BeOs then tried to penetrate the personal computer market but ran into compatibility issues. BeOs needed custom hardware to function correctly. Development ensued to port BeOs onto standard PCs, but ultimately fell flat when Be could not convince any manufacture to use their operating system. This was largely due to restriction and exclusive rights held by Microsoft which blocked BeOs' entry into the market. By 2001, BeOs had failed to obtain even marginal market share and was acquired by Palm to be integrated into Palm OS. An small community driven open source project named Haiku OS has resumed development of BeOs and released Haiku Alpha 2 in May 2010.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
{{reflist}}
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:33:44Z

Tpham3: /* Answer */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database-like stores?**

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the journaling file system implemented by BFS, stores all data in a journaled database, but allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any possible data format of their choosing.

Since BFS is a custom file system developed solely to be used by BeOs its success is intrinsically tied to that of its operating system. BeOs poses a threat to BFS both because of its inability to reach the masses as well as the potential memory errors caused by its unfiltered multithreading capability. Because, system reliability is placed in the hands of the programmer, BeOs's multithreading capability is a potential cause for concern especially when combined with a slow access file system.

===Fate===

Schmalensee didn't mention that Be had trou- 34
ble giving away the BeOS operating system.
Gassée approached a number of PC manufacturers
to see if they would include BeOS
on their machines and give users the chance
to switch between two operating systems.
Gassée found, to no one's real surprise,
that Microsoft's contracts with manufacturers
made it difficult, if not practically impossible,
to get BeOS in customers' hands. Microsoft
controlled much of what the user got to see
and insisted on almost total control over the
viewer's experience. Schmalensee didn't
mention these details in his testimony. BeOS

Microsoft execlusive lisence with hardware manufacture made harder to enter the market

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:15:06Z

Tpham3: /* Flaws of Newton OS */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database stores?**

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

**I dont think the sentence with the simpsons part is a good idea because it is not likely for essays to have something like that**

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the journaling file system implemented by BFS, stores all data in a journaled database, but allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any posible data format of their choosing.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T18:11:23Z

Tpham3: /* Answer */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

**For the last two paragraphs did you mean database stores?**

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data"[http://msdn.microsoft.com/en-US/library/aa480687.aspx], but with data treated as data inside a database, we could find what we are looking for quite easily.

Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.[http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===
The file system used by BeOS is called BFS. BFS is a custom file system developed by the same team as BeOs. BFS is a 64-bit journaling file system which logs all data before storing it. [http://www.citeulike.org/user/msakai/article/638204] This journaling increases both the stability and security of BeOs, by preventing system crashes from jeopardizing the file system’s integrity as well as offering quicker recovery from crashes. However, this comes at the expensive of rapid data acess since a journaling file system is based on random memory access. Uniquely, the journaling file system implemented by BFS, stores all data in a journaled database, but allows users to view it in a more traditional heirachical way. [http://www.letterp.com/~dbg/practical-file-system-design.pdf] It also boost flexibility by allowing users to store any posible data format of their choosing.

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great alternative to the traditional file system. However, there has not yet been developed a highly flexible design that would be useful on the average PC, as we can see from the failure of WinFS and ReiserFS.

The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice.

It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_9], so there is no guarantee that object stores will ever become a widely used technology, as they may not mature quickly enough to beat out other technologies.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html

COMP 3000 Essay 1 2010 Question 12

2010-10-14T17:53:25Z

Tpham3: /* Brief Concept */

==Question==

There have been multiple attempts to have operating systems use databases or database-like stores. What have been some of the major past attempts at this? What was their fate? Why? Key examples (not exhaustive): WinFS, ReiserFS, PalmOS, Newton OS, BeOS

==Answer==

There have been many attempts at creating file systems that use database-like stores. While the idea is an interesting one, database stores are just not ready for the consumer market.

Traditionally, databases are used in applications where a project focuses on accessing large amounts of data quickly and efficiently, such as banking systems, telecommunications, and web servers. A personal computer did not traditionally need as much storage, and is organized in an easy-to-navigate tree structure. However, the recent shift towards object-oriented programming styles, along with the tremendous increase in the amount of data that can be stored on a home computer, has led to the idea of object stores, or file systems that function as databases of objects [http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_11].

There have been various implementations of object stores in the real world, all chosen for certain features that made sense in that application. For PalmOS and NewtonOS, the security and modularity object stores provided was an attractive fit for mobile devices. Windows was drawn to object stores because of the possible speed increases, and so they developed WinFS. ReiserFS seemed to have started as a proof-of-concept, which found it's place with Linux.

Object stores have caught on in a few niche applications, mostly scientific. However, this technology is in its infancy, still trying to engineer issues that are arising while implementing the concept. Combined with issues with management that surround most attempts at creating an object store, it is clear that object stores are a long way off from becoming a common file system.

==Palm OS==

===History===
PalmOS was developed from Graffiti, a hand-recognition software which was believed to be doomed from the very beginning[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Graffiti’s fast and accurate hand-recognition software was thought to be useless because many companies did not see a purpose for it; there were no companies that wanted to make the hardware for it[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. The company Palm then decided to make their own hardware with Graffiti as the OS, which became what is known as Palm OS today.

===Database Structure===
Palm OS does not use a relational or XML database[http://www.iam.unibe.ch/~rvs/teaching/SS02/bs/arbeiten/Guido_Gloor-PalmOS.pdf]. Instead Palm OS has two different database types, record and resource database. There are similarities between these two databases, they both have headers which stores information onto the database and are both stored in the storage heap. The difference between the record database and resource database is how the information is stored, called, handled and how it is named. The record database is used to store data, and the resource database contains all applications.

Furthermore a record database is a collection of records (blocks of memory) each record can only store up to 64KB of memory. It has information that is unique to the record; it has the location of the record, an ID, and an attribute, which contains delete, dirty, busy, and secret bits [book ref].

The resource database stores application code, name, icon, forms, alerts, menus, strings, and all other elements of the application [book ref]. Applications can be referenced by an ID number and a type (four-character constant).

===Problem===
Record and resource databases are stored in storage/database heap, which in turn is stored in the RAM (Random Access Memory). The database heap has a limit of 64KB of memory and since a record had to be small enough to fit within the heap, this made memory hard to manage. For example, when a record is 50KB and there are three heaps which can only hold 64KB each, a total of 192KB of memory; say these three heaps are half full therefore there is 96KB of free memory, but the record of size 50KB cannot be stored because not a single heap has enough memory to store the record. [book ref] This problem arose in versions before OS 3.0 because the database heap was divided into smaller parts. This allowed for free space in memory, but some of those heaps were occupied by data, and there might not be enough space to store the record into the heap. Palm OS 3.0 and later solved this problem by to making a single large storage heap instead of multiple smaller heaps.

===Fate===

In 2005, Palm OS was acquired by Access Systems Americas, Inc. Access decided to expand the breadth of the operating system’s offering to include more tools, a better user experience, and increased compatibility across many more devices.[http://azul.iing.mxl.uabc.mx/~renecruz/papers/Unpublished-Cruz-Flores-2.pdf] In 2007, development of Palm OS shifted to a new operating system called Garnet OS. [http://www.access-company.com/products/platforms/garnet/index.html] Its latest release, Garnet version 6, still includes a database based file system.

==WinFS==

===History===

The history of WinFS is relatively long, as we can find trace of the project back in mid-90's with Storage +. Microsoft had the idea to remove the NTFS file system and instead use a relational object-oriented file storage, which was based on SQL server 8.0. This new file system was supposed to be implemented in Windows 2003 Server. But then in 2000, Windows announced that Storage + was to be forgotten, with Relational File System (RFS) as its successor. RFS was supposed to be included in SQL Server 2000, but never made the cut. Another reason for the continual delays was that, in 2000, Oracle announced a new file system which was also a relational file system, Independent Internet File System. Microsoft had to rethink RFS to be ahead of the competition and that added additional delay.[http://www.winsupersite.com/showcase/winfs_preview.asp]

RFS was forgotten about until the public heard in 2002 about a new file system that would be present in Longhorn (later renamed Vista). The system would be once again be based on its predecessor, RFS, but it would have run on top of a NTFS file system. WinFS was included in a few public builds for Vista, but in 2004 it was removed from the beta builds. It was said it would be downloadable again after the release, but was definitely cut for good in 2006 from Vista.

===Brief Concept===

With today’s data, we are facing a crisis of finding what we want, when we want, at a reasonable speed on our own computer. Data is stored in many different ways. We can recognize the uses of certain files by their file extensions, but the amount of file extensions that exist is astonishing, so it is quite difficult to remember each one. Also, a simple file may be stored with different extensions, in different databases, which makes finding, relating to, and acting on the file quite difficult to achieve. [http://www.slideshare.net/Severus/winfs]

**Obscure 1st sentence: data would be simply data? Elaborate.**
Microsoft had an idea to solve this problem by using a relational database as a file system, where data would be treated like it should be, as data and nothing else. To understand how WinFS works, we must have a general idea of the relational database. The data in this kind of database is spread into specific tables, like in a normal database, but there are multiple relations between these tables. This gives the programmer the power to search, find and present the result in an efficient way. In the case of WinFS, the main goal was to "Enable people to Find, Relate, and Act on their information"[http://msdn.microsoft.com/en-US/library/aa480687.aspx].

**Reference for quotation needed**
As already pointed out, data files are very broad nowadays. With that many file formats using complicated data storage methods, the "...current file system does not know how to collect and find information within these new types of data", but with data treated as data inside a database, we could find what we are looking for quite easily.

**"In our...to Bob." Needs to be easier to read/understand. Also, treat data as data? Elaborate. Also, any reference?**
Another important point of WinFS is the notion of how data relates to each other. In our current file system we can't, unless doing it manually, add a picture of our good friend Bob, and in the same time see all the picture related to Bob. On top of that, we can't find the picture of Bob, all the received emails, documents, movies, or whatever else we would want to find in the same request. In order to find them, we have to search for them one by one. If we treat data as data, once again, we can simply search in the tables of the database for the pictures, received emails, videos, and documents containing the name Bob, and present them to the user. As we can see, data can be related to each other with keywords, in our example it was the name Bob, but it can be anything.

What if data could do actions that follow a specific set of rules? This is exactly what WinFS wanted to implement. "WinFS Rules are a built-in component of the system that allows you to tell the system how to work with, sort, and deliver your data." Also, it could make use of other applications on the system. So if we received a picture from our friend Bob, we could automatically transfer it elsewhere. [http://channel9.msdn.com/Blogs/scobleizer/Shishir-Mehrotra-WinFS-beta-1-team-meeting]

The last problem that WinFS was aiming to overcome was to run on top of NTFS. Basically, WinFS would scan all the data in the NTFS file system, and put it into its database. Thus, it would work as a file system, but it would be totally dependent of NTFS.

===Fate of WinFS===

As we have seen in the history of WinFS, its fate wasn't as desired. In 2006, on the team blog, Microsoft announced that it wouldn't include WinFS as a system file package, but instead deliver it into the next Microsoft SQL server, which was SQL Server 2008. "These changes do mean that we are not pursuing a separate delivery of WinFS, including the previously planned Beta 2 release"[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx], but instead, they will keep working on it and some "...may be used by other Microsoft products going forward."[http://blogs.msdn.com/b/winfs/archive/2006/06/26/648075.aspx] So, we can see that WinFS didn't succeed as a file system, but some of the logistics will continue to live on in Microsoft’s database software.

===Why===

Microsoft never released publicly what exactly went wrong. There was much speculation about the design of WinFS, but on the team blog Quentin Clark, Product Unit Manager of WinFS, answered that "[i]n fact, the Beta was coming together really well."[http://msdn.microsoft.com/en-US/library/aa480687.aspx] He then replied that the technology used wasn't easy to build onto, so they had to rewrite some parts, but that this wouldn't have caused the end of WinFS. Some also speculate that no serious software used WinFS, nor did it receive the attention needed from the developers to have a good start. In an interview with Channel 9, Quentin Clark said, "We were building too much of the house at once. We had guys working on the roof while we were still pouring concrete for the foundation."[http://channel9.msdn.com/blogs/jonudell/where-is-winfs-now] This shows that the team might have had some management problem, which evidently led to the termination of the project.

==Newton OS==

===Brief History of Newton OS===

Newton OS was created by Apple and was used with their line of PDAs, becoming one of the world's first PDAs. Newton was originally meant to be an innovative OS to reinvent personal computing, but it was changed to become a PDA, due to fear of eating up Macintosh sales and because of project delays [http://www.techeblog.com/index.php/tech-gadget/a-look-back-apple-newton]. The first iteration of the Newton system came as the MessagePad. The MessagePad sold out its 5,000 copies within a few hours of release, despite its price of $800 US [http://www.pencomputing.com/frames/newton_obituary.html]. Apple released several different PDAs with Newton OS between 1993 and 1997, which despite its popularity were plagued by flaws in the applications of the device [http://news.cnet.com/Apple-scraps-Newton/2100-1001_3-208551.html].

The main reason why Newton PDAs became so well-known was because the Newton OS was the most advanced operating system of any personal computing device of its time. The OS kept the user from accessing the inner workings of the device, keeping users away from creating problems by tampering with the wrong settings, and used a database-based file management that simplified the system to a higher degree than any previous OS [http://www.pencomputing.com/frames/newton_obituary.html].

===Flaws of Newton OS===

Even with the innovation that came from Newton OS and its computing devices, it had several deep-running flaws that caused critics to pan the Newton devices. First and foremost, the Newton devices were known for their failure at implementing their handwriting system, which was supposed to recognize entire words. The Simpsons television show even had a joke about this flaw in one of their episodes [http://www.youtube.com/watch?v=xc3JzS0K3ys]. The main problem behind the handwriting system was that it had a hard time recognizing cursive writing, even though Apple insisted that its engineers ensured it worked correctly [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Another main problem with the Newton devices was that their overall size was too large for most pockets. Since Newton devices were expected to be carried similarly like a wallet or a cellphone, most people found their size too great for daily use [http://news.cnet.com/8301-13512_3-9754359-23.html][http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. Earlier Newton devices were also found to be very slow due to the virtualization of the NewtonScript and the lack of necessary RAM [http://www.roughlydrafted.com/RD/Q4.06/600D65E6-A31E-45CA-AFC5-42BC253F5337.html]. In the end, the Newton devices were fated to fail.

===Fate===

Although the Newton OS was revolutionary when it was released, it was doomed to fail due mostly to impracticality. In early 1997, Newton Inc. was created as a subsidiary company of Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2169_v43/ai_19494161/], but after a relatively short run, Newton Inc. was reabsorbed into Apple [http://findarticles.com/p/articles/mi_m0EKF/is_n2189_v43/ai_19892982/]. In December of 1997, Apple effectively ceased development efforts on the Newton OS [http://news.cnet.com/2100-1001-206664.html], and in February 1998, Apple announced the discontinuation of the Newton OS development [http://www.apple.com/ca/press/1998/02/NewtonDisco.html].

===Why===

After his return as CEO of Apple, Steve Jobs canceled or restructured many of Apple's failing products to refocus Apple's energy towards more successful endeavors, like the iPod and Macintosh computers. Not surprisingly, Newton OS was one of these products [http://www.roughlydrafted.com/2010/01/26/steve-jobs-apple-tablet-%E2%80%9Cthe-most-important-thing-ive-ever-done%E2%80%9D/]. Although Newton was canceled, the development of the OS and its devices have had impacts on other Apple products. Mac OS X 10.2 has a handwriting recognition software called Inkwell that uses an external tablet to recognize words, but like the Newton handwriting software, it requires you to right each letter individually [http://lowendmac.com/osx/jaguar-10.2/index.html][http://everything2.com/title/Inkwell]. Pixo, the company that created the operating system for the iPod, was founded by two of the developers that worked on the Newton OS [http://www.starmenusa.com/blog/category/Culture][http://articles.sfgate.com/2004-08-16/business/17438885_1_pixo-user-interface-ipod]. Pixo was subsequently acquired by Apple after the shipping of the first iPods [http://www.servinghistory.com/topics/Pixo].

==BeOS==

===Brief History===

BeOS is released by Be, Inc which is founded in 1990 by Jean-louis Gasee and Steve Sakoman. Be OS is a brand new operating system running on it’s own hardware called BeBox. The system is giving a better performance on digital media works because of better use on system resources. This system is initially made for AT&T Hobbit hardware, but since AT&T decide to stop producing Hobbit processor, the system is modified to Apple’s PowerPC hardware.

===Operating System===

===File System===

==ReiserFS==

===History===
ReiserFS is the product of Namesys, a Californian company stared by Hans Reiser [http://web.archive.org/web/20071024001500/http://www.namesys.com/v4/v4.html]. ReiserFS was first introduced in version 2.4.1 of the Linux kernel, and was the default file system on several Linux distributions, most notably SUSE. ReiserFS's successor is Reiser4, which released in 2004[http://en.wikipedia.org/wiki/Reiser4]. However, in 2008, Namesys dissolved and commercial production of ResierFS and Reiser4 halted [http://news.cnet.com/8301-13580_3-9851703-39.html]. Since the removal of commercial support, ReiserFS has become less popular, probably due to the many bugs that have no hope of being resolved.

===Concept===
ReiserFS is driven by a few major concepts. The first and probably main idea is the ability to handle many files that are smaller than a block of storage on the storage media. This is accomplished through the use of B+ Trees. [http://delivery.acm.org.proxy.library.carleton.ca/10.1145/610000/603630/6267.html?key1=603630&key2=4060996821&coll=ACM&dl=ACM&CFID=105645345&CFTOKEN=72691832]. Instead of balancing in order to keep the height of the tree fairly stable (as an AVL tree balances), ReiserFS balances so that the height of the storage tree is constant. This type of balanced tree reduces overhead by reducing the number of internal nodes, or i-nodes needed. Each node in the B+ tree maps to a block on the storage device, and each node can hold multiple objects, each of which has a unique key[http://homes.cerias.purdue.edu/~florian/reiser/reiserfs.php]
Another concept in ReiserFS is the use of unified name spaces[http://www4.informatik.uni-erlangen.de/Lehre/WS01/PS_KVBK/docs/reiserfs/MosheBarReiserFS.pdf]. Unified name spaces are simply a more refined definition of an object store, in which all stored data is made up of objects that are both 'files' and 'directories'. Objects are stored in such a way so that you can use directories to quickly access different types of objects, then further traverse the directories in order to access specific objects, and go even further to access an object's attributes. This maps extremely well to different programming styles, especially object oriented programming. Lookup of data can also done using attributes of the data, much like in WinFS[http://homeostasis.scs.carleton.ca/wiki/index.php/COMP_3000_Essay_1_2010_Question_12#WinFS]. This type of access would be more often used by the end user.

===Flaws and Fate===
While ResierFS preforms very well in terms of speed [http://www.sun.com/software/whitepapers/solaris10/fs_performance.pdf], ReiserFS has many bugs that can lead to instability of files. One problem is that link() and unlink() are not synchronous in ReiserFS, which can lead to data corruption[http://archives.neohapsis.com/archives/postfix/2001-05/1749.html]. Another is that, if ReiserFS's tree structure becomes unusable for any reason, a rebuild risks further corrupting the file system [http://lkml.indiana.edu/hypermail/linux/kernel/0506.3/0219.html]. There are also issues with the journaling which can lead to corruption [http://users.cis.fiu.edu/~yangz/teaching/reading_list/Analysis%20and%20Evolution%20of%20Journaling%20File%20Systems.pdf]. Issues like these are a bad idea in any file system, as the risk of unstable data is a serious drawback. Considering that the root of these issues are often synchronization problems, ReiserFS is a very poor choice for many modern, multi-core systems, as the increased need for synchronization would increase the level of instability.

In addition to the stability issues, ReiserFS has some design issues mainly contribute to why it is no longer used today. Since ReiserFS was designed to handle numerators small files, when trying to scale a ReiserFS system, behavior is inconsistent, again spawning from issues in synchronization. In addition, moving from the bug-ridden ReiserFS v3 to v4 required a reformat, as Reiser4 was re-written from scratch. This prompted SUSE to drop ReiserFS as it's default file system, as they preferred to use a tried and tested sytem update (ext3) instead of a new, yet-untested file system in their distribution [http://lwn.net/Articles/202780/].

Interestingly enough, the SUSE drop coincided eerily well with Hans Reiser being sent to jail. Left without an owner, Namesys lost most corporate sponsors, eventually only receiving funding from the DARPA initiative. With Namesys (mostly) gone, ReiserFS no longer has commercial support, and so the stability and scalability bugs are taking a fair amount of time to be resolved. This combined with low visibility that stems from no longer being used in any major Linux distributions largely explains why ReiserFS can be considered a dead project.

==Conclusion==
Looking at these object based file systems, it is easy to see that, despite object stores being an interesting and potentially useful idea, they are just not ready yet for the desktop. For certain applications, such as systems designed to handle large amounts of data for programming projects, such as ones used for models in physics; or in small, modular systems such as PalmOS and NewtonOS, object stores are a great improvement on the traditional file system. However, for the average desktop, object stores preform somewhat less efficiently than the traditional model, due to scalability issues. The increasing importance of digital information in today's world also means that people are extremely reluctant to switch a tried and true system for newer and possibly unstable technology. In the case of object stores, what few implementations are available to the public for the desktop are unstable, and therefore an unattractive choice. It is important to note that object stores are not the only alternative file system competing to improve upon traditional block storage, so there is no guarantee that object stores will become a widely used technology even if these problems are worked out.

==Reference==
[1] Palm OS programming: the Developer’s guide by Neil Rhodes, Julie McKeehan

http://membres.multimania.fr/microfirst/palm/pdb.html

http://mobile.eric-poncet.com/palm/tutorial/db.html