Soma-notes - User contributions [en]

COMP 3000 Essay 2 2010 Question 11

2010-12-03T00:35:17Z

Jjpwilso: /* Contribution */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running the whole time. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts behind it. For example, we all know what clocks are in our day-to-day lives but how are they different in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, explore the advantages and disadvantages of the different available counters and synchronization algorithms, and explain what a para-virtualized system is about.

===Timekeeping===

Computers typically measure time in one of two ways: tick counting and tickless timekeeping[2]. Tick counting is when the operating system sets up a hardware device, generally a CPU, to interrupt at a certain rate. A counter is updated each time one of these interrupts occurs. It is this counter that allows the system to keep track of the passage of time.

In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead. This device has its own counter which starts when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts, however its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift (see below). But neither of these methods knows the actual time, they only know how long it has been since they last checked an authoritative source. Personal computers typically get their time from a battery-backed real-time clock (i.e. a CMOS clock). Networked machines often need a more precise time, with a resolution in the millisecond range or below. In these cases a machine can query another source such as one based on Network Time Protocol (NTP).

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [3]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [4]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [5]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [9]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [10]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [11]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping: either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate to the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, otherwise known as drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when they are resumed from a suspended state, or restored from a snapshot. It is easy to reason that, because the guest's clock starts off correctly, it will always be correct from then on. Unfortunately, this is not the case. The first source of drift is simply due to electronics. A clock is almost never entirely accurate, having a slight error due to the effects of ambient temperature on oscillator frequency, even on the host system. Since the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time, the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that it is treated like a process by the host. In and of itself this does not seem like a problem, but the guest system can be suspended just before it tries to update its perception of time. With restricted CPU time, it is easy for the lost ticks to pile up and create a backlog. Even if the guest is checking over the network with a time server, its network conversation can be suspended before the answer comes back. When the process resumes the guest has a wildly inaccurate perception of the elapsed time (known as Round-Trip Time or RTT) and its incorrect adjustments for the network delay will throw off the clock. Problems can also come from memory swaps performed by the host. If the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

Clearly the errors in virtual machine timekeeping come from algorithms that were simply not designed for virtual environments. They assume a more stable physical world than they have.

===Impact of Drift===

The sources of drift essentially boil down to round-off errors and lost ticks. But the practical impact of drift is quite apparent in any automated system. For a real-world analogy consider a factory's assembly line, where the machinery is finely tuned to do its own specific part at certain intervals, and generally does this with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt.

In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

The impacts of drift are even more apparent when virtual machines must communicate with one another. Consider the case of a farm of distributed gaming servers, where differences in timing can mean virtual characters suddenly warp at super-human speed across the landscape. Or for a more serious example consider investment trading, where missing the moment to bid can mean a difference of millions of dollars.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [6]
For demanding applications, this is neither robust nor reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduce the offset error. Unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. Thus, it would not work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[7] Such clocks are unsuitable for applications that deal with network management and measurements. The reason for this is that NTP focuses on the offset and not on hardware clock's oscillator rate. For example, when calculating delay variations, the offset error does not change anything in the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved but when it is enabled again on the new system problems occur: because no two hardware clocks drift the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This consequences could prove disastrous to the system. These could range anywhere from slowly recoverable errors to ones where the ntpd might never recover and where the virtualized OS could become unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked environments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms confers advantages in VM migrations since a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information. The RADclock implementation makes use of this type of feed-forward design.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem. Of course, to be fair to these academics, their paper wasn't written for students. But the problem can be summed up succinctly: If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so find the fastest way there.

=References=

1. Broomhead, Cremean, Ridoux, Veitch. "Virtualize Everything But Time" . 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. Web. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf Accessed: Nov. 2010.

3. "Bran's Kernel Development Tutorial" from ''Bona Fide OS Developer'' . Web. http://www.osdever.net/bkerndev/Docs/pit.htm . Accessed: Nov. 2010.

4. "What is a CMOS battery, and why does my computer need one?" ''Indiana University's Knowledge Base'', 2010. Web. http://kb.iu.edu/data/adoy.html . Accessed: Nov. 2010.

5. "Multiprocessor Specification version 1.4". ''Intel''. May 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf .

6. Pasztor and Veitch. "PC Based Precision Timing Without GPS". 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf .

7. Veitch, Ridoux, Korada. "Robust Synchronization of Absolute and Difference Clocks over Networks". 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

9. "Advanced Configuration and Power Interface Specification". ''Intel Corporation''. April 2010. http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf.

10. "Game timing and Multicore Processors". ''msdn'' . Dec 2005. Web. http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx . Accessed Nov. 2010.

11. "IA-PC HPET (High Precision Event Timers) Specification. ''Intel Corporation''. Oct. 2004. http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf .

COMP 3000 Essay 2 2010 Question 11

2010-12-02T07:05:30Z

Jjpwilso: /* Background Concepts */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running the whole time. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts behind it. For example, we all know what clocks are in our day-to-day lives but how are they different in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, explore the advantages and disadvantages of the different available counters and synchronization algorithms, and explain what a para-virtualized system is about.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf].

Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed.

In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts, however its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift (see below). But neither of these methods knows the actual time, they only know how long it has been since they last checked an authoritative source. Personal computers typically get their time from a battery-backed real-time clock (i.e. a CMOS clock). Networked machines often need a more precise time, with a resolution in the millisecond range or below. In these cases a machine can query another source such as one based on Network Time Protocol (NTP).

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to electronics. A clock is almost never entirely accurate, having a slight error due to the effects of ambient temperature on oscillator frequency, even on the host system. Since the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but the guest system can be suspended just before it tries to update its perception of time. With restricted CPU time, it's easy for the lost ticks to pile up and create a backlog. Even if the guest is checking over the network with a time server, its network conversation can be suspended before the answer comes back. When the process resumes the guest has a wildly inaccurate perception of the elapsed time (known as Round-Trip Time or RTT) and its incorrect adjustments for the network delay will throw off the clock. Problems can also come from memory swaps performed by the host. If the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

Clearly the errors in virtual machine timekeeping come from algorithms that were simply not designed for virtual environments. They assume a more stable physical world than they have.

===Impact of Drift===

The sources of drift essentially boil down to round-off errors and lost ticks. But the practical impact of drift is quite apparent in any automated system. For a real-world analogy consider a factory's assembly line, where the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt.

In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

The impacts of drift are even more apparent when virtual machines must communicate with one another. Consider the case of a farm of distributed gaming servers, where differences in timing can mean virtual characters suddenly warp at super-human speed across the landscape. Or for a more serious example consider investment trading, where missing the moment to bid can mean a difference of millions of dollars.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem. Of course, to be fair to these academics, their paper wasn't written for students. But the problem can be summed up succinctly: If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so find the fastest way there.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T07:03:58Z

Jjpwilso: /* Sources of Drift */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts behind it. For example, we all know what clocks are in our day-to-day lives but how are they different in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, explore the advantages and disadvantages of the different available counters and synchronization algorithms, and explain what a para-virtualized system is about.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf].

Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed.

In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts, however its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift (see below). But neither of these methods knows the actual time, they only know how long it has been since they last checked an authoritative source. Personal computers typically get their time from a battery-backed real-time clock (i.e. a CMOS clock). Networked machines often need a more precise time, with a resolution in the millisecond range or below. In these cases a machine can query another source such as one based on Network Time Protocol (NTP).

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to electronics. A clock is almost never entirely accurate, having a slight error due to the effects of ambient temperature on oscillator frequency, even on the host system. Since the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but the guest system can be suspended just before it tries to update its perception of time. With restricted CPU time, it's easy for the lost ticks to pile up and create a backlog. Even if the guest is checking over the network with a time server, its network conversation can be suspended before the answer comes back. When the process resumes the guest has a wildly inaccurate perception of the elapsed time (known as Round-Trip Time or RTT) and its incorrect adjustments for the network delay will throw off the clock. Problems can also come from memory swaps performed by the host. If the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

Clearly the errors in virtual machine timekeeping come from algorithms that were simply not designed for virtual environments. They assume a more stable physical world than they have.

===Impact of Drift===

The sources of drift essentially boil down to round-off errors and lost ticks. But the practical impact of drift is quite apparent in any automated system. For a real-world analogy consider a factory's assembly line, where the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt.

In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

The impacts of drift are even more apparent when virtual machines must communicate with one another. Consider the case of a farm of distributed gaming servers, where differences in timing can mean virtual characters suddenly warp at super-human speed across the landscape. Or for a more serious example consider investment trading, where missing the moment to bid can mean a difference of millions of dollars.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem. Of course, to be fair to these academics, their paper wasn't written for students. But the problem can be summed up succinctly: If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so find the fastest way there.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T07:01:11Z

Jjpwilso: /* Critique */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts behind it. For example, we all know what clocks are in our day-to-day lives but how are they different in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, explore the advantages and disadvantages of the different available counters and synchronization algorithms, and explain what a para-virtualized system is about.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf].

Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed.

In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts, however its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift (see below). But neither of these methods knows the actual time, they only know how long it has been since they last checked an authoritative source. Personal computers typically get their time from a battery-backed real-time clock (i.e. a CMOS clock). Networked machines often need a more precise time, with a resolution in the millisecond range or below. In these cases a machine can query another source such as one based on Network Time Protocol (NTP).

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to electronics. A clock is almost never entirely accurate, having a slight error due to the effects of ambient temperature on oscillator frequency, even on the host system. Since the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but the guest system can be suspended just before it tried to update its perception of time. With restricted CPU time, it's easy for the lost ticks to pile up and create a backlog, or sometimes its network conversation to a time server is suspended before the answer comes back. When the process resumes, the guest system has a wildly inaccurate perception of the elapsed time (known as Round-Trip Time or RTT) and its incorrect adjustments for that time will throw off the clock. Problems can also come from memory swaps performed by the host. If the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

Clearly the errors in virtual machine timekeeping come from algorithms that were simply not designed for virtual environments. They assume a more stable physical world than they have.

===Impact of Drift===

The sources of drift essentially boil down to round-off errors and lost ticks. But the practical impact of drift is quite apparent in any automated system. For a real-world analogy consider a factory's assembly line, where the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt.

In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

The impacts of drift are even more apparent when virtual machines must communicate with one another. Consider the case of a farm of distributed gaming servers, where differences in timing can mean virtual characters suddenly warp at super-human speed across the landscape. Or for a more serious example consider investment trading, where missing the moment to bid can mean a difference of millions of dollars.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem. Of course, to be fair to these academics, their paper wasn't written for students. But the problem can be summed up succinctly: If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so find the fastest way there.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T06:58:25Z

Jjpwilso: /* Sources of Drift */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts behind it. For example, we all know what clocks are in our day-to-day lives but how are they different in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, explore the advantages and disadvantages of the different available counters and synchronization algorithms, and explain what a para-virtualized system is about.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf].

Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed.

In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts, however its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift (see below). But neither of these methods knows the actual time, they only know how long it has been since they last checked an authoritative source. Personal computers typically get their time from a battery-backed real-time clock (i.e. a CMOS clock). Networked machines often need a more precise time, with a resolution in the millisecond range or below. In these cases a machine can query another source such as one based on Network Time Protocol (NTP).

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to electronics. A clock is almost never entirely accurate, having a slight error due to the effects of ambient temperature on oscillator frequency, even on the host system. Since the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but the guest system can be suspended just before it tried to update its perception of time. With restricted CPU time, it's easy for the lost ticks to pile up and create a backlog, or sometimes its network conversation to a time server is suspended before the answer comes back. When the process resumes, the guest system has a wildly inaccurate perception of the elapsed time (known as Round-Trip Time or RTT) and its incorrect adjustments for that time will throw off the clock. Problems can also come from memory swaps performed by the host. If the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

Clearly the errors in virtual machine timekeeping come from algorithms that were simply not designed for virtual environments. They assume a more stable physical world than they have.

===Impact of Drift===

The sources of drift essentially boil down to round-off errors and lost ticks. But the practical impact of drift is quite apparent in any automated system. For a real-world analogy consider a factory's assembly line, where the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt.

In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

The impacts of drift are even more apparent when virtual machines must communicate with one another. Consider the case of a farm of distributed gaming servers, where differences in timing can mean virtual characters suddenly warp at super-human speed across the landscape. Or for a more serious example consider investment trading, where missing the moment to bid can mean a difference of millions of dollars.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem (to be fair, this wasn't written for students). If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so this seemed like an obvious answer.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T06:22:17Z

Jjpwilso: /* Timekeeping */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts behind it. For example, we all know what clocks are in our day-to-day lives but how are they different in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, explore the advantages and disadvantages of the different available counters and synchronization algorithms, and explain what a para-virtualized system is about.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf].

Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed.

In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts, however its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift (see below). But neither of these methods knows the actual time, they only know how long it has been since they last checked an authoritative source. Personal computers typically get their time from a battery-backed real-time clock (i.e. a CMOS clock). Networked machines often need a more precise time, with a resolution in the millisecond range or below. In these cases a machine can query another source such as one based on Network Time Protocol (NTP).

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the effects of ambient temperature on oscillator frequency, even on the host system. Since the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

But perhaps the largest source of errors in virtual timekeeping come from the fact that their algorithms were not designed for virtual environments. They assume a more stable physical world. Think of a Florida pickup on an icy road.

===Impact of Drift===

The sources of drift essentially boil down to round-off errors and lost ticks. But the practical impact of drift is quite apparent in any automated system. For a real-world analogy consider a factory's assembly line, where the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt.

In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

The impacts of drift are even more apparent when virtual machines must communicate with one another. Consider the case of a farm of distributed gaming servers, where differences in timing can mean virtual characters suddenly warp at super-human speed across the landscape. Or for a more serious example consider investment trading, where missing the moment to bid can mean a difference of millions of dollars.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem (to be fair, this wasn't written for students). If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so this seemed like an obvious answer.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T06:14:21Z

Jjpwilso: /* Background Concepts */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts behind it. For example, we all know what clocks are in our day-to-day lives but how are they different in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, explore the advantages and disadvantages of the different available counters and synchronization algorithms, and explain what a para-virtualized system is about.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the effects of ambient temperature on oscillator frequency, even on the host system. Since the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

But perhaps the largest source of errors in virtual timekeeping come from the fact that their algorithms were not designed for virtual environments. They assume a more stable physical world. Think of a Florida pickup on an icy road.

===Impact of Drift===

The sources of drift essentially boil down to round-off errors and lost ticks. But the practical impact of drift is quite apparent in any automated system. For a real-world analogy consider a factory's assembly line, where the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt.

In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

The impacts of drift are even more apparent when virtual machines must communicate with one another. Consider the case of a farm of distributed gaming servers, where differences in timing can mean virtual characters suddenly warp at super-human speed across the landscape. Or for a more serious example consider investment trading, where missing the moment to bid can mean a difference of millions of dollars.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem (to be fair, this wasn't written for students). If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so this seemed like an obvious answer.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T06:10:23Z

Jjpwilso: /* Impact of Drift */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the effects of ambient temperature on oscillator frequency, even on the host system. Since the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

But perhaps the largest source of errors in virtual timekeeping come from the fact that their algorithms were not designed for virtual environments. They assume a more stable physical world. Think of a Florida pickup on an icy road.

===Impact of Drift===

The sources of drift essentially boil down to round-off errors and lost ticks. But the practical impact of drift is quite apparent in any automated system. For a real-world analogy consider a factory's assembly line, where the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt.

In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

The impacts of drift are even more apparent when virtual machines must communicate with one another. Consider the case of a farm of distributed gaming servers, where differences in timing can mean virtual characters suddenly warp at super-human speed across the landscape. Or for a more serious example consider investment trading, where missing the moment to bid can mean a difference of millions of dollars.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem (to be fair, this wasn't written for students). If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so this seemed like an obvious answer.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T06:01:20Z

Jjpwilso: /* Sources of Drift */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the effects of ambient temperature on oscillator frequency, even on the host system. Since the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

But perhaps the largest source of errors in virtual timekeeping come from the fact that their algorithms were not designed for virtual environments. They assume a more stable physical world. Think of a Florida pickup on an icy road.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem (to be fair, this wasn't written for students). If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so this seemed like an obvious answer.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T05:59:57Z

Jjpwilso: /* Sources of Drift */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the effects of ambient temperature on oscillator frequency, even on the host system. Since the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

But perhaps the largest source of errors in virtual timekeeping come from the fact that their algorithms were not designed for virtual environments. They assume a more stable physical world. Their over-compensation can be like a Florida pickup on an icy road.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem (to be fair, this wasn't written for students). If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so this seemed like an obvious answer.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T05:56:39Z

Jjpwilso: /* Sources of Drift */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the effects of ambient temperature on oscillator frequency, even on the host system. Since the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

But perhaps the largest source of errors in virtual timekeeping come from the fact that their algorithms were not designed for virtual environments. They assume a more predictable physical world and so their compensations can be like oversteering on an icy road.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem (to be fair, this wasn't written for students). If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so this seemed like an obvious answer.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T05:45:44Z

Jjpwilso: /* Timekeeping */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS simply reads the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem (to be fair, this wasn't written for students). If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so this seemed like an obvious answer.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T05:44:25Z

Jjpwilso: /* Contribution */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls (''rdtsc()'' in [8]) they tried several clock sources, seeking to eliminate variability from power management and CPU load when setting ''raw'' timestamps for use in guest machines.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem (to be fair, this wasn't written for students). If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so this seemed like an obvious answer.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T05:25:29Z

Jjpwilso: /* Critique */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls they tried several clock sources, seeking to eliminate variability from power management and CPU load.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem (to be fair, this wasn't written for students). If you can't trust your own perception of time, ask the closest agent you ''can'' trust -- and make sure they've check their watch. There is no closer source to a VM than its host, so this seemed like an obvious answer.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T05:20:02Z

Jjpwilso: /* Critique */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls they tried several clock sources, seeking to eliminate variability from power management and CPU load.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=

It is difficult to critique the authors' work since they did a great job of finding a meaningful set of timestamps and counters and clearly demonstrated an advantage in their field of study. They also compared a number of time sources to ensure that their selection was meaningful and stable. And it's hard to argue with success. Their results approximate the variation that comes from CPU temperature. That's quite impressive.

As a student, one criticism might be that they found quite an obscure way of describing what was at heart a very simple problem. If you can't trust your own perception of time, ask the closest agent you ''can'' trust. There is no closer source to a VM than its host, so this seemed like an obvious answer. Also they skirted around the problem of drift in full virtualization, but at least there isn't a migration issue there.

=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T05:05:39Z

Jjpwilso: /* Contribution */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally explored to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets. Rather than use TSC calls they tried several clock sources, seeking to eliminate variability from power management and CPU load.

The paper makes several references to ''feed-forward'' and ''feedback'' mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTPd) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like ''sysfs'' or ''procfs'' that permits communications between virtual domains using shared memory. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=
=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T05:02:11Z

Jjpwilso: /* Contribution */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. It is industry-standard through its creators, Intel, Microsoft, Phoenix, Hewlett Packard and Toshiba. Its power management includes all forms: notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET defines a set of timers that the OS has access to and can assign to applications. Each timer can generate an interrupt when the least significant bits are equal to the equivalent bits of the 64-bit counter value. However, a race case can occur in which the target time has already passed. This causes more interrupts and more work even if the task is a simple one. It does produce less interrupts than its predecessors PIT and CMOS RTC giving it an edge. Despite its race condition, this modern timer is improvement upon old practices. [http://hackipedia.org/Hardware/HPET,%20High%20Performance%20Event%20Timer/IA-PC%20HPET%20%28High%20Precision%20Event%20Timers%29%20Specification.pdf]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

The authors of this timekeeping paper have done previous work exploring the feed-forward and feedback mechanisms for clock algorithm adjustment in non-virtual systems [7][8]. The RADclock algorithm (Robust Absolute Difference) was originally meant to address the drift resulting from NTP's feedback algorithm, and how non-ideal network conditions (a circumstance that is quite common) can have serious effects. In their original paper they improved network synchronization using the TimeStamp Counter (TSC) register a system call introduced in Pentium class machines as a source for a CPU cycle count. The use of a more reliable timestamp and counter provided "GPS-like" reliability in networked enviroments.

This new paper seeks to take a similar approach in a virtual machine setting where VM migration can cause much more severe disruption than simply lost UDP packets.

The paper makes several references to feed-forward and feedback mechanisms, and so a quick discussion of these control theories is in order. In feed-forward mechanisms, inputs to a process or calculation may be modified in advance, but the resulting output plays no part in subsequent calculations. In feedback mechanisms (such as NTP) inputs to a calculation can be modified by outputs from previous ones. As a result, feedback mechanisms carry state from one step to the next. Since the state of a virtual environment may be rendered inaccurate by so many sources a feedback mechanism as a bad idea. The statelessness of feed-forward mechanisms offers VM migrations a particular advantage since after migration a guest OS can simply discover the actual facts of timestamps rather than try to estimate them from their own invalid state information.

The mechanism the authors used in the Xen environment was the XenStore, a filesystem structure like sysfs or procfs that permits communications between virtual domains. Dom0 (Xen terminology for the hypervisor) takes sychronization from its own time server and saves the calculated clock parameters to the shared XenStore. On DomU (Xen for any guest OS) an application would simply use the shared information as a raw timestamp or a time difference. Their extensive testing showed a clear -- if not surprising -- advantage over NTPd.

=Critique=
=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T04:31:03Z

Jjpwilso: /* References */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. Power Management includes notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# RDTSC is based on the x86 P5 instruction set and perform high-resolution timing, however, it suffers from several flaws. Discontinuous values from the processor are caused as a result of not using the same thread to the processor each time, which can also be caused by having a multicore processor. This is made worse by ACPI which will eventually lead to the cores being completely out of sync. Availability of dedicated hardware: "RDTSC locks the timing information that the application requests to the processor's cycle counter." With dedicated timing devices included on modern motherboards this method of locking the timing information will become obsolete. Lastly, the variability of the CPU's frequency needs to be taken into account. With modern day laptops, most CPU frequencies are adjusted on the fly to respond to the users demand when needed and to lower themselves when idle, this results in longer battery life and less heat generated by the laptop but regretfully affects RDTSC making it unreliable. [http://msdn.microsoft.com/en-us/library/ee417693%28VS.85%29.aspx]
# HPET (I will fill these in later, Dec. 1st Update) --[[User:Spanke|Spanke]]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

(sections are stubs for the moment ... more to come)
The contributions of this paper were:

* baseline evaluations of:
** performance of NTP in dependent and independent configurations
** Xen Clocksource as a basis counter under NTP
** latencies of different clock sources
** implications of Power Management

* new architecture
** RADclock
** XenStore as holder of clock parameter data
** feed-forward versus feedback

* evaluation of RADclock vs ntpd

=Critique=
=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

8. Broomhead, T.; Ridoux, J.; Veitch, D.; , "Counter availability and characteristics for feed-forward based synchronization," Precision Clock Synchronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1-6, 12-16 Oct. 2009

COMP 3000 Essay 2 2010 Question 11

2010-12-02T03:48:03Z

Jjpwilso: /* Contribution */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

The next time you notice one stranger ask another for the time and you see them check their watch, try this experiment: immediately ask too. Chances are the person will check their watch again. Why? Human internal clocks are notoriously unreliable. Our sense of time contracts and expands all day long. We seem to believe that a definitive report of time can only come from some mechanical or electronic source. So social norms require that the watch owner provides you with two things: 1) the time, and 2) a gesture of external authority, i.e. a glance at their watch.

The story of time inside a virtual machine is almost as unreliable as our own internal clocks. How much time has elapsed since a VM client got the CPU's attention? At the best of times there's no way for it to guess because it wasn't actually running. If the VM was suspended and migrated from one physical host to another its concept of time is even worse. This paper is about how a computer glances at its metaphorical watch, and what kinds of timepieces it has at hand.

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

For thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represents how tick counting works. The oscillator runs at a predetermined frequency. The operating system might have to measure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. Not all hardware timers work exactly like that. For instance, some actually count up, others don't use interrupts, and yet others don't keep an initial counter. The general principle of hardware counters is the however the same. There is some kind of fixed interval at the end of which the current time is updated by an appropriate number of units (i.e. nanoseconds).

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI establishes industry-standard interfaces configuration guided by the OS and power management. Power Management includes notebooks, desktops, and servers. ACPI's goal is to improve current power and configuration standards for hardware devices by transitioning to ACPI-compliant hardware. This allows the OS as well as the VM to have control over power management. [http://www.intel.com/technology/iapc/acpi/][http://www.acpi.info/][http://www.acpi.info/DOWNLOADS/ACPIspec40a.pdf]
# TSC
# HPET (I will fill these in later, Dec. 1st Update) --[[User:Spanke|Spanke]]

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=

(sections are stubs for the moment ... more to come)
The contributions of this paper were:

* baseline evaluations of:
** performance of NTP in dependent and independent configurations
** Xen Clocksource as a basis counter under NTP
** latencies of different clock sources
** implications of Power Management

* new architecture
** RADclock
** XenStore as holder of clock parameter data
** feed-forward versus feedback

* evaluation of RADclock vs ntpd

=Critique=
=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

COMP 3000 Essay 2 2010 Question 11

2010-12-02T03:11:31Z

Jjpwilso: /* Contribution */

COMP 3000 Essay 2 2010 Question 11

2010-12-02T01:15:05Z

Jjpwilso: stubs for contribution section

COMP 3000 Essay 2 2010 Question 11

2010-12-02T00:40:57Z

Jjpwilso: minor spelling edit

COMP 3000 Essay 2 2010 Question 11

2010-12-02T00:10:35Z

Jjpwilso: /* Background Concepts */

COMP 3000 Essay 2 2010 Question 11

2010-12-02T00:09:46Z

Jjpwilso: /* Background Concepts */

Talk:COMP 3000 Essay 2 2010 Question 11

2010-12-02T00:03:38Z

Jjpwilso:

--[[User:Jjpwilso|Jjpwilso]] 00:03, 2 December 2010 (UTC) I'll check for some more references on the ACM and IEEE databases. In the meantime I thought I'd mention what Anil said regarding critique. He suggested we should consider other approaches to the same solution, such as modifying NTP with a different heuristic. I'll see what I can dig up in other papers on NTP.

--[[User:ScottG|ScottG]] 22:06, 1 December 2010 (UTC) I'm assuming you meant for me to add my references, yes? I really only used the article, and 'Timekeeping in Virtual Machines' which I went to add, but is already on there. I've looked for other articles to try to get how others have looked at it that aren't VMware, but there really isn't a huge amount out there dealing ''specifically'' with guest timekeeping (unless I've gone Google-blind, which has admittedly happened before). Mostly I ran into links pointing to that specific article.

--[[User:Sblais2|Sblais2]] 17:50, 1 December 2010 (UTC) I added stuf into the Research problems. I think I summarized most of them. If I forgot any, please add them in. I also added the missing references in the reference section. For Fedor, we seem to miss some content in 2 sections. Also, you could read through the other section and add/change some pertinent information that might've been missed that would make this essay even better.

--[[User:Sblais2|Sblais2]] 15:11, 1 December 2010 (UTC) Would it be possible to add your references at the bottom please? Even if it is a link. I have added the article link at the top of the essay.

Hey guys, sorry its taken me a while to post here. If there is a particular topic that needs researching, I could spend some hours doing that tomorrow - suggestions? Also, I intend to fix up the style&structure after everything is done as I am quite good with that.

Fedor

--[[User:ScottG|ScottG]] 21:36, 26 November 2010 (UTC) So I was a little (more than a little) behind on my initially estimated time for getting stuff up on Guest Timekeeping, but that's the gist of it there now. I'm going to try to buff it up a bit before it's due, since what I put in is a bit rougher than I'd like. If I seem to be missing something that should be pretty obvious, let me know and I'll work it in.

--[[User:Jjpwilso|Jjpwilso]] 15:49, 23 November 2010 (UTC) I've been completely swamped with COMP3004 stuff (among other things) and feeling guilty as hell about this essay. The good news, for those who might have missed today's lecture, is we have an extension of one week. Phew!!

--[[User:Sblais2|Sblais2]] 21:29, 22 November 2010 (UTC) I have added a small part to the background section. I have created by hand a diagram explaining how it works. I tried to find an original way of doing it but it is the same diagram everywhere. Please feel free to comment here or by sending me an email.

--[[User:AbsMechanik|AbsMechanik]] 19:46, 22 November 2010 (UTC) Here's what my research has led me to so far. I'm trying to come up with good points for the research problem, contribution and critique part of this essay. Here's a bunch of links, I've come across. I think there will be a few more tonight. Feel free to read through 'em:
 http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
 http://www.xen.org/files/xen_interface.pdf
 http://www.microsoft.com/whdc/system/sysinternals/mm-timer.mspx
 http://www.intel.com/hardwaredesign/hpetspec_1.pdf
 http://www.cubinlab.ee.unimelb.edu.au/radclock/

--[[User:ScottG|ScottG]] 18:55, 22 November 2010 (UTC) I'm good taking the Guest Timekeeping section. Hopefully I'll have some stuff up tonight or early tomorrow for it.

--[[User:Sblais2|Sblais2]] 17:14, 22 November 2010 (UTC) I will be working on the Background section. I will dedicate it to explain some of the key concepts that are used in the research paper that will allow the readers to have a better understanding on the rest of our essay. The structure you've put in place looks good but it might get modified, depending on the text will flow. The diagram is a good idea. I will drawn a simple one and add it in. Feel free again to critique.

--[[User:Jjpwilso|Jjpwilso]] 15:12, 16 November 2010 (UTC) I wanted to get a structure started, so I have stubbed out the first section. Note: some of the sub-sections might belong in the Research Problem section but we can easily move them if they fit there. Let's use this area to plan who is doing what. Feel free to critique any of my submissions. When you comment here, please put your comments at the very top so we can easily see recent posts.

=Participants=
(X) Blais Sylvain sblais2 - Email: syl20blais@gmail.com 
(X) Graham Scott sgraham6 
(X) Ilitchev Fedor filitche fedor dot ilitchev at gmail dot com 
(X) Panke Shane spanke 
(X) Shukla Abhinav ashukla2 
(X) Wilson Robert jjpwilso

COMP 3000 Essay 2 2010 Question 11

2010-12-01T23:53:41Z

Jjpwilso: /* Background Concepts */

Talk:COMP 3000 Essay 2 2010 Question 11

2010-11-23T15:49:01Z

Jjpwilso:

--[[User:Jjpwilso|Jjpwilso]] 15:49, 23 November 2010 (UTC) I've been completely swamped with COMP3004 stuff (among other things) and feeling guilty as hell about this essay. The good news, for those who might have missed today's lecture, is we have an extension of one week. Phew!!

--[[User:Sblais2|Sblais2]] 21:29, 22 November 2010 (UTC) I have added a small part to the background section. I have created by hand a diagram explaining how it works. I tried to find an original way of doing it but it is the same diagram everywhere. Please feel free to comment here or by sending me an email.

--[[User:AbsMechanik|AbsMechanik]] 19:46, 22 November 2010 (UTC) Here's what my research has led me to so far. I'm trying to come up with good points for the research problem, contribution and critique part of this essay. Here's a bunch of links, I've come across. I think there will be a few more tonight. Feel free to read through 'em:
 http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
 http://www.xen.org/files/xen_interface.pdf
 http://www.microsoft.com/whdc/system/sysinternals/mm-timer.mspx
 http://www.intel.com/hardwaredesign/hpetspec_1.pdf
 http://www.cubinlab.ee.unimelb.edu.au/radclock/

--[[User:ScottG|ScottG]] 18:55, 22 November 2010 (UTC) I'm good taking the Guest Timekeeping section. Hopefully I'll have some stuff up tonight or early tomorrow for it.

--[[User:Sblais2|Sblais2]] 17:14, 22 November 2010 (UTC) I will be working on the Background section. I will dedicate it to explain some of the key concepts that are used in the research paper that will allow the readers to have a better understanding on the rest of our essay. The structure you've put in place looks good but it might get modified, depending on the text will flow. The diagram is a good idea. I will drawn a simple one and add it in. Feel free again to critique.

--[[User:Jjpwilso|Jjpwilso]] 15:12, 16 November 2010 (UTC) I wanted to get a structure started, so I have stubbed out the first section. Note: some of the sub-sections might belong in the Research Problem section but we can easily move them if they fit there. Let's use this area to plan who is doing what. Feel free to critique any of my submissions. When you comment here, please put your comments at the very top so we can easily see recent posts.

=Participants=
(X) Blais Sylvain sblais2 - Email: syl20blais@gmail.com 
(X) Graham Scott sgraham6 
(X) Ilitchev Fedor filitche fedor dot ilitchev at gmail dot com 
(X) Panke Shane spanke 
(X) Shukla Abhinav ashukla2 
(X) Wilson Robert jjpwilso

Talk:COMP 3000 Essay 2 2010 Question 11

2010-11-16T15:12:19Z

Jjpwilso: first section stubs

--[[User:Jjpwilso|Jjpwilso]] 15:12, 16 November 2010 (UTC) I wanted to get a structure started, so I have stubbed out the first section. Note: some of the sub-sections might belong in the Research Problem section but we can easily move them if they fit there. Let's use this area to plan who is doing what. Feel free to critique any of my submissions. When you comment here, please put your comments at the very top so we can easily see recent posts.

=Participants=
(X) Blais Sylvain sblais2 - Email: syl20blais@gmail.com 
(X) Graham Scott sgraham6 
(X) Ilitchev Fedor filitche fedor dot ilitchev at gmail dot com 
(X) Panke Shane spanke 
(X) Shukla Abhinav ashukla2 
(X) Wilson Robert jjpwilso

COMP 3000 Essay 2 2010 Question 11

2010-11-16T15:04:41Z

Jjpwilso: proposed background concept sections

=Background Concepts=

In this section we will discuss the fundamentals of timekeeping in real and virtual machines.

==Host Timekeeping==
This section will provide a brief overview of timekeeping in stand-alone or host PCs/servers.
#todo: add a diagram of a simplified timer showing oscillator, counter, counter input, and interrupt
#todo: explain the parts of the diagram

===Counting Strategies===
# tick counting
# tickless counting

===Timers===
# PIT
# CMOS RTC
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

COMP 3000 Essay 2 2010 Question 11

2010-11-14T05:06:32Z

Jjpwilso: structure only

=Background Concepts=
=Research problem=
=Contribution=
=Critique=
=References=

Talk:COMP 3000 Essay 2 2010 Question 11

2010-11-11T18:38:00Z

Jjpwilso: Team Ping

Please mark an X if you are able to participate.

( ) Blais Sylvain sblais2 
( ) Graham Scott sgraham6 
( ) Ilitchev Fedor filitche 
( ) Panke Shane spanke 
( ) Shukla Abhinav ashukla2 
(X) Wilson Robert jjpwilso

COMP 3000 Midterm exam 2010

2010-10-31T13:34:21Z

Jjpwilso: clarity

==Question 1==
Q: On an x86-class processor that supports hardware virtualization, it is possible in principle to virtualize any OS that can run on the native hardware. In practice, only certain operating system types and versions will run well on a given VM platform (e.g., VirtualBox, VMWare Player). What is one reason for this discrepancy?

A:

===Comments===

==Question 2==
Q: Typically, what does a guest OS's hard disk look like to the host OS?

A: The host OS typically sees one big file for a guest OS hard disk. Some VM software (e.g. VMWare) lets you break up this file into segments for performance and/or administrative reasons (e.g. the file size exceeds space on one volume), but this has no direct impact on the guest filesystem.

===Comments===

==Question 3==
Q: The execve system call gets three arguments: a full pathname to an executable, arguments, and environment variables. What are each of these for when "chmod a+r foo.txt" is run?

A:

''Path to executable:'' <code>/bin/chmod</code> (partly derived ... see note below) 
''Arguments:'' <code>a+r</code> <code>foo.txt</code> 
''Environment:'' (not shown ... depends on the shell)

Note: the full path to the executable is derived by the shell when the command is processed. It is based on the PATH environment variable.

===Comments===

==Question 4==
Q: Why doesn't <code>ldd</code> report all the libraries that are listed in /proc/<PID>/maps?

A: The <code>ldd</code> command only lists libraries that were statically linked at compile time. The proc mapping lists those and the ones linked dynamically at runtime.

===Comments===

==Question 5==
Q: When a process becomes undead (a zombie), who can send it to its grave permanently? What must it do?

A: The ancestor of all programs is <code>init</code> (PID = 1), and its job is to kill off any zombie children not properly killed by their parents before their own deaths. Gruesome stuff.

===Comments===

==Question 6==
Q: UNIX files each have permissions for three classes of users. What are those classes?

A: Owner, Group, Others. A file is typically created with owner set to the user running the program and the group set to that user's primary group. The value of the specific permissions (i.e. _rw_rw_r__) is determined at file creation by prior use of the <code>umask</code> command. Permissions can be changed after file creation with the <code>chown</code> command.

===Comments===

==Question 7==
Q: A solution to the producer/consumer problem must take into account many conditions. What are three of them?

A:

===Comments===

==Question 8==
Q: What are three events that cause signals to be sent in UNIX?

A:

===Comments===

==Question 9==
Q: What system call do you use to receive data over an already existing pipe?

A: The <code>read</code> call receives data in a pipe. Note: this will block if the pipe is empty. When a sender issues the <code>write</code> command the receiver will unblock and receive a data from the pipe.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:28, 27 October 2010 (UTC) If I recall correctly, the amount of data read depends on: 1) a record separator such as a line-feed (possibly based on IFS?) or 2) the length of PIPEBUF as set during OS compilation.

==Question 10==
Q: How is an exokernel library OS like a guest OS running on a VM? How are they different?

A:

===Comments===

==Question 11==
Q: What is one reason the UNIX system call interface has had to change in response to computer architectural changes?

A:

===Comments===

==Question 12==
Q: What is one reason why mainframes support near perfect hardware virtualization of CPU, memory, and all I/O devices?

A:

===Comments===

==Question 13==
Q: How can a cluster have higher overall reliability than the individual computers that it consists of?

A:

===Comments===

==Question 14==
Q: What was a key advantage of the Lisp machine OS, Genera, over current operating systems? What was a key disadvantage?

A:

===Comments===

==Question 15==
Q: Why did kernel extensions in SPIN need to be written in a "safe" language? Why don't kernel extensions in Windows and Linux (drivers, modules) need to be written in a safe language?

A:

===Comments===

==Question 16==
Q: What is an important reason why modern schedulers, like the ULE and recent Linux schedulers, require a scheduling algorithm which operates in sublinear time? Correspondingly, why was a linear-time scheduler okay for the original UNIX? Explain briefly.

A:

===Comments===

==Question 17==
Q: Describe a race condition error and a test that would catch that error in testing. (You may describe a ''well-known'' race condition or one that you make up.)

A:

===Comments===

==Question 18==
Q: Can a threading library like pthreads be implemented on a system that has no native support for threads, such as the original UNIX? What is required for such an implementation?

A:

===Comments===

==Question 19==
Q: Despite their performance disadvantage, what is one reason kernel space thread schedulers are generally favored over userspace implementations?

A:

===Comments===

==Question 20==
Q: Linux's logical volume manager (LVM) allows for the creation of synthetic block devices that span physical disks, e.g., the blocks in two hard disks can be appended or striped so they appear to be one large hard disk. LVM, however, implements no file system abstraction---the unified block device must be formatted with a regular filesystem. What is one way LVM plus ext4 (a standard Linux filesystem) is similar to ZFS? What is one way in which they are different?

A:

===Comments===

==Question 21==
Q: Hard disk-based filesystems must design around the slow seek times of hard disks (relative to their sequential transfer speed). What characteristic(s) of flash (solid state) storage must flash-based filesystems design around to achieve high performance and reliability?

A:

===Comments===

==Question 22==
Q: Storage area networks (SANs) allow many computers to share access to the same storage devices via a special trusted network. Standard SAN access protocols work in terms of blocks. By reading from multiple devices concurrently, SANs can provide impressive I/O performance. Given that SANs already are extremely high performance, why are many researchers arguing that they should transition to object-based stores?

A:

===Comments===

==Question 23==
Q: What key properties of a database are developers trying to capture with database-filesystem hybrids?

A:

===Comments===

==Question 24==
Q: Why aren't standard databases a drop-in replacement for filesystems?

A:

===Comments===

COMP 3000 Midterm exam 2010

2010-10-27T14:53:33Z

Jjpwilso: formatting

==Question 1==
Q: On an x86-class processor that supports hardware virtualization, it is possible in principle to virtualize any OS that can run on the native hardware. In practice, only certain operating system types and versions will run well on a given VM platform (e.g., VirtualBox, VMWare Player). What is one reason for this discrepancy?

A:

===Comments===

==Question 2==
Q: Typically, what does a guest OS's hard disk look like to the host OS?

A: The host OS typically sees one big file for a guest OS hard disk. Some VM software (e.g. VMWare) lets you break up this file into segments for performance and/or administrative reasons (e.g. the file size exceeds space on one volume), but this has no direct impact on the guest filesystem.

===Comments===

==Question 3==
Q: The execve system call gets three arguments: a full pathname to an executable, arguments, and environment variables. What are each of these for when "chmod a+r foo.txt" is run?

A:

''Path to executable:'' <code>/bin/chmod</code> (partly derived ... see note below) 
''Arguments:'' <code>a+r</code> <code>foo.txt</code> 
''Environment:'' (not shown ... depends on the shell)

Note: the full path to the executable is derived by the shell when the command is processed. It is based on the PATH environment variable.

===Comments===

==Question 4==
Q: Why doesn't <code>ldd</code> report all the libraries that are listed in /proc/$<$PID$>$/maps?

A: The <code>ldd</code> command only lists libraries that were statically linked at compile time. The proc mapping lists those and the ones linked dynamically at runtime.

===Comments===

==Question 5==
Q: When a process becomes undead (a zombie), who can send it to its grave permanently? What must it do?

A: The ancestor of all programs is <code>init</code> (PID = 1), and its job is to kill off any zombie children not properly killed by their parents before their own deaths. Gruesome stuff.

===Comments===

==Question 6==
Q: UNIX files each have permissions for three classes of users. What are they?

A: Owner, Group, Others. A file is typically created with owner set to the user running the program and the group set to that user's primary group. The value of the permissions (i.e. _rw_rw_r__) is set by the <code>umask</code> command. They can be changed after file creation with the <code>chown</code> command.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:19, 27 October 2010 (UTC) This question was truncated from what we had on the midterm. I tacked on "What are they?" Was there something else here?

==Question 7==
Q: A solution to the producer/consumer problem must take into account many conditions. What are three of them?

A:

===Comments===

==Question 8==
Q: What are three events that cause signals to be sent in UNIX?

A:

===Comments===

==Question 9==
Q: What system call do you use to receive data over an already existing pipe?

A: The <code>read</code> call receives data in a pipe. Note: this will block if the pipe is empty. When a sender issues the <code>write</code> command the receiver will unblock and receive a data from the pipe.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:28, 27 October 2010 (UTC) If I recall correctly, the amount of data read depends on: 1) a record separator such as a line-feed (possibly based on IFS?) or 2) the length of PIPEBUF as set during OS compilation.

==Question 10==
Q: How is an exokernel library OS like a guest OS running on a VM? How are they different?

A:

===Comments===

==Question 11==
Q: What is one reason the UNIX system call interface has had to change in response to computer architectural changes?

A:

===Comments===

==Question 12==
Q: What is one reason why mainframes support near perfect hardware virtualization of CPU, memory, and all I/O devices?

A:

===Comments===

==Question 13==
Q: How can a cluster have higher overall reliability than the individual computers that it consists of?

A:

===Comments===

==Question 14==
Q: What was a key advantage of the Lisp machine OS, Genera, over current operating systems? What was a key disadvantage?

A:

===Comments===

==Question 15==
Q: Why did kernel extensions in SPIN need to be written in a "safe" language? Why don't kernel extensions in Windows and Linux (drivers, modules) need to be written in a safe language?

A:

===Comments===

==Question 16==
Q: What is an important reason why modern schedulers, like the ULE and recent Linux schedulers, require a scheduling algorithm which operates in sublinear time? Correspondingly, why was a linear-time scheduler okay for the original UNIX? Explain briefly.

A:

===Comments===

==Question 17==
Q: Describe a race condition error and a test that would catch that error in testing. (You may describe a ''well-known'' race condition or one that you make up.)

A:

===Comments===

==Question 18==
Q: Can a threading library like pthreads be implemented on a system that has no native support for threads, such as the original UNIX? What is required for such an implementation?

A:

===Comments===

==Question 19==
Q: Despite their performance disadvantage, what is one reason kernel space thread schedulers are generally favored over userspace implementations?

A:

===Comments===

==Question 20==
Q: Linux's logical volume manager (LVM) allows for the creation of synthetic block devices that span physical disks, e.g., the blocks in two hard disks can be appended or striped so they appear to be one large hard disk. LVM, however, implements no file system abstraction---the unified block device must be formatted with a regular filesystem. What is one way LVM plus ext4 (a standard Linux filesystem) is similar to ZFS? What is one way in which they are different?

A:

===Comments===

==Question 21==
Q: Hard disk-based filesystems must design around the slow seek times of hard disks (relative to their sequential transfer speed). What characteristic(s) of flash (solid state) storage must flash-based filesystems design around to achieve high performance and reliability?

A:

===Comments===

==Question 22==
Q: Storage area networks (SANs) allow many computers to share access to the same storage devices via a special trusted network. Standard SAN access protocols work in terms of blocks. By reading from multiple devices concurrently, SANs can provide impressive I/O performance. Given that SANs already are extremely high performance, why are many researchers arguing that they should transition to object-based stores?

A:

===Comments===

==Question 23==
Q: What key properties of a database are developers trying to capture with database-filesystem hybrids?

A:

===Comments===

==Question 24==
Q: Why aren't standard databases a drop-in replacement for filesystems?

A:

===Comments===

COMP 3000 Midterm exam 2010

2010-10-27T14:50:11Z

Jjpwilso: /* Question 2 */

==Question 1==
Q: On an x86-class processor that supports hardware virtualization, it is possible in principle to virtualize any OS that can run on the native hardware. In practice, only certain operating system types and versions will run well on a given VM platform (e.g., VirtualBox, VMWare Player). What is one reason for this discrepancy?

A:

===Comments===

==Question 2==
Q: Typically, what does a guest OS's hard disk look like to the host OS?

A: The host OS typically sees one big file for a guest OS hard disk. Some VM software (e.g. VMWare) lets you break up this file into segments for performance and/or administrative reasons (e.g. the file size exceeds space on one volume), but this has no direct impact on the guest filesystem.

===Comments===

==Question 3==
Q: The execve system call gets three arguments: a full pathname to an executable, arguments, and environment variables. What are each of these for when "chmod a+r foo.txt" is run?

A:

''Path to executable:'' <code>/bin/chmod</code> (partly derived ... see note below) 
''Arguments:'' <code>a+r</code> <code>foo.txt</code> 
''Environment:'' (not shown ... depends on the shell)

Note: the full path to the executable is derived by the shell when the command is processed. It is based on the PATH environment variable.

===Comments===

==Question 4==
Q: Why doesn't <code>ldd</code> report all the libraries that are listed in /proc/$<$PID$>$/maps?

A: The <code>ldd</code> command only lists libraries that were statically linked at compile time. The proc mapping lists those and the ones linked dynamically at runtime.

===Comments===

==Question 5==
Q: When a process becomes undead (a zombie), who can send it to its grave permanently? What must it do?

A: The ancestor of all programs is <code>init</code> (PID = 1), and its job is to kill off any zombie children not properly killed by their parents before their own deaths. Gruesome stuff.

===Comments===

==Question 6==
Q: UNIX files each have permissions for three classes of users. What are they?

A: Owner, Group, Others. A file is typically created with owner set to the user running the program and the group set to that user's primary group. The value of the permissions (i.e. _rw_rw_r__) is set by the <code>umask</code> command. They can be changed after file creation with the <code>chown</code> command.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:19, 27 October 2010 (UTC) This question was truncated from what we had on the midterm. I tacked on "What are they?" Was there something else here?

==Question 7==
Q: A solution to the producer/consumer problem must take into account many conditions. What are three of them?

A:

===Comments===

==Question 8==
Q: What are three events that cause signals to be sent in UNIX?

A:

===Comments===

==Question 9==
Q: What system call do you use to receive data over an already existing pipe?

A: The <code>read</code> call receives data in a pipe. Note: this will block if the pipe is empty. When a sender issues the <code>write</code> command the receiver will unblock and receive a data from the pipe.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:28, 27 October 2010 (UTC) If I recall correctly, the amount of data read depends on: 1) a record separator such as a line-feed (possibly based on IFS?) or 2) the length of PIPEBUF as set during OS compilation.

==Question 10==
Q: How is an exokernel library OS like a guest OS running on a VM? How are they different?

A:

===Comments===

==Question 11==
Q: What is one reason the UNIX system call interface has had to change in response to computer architectural changes?

A:

===Comments===

==Question 12==
Q: What is one reason why mainframes support near perfect hardware virtualization of CPU, memory, and all I/O devices?

A:

===Comments===

==Question 13==
Q: How can a cluster have higher overall reliability than the individual computers that it consists of?

A:

===Comments===

==Question 14==
Q: What was a key advantage of the Lisp machine OS, Genera, over current operating systems? What was a key disadvantage?

A:

===Comments===

==Question 15==
Q: Why did kernel extensions in SPIN need to be written in a "safe" language? Why don't kernel extensions in Windows and Linux (drivers, modules) need to be written in a safe language?

A:

===Comments===

==Question 16==
Q: What is an important reason why modern schedulers, like the ULE and recent Linux schedulers, require a scheduling algorithm which operates in sublinear time? Correspondingly, why was a linear-time scheduler okay for the original UNIX? Explain briefly.

A:

===Comments===

==Question 17==
Q: Describe a race condition error and a test that would catch that error in testing. (You may describe a ``well-known'' race condition or one that you make up.)

A:

===Comments===

==Question 18==
Q: Can a threading library like pthreads be implemented on a system that has no native support for threads, such as the original UNIX? What is required for such an implementation?

A:

===Comments===

==Question 19==
Q: Despite their performance disadvantage, what is one reason kernel space thread schedulers are generally favored over userspace implementations?

A:

===Comments===

==Question 20==
Q: Linux's logical volume manager (LVM) allows for the creation of synthetic block devices that span physical disks, e.g., the blocks in two hard disks can be appended or striped so they appear to be one large hard disk. LVM, however, implements no file system abstraction---the unified block device must be formatted with a regular filesystem. What is one way LVM plus ext4 (a standard Linux filesystem) is similar to ZFS? What is one way in which they are different?

A:

===Comments===

==Question 21==
Q: Hard disk-based filesystems must design around the slow seek times of hard disks (relative to their sequential transfer speed). What characteristic(s) of flash (solid state) storage must flash-based filesystems design around to achieve high performance and reliability?

A:

===Comments===

==Question 22==
Q: Storage area networks (SANs) allow many computers to share access to the same storage devices via a special trusted network. Standard SAN access protocols work in terms of blocks. By reading from multiple devices concurrently, SANs can provide impressive I/O performance. Given that SANs already are extremely high performance, why are many researchers arguing that they should transition to object-based stores?

A:

===Comments===

==Question 23==
Q: What key properties of a database are developers trying to capture with database-filesystem hybrids?

A:

===Comments===

==Question 24==
Q: Why aren't standard databases a drop-in replacement for filesystems?

A:

===Comments===

COMP 3000 Midterm exam 2010

2010-10-27T14:43:22Z

Jjpwilso: /* Question 3 */

==Question 1==
Q: On an x86-class processor that supports hardware virtualization, it is possible in principle to virtualize any OS that can run on the native hardware. In practice, only certain operating system types and versions will run well on a given VM platform (e.g., VirtualBox, VMWare Player). What is one reason for this discrepancy?

A:

===Comments===

==Question 2==
Q: Typically, what does a guest OS's hard disk look like to the host OS?

A:

===Comments===

==Question 3==
Q: The execve system call gets three arguments: a full pathname to an executable, arguments, and environment variables. What are each of these for when "chmod a+r foo.txt" is run?

A:

''Path to executable:'' <code>/bin/chmod</code> (partly derived ... see note below) 
''Arguments:'' <code>a+r</code> <code>foo.txt</code> 
''Environment:'' (not shown ... depends on the shell)

Note: the full path to the executable is derived by the shell when the command is processed. It is based on the PATH environment variable.

===Comments===

==Question 4==
Q: Why doesn't <code>ldd</code> report all the libraries that are listed in /proc/$<$PID$>$/maps?

A: The <code>ldd</code> command only lists libraries that were statically linked at compile time. The proc mapping lists those and the ones linked dynamically at runtime.

===Comments===

==Question 5==
Q: When a process becomes undead (a zombie), who can send it to its grave permanently? What must it do?

A: The ancestor of all programs is <code>init</code> (PID = 1), and its job is to kill off any zombie children not properly killed by their parents before their own deaths. Gruesome stuff.

===Comments===

==Question 6==
Q: UNIX files each have permissions for three classes of users. What are they?

A: Owner, Group, Others. A file is typically created with owner set to the user running the program and the group set to that user's primary group. The value of the permissions (i.e. _rw_rw_r__) is set by the <code>umask</code> command. They can be changed after file creation with the <code>chown</code> command.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:19, 27 October 2010 (UTC) This question was truncated from what we had on the midterm. I tacked on "What are they?" Was there something else here?

==Question 7==
Q: A solution to the producer/consumer problem must take into account many conditions. What are three of them?

A:

===Comments===

==Question 8==
Q: What are three events that cause signals to be sent in UNIX?

A:

===Comments===

==Question 9==
Q: What system call do you use to receive data over an already existing pipe?

A: The <code>read</code> call receives data in a pipe. Note: this will block if the pipe is empty. When a sender issues the <code>write</code> command the receiver will unblock and receive a data from the pipe.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:28, 27 October 2010 (UTC) If I recall correctly, the amount of data read depends on: 1) a record separator such as a line-feed (possibly based on IFS?) or 2) the length of PIPEBUF as set during OS compilation.

==Question 10==
Q: How is an exokernel library OS like a guest OS running on a VM? How are they different?

A:

===Comments===

==Question 11==
Q: What is one reason the UNIX system call interface has had to change in response to computer architectural changes?

A:

===Comments===

==Question 12==
Q: What is one reason why mainframes support near perfect hardware virtualization of CPU, memory, and all I/O devices?

A:

===Comments===

==Question 13==
Q: How can a cluster have higher overall reliability than the individual computers that it consists of?

A:

===Comments===

==Question 14==
Q: What was a key advantage of the Lisp machine OS, Genera, over current operating systems? What was a key disadvantage?

A:

===Comments===

==Question 15==
Q: Why did kernel extensions in SPIN need to be written in a "safe" language? Why don't kernel extensions in Windows and Linux (drivers, modules) need to be written in a safe language?

A:

===Comments===

==Question 16==
Q: What is an important reason why modern schedulers, like the ULE and recent Linux schedulers, require a scheduling algorithm which operates in sublinear time? Correspondingly, why was a linear-time scheduler okay for the original UNIX? Explain briefly.

A:

===Comments===

==Question 17==
Q: Describe a race condition error and a test that would catch that error in testing. (You may describe a ``well-known'' race condition or one that you make up.)

A:

===Comments===

==Question 18==
Q: Can a threading library like pthreads be implemented on a system that has no native support for threads, such as the original UNIX? What is required for such an implementation?

A:

===Comments===

==Question 19==
Q: Despite their performance disadvantage, what is one reason kernel space thread schedulers are generally favored over userspace implementations?

A:

===Comments===

==Question 20==
Q: Linux's logical volume manager (LVM) allows for the creation of synthetic block devices that span physical disks, e.g., the blocks in two hard disks can be appended or striped so they appear to be one large hard disk. LVM, however, implements no file system abstraction---the unified block device must be formatted with a regular filesystem. What is one way LVM plus ext4 (a standard Linux filesystem) is similar to ZFS? What is one way in which they are different?

A:

===Comments===

==Question 21==
Q: Hard disk-based filesystems must design around the slow seek times of hard disks (relative to their sequential transfer speed). What characteristic(s) of flash (solid state) storage must flash-based filesystems design around to achieve high performance and reliability?

A:

===Comments===

==Question 22==
Q: Storage area networks (SANs) allow many computers to share access to the same storage devices via a special trusted network. Standard SAN access protocols work in terms of blocks. By reading from multiple devices concurrently, SANs can provide impressive I/O performance. Given that SANs already are extremely high performance, why are many researchers arguing that they should transition to object-based stores?

A:

===Comments===

==Question 23==
Q: What key properties of a database are developers trying to capture with database-filesystem hybrids?

A:

===Comments===

==Question 24==
Q: Why aren't standard databases a drop-in replacement for filesystems?

A:

===Comments===

COMP 3000 Midterm exam 2010

2010-10-27T14:34:15Z

Jjpwilso: clarity

==Question 1==
Q: On an x86-class processor that supports hardware virtualization, it is possible in principle to virtualize any OS that can run on the native hardware. In practice, only certain operating system types and versions will run well on a given VM platform (e.g., VirtualBox, VMWare Player). What is one reason for this discrepancy?

A:

===Comments===

==Question 2==
Q: Typically, what does a guest OS's hard disk look like to the host OS?

A:

===Comments===

==Question 3==
Q: The execve system call gets three arguments: a full pathname to an executable, arguments, and environment variables. What are each of these for when "chmod a+r foo.txt" is run?

A:

===Comments===

==Question 4==
Q: Why doesn't <code>ldd</code> report all the libraries that are listed in /proc/$<$PID$>$/maps?

A: The <code>ldd</code> command only lists libraries that were statically linked at compile time. The proc mapping lists those and the ones linked dynamically at runtime.

===Comments===

==Question 5==
Q: When a process becomes undead (a zombie), who can send it to its grave permanently? What must it do?

A: The ancestor of all programs is <code>init</code> (PID = 1), and its job is to kill off any zombie children not properly killed by their parents before their own deaths. Gruesome stuff.

===Comments===

==Question 6==
Q: UNIX files each have permissions for three classes of users. What are they?

A: Owner, Group, Others. A file is typically created with owner set to the user running the program and the group set to that user's primary group. The value of the permissions (i.e. _rw_rw_r__) is set by the <code>umask</code> command. They can be changed after file creation with the <code>chown</code> command.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:19, 27 October 2010 (UTC) This question was truncated from what we had on the midterm. I tacked on "What are they?" Was there something else here?

==Question 7==
Q: A solution to the producer/consumer problem must take into account many conditions. What are three of them?

A:

===Comments===

==Question 8==
Q: What are three events that cause signals to be sent in UNIX?

A:

===Comments===

==Question 9==
Q: What system call do you use to receive data over an already existing pipe?

A: The <code>read</code> call receives data in a pipe. Note: this will block if the pipe is empty. When a sender issues the <code>write</code> command the receiver will unblock and receive a data from the pipe.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:28, 27 October 2010 (UTC) If I recall correctly, the amount of data read depends on: 1) a record separator such as a line-feed (possibly based on IFS?) or 2) the length of PIPEBUF as set during OS compilation.

==Question 10==
Q: How is an exokernel library OS like a guest OS running on a VM? How are they different?

A:

===Comments===

==Question 11==
Q: What is one reason the UNIX system call interface has had to change in response to computer architectural changes?

A:

===Comments===

==Question 12==
Q: What is one reason why mainframes support near perfect hardware virtualization of CPU, memory, and all I/O devices?

A:

===Comments===

==Question 13==
Q: How can a cluster have higher overall reliability than the individual computers that it consists of?

A:

===Comments===

==Question 14==
Q: What was a key advantage of the Lisp machine OS, Genera, over current operating systems? What was a key disadvantage?

A:

===Comments===

==Question 15==
Q: Why did kernel extensions in SPIN need to be written in a "safe" language? Why don't kernel extensions in Windows and Linux (drivers, modules) need to be written in a safe language?

A:

===Comments===

==Question 16==
Q: What is an important reason why modern schedulers, like the ULE and recent Linux schedulers, require a scheduling algorithm which operates in sublinear time? Correspondingly, why was a linear-time scheduler okay for the original UNIX? Explain briefly.

A:

===Comments===

==Question 17==
Q: Describe a race condition error and a test that would catch that error in testing. (You may describe a ``well-known'' race condition or one that you make up.)

A:

===Comments===

==Question 18==
Q: Can a threading library like pthreads be implemented on a system that has no native support for threads, such as the original UNIX? What is required for such an implementation?

A:

===Comments===

==Question 19==
Q: Despite their performance disadvantage, what is one reason kernel space thread schedulers are generally favored over userspace implementations?

A:

===Comments===

==Question 20==
Q: Linux's logical volume manager (LVM) allows for the creation of synthetic block devices that span physical disks, e.g., the blocks in two hard disks can be appended or striped so they appear to be one large hard disk. LVM, however, implements no file system abstraction---the unified block device must be formatted with a regular filesystem. What is one way LVM plus ext4 (a standard Linux filesystem) is similar to ZFS? What is one way in which they are different?

A:

===Comments===

==Question 21==
Q: Hard disk-based filesystems must design around the slow seek times of hard disks (relative to their sequential transfer speed). What characteristic(s) of flash (solid state) storage must flash-based filesystems design around to achieve high performance and reliability?

A:

===Comments===

==Question 22==
Q: Storage area networks (SANs) allow many computers to share access to the same storage devices via a special trusted network. Standard SAN access protocols work in terms of blocks. By reading from multiple devices concurrently, SANs can provide impressive I/O performance. Given that SANs already are extremely high performance, why are many researchers arguing that they should transition to object-based stores?

A:

===Comments===

==Question 23==
Q: What key properties of a database are developers trying to capture with database-filesystem hybrids?

A:

===Comments===

==Question 24==
Q: Why aren't standard databases a drop-in replacement for filesystems?

A:

===Comments===

COMP 3000 Midterm exam 2010

2010-10-27T14:28:33Z

Jjpwilso: /* Question 9 */

==Question 1==
Q: On an x86-class processor that supports hardware virtualization, it is possible in principle to virtualize any OS that can run on the native hardware. In practice, only certain operating system types and versions will run well on a given VM platform (e.g., VirtualBox, VMWare Player). What is one reason for this discrepancy?

A:

===Comments===

==Question 2==
Q: Typically, what does a guest OS's hard disk look like to the host OS?

A:

===Comments===

==Question 3==
Q: The execve system call gets three arguments: a full pathname to an executable, arguments, and environment variables. What are each of these for when "chmod a+r foo.txt" is run?

A:

===Comments===

==Question 4==
Q: Why doesn't <code>ldd</code> report all the libraries that are listed in /proc/$<$PID$>$/maps?

A: The <code>ldd</code> command only lists libraries that were statically linked at compile time. The proc mapping lists those and the ones linked dynamically at runtime.

===Comments===

==Question 5==
Q: When a process becomes undead (a zombie), who can send it to its grave permanently? What must it do?

A: The ancestor of all programs is <code>init</code> (PID = 1), and its job is to kill off any zombie children not properly killed by their parents before their own deaths. Gruesome stuff.

===Comments===

==Question 6==
Q: UNIX files each have permissions for three classes of users. What are they?

A: Owner, Group, Others. A file is typically created with owner set to the user running the program and the group set to that user's primary group. The value of the permissions (i.e. _rw_rw_r__) is set by the <code>umask</code> command. They can be changed after file creation with the <code>chown</code> command.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:19, 27 October 2010 (UTC) This question was truncated from what we had on the midterm. I tacked on "What are they?" Was there something else here?

==Question 7==
Q: A solution to the producer/consumer problem must take into account many conditions. What are three of them?

A:

===Comments===

==Question 8==
Q: What are three events that cause signals to be sent in UNIX?

A:

===Comments===

==Question 9==
Q: What system call do you use to receive data over an already existing pipe?

A: The <code>read</code> call receives data in a pipe. Note: this will block if the pipe is empty. When a sender issues the <code>write</code> command the receiver will unblock and receive a data from the pipe.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:28, 27 October 2010 (UTC) If I recall correctly, the amount of data read depends on: 1) a record separator such as a line-feed (possibly based on IFS?) or 2) the length of PIPEBUF as set during compile.

==Question 10==
Q: How is an exokernel library OS like a guest OS running on a VM? How are they different?

A:

===Comments===

==Question 11==
Q: What is one reason the UNIX system call interface has had to change in response to computer architectural changes?

A:

===Comments===

==Question 12==
Q: What is one reason why mainframes support near perfect hardware virtualization of CPU, memory, and all I/O devices?

A:

===Comments===

==Question 13==
Q: How can a cluster have higher overall reliability than the individual computers that it consists of?

A:

===Comments===

==Question 14==
Q: What was a key advantage of the Lisp machine OS, Genera, over current operating systems? What was a key disadvantage?

A:

===Comments===

==Question 15==
Q: Why did kernel extensions in SPIN need to be written in a "safe" language? Why don't kernel extensions in Windows and Linux (drivers, modules) need to be written in a safe language?

A:

===Comments===

==Question 16==
Q: What is an important reason why modern schedulers, like the ULE and recent Linux schedulers, require a scheduling algorithm which operates in sublinear time? Correspondingly, why was a linear-time scheduler okay for the original UNIX? Explain briefly.

A:

===Comments===

==Question 17==
Q: Describe a race condition error and a test that would catch that error in testing. (You may describe a ``well-known'' race condition or one that you make up.)

A:

===Comments===

==Question 18==
Q: Can a threading library like pthreads be implemented on a system that has no native support for threads, such as the original UNIX? What is required for such an implementation?

A:

===Comments===

==Question 19==
Q: Despite their performance disadvantage, what is one reason kernel space thread schedulers are generally favored over userspace implementations?

A:

===Comments===

==Question 20==
Q: Linux's logical volume manager (LVM) allows for the creation of synthetic block devices that span physical disks, e.g., the blocks in two hard disks can be appended or striped so they appear to be one large hard disk. LVM, however, implements no file system abstraction---the unified block device must be formatted with a regular filesystem. What is one way LVM plus ext4 (a standard Linux filesystem) is similar to ZFS? What is one way in which they are different?

A:

===Comments===

==Question 21==
Q: Hard disk-based filesystems must design around the slow seek times of hard disks (relative to their sequential transfer speed). What characteristic(s) of flash (solid state) storage must flash-based filesystems design around to achieve high performance and reliability?

A:

===Comments===

==Question 22==
Q: Storage area networks (SANs) allow many computers to share access to the same storage devices via a special trusted network. Standard SAN access protocols work in terms of blocks. By reading from multiple devices concurrently, SANs can provide impressive I/O performance. Given that SANs already are extremely high performance, why are many researchers arguing that they should transition to object-based stores?

A:

===Comments===

==Question 23==
Q: What key properties of a database are developers trying to capture with database-filesystem hybrids?

A:

===Comments===

==Question 24==
Q: Why aren't standard databases a drop-in replacement for filesystems?

A:

===Comments===

COMP 3000 Midterm exam 2010

2010-10-27T14:22:54Z

Jjpwilso: /* Question 5 */

==Question 1==
Q: On an x86-class processor that supports hardware virtualization, it is possible in principle to virtualize any OS that can run on the native hardware. In practice, only certain operating system types and versions will run well on a given VM platform (e.g., VirtualBox, VMWare Player). What is one reason for this discrepancy?

A:

===Comments===

==Question 2==
Q: Typically, what does a guest OS's hard disk look like to the host OS?

A:

===Comments===

==Question 3==
Q: The execve system call gets three arguments: a full pathname to an executable, arguments, and environment variables. What are each of these for when "chmod a+r foo.txt" is run?

A:

===Comments===

==Question 4==
Q: Why doesn't <code>ldd</code> report all the libraries that are listed in /proc/$<$PID$>$/maps?

A: The <code>ldd</code> command only lists libraries that were statically linked at compile time. The proc mapping lists those and the ones linked dynamically at runtime.

===Comments===

==Question 5==
Q: When a process becomes undead (a zombie), who can send it to its grave permanently? What must it do?

A: The ancestor of all programs is <code>init</code> (PID = 1), and its job is to kill off any zombie children not properly killed by their parents before their own deaths. Gruesome stuff.

===Comments===

==Question 6==
Q: UNIX files each have permissions for three classes of users. What are they?

A: Owner, Group, Others. A file is typically created with owner set to the user running the program and the group set to that user's primary group. The value of the permissions (i.e. _rw_rw_r__) is set by the <code>umask</code> command. They can be changed after file creation with the <code>chown</code> command.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:19, 27 October 2010 (UTC) This question was truncated from what we had on the midterm. I tacked on "What are they?" Was there something else here?

==Question 7==
Q: A solution to the producer/consumer problem must take into account many conditions. What are three of them?

A:

===Comments===

==Question 8==
Q: What are three events that cause signals to be sent in UNIX?

A:

===Comments===

==Question 9==
Q: What system call do you use to receive data over an already existing pipe?

A:

===Comments===

==Question 10==
Q: How is an exokernel library OS like a guest OS running on a VM? How are they different?

A:

===Comments===

==Question 11==
Q: What is one reason the UNIX system call interface has had to change in response to computer architectural changes?

A:

===Comments===

==Question 12==
Q: What is one reason why mainframes support near perfect hardware virtualization of CPU, memory, and all I/O devices?

A:

===Comments===

==Question 13==
Q: How can a cluster have higher overall reliability than the individual computers that it consists of?

A:

===Comments===

==Question 14==
Q: What was a key advantage of the Lisp machine OS, Genera, over current operating systems? What was a key disadvantage?

A:

===Comments===

==Question 15==
Q: Why did kernel extensions in SPIN need to be written in a "safe" language? Why don't kernel extensions in Windows and Linux (drivers, modules) need to be written in a safe language?

A:

===Comments===

==Question 16==
Q: What is an important reason why modern schedulers, like the ULE and recent Linux schedulers, require a scheduling algorithm which operates in sublinear time? Correspondingly, why was a linear-time scheduler okay for the original UNIX? Explain briefly.

A:

===Comments===

==Question 17==
Q: Describe a race condition error and a test that would catch that error in testing. (You may describe a ``well-known'' race condition or one that you make up.)

A:

===Comments===

==Question 18==
Q: Can a threading library like pthreads be implemented on a system that has no native support for threads, such as the original UNIX? What is required for such an implementation?

A:

===Comments===

==Question 19==
Q: Despite their performance disadvantage, what is one reason kernel space thread schedulers are generally favored over userspace implementations?

A:

===Comments===

==Question 20==
Q: Linux's logical volume manager (LVM) allows for the creation of synthetic block devices that span physical disks, e.g., the blocks in two hard disks can be appended or striped so they appear to be one large hard disk. LVM, however, implements no file system abstraction---the unified block device must be formatted with a regular filesystem. What is one way LVM plus ext4 (a standard Linux filesystem) is similar to ZFS? What is one way in which they are different?

A:

===Comments===

==Question 21==
Q: Hard disk-based filesystems must design around the slow seek times of hard disks (relative to their sequential transfer speed). What characteristic(s) of flash (solid state) storage must flash-based filesystems design around to achieve high performance and reliability?

A:

===Comments===

==Question 22==
Q: Storage area networks (SANs) allow many computers to share access to the same storage devices via a special trusted network. Standard SAN access protocols work in terms of blocks. By reading from multiple devices concurrently, SANs can provide impressive I/O performance. Given that SANs already are extremely high performance, why are many researchers arguing that they should transition to object-based stores?

A:

===Comments===

==Question 23==
Q: What key properties of a database are developers trying to capture with database-filesystem hybrids?

A:

===Comments===

==Question 24==
Q: Why aren't standard databases a drop-in replacement for filesystems?

A:

===Comments===

COMP 3000 Midterm exam 2010

2010-10-27T14:20:59Z

Jjpwilso: /* Question 4 */

==Question 1==
Q: On an x86-class processor that supports hardware virtualization, it is possible in principle to virtualize any OS that can run on the native hardware. In practice, only certain operating system types and versions will run well on a given VM platform (e.g., VirtualBox, VMWare Player). What is one reason for this discrepancy?

A:

===Comments===

==Question 2==
Q: Typically, what does a guest OS's hard disk look like to the host OS?

A:

===Comments===

==Question 3==
Q: The execve system call gets three arguments: a full pathname to an executable, arguments, and environment variables. What are each of these for when "chmod a+r foo.txt" is run?

A:

===Comments===

==Question 4==
Q: Why doesn't <code>ldd</code> report all the libraries that are listed in /proc/$<$PID$>$/maps?

A: The <code>ldd</code> command only lists libraries that were statically linked at compile time. The proc mapping lists those and the ones linked dynamically at runtime.

===Comments===

==Question 5==
Q: When a process becomes undead (a zombie), who can send it to its grave permanently? What must it do?

A:

===Comments===

==Question 6==
Q: UNIX files each have permissions for three classes of users. What are they?

A: Owner, Group, Others. A file is typically created with owner set to the user running the program and the group set to that user's primary group. The value of the permissions (i.e. _rw_rw_r__) is set by the <code>umask</code> command. They can be changed after file creation with the <code>chown</code> command.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:19, 27 October 2010 (UTC) This question was truncated from what we had on the midterm. I tacked on "What are they?" Was there something else here?

==Question 7==
Q: A solution to the producer/consumer problem must take into account many conditions. What are three of them?

A:

===Comments===

==Question 8==
Q: What are three events that cause signals to be sent in UNIX?

A:

===Comments===

==Question 9==
Q: What system call do you use to receive data over an already existing pipe?

A:

===Comments===

==Question 10==
Q: How is an exokernel library OS like a guest OS running on a VM? How are they different?

A:

===Comments===

==Question 11==
Q: What is one reason the UNIX system call interface has had to change in response to computer architectural changes?

A:

===Comments===

==Question 12==
Q: What is one reason why mainframes support near perfect hardware virtualization of CPU, memory, and all I/O devices?

A:

===Comments===

==Question 13==
Q: How can a cluster have higher overall reliability than the individual computers that it consists of?

A:

===Comments===

==Question 14==
Q: What was a key advantage of the Lisp machine OS, Genera, over current operating systems? What was a key disadvantage?

A:

===Comments===

==Question 15==
Q: Why did kernel extensions in SPIN need to be written in a "safe" language? Why don't kernel extensions in Windows and Linux (drivers, modules) need to be written in a safe language?

A:

===Comments===

==Question 16==
Q: What is an important reason why modern schedulers, like the ULE and recent Linux schedulers, require a scheduling algorithm which operates in sublinear time? Correspondingly, why was a linear-time scheduler okay for the original UNIX? Explain briefly.

A:

===Comments===

==Question 17==
Q: Describe a race condition error and a test that would catch that error in testing. (You may describe a ``well-known'' race condition or one that you make up.)

A:

===Comments===

==Question 18==
Q: Can a threading library like pthreads be implemented on a system that has no native support for threads, such as the original UNIX? What is required for such an implementation?

A:

===Comments===

==Question 19==
Q: Despite their performance disadvantage, what is one reason kernel space thread schedulers are generally favored over userspace implementations?

A:

===Comments===

==Question 20==
Q: Linux's logical volume manager (LVM) allows for the creation of synthetic block devices that span physical disks, e.g., the blocks in two hard disks can be appended or striped so they appear to be one large hard disk. LVM, however, implements no file system abstraction---the unified block device must be formatted with a regular filesystem. What is one way LVM plus ext4 (a standard Linux filesystem) is similar to ZFS? What is one way in which they are different?

A:

===Comments===

==Question 21==
Q: Hard disk-based filesystems must design around the slow seek times of hard disks (relative to their sequential transfer speed). What characteristic(s) of flash (solid state) storage must flash-based filesystems design around to achieve high performance and reliability?

A:

===Comments===

==Question 22==
Q: Storage area networks (SANs) allow many computers to share access to the same storage devices via a special trusted network. Standard SAN access protocols work in terms of blocks. By reading from multiple devices concurrently, SANs can provide impressive I/O performance. Given that SANs already are extremely high performance, why are many researchers arguing that they should transition to object-based stores?

A:

===Comments===

==Question 23==
Q: What key properties of a database are developers trying to capture with database-filesystem hybrids?

A:

===Comments===

==Question 24==
Q: Why aren't standard databases a drop-in replacement for filesystems?

A:

===Comments===

COMP 3000 Midterm exam 2010

2010-10-27T14:19:17Z

Jjpwilso: /* Question 6 */

==Question 1==
Q: On an x86-class processor that supports hardware virtualization, it is possible in principle to virtualize any OS that can run on the native hardware. In practice, only certain operating system types and versions will run well on a given VM platform (e.g., VirtualBox, VMWare Player). What is one reason for this discrepancy?

A:

===Comments===

==Question 2==
Q: Typically, what does a guest OS's hard disk look like to the host OS?

A:

===Comments===

==Question 3==
Q: The execve system call gets three arguments: a full pathname to an executable, arguments, and environment variables. What are each of these for when "chmod a+r foo.txt" is run?

A:

===Comments===

==Question 4==
Q: Why doesn't <code>ldd</code> report all the libraries that are listed in /proc/$<$PID$>$/maps?

A:

===Comments===

==Question 5==
Q: When a process becomes undead (a zombie), who can send it to its grave permanently? What must it do?

A:

===Comments===

==Question 6==
Q: UNIX files each have permissions for three classes of users. What are they?

A: Owner, Group, Others. A file is typically created with owner set to the user running the program and the group set to that user's primary group. The value of the permissions (i.e. _rw_rw_r__) is set by the <code>umask</code> command. They can be changed after file creation with the <code>chown</code> command.

===Comments===

--[[User:Jjpwilso|Jjpwilso]] 14:19, 27 October 2010 (UTC) This question was truncated from what we had on the midterm. I tacked on "What are they?" Was there something else here?

==Question 7==
Q: A solution to the producer/consumer problem must take into account many conditions. What are three of them?

A:

===Comments===

==Question 8==
Q: What are three events that cause signals to be sent in UNIX?

A:

===Comments===

==Question 9==
Q: What system call do you use to receive data over an already existing pipe?

A:

===Comments===

==Question 10==
Q: How is an exokernel library OS like a guest OS running on a VM? How are they different?

A:

===Comments===

==Question 11==
Q: What is one reason the UNIX system call interface has had to change in response to computer architectural changes?

A:

===Comments===

==Question 12==
Q: What is one reason why mainframes support near perfect hardware virtualization of CPU, memory, and all I/O devices?

A:

===Comments===

==Question 13==
Q: How can a cluster have higher overall reliability than the individual computers that it consists of?

A:

===Comments===

==Question 14==
Q: What was a key advantage of the Lisp machine OS, Genera, over current operating systems? What was a key disadvantage?

A:

===Comments===

==Question 15==
Q: Why did kernel extensions in SPIN need to be written in a "safe" language? Why don't kernel extensions in Windows and Linux (drivers, modules) need to be written in a safe language?

A:

===Comments===

==Question 16==
Q: What is an important reason why modern schedulers, like the ULE and recent Linux schedulers, require a scheduling algorithm which operates in sublinear time? Correspondingly, why was a linear-time scheduler okay for the original UNIX? Explain briefly.

A:

===Comments===

==Question 17==
Q: Describe a race condition error and a test that would catch that error in testing. (You may describe a ``well-known'' race condition or one that you make up.)

A:

===Comments===

==Question 18==
Q: Can a threading library like pthreads be implemented on a system that has no native support for threads, such as the original UNIX? What is required for such an implementation?

A:

===Comments===

==Question 19==
Q: Despite their performance disadvantage, what is one reason kernel space thread schedulers are generally favored over userspace implementations?

A:

===Comments===

==Question 20==
Q: Linux's logical volume manager (LVM) allows for the creation of synthetic block devices that span physical disks, e.g., the blocks in two hard disks can be appended or striped so they appear to be one large hard disk. LVM, however, implements no file system abstraction---the unified block device must be formatted with a regular filesystem. What is one way LVM plus ext4 (a standard Linux filesystem) is similar to ZFS? What is one way in which they are different?

A:

===Comments===

==Question 21==
Q: Hard disk-based filesystems must design around the slow seek times of hard disks (relative to their sequential transfer speed). What characteristic(s) of flash (solid state) storage must flash-based filesystems design around to achieve high performance and reliability?

A:

===Comments===

==Question 22==
Q: Storage area networks (SANs) allow many computers to share access to the same storage devices via a special trusted network. Standard SAN access protocols work in terms of blocks. By reading from multiple devices concurrently, SANs can provide impressive I/O performance. Given that SANs already are extremely high performance, why are many researchers arguing that they should transition to object-based stores?

A:

===Comments===

==Question 23==
Q: What key properties of a database are developers trying to capture with database-filesystem hybrids?

A:

===Comments===

==Question 24==
Q: Why aren't standard databases a drop-in replacement for filesystems?

A:

===Comments===

COMP 3000 Midterm exam 2010

2010-10-27T14:08:01Z

Jjpwilso: comment stubs

==Question 1==
Q: On an x86-class processor that supports hardware virtualization, it is possible in principle to virtualize any OS that can run on the native hardware. In practice, only certain operating system types and versions will run well on a given VM platform (e.g., VirtualBox, VMWare Player). What is one reason for this discrepancy?

A:

===Comments===

==Question 2==
Q: Typically, what does a guest OS's hard disk look like to the host OS?

A:

===Comments===

==Question 3==
Q: The execve system call gets three arguments: a full pathname to an executable, arguments, and environment variables. What are each of these for when "chmod a+r foo.txt" is run?

A:

===Comments===

==Question 4==
Q: Why doesn't <code>ldd</code> report all the libraries that are listed in /proc/$<$PID$>$/maps?

A:

===Comments===

==Question 5==
Q: When a process becomes undead (a zombie), who can send it to its grave permanently? What must it do?

A:

===Comments===

==Question 6==
Q: UNIX files each have permissions for three classes of users.

A:

===Comments===

==Question 7==
Q: A solution to the producer/consumer problem must take into account many conditions. What are three of them?

A:

===Comments===

==Question 8==
Q: What are three events that cause signals to be sent in UNIX?

A:

===Comments===

==Question 9==
Q: What system call do you use to receive data over an already existing pipe?

A:

===Comments===

==Question 10==
Q: How is an exokernel library OS like a guest OS running on a VM? How are they different?

A:

===Comments===

==Question 11==
Q: What is one reason the UNIX system call interface has had to change in response to computer architectural changes?

A:

===Comments===

==Question 12==
Q: What is one reason why mainframes support near perfect hardware virtualization of CPU, memory, and all I/O devices?

A:

===Comments===

==Question 13==
Q: How can a cluster have higher overall reliability than the individual computers that it consists of?

A:

===Comments===

==Question 14==
Q: What was a key advantage of the Lisp machine OS, Genera, over current operating systems? What was a key disadvantage?

A:

===Comments===

==Question 15==
Q: Why did kernel extensions in SPIN need to be written in a "safe" language? Why don't kernel extensions in Windows and Linux (drivers, modules) need to be written in a safe language?

A:

===Comments===

==Question 16==
Q: What is an important reason why modern schedulers, like the ULE and recent Linux schedulers, require a scheduling algorithm which operates in sublinear time? Correspondingly, why was a linear-time scheduler okay for the original UNIX? Explain briefly.

A:

===Comments===

==Question 17==
Q: Describe a race condition error and a test that would catch that error in testing. (You may describe a ``well-known'' race condition or one that you make up.)

A:

===Comments===

==Question 18==
Q: Can a threading library like pthreads be implemented on a system that has no native support for threads, such as the original UNIX? What is required for such an implementation?

A:

===Comments===

==Question 19==
Q: Despite their performance disadvantage, what is one reason kernel space thread schedulers are generally favored over userspace implementations?

A:

===Comments===

==Question 20==
Q: Linux's logical volume manager (LVM) allows for the creation of synthetic block devices that span physical disks, e.g., the blocks in two hard disks can be appended or striped so they appear to be one large hard disk. LVM, however, implements no file system abstraction---the unified block device must be formatted with a regular filesystem. What is one way LVM plus ext4 (a standard Linux filesystem) is similar to ZFS? What is one way in which they are different?

A:

===Comments===

==Question 21==
Q: Hard disk-based filesystems must design around the slow seek times of hard disks (relative to their sequential transfer speed). What characteristic(s) of flash (solid state) storage must flash-based filesystems design around to achieve high performance and reliability?

A:

===Comments===

==Question 22==
Q: Storage area networks (SANs) allow many computers to share access to the same storage devices via a special trusted network. Standard SAN access protocols work in terms of blocks. By reading from multiple devices concurrently, SANs can provide impressive I/O performance. Given that SANs already are extremely high performance, why are many researchers arguing that they should transition to object-based stores?

A:

===Comments===

==Question 23==
Q: What key properties of a database are developers trying to capture with database-filesystem hybrids?

A:

===Comments===

==Question 24==
Q: Why aren't standard databases a drop-in replacement for filesystems?

A:

===Comments===

COMP 3000 Midterm exam 2010

2010-10-27T14:06:23Z

Jjpwilso: q4 formatting (latex gave error)

COMP 3000 Midterm exam 2010

2010-10-27T14:03:27Z

Jjpwilso: broke into sections (was messed up after q3)

COMP 3000 Essay 1 2010 Question 4

2010-10-15T04:31:58Z

Jjpwilso: /* References */

=Question=

What "operating systems" have been implemented in the following languages: LISP, Modula-3, Smalltalk, Java? To what extent do these systems match the capabilities of operating systems implemented in C and C++?

=Answer=

== Introduction ==

Not so long ago people believed the Earth was a flat world at the center of the universe. This essay addresses a more recent falsehood: that all operating systems are written in assembly language and C. It's not surprising that students of computing in this century would genuflect at the academic altars of Kernighan & Ritchie. After all we grew up with our computer worlds already pre-formed into the conceptual continents of Apple OS, UNIX, and Windows. The more historically curious among us are vaguely aware that other island cultures do exist but they represent civilizations defeated in the marketplace. Explorations into these ancient worlds resemble documentaries about archeologists decoding rediscovered languages etched in stone. But scratch the surface of any of our so-called modern operating systems and you'll find echoes of these ancient languages in our own familiar worlds. Ellen Ullman said it best when she wrote, <blockquote>''We build our computer systems like we build our cities: over time, without a plan, on top of ruins.''</blockquote>

In the sections below we present our explorations into a few truly foundational operating systems. We will also see some brand new ones that prove we stand, as one twelfth century scholar put it, [http://en.wikipedia.org/wiki/Standing_on_the_shoulders_of_giants on the shoulders of giants].

{| class="wikitable" style="text-align:center" border="1"
|+ '''Operating Systems By Language'''
|-
!width="15%"|Language
!width="85%"|OS List
|-
! Lisp
| MIT's Lisp Machines, Genera
|-
! Modula-3
| SPIN OS, Oberon
|-
! Smalltalk
| Smalltalk-80 on Xerox Alto, Squeak
|-
! Java
| JavaOS, JNode, JX, Android
|}

==Lisp Based==
===Overview===
Lisp is the second oldest programming languages, established in the late 1960's as a list-processing language. It started out being perfect for math but when it came to programming with it, it left a lot to be desired. Since then there have been a number of different versions of it, from the first Lisp – called “pure Lisp” – to Scheme and Common Lisp. Unlike most object-oriented languages which are focused on classes and instances among other things, Lisp was initially focused more on functions, in which functions call other functions, though later versions of Lisp did eventually add in classes and other typical object-oriented features. It was after this that Genera was created, an operating system written entirely in Lisp, and developed from an earlier operating system on MIT's Lisp Machine.

The Lisp Machine project in began in 1974, where Richard Greenblatt and Thomas Knight, programmers at MIT, designed a computer for general use that was designed for a single user, in which the operating system was programmed entirely in Lisp. It was designed to be completely open to the user, so any changes the user wanted to make could be done during run-time. It was also one of the first systems to be programmed in a 32-bit architecture, making it far more versatile than a number of other operating systems at the time. Despite this, however, the first Lisp Machine was quite basic compared to later iterations, namely Genera.

===Motivation===
Genera is an evolution of the initial Lisp Machine operating system, and as such improved upon many of the basic features of the system. Genera was the first object-oriented operating system, making it stand out among the already well-known Lisp Machines, especially given the fact that it isn't a composition of languages like other operating systems – most using the primary language for the operating system and another language, such as assembly, to deal with hardware directly – but written entirely in Lisp – any of five different dialects of the language – including the lowest levels of the operating system that interact with hardware due to its effective lack of a kernel. Not having to deal with a kernel allows Genera to be very powerful and efficient in certain areas, most especially in the most complex applications such as those dealing with intelligent CAD and graphics, though when given very simple, very routine applications or processes it is much less efficient than other operating systems.

===Achievements===
Another primary difference from other operating systems is that Genera is a fully open system like the first Lisp Machines. Having an open system has its positive and negative aspects. Among the foremost positive aspects is that the user is in control and not left to the whims of the operating system as is common in non-open systems – if the user doesn't like how something is happening, they can simply go in and change it from happening that way again – and allows for very quick prototyping, not needing to wait to compile every little change. Also, due to utilizing very lightweight objects and the fact that everything is in a single address space rather than split across multiple spaces greatly increases efficiency, though again mainly in non-simple tasks.

Like operating systems written in C/C++, Genera has multiple inheritance and automatic garbage collection. Unlike C/C++, however, Genera uses object-oriented memory. This gives Genera a very useful feature that many other systems cannot do, which is that, given a memory address, the system is able to provide the start address, size, and the type of object in that memory location, which increases efficiency. Even the way files are accessed is very different from other operating systems, as it uses a generic file access system; in a generic file access system, rather than following a typical structure of keeping the commands needed for local and networked files different, the commands are the same, meaning that the system essentially acts like a file on a system on the other side of the world is a local file.

===Problems===
Not everything is great with an open system, however, as there are definite security concerns: with an open system, the user can change anything, and programs can interact with other programs and change anything, meaning that if a virus were unleashed on a Genera system it could be catastrophic. To counter this problem Genera systems are protected by a very simple method: keeping the physical system itself out of reach of all but authorized users and behind locked doors. To compare to other operating systems, such as those written in C/C++, it is generally less secure because of the open system, though they are both equally at risk when installing software, as both systems are completely open at that specific junction. Genera also has issues with modularity, as it allows for something to be called even when it is not supposed to be able to be called because, as it is an open system, there is nothing really stopping this from happening. Some versions of Genera and Genera applications are also very well-documented and run quite smoothly, but others are poorly documented and run quite slowly as they are built on top of older versions without any real plan beyond adding on an extra feature here or there over and over again.

===Current Status===
Genera and Lisp Machines, while still being used in a select few locations, are not currently being developed, as their time has passed. Their legacy still lives on, however, in the object-oriented operating systems still in use today.

==Modula-3 Based==
===Overview===

An idea came to mind. A question was proposed. Can an operating system have extensibility, safety, and good performance? A group of computer scientists from Washington University took on this question and tried to come up with an answer. That’s how SPIN operating system was created. SPIN is an operating system that blends the user-level with the kernel level. The main feature is that the kernel can be extended using modules that implement interfaces to meet the applications needs to optimise performance and safety. The programming language used is Modula-3, and the reason for using this language is for its type safe properties like interfaces, extension, and its automatic storage management. The operating system sits on the traditional monolithic kernel. This is because the authors of SPIN argue that microkernels are not extendible due to the massive overhead created from switching in and out of the kernel.

===Motivation===

A poorly matched implementation prevents an
application from working well, while a poorly
matched interface prevents it from working at all.

This is the sole purpose for creating SPIN. The team had to meet three important features to make this work, a Flexible kernel with safe extensions and a good overall performance. Operating systems based on C/C++ are too general that handle many applications. Therefore, some programs may fall through the cracks and suffer the lack of safety and performance. The goal for SPIN however, was to create a specialized operating system that can run a range of applications without sacrificing safety or performance. This system would allow the user programs and applications to manipulate and change the system’s interface to load and access the data or memory needed safely and with a good overall performance. This was possible through the type safe extensions of Modula-3. Unlike C/C++, Modula-3 is considered a safe language; it guarantees the protection of the kernel from dangerous extension by forcing them to interact with the kernel through specific interfaces. Furthermore, the extensions are executed in the virtual address space not the user space.

===Achievements===

SPIN operating system came through with positive results, it actually works. Most of the goals set by the creators were accomplished. SPIN overcame the large overhead of microkernels by enforcing the extensions to be run in the kernel’s address space. The number of switches dropped which actually improved efficiency significantly.
By using Modula-3’s safe interfaces, there was no run-time checking of the code of the extensions. This was a huge advantage over languages like C/C++ due to the fact that a significant overhead was cleared from the execution time. SPIN addressed the latency issue by only allowing code with low latency, which is quite small, to access the system through extensions that access the kernel directly. This is another advantage of using a safe programming language.

===Problems===

Two general issues with extendible systems are adaptability and reliability. Using a safe programming language comes with a price. In SPIN, there is a loss of efficiency when the dynamic checks run for array index bounds. These checks also add a significant overhead to the execution time.
SPIN is written with very few Modula-3 code lines and are much easier to understand compared to other operating systems written in C or C++. However, the kernel of the system is almost entirely written in C and Assembly. These two languages are considered unsafe in contrast to Modula-3. The mix of safe and unsafe code is a huge complication which leads to many bugs, which compromises the safe extensions and interfaces used by applications.

===Current Status===

SPIN, and extendible operating systems in general, is becoming the playground for research. The authors of SPIN at the University of Washington are performing projects and continuing the on-going research of improving extendible operating systems. The solutions they might come up with could greatly influence the operating systems as we see them today.

==Smalltalk Based==
===Overview===

<blockquote>The best way to predict the future is to invent it. -- Alan Kay</blockquote>

If you are reading this paper on a personal computer, and it has a GUI with overlapping windows, desktop icons, and a mouse pointer then you owe a debt to a group of researchers led by Alan Kay at Xerox's Palo Alto Research Center, or Xerox PARC. Many of those ideas had appeared elsewhere in one form or another, but the first time they came together in a demonstrable and portable form was in the early 1970s at Xerox on a machine called the Alto [http://en.wikipedia.org/wiki/Xerox_Alto]. Their example of a working personal computer was to be the inspiration that launched Apple into the history books.

===Motivation===

Searching for operating systems written in Smalltalk is an exercise in recursion. The Smalltalk OS was written in -- what else -- Smalltalk. But Alan Kay's Learning Research Group (LRG) didn't just set out to write a clever new language that could bootstrap its own environment. Their goal was to produce an entirely new learning environment to "amplify human reach and bring new ways of thinking to a faltering civilization that desperately needed it." [http://www.smalltalk.org/smalltalk/TheEarlyHistoryOfSmalltalk_III.html]

===Achievements===

They did manage to invent new ways of think about computing. Alan Kay coined the term "object-oriented programming". His inspiration had come from notions of objects already well known in LISP but he wasn't shy about giving credit where it was due. Kay believed his knowledge of LISP helped him think more clearly about computer problems. He felt it contained "great thinking patterns" and "deep beauty" and he vowed to preserve these qualities in the language that would become Smalltalk.

But Kay wasn't just interested the symbolic language at the heart of a system. He incorporated recent studies on learning and cognition from such experts such as Maria Montessori, Jean Piaget, and Jerome Bruner in his goal to form a complete vision of the personal computer. For instance, Bruner's research implied that people learn new thoughts in a loose sequence of translations [http://en.wikipedia.org/wiki/Jerome_Bruner]. First there are actions, or ''enactive'' representations which transform into images or ''iconic'' representations, and finally into language or ''symbolic'' representations. We may take it for granted today, but it was human insights such as these that helped Kay form the basis of graphical user interfaces as channels through which we perceive ideas represented in a computer.

Still the language is no small point. The central idea in Smalltalk (and Smalltalk-80 as an example of a whole system) is that "everything is an object". This includes what we already might think of as objects, such as files, forms, and fields but it also includes transformational methods such as actions, behaviors, and calculations. This places it in distinct contrast with operating systems written in C, and even those written in C++. Gone are such notions as ''process'' and ''semaphore''. In their place are ''messages'' between objects and ''events'' to which interested objects may react.

Another difference in Smalltalk from C-based operating systems is the lack of security. The only means Smalltalk has of protecting anything is through the visibility (private or public) of an object's methods and properties. But if you know how to dig through the system you can modify -- and break -- just about everything. This may be offer a sense of freedom to some. In the networked world this seems more than a bit risky. Yes, Smalltalk has networking.

===Problems===

Performance in Smalltalk-80 wasn't what we expect from our machines today. A later version of the system called Squeak made some attempt to address this issue (and that of portability) by writing a C language generator to create its virtual machine. This was reasonably successful, and you can now run Squeak on a number of platforms including stand-alone inside of VirtualBox.

[[File:Squeak-Hello-Carleton.png]]

Most non-programmers don't know much about Smalltalk-80 or Squeak though it's tempting to speculate what might have happened if Xerox had chosen to keep its secrets from Apple and commercialize the Alto. The decision not to commercialize was one that hurt Alan Kay's team, one of whom in fact seized the moment and went to work for Apple. By the mid-seventies Kay wasn't sure they had completely solved the problems for which they had set out. In his disillusionment he sought to burn everything and start over. He was concerned he would be fulfilling both claims of Marshall McLuhan, namely that "our tools reshape us" and "inadequate tools ''still'' reshape us", though presumably not always for the better.

===Current Status===

The Smalltalk world continues to have ripples around the world, though perhaps less than the incredible splash they made in the 1970s.

==Java Based==
===Overview===

Java is used on a plethora of devices and systems throughout the industry from cell phones to web applets. With Java being a language that creates a virtual machine for each application, one would think that it is already suited to be an operating system in itself but this is not the case. Java is built to run on top of other operating systems such as Microsoft Windows, Mac OS X and Ubuntu Linux rather than being a standalone system. This section will discuss various operating systems that are written in Java such as JavaOS, Android, JNode and JX.

[http://java.sun.com/developer/products/JavaOS/ JavaOS] is Sun Microsystems very own creation. This system runs on different layers to make a scalable and easily updateable operating system [http://java.sun.com/developer/products/JavaOS/OverView/index.html ]. The first layer is the microkernel which handles the memory architecture, booting, interrupt handling, threading, traps and DMA handling. The Java Virtual Machine is also compiled into native code for the system and is run on top of the microkernel. The second layer consists of the JavaOS for Business software which extends the memory module to optimize for systems with limited memory[http://java.sun.com/developer/products/JavaOS/OverView/index.html ]. All device drivers for the system are written and run in Java and are what the third layer is consisted of. Finally, the fourth layer is a stand-alone JDK runtime environment used to run user applications.

[http://www.android.com/ Android] was created by a very ambitious team at Google for use with cell phones. It is basically an operating system running on top of another minimalistic operating system. At the very lowest end of Android, a Linux 2.6 kernel is what powers it. All device drivers are written and compiled for the native hardware or compiled using Google’s Native Development Kit and core system libraries such as libc, OpenGL | ES and SQLite are dynamically linked in per application [http://developer.android.com/guide/basics/what-is-android.html]. In the Android runtime, the Dalvik Virtual Machine (DVM) controls applications similar to the Java Virtual Machine. Dalvik is similar to the aforementioned JavaOS as it relies on the kernel to manage threading and low-level memory management. Running on top of the DVM are core libraries and application frameworks for the Android operating system. These frameworks include resource management, window management, notification manage just to name a few. Applications are then built on top of these frameworks and are the end result in which the user will actually interact with.

[http://www.jnode.org/ JNode] began as the Java Bootable System (JBS) in 1995. Ewout Prangsma, the creator, was unhappy with the amount of native C and assembly used for the system so he began a new project, JNode. JNode only uses a small amount of assembly code for booting the system now compared to the initial JBS [http://www.jnode.org/node/174]. The rest of the system is completely written in Java including its graphical user interface. Applications in JNode are referred to as plugins [http://www.jnode.org/node/175] including the device drivers, filesystems, networking and user applications.

[http://www4.informatik.uni-erlangen.de/Projects/JX/index.html JX] was created at the University of Erlangen. At it’s base is a microkernel that the basic Java Virtual Machine runs on which is similar to JavaOS. The system runs on the idea of domains where the microkernel runs at domain level zero and subsequent programs run in domain A, B, C, etc. Domains contain their own threads, heap and garbage collector and can communicate with other domains using portals. A portal is essentially the same as inter-process communication but using domains as processes instead [http://www.usenix.org/events/usenix02/full_papers/golm/golm.pdf].

===Motivation===

Java is a powerful language that already contains the code necessary for running on many different platforms. With the concept of virtual machines for each individual application it provides a layer of security that not every operating system has. Each virtual machine has it’s own heap which keeps other processes from accessing and writing over already allocated memory since the kernel manages the memory paging. With the use of just-in-time (JIT) compilers, these operating systems can almost achieve comparable run times when compared to native compiled applications.

Since all Java applications are compiled into the same set of bytecode, third-parties can develop their own implementation of the Java runtime. For example, Google has created the Dalvik Virtual Machine for use with the Android platform so it will run more efficiently on small devices while reducing the memory footprint [http://developer.android.com/guide/basics/what-is-android.html]. Google has just recently released Android 2.2 which includes a new JIT compiler for their Dalvik Virtual Machine which can improve speeds of applications up to 2-5 times [http://android-developers.blogspot.com/2010/05/dalvik-jit.html]. IBM also has their own version of the Java Virtual Machine, J9, that is used in many of their own pieces of software and also includes its own implementation of a JIT compiler.

===Achievements===

Android is probably one of the most well-known and mainstream Java based operating system currently on the market. Even though it’s the youngest of the operating systems discussed, it has become one of the newest standards for smartphones. With over 150 devices and counting, Android continues to grow and develop.

===Problems===

One major issue with using Java for an operating system is completely relying on the operating system and kernel to manage memory. Since Java is a garbage collecting language, developers do not have direct access to the memory and have to rely on the operating system to clean up after any objects have been left behind. This can be an issue with lower memory systems while running multiple applications at the same time.

===Current Status===

Each operating system has been created or worked on in the last ten years but some have either halted development or have not seen a major stable release in quite a while. JavaOS is still being maintained by Oracle after they had purchased Sun Microsystems. JNode has not seen an update in over a year and a half [http://sourceforge.net/projects/jnode/files/]. JX also seems to be at a stand still in development with only a minor update after its 0.1 release [http://www4.informatik.uni-erlangen.de/Projects/JX/download-sources.html]. Finally, Android is the most active with minor or major updates coming out every few months. The current state Android is in is Android 2.2 with Android 3.0 currently being hinted for quarter four of 2010 [http://www.techradar.com/news/phone-and-communications/mobile-phones/android-3-0-details-you-need-to-know-706243].

== References ==

Moon, David A. "Genera Retrospective." ''1991 International Workshop on Object Orientation in Operating Systems'' (1991): 2-8. Print.

Moon, David A., Thomas F. Knight, John T. Holloway, and Richard D. Greenblatt. "A Lisp Machine." ''ACM SIGIR Forum'' 15.2 (1980): 137-38. Print.

Pleszkun, A. R., and M. J. Thazhuthaveetil. "The Architecture of Lisp Machines." ''Computer'' 20.3 (1987): 35-44. Print.

FOR SPIN

B. N. Bershad, S. Savage, P. Pardyak, E. G. Sirer, M. Fiuczynski, D. Becker, S. Eggers, and C. Chambers. Extensibility, Safety and Performance in the SPIN Operating System. In the Proceedings of the 15th ACM Symposium on Operating System Principles (SOSP), Copper Mountain, CO, December 1995.

W. C. Hsieh, M. E. Fiuczynski, C. Garrett, S. Savage, D. Becker and B. N. Bershad. Language Support for Extensible Operating Systems. Workshop on Compiler Support for System Software, University of Washington, February 1996.

S. Savage, B. N. Bershad. Issues in the Design of an Extensible Operating System. Proceedings of the First USENIX Symposium on Operatings Systems Design and Implementation (OSDI), November 1994 (Unpublished).

University of Washington, Dept. of Computer Science. The SPIN Operating System. [http://www.cs.washington.edu.html]

For Smalltalk:

Alan Kay, The Early History of Smalltalk [http://www.smalltalk.org/downloads/papers/SmalltalkHistoryHOPL.pdf]

COMP 3000 Essay 1 2010 Question 4

2010-10-15T04:31:28Z

Jjpwilso: /* References */

COMP 3000 Essay 1 2010 Question 4

2010-10-15T04:27:47Z

Jjpwilso: /* Team Note (to be removed by delivery date) */

COMP 3000 Essay 1 2010 Question 4

2010-10-15T04:27:23Z

Jjpwilso: /* Introduction */

=Question=

What "operating systems" have been implemented in the following languages: LISP, Modula-3, Smalltalk, Java? To what extent do these systems match the capabilities of operating systems implemented in C and C++?

=Team Note (to be removed by delivery date)=

Please use the [[Talk:COMP_3000_Essay_1_2010_Question_4|discussion page]] for any planning and comments.

=Answer=

== Introduction ==

Not so long ago people believed the Earth was a flat world at the center of the universe. This essay addresses a more recent falsehood: that all operating systems are written in assembly language and C. It's not surprising that students of computing in this century would genuflect at the academic altars of Kernighan & Ritchie. After all we grew up with our computer worlds already pre-formed into the conceptual continents of Apple OS, UNIX, and Windows. The more historically curious among us are vaguely aware that other island cultures do exist but they represent civilizations defeated in the marketplace. Explorations into these ancient worlds resemble documentaries about archeologists decoding rediscovered languages etched in stone. But scratch the surface of any of our so-called modern operating systems and you'll find echoes of these ancient languages in our own familiar worlds. Ellen Ullman said it best when she wrote, <blockquote>''We build our computer systems like we build our cities: over time, without a plan, on top of ruins.''</blockquote>

In the sections below we present our explorations into a few truly foundational operating systems. We will also see some brand new ones that prove we stand, as one twelfth century scholar put it, [http://en.wikipedia.org/wiki/Standing_on_the_shoulders_of_giants on the shoulders of giants].

{| class="wikitable" style="text-align:center" border="1"
|+ '''Operating Systems By Language'''
|-
!width="15%"|Language
!width="85%"|OS List
|-
! Lisp
| MIT's Lisp Machines, Genera
|-
! Modula-3
| SPIN OS, Oberon
|-
! Smalltalk
| Smalltalk-80 on Xerox Alto, Squeak
|-
! Java
| JavaOS, JNode, JX, Android
|}

==Lisp Based==
===Overview===
Lisp is the second oldest programming languages, established in the late 1960's as a list-processing language. It started out being perfect for math but when it came to programming with it, it left a lot to be desired. Since then there have been a number of different versions of it, from the first Lisp – called “pure Lisp” – to Scheme and Common Lisp. Unlike most object-oriented languages which are focused on classes and instances among other things, Lisp was initially focused more on functions, in which functions call other functions, though later versions of Lisp did eventually add in classes and other typical object-oriented features. It was after this that Genera was created, an operating system written entirely in Lisp, and developed from an earlier operating system on MIT's Lisp Machine.

The Lisp Machine project in began in 1974, where Richard Greenblatt and Thomas Knight, programmers at MIT, designed a computer for general use that was designed for a single user, in which the operating system was programmed entirely in Lisp. It was designed to be completely open to the user, so any changes the user wanted to make could be done during run-time. It was also one of the first systems to be programmed in a 32-bit architecture, making it far more versatile than a number of other operating systems at the time. Despite this, however, the first Lisp Machine was quite basic compared to later iterations, namely Genera.

===Motivation===
Genera is an evolution of the initial Lisp Machine operating system, and as such improved upon many of the basic features of the system. Genera was the first object-oriented operating system, making it stand out among the already well-known Lisp Machines, especially given the fact that it isn't a composition of languages like other operating systems – most using the primary language for the operating system and another language, such as assembly, to deal with hardware directly – but written entirely in Lisp – any of five different dialects of the language – including the lowest levels of the operating system that interact with hardware due to its effective lack of a kernel. Not having to deal with a kernel allows Genera to be very powerful and efficient in certain areas, most especially in the most complex applications such as those dealing with intelligent CAD and graphics, though when given very simple, very routine applications or processes it is much less efficient than other operating systems.

===Achievements===
Another primary difference from other operating systems is that Genera is a fully open system like the first Lisp Machines. Having an open system has its positive and negative aspects. Among the foremost positive aspects is that the user is in control and not left to the whims of the operating system as is common in non-open systems – if the user doesn't like how something is happening, they can simply go in and change it from happening that way again – and allows for very quick prototyping, not needing to wait to compile every little change. Also, due to utilizing very lightweight objects and the fact that everything is in a single address space rather than split across multiple spaces greatly increases efficiency, though again mainly in non-simple tasks.

Like operating systems written in C/C++, Genera has multiple inheritance and automatic garbage collection. Unlike C/C++, however, Genera uses object-oriented memory. This gives Genera a very useful feature that many other systems cannot do, which is that, given a memory address, the system is able to provide the start address, size, and the type of object in that memory location, which increases efficiency. Even the way files are accessed is very different from other operating systems, as it uses a generic file access system; in a generic file access system, rather than following a typical structure of keeping the commands needed for local and networked files different, the commands are the same, meaning that the system essentially acts like a file on a system on the other side of the world is a local file.

===Problems===
Not everything is great with an open system, however, as there are definite security concerns: with an open system, the user can change anything, and programs can interact with other programs and change anything, meaning that if a virus were unleashed on a Genera system it could be catastrophic. To counter this problem Genera systems are protected by a very simple method: keeping the physical system itself out of reach of all but authorized users and behind locked doors. To compare to other operating systems, such as those written in C/C++, it is generally less secure because of the open system, though they are both equally at risk when installing software, as both systems are completely open at that specific junction. Genera also has issues with modularity, as it allows for something to be called even when it is not supposed to be able to be called because, as it is an open system, there is nothing really stopping this from happening. Some versions of Genera and Genera applications are also very well-documented and run quite smoothly, but others are poorly documented and run quite slowly as they are built on top of older versions without any real plan beyond adding on an extra feature here or there over and over again.

===Current Status===
Genera and Lisp Machines, while still being used in a select few locations, are not currently being developed, as their time has passed. Their legacy still lives on, however, in the object-oriented operating systems still in use today.

==Modula-3 Based==
===Overview===

An idea came to mind. A question was proposed. Can an operating system have extensibility, safety, and good performance? A group of computer scientists from Washington University took on this question and tried to come up with an answer. That’s how SPIN operating system was created. SPIN is an operating system that blends the user-level with the kernel level. The main feature is that the kernel can be extended using modules that implement interfaces to meet the applications needs to optimise performance and safety. The programming language used is Modula-3, and the reason for using this language is for its type safe properties like interfaces, extension, and its automatic storage management. The operating system sits on the traditional monolithic kernel. This is because the authors of SPIN argue that microkernels are not extendible due to the massive overhead created from switching in and out of the kernel.

===Motivation===

A poorly matched implementation prevents an
application from working well, while a poorly
matched interface prevents it from working at all.

This is the sole purpose for creating SPIN. The team had to meet three important features to make this work, a Flexible kernel with safe extensions and a good overall performance. Operating systems based on C/C++ are too general that handle many applications. Therefore, some programs may fall through the cracks and suffer the lack of safety and performance. The goal for SPIN however, was to create a specialized operating system that can run a range of applications without sacrificing safety or performance. This system would allow the user programs and applications to manipulate and change the system’s interface to load and access the data or memory needed safely and with a good overall performance. This was possible through the type safe extensions of Modula-3. Unlike C/C++, Modula-3 is considered a safe language; it guarantees the protection of the kernel from dangerous extension by forcing them to interact with the kernel through specific interfaces. Furthermore, the extensions are executed in the virtual address space not the user space.

===Achievements===

SPIN operating system came through with positive results, it actually works. Most of the goals set by the creators were accomplished. SPIN overcame the large overhead of microkernels by enforcing the extensions to be run in the kernel’s address space. The number of switches dropped which actually improved efficiency significantly.
By using Modula-3’s safe interfaces, there was no run-time checking of the code of the extensions. This was a huge advantage over languages like C/C++ due to the fact that a significant overhead was cleared from the execution time. SPIN addressed the latency issue by only allowing code with low latency, which is quite small, to access the system through extensions that access the kernel directly. This is another advantage of using a safe programming language.

===Problems===

Two general issues with extendible systems are adaptability and reliability. Using a safe programming language comes with a price. In SPIN, there is a loss of efficiency when the dynamic checks run for array index bounds. These checks also add a significant overhead to the execution time.
SPIN is written with very few Modula-3 code lines and are much easier to understand compared to other operating systems written in C or C++. However, the kernel of the system is almost entirely written in C and Assembly. These two languages are considered unsafe in contrast to Modula-3. The mix of safe and unsafe code is a huge complication which leads to many bugs, which compromises the safe extensions and interfaces used by applications.

===Current Status===

SPIN, and extendible operating systems in general, is becoming the playground for research. The authors of SPIN at the University of Washington are performing projects and continuing the on-going research of improving extendible operating systems. The solutions they might come up with could greatly influence the operating systems as we see them today.

==Smalltalk Based==
===Overview===

<blockquote>The best way to predict the future is to invent it. -- Alan Kay</blockquote>

If you are reading this paper on a personal computer, and it has a GUI with overlapping windows, desktop icons, and a mouse pointer then you owe a debt to a group of researchers led by Alan Kay at Xerox's Palo Alto Research Center, or Xerox PARC. Many of those ideas had appeared elsewhere in one form or another, but the first time they came together in a demonstrable and portable form was in the early 1970s at Xerox on a machine called the Alto [http://en.wikipedia.org/wiki/Xerox_Alto]. Their example of a working personal computer was to be the inspiration that launched Apple into the history books.

===Motivation===

Searching for operating systems written in Smalltalk is an exercise in recursion. The Smalltalk OS was written in -- what else -- Smalltalk. But Alan Kay's Learning Research Group (LRG) didn't just set out to write a clever new language that could bootstrap its own environment. Their goal was to produce an entirely new learning environment to "amplify human reach and bring new ways of thinking to a faltering civilization that desperately needed it." [http://www.smalltalk.org/smalltalk/TheEarlyHistoryOfSmalltalk_III.html]

===Achievements===

They did manage to invent new ways of think about computing. Alan Kay coined the term "object-oriented programming". His inspiration had come from notions of objects already well known in LISP but he wasn't shy about giving credit where it was due. Kay believed his knowledge of LISP helped him think more clearly about computer problems. He felt it contained "great thinking patterns" and "deep beauty" and he vowed to preserve these qualities in the language that would become Smalltalk.

But Kay wasn't just interested the symbolic language at the heart of a system. He incorporated recent studies on learning and cognition from such experts such as Maria Montessori, Jean Piaget, and Jerome Bruner in his goal to form a complete vision of the personal computer. For instance, Bruner's research implied that people learn new thoughts in a loose sequence of translations [http://en.wikipedia.org/wiki/Jerome_Bruner]. First there are actions, or ''enactive'' representations which transform into images or ''iconic'' representations, and finally into language or ''symbolic'' representations. We may take it for granted today, but it was human insights such as these that helped Kay form the basis of graphical user interfaces as channels through which we perceive ideas represented in a computer.

Still the language is no small point. The central idea in Smalltalk (and Smalltalk-80 as an example of a whole system) is that "everything is an object". This includes what we already might think of as objects, such as files, forms, and fields but it also includes transformational methods such as actions, behaviors, and calculations. This places it in distinct contrast with operating systems written in C, and even those written in C++. Gone are such notions as ''process'' and ''semaphore''. In their place are ''messages'' between objects and ''events'' to which interested objects may react.

Another difference in Smalltalk from C-based operating systems is the lack of security. The only means Smalltalk has of protecting anything is through the visibility (private or public) of an object's methods and properties. But if you know how to dig through the system you can modify -- and break -- just about everything. This may be offer a sense of freedom to some. In the networked world this seems more than a bit risky. Yes, Smalltalk has networking.

===Problems===

Performance in Smalltalk-80 wasn't what we expect from our machines today. A later version of the system called Squeak made some attempt to address this issue (and that of portability) by writing a C language generator to create its virtual machine. This was reasonably successful, and you can now run Squeak on a number of platforms including stand-alone inside of VirtualBox.

[[File:Squeak-Hello-Carleton.png]]

Most non-programmers don't know much about Smalltalk-80 or Squeak though it's tempting to speculate what might have happened if Xerox had chosen to keep its secrets from Apple and commercialize the Alto. The decision not to commercialize was one that hurt Alan Kay's team, one of whom in fact seized the moment and went to work for Apple. By the mid-seventies Kay wasn't sure they had completely solved the problems for which they had set out. In his disillusionment he sought to burn everything and start over. He was concerned he would be fulfilling both claims of Marshall McLuhan, namely that "our tools reshape us" and "inadequate tools ''still'' reshape us", though presumably not always for the better.

===Current Status===

The Smalltalk world continues to have ripples around the world, though perhaps less than the incredible splash they made in the 1970s.

==Java Based==
===Overview===

Java is used on a plethora of devices and systems throughout the industry from cell phones to web applets. With Java being a language that creates a virtual machine for each application, one would think that it is already suited to be an operating system in itself but this is not the case. Java is built to run on top of other operating systems such as Microsoft Windows, Mac OS X and Ubuntu Linux rather than being a standalone system. This section will discuss various operating systems that are written in Java such as JavaOS, Android, JNode and JX.

[http://java.sun.com/developer/products/JavaOS/ JavaOS] is Sun Microsystems very own creation. This system runs on different layers to make a scalable and easily updateable operating system [http://java.sun.com/developer/products/JavaOS/OverView/index.html ]. The first layer is the microkernel which handles the memory architecture, booting, interrupt handling, threading, traps and DMA handling. The Java Virtual Machine is also compiled into native code for the system and is run on top of the microkernel. The second layer consists of the JavaOS for Business software which extends the memory module to optimize for systems with limited memory[http://java.sun.com/developer/products/JavaOS/OverView/index.html ]. All device drivers for the system are written and run in Java and are what the third layer is consisted of. Finally, the fourth layer is a stand-alone JDK runtime environment used to run user applications.

[http://www.android.com/ Android] was created by a very ambitious team at Google for use with cell phones. It is basically an operating system running on top of another minimalistic operating system. At the very lowest end of Android, a Linux 2.6 kernel is what powers it. All device drivers are written and compiled for the native hardware or compiled using Google’s Native Development Kit and core system libraries such as libc, OpenGL | ES and SQLite are dynamically linked in per application [http://developer.android.com/guide/basics/what-is-android.html]. In the Android runtime, the Dalvik Virtual Machine (DVM) controls applications similar to the Java Virtual Machine. Dalvik is similar to the aforementioned JavaOS as it relies on the kernel to manage threading and low-level memory management. Running on top of the DVM are core libraries and application frameworks for the Android operating system. These frameworks include resource management, window management, notification manage just to name a few. Applications are then built on top of these frameworks and are the end result in which the user will actually interact with.

[http://www.jnode.org/ JNode] began as the Java Bootable System (JBS) in 1995. Ewout Prangsma, the creator, was unhappy with the amount of native C and assembly used for the system so he began a new project, JNode. JNode only uses a small amount of assembly code for booting the system now compared to the initial JBS [http://www.jnode.org/node/174]. The rest of the system is completely written in Java including its graphical user interface. Applications in JNode are referred to as plugins [http://www.jnode.org/node/175] including the device drivers, filesystems, networking and user applications.

[http://www4.informatik.uni-erlangen.de/Projects/JX/index.html JX] was created at the University of Erlangen. At it’s base is a microkernel that the basic Java Virtual Machine runs on which is similar to JavaOS. The system runs on the idea of domains where the microkernel runs at domain level zero and subsequent programs run in domain A, B, C, etc. Domains contain their own threads, heap and garbage collector and can communicate with other domains using portals. A portal is essentially the same as inter-process communication but using domains as processes instead [http://www.usenix.org/events/usenix02/full_papers/golm/golm.pdf].

===Motivation===

Java is a powerful language that already contains the code necessary for running on many different platforms. With the concept of virtual machines for each individual application it provides a layer of security that not every operating system has. Each virtual machine has it’s own heap which keeps other processes from accessing and writing over already allocated memory since the kernel manages the memory paging. With the use of just-in-time (JIT) compilers, these operating systems can almost achieve comparable run times when compared to native compiled applications.

Since all Java applications are compiled into the same set of bytecode, third-parties can develop their own implementation of the Java runtime. For example, Google has created the Dalvik Virtual Machine for use with the Android platform so it will run more efficiently on small devices while reducing the memory footprint [http://developer.android.com/guide/basics/what-is-android.html]. Google has just recently released Android 2.2 which includes a new JIT compiler for their Dalvik Virtual Machine which can improve speeds of applications up to 2-5 times [http://android-developers.blogspot.com/2010/05/dalvik-jit.html]. IBM also has their own version of the Java Virtual Machine, J9, that is used in many of their own pieces of software and also includes its own implementation of a JIT compiler.

===Achievements===

Android is probably one of the most well-known and mainstream Java based operating system currently on the market. Even though it’s the youngest of the operating systems discussed, it has become one of the newest standards for smartphones. With over 150 devices and counting, Android continues to grow and develop.

===Problems===

One major issue with using Java for an operating system is completely relying on the operating system and kernel to manage memory. Since Java is a garbage collecting language, developers do not have direct access to the memory and have to rely on the operating system to clean up after any objects have been left behind. This can be an issue with lower memory systems while running multiple applications at the same time.

===Current Status===

Each operating system has been created or worked on in the last ten years but some have either halted development or have not seen a major stable release in quite a while. JavaOS is still being maintained by Oracle after they had purchased Sun Microsystems. JNode has not seen an update in over a year and a half [http://sourceforge.net/projects/jnode/files/]. JX also seems to be at a stand still in development with only a minor update after its 0.1 release [http://www4.informatik.uni-erlangen.de/Projects/JX/download-sources.html]. Finally, Android is the most active with minor or major updates coming out every few months. The current state Android is in is Android 2.2 with Android 3.0 currently being hinted for quarter four of 2010 [http://www.techradar.com/news/phone-and-communications/mobile-phones/android-3-0-details-you-need-to-know-706243].

== References ==

Moon, David A. "Genera Retrospective." ''1991 International Workshop on Object Orientation in Operating Systems'' (1991): 2-8. Print.

Moon, David A., Thomas F. Knight, John T. Holloway, and Richard D. Greenblatt. "A Lisp Machine." ''ACM SIGIR Forum'' 15.2 (1980): 137-38. Print.

Pleszkun, A. R., and M. J. Thazhuthaveetil. "The Architecture of Lisp Machines." ''Computer'' 20.3 (1987): 35-44. Print.

FOR SPIN

B. N. Bershad, S. Savage, P. Pardyak, E. G. Sirer, M. Fiuczynski, D. Becker, S. Eggers, and C. Chambers. Extensibility, Safety and Performance in the SPIN Operating System. In the Proceedings of the 15th ACM Symposium on Operating System Principles (SOSP), Copper Mountain, CO, December 1995.

W. C. Hsieh, M. E. Fiuczynski, C. Garrett, S. Savage, D. Becker and B. N. Bershad. Language Support for Extensible Operating Systems. Workshop on Compiler Support for System Software, University of Washington, February 1996.

S. Savage, B. N. Bershad. Issues in the Design of an Extensible Operating System. Proceedings of the First USENIX Symposium on Operatings Systems Design and Implementation (OSDI), November 1994 (Unpublished).

University of Washington, Dept. of Computer Science. The SPIN Operating System. [http://www.cs.washington.edu.html]

COMP 3000 Essay 1 2010 Question 4

2010-10-15T04:25:37Z

Jjpwilso: /* Problems */

=Question=

What "operating systems" have been implemented in the following languages: LISP, Modula-3, Smalltalk, Java? To what extent do these systems match the capabilities of operating systems implemented in C and C++?

=Team Note (to be removed by delivery date)=

Please use the [[Talk:COMP_3000_Essay_1_2010_Question_4|discussion page]] for any planning and comments.

=Answer=

== Introduction ==

Not so long ago people believed the Earth was a flat world at the center of the universe. This essay addresses a more recent falsehood: that all operating systems are written in assembly language and C. It's not surprising that students of computing in this century would genuflect at the academic altars of Kernighan & Ritchie. After all we grew up with our computer worlds already pre-formed into the conceptual continents of Apple OS, UNIX, and Windows. The more historically curious among us are vaguely aware that other island cultures do exist but they represent civilizations defeated in the marketplace. Explorations into these ancient worlds resemble documentaries about archeologists decoding rediscovered languages etched in stone. But scratch the surface of any of our so-called modern operating systems and you'll find echoes of these ancient languages in our own familiar worlds. Ellen Ullman said it best when she wrote, <blockquote>''We build our computer systems like we build our cities: over time, without a plan, on top of ruins.''</blockquote>

In the sections below we present our explorations into a few truly foundational operating systems. We will also see some brand new ones that prove we stand, as one twelfth century scholar put it, [http://en.wikipedia.org/wiki/Standing_on_the_shoulders_of_giants on the shoulders of giants].

The following is a short list of operating systems written partially or completely with the given language. We will go into more detail in the language-specific sections below.

{| class="wikitable" style="text-align:center" border="1"
|+ '''Operating Systems By Language'''
|-
!width="15%"|Language
!width="85%"|OS List
|-
! Lisp
| MIT's Lisp Machines, Genera
|-
! Modula-3
| SPIN OS, Oberon
|-
! Smalltalk
| Smalltalk-80 on Xerox Alto, Squeak
|-
! Java
| JavaOS, JNode, JX, Android
|}

==Lisp Based==
===Overview===
Lisp is the second oldest programming languages, established in the late 1960's as a list-processing language. It started out being perfect for math but when it came to programming with it, it left a lot to be desired. Since then there have been a number of different versions of it, from the first Lisp – called “pure Lisp” – to Scheme and Common Lisp. Unlike most object-oriented languages which are focused on classes and instances among other things, Lisp was initially focused more on functions, in which functions call other functions, though later versions of Lisp did eventually add in classes and other typical object-oriented features. It was after this that Genera was created, an operating system written entirely in Lisp, and developed from an earlier operating system on MIT's Lisp Machine.

The Lisp Machine project in began in 1974, where Richard Greenblatt and Thomas Knight, programmers at MIT, designed a computer for general use that was designed for a single user, in which the operating system was programmed entirely in Lisp. It was designed to be completely open to the user, so any changes the user wanted to make could be done during run-time. It was also one of the first systems to be programmed in a 32-bit architecture, making it far more versatile than a number of other operating systems at the time. Despite this, however, the first Lisp Machine was quite basic compared to later iterations, namely Genera.

===Motivation===
Genera is an evolution of the initial Lisp Machine operating system, and as such improved upon many of the basic features of the system. Genera was the first object-oriented operating system, making it stand out among the already well-known Lisp Machines, especially given the fact that it isn't a composition of languages like other operating systems – most using the primary language for the operating system and another language, such as assembly, to deal with hardware directly – but written entirely in Lisp – any of five different dialects of the language – including the lowest levels of the operating system that interact with hardware due to its effective lack of a kernel. Not having to deal with a kernel allows Genera to be very powerful and efficient in certain areas, most especially in the most complex applications such as those dealing with intelligent CAD and graphics, though when given very simple, very routine applications or processes it is much less efficient than other operating systems.

===Achievements===
Another primary difference from other operating systems is that Genera is a fully open system like the first Lisp Machines. Having an open system has its positive and negative aspects. Among the foremost positive aspects is that the user is in control and not left to the whims of the operating system as is common in non-open systems – if the user doesn't like how something is happening, they can simply go in and change it from happening that way again – and allows for very quick prototyping, not needing to wait to compile every little change. Also, due to utilizing very lightweight objects and the fact that everything is in a single address space rather than split across multiple spaces greatly increases efficiency, though again mainly in non-simple tasks.

Like operating systems written in C/C++, Genera has multiple inheritance and automatic garbage collection. Unlike C/C++, however, Genera uses object-oriented memory. This gives Genera a very useful feature that many other systems cannot do, which is that, given a memory address, the system is able to provide the start address, size, and the type of object in that memory location, which increases efficiency. Even the way files are accessed is very different from other operating systems, as it uses a generic file access system; in a generic file access system, rather than following a typical structure of keeping the commands needed for local and networked files different, the commands are the same, meaning that the system essentially acts like a file on a system on the other side of the world is a local file.

===Problems===
Not everything is great with an open system, however, as there are definite security concerns: with an open system, the user can change anything, and programs can interact with other programs and change anything, meaning that if a virus were unleashed on a Genera system it could be catastrophic. To counter this problem Genera systems are protected by a very simple method: keeping the physical system itself out of reach of all but authorized users and behind locked doors. To compare to other operating systems, such as those written in C/C++, it is generally less secure because of the open system, though they are both equally at risk when installing software, as both systems are completely open at that specific junction. Genera also has issues with modularity, as it allows for something to be called even when it is not supposed to be able to be called because, as it is an open system, there is nothing really stopping this from happening. Some versions of Genera and Genera applications are also very well-documented and run quite smoothly, but others are poorly documented and run quite slowly as they are built on top of older versions without any real plan beyond adding on an extra feature here or there over and over again.

===Current Status===
Genera and Lisp Machines, while still being used in a select few locations, are not currently being developed, as their time has passed. Their legacy still lives on, however, in the object-oriented operating systems still in use today.

==Modula-3 Based==
===Overview===

An idea came to mind. A question was proposed. Can an operating system have extensibility, safety, and good performance? A group of computer scientists from Washington University took on this question and tried to come up with an answer. That’s how SPIN operating system was created. SPIN is an operating system that blends the user-level with the kernel level. The main feature is that the kernel can be extended using modules that implement interfaces to meet the applications needs to optimise performance and safety. The programming language used is Modula-3, and the reason for using this language is for its type safe properties like interfaces, extension, and its automatic storage management. The operating system sits on the traditional monolithic kernel. This is because the authors of SPIN argue that microkernels are not extendible due to the massive overhead created from switching in and out of the kernel.

===Motivation===

A poorly matched implementation prevents an
application from working well, while a poorly
matched interface prevents it from working at all.

This is the sole purpose for creating SPIN. The team had to meet three important features to make this work, a Flexible kernel with safe extensions and a good overall performance. Operating systems based on C/C++ are too general that handle many applications. Therefore, some programs may fall through the cracks and suffer the lack of safety and performance. The goal for SPIN however, was to create a specialized operating system that can run a range of applications without sacrificing safety or performance. This system would allow the user programs and applications to manipulate and change the system’s interface to load and access the data or memory needed safely and with a good overall performance. This was possible through the type safe extensions of Modula-3. Unlike C/C++, Modula-3 is considered a safe language; it guarantees the protection of the kernel from dangerous extension by forcing them to interact with the kernel through specific interfaces. Furthermore, the extensions are executed in the virtual address space not the user space.

===Achievements===

SPIN operating system came through with positive results, it actually works. Most of the goals set by the creators were accomplished. SPIN overcame the large overhead of microkernels by enforcing the extensions to be run in the kernel’s address space. The number of switches dropped which actually improved efficiency significantly.
By using Modula-3’s safe interfaces, there was no run-time checking of the code of the extensions. This was a huge advantage over languages like C/C++ due to the fact that a significant overhead was cleared from the execution time. SPIN addressed the latency issue by only allowing code with low latency, which is quite small, to access the system through extensions that access the kernel directly. This is another advantage of using a safe programming language.

===Problems===

Two general issues with extendible systems are adaptability and reliability. Using a safe programming language comes with a price. In SPIN, there is a loss of efficiency when the dynamic checks run for array index bounds. These checks also add a significant overhead to the execution time.
SPIN is written with very few Modula-3 code lines and are much easier to understand compared to other operating systems written in C or C++. However, the kernel of the system is almost entirely written in C and Assembly. These two languages are considered unsafe in contrast to Modula-3. The mix of safe and unsafe code is a huge complication which leads to many bugs, which compromises the safe extensions and interfaces used by applications.

===Current Status===

SPIN, and extendible operating systems in general, is becoming the playground for research. The authors of SPIN at the University of Washington are performing projects and continuing the on-going research of improving extendible operating systems. The solutions they might come up with could greatly influence the operating systems as we see them today.

==Smalltalk Based==
===Overview===

<blockquote>The best way to predict the future is to invent it. -- Alan Kay</blockquote>

If you are reading this paper on a personal computer, and it has a GUI with overlapping windows, desktop icons, and a mouse pointer then you owe a debt to a group of researchers led by Alan Kay at Xerox's Palo Alto Research Center, or Xerox PARC. Many of those ideas had appeared elsewhere in one form or another, but the first time they came together in a demonstrable and portable form was in the early 1970s at Xerox on a machine called the Alto [http://en.wikipedia.org/wiki/Xerox_Alto]. Their example of a working personal computer was to be the inspiration that launched Apple into the history books.

===Motivation===

Searching for operating systems written in Smalltalk is an exercise in recursion. The Smalltalk OS was written in -- what else -- Smalltalk. But Alan Kay's Learning Research Group (LRG) didn't just set out to write a clever new language that could bootstrap its own environment. Their goal was to produce an entirely new learning environment to "amplify human reach and bring new ways of thinking to a faltering civilization that desperately needed it." [http://www.smalltalk.org/smalltalk/TheEarlyHistoryOfSmalltalk_III.html]

===Achievements===

They did manage to invent new ways of think about computing. Alan Kay coined the term "object-oriented programming". His inspiration had come from notions of objects already well known in LISP but he wasn't shy about giving credit where it was due. Kay believed his knowledge of LISP helped him think more clearly about computer problems. He felt it contained "great thinking patterns" and "deep beauty" and he vowed to preserve these qualities in the language that would become Smalltalk.

But Kay wasn't just interested the symbolic language at the heart of a system. He incorporated recent studies on learning and cognition from such experts such as Maria Montessori, Jean Piaget, and Jerome Bruner in his goal to form a complete vision of the personal computer. For instance, Bruner's research implied that people learn new thoughts in a loose sequence of translations [http://en.wikipedia.org/wiki/Jerome_Bruner]. First there are actions, or ''enactive'' representations which transform into images or ''iconic'' representations, and finally into language or ''symbolic'' representations. We may take it for granted today, but it was human insights such as these that helped Kay form the basis of graphical user interfaces as channels through which we perceive ideas represented in a computer.

Still the language is no small point. The central idea in Smalltalk (and Smalltalk-80 as an example of a whole system) is that "everything is an object". This includes what we already might think of as objects, such as files, forms, and fields but it also includes transformational methods such as actions, behaviors, and calculations. This places it in distinct contrast with operating systems written in C, and even those written in C++. Gone are such notions as ''process'' and ''semaphore''. In their place are ''messages'' between objects and ''events'' to which interested objects may react.

Another difference in Smalltalk from C-based operating systems is the lack of security. The only means Smalltalk has of protecting anything is through the visibility (private or public) of an object's methods and properties. But if you know how to dig through the system you can modify -- and break -- just about everything. This may be offer a sense of freedom to some. In the networked world this seems more than a bit risky. Yes, Smalltalk has networking.

===Problems===

Performance in Smalltalk-80 wasn't what we expect from our machines today. A later version of the system called Squeak made some attempt to address this issue (and that of portability) by writing a C language generator to create its virtual machine. This was reasonably successful, and you can now run Squeak on a number of platforms including stand-alone inside of VirtualBox.

[[File:Squeak-Hello-Carleton.png]]

Most non-programmers don't know much about Smalltalk-80 or Squeak though it's tempting to speculate what might have happened if Xerox had chosen to keep its secrets from Apple and commercialize the Alto. The decision not to commercialize was one that hurt Alan Kay's team, one of whom in fact seized the moment and went to work for Apple. By the mid-seventies Kay wasn't sure they had completely solved the problems for which they had set out. In his disillusionment he sought to burn everything and start over. He was concerned he would be fulfilling both claims of Marshall McLuhan, namely that "our tools reshape us" and "inadequate tools ''still'' reshape us", though presumably not always for the better.

===Current Status===

The Smalltalk world continues to have ripples around the world, though perhaps less than the incredible splash they made in the 1970s.

==Java Based==
===Overview===

Java is used on a plethora of devices and systems throughout the industry from cell phones to web applets. With Java being a language that creates a virtual machine for each application, one would think that it is already suited to be an operating system in itself but this is not the case. Java is built to run on top of other operating systems such as Microsoft Windows, Mac OS X and Ubuntu Linux rather than being a standalone system. This section will discuss various operating systems that are written in Java such as JavaOS, Android, JNode and JX.

[http://java.sun.com/developer/products/JavaOS/ JavaOS] is Sun Microsystems very own creation. This system runs on different layers to make a scalable and easily updateable operating system [http://java.sun.com/developer/products/JavaOS/OverView/index.html ]. The first layer is the microkernel which handles the memory architecture, booting, interrupt handling, threading, traps and DMA handling. The Java Virtual Machine is also compiled into native code for the system and is run on top of the microkernel. The second layer consists of the JavaOS for Business software which extends the memory module to optimize for systems with limited memory[http://java.sun.com/developer/products/JavaOS/OverView/index.html ]. All device drivers for the system are written and run in Java and are what the third layer is consisted of. Finally, the fourth layer is a stand-alone JDK runtime environment used to run user applications.

[http://www.android.com/ Android] was created by a very ambitious team at Google for use with cell phones. It is basically an operating system running on top of another minimalistic operating system. At the very lowest end of Android, a Linux 2.6 kernel is what powers it. All device drivers are written and compiled for the native hardware or compiled using Google’s Native Development Kit and core system libraries such as libc, OpenGL | ES and SQLite are dynamically linked in per application [http://developer.android.com/guide/basics/what-is-android.html]. In the Android runtime, the Dalvik Virtual Machine (DVM) controls applications similar to the Java Virtual Machine. Dalvik is similar to the aforementioned JavaOS as it relies on the kernel to manage threading and low-level memory management. Running on top of the DVM are core libraries and application frameworks for the Android operating system. These frameworks include resource management, window management, notification manage just to name a few. Applications are then built on top of these frameworks and are the end result in which the user will actually interact with.

[http://www.jnode.org/ JNode] began as the Java Bootable System (JBS) in 1995. Ewout Prangsma, the creator, was unhappy with the amount of native C and assembly used for the system so he began a new project, JNode. JNode only uses a small amount of assembly code for booting the system now compared to the initial JBS [http://www.jnode.org/node/174]. The rest of the system is completely written in Java including its graphical user interface. Applications in JNode are referred to as plugins [http://www.jnode.org/node/175] including the device drivers, filesystems, networking and user applications.

[http://www4.informatik.uni-erlangen.de/Projects/JX/index.html JX] was created at the University of Erlangen. At it’s base is a microkernel that the basic Java Virtual Machine runs on which is similar to JavaOS. The system runs on the idea of domains where the microkernel runs at domain level zero and subsequent programs run in domain A, B, C, etc. Domains contain their own threads, heap and garbage collector and can communicate with other domains using portals. A portal is essentially the same as inter-process communication but using domains as processes instead [http://www.usenix.org/events/usenix02/full_papers/golm/golm.pdf].

===Motivation===

Java is a powerful language that already contains the code necessary for running on many different platforms. With the concept of virtual machines for each individual application it provides a layer of security that not every operating system has. Each virtual machine has it’s own heap which keeps other processes from accessing and writing over already allocated memory since the kernel manages the memory paging. With the use of just-in-time (JIT) compilers, these operating systems can almost achieve comparable run times when compared to native compiled applications.

Since all Java applications are compiled into the same set of bytecode, third-parties can develop their own implementation of the Java runtime. For example, Google has created the Dalvik Virtual Machine for use with the Android platform so it will run more efficiently on small devices while reducing the memory footprint [http://developer.android.com/guide/basics/what-is-android.html]. Google has just recently released Android 2.2 which includes a new JIT compiler for their Dalvik Virtual Machine which can improve speeds of applications up to 2-5 times [http://android-developers.blogspot.com/2010/05/dalvik-jit.html]. IBM also has their own version of the Java Virtual Machine, J9, that is used in many of their own pieces of software and also includes its own implementation of a JIT compiler.

===Achievements===

Android is probably one of the most well-known and mainstream Java based operating system currently on the market. Even though it’s the youngest of the operating systems discussed, it has become one of the newest standards for smartphones. With over 150 devices and counting, Android continues to grow and develop.

===Problems===

One major issue with using Java for an operating system is completely relying on the operating system and kernel to manage memory. Since Java is a garbage collecting language, developers do not have direct access to the memory and have to rely on the operating system to clean up after any objects have been left behind. This can be an issue with lower memory systems while running multiple applications at the same time.

===Current Status===

Each operating system has been created or worked on in the last ten years but some have either halted development or have not seen a major stable release in quite a while. JavaOS is still being maintained by Oracle after they had purchased Sun Microsystems. JNode has not seen an update in over a year and a half [http://sourceforge.net/projects/jnode/files/]. JX also seems to be at a stand still in development with only a minor update after its 0.1 release [http://www4.informatik.uni-erlangen.de/Projects/JX/download-sources.html]. Finally, Android is the most active with minor or major updates coming out every few months. The current state Android is in is Android 2.2 with Android 3.0 currently being hinted for quarter four of 2010 [http://www.techradar.com/news/phone-and-communications/mobile-phones/android-3-0-details-you-need-to-know-706243].

== References ==

Moon, David A. "Genera Retrospective." ''1991 International Workshop on Object Orientation in Operating Systems'' (1991): 2-8. Print.

Moon, David A., Thomas F. Knight, John T. Holloway, and Richard D. Greenblatt. "A Lisp Machine." ''ACM SIGIR Forum'' 15.2 (1980): 137-38. Print.

Pleszkun, A. R., and M. J. Thazhuthaveetil. "The Architecture of Lisp Machines." ''Computer'' 20.3 (1987): 35-44. Print.

FOR SPIN

B. N. Bershad, S. Savage, P. Pardyak, E. G. Sirer, M. Fiuczynski, D. Becker, S. Eggers, and C. Chambers. Extensibility, Safety and Performance in the SPIN Operating System. In the Proceedings of the 15th ACM Symposium on Operating System Principles (SOSP), Copper Mountain, CO, December 1995.

W. C. Hsieh, M. E. Fiuczynski, C. Garrett, S. Savage, D. Becker and B. N. Bershad. Language Support for Extensible Operating Systems. Workshop on Compiler Support for System Software, University of Washington, February 1996.

S. Savage, B. N. Bershad. Issues in the Design of an Extensible Operating System. Proceedings of the First USENIX Symposium on Operatings Systems Design and Implementation (OSDI), November 1994 (Unpublished).

University of Washington, Dept. of Computer Science. The SPIN Operating System. [http://www.cs.washington.edu.html]

COMP 3000 Essay 1 2010 Question 4

2010-10-15T04:24:03Z

Jjpwilso: /* Current Status */

=Question=

What "operating systems" have been implemented in the following languages: LISP, Modula-3, Smalltalk, Java? To what extent do these systems match the capabilities of operating systems implemented in C and C++?

=Team Note (to be removed by delivery date)=

Please use the [[Talk:COMP_3000_Essay_1_2010_Question_4|discussion page]] for any planning and comments.

=Answer=

== Introduction ==

Not so long ago people believed the Earth was a flat world at the center of the universe. This essay addresses a more recent falsehood: that all operating systems are written in assembly language and C. It's not surprising that students of computing in this century would genuflect at the academic altars of Kernighan & Ritchie. After all we grew up with our computer worlds already pre-formed into the conceptual continents of Apple OS, UNIX, and Windows. The more historically curious among us are vaguely aware that other island cultures do exist but they represent civilizations defeated in the marketplace. Explorations into these ancient worlds resemble documentaries about archeologists decoding rediscovered languages etched in stone. But scratch the surface of any of our so-called modern operating systems and you'll find echoes of these ancient languages in our own familiar worlds. Ellen Ullman said it best when she wrote, <blockquote>''We build our computer systems like we build our cities: over time, without a plan, on top of ruins.''</blockquote>

In the sections below we present our explorations into a few truly foundational operating systems. We will also see some brand new ones that prove we stand, as one twelfth century scholar put it, [http://en.wikipedia.org/wiki/Standing_on_the_shoulders_of_giants on the shoulders of giants].

The following is a short list of operating systems written partially or completely with the given language. We will go into more detail in the language-specific sections below.

{| class="wikitable" style="text-align:center" border="1"
|+ '''Operating Systems By Language'''
|-
!width="15%"|Language
!width="85%"|OS List
|-
! Lisp
| MIT's Lisp Machines, Genera
|-
! Modula-3
| SPIN OS, Oberon
|-
! Smalltalk
| Smalltalk-80 on Xerox Alto, Squeak
|-
! Java
| JavaOS, JNode, JX, Android
|}

==Lisp Based==
===Overview===
Lisp is the second oldest programming languages, established in the late 1960's as a list-processing language. It started out being perfect for math but when it came to programming with it, it left a lot to be desired. Since then there have been a number of different versions of it, from the first Lisp – called “pure Lisp” – to Scheme and Common Lisp. Unlike most object-oriented languages which are focused on classes and instances among other things, Lisp was initially focused more on functions, in which functions call other functions, though later versions of Lisp did eventually add in classes and other typical object-oriented features. It was after this that Genera was created, an operating system written entirely in Lisp, and developed from an earlier operating system on MIT's Lisp Machine.

The Lisp Machine project in began in 1974, where Richard Greenblatt and Thomas Knight, programmers at MIT, designed a computer for general use that was designed for a single user, in which the operating system was programmed entirely in Lisp. It was designed to be completely open to the user, so any changes the user wanted to make could be done during run-time. It was also one of the first systems to be programmed in a 32-bit architecture, making it far more versatile than a number of other operating systems at the time. Despite this, however, the first Lisp Machine was quite basic compared to later iterations, namely Genera.

===Motivation===
Genera is an evolution of the initial Lisp Machine operating system, and as such improved upon many of the basic features of the system. Genera was the first object-oriented operating system, making it stand out among the already well-known Lisp Machines, especially given the fact that it isn't a composition of languages like other operating systems – most using the primary language for the operating system and another language, such as assembly, to deal with hardware directly – but written entirely in Lisp – any of five different dialects of the language – including the lowest levels of the operating system that interact with hardware due to its effective lack of a kernel. Not having to deal with a kernel allows Genera to be very powerful and efficient in certain areas, most especially in the most complex applications such as those dealing with intelligent CAD and graphics, though when given very simple, very routine applications or processes it is much less efficient than other operating systems.

===Achievements===
Another primary difference from other operating systems is that Genera is a fully open system like the first Lisp Machines. Having an open system has its positive and negative aspects. Among the foremost positive aspects is that the user is in control and not left to the whims of the operating system as is common in non-open systems – if the user doesn't like how something is happening, they can simply go in and change it from happening that way again – and allows for very quick prototyping, not needing to wait to compile every little change. Also, due to utilizing very lightweight objects and the fact that everything is in a single address space rather than split across multiple spaces greatly increases efficiency, though again mainly in non-simple tasks.

Like operating systems written in C/C++, Genera has multiple inheritance and automatic garbage collection. Unlike C/C++, however, Genera uses object-oriented memory. This gives Genera a very useful feature that many other systems cannot do, which is that, given a memory address, the system is able to provide the start address, size, and the type of object in that memory location, which increases efficiency. Even the way files are accessed is very different from other operating systems, as it uses a generic file access system; in a generic file access system, rather than following a typical structure of keeping the commands needed for local and networked files different, the commands are the same, meaning that the system essentially acts like a file on a system on the other side of the world is a local file.

===Problems===
Not everything is great with an open system, however, as there are definite security concerns: with an open system, the user can change anything, and programs can interact with other programs and change anything, meaning that if a virus were unleashed on a Genera system it could be catastrophic. To counter this problem Genera systems are protected by a very simple method: keeping the physical system itself out of reach of all but authorized users and behind locked doors. To compare to other operating systems, such as those written in C/C++, it is generally less secure because of the open system, though they are both equally at risk when installing software, as both systems are completely open at that specific junction. Genera also has issues with modularity, as it allows for something to be called even when it is not supposed to be able to be called because, as it is an open system, there is nothing really stopping this from happening. Some versions of Genera and Genera applications are also very well-documented and run quite smoothly, but others are poorly documented and run quite slowly as they are built on top of older versions without any real plan beyond adding on an extra feature here or there over and over again.

===Current Status===
Genera and Lisp Machines, while still being used in a select few locations, are not currently being developed, as their time has passed. Their legacy still lives on, however, in the object-oriented operating systems still in use today.

==Modula-3 Based==
===Overview===

An idea came to mind. A question was proposed. Can an operating system have extensibility, safety, and good performance? A group of computer scientists from Washington University took on this question and tried to come up with an answer. That’s how SPIN operating system was created. SPIN is an operating system that blends the user-level with the kernel level. The main feature is that the kernel can be extended using modules that implement interfaces to meet the applications needs to optimise performance and safety. The programming language used is Modula-3, and the reason for using this language is for its type safe properties like interfaces, extension, and its automatic storage management. The operating system sits on the traditional monolithic kernel. This is because the authors of SPIN argue that microkernels are not extendible due to the massive overhead created from switching in and out of the kernel.

===Motivation===

A poorly matched implementation prevents an
application from working well, while a poorly
matched interface prevents it from working at all.

This is the sole purpose for creating SPIN. The team had to meet three important features to make this work, a Flexible kernel with safe extensions and a good overall performance. Operating systems based on C/C++ are too general that handle many applications. Therefore, some programs may fall through the cracks and suffer the lack of safety and performance. The goal for SPIN however, was to create a specialized operating system that can run a range of applications without sacrificing safety or performance. This system would allow the user programs and applications to manipulate and change the system’s interface to load and access the data or memory needed safely and with a good overall performance. This was possible through the type safe extensions of Modula-3. Unlike C/C++, Modula-3 is considered a safe language; it guarantees the protection of the kernel from dangerous extension by forcing them to interact with the kernel through specific interfaces. Furthermore, the extensions are executed in the virtual address space not the user space.

===Achievements===

SPIN operating system came through with positive results, it actually works. Most of the goals set by the creators were accomplished. SPIN overcame the large overhead of microkernels by enforcing the extensions to be run in the kernel’s address space. The number of switches dropped which actually improved efficiency significantly.
By using Modula-3’s safe interfaces, there was no run-time checking of the code of the extensions. This was a huge advantage over languages like C/C++ due to the fact that a significant overhead was cleared from the execution time. SPIN addressed the latency issue by only allowing code with low latency, which is quite small, to access the system through extensions that access the kernel directly. This is another advantage of using a safe programming language.

===Problems===

Two general issues with extendible systems are adaptability and reliability. Using a safe programming language comes with a price. In SPIN, there is a loss of efficiency when the dynamic checks run for array index bounds. These checks also add a significant overhead to the execution time.
SPIN is written with very few Modula-3 code lines and are much easier to understand compared to other operating systems written in C or C++. However, the kernel of the system is almost entirely written in C and Assembly. These two languages are considered unsafe in contrast to Modula-3. The mix of safe and unsafe code is a huge complication which leads to many bugs, which compromises the safe extensions and interfaces used by applications.

===Current Status===

SPIN, and extendible operating systems in general, is becoming the playground for research. The authors of SPIN at the University of Washington are performing projects and continuing the on-going research of improving extendible operating systems. The solutions they might come up with could greatly influence the operating systems as we see them today.

==Smalltalk Based==
===Overview===

<blockquote>The best way to predict the future is to invent it. -- Alan Kay</blockquote>

If you are reading this paper on a personal computer, and it has a GUI with overlapping windows, desktop icons, and a mouse pointer then you owe a debt to a group of researchers led by Alan Kay at Xerox's Palo Alto Research Center, or Xerox PARC. Many of those ideas had appeared elsewhere in one form or another, but the first time they came together in a demonstrable and portable form was in the early 1970s at Xerox on a machine called the Alto [http://en.wikipedia.org/wiki/Xerox_Alto]. Their example of a working personal computer was to be the inspiration that launched Apple into the history books.

===Motivation===

Searching for operating systems written in Smalltalk is an exercise in recursion. The Smalltalk OS was written in -- what else -- Smalltalk. But Alan Kay's Learning Research Group (LRG) didn't just set out to write a clever new language that could bootstrap its own environment. Their goal was to produce an entirely new learning environment to "amplify human reach and bring new ways of thinking to a faltering civilization that desperately needed it." [http://www.smalltalk.org/smalltalk/TheEarlyHistoryOfSmalltalk_III.html]

===Achievements===

They did manage to invent new ways of think about computing. Alan Kay coined the term "object-oriented programming". His inspiration had come from notions of objects already well known in LISP but he wasn't shy about giving credit where it was due. Kay believed his knowledge of LISP helped him think more clearly about computer problems. He felt it contained "great thinking patterns" and "deep beauty" and he vowed to preserve these qualities in the language that would become Smalltalk.

But Kay wasn't just interested the symbolic language at the heart of a system. He incorporated recent studies on learning and cognition from such experts such as Maria Montessori, Jean Piaget, and Jerome Bruner in his goal to form a complete vision of the personal computer. For instance, Bruner's research implied that people learn new thoughts in a loose sequence of translations [http://en.wikipedia.org/wiki/Jerome_Bruner]. First there are actions, or ''enactive'' representations which transform into images or ''iconic'' representations, and finally into language or ''symbolic'' representations. We may take it for granted today, but it was human insights such as these that helped Kay form the basis of graphical user interfaces as channels through which we perceive ideas represented in a computer.

Still the language is no small point. The central idea in Smalltalk (and Smalltalk-80 as an example of a whole system) is that "everything is an object". This includes what we already might think of as objects, such as files, forms, and fields but it also includes transformational methods such as actions, behaviors, and calculations. This places it in distinct contrast with operating systems written in C, and even those written in C++. Gone are such notions as ''process'' and ''semaphore''. In their place are ''messages'' between objects and ''events'' to which interested objects may react.

Another difference in Smalltalk from C-based operating systems is the lack of security. The only means Smalltalk has of protecting anything is through the visibility (private or public) of an object's methods and properties. But if you know how to dig through the system you can modify -- and break -- just about everything. This may be offer a sense of freedom to some. In the networked world this seems more than a bit risky. Yes, Smalltalk has networking.

===Problems===

Performance in Smalltalk-80 wasn't what we expect from our machines today. A later version of the system called Squeak made some attempt to address this issue (and that of portability) by writing a C language generator to create its virtual machine. This was reasonably successful, and you can now run Squeak on a number of platforms including stand-alone inside of VirtualBox.

[[File:Squeak-Hello-Carleton.png]]

Most non-programmers don't know much about Smalltalk-80 or Squeak though it's tempting to speculate what might have happened if Xerox had chosen to keep its secrets from Apple and commercialize the Alto. The decision not to commercialize was one that hurt Alan Kay's team, one of whom in fact seized the moment and went to work for Apple. By the mid-seventies Kay wasn't sure they had completely solved the problems for which they had set. In his disillusionment he sought to burn everything and start over. He was concerned he would be fulfilling both claims of Marshall McLuhan, namely that "our tools reshape us" and "inadequate tools ''still'' reshape us", though presumably not always for the better.

===Current Status===

The Smalltalk world continues to have ripples around the world, though perhaps less than the incredible splash they made in the 1970s.

==Java Based==
===Overview===

Java is used on a plethora of devices and systems throughout the industry from cell phones to web applets. With Java being a language that creates a virtual machine for each application, one would think that it is already suited to be an operating system in itself but this is not the case. Java is built to run on top of other operating systems such as Microsoft Windows, Mac OS X and Ubuntu Linux rather than being a standalone system. This section will discuss various operating systems that are written in Java such as JavaOS, Android, JNode and JX.

[http://java.sun.com/developer/products/JavaOS/ JavaOS] is Sun Microsystems very own creation. This system runs on different layers to make a scalable and easily updateable operating system [http://java.sun.com/developer/products/JavaOS/OverView/index.html ]. The first layer is the microkernel which handles the memory architecture, booting, interrupt handling, threading, traps and DMA handling. The Java Virtual Machine is also compiled into native code for the system and is run on top of the microkernel. The second layer consists of the JavaOS for Business software which extends the memory module to optimize for systems with limited memory[http://java.sun.com/developer/products/JavaOS/OverView/index.html ]. All device drivers for the system are written and run in Java and are what the third layer is consisted of. Finally, the fourth layer is a stand-alone JDK runtime environment used to run user applications.

[http://www.android.com/ Android] was created by a very ambitious team at Google for use with cell phones. It is basically an operating system running on top of another minimalistic operating system. At the very lowest end of Android, a Linux 2.6 kernel is what powers it. All device drivers are written and compiled for the native hardware or compiled using Google’s Native Development Kit and core system libraries such as libc, OpenGL | ES and SQLite are dynamically linked in per application [http://developer.android.com/guide/basics/what-is-android.html]. In the Android runtime, the Dalvik Virtual Machine (DVM) controls applications similar to the Java Virtual Machine. Dalvik is similar to the aforementioned JavaOS as it relies on the kernel to manage threading and low-level memory management. Running on top of the DVM are core libraries and application frameworks for the Android operating system. These frameworks include resource management, window management, notification manage just to name a few. Applications are then built on top of these frameworks and are the end result in which the user will actually interact with.

[http://www.jnode.org/ JNode] began as the Java Bootable System (JBS) in 1995. Ewout Prangsma, the creator, was unhappy with the amount of native C and assembly used for the system so he began a new project, JNode. JNode only uses a small amount of assembly code for booting the system now compared to the initial JBS [http://www.jnode.org/node/174]. The rest of the system is completely written in Java including its graphical user interface. Applications in JNode are referred to as plugins [http://www.jnode.org/node/175] including the device drivers, filesystems, networking and user applications.

[http://www4.informatik.uni-erlangen.de/Projects/JX/index.html JX] was created at the University of Erlangen. At it’s base is a microkernel that the basic Java Virtual Machine runs on which is similar to JavaOS. The system runs on the idea of domains where the microkernel runs at domain level zero and subsequent programs run in domain A, B, C, etc. Domains contain their own threads, heap and garbage collector and can communicate with other domains using portals. A portal is essentially the same as inter-process communication but using domains as processes instead [http://www.usenix.org/events/usenix02/full_papers/golm/golm.pdf].

===Motivation===

Java is a powerful language that already contains the code necessary for running on many different platforms. With the concept of virtual machines for each individual application it provides a layer of security that not every operating system has. Each virtual machine has it’s own heap which keeps other processes from accessing and writing over already allocated memory since the kernel manages the memory paging. With the use of just-in-time (JIT) compilers, these operating systems can almost achieve comparable run times when compared to native compiled applications.

Since all Java applications are compiled into the same set of bytecode, third-parties can develop their own implementation of the Java runtime. For example, Google has created the Dalvik Virtual Machine for use with the Android platform so it will run more efficiently on small devices while reducing the memory footprint [http://developer.android.com/guide/basics/what-is-android.html]. Google has just recently released Android 2.2 which includes a new JIT compiler for their Dalvik Virtual Machine which can improve speeds of applications up to 2-5 times [http://android-developers.blogspot.com/2010/05/dalvik-jit.html]. IBM also has their own version of the Java Virtual Machine, J9, that is used in many of their own pieces of software and also includes its own implementation of a JIT compiler.

===Achievements===

Android is probably one of the most well-known and mainstream Java based operating system currently on the market. Even though it’s the youngest of the operating systems discussed, it has become one of the newest standards for smartphones. With over 150 devices and counting, Android continues to grow and develop.

===Problems===

One major issue with using Java for an operating system is completely relying on the operating system and kernel to manage memory. Since Java is a garbage collecting language, developers do not have direct access to the memory and have to rely on the operating system to clean up after any objects have been left behind. This can be an issue with lower memory systems while running multiple applications at the same time.

===Current Status===

Each operating system has been created or worked on in the last ten years but some have either halted development or have not seen a major stable release in quite a while. JavaOS is still being maintained by Oracle after they had purchased Sun Microsystems. JNode has not seen an update in over a year and a half [http://sourceforge.net/projects/jnode/files/]. JX also seems to be at a stand still in development with only a minor update after its 0.1 release [http://www4.informatik.uni-erlangen.de/Projects/JX/download-sources.html]. Finally, Android is the most active with minor or major updates coming out every few months. The current state Android is in is Android 2.2 with Android 3.0 currently being hinted for quarter four of 2010 [http://www.techradar.com/news/phone-and-communications/mobile-phones/android-3-0-details-you-need-to-know-706243].

== References ==

Moon, David A. "Genera Retrospective." ''1991 International Workshop on Object Orientation in Operating Systems'' (1991): 2-8. Print.

Moon, David A., Thomas F. Knight, John T. Holloway, and Richard D. Greenblatt. "A Lisp Machine." ''ACM SIGIR Forum'' 15.2 (1980): 137-38. Print.

Pleszkun, A. R., and M. J. Thazhuthaveetil. "The Architecture of Lisp Machines." ''Computer'' 20.3 (1987): 35-44. Print.

FOR SPIN

B. N. Bershad, S. Savage, P. Pardyak, E. G. Sirer, M. Fiuczynski, D. Becker, S. Eggers, and C. Chambers. Extensibility, Safety and Performance in the SPIN Operating System. In the Proceedings of the 15th ACM Symposium on Operating System Principles (SOSP), Copper Mountain, CO, December 1995.

W. C. Hsieh, M. E. Fiuczynski, C. Garrett, S. Savage, D. Becker and B. N. Bershad. Language Support for Extensible Operating Systems. Workshop on Compiler Support for System Software, University of Washington, February 1996.

S. Savage, B. N. Bershad. Issues in the Design of an Extensible Operating System. Proceedings of the First USENIX Symposium on Operatings Systems Design and Implementation (OSDI), November 1994 (Unpublished).

University of Washington, Dept. of Computer Science. The SPIN Operating System. [http://www.cs.washington.edu.html]

COMP 3000 Essay 1 2010 Question 4

2010-10-15T04:20:32Z

Jjpwilso: /* Motivation */

=Question=

What "operating systems" have been implemented in the following languages: LISP, Modula-3, Smalltalk, Java? To what extent do these systems match the capabilities of operating systems implemented in C and C++?

=Team Note (to be removed by delivery date)=

Please use the [[Talk:COMP_3000_Essay_1_2010_Question_4|discussion page]] for any planning and comments.

=Answer=

== Introduction ==

Not so long ago people believed the Earth was a flat world at the center of the universe. This essay addresses a more recent falsehood: that all operating systems are written in assembly language and C. It's not surprising that students of computing in this century would genuflect at the academic altars of Kernighan & Ritchie. After all we grew up with our computer worlds already pre-formed into the conceptual continents of Apple OS, UNIX, and Windows. The more historically curious among us are vaguely aware that other island cultures do exist but they represent civilizations defeated in the marketplace. Explorations into these ancient worlds resemble documentaries about archeologists decoding rediscovered languages etched in stone. But scratch the surface of any of our so-called modern operating systems and you'll find echoes of these ancient languages in our own familiar worlds. Ellen Ullman said it best when she wrote, <blockquote>''We build our computer systems like we build our cities: over time, without a plan, on top of ruins.''</blockquote>

In the sections below we present our explorations into a few truly foundational operating systems. We will also see some brand new ones that prove we stand, as one twelfth century scholar put it, [http://en.wikipedia.org/wiki/Standing_on_the_shoulders_of_giants on the shoulders of giants].

The following is a short list of operating systems written partially or completely with the given language. We will go into more detail in the language-specific sections below.

{| class="wikitable" style="text-align:center" border="1"
|+ '''Operating Systems By Language'''
|-
!width="15%"|Language
!width="85%"|OS List
|-
! Lisp
| MIT's Lisp Machines, Genera
|-
! Modula-3
| SPIN OS, Oberon
|-
! Smalltalk
| Smalltalk-80 on Xerox Alto, Squeak
|-
! Java
| JavaOS, JNode, JX, Android
|}

==Lisp Based==
===Overview===
Lisp is the second oldest programming languages, established in the late 1960's as a list-processing language. It started out being perfect for math but when it came to programming with it, it left a lot to be desired. Since then there have been a number of different versions of it, from the first Lisp – called “pure Lisp” – to Scheme and Common Lisp. Unlike most object-oriented languages which are focused on classes and instances among other things, Lisp was initially focused more on functions, in which functions call other functions, though later versions of Lisp did eventually add in classes and other typical object-oriented features. It was after this that Genera was created, an operating system written entirely in Lisp, and developed from an earlier operating system on MIT's Lisp Machine.

The Lisp Machine project in began in 1974, where Richard Greenblatt and Thomas Knight, programmers at MIT, designed a computer for general use that was designed for a single user, in which the operating system was programmed entirely in Lisp. It was designed to be completely open to the user, so any changes the user wanted to make could be done during run-time. It was also one of the first systems to be programmed in a 32-bit architecture, making it far more versatile than a number of other operating systems at the time. Despite this, however, the first Lisp Machine was quite basic compared to later iterations, namely Genera.

===Motivation===
Genera is an evolution of the initial Lisp Machine operating system, and as such improved upon many of the basic features of the system. Genera was the first object-oriented operating system, making it stand out among the already well-known Lisp Machines, especially given the fact that it isn't a composition of languages like other operating systems – most using the primary language for the operating system and another language, such as assembly, to deal with hardware directly – but written entirely in Lisp – any of five different dialects of the language – including the lowest levels of the operating system that interact with hardware due to its effective lack of a kernel. Not having to deal with a kernel allows Genera to be very powerful and efficient in certain areas, most especially in the most complex applications such as those dealing with intelligent CAD and graphics, though when given very simple, very routine applications or processes it is much less efficient than other operating systems.

===Achievements===
Another primary difference from other operating systems is that Genera is a fully open system like the first Lisp Machines. Having an open system has its positive and negative aspects. Among the foremost positive aspects is that the user is in control and not left to the whims of the operating system as is common in non-open systems – if the user doesn't like how something is happening, they can simply go in and change it from happening that way again – and allows for very quick prototyping, not needing to wait to compile every little change. Also, due to utilizing very lightweight objects and the fact that everything is in a single address space rather than split across multiple spaces greatly increases efficiency, though again mainly in non-simple tasks.

Like operating systems written in C/C++, Genera has multiple inheritance and automatic garbage collection. Unlike C/C++, however, Genera uses object-oriented memory. This gives Genera a very useful feature that many other systems cannot do, which is that, given a memory address, the system is able to provide the start address, size, and the type of object in that memory location, which increases efficiency. Even the way files are accessed is very different from other operating systems, as it uses a generic file access system; in a generic file access system, rather than following a typical structure of keeping the commands needed for local and networked files different, the commands are the same, meaning that the system essentially acts like a file on a system on the other side of the world is a local file.

===Problems===
Not everything is great with an open system, however, as there are definite security concerns: with an open system, the user can change anything, and programs can interact with other programs and change anything, meaning that if a virus were unleashed on a Genera system it could be catastrophic. To counter this problem Genera systems are protected by a very simple method: keeping the physical system itself out of reach of all but authorized users and behind locked doors. To compare to other operating systems, such as those written in C/C++, it is generally less secure because of the open system, though they are both equally at risk when installing software, as both systems are completely open at that specific junction. Genera also has issues with modularity, as it allows for something to be called even when it is not supposed to be able to be called because, as it is an open system, there is nothing really stopping this from happening. Some versions of Genera and Genera applications are also very well-documented and run quite smoothly, but others are poorly documented and run quite slowly as they are built on top of older versions without any real plan beyond adding on an extra feature here or there over and over again.

===Current Status===
Genera and Lisp Machines, while still being used in a select few locations, are not currently being developed, as their time has passed. Their legacy still lives on, however, in the object-oriented operating systems still in use today.

==Modula-3 Based==
===Overview===

An idea came to mind. A question was proposed. Can an operating system have extensibility, safety, and good performance? A group of computer scientists from Washington University took on this question and tried to come up with an answer. That’s how SPIN operating system was created. SPIN is an operating system that blends the user-level with the kernel level. The main feature is that the kernel can be extended using modules that implement interfaces to meet the applications needs to optimise performance and safety. The programming language used is Modula-3, and the reason for using this language is for its type safe properties like interfaces, extension, and its automatic storage management. The operating system sits on the traditional monolithic kernel. This is because the authors of SPIN argue that microkernels are not extendible due to the massive overhead created from switching in and out of the kernel.

===Motivation===

A poorly matched implementation prevents an
application from working well, while a poorly
matched interface prevents it from working at all.

This is the sole purpose for creating SPIN. The team had to meet three important features to make this work, a Flexible kernel with safe extensions and a good overall performance. Operating systems based on C/C++ are too general that handle many applications. Therefore, some programs may fall through the cracks and suffer the lack of safety and performance. The goal for SPIN however, was to create a specialized operating system that can run a range of applications without sacrificing safety or performance. This system would allow the user programs and applications to manipulate and change the system’s interface to load and access the data or memory needed safely and with a good overall performance. This was possible through the type safe extensions of Modula-3. Unlike C/C++, Modula-3 is considered a safe language; it guarantees the protection of the kernel from dangerous extension by forcing them to interact with the kernel through specific interfaces. Furthermore, the extensions are executed in the virtual address space not the user space.

===Achievements===

SPIN operating system came through with positive results, it actually works. Most of the goals set by the creators were accomplished. SPIN overcame the large overhead of microkernels by enforcing the extensions to be run in the kernel’s address space. The number of switches dropped which actually improved efficiency significantly.
By using Modula-3’s safe interfaces, there was no run-time checking of the code of the extensions. This was a huge advantage over languages like C/C++ due to the fact that a significant overhead was cleared from the execution time. SPIN addressed the latency issue by only allowing code with low latency, which is quite small, to access the system through extensions that access the kernel directly. This is another advantage of using a safe programming language.

===Problems===

Two general issues with extendible systems are adaptability and reliability. Using a safe programming language comes with a price. In SPIN, there is a loss of efficiency when the dynamic checks run for array index bounds. These checks also add a significant overhead to the execution time.
SPIN is written with very few Modula-3 code lines and are much easier to understand compared to other operating systems written in C or C++. However, the kernel of the system is almost entirely written in C and Assembly. These two languages are considered unsafe in contrast to Modula-3. The mix of safe and unsafe code is a huge complication which leads to many bugs, which compromises the safe extensions and interfaces used by applications.

===Current Status===

SPIN, and extendible operating systems in general, is becoming the playground for research. The authors of SPIN at the University of Washington are performing projects and continuing the on-going research of improving extendible operating systems. The solutions they might come up with could greatly influence the operating systems as we see them today.

==Smalltalk Based==
===Overview===

<blockquote>The best way to predict the future is to invent it. -- Alan Kay</blockquote>

If you are reading this paper on a personal computer, and it has a GUI with overlapping windows, desktop icons, and a mouse pointer then you owe a debt to a group of researchers led by Alan Kay at Xerox's Palo Alto Research Center, or Xerox PARC. Many of those ideas had appeared elsewhere in one form or another, but the first time they came together in a demonstrable and portable form was in the early 1970s at Xerox on a machine called the Alto [http://en.wikipedia.org/wiki/Xerox_Alto]. Their example of a working personal computer was to be the inspiration that launched Apple into the history books.

===Motivation===

Searching for operating systems written in Smalltalk is an exercise in recursion. The Smalltalk OS was written in -- what else -- Smalltalk. But Alan Kay's Learning Research Group (LRG) didn't just set out to write a clever new language that could bootstrap its own environment. Their goal was to produce an entirely new learning environment to "amplify human reach and bring new ways of thinking to a faltering civilization that desperately needed it." [http://www.smalltalk.org/smalltalk/TheEarlyHistoryOfSmalltalk_III.html]

===Achievements===

They did manage to invent new ways of think about computing. Alan Kay coined the term "object-oriented programming". His inspiration had come from notions of objects already well known in LISP but he wasn't shy about giving credit where it was due. Kay believed his knowledge of LISP helped him think more clearly about computer problems. He felt it contained "great thinking patterns" and "deep beauty" and he vowed to preserve these qualities in the language that would become Smalltalk.

But Kay wasn't just interested the symbolic language at the heart of a system. He incorporated recent studies on learning and cognition from such experts such as Maria Montessori, Jean Piaget, and Jerome Bruner in his goal to form a complete vision of the personal computer. For instance, Bruner's research implied that people learn new thoughts in a loose sequence of translations [http://en.wikipedia.org/wiki/Jerome_Bruner]. First there are actions, or ''enactive'' representations which transform into images or ''iconic'' representations, and finally into language or ''symbolic'' representations. We may take it for granted today, but it was human insights such as these that helped Kay form the basis of graphical user interfaces as channels through which we perceive ideas represented in a computer.

Still the language is no small point. The central idea in Smalltalk (and Smalltalk-80 as an example of a whole system) is that "everything is an object". This includes what we already might think of as objects, such as files, forms, and fields but it also includes transformational methods such as actions, behaviors, and calculations. This places it in distinct contrast with operating systems written in C, and even those written in C++. Gone are such notions as ''process'' and ''semaphore''. In their place are ''messages'' between objects and ''events'' to which interested objects may react.

Another difference in Smalltalk from C-based operating systems is the lack of security. The only means Smalltalk has of protecting anything is through the visibility (private or public) of an object's methods and properties. But if you know how to dig through the system you can modify -- and break -- just about everything. This may be offer a sense of freedom to some. In the networked world this seems more than a bit risky. Yes, Smalltalk has networking.

===Problems===

Performance in Smalltalk-80 wasn't what we expect from our machines today. A later version of the system called Squeak made some attempt to address this issue (and that of portability) by writing a C language generator to create its virtual machine. This was reasonably successful, and you can now run Squeak on a number of platforms including stand-alone inside of VirtualBox.

[[File:Squeak-Hello-Carleton.png]]

Most non-programmers don't know much about Smalltalk-80 or Squeak though it's tempting to speculate what might have happened if Xerox had chosen to keep its secrets from Apple and commercialize the Alto. The decision not to commercialize was one that hurt Alan Kay's team, one of whom in fact seized the moment and went to work for Apple. By the mid-seventies Kay wasn't sure they had completely solved the problems for which they had set. In his disillusionment he sought to burn everything and start over. He was concerned he would be fulfilling both claims of Marshall McLuhan, namely that "our tools reshape us" and "inadequate tools ''still'' reshape us", though presumably not always for the better.

===Current Status===

* Continuing Research
* OLPC

==Java Based==
===Overview===

Java is used on a plethora of devices and systems throughout the industry from cell phones to web applets. With Java being a language that creates a virtual machine for each application, one would think that it is already suited to be an operating system in itself but this is not the case. Java is built to run on top of other operating systems such as Microsoft Windows, Mac OS X and Ubuntu Linux rather than being a standalone system. This section will discuss various operating systems that are written in Java such as JavaOS, Android, JNode and JX.

[http://java.sun.com/developer/products/JavaOS/ JavaOS] is Sun Microsystems very own creation. This system runs on different layers to make a scalable and easily updateable operating system [http://java.sun.com/developer/products/JavaOS/OverView/index.html ]. The first layer is the microkernel which handles the memory architecture, booting, interrupt handling, threading, traps and DMA handling. The Java Virtual Machine is also compiled into native code for the system and is run on top of the microkernel. The second layer consists of the JavaOS for Business software which extends the memory module to optimize for systems with limited memory[http://java.sun.com/developer/products/JavaOS/OverView/index.html ]. All device drivers for the system are written and run in Java and are what the third layer is consisted of. Finally, the fourth layer is a stand-alone JDK runtime environment used to run user applications.

[http://www.android.com/ Android] was created by a very ambitious team at Google for use with cell phones. It is basically an operating system running on top of another minimalistic operating system. At the very lowest end of Android, a Linux 2.6 kernel is what powers it. All device drivers are written and compiled for the native hardware or compiled using Google’s Native Development Kit and core system libraries such as libc, OpenGL | ES and SQLite are dynamically linked in per application [http://developer.android.com/guide/basics/what-is-android.html]. In the Android runtime, the Dalvik Virtual Machine (DVM) controls applications similar to the Java Virtual Machine. Dalvik is similar to the aforementioned JavaOS as it relies on the kernel to manage threading and low-level memory management. Running on top of the DVM are core libraries and application frameworks for the Android operating system. These frameworks include resource management, window management, notification manage just to name a few. Applications are then built on top of these frameworks and are the end result in which the user will actually interact with.

[http://www.jnode.org/ JNode] began as the Java Bootable System (JBS) in 1995. Ewout Prangsma, the creator, was unhappy with the amount of native C and assembly used for the system so he began a new project, JNode. JNode only uses a small amount of assembly code for booting the system now compared to the initial JBS [http://www.jnode.org/node/174]. The rest of the system is completely written in Java including its graphical user interface. Applications in JNode are referred to as plugins [http://www.jnode.org/node/175] including the device drivers, filesystems, networking and user applications.

[http://www4.informatik.uni-erlangen.de/Projects/JX/index.html JX] was created at the University of Erlangen. At it’s base is a microkernel that the basic Java Virtual Machine runs on which is similar to JavaOS. The system runs on the idea of domains where the microkernel runs at domain level zero and subsequent programs run in domain A, B, C, etc. Domains contain their own threads, heap and garbage collector and can communicate with other domains using portals. A portal is essentially the same as inter-process communication but using domains as processes instead [http://www.usenix.org/events/usenix02/full_papers/golm/golm.pdf].

===Motivation===

Java is a powerful language that already contains the code necessary for running on many different platforms. With the concept of virtual machines for each individual application it provides a layer of security that not every operating system has. Each virtual machine has it’s own heap which keeps other processes from accessing and writing over already allocated memory since the kernel manages the memory paging. With the use of just-in-time (JIT) compilers, these operating systems can almost achieve comparable run times when compared to native compiled applications.

Since all Java applications are compiled into the same set of bytecode, third-parties can develop their own implementation of the Java runtime. For example, Google has created the Dalvik Virtual Machine for use with the Android platform so it will run more efficiently on small devices while reducing the memory footprint [http://developer.android.com/guide/basics/what-is-android.html]. Google has just recently released Android 2.2 which includes a new JIT compiler for their Dalvik Virtual Machine which can improve speeds of applications up to 2-5 times [http://android-developers.blogspot.com/2010/05/dalvik-jit.html]. IBM also has their own version of the Java Virtual Machine, J9, that is used in many of their own pieces of software and also includes its own implementation of a JIT compiler.

===Achievements===

Android is probably one of the most well-known and mainstream Java based operating system currently on the market. Even though it’s the youngest of the operating systems discussed, it has become one of the newest standards for smartphones. With over 150 devices and counting, Android continues to grow and develop.

===Problems===

One major issue with using Java for an operating system is completely relying on the operating system and kernel to manage memory. Since Java is a garbage collecting language, developers do not have direct access to the memory and have to rely on the operating system to clean up after any objects have been left behind. This can be an issue with lower memory systems while running multiple applications at the same time.

===Current Status===

Each operating system has been created or worked on in the last ten years but some have either halted development or have not seen a major stable release in quite a while. JavaOS is still being maintained by Oracle after they had purchased Sun Microsystems. JNode has not seen an update in over a year and a half [http://sourceforge.net/projects/jnode/files/]. JX also seems to be at a stand still in development with only a minor update after its 0.1 release [http://www4.informatik.uni-erlangen.de/Projects/JX/download-sources.html]. Finally, Android is the most active with minor or major updates coming out every few months. The current state Android is in is Android 2.2 with Android 3.0 currently being hinted for quarter four of 2010 [http://www.techradar.com/news/phone-and-communications/mobile-phones/android-3-0-details-you-need-to-know-706243].

== References ==

Moon, David A. "Genera Retrospective." ''1991 International Workshop on Object Orientation in Operating Systems'' (1991): 2-8. Print.

Moon, David A., Thomas F. Knight, John T. Holloway, and Richard D. Greenblatt. "A Lisp Machine." ''ACM SIGIR Forum'' 15.2 (1980): 137-38. Print.

Pleszkun, A. R., and M. J. Thazhuthaveetil. "The Architecture of Lisp Machines." ''Computer'' 20.3 (1987): 35-44. Print.

FOR SPIN

B. N. Bershad, S. Savage, P. Pardyak, E. G. Sirer, M. Fiuczynski, D. Becker, S. Eggers, and C. Chambers. Extensibility, Safety and Performance in the SPIN Operating System. In the Proceedings of the 15th ACM Symposium on Operating System Principles (SOSP), Copper Mountain, CO, December 1995.

W. C. Hsieh, M. E. Fiuczynski, C. Garrett, S. Savage, D. Becker and B. N. Bershad. Language Support for Extensible Operating Systems. Workshop on Compiler Support for System Software, University of Washington, February 1996.

S. Savage, B. N. Bershad. Issues in the Design of an Extensible Operating System. Proceedings of the First USENIX Symposium on Operatings Systems Design and Implementation (OSDI), November 1994 (Unpublished).

University of Washington, Dept. of Computer Science. The SPIN Operating System. [http://www.cs.washington.edu.html]