Soma-notes - User contributions [en]

Talk:COMP 3000 Essay 2 2010 Question 11

2010-12-01T17:50:30Z

Sblais2:

--[[User:Sblais2|Sblais2]] 17:50, 1 December 2010 (UTC) I added stuf into the Research problems. I think I summarized most of them. If I forgot any, please add them in. I also added the missing references in the reference section. For Fedor, we seem to miss some content in 2 sections. Also, you could read through the other section and add/change some pertinent information that might've been missed that would make this essay even better.

--[[User:Sblais2|Sblais2]] 15:11, 1 December 2010 (UTC) Would it be possible to add your references at the bottom please? Even if it is a link. I have added the article link at the top of the essay.

Hey guys, sorry its taken me a while to post here. If there is a particular topic that needs researching, I could spend some hours doing that tomorrow - suggestions? Also, I intend to fix up the style&structure after everything is done as I am quite good with that.

Fedor

--[[User:ScottG|ScottG]] 21:36, 26 November 2010 (UTC) So I was a little (more than a little) behind on my initially estimated time for getting stuff up on Guest Timekeeping, but that's the gist of it there now. I'm going to try to buff it up a bit before it's due, since what I put in is a bit rougher than I'd like. If I seem to be missing something that should be pretty obvious, let me know and I'll work it in.

--[[User:Jjpwilso|Jjpwilso]] 15:49, 23 November 2010 (UTC) I've been completely swamped with COMP3004 stuff (among other things) and feeling guilty as hell about this essay. The good news, for those who might have missed today's lecture, is we have an extension of one week. Phew!!

--[[User:Sblais2|Sblais2]] 21:29, 22 November 2010 (UTC) I have added a small part to the background section. I have created by hand a diagram explaining how it works. I tried to find an original way of doing it but it is the same diagram everywhere. Please feel free to comment here or by sending me an email.

--[[User:AbsMechanik|AbsMechanik]] 19:46, 22 November 2010 (UTC) Here's what my research has led me to so far. I'm trying to come up with good points for the research problem, contribution and critique part of this essay. Here's a bunch of links, I've come across. I think there will be a few more tonight. Feel free to read through 'em:
 http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
 http://www.xen.org/files/xen_interface.pdf
 http://www.microsoft.com/whdc/system/sysinternals/mm-timer.mspx
 http://www.intel.com/hardwaredesign/hpetspec_1.pdf
 http://www.cubinlab.ee.unimelb.edu.au/radclock/

--[[User:ScottG|ScottG]] 18:55, 22 November 2010 (UTC) I'm good taking the Guest Timekeeping section. Hopefully I'll have some stuff up tonight or early tomorrow for it.

--[[User:Sblais2|Sblais2]] 17:14, 22 November 2010 (UTC) I will be working on the Background section. I will dedicate it to explain some of the key concepts that are used in the research paper that will allow the readers to have a better understanding on the rest of our essay. The structure you've put in place looks good but it might get modified, depending on the text will flow. The diagram is a good idea. I will drawn a simple one and add it in. Feel free again to critique.

--[[User:Jjpwilso|Jjpwilso]] 15:12, 16 November 2010 (UTC) I wanted to get a structure started, so I have stubbed out the first section. Note: some of the sub-sections might belong in the Research Problem section but we can easily move them if they fit there. Let's use this area to plan who is doing what. Feel free to critique any of my submissions. When you comment here, please put your comments at the very top so we can easily see recent posts.

=Participants=
(X) Blais Sylvain sblais2 - Email: syl20blais@gmail.com 
(X) Graham Scott sgraham6 
(X) Ilitchev Fedor filitche fedor dot ilitchev at gmail dot com 
(X) Panke Shane spanke 
(X) Shukla Abhinav ashukla2 
(X) Wilson Robert jjpwilso

COMP 3000 Essay 2 2010 Question 11

2010-12-01T17:46:19Z

Sblais2: /* References */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI (I will fill these in later, Nov. 29th Update) --[[User:Spanke|Spanke]]
# TSC
# HPET

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=
=Critique=
=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

7. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

COMP 3000 Essay 2 2010 Question 11

2010-12-01T17:45:55Z

Sblais2:

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI (I will fill these in later, Nov. 29th Update) --[[User:Spanke|Spanke]]
# TSC
# HPET

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=
=Critique=
=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4. "What is a CMOS battery, and why does my computer need one?" from the Indiana University's Knowledge Base, 2010. http://kb.iu.edu/data/adoy.html

5. "Multiprocessor Specification version 1.4" from Intel, 1997. http://developer.intel.com/design/pentium/datashts/24201606.pdf

6.

7. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

8. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

COMP 3000 Essay 2 2010 Question 11

2010-12-01T17:41:22Z

Sblais2: /* References */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI (I will fill these in later, Nov. 29th Update) --[[User:Spanke|Spanke]]
# TSC
# HPET

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=
=Critique=
=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. "Bran's Kernel Development Tutorial" from Bona Fide OS Developer website. http://www.osdever.net/bkerndev/Docs/pit.htm

4.

7. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

8. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

COMP 3000 Essay 2 2010 Question 11

2010-12-01T17:37:19Z

Sblais2: /* References */

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI (I will fill these in later, Nov. 29th Update) --[[User:Spanke|Spanke]]
# TSC
# HPET

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=
=Critique=
=References=

1. "Virtualize Everything But Time" by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. "Timekeeping in Virtual Machines, Information Guide" from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

7. "PC Based Precision Timing Without GPS" by Attila Pa ́sztor and Darryl Veitch, 2002. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf

8. "Robust Synchronization of Absolute and Difference Clocks over Networks" by Darryl Veitch, Julien Ridoux and Satish Babu Korada, 2009. http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf

COMP 3000 Essay 2 2010 Question 11

2010-12-01T17:32:01Z

Sblais2:

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI (I will fill these in later, Nov. 29th Update) --[[User:Spanke|Spanke]]
# TSC
# HPET

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=

Today, the use of the Network Time Protocol and of daemons like ntpd is the dominant solution for accurate timekeeping. In optimal conditions, the ntpd can be very good but these situations rarely happen. Network congestion, disconnections, lower quality networking hardware and unsuspected system events can create offsets errors in the order of 10‘s or even 100 milliseconds(ms). [http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/tscclock_final.pdf]
For demanding applications, this is neither robust or reliable. One way to enhance the performance of ntpd would be to poll from the NTP server more often as this would reduced the offset error but unfortunately, this would increase the network traffic which could cause network congestions which would raise the offset error. So this won’t work.

Another problem with current system software clocks using NTP(like ntpd), is that they provide only an absolute clock.[http://www.cubinlab.ee.unimelb.edu.au/~darryl/Publications/synch_ToN.pdf]
So for applications that deals with network managements and measurements, this is unsuitable. Why? Because NTP focus on offset and not on hardware clock oscillator rate. For example, when calculating delay variations, the offset error doesn’t change anything to the calculations but the clocks’ oscillator rate variation does affect it. So having a more accurate timestamp would make those calculation more precise. Which mean we would need another system software clock.

In virtualization(in this case Xen), when migrating a running system from one system to another can cause issues and this is again caused by the ntpd daemon. By default, each guest OS runs its own instance of the ntpd daemon. So the synchronization algorithm keeps track of the reference wallclock time, rate-of-drift and current clock error, which are defined by the hardware clock on the system. So when migrating the virtualized OS to another system, the ntpd state is saved and when it is enabled again on the new system, thats where the problems starts. Because no two hardware clocks drifts the same way or have the exact same wallclock time, all the information traced by the daemon are all of a sudden inaccurate. This could prove disastrous to the system. This could go from a slowly recoverable error to one where ntpd might never recover, making the virtualized OS unstable.

=Contribution=
=Critique=
=References=

1. Virtualize Everything But Time by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

Talk:COMP 3000 Essay 2 2010 Question 11

2010-12-01T15:11:42Z

Sblais2:

--[[User:Sblais2|Sblais2]] 15:11, 1 December 2010 (UTC) Would it be possible to add your references at the bottom please? Even if it is a link. I have added the article link at the top of the essay.

Hey guys, sorry its taken me a while to post here. If there is a particular topic that needs researching, I could spend some hours doing that tomorrow - suggestions? Also, I intend to fix up the style&structure after everything is done as I am quite good with that.

Fedor

--[[User:ScottG|ScottG]] 21:36, 26 November 2010 (UTC) So I was a little (more than a little) behind on my initially estimated time for getting stuff up on Guest Timekeeping, but that's the gist of it there now. I'm going to try to buff it up a bit before it's due, since what I put in is a bit rougher than I'd like. If I seem to be missing something that should be pretty obvious, let me know and I'll work it in.

--[[User:Jjpwilso|Jjpwilso]] 15:49, 23 November 2010 (UTC) I've been completely swamped with COMP3004 stuff (among other things) and feeling guilty as hell about this essay. The good news, for those who might have missed today's lecture, is we have an extension of one week. Phew!!

--[[User:Sblais2|Sblais2]] 21:29, 22 November 2010 (UTC) I have added a small part to the background section. I have created by hand a diagram explaining how it works. I tried to find an original way of doing it but it is the same diagram everywhere. Please feel free to comment here or by sending me an email.

--[[User:AbsMechanik|AbsMechanik]] 19:46, 22 November 2010 (UTC) Here's what my research has led me to so far. I'm trying to come up with good points for the research problem, contribution and critique part of this essay. Here's a bunch of links, I've come across. I think there will be a few more tonight. Feel free to read through 'em:
 http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
 http://www.xen.org/files/xen_interface.pdf
 http://www.microsoft.com/whdc/system/sysinternals/mm-timer.mspx
 http://www.intel.com/hardwaredesign/hpetspec_1.pdf
 http://www.cubinlab.ee.unimelb.edu.au/radclock/

--[[User:ScottG|ScottG]] 18:55, 22 November 2010 (UTC) I'm good taking the Guest Timekeeping section. Hopefully I'll have some stuff up tonight or early tomorrow for it.

--[[User:Sblais2|Sblais2]] 17:14, 22 November 2010 (UTC) I will be working on the Background section. I will dedicate it to explain some of the key concepts that are used in the research paper that will allow the readers to have a better understanding on the rest of our essay. The structure you've put in place looks good but it might get modified, depending on the text will flow. The diagram is a good idea. I will drawn a simple one and add it in. Feel free again to critique.

--[[User:Jjpwilso|Jjpwilso]] 15:12, 16 November 2010 (UTC) I wanted to get a structure started, so I have stubbed out the first section. Note: some of the sub-sections might belong in the Research Problem section but we can easily move them if they fit there. Let's use this area to plan who is doing what. Feel free to critique any of my submissions. When you comment here, please put your comments at the very top so we can easily see recent posts.

=Participants=
(X) Blais Sylvain sblais2 - Email: syl20blais@gmail.com 
(X) Graham Scott sgraham6 
(X) Ilitchev Fedor filitche fedor dot ilitchev at gmail dot com 
(X) Panke Shane spanke 
(X) Shukla Abhinav ashukla2 
(X) Wilson Robert jjpwilso

COMP 3000 Essay 2 2010 Question 11

2010-12-01T15:09:47Z

Sblais2:

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI (I will fill these in later, Nov. 29th Update) --[[User:Spanke|Spanke]]
# TSC
# HPET

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=
=Contribution=
=Critique=
=References=

1. Virtualize Everything But Time by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

COMP 3000 Essay 2 2010 Question 11

2010-12-01T15:03:47Z

Sblais2:

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI (I will fill these in later, Nov. 29th Update) --[[User:Spanke|Spanke]]
# TSC
# HPET

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=
=Contribution=
=Critique=
=References=

1. Virtualize Everything But Time by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf

2. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

3. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

COMP 3000 Essay 2 2010 Question 11

2010-12-01T14:48:07Z

Sblais2:

= Virtualize Everything But Time =
Article written by Timothy Broomhead, Laurence Cremean, Julien Ridoux and Darrel Veitch. They are working for the Center for Ultra-Broadband Information Networks (CUBIN) Department of Electrical & Electronic Engineering at the University of Melbourne in Australia. Here is the link to the article: [http://www.usenix.org/events/osdi10/tech/full_papers/Broomhead.pdf]

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC, also known as a CMOS battery, allows the CMOS chip to remain powered to keep track of things like time even while the physical PC unit has no source of power. If there is no CMOS battery on the motherboard, the computer would reset to its default time each restart. The battery itself can die, as expected, if the computer is powered off and not used for a long period of time. This can cause issues with the main OS as well as the VM. [http://kb.iu.edu/data/adoy.html]
# Local APIC handles all external interrupts for the processor in the system. It can also accept and generate inter-processor interrupts between Local APICs. [http://developer.intel.com/design/pentium/datashts/24201606.pdf]
# ACPI (I will fill these in later, Nov. 29th Update) --[[User:Spanke|Spanke]]
# TSC
# HPET

==Guest Timekeeping==

Guest timekeeping is done using the same general methods as any computer timekeeping, using either tick counting or tickless systems. Where the two begin to differ, however, is that a host operating system is able to communicate directly with the physical hardware, while the guest operating system is unable to do so, having to communicate with the host system that it wants to communicate with the hardware. Having to do this is the greatest source of the guest operating system's clock losing accuracy, or more simply called drifting.

===Sources of Drift===

When a guest operating system is started, its clock simply synchronizes with the host's – some virtual machines such as VMware also do this when it is resumed from a suspended state, or restored from a snapshot – so it is easy to think that, since it starts off correctly the guest's clock will continue to be correct. That is, of course, incorrect. The first source of drift is simply due to the drift a host incurs in its own timekeeping. A clock is almost never entirely accurate, having a slight error due to the time used to communicate with the counter, even on the host system, and because the guest communicates with the host in order to keep track of its time, an error in the host's time is not only passed on to the guest, but because the host is trying to correct its own time the guest's request for a count is given slightly less priority, making it yet again lose accuracy. The larger the drift in the host, the larger the drift in the guest, as the host's drift simply compounds the issue.

Aside from the host's own drift, the other cause of drift in the virtual environment is the fact that the it is treated like a process by the host. In and of itself this doesn't seem like a problem, but because of it it can be denied the CPU time required, or allocated less memory than needed. With restricted CPU time, it's easy for the requested ticks or requested read of a counter to pile up and create a backlog of requests, or simply receive the requested data late enough to throw its clock off. With memory, if the virtual environment does not have enough allocated to it by the host it can run into the problem of swapping out pages that are needed soon. Swapping the pages back in will momentarily bring the entire virtual environment to a halt, so ticks are missed and the clock falls behind.

===Impact of Drift===

The impact of drift essentially boils down to round-off errors and lost ticks. The practical impact of drift, however, is quite apparent in any automated system. For a relatable real-world example, though not in a virtual environment, in a factory's assembly line, the machinery is finely tuned to do its own specific part at certain intervals, and it generally does so with impressive efficiency. If the clock in the system were to drift, however, a specific machine may move too soon or too late, bringing the line to a potentially catastrophic halt. In a virtual environment, drift is a bit more subtle, as one result of it could be skewed process scheduling – some schedulers give a certain amount of time to a process before moving on, but if the guest's time has drifted substantially, when it tries to correct its time it could give more or less time to the processes in the scheduler.

===Compensation Strategies===

There are a number of compensation strategies for dealing with drift, depending on the cause of it. If the problem is due to CPU management issues, then the host can give more CPU time to the virtual machine, or it can lower the timer interrupt rate – or simply use a tickless counter. If it is due to a memory management issue, allocating more memory to the virtual environment should prevent the system from needing to swap out page files so often.

If the issue is from neither of those, but simply due to the inevitable lag when the guest communicates with the hardware via the host, then there are other methods to correct the drift. Most systems natively have algorithms built in to correct the time if it gets too far ahead or behind real time, though they are not without their own faults; if the time is set ahead when catching up, the backlog of ticks it has built up may not be cleared, so it could potentially set itself ahead multiple times until the backlog is dealt with. Tools built into the virtual machine itself can also deal with drift to an extent, as VMware Tools does. This kind of tool checks to see if the clock's error is within a certain margin. If it exceeds the margin, then the backlog is set to zero – to prevent the issue mentioned with the native algorithms – and resynchronizes with the host clock before the guest goes back to keeping track of time as it normally would.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

COMP 3000 Essay 2 2010 Question 11

2010-11-23T11:40:22Z

Sblais2:

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again. As I said earlier, not all hardware timer works exactly like that. Some actually counts up, doesn't use interrupts or doesn't have an initial counter value but they still follow the same principle.

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC (I will fill these in later) --[[User:Spanke|Spanke]]
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

COMP 3000 Essay 2 2010 Question 11

2010-11-23T11:37:15Z

Sblais2:

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.

===Timers===
# PIT is useful for generating interrupts at regular intervals through its three channels. Channel 0 is bound to IRQ0 which interrupts the CPU at regular intervals. Channel 1 is specific to each system and Channel 2 is connected to the speaker system. As such, we only need to concern ourselves with Channel 0. [http://www.osdever.net/bkerndev/Docs/pit.htm]
# CMOS RTC (I will fill these in later) --[[User:Spanke|Spanke]]
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

COMP 3000 Essay 2 2010 Question 11

2010-11-22T22:15:36Z

Sblais2:

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.

===Timers===
# PIT
# CMOS RTC
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

COMP 3000 Essay 2 2010 Question 11

2010-11-22T22:14:57Z

Sblais2:

[[File:Example.jpg]]=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.

===Timers===
# PIT
# CMOS RTC
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

COMP 3000 Essay 2 2010 Question 11

2010-11-22T22:13:39Z

Sblais2:

[[File:Example.jpg]]=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[Image:http://homeostasis.scs.carleton.ca/wiki/index.php/File:Timerabstract.jpg]]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.

===Timers===
# PIT
# CMOS RTC
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

COMP 3000 Essay 2 2010 Question 11

2010-11-22T22:10:34Z

Sblais2:

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[Image:Timerdiagram.jpg]]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.

===Timers===
# PIT
# CMOS RTC
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

COMP 3000 Essay 2 2010 Question 11

2010-11-22T22:08:08Z

Sblais2:

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerdiagram.jpg]]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.

===Timers===
# PIT
# CMOS RTC
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

COMP 3000 Essay 2 2010 Question 11

2010-11-22T22:06:59Z

Sblais2:

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[Media:Timerdiagram.jpg]]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.

===Timers===
# PIT
# CMOS RTC
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

COMP 3000 Essay 2 2010 Question 11

2010-11-22T22:06:32Z

Sblais2:

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[File:Timerdiagram.jpg]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.

===Timers===
# PIT
# CMOS RTC
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

COMP 3000 Essay 2 2010 Question 11

2010-11-22T22:05:10Z

Sblais2:

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

[[File:Timerdiagram.jpg]]

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.

===Timers===
# PIT
# CMOS RTC
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

File:Timerabstract.jpg

2010-11-22T22:02:47Z

Sblais2: uploaded a new version of "File:Timerabstract.jpg"

Describes a general abstraction on the workings of a clock(timer).

File:Timerabstract.jpg

2010-11-22T21:42:28Z

Sblais2: Describes a general abstraction on the workings of a clock(timer).

Describes a general abstraction on the workings of a clock(timer).

COMP 3000 Essay 2 2010 Question 11

2010-11-22T21:40:43Z

Sblais2:

=Background Concepts=

To better understand this paper, it is very important to have a good understanding of the general concepts breached in it. For example, we all know what clocks are in our day-to-day life but what are they in the context of computing? In this section, we will describe concepts like timekeeping, hardware/software clocks, the advantages and disadvantages of the different available counters, synchronization algorithms and explains what is a para-virtualized system.

===Timekeeping===

Since thousands of years, men have tried to find better ways to keep track of time. From sundials to atomic clocks, they were all made for the specific purpose of measuring the passage of time. This is not so different in computer operating systems. It is typically done in one of two ways: tick counting and tickless timekeeping[1]. Tick counting is when the operating system sets up an hardware device, generally a CPU, to interrupt at a certain rate. So each time one of those interrupts are called(a tick), the operating system will keep track of it in a counter. That will tell the system how much time has passed. In tickless timekeeping, instead of the OS keeping track of time through interrupts, a hardware device is used instead starting its own counter when the system is booted. The OS just need to read the counter from it when needed. Tickless timekeeping seems to be the better way to keep track of time because it doesn’t hog the CPU with hardware interrupts however, its performance is very dependent on the type of hardware used. Another disadvantages is that they tend to drift and can cause inaccuracy. I will explains those drifts later. But both of these are just counters. They don’t know what is the actual real-time. To remedy that, either a computer gets its time from a battery-backed real-time clock or it queries a network time server(NTP) to get the current time. The computer can also use software in the form of a daemon that will run periodically to make adjustments to the time.

===Clocks===

Computer “clocks” or “timers” can be hardware based, software based or they can even be an hybrid. The most commonly found timer is the hardware timer. All of the hardware timers can be generally described by this diagram where some have either more or less features:

Diagram1. Timer Abstraction

This diagram nicely represent how tick counting works[2]. The oscillator runs at a predetermine frequency. The operating system might have to mesure it when the system boots. The counter starts with a predetermined value which can be set by software. For every cycle of the oscillator, the counter counts down one unit. When it reaches zero, its generates an output signal that might interrupt the CPU. That same interrupt will then allow the counter’s initial value to be reloaded into the counter and the process begins again.

===Timers===
# PIT
# CMOS RTC
# Local APIC
# ACPI
# TSC
# HPET

==Guest Timekeeping==
This section will explain why timing is different for guest operating systems.
===Sources of Drift===
Discuss what causes the problem in virtual environments. Consider para-virtual and full-virtual architectures.
===Impact of Drift===
Discuss the impact of error on a few real-world applications.
===Compensation Strategies===
Discuss common strategies in VM software and in operating systems to compensate for drift. Discuss what happens when the clock is slow and fast.

=Research problem=
=Contribution=
=Critique=
=References=

1. Timekeeping in Virtual Machines, Information Guide from VMWare. http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
2. Modern Operating System 3rd Edition, by Andrew S. Tanenbaum, published by Pearson.

Talk:COMP 3000 Essay 2 2010 Question 11

2010-11-22T21:29:12Z

Sblais2:

--[[User:Sblais2|Sblais2]] 21:29, 22 November 2010 (UTC) I have added a small part to the background section. I have created by hand a diagram explaining how it works. I tried to find an original way of doing it but it is the same diagram everywhere. Please feel free to comment here or by sending me an email.

--[[User:AbsMechanik|AbsMechanik]] 19:46, 22 November 2010 (UTC) Here's what my research has led me to so far. I'm trying to come up with good points for the research problem, contribution and critique part of this essay. Here's a bunch of links, I've come across. I think there will be a few more tonight. Feel free to read through 'em:
 http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
 http://www.xen.org/files/xen_interface.pdf
 http://www.microsoft.com/whdc/system/sysinternals/mm-timer.mspx
 http://www.intel.com/hardwaredesign/hpetspec_1.pdf
 http://www.cubinlab.ee.unimelb.edu.au/radclock/

--[[User:ScottG|ScottG]] 18:55, 22 November 2010 (UTC) I'm good taking the Guest Timekeeping section. Hopefully I'll have some stuff up tonight or early tomorrow for it.

--[[User:Sblais2|Sblais2]] 17:14, 22 November 2010 (UTC) I will be working on the Background section. I will dedicate it to explain some of the key concepts that are used in the research paper that will allow the readers to have a better understanding on the rest of our essay. The structure you've put in place looks good but it might get modified, depending on the text will flow. The diagram is a good idea. I will drawn a simple one and add it in. Feel free again to critique.

--[[User:Jjpwilso|Jjpwilso]] 15:12, 16 November 2010 (UTC) I wanted to get a structure started, so I have stubbed out the first section. Note: some of the sub-sections might belong in the Research Problem section but we can easily move them if they fit there. Let's use this area to plan who is doing what. Feel free to critique any of my submissions. When you comment here, please put your comments at the very top so we can easily see recent posts.

=Participants=
(X) Blais Sylvain sblais2 - Email: syl20blais@gmail.com 
(X) Graham Scott sgraham6 
(X) Ilitchev Fedor filitche fedor dot ilitchev at gmail dot com 
(X) Panke Shane spanke 
(X) Shukla Abhinav ashukla2 
(X) Wilson Robert jjpwilso

Talk:COMP 3000 Essay 2 2010 Question 11

2010-11-22T17:15:04Z

Sblais2:

--[[User:Sblais2|Sblais2]] 17:14, 22 November 2010 (UTC) I will be working on the Background section. I will dedicate it to explain some of the key concepts that are used in the research paper that will allow the readers to have a better understanding on the rest of our essay. The structure you've put in place looks good but it might get modified, depending on the text will flow. The diagram is a good idea. I will drawn a simple one and add it in. Feel free again to critique.

--[[User:Jjpwilso|Jjpwilso]] 15:12, 16 November 2010 (UTC) I wanted to get a structure started, so I have stubbed out the first section. Note: some of the sub-sections might belong in the Research Problem section but we can easily move them if they fit there. Let's use this area to plan who is doing what. Feel free to critique any of my submissions. When you comment here, please put your comments at the very top so we can easily see recent posts.

=Participants=
(X) Blais Sylvain sblais2 - Email: syl20blais@gmail.com 
(X) Graham Scott sgraham6 
(X) Ilitchev Fedor filitche fedor dot ilitchev at gmail dot com 
(X) Panke Shane spanke 
(X) Shukla Abhinav ashukla2 
(X) Wilson Robert jjpwilso

Talk:COMP 3000 Essay 2 2010 Question 11

2010-11-22T17:14:43Z

Sblais2:

----[[User:Sblais2|Sblais2]] 17:14, 22 November 2010 (UTC) I will be working on the Background section. I will dedicate it to explain some of the key concepts that are used in the research paper that will allow the readers to have a better understanding on the rest of our essay. The structure you've put in place looks good but it might get modified, depending on the text will flow. The diagram is a good idea. I will drawn a simple one and add it in. Feel free again to critique.

--[[User:Jjpwilso|Jjpwilso]] 15:12, 16 November 2010 (UTC) I wanted to get a structure started, so I have stubbed out the first section. Note: some of the sub-sections might belong in the Research Problem section but we can easily move them if they fit there. Let's use this area to plan who is doing what. Feel free to critique any of my submissions. When you comment here, please put your comments at the very top so we can easily see recent posts.

=Participants=
(X) Blais Sylvain sblais2 - Email: syl20blais@gmail.com 
(X) Graham Scott sgraham6 
(X) Ilitchev Fedor filitche fedor dot ilitchev at gmail dot com 
(X) Panke Shane spanke 
(X) Shukla Abhinav ashukla2 
(X) Wilson Robert jjpwilso

Talk:COMP 3000 Essay 2 2010 Question 11

2010-11-15T18:48:34Z

Sblais2:

Please mark an X if you are able to participate.

(X) Blais Sylvain sblais2 - Email: syl20blais@gmail.com 
(X) Graham Scott sgraham6 
(X) Ilitchev Fedor filitche fedor dot ilitchev at gmail dot com 
(X) Panke Shane spanke 
(X) Shukla Abhinav ashukla2 
(X) Wilson Robert jjpwilso

Talk:COMP 3000 Essay 2 2010 Question 11

2010-11-11T23:46:02Z

Sblais2:

Please mark an X if you are able to participate.

(X) Blais Sylvain sblais2 
(X) Graham Scott sgraham6 
( ) Ilitchev Fedor filitche 
( ) Panke Shane spanke 
( ) Shukla Abhinav ashukla2 
(X) Wilson Robert jjpwilso

COMP 3000 Essay 1 2010 Question 2

2010-10-15T10:15:06Z

Sblais2: /* File Management Calls */

=Question=
How do the available system calls in modern versions of the Linux Kernel (2.6.30+) compare with the system calls available in the earliest versions of UNIX? How has the system call interface been expanded, and why? Focus on major changes or extensions in functionality.

=Answer=

A system call is a mean by which programs in the user space can access kernel services. Systems calls vary from operating system to operating system, although the underlying concepts tends to be the same. In general, a process is not supposed to be able to access the kernel directly. It can't access kernel memory and it can't call kernel functions. When the CPU prevents a process from accessing the kernel, this prevention is commonly known as, the protected mode. On the other hand, system calls are an exception to this rule. For example, older x86 processors used an interrupt mechanism to go from user-space to kernel-space, but newer processor (PentiumII+) provided instructions that optimize this transition using sysenter and sysexit instructions (Hayward, Mike. Intel P6 vs P7 system call performance. [http://lkml.org/lkml/2002/12/9/13], December 9, 2002). All system calls are small programs built using the C programming language.

The Unix and Linux systems calls are roughly grouped into 6 major categories: file management, device management, information maintenance, process control, communications and miscellaneous calls. The miscellaneous calls are all the ones that don’t really fit in the other categories, like system calls dealing with errors. Today, the Unix and Linux operating system contains hundreds of system calls but in general, they all came from the 35 system calls that came with one of the original UNIX OS in the early 70s. In the next paragraphs, we’re going to describe the various system calls in each of the categories mentioned above, their evolution through history (major changes in functionality) and a comparison with the earliest versions of UNIX.

== File Management Calls==

The system calls in this group deal with every type of operation that is required to run a file system in the operating system. Create file, delete file, opening a file and closing a file are just a few examples of them and most of them hardly changed throughout the years.

''chmod'', ''chown'' and ''chdir'' have been available since the first original UNIX (1971) and they are still used in today’s Linux kernels. The ''chmod'' and ''chown'' calls allows the users to change the file attributes and implements security to the file system. The system call ''chdir'' allows the process to change the current working directory. In the 4th distribution of UNIX from Berkeley (4BSD), new system calls were added to give more control of the file system to the applications. The call ''chroot'' allows the process to change the current root directory with one specified in a path argument. ''fchmod'' and ''fchdir'' are the same as ''chmod'' and ''chdir'' except they takes file descriptors as arguments. As of Linux kernel 2.1.81, the ''chown'' system call now follows symbolic links and therefore, they introduced a new system call, ''lchown'', that does not follow symbolic links.

Another four of the original UNIX system calls are ''open'', ''creat'', ''mkdir'' and ''close''. The ''open'' and ''creat'' calls allows processes to open and possibly create a file or device. Arguments flags are used to set from access modes, like O_RDONLY(read-only), to status flags, like O_APPEND(append mode). The only modifications made to the system calls were the addition of status flags where some of them are linux-specific. The ''close'' call allows processes to close a file descriptor preventing it to be reused. No changes were made to it. ''mkdir'' allows the creation of a file directory. In the earliest version of Unix, to delete a directory, users needed to make a series of ''link'' and ''unlink'' system calls. With Unix 4.2BSD, ''rmdir'' was added and helped solve the problem. The ''rename'' call was also added in 4.2BSD allowing processes to change the name or the location of a file. As file system became more complex, these new system calls helped the users gain better control over them.

There is also system calls used to find, access and modify files. They are ''read'', ''write'', ''seek'' and ''stat''. These were also part of the first UNIX built. The ''read'' and ''write'' system calls allows to read and write from a file (assigned to a file descriptor). The only change was in the Unix System V release 4(SVR4) where a ''write'' call could be interrupted at anytime. The ''seek'' system calls is used to go to a specified position in a file. This calls used a 16 bit address to determine its position in a file(also called offset). But this was replaced very quickly for ''lseek'' as early as SVR4. It allows the call to use 32 bit address enabling the users more flexibility when reading or writing files especially for large ones. It is still used in modern Linux and Unix systems. As of now, developers are trying to implement ''lseek64'', a system call that will use 64 bit addresses. The ''stat'' system calls allows processes to get the status of a file. With SVR4, 2 other version of that system call were created: ''fstat'' and ''lstat''. They both do the same thing except ''lstat'' give the status of symbolic links and ''fstat'' give the status of a file specified by a file descriptor. Different operating systems will output different values to represent the state of a file. Since kernel 2.5.48, the stat returned a nanoseconds field in the file’s timestamp. With the release of 4.4BSD, two new system calls called ''statvfs'' and ''fstatvfs'' were introduced to provide information about a mounted file system. They both do the same thing except fstatvfs takes file descriptors as an argument. These calls are only used in an UNIX environment. In Linux, it has ''statfs'' and ''fstatfs'' to support that same call.

The last two original UNIX system calls in this category that are still used today are ''link'' and ''unlink''. ''link'' creates a hard link to an existing file and ''unlink'' deletes a file link’s name and possibly the file it refers to. If the name refers to a symbolic link, only the link is removed. No major changes were done to the ''unlink'' system calls but new system calls were create from ''link''. The ''symlink'' system call was added in 4.2BSD to allow the creation of symbolic links in the file system.

In the Linux 2.6.16 build, multiple system calls were created so that the calls could deal with relative pathnames as arguments. They can easily be spotted as the system call names all finish with 'at'. Here is a sample list of the created system calls: ''openat'', ''mkdirat'', ''fchmodat'', ''fchownat'', ''fstatat'', ''linkat'', ''unlinkat'', ''renameat'' and ''fchmodat''.

== Device Management Calls==

The device management system calls are linked to hardware and they are mainly used to requests for devices, release devices, to logically attach a device or to detach it, get and modified device attributes and read and write to them.

Two of the most important system calls for the UNIX and Linux operating system is ''mount'' and ''umount''. These were among the few system calls available in the first version of UNIX in 1971. The two calls allowed the operating system to load file systems on storage devices. A few changes were done to the mount system calls. Most of these changes were the creation of new mount flags to enhance performance. For example, since Linux 2.5.19, the MS_DIRSYNC flag permits the directory changes on a file system synchronous. Another Linux improvement was to provide per-process mount namespaces. This was added on the 2.4.19 kernel. If a process was created using the ''clone'' system call with the CLONE_NEWNS flag, the process will have a new namespace initialized to be a copy of the namespace of the process that was cloned. The ''umount'' system call unmounted the file system from the storage device. The only noteworthy change to ''umount'' was the creation of ‘‘umount2’’ in Linux 2.1.116. It is the same as ''umount'' except it allows different flags to control the operation.

The ''open'', ''read'' and ''write'' calls can also be used to access devices. As discussed in the previous section, arguments flags are used to better control the device. You would use them as if the devices were files using the appropriate flags.

With the SVR4 came the system call ''mmap''. This system call is used to map or unmap files or devices into memory. Once a device is mapped, the system call returns a pointer to the mapped area allowing processes to access that device. This system call is still used in a Unix environment but since Linux 2.4, Linux replaced it by the mmap2 system call. It is basically the same as mmap except for a final argument specifying the offset into a file in 4096-byte units. This enables the mapping of large files.

In version 7 of Unix, ''ioctl'' system call is used for device-specific operations that can’t be done using the standard system calls. This helps to deal with a multitude of devices. Each device drivers would provide a set of ioctl request code to allow various operations on their device. Each various request code are hardware dependent so there is no standard available for this system call.

== Information Maintenance Calls==
Information maintenance calls are system calls that return the computers personal information back to the user or change it completely. These type of calls can be split up into three groups get/set time or date, get/set system data and get/set process, file, or device attributes. To fully understand the difference between Linux and UNIX in regrades to system calls, one must explore the three sub-types of information maintenance calls and see how they have changed over time.

The first sub type is Get/set of time and/or date. In Linux, this can be done by a few different system calls, there are: 'gettimeofday' to get the time, 'settimeofday' to set it, 'time' returns the time in seconds and a few other ones like 'ftime'.In the earliest versions UNIX the used the system call was 'stime', which was used to interact with time and dates. 'stime' could return the time and date and sets the system’s idea of the time and date by altering the seconds. 'stime' is still being used by Linux because it is successful, unlike 'settimeofday', which was created to change timezones (tz_dsttime) as well as the time but each occurrence of this field call in the kernel source (apart from declaration) is a bug thus failing.

The second sub type is get/set system data. UNIX does this using the following commands: 'open', 'read', 'close', and 'write'. 'open' opens a file so the file can be written to or read from. 'read' retrieves data from the file, and 'write' modifies data in the file. 'close' is used to indicate that the file is no longer in use. Linux uses the same set of commands for the same purposes.In addition to those system calls there Linux has there own unique system calls which are: 'olduname' gets name and information about current kernel, similar to that is 'uname' gets name and information about current kernel (which is used in the newer versions of UNIX not the older ones), 'iopl' which changes I/O privilege level and 'sysfs' which gets file system type information.

The third sub type is get/set process, file, or device attributes, in UNIX there are several system calls for processing file and device attributes, some of these examples are common to both UNIX and Linux: 'stat' gets file status, 'fork' which spawns a new process, and 'stty' which sets the mode of typewriter.The 'wait' system call is used in both as well the only really difference is that in the Linux version wait store status information in a integer which take the integer itself as an argument, not a pointer to itself. In Linux there are a lot more system calls regarding this type and here are a few of them: 'capget' gets the capabilities of the process, 'capset' sets the capabilities process, 'getppid' gets process identification. The 'capget' and 'capset' interact with the raw kernel interface to getting and setting thread capabilities. These two system calls are specific to Linux and as such the use of these functions (in particular the format of the cap_user_*_t types) are updated as the kernel is updated. The 'getppid' returns the process ID of the calling process and never has any errors.

== Process Control Calls==

Process Control calls are system calls that handle the start, termination and other tasks that might be required
for a process to run correctly.

In unix there are 10 system calls that make up Process Control Calls. These are:
''fork()'',''wait()'',''execl()'',''execlp()'',''execle()'',''execvp()'',''execv()'',''execve()'',''exit()'',''signal()'',''kill()''.

''fork()'': It takes a process and creates an identical processes, which in turn makes one the parent process and the
other the child process. When ''fork()'' succeds it returns 0 to the child process and returns the PID of the child process to the parent process. When it fails, ''fork()'' returns -1 to the parent process.

''wait()'': This call makes a parent process wait for the child process to end. It returns the pid of the child process that is
done. Wait fails if the process has no child process to wait for or its points to an invalid address.

''execl()'',''execlp()'',''execle()'',''execvp()'',''execv()'', are system calls based on the same principle that the system call
takes as an argument a binary file and converts it into a process. When the system call works properly it does
not return, instead it gives control to the new process which replaces the process that called the system call.
each of these are called when different arguments are given.

The following are the definitions for the these system calls as described by this [http://www.di.uevora.pt/~lmr/syscalls.html]

''execl()'': Takes the path name of an executable program (binary file) as its first argument. The rest of the arguments are a list of command
line arguments to the new program (argv[]). The list terminated with a null pointer

''execle()'': Same as execl(), except that the end of the argument list is followed by a pointer to a null-terminated list of character
pointers that is passed a the environment of the new program

''execv()'': Takes the path name of an executable program (binary file) as it first argument. The second argument is a pointer to a list of
character pointers (like argv[]) that is passed as command line arguments to the new program.

''execve()'': Same as execv(), except that a third argument is given as a pointer to a list of character pointers (like argv[]) that is passed as the environment of the new program.

''execlp()'': Same as execl(), except that the program name doesn't have to be a full path name, and it can be a shell program instead of executable module.
''execvp()'': Same as execv(), except that the program name doesn't have to be a full path name, and it can be a shell program instead of an executable module.

''signal()'': This system call is sent to the process when the proper conditions are met. When the program receives the signal it can act in three different ways. The first is to ignore completely, it wont matter how many times the signal is sent the process will not do anything because of it. The only signal that can't be ignored or caught is SIGKILL(). The second is to have the signal set to its default state which means when the process receives it, the process will end. The last option is to catch the signal, when this occurs the unix system will give control to a function that will execute according to how appropriate it is for the process.

''kill()'': The system sends a signal to the process when something occurs. It fails if the signal_name is not a correct signal. There is no process with the PID that matches the argument value.

''exit()'': This call ends the process that calls it and returns the exit status value.

In linux, all of these unix system calls have couterparts in linux except for the exec group of system calls, only execve exists. Also these system calls behave the same way in linux. However the system call ''signal()'' is not recommended to use because of its different implementations in different versions of linux and unix. It is better to use ''sigaction()''. It changes the actions of the process when it receives a valid signal except SIGKILL and SIGSTOP. As newer versions of linux are released, these system calls will always never have major modifications but other system calls, based on these, may be created because specific cases which would make it easier to write programs.

== Communications Calls==

The communication calls relates to the concept of processes having the ability to communicate with one another. Similar to how humans use a telephone as their portal to communicate with eachother, communication calls use "pipes" as their gateway.

In unix there are four subgroups of system calls that are related to communications calls: pipelines, messages, semaphores, and shared memory.
The following are the system calls that belong to each of the subgroups.

Pipelines: ''pipe()'' The pipe() command consists of two components. int pipe (file_descriptors) & int file_descriptors[2]. File_descriptors is an array consisting of two parts as well. One is for reading the data, and the other is for writing the data. Both writing and reading data will read in sequential order along with fully completing it's task. I.e.) There are no partial writes, the pipe will write the whole data that was sent and complete the transmission. The same concept holds for the reading where it will be read all the way through before reading another pipe or new information coming into the pipe. A specially named pipe is FIFO. Standing First In First Out. It is accessed as part of a file system through idea of pipes.

Messages: These functions all consist of recieving and sending messages from the queue usually involving ID's. ''msgget()'' acquires a message from the queue identifier relating to the key. Closely related, but not the same the ''msgrcv()'' command is used to recieve a message from the queue relating to the msqid parameter. This parameter involves the ID of where to recieve the message. ''msgsnd()'' sends a message to the queue. This command can be thought of as the reverse of the ''msgget()''. Lastly, ''msgctl()'' performs message control operations through queries.

Semaphores: This idea of semaphores consists of setting it or checking it. They are used to control access to files. One can use the concept of file locking to get a better understanding of Semaphores. Semaphores aren't usually held together in singles, but rather in groups. This is done by creating a set that can contain several semaphores through the ''semget()'' command. '' semop()'' decides what we the semaphor to accomplish. I.e) depending on whether we have a positive, 0 or negative value, the semaphor shall be added, will wait, or be blocked until positive, respectively. Semaphores were first thought of by Dijkstra and used in computers in the late 60's

Shared Memory: Functions involving shared memory allow the user to be able to access, detach and combine shared addresses. ''shmget()'' command returns the ID for the shared memory region. It can also create it if it doesn't already exist. ''shmat()'' function attaches the shared memory to the virtual address of the calling process. ''shmdt()'' reverses the ''shmat()'' command, and detaches shared memory.

Unix and Linux use the same calls for the majority of the functions now, except for a few which are slighly different.

== Miscellaneous System Calls ==
This category of system calls contains the system calls that do not enough similar calls to form its own group. To avoid random calls floating around, we simply group them into this category.

Directories: These are special files that contain a number of filenames. There are different variations of directories, i.e.) System V, Berkely style directories.

Time: Intuitively, this call allows the user to access the time of day. Specifics on time can be obtained through the structure given by these attributes: ''tm_secint'', ''tm_min'', ''tm_hour'', ''tm_mday'', ''tm_mon'', '' tm_year'', just to list a few.
Parsing Input: Parsing is often used when the user enters in data and the program must parse this data into appropriate divisions in order to obtain specific parts of the data. I.e.)Seperate words from each other in the program, seperate numbers from characters. There are several different ways a programmer can parse the data in order to achieve specific pieces of the data that are needed to be analyzed.

Lastly, there are some system calls which overlap and can be considered in a specific category or mentioned within the Miscellaneous System Calls. Referring to the ``3rd edition, Modern Operating Systems`` textbook, the command, ``chmod`` described above in File Management Calls is considered Miscellaneous. Similarly, the kill() command is mentioned as a Miscellaneous System Call. Hence, it is difficult to decifer whether a system call can be placed into a specific category or simply placed in the ``Other`` bin.

== Conclusion ==

System Calls have been an essential component to the structure of the Linux Kernel(2.6.30+) and Unix Operating Systems for a long period of time. They are the gateway between the user space and the kernel services. More specifically, it allows the User space to acquire the kernel services unlike processes which do not have this authority. Over the years of development in the Linux and Unix OS, the system calls have not had drastic changes to them. Rather than having radical changes to system calls, the development of system calls has merely added more specific system calls to solve new issues that occur within the OS. Hence, this concept has led the birth of 35 system calls to grow to an astonishing quantity consisting of hundreds of system calls. With hundreds of system calls available at ones disposable, all can be catgeorized into 6 major groups: file management, device management, information maintenance, process control, communications and miscellaneous calls. Operating Systems are a colossal program consisiting of very intrinsic pieces all coming together to form what we now know today as Linux Kernel(2.6.30+) or Unix. System calls are simply a small building block, but nevertheless an essential piece, to the tower that is our Operating System.

= References =

Salus, Peter H. A Quarter Century of Unix, Publisher: Addison-Wesly Professional, June 10, 1994.

Unix Programming Manual, http://cm.bell-labs.com/cm/cs/who/dmr/1stEdman.html, November 3, 1971.

BSD System Calls Manual. http://www.unix.com/man-page/FreeBSD/2/, The Unix and Linux Forums.

Linux Programmer's Manual, Linux man-pages project.
http://www.kernel.org/doc/man-pages/

Mendonça Rato, Luís Miguel,professor,university of evora. http://www.di.uevora.pt/~lmr/syscalls.html

Tanenbaum, Andrew S. 3rd edition Modern Operating Systems.

COMP 3000 Essay 1 2010 Question 2

2010-10-15T10:10:14Z

Sblais2: /* Device Management Calls */

=Question=
How do the available system calls in modern versions of the Linux Kernel (2.6.30+) compare with the system calls available in the earliest versions of UNIX? How has the system call interface been expanded, and why? Focus on major changes or extensions in functionality.

=Answer=

A system call is a mean by which programs in the user space can access kernel services. Systems calls vary from operating system to operating system, although the underlying concepts tends to be the same. In general, a process is not supposed to be able to access the kernel directly. It can't access kernel memory and it can't call kernel functions. When the CPU prevents a process from accessing the kernel, this prevention is commonly known as, the protected mode. On the other hand, system calls are an exception to this rule. For example, older x86 processors used an interrupt mechanism to go from user-space to kernel-space, but newer processor (PentiumII+) provided instructions that optimize this transition using sysenter and sysexit instructions (Hayward, Mike. Intel P6 vs P7 system call performance. [http://lkml.org/lkml/2002/12/9/13], December 9, 2002). All system calls are small programs built using the C programming language.

The Unix and Linux systems calls are roughly grouped into 6 major categories: file management, device management, information maintenance, process control, communications and miscellaneous calls. The miscellaneous calls are all the ones that don’t really fit in the other categories, like system calls dealing with errors. Today, the Unix and Linux operating system contains hundreds of system calls but in general, they all came from the 35 system calls that came with one of the original UNIX OS in the early 70s. In the next paragraphs, we’re going to describe the various system calls in each of the categories mentioned above, their evolution through history (major changes in functionality) and a comparison with the earliest versions of UNIX.

== File Management Calls==

The system calls in this group deal with every type of operation that is required to run a file system in the operating system. Create file, delete file, opening a file and closing a file are just a few examples of them and most of them hardly changed throughout the years.

''chmod'', ''chown'' and ''chdir'' have been available since the first original UNIX (1971) and they are still used in today’s Linux kernels. The ''chmod'' and ''chown'' calls allows the users to change the file attributes and implements security to the file system. The system call ''chdir'' allows the process to change the current working directory. In the 4th distribution of UNIX from Berkeley (4BSD), new system calls were added to give more control of the file system to the applications. The call ''chroot'' allows the process to change the current root directory with one specified in a path argument. ''fchmod'' and ''fchdir'' are the same as ''chmod'' and ''chdir'' except they takes file descriptors as arguments. As of Linux kernel 2.1.81, the ''chown'' system call now follows symbolic links and therefore, they introduced a new system call, ''lchown'', that does not follow symbolic links.

Another four of the original UNIX system calls are ''open'', ''creat'', ''mkdir'' and ''close''. The ''open'' and ''creat'' calls allows processes to open and possibly create a file or device. Arguments flags are used to set from access modes, like O_RDONLY(read-only), to status flags, like O_APPEND(append mode). The only modifications made to the system calls were the addition of status flags where some of them are linux-specific. The ''close'' call allows processes to close a file descriptor preventing it to be reused. No changes were made to it. ''mkdir'' allows the creation of a file directory. In the earliest version of Unix, to delete a directory, users needed to make a series of ''link'' and ''unlink'' system calls. With Unix 4.2BSD, ''rmdir'' was added and helped solve the problem. The ''rename'' call was also added in 4.2BSD allowing processes to change the name or the location of a file. As file system became more complex, these new system calls helped the users gain better control over them.

There is also system calls used to find, access and modify files. They are ''read'', ''write'', ''seek'' and ''stat''. These were also part of the first UNIX built. The ''read'' and ''write'' system calls allows to read and write from a file (assigned to a file descriptor). The only change was in the Unix System V release 4(SVR4) where a ''write'' call could be interrupted at anytime. The ''seek'' system calls is used to go to a specified position in a file. This calls used a 16 bit address offset. But this was replaced very quickly for ''lseek'' as early as SVR4. It allows the call to use 32 bit address offsets enabling the users more flexibility when accessing or writing to files especially for large ones. It is still used in modern Linux and Unix systems. As of now, developers are trying to implement ''lseek64'', a system call that will use 64 bit addresses. The ''stat'' system calls allows processes to get the status of a file. With SVR4, 2 other version of that system call were created: ''fstat'' and ''lstat''. They both do the same thing except ''lstat'' give the status of symbolic links and ''fstat'' give the status of a file specified by a file descriptor. Different operating systems will output different values to represent the state of a file. Since kernel 2.5.48, the stat returned a nanoseconds field in the file’s timestamp. With the release of 4.4BSD, two new system calls called ''statvfs'' and ''fstatvfs'' were introduced to provide information about a mounted file system. They both do the same thing except fstatvfs takes file descriptors as an argument. These calls are only used in an UNIX environment. In Linux, it has ''statfs'' and ''fstatfs'' to support that same call.

The last two original UNIX system calls in this category that are still used today are ''link'' and ''unlink''. ''link'' creates a hard link to an existing file and ''unlink'' deletes a file link’s name and possibly the file it refers to. If the name refers to a symbolic link, only the link is removed. No major changes were done to the ''unlink'' system calls but new system calls were create from ''link''. The ''symlink'' system call was added in 4.2BSD to allow the creation of symbolic links in the file system.

In the Linux 2.6.16 build, multiple system calls were created so that the calls could deal with relative pathnames as arguments. They can easily be spotted as the system call names all finish with 'at'. Here is a sample list of the created system calls: ''openat'', ''mkdirat'', ''fchmodat'', ''fchownat'', ''fstatat'', ''linkat'', ''unlinkat'', ''renameat'' and ''fchmodat''.

== Device Management Calls==

The device management system calls are linked to hardware and they are mainly used to requests for devices, release devices, to logically attach a device or to detach it, get and modified device attributes and read and write to them.

Two of the most important system calls for the UNIX and Linux operating system is ''mount'' and ''umount''. These were among the few system calls available in the first version of UNIX in 1971. The two calls allowed the operating system to load file systems on storage devices. A few changes were done to the mount system calls. Most of these changes were the creation of new mount flags to enhance performance. For example, since Linux 2.5.19, the MS_DIRSYNC flag permits the directory changes on a file system synchronous. Another Linux improvement was to provide per-process mount namespaces. This was added on the 2.4.19 kernel. If a process was created using the ''clone'' system call with the CLONE_NEWNS flag, the process will have a new namespace initialized to be a copy of the namespace of the process that was cloned. The ''umount'' system call unmounted the file system from the storage device. The only noteworthy change to ''umount'' was the creation of ‘‘umount2’’ in Linux 2.1.116. It is the same as ''umount'' except it allows different flags to control the operation.

The ''open'', ''read'' and ''write'' calls can also be used to access devices. As discussed in the previous section, arguments flags are used to better control the device. You would use them as if the devices were files using the appropriate flags.

With the SVR4 came the system call ''mmap''. This system call is used to map or unmap files or devices into memory. Once a device is mapped, the system call returns a pointer to the mapped area allowing processes to access that device. This system call is still used in a Unix environment but since Linux 2.4, Linux replaced it by the mmap2 system call. It is basically the same as mmap except for a final argument specifying the offset into a file in 4096-byte units. This enables the mapping of large files.

In version 7 of Unix, ''ioctl'' system call is used for device-specific operations that can’t be done using the standard system calls. This helps to deal with a multitude of devices. Each device drivers would provide a set of ioctl request code to allow various operations on their device. Each various request code are hardware dependent so there is no standard available for this system call.

== Information Maintenance Calls==
Information maintenance calls are system calls that return the computers personal information back to the user or change it completely. These type of calls can be split up into three groups get/set time or date, get/set system data and get/set process, file, or device attributes. To fully understand the difference between Linux and UNIX in regrades to system calls, one must explore the three sub-types of information maintenance calls and see how they have changed over time.

The first sub type is Get/set of time and/or date. In Linux, this can be done by a few different system calls, there are: 'gettimeofday' to get the time, 'settimeofday' to set it, 'time' returns the time in seconds and a few other ones like 'ftime'.In the earliest versions UNIX the used the system call was 'stime', which was used to interact with time and dates. 'stime' could return the time and date and sets the system’s idea of the time and date by altering the seconds. 'stime' is still being used by Linux because it is successful, unlike 'settimeofday', which was created to change timezones (tz_dsttime) as well as the time but each occurrence of this field call in the kernel source (apart from declaration) is a bug thus failing.

The second sub type is get/set system data. UNIX does this using the following commands: 'open', 'read', 'close', and 'write'. 'open' opens a file so the file can be written to or read from. 'read' retrieves data from the file, and 'write' modifies data in the file. 'close' is used to indicate that the file is no longer in use. Linux uses the same set of commands for the same purposes.In addition to those system calls there Linux has there own unique system calls which are: 'olduname' gets name and information about current kernel, similar to that is 'uname' gets name and information about current kernel (which is used in the newer versions of UNIX not the older ones), 'iopl' which changes I/O privilege level and 'sysfs' which gets file system type information.

The third sub type is get/set process, file, or device attributes, in UNIX there are several system calls for processing file and device attributes, some of these examples are common to both UNIX and Linux: 'stat' gets file status, 'fork' which spawns a new process, and 'stty' which sets the mode of typewriter.The 'wait' system call is used in both as well the only really difference is that in the Linux version wait store status information in a integer which take the integer itself as an argument, not a pointer to itself. In Linux there are a lot more system calls regarding this type and here are a few of them: 'capget' gets the capabilities of the process, 'capset' sets the capabilities process, 'getppid' gets process identification. The 'capget' and 'capset' interact with the raw kernel interface to getting and setting thread capabilities. These two system calls are specific to Linux and as such the use of these functions (in particular the format of the cap_user_*_t types) are updated as the kernel is updated. The 'getppid' returns the process ID of the calling process and never has any errors.

== Process Control Calls==

Process Control calls are system calls that handle the start, termination and other tasks that might be required
for a process to run correctly.

In unix there are 10 system calls that make up Process Control Calls. These are:
''fork()'',''wait()'',''execl()'',''execlp()'',''execle()'',''execvp()'',''execv()'',''execve()'',''exit()'',''signal()'',''kill()''.

''fork()'': It takes a process and creates an identical processes, which in turn makes one the parent process and the
other the child process. When ''fork()'' succeds it returns 0 to the child process and returns the PID of the child process to the parent process. When it fails, ''fork()'' returns -1 to the parent process.

''wait()'': This call makes a parent process wait for the child process to end. It returns the pid of the child process that is
done. Wait fails if the process has no child process to wait for or its points to an invalid address.

''execl()'',''execlp()'',''execle()'',''execvp()'',''execv()'', are system calls based on the same principle that the system call
takes as an argument a binary file and converts it into a process. When the system call works properly it does
not return, instead it gives control to the new process which replaces the process that called the system call.
each of these are called when different arguments are given.

The following are the definitions for the these system calls as described by this [http://www.di.uevora.pt/~lmr/syscalls.html]

''execl()'': Takes the path name of an executable program (binary file) as its first argument. The rest of the arguments are a list of command
line arguments to the new program (argv[]). The list terminated with a null pointer

''execle()'': Same as execl(), except that the end of the argument list is followed by a pointer to a null-terminated list of character
pointers that is passed a the environment of the new program

''execv()'': Takes the path name of an executable program (binary file) as it first argument. The second argument is a pointer to a list of
character pointers (like argv[]) that is passed as command line arguments to the new program.

''execve()'': Same as execv(), except that a third argument is given as a pointer to a list of character pointers (like argv[]) that is passed as the environment of the new program.

''execlp()'': Same as execl(), except that the program name doesn't have to be a full path name, and it can be a shell program instead of executable module.
''execvp()'': Same as execv(), except that the program name doesn't have to be a full path name, and it can be a shell program instead of an executable module.

''signal()'': This system call is sent to the process when the proper conditions are met. When the program receives the signal it can act in three different ways. The first is to ignore completely, it wont matter how many times the signal is sent the process will not do anything because of it. The only signal that can't be ignored or caught is SIGKILL(). The second is to have the signal set to its default state which means when the process receives it, the process will end. The last option is to catch the signal, when this occurs the unix system will give control to a function that will execute according to how appropriate it is for the process.

''kill()'': The system sends a signal to the process when something occurs. It fails if the signal_name is not a correct signal. There is no process with the PID that matches the argument value.

''exit()'': This call ends the process that calls it and returns the exit status value.

In linux, all of these unix system calls have couterparts in linux except for the exec group of system calls, only execve exists. Also these system calls behave the same way in linux. However the system call ''signal()'' is not recommended to use because of its different implementations in different versions of linux and unix. It is better to use ''sigaction()''. It changes the actions of the process when it receives a valid signal except SIGKILL and SIGSTOP. As newer versions of linux are released, these system calls will always never have major modifications but other system calls, based on these, may be created because specific cases which would make it easier to write programs.

== Communications Calls==

The communication calls relates to the concept of processes having the ability to communicate with one another. Similar to how humans use a telephone as their portal to communicate with eachother, communication calls use "pipes" as their gateway.

In unix there are four subgroups of system calls that are related to communications calls: pipelines, messages, semaphores, and shared memory.
The following are the system calls that belong to each of the subgroups.

Pipelines: ''pipe()'' The pipe() command consists of two components. int pipe (file_descriptors) & int file_descriptors[2]. File_descriptors is an array consisting of two parts as well. One is for reading the data, and the other is for writing the data. Both writing and reading data will read in sequential order along with fully completing it's task. I.e.) There are no partial writes, the pipe will write the whole data that was sent and complete the transmission. The same concept holds for the reading where it will be read all the way through before reading another pipe or new information coming into the pipe. A specially named pipe is FIFO. Standing First In First Out. It is accessed as part of a file system through idea of pipes.

Messages: These functions all consist of recieving and sending messages from the queue usually involving ID's. ''msgget()'' acquires a message from the queue identifier relating to the key. Closely related, but not the same the ''msgrcv()'' command is used to recieve a message from the queue relating to the msqid parameter. This parameter involves the ID of where to recieve the message. ''msgsnd()'' sends a message to the queue. This command can be thought of as the reverse of the ''msgget()''. Lastly, ''msgctl()'' performs message control operations through queries.

Semaphores: This idea of semaphores consists of setting it or checking it. They are used to control access to files. One can use the concept of file locking to get a better understanding of Semaphores. Semaphores aren't usually held together in singles, but rather in groups. This is done by creating a set that can contain several semaphores through the ''semget()'' command. '' semop()'' decides what we the semaphor to accomplish. I.e) depending on whether we have a positive, 0 or negative value, the semaphor shall be added, will wait, or be blocked until positive, respectively. Semaphores were first thought of by Dijkstra and used in computers in the late 60's

Shared Memory: Functions involving shared memory allow the user to be able to access, detach and combine shared addresses. ''shmget()'' command returns the ID for the shared memory region. It can also create it if it doesn't already exist. ''shmat()'' function attaches the shared memory to the virtual address of the calling process. ''shmdt()'' reverses the ''shmat()'' command, and detaches shared memory.

Unix and Linux use the same calls for the majority of the functions now, except for a few which are slighly different.

== Miscellaneous System Calls ==
This category of system calls contains the system calls that do not enough similar calls to form its own group. To avoid random calls floating around, we simply group them into this category.

Directories: These are special files that contain a number of filenames. There are different variations of directories, i.e.) System V, Berkely style directories.

Time: Intuitively, this call allows the user to access the time of day. Specifics on time can be obtained through the structure given by these attributes: ''tm_secint'', ''tm_min'', ''tm_hour'', ''tm_mday'', ''tm_mon'', '' tm_year'', just to list a few.
Parsing Input: Parsing is often used when the user enters in data and the program must parse this data into appropriate divisions in order to obtain specific parts of the data. I.e.)Seperate words from each other in the program, seperate numbers from characters. There are several different ways a programmer can parse the data in order to achieve specific pieces of the data that are needed to be analyzed.

Lastly, there are some system calls which overlap and can be considered in a specific category or mentioned within the Miscellaneous System Calls. Referring to the ``3rd edition, Modern Operating Systems`` textbook, the command, ``chmod`` described above in File Management Calls is considered Miscellaneous. Similarly, the kill() command is mentioned as a Miscellaneous System Call. Hence, it is difficult to decifer whether a system call can be placed into a specific category or simply placed in the ``Other`` bin.

== Conclusion ==

System Calls have been an essential component to the structure of the Linux Kernel(2.6.30+) and Unix Operating Systems for a long period of time. They are the gateway between the user space and the kernel services. More specifically, it allows the User space to acquire the kernel services unlike processes which do not have this authority. Over the years of development in the Linux and Unix OS, the system calls have not had drastic changes to them. Rather than having radical changes to system calls, the development of system calls has merely added more specific system calls to solve new issues that occur within the OS. Hence, this concept has led the birth of 35 system calls to grow to an astonishing quantity consisting of hundreds of system calls. With hundreds of system calls available at ones disposable, all can be catgeorized into 6 major groups: file management, device management, information maintenance, process control, communications and miscellaneous calls. Operating Systems are a colossal program consisiting of very intrinsic pieces all coming together to form what we now know today as Linux Kernel(2.6.30+) or Unix. System calls are simply a small building block, but nevertheless an essential piece, to the tower that is our Operating System.

= References =

Salus, Peter H. A Quarter Century of Unix, Publisher: Addison-Wesly Professional, June 10, 1994.

Unix Programming Manual, http://cm.bell-labs.com/cm/cs/who/dmr/1stEdman.html, November 3, 1971.

BSD System Calls Manual. http://www.unix.com/man-page/FreeBSD/2/, The Unix and Linux Forums.

Linux Programmer's Manual, Linux man-pages project.
http://www.kernel.org/doc/man-pages/

Mendonça Rato, Luís Miguel,professor,university of evora. http://www.di.uevora.pt/~lmr/syscalls.html

COMP 3000 Essay 1 2010 Question 2

2010-10-15T10:08:03Z

Sblais2: /* File Management Calls */

=Question=
How do the available system calls in modern versions of the Linux Kernel (2.6.30+) compare with the system calls available in the earliest versions of UNIX? How has the system call interface been expanded, and why? Focus on major changes or extensions in functionality.

=Answer=

A system call is a mean by which programs in the user space can access kernel services. Systems calls vary from operating system to operating system, although the underlying concepts tends to be the same. In general, a process is not supposed to be able to access the kernel directly. It can't access kernel memory and it can't call kernel functions. When the CPU prevents a process from accessing the kernel, this prevention is commonly known as, the protected mode. On the other hand, system calls are an exception to this rule. For example, older x86 processors used an interrupt mechanism to go from user-space to kernel-space, but newer processor (PentiumII+) provided instructions that optimize this transition using sysenter and sysexit instructions (Hayward, Mike. Intel P6 vs P7 system call performance. [http://lkml.org/lkml/2002/12/9/13], December 9, 2002). All system calls are small programs built using the C programming language.

The Unix and Linux systems calls are roughly grouped into 6 major categories: file management, device management, information maintenance, process control, communications and miscellaneous calls. The miscellaneous calls are all the ones that don’t really fit in the other categories, like system calls dealing with errors. Today, the Unix and Linux operating system contains hundreds of system calls but in general, they all came from the 35 system calls that came with one of the original UNIX OS in the early 70s. In the next paragraphs, we’re going to describe the various system calls in each of the categories mentioned above, their evolution through history (major changes in functionality) and a comparison with the earliest versions of UNIX.

== File Management Calls==

The system calls in this group deal with every type of operation that is required to run a file system in the operating system. Create file, delete file, opening a file and closing a file are just a few examples of them and most of them hardly changed throughout the years.

''chmod'', ''chown'' and ''chdir'' have been available since the first original UNIX (1971) and they are still used in today’s Linux kernels. The ''chmod'' and ''chown'' calls allows the users to change the file attributes and implements security to the file system. The system call ''chdir'' allows the process to change the current working directory. In the 4th distribution of UNIX from Berkeley (4BSD), new system calls were added to give more control of the file system to the applications. The call ''chroot'' allows the process to change the current root directory with one specified in a path argument. ''fchmod'' and ''fchdir'' are the same as ''chmod'' and ''chdir'' except they takes file descriptors as arguments. As of Linux kernel 2.1.81, the ''chown'' system call now follows symbolic links and therefore, they introduced a new system call, ''lchown'', that does not follow symbolic links.

Another four of the original UNIX system calls are ''open'', ''creat'', ''mkdir'' and ''close''. The ''open'' and ''creat'' calls allows processes to open and possibly create a file or device. Arguments flags are used to set from access modes, like O_RDONLY(read-only), to status flags, like O_APPEND(append mode). The only modifications made to the system calls were the addition of status flags where some of them are linux-specific. The ''close'' call allows processes to close a file descriptor preventing it to be reused. No changes were made to it. ''mkdir'' allows the creation of a file directory. In the earliest version of Unix, to delete a directory, users needed to make a series of ''link'' and ''unlink'' system calls. With Unix 4.2BSD, ''rmdir'' was added and helped solve the problem. The ''rename'' call was also added in 4.2BSD allowing processes to change the name or the location of a file. As file system became more complex, these new system calls helped the users gain better control over them.

There is also system calls used to find, access and modify files. They are ''read'', ''write'', ''seek'' and ''stat''. These were also part of the first UNIX built. The ''read'' and ''write'' system calls allows to read and write from a file (assigned to a file descriptor). The only change was in the Unix System V release 4(SVR4) where a ''write'' call could be interrupted at anytime. The ''seek'' system calls is used to go to a specified position in a file. This calls used a 16 bit address offset. But this was replaced very quickly for ''lseek'' as early as SVR4. It allows the call to use 32 bit address offsets enabling the users more flexibility when accessing or writing to files especially for large ones. It is still used in modern Linux and Unix systems. As of now, developers are trying to implement ''lseek64'', a system call that will use 64 bit addresses. The ''stat'' system calls allows processes to get the status of a file. With SVR4, 2 other version of that system call were created: ''fstat'' and ''lstat''. They both do the same thing except ''lstat'' give the status of symbolic links and ''fstat'' give the status of a file specified by a file descriptor. Different operating systems will output different values to represent the state of a file. Since kernel 2.5.48, the stat returned a nanoseconds field in the file’s timestamp. With the release of 4.4BSD, two new system calls called ''statvfs'' and ''fstatvfs'' were introduced to provide information about a mounted file system. They both do the same thing except fstatvfs takes file descriptors as an argument. These calls are only used in an UNIX environment. In Linux, it has ''statfs'' and ''fstatfs'' to support that same call.

The last two original UNIX system calls in this category that are still used today are ''link'' and ''unlink''. ''link'' creates a hard link to an existing file and ''unlink'' deletes a file link’s name and possibly the file it refers to. If the name refers to a symbolic link, only the link is removed. No major changes were done to the ''unlink'' system calls but new system calls were create from ''link''. The ''symlink'' system call was added in 4.2BSD to allow the creation of symbolic links in the file system.

In the Linux 2.6.16 build, multiple system calls were created so that the calls could deal with relative pathnames as arguments. They can easily be spotted as the system call names all finish with 'at'. Here is a sample list of the created system calls: ''openat'', ''mkdirat'', ''fchmodat'', ''fchownat'', ''fstatat'', ''linkat'', ''unlinkat'', ''renameat'' and ''fchmodat''.

== Device Management Calls==

The device management system calls are linked to hardware and they are mainly used to requests for devices, release devices, to logically attach a device or to detach it, get and modified device attributes and read and write to them.

Two of the most important system calls for the UNIX and Linux operating system is ‘‘mount’’ and ‘‘umount’’. These were among the few system calls available in the first version of UNIX in 1971. The two calls allowed the operating system to load file systems on storage devices. A few changes were done to the mount system calls. Most of these changes were the creation of new mount flags to enhance performance. For example, since Linux 2.5.19, the MS_DIRSYNC flag permits the directory changes on a file system synchronous. Another Linux improvement was to provide per-process mount namespaces. This was added on the 2.4.19 kernel. If a process was created using clone() with the CLONE_NEWNS flag, the process will have a new namespace initialized to be a copy of the namespace of the process that was cloned. The ‘‘umount’’ system call unmounted the file system from the storage device. The only noteworthy change to ‘‘umount’’ was the creation of ‘‘umount2’’ in Linux 2.1.116. It is the same as ‘‘umount’’ except it allows different flags to control the operation.

The ‘‘open’’, ‘‘read’’ and ‘‘write’’ calls can also be used to access devices. As discussed in the previous section, arguments flags are used to better control the device. You would use them as if the devices were files using the appropriate flags.

With the SVR4 came the system call ‘‘mmap’’. This system call is used to map or unmap files or devices into memory. Once a device is mapped, the system call returns a pointer to the mapped area allowing processes to access that device. This system call is still used in a Unix environment but since Linux 2.4, Linux replaced it by the mmap2 system call. It is basically the same as mmap except for a final argument specifying the offset into a file in 4096-byte units. This enables the mapping of large files.

In version 7 of Unix, ‘‘ioctl’’ system call is used for device-specific operations that can’t be done using the standard system calls. This helps to deal with a multitude of devices. Each device drivers would provide a set of ioctl request code to allow various operations on their device. Each various request code are hardware dependent so there is no standard available for this system call.

== Information Maintenance Calls==
Information maintenance calls are system calls that return the computers personal information back to the user or change it completely. These type of calls can be split up into three groups get/set time or date, get/set system data and get/set process, file, or device attributes. To fully understand the difference between Linux and UNIX in regrades to system calls, one must explore the three sub-types of information maintenance calls and see how they have changed over time.

The first sub type is Get/set of time and/or date. In Linux, this can be done by a few different system calls, there are: 'gettimeofday' to get the time, 'settimeofday' to set it, 'time' returns the time in seconds and a few other ones like 'ftime'.In the earliest versions UNIX the used the system call was 'stime', which was used to interact with time and dates. 'stime' could return the time and date and sets the system’s idea of the time and date by altering the seconds. 'stime' is still being used by Linux because it is successful, unlike 'settimeofday', which was created to change timezones (tz_dsttime) as well as the time but each occurrence of this field call in the kernel source (apart from declaration) is a bug thus failing.

The second sub type is get/set system data. UNIX does this using the following commands: 'open', 'read', 'close', and 'write'. 'open' opens a file so the file can be written to or read from. 'read' retrieves data from the file, and 'write' modifies data in the file. 'close' is used to indicate that the file is no longer in use. Linux uses the same set of commands for the same purposes.In addition to those system calls there Linux has there own unique system calls which are: 'olduname' gets name and information about current kernel, similar to that is 'uname' gets name and information about current kernel (which is used in the newer versions of UNIX not the older ones), 'iopl' which changes I/O privilege level and 'sysfs' which gets file system type information.

The third sub type is get/set process, file, or device attributes, in UNIX there are several system calls for processing file and device attributes, some of these examples are common to both UNIX and Linux: 'stat' gets file status, 'fork' which spawns a new process, and 'stty' which sets the mode of typewriter.The 'wait' system call is used in both as well the only really difference is that in the Linux version wait store status information in a integer which take the integer itself as an argument, not a pointer to itself. In Linux there are a lot more system calls regarding this type and here are a few of them: 'capget' gets the capabilities of the process, 'capset' sets the capabilities process, 'getppid' gets process identification. The 'capget' and 'capset' interact with the raw kernel interface to getting and setting thread capabilities. These two system calls are specific to Linux and as such the use of these functions (in particular the format of the cap_user_*_t types) are updated as the kernel is updated. The 'getppid' returns the process ID of the calling process and never has any errors.

== Process Control Calls==

Process Control calls are system calls that handle the start, termination and other tasks that might be required
for a process to run correctly.

In unix there are 10 system calls that make up Process Control Calls. These are:
''fork()'',''wait()'',''execl()'',''execlp()'',''execle()'',''execvp()'',''execv()'',''execve()'',''exit()'',''signal()'',''kill()''.

''fork()'': It takes a process and creates an identical processes, which in turn makes one the parent process and the
other the child process. When ''fork()'' succeds it returns 0 to the child process and returns the PID of the child process to the parent process. When it fails, ''fork()'' returns -1 to the parent process.

''wait()'': This call makes a parent process wait for the child process to end. It returns the pid of the child process that is
done. Wait fails if the process has no child process to wait for or its points to an invalid address.

''execl()'',''execlp()'',''execle()'',''execvp()'',''execv()'', are system calls based on the same principle that the system call
takes as an argument a binary file and converts it into a process. When the system call works properly it does
not return, instead it gives control to the new process which replaces the process that called the system call.
each of these are called when different arguments are given.

The following are the definitions for the these system calls as described by this [http://www.di.uevora.pt/~lmr/syscalls.html]

''execl()'': Takes the path name of an executable program (binary file) as its first argument. The rest of the arguments are a list of command
line arguments to the new program (argv[]). The list terminated with a null pointer

''execle()'': Same as execl(), except that the end of the argument list is followed by a pointer to a null-terminated list of character
pointers that is passed a the environment of the new program

''execv()'': Takes the path name of an executable program (binary file) as it first argument. The second argument is a pointer to a list of
character pointers (like argv[]) that is passed as command line arguments to the new program.

''execve()'': Same as execv(), except that a third argument is given as a pointer to a list of character pointers (like argv[]) that is passed as the environment of the new program.

''execlp()'': Same as execl(), except that the program name doesn't have to be a full path name, and it can be a shell program instead of executable module.
''execvp()'': Same as execv(), except that the program name doesn't have to be a full path name, and it can be a shell program instead of an executable module.

''signal()'': This system call is sent to the process when the proper conditions are met. When the program receives the signal it can act in three different ways. The first is to ignore completely, it wont matter how many times the signal is sent the process will not do anything because of it. The only signal that can't be ignored or caught is SIGKILL(). The second is to have the signal set to its default state which means when the process receives it, the process will end. The last option is to catch the signal, when this occurs the unix system will give control to a function that will execute according to how appropriate it is for the process.

''kill()'': The system sends a signal to the process when something occurs. It fails if the signal_name is not a correct signal. There is no process with the PID that matches the argument value.

''exit()'': This call ends the process that calls it and returns the exit status value.

In linux, all of these unix system calls have couterparts in linux except for the exec group of system calls, only execve exists. Also these system calls behave the same way in linux. However the system call ''signal()'' is not recommended to use because of its different implementations in different versions of linux and unix. It is better to use ''sigaction()''. It changes the actions of the process when it receives a valid signal except SIGKILL and SIGSTOP. As newer versions of linux are released, these system calls will always never have major modifications but other system calls, based on these, may be created because specific cases which would make it easier to write programs.

== Communications Calls==

The communication calls relates to the concept of processes having the ability to communicate with one another. Similar to how humans use a telephone as their portal to communicate with eachother, communication calls use "pipes" as their gateway.

In unix there are four subgroups of system calls that are related to communications calls: pipelines, messages, semaphores, and shared memory.
The following are the system calls that belong to each of the subgroups.

Pipelines: ''pipe()'' The pipe() command consists of two components. int pipe (file_descriptors) & int file_descriptors[2]. File_descriptors is an array consisting of two parts as well. One is for reading the data, and the other is for writing the data. Both writing and reading data will read in sequential order along with fully completing it's task. I.e.) There are no partial writes, the pipe will write the whole data that was sent and complete the transmission. The same concept holds for the reading where it will be read all the way through before reading another pipe or new information coming into the pipe. A specially named pipe is FIFO. Standing First In First Out. It is accessed as part of a file system through idea of pipes.

Messages: These functions all consist of recieving and sending messages from the queue usually involving ID's. ''msgget()'' acquires a message from the queue identifier relating to the key. Closely related, but not the same the ''msgrcv()'' command is used to recieve a message from the queue relating to the msqid parameter. This parameter involves the ID of where to recieve the message. ''msgsnd()'' sends a message to the queue. This command can be thought of as the reverse of the ''msgget()''. Lastly, ''msgctl()'' performs message control operations through queries.

Semaphores: This idea of semaphores consists of setting it or checking it. They are used to control access to files. One can use the concept of file locking to get a better understanding of Semaphores. Semaphores aren't usually held together in singles, but rather in groups. This is done by creating a set that can contain several semaphores through the ''semget()'' command. '' semop()'' decides what we the semaphor to accomplish. I.e) depending on whether we have a positive, 0 or negative value, the semaphor shall be added, will wait, or be blocked until positive, respectively. Semaphores were first thought of by Dijkstra and used in computers in the late 60's

Shared Memory: Functions involving shared memory allow the user to be able to access, detach and combine shared addresses. ''shmget()'' command returns the ID for the shared memory region. It can also create it if it doesn't already exist. ''shmat()'' function attaches the shared memory to the virtual address of the calling process. ''shmdt()'' reverses the ''shmat()'' command, and detaches shared memory.

Unix and Linux use the same calls for the majority of the functions now, except for a few which are slighly different.

== Miscellaneous System Calls ==
This category of system calls contains the system calls that do not enough similar calls to form its own group. To avoid random calls floating around, we simply group them into this category.

Directories: These are special files that contain a number of filenames. There are different variations of directories, i.e.) System V, Berkely style directories.

Time: Intuitively, this call allows the user to access the time of day. Specifics on time can be obtained through the structure given by these attributes: ''tm_secint'', ''tm_min'', ''tm_hour'', ''tm_mday'', ''tm_mon'', '' tm_year'', just to list a few.
Parsing Input: Parsing is often used when the user enters in data and the program must parse this data into appropriate divisions in order to obtain specific parts of the data. I.e.)Seperate words from each other in the program, seperate numbers from characters. There are several different ways a programmer can parse the data in order to achieve specific pieces of the data that are needed to be analyzed.

Lastly, there are some system calls which overlap and can be considered in a specific category or mentioned within the Miscellaneous System Calls. Referring to the ``3rd edition, Modern Operating Systems`` textbook, the command, ``chmod`` described above in File Management Calls is considered Miscellaneous. Similarly, the kill() command is mentioned as a Miscellaneous System Call. Hence, it is difficult to decifer whether a system call can be placed into a specific category or simply placed in the ``Other`` bin.

== Conclusion ==

System Calls have been an essential component to the structure of the Linux Kernel(2.6.30+) and Unix Operating Systems for a long period of time. They are the gateway between the user space and the kernel services. More specifically, it allows the User space to acquire the kernel services unlike processes which do not have this authority. Over the years of development in the Linux and Unix OS, the system calls have not had drastic changes to them. Rather than having radical changes to system calls, the development of system calls has merely added more specific system calls to solve new issues that occur within the OS. Hence, this concept has led the birth of 35 system calls to grow to an astonishing quantity consisting of hundreds of system calls. With hundreds of system calls available at ones disposable, all can be catgeorized into 6 major groups: file management, device management, information maintenance, process control, communications and miscellaneous calls. Operating Systems are a colossal program consisiting of very intrinsic pieces all coming together to form what we now know today as Linux Kernel(2.6.30+) or Unix. System calls are simply a small building block, but nevertheless an essential piece, to the tower that is our Operating System.

= References =

Salus, Peter H. A Quarter Century of Unix, Publisher: Addison-Wesly Professional, June 10, 1994.

Unix Programming Manual, http://cm.bell-labs.com/cm/cs/who/dmr/1stEdman.html, November 3, 1971.

BSD System Calls Manual. http://www.unix.com/man-page/FreeBSD/2/, The Unix and Linux Forums.

Linux Programmer's Manual, Linux man-pages project.
http://www.kernel.org/doc/man-pages/

Mendonça Rato, Luís Miguel,professor,university of evora. http://www.di.uevora.pt/~lmr/syscalls.html

COMP 3000 Essay 1 2010 Question 2

2010-10-15T00:16:37Z

Sblais2: /* File Management Calls */

COMP 3000 Essay 1 2010 Question 2

2010-10-15T00:15:54Z

Sblais2: /* Device Management Calls */

COMP 3000 Essay 1 2010 Question 2

2010-10-15T00:12:27Z

Sblais2: /* File Management Calls */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T23:55:26Z

Sblais2: /* File Management Calls */

Talk:COMP 3000 Essay 1 2010 Question 2

2010-10-14T23:54:13Z

Sblais2:

Not in this group and I'm not completely sure if this is relevant but I found that UNIX used the POSIX standard while Linux used LSB which is based on the POSIX standard.
This article outlines some conflicts between them [https://www.opengroup.org/platform/single_unix_specification/uploads/40/13450/POSIX_and_Linux_Application_Compatibility_Final_-_v1.0.pdf]. I didn't find the actual comparisons very comprehensible but the ideas are there. --[[User:Slay|Slay]] 15:05, 7 October 2010 (UTC)

Uh, where did Figure 1 and much of the current text come from? It looks like it was cut and pasted from random source. Please don't plagarize! --[[User:Soma|Anil]] (19:24, 8 October 2010 (UTC))

Look into the reference article "Kernel command using Linux system calls". Plagiarism is not my goal. I'm using my own words to make a simple but complete description of a system call using the interrupt method. Check the references and If you think it is too close, please let me know. It is hard when an author makes such a good and clear description.--[[User:Sblais2|Sblais2]] 21:02, 8 October 2010 (UTC)

I thought it would be nice to first describe what is a system calls and the two current methods of doing them. The first is the interrupt method. The second which is used in Linux 2.6.18+ is using the sysenter and sysexit instructions.--[[User:Sblais2|Sblais2]] 19:56, 8 October 2010 (UTC)

You can't use that figure. And you can't copy the text either, even if you change the words slightly. But really, you're just wasting your time. This question is not talking about how system calls are invoked; if you wanted to discuss this, you should be discussing system call invocation mechanisms on the PDP-11 and VAX systems! Here I'm interested in what are the calls, i.e., kernel functions that can be invoked by a regular program.--[[User:Soma|Anil]]

This link provides about 40 UNIX system calls along with example on where they would be used from the looks of it: [http://www.di.uevora.pt/~lmr/syscalls.html]. --[[User:Apharan2|Apharan2]]

Thank you for clarifying things. I will go that route. --[[User:Sblais2|Sblais2]] 13:08, 12 October 2010 (UTC)

I don't see everyone contributing to this group. Please do let the others in your group know, divide your work into sections and discuss here. If you have questions - ask.--[[User:praman|Preeti]]

This link shows all the system calls from Linux 2.6.33 [http://www.kernel.org/doc/man-pages/online/pages/man2/syscalls.2.html]
--[[User:Sblais2|Sblais2]] 23:36, 12 October 2010 (UTC)

As no one in our group made suggestion on the format of our essay, I've put one in place. In your research, each system calls should fit in one of the category. If someone picks one up. Please let me know ASAP. I will be working on that all day. Read the intro's last paragraph if your not sure what you should write on. My english writing skills are not perfect so if one of you guys see ways to improved the text, please do. --[[User:Sblais2|Sblais2]] 14:29, 13 October 2010 (UTC)

Well I feel like I'm the only one in that team but...Anyway I've completed the first 2 sections. Please try working on the next 4. If you want to modify something, please post a small gist of it in here so we can all validate. Thanks. --[[User:Sblais2|Sblais2]] 20:10, 13 October 2010 (UTC)

I am wondering if we have actually split up the work accordingly i am going to a temped to answer Information Maintenance if anyone has dibs please let me know - Csulliva
I am finding this site that I find very helpfull to understand system called for linux an UNIX
http://ss64.com/bash/
-Csulliva

I'll do process control calls. and help out on the last part that is not written up yet.I'll read the other parts as well just to get an understanding. [[User:Apharan2|Apharan2]]

Csullliva, be careful not to confuse system calls and shell commands. Some of them have actually the same name, like ''mkdir''. But shell commands is on the user-level. Some of them will actually do system calls to complete the operation.
It's good to finally hear from you guys.--[[User:Sblais2|Sblais2]] 01:39, 14 October 2010 (UTC)

I'll work on the communication calls and the miscellaneous system calls. Not sure if you guys wanted to add more to this, but I can help out with writing a conclusion as well. I'm somewhat good at writing, so if i see any little things that I could touch up on, i'll help out with that.-R.arteaga

after my confusion with bash and system calls i had some trouble find a system calls for linux that would effect time here a web site I found that helped me out with description regarding the calls http://www.digilife.be/quickreferences/QRC/LINUX%20System%20Call%20Quick%20Reference.pdf
hopefully i am on the write track now....some one stop me if i am not -Csulliva

Looks ok to me Csulliva, might want to check the link I posted in this discussion. It shows all the system calls in the Linux kernel 2.6.30. Then even show history information in it. It is then easy to track early Unix implementation. R.arteaga -> That would be great. I started thinking about a conclusion but writing is not my forte (unless I am underestimating myself). --[[User:Sblais2|Sblais2]] 11:59, 14 October 2010 (UTC)

Also not to forget to add your references. I added mine in the reference section. --[[User:Sblais2|Sblais2]] 19:02, 14 October 2010 (UTC)

My writing skills have never been any good. I've read through a whole bunch of the page and fixed a few typos here and there. I want to add to the "Information Maintenance Calls" section, but I can't promise that it will be any good. So feel free to help me out. -Dlangloi

okay well i am currently working on the information maintenance calls part and i am a little stuck on the type system data so by all means help out.. that being said can anyone give me a hint on a system call that works with a system data in UNIX because i read the manual and i still drawing a blank, thanks-Csulliva

Ya I have also had troubles trying to find system calls that affect system data. I have read through about half of that manuel in the references, but nothing seems to be related. Anyways, my eyes are starting to hurt. I will try again later and see if something turns up. -Dlangloi
I've added to it but I have my doubts that it is correct. Please revise my work as it could be wrong. -Dlangloi

Just to make sure before I add stuff to the page: are the exec(),fork(), wait(), and exit() calls be considered part of process control calls ? [[User:Apharan2|Apharan2]]

Apharan2=> yes they are considered part of process control as they deal directly with processes. Check this link out for more detail [http://www.softpanorama.org/Internals/unix_system_calls.shtml]. --[[User:Sblais2|Sblais2]] 23:54, 14 October 2010 (UTC)

Talk:COMP 3000 Essay 1 2010 Question 2

2010-10-14T19:02:08Z

Sblais2:

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:57:47Z

Sblais2: /* References */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:57:19Z

Sblais2: /* References */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:51:56Z

Sblais2: /* File Management Calls */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:50:49Z

Sblais2: /* File Management Calls */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:44:23Z

Sblais2: /* File Management Calls */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:43:10Z

Sblais2: /* References */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:37:34Z

Sblais2: /* File Management Calls */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:30:55Z

Sblais2: /* References */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:30:21Z

Sblais2: /* References */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:28:51Z

Sblais2: /* References */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:25:27Z

Sblais2: /* Answer */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:24:20Z

Sblais2: /* Answer */

COMP 3000 Essay 1 2010 Question 2

2010-10-14T18:22:00Z

Sblais2: /* References */